Progress Report

May 31st, 2005

Overview*

• Recent Work
• Explanation
• Results
• Thoughts
• Future Paths
•    *Not necessarily covered in this order

Recent Work in Brief

• IBIM Data
• Improved evaluation based on shuffle distance
• Histogram-based evaluation
• ITK (image registration toolkit from Imperial College) registration
• Text for a comprehensive overview on NRR and AAM evaluation
• More minor activities

IBIM and Evaluation

• My involvement was interrupted by a break, which had been scheduled long beforehand (no Easter break was taken in April)

Data examples below: (arbitrarily picked from IBIM sets)

IBIM Data Model

• Some data is noisy and/or distorted
• The model is affected negatively
• Model sharpness does not appear to affect evaluation significantly

IBIM Data Model - Ctd.

IBIM Data Collaboration - Conclusion

• Correlation was demonstrated between:
  • Labels overlap measures and...
  • Model evaluation using shuffle distance
• Further work to follow (see future plans -- yet to come in this report)

Histogram-based Distances

• Motivation to move onwards from published work on the shuffle, so:
  • Histogram-based difference images
  • Essentially, images are orderless within localised regions
  • Possibly a superset of the shuffle in some sense (needs formulation)
  • Evaluation of image similarity can be improved
  • No empirical proof of that yet

Histogram-based Distances - Ctd.

• Comparisons with shuffle distance
• Varying resolution levels for speed and hence more rapid tweaking
• Conclusion: if image resolution is too low (near 30 x 30 pixels), evaluation becomes very weak
• This weakness is apparent even when robust shuffle distance is used

Histogram-based Distances - Ctd.

• Further experiments to complete this weekend
• The significant question: will this new measure be more sensitive?

Histogram-based Distances - Explanation

• Explained in the past, but has been modified since
• The steps involved:
  • Pick pair of fix-sized squares (or rectangles, or discs...) in a pair of images
  • Build a histogram for the squares (windows)
  • Apply smoothing to the histogram depending on number of bins and window size
  • Calculate the sum of squared differences in corresponding bins
  • Repeat the above for each pair of corresponding pixels to form a new image

Histogram-based Distances - Explanation - Ctd.

• This is a slow algorithm
• Its particular implementation makes it even slower
• Simple (naïve) algorithm, hence fast to pull together and easy to change
• Some visual examples are shown in the next few slides

Histogram-based Distances - Examples

5 bins, 5x5 window for the histogram, no smoothing

Histogram-based Distances - Examples

2 bins, 5x5 window for the histogram, no smoothing

Histogram-based Distances - Examples

10 bins, 5x5 window for the histogram, histogram smoothing with kernel width 5

Histogram-based Distances - Examples

7 bins, 5x5 window for the histogram, histogram smoothing with kernel width 5

Histogram-based Distances - Examples

3 bins, 5x5 window for the histogram, no histogram smoothing

Histogram-based Distances - Examples

10 bins, 3x3 window for the histogram, no histogram smoothing

Histogram-based Distances - Examples

25 bins, 5x5 window for the histogram, no histogram smoothing

Histogram-based Distances - Examples

5 bins, 3x3 window for the histogram, no histogram smoothing

Histogram-based Distances - Examples

25 bins, 5x5 window for the histogram, histogram smoothing with kernel width 7

Pending Work on Model Evaluation using the Shuffle Distance

• Perturbation in terms of pixel displacement - now determined
• Show that the shuffle transform is a metric
• Normalisation for set sizes
• Publication of work on comparison between overlap and shuffle

Pending Work on Model Evaluation using the Histograms

• Continue evaluation that is histogram-based (or anything which outperforms shuffle)
• Demonstrate that the new method is more sensitive than shuffle (with varying radii)
• Tim says it is plausible as work towards an ECCV submission

Evaluation in 3-D

• Vlad: towards 3-D automatic model construction
• Might need access to grid due to heavy computations
• Evaluation efficency improvements, e.g. Euclidean distance for some points, transitive inference
• 3-D evaluation: a possible implementation with efficiency in mind

Registration Code from Imperial College: How it Came About

• Request for the source code
• Open Source in principle, but not shared openly
• Further advice given in the IRC Plenary Meeting
• Rueckert was willing to distribute the code within the IRC
• Code from Kola for conversion of data

Current work on Evaluation (Shuffle)

• ITK experiments in 3-D
• Overly long process. Takes over a night for an image pair using NMI
• Comparison with registration in VXL is hard because of different data formats and interfaces

Next Steps in Evaluation (Shuffle)

• Compare methods of NRR
• Rueckert's code (ITK) works properly in ISBE
• Similarity measures include: CC, CR, MI, NMI...
• Compare against VXL registration - MDL, Pair-wise, group-wise, etc.

Next Steps in Evaluation (Shuffle) - Ctd.

• Using large set of images (200+)
• Based on shuffling (proven to behave like/emulate overlap)
• Collaborative work which involves large comparison scale and little effort
• Vlad is committed to help in evaluating more registration algorithms

Next Steps - Possibilities

• Possibility: get code from UCL (formerly the KCL-based group) -- SPM -- for further evaluation.
• Maybe a full review as Crum did (inter-site comparison)
• Issues concerning data compatibility and programming languages

ITK Registration - Examples

Source image

ITK Registration - Examples

Target image

ITK Registration - Examples

Source to target using joint entropy

ITK Registration - Examples

Difference image: Joint entropy

ITK Registration - Examples

Source to target using mutual information

ITK Registration - Examples

Difference image: Mutual information

ITK Registration - Examples

Source to target using normalised mutual information

ITK Registration - Examples

Difference image: Normalised mutual information

ITK Registration - Examples

Source to target using sums of squared differences

ITK Registration - Examples

Difference image: Sums of squared differences

ITK Registration - Examples

Source to target using correlation coefficient

ITK Registration - Examples

Difference image: Correlation coefficient

ITK Registration - Examples

Source to target using correlation ratio

ITK Registration - Examples

Difference image: Correlation ratio

Thoughts: Segmentation, Models and NRR

• Registration aims at getting segments to overlap
• Maybe segment one image and propagate to all others via model
• Use segmentation to improve evaluation (similar to label)
• Conversely, improve segmentation using dynamics of models (built using NRR)

Thoughts: Segmentation, Models and NRR - Ctd.

• Advantage is in having what is a missing link to others -- models
• Look at many images (sequence) of modelled images, project back from model to get segmentation
• The approach gives registration, model, and segmentation directly from the raw data.
• Segmentation is arbitrary (not rational but data-driven), just like choice of landmark points

Thoughts: Segmentation, Models and NRR - Ctd.

• Create small compartmentalised models of constituent segments
• When model is built, see variances in mean and optimise points, then re-run model-building
• This approach can progressively improve identification of landmarks
• Hence, better models can be built automatically
• Expensive approach as models are built over and over again to get refined

Towards Publication

• First steps towards assembling a journal paper
• production of new graphics
• New content
• Based profoundly on the submissions to BMVC and MICCAI
• Still a draft, which needs criticism before resumption

Towards Publication - Ctd.

• Incorporates symmetric shuffle and other minor developments
• Text re-use from IPMI, BMVC, MICCAI (also graphics)
• More experiments ought to be run, but need to reach agreement on which ones to run first
• Compiled document in its present form:
     Evaluation of AAM's and NRR (PDF)

Small Activities

• Ph.D. 2^nd Year Workshop - useful and constructive group discussion about career paths in academia
• Vladimir Vapnik presentation in RSS - bland, but very personal talk. Heavily attended by people affiliated with the School of Computer Science.
• Possibility: Volunteering for a Monday meeting talk or Machine Learning and Pattern Recognition seminar (Computer Science)

Summary

• Evaluation methods
  • Evaluation using shuffle distance
  • Further improvement of evaluation
  • Prospects of evaluating algorithms from IC
• Improving automatic model construction (segmentation, 3-D, etc.)

Future Directions

• Further work on evaluation methods
• Further investigation of data and NRR algorithms
• Automatic model construction
• Writing up of more comprehensive text