PDF version of this entire document

Results

The results of the validation experiment are shown in Figure 4. Note that $\mathcal{O}$ is expected to decrease with increasing perturbation of the registration, whilst $G$ and $S$ are expected to increase. All three metrics are generally well-behaved and show a monotonic response to increasing perturbation. This validates the model-based measures of registration quality, which are shown both to change monotonically with increasing perturbation of the registration and to correlate with the gold-standard approach based on manually annotated ground truth.

Figure 5: The sensitivities of the different registration assessment methods and their standard errors.

The results for different values of $r$ (shuffle radius) and $\beta_{l}$ all demonstrate monotonic behaviour with increasing perturbation, but the slopes and errors vary systematically. This affects the size of perturbation that can be detected. To make a quantitative comparison of the different methods, we define the sensitivity, as a function of perturbation $(\frac{1}{\overline{\sigma}})\frac{M-M_{0}}{d}$, where $M$ is the quality measured for a given degree of deformation $d$, $M_{0}$ is the measured quality at registration (no deformation) and $\overline{\sigma}$ is the mean error in the estimate of $M$ over the range.

Sensitivities of the different methods, averaged over the range of perturbations shown in Figure 4, are summarised in Figure 5 for all the methods of assessment. Since sensitivity across the whole range is desired, this average shows that the Specificity measure with shuffle radius 1.5 or 2.1 is the most sensitive of the measures studied, and that this advantage is statistically significant. Exceeding this shuffle radius may lead to performance degradation, as deformations will be obscured by the shuffling.

Figure 6: Appearance model constructed from groupwise registered images. First mode of variation is shown, $\pm 2.5$ standard deviations.

The results for different registration algorithms are shown in Figure 7. The specificity obtained for the two groupwise methods is significantly better than that obtained using the pairwise approach, implying better registration, but it is not possible to distinguish between the two groupwise methods. By applying the same NRR algorithms to an annotated dataset, it becomes evident that generalised overlap measures agree with this assessment, for all possible assignments to $\beta_{l}$. As might be expected from the sensitivity results presented above, it is not possible to distinguish between any of the methods using generalisation.

Figure 7: Generalisation and Specificity of the three registration methods as a function of the number of modes included in the appearance model.