The secret assumption of transfer functions: problems with spatial autocorrelation in evaluating model performance

• Published in: Quaternary science reviews 2005-11, Vol.24 (20), p.2173-2179
• Main Authors: Telford, R.J., Birks, H.J.B.
• Format: Article
• Language: English
• ISSN: 0277-3791; 1873-457X
• DOI: 10.1016/j.quascirev.2005.05.001

The estimation of the predictive power of transfer functions assumes that the test sites are independent of the modelling sites. Cross-validation in the presence of spatial autocorrelation seriously violates this assumption. This assumption and the consequences of its violation have not been discussed before. We show, by simulation, that the expected r 2 of a transfer function model from an autocorrelated environment can be high, and is not near zero as commonly assumed. We investigate a foraminiferal sea surface temperature training set for the North Atlantic, for which, with cross-validation, the modern analogue technique (MAT) and artificial neural networks (ANN) outperform transfer function methods based on a unimodal species-environment response model. However, when a spatially independent test set, the South Atlantic, is used, all models have a similar predictive power. We show that there is a spatial structure in the foraminiferal assemblages even after accounting for temperature, presumably due to autocorrelations in other environmental variables. Since the residuals from MAT show little spatial structure, in contrast to the residuals of unimodal response models, we contend that MAT has inappropriately internalized the non-temperature spatial structure to improve its performance. We argue that most, if not all, estimates of the predictive power of MAT and ANN models for sea surface temperatures hitherto published are over-optimistic and misleading.