Open Access Open Access  Restricted Access Subscription Access

doi:10.3808/jei.201100191
Copyright © 2024 ISEIS. All rights reserved

A Robust Test of Spatial Predictive Models: Geographic Cross-Validation

D. J. Lieske1,2* and D. J. Bender1

  1. Department of Geography, University of Calgary, 2500 University Dr. NW, Calgary, Alberta T2N 4N1, Canada
  2. Department of Geography and Environment, Mount Allison University, 144 Main Street, Sackville, New Brunswick E4L 1A7, Canada

*Corresponding author. Email: dlieske@mta.ca

Abstract


Predictive modeling is an important tool for identifying areas for conservation prioritization. But the reliability of any model depends on how well its predictions can be generalized beyond the area surveyed. Recent work points to the potential for enhancing predictive power by incorporating such spatial processes as autocorrelation or the influence of location, so this study addressed two questions: (1) what affect does model complexity, spatial autocorrelation and spatial location have on model accuracy? (2) how generalizable are different methods when applied to new geographic test regions? On average, predictive power declined 22.7% ± 2.7% SE when models were used to predict occurrences in “unsampled†geographic test regions. Overall variability in performance depended on the method used. AUTO and GAM models tended to be amongst the least variable, but results depended upon species. Our results suggest that models with complex functional relationships between the response and predictor variables (such as GAMs fit with up to 5 knots) tended to either improve accuracy, or perform more consistently across species, but not both at the same time. In general, it is very difficult to accurately extrapolate model predictions into unsampled geographic areas. However, we found that habitat specialists such as the Sedge Wren were consistently well predicted, regardless of method, and that autocorrelated regression (using a Gibbs sampler and simulation of presence/absence) could be more reliably generalized for species showing strong social structure (e.g., patchiness). GWR was especially sensitive to the plots used to train the model.

Keywords: geographic cross-validation, predictive models, generalizability, accuracy


Full Text:

PDF

Supplementary Files:

Refbacks

  • There are currently no refbacks.