Fig. 2
Model results for the Pinus contorta Ryan and Amman (R-A) model, from our study to evaluate post-fire tree mortality models. This figure allows for a thorough assessment of model quality and data quality for the P. contorta R-A model. (A) Map shows locations of fires occurring from 1981 to 2016 within the USA from which data to evaluate the model were sampled. Fire locations are plotted over the species’ range (green polygons). P. contorta had excellent data quality, with observations coming from 34 fires, dispersed across the species range within the USA. BTcoef = species-specific bark thickness coefficient. (B) The bi-plot shows where the observations used to evaluate models (orange points) fall within the species’ bioclimatic niche space (black points) in terms of temperature (x-axis) and precipitation (y-axis). Fires were located across the temperature niche of the species, but on the lower range of the precipitation niche. (C) Model evaluation summary statistics including the AUC (area under the receiver operator characteristic curve) at 0.5 threshold for determining mortality, and confidence intervals (CI) around the AUC. Model evaluation statistics include accuracy, sensitivity (Sens.), specificity (Spec.), positive predicted values (PPV), and negative predictive values (NPV), summarized over a range of probability thresholds (0.1 to 0.9; rows), with the commonly used threshold of 0.5 shown in bold. Warmer colors indicate greater values. The top three bold rows show model performance metrics for the “best” threshold, which optimizes sensitivity and specificity, the best threshold with sensitivity >0.8, and the best threshold with specificity >0.8. The model accuracy statistics indicate a high AUC (0.803), but at the typically used 0.5 threshold, model sensitivity is very high and model specificity is very low. This means that the model accurately predicts which trees are going to die, but makes inaccurate predictions regarding which trees are going to live, which is reflected in the low positive predicted values (PPV) and high negative predictive values (NPV): many trees predicted to die do not actually die, while most trees predicted to live do live. By adjusting the threshold used to assign either trees to live or dead classes to a high value, either high sensitivity or specificity can be obtained with this model with the evaluation data (top bold rows). (D) The distributions of defense (diameter at breast height [DBH], as an interpretable representation of bark thickness) and injury (crown volume scorch) variables used in the model are shown with bi-plots. Box plots in the margins of (D) show median (bar), interquartile range (IQR; box; 25th and 75th percentiles), and whiskers show the minimum and maximum values that do not exceed a 1.5 × IQR. Dots are values outside IQR. There were 6006 total observations, and 1875 and 4131 observations in the live and dead categories, respectively, well distributed over the primary injury and defense variables. (E and F) Assessment of species-level error comparing the predicted probability of mortality using a 0.5 threshold (Pm; orange points show values and shading shows range) and the observed proportion of trees or stems killed (gray points) within binned observations of the primary injury variables (E), and the DBH (F). (E) The model accurately predicts mortality over the range of percentage crown volume scorched (CVS) and (F) tree diameter at breast height. Qualitative ratings of data quality, model performance, and direction or error in model predictions are listed at the bottom of the figure. Model evaluation figures, such as this one, are available for each model evaluated for individual species in Additional file 1