Skip to main content

Table 7 Criteria for qualitative ratings of data quality, model performance, and direction or error in model predictions, in this study of post-fire tree mortality models from the USA, from fires occurring from 1981 to 2016. Most model evaluation criteria are defined in Table 1; AUC = area under the receiver operating characteristic curve. Defense and injury variables are defined in Table 2. SD = standard deviation

From: A large database supports the use of simple models of post-fire tree mortality for thick-barked conifers, with less support for other species

Type

Rating

Criteria

Data quality

Poora

 

<50 observations total

or

<10 live observations

or

<10 dead observations

Fair

 

≥50 observations total

and

≥10 live observations

and

≥10 dead observations

Acceptable

 

Meets “Fair” standards

and

≥5 fires were sampled

and

≥50 live observations

and

≥50 dead observations

Excellent

 

Meets “Acceptable” standards

and

at least 10 observations for every injury level bin (for models with CVS, CLS, CVK, only)

and

minimum injury variable (CVS, CLS, CVK, BCH) = 0

and

maximum injury variable (CVS, CLS, CVK) = 100

Outstanding

 

Meets “Excellent” standards

and

maximum DBH ≥100 cm

and

maximum DBH ≥ large-tree DBH for speciesb

and

observations are from sites ≥1 SD of the temperature range for the species

and

observations are from sites ≤1 SD of the temperature range for the species

and

observations are from sites ≥1 SD of the precipitation range for the species

and

observations are from sites ≤1 SD of the precipitation range for the species

Model performance

Poor

 

AUC <0.7

and

PPV <0.6

and

NPV <0.6

Acceptable

 

AUC ≥0.7

and either

PPV ≥0.6

or

NPV ≥0.6

Excellent

and

AUC ≥0.8

and

PPV ≥0.7

and

NPV ≥0.7

Outstanding

 

Meets “Excellent” standards

and

AUC ≥0.9

Over-predicts

Rarely

 

Species-level error rate >0.25 across <30% of the range of primary damage variable

Sometimes

 

Species-level error rate >0.25 across >30% and <50% of the range of primary damage variable

Often

or

Species-level error rate >0.25 over 50% of the range of primary damage variable species-level error rate >0.5 over 30% of the range of primary damage variable

Always

 

Species-level error rate >0.5 over 50% of the range of primary damage variable

Under-predicts

Rarely

 

Species-level error rate <−0.25 across <30% of the range of primary damage variable

Sometimes

 

Species-level error rate >−0.25 across >30% and <50% of the range of primary damage variable

Often

or

Species-level error rate <−0. 25 over 50% of the range of primary damage variable species-level error rate <−0.5 over 30% of the range of primary damage variable

Always

 

Species-level error rate <−0.5 over 50% of the range of primary damage variable

  1. aExcluded from model evaluation analysis
  2. bLarge-tree DBH values were the higher values of the typical DBH range for each species in the Silvics of North America (Burns and Honkala 1990). If not available from that source, we drew from the Fire Effects Information System database (USDA Forest Service 2019), or the Gymnosperm Database (Earle 2019)