Type | Rating | Criteria | |
---|---|---|---|
Data quality | Poora | <50 observations total | |
or | <10 live observations | ||
or | <10 dead observations | ||
Fair | ≥50 observations total | ||
and | ≥10 live observations | ||
and | ≥10 dead observations | ||
Acceptable | Meets “Fair” standards | ||
and | ≥5 fires were sampled | ||
and | ≥50 live observations | ||
and | ≥50 dead observations | ||
Excellent | Meets “Acceptable” standards | ||
and | at least 10 observations for every injury level bin (for models with CVS, CLS, CVK, only) | ||
and | minimum injury variable (CVS, CLS, CVK, BCH) = 0 | ||
and | maximum injury variable (CVS, CLS, CVK) = 100 | ||
Outstanding | Meets “Excellent” standards | ||
and | maximum DBH ≥100 cm | ||
and | maximum DBH ≥ large-tree DBH for speciesb | ||
and | observations are from sites ≥1 SD of the temperature range for the species | ||
and | observations are from sites ≤1 SD of the temperature range for the species | ||
and | observations are from sites ≥1 SD of the precipitation range for the species | ||
and | observations are from sites ≤1 SD of the precipitation range for the species | ||
Model performance | Poor | AUC <0.7 | |
and | PPV <0.6 | ||
and | NPV <0.6 | ||
Acceptable | AUC ≥0.7 | ||
and either | PPV ≥0.6 | ||
or | NPV ≥0.6 | ||
Excellent | and | AUC ≥0.8 | |
and | PPV ≥0.7 | ||
and | NPV ≥0.7 | ||
Outstanding | Meets “Excellent” standards | ||
and | AUC ≥0.9 | ||
Over-predicts | Rarely | Species-level error rate >0.25 across <30% of the range of primary damage variable | |
Sometimes | Species-level error rate >0.25 across >30% and <50% of the range of primary damage variable | ||
Often | or | Species-level error rate >0.25 over 50% of the range of primary damage variable species-level error rate >0.5 over 30% of the range of primary damage variable | |
Always | Species-level error rate >0.5 over 50% of the range of primary damage variable | ||
Under-predicts | Rarely | Species-level error rate <−0.25 across <30% of the range of primary damage variable | |
Sometimes | Species-level error rate >−0.25 across >30% and <50% of the range of primary damage variable | ||
Often | or | Species-level error rate <−0. 25 over 50% of the range of primary damage variable species-level error rate <−0.5 over 30% of the range of primary damage variable | |
Always | Species-level error rate <−0.5 over 50% of the range of primary damage variable |