Calibration and Validation of Immediate Post-Fire Satellite-Derived Data to Three Severity Metrics

Abstract

Since 2007, the USDA Forest Service’s Remote Sensing Applications Center (RSAC) has been producing fire severity data within the first 30 to 45 days after wildfire containment (i.e., initial assessments [IA]), for wildfires that occur on USDA Forest Service managed lands, to support post-fire management actions. Satellite image-derived map products are produced using calibrations of the relativized differenced normalized burn ratio (RdNBR) to the Composite Burn Index (CBI), percent change in tree basal area (BA), and percent change in canopy cover (CC). Calibrations for extended assessments (EA) based upon one-year post-fire images have previously been published. Given that RdNBR is sensitive to ash cover, which declines with time since fire, RdNBR values that represent total mortality can be different immediately post fire compared with one year post fire. Therefore, new calibrations are required for IAs. In this manuscript, we describe how we modified the EA calibrations to be used for IAs using an adjustment factor to account for changes in ash cover computed through regression of IA and EA RdNBR values. We evaluate whether the accuracy of IA and EA maps are significantly different using ground measurements of live and dead trees, and CBI taken one year post fire in 11 fires in the Sierra Nevada and northwestern California. We compare differences between error matrices using Z-tests of Kappa statistics and differences between mean plot values in mapped categories using Generalized Linear Models (GLM). We also investigate whether map accuracy is dependent upon plot distance from boundaries delineating mapped categories. The IAs and EAs produced similarly accurate broad-scale estimates of tree mortality. Between IAs and EAs of each severity metric, the Kappa statistics of error matrices were not significantly different (P > 0.674) nor were mean plot values within mapped categories (P > 0.077). Plots <30 m (one Landsat pixel) distance from mapped polygon boundaries were less accurate than plots ≥30 m inside mapped polygons (P < 0.001). As land managers concentrate most post-fire management actions where tree mortality is high, it is desirable for map accuracy of severely burned areas to be high. Plots that were ≥30 m inside polygons depicting ≥75 % or ≥90 % BA mortality were correctly classified (producer’s accuracy) >92.3 % of the time, regardless of IA or EA.

Resumen

Desde 2007, el Centro de Sensores Remotos Aplicados (RSAC) del Servicio Forestal de los Estados Unidos de América, ha estado produciendo datos de severidad del fuego a los 30 a 45 días de haber sido controlado el incendio (i.e., determinaciones iniciales [IA]), para apoyar acciones de manejo post-fuego en áreas gestionadas por el Servicio Forestal. Los mapas producidos mediante imágenes de satélite se confeccionan utilizando calibraciones de la diferencia relativa de la tasa normalizada de quema (RdNBR; relativized differenced normalized burn ratio) para el índice de quema compuesta (CBI; composite burn index), porcentaje de cambio en el área basal de los árboles (BA; basal area) y porcentaje de cambio en la cobertura del dosel arbóreo (CC; canopy cover). Las calibraciones para las determinaciones extendidas (EA; extended assessments), basadas en imágenes de un año posterior al fuego, han sido publicadas previamente. Dado que la RdNBR es sensible a la cobertura de ceniza, la cual disminuye a medida que transcurre el tiempo tras el incendio, los valores de RdNBR que representan la mortalidad total, pueden ser diferentes inmediatamente después del fuego que los obtenidos un año después. Por ese motivo, se requieren nuevas calibraciones para las IA. En este trabajo, describimos como hemos modificado las calibraciones de las EA para ser utilizadas por las IA, usando un factor de ajuste que tiene en cuenta cambios en la cobertura de las cenizas, calculado a través de una regresión de valores de IA y EA RdNBR. Evaluamos si la exactitud de los mapas de la IA y la EA son significativamente diferentes utilizando mediciones en terreno de árboles vivos y muertos, y el CBI tomado un año después del fuego en 11 incendios en Sierra Nevada y en el noroeste de California. Comparamos las diferencias entre los errores matriciales con los Z-tests de las estadísticas de Kappa y las diferencias entre los valores medios de las parcelas en categorías mapeadas utilizando el Modelo Lineal Generalizado (GLM; generalizad linear model). También investigamos si la exactitud de los mapas es dependiente de la distancia de la parcela hasta los límites marcados en las categorías mapeadas. Los IA y EA produjeron estimadores de mortalidad de árboles de exactitud similar y en una escala amplia. Entre los IA y EA de cada medida de severidad, las matrices de error según las estadísticas de Kappa no resultaron diferentes de forma significativa (P > 0.674), como tampoco los valores medios de las parcelas en las categorías mapeadas (P > 0.077). Las parcelas a <30 m (un píxel de Landsat) de distancia, desde los límites del polígono mapeado fueron menos exactas que las parcelas a ≥30 m dentro de los polígonos mapeados (P < 0.001). Como los gestores del manejo de tierras concentran la mayor parte de las acciones de manejo post-fuego, en donde la mortalidad de los árboles es grande, es deseable que la exactitud de los mapas sea alta, especialmente en áreas severamente quemadas. Las parcelas que estaban a ≥30 m dentro de los polígonos representando una mortalidad del área basal (BA) de ≥75 % o ≥90 % fueron correctamente clasificadas (exactitud del productor) más del 92.3 % de las veces, independientemente del IA o EA.

Introduction

Multispectral satellite data have become an important data source used by the US federal land management agencies for mapping wildland fire effects, commonly called “severity” maps (Eidenshink et al. 2007, USDA 2007, Parsons et al. 2010). These data not only assist land managers in making post-fire management decisions, but they can provide insights into basic fire ecology, serve as a broad-scale monitoring tool, and provide information on modern fire regimes (e.g., van Wagtendonk and Lutz 2007, Miller et al. 2009b, Dillon et al. 2011, Miller et al. 2012a, Miller et al. 2012b).

There are two closely linked issues that complicate interpreting fire severity maps. First, the degree of severity is based upon an interpretation of how much a site has been altered, which can vary depending upon vegetation type and the expected amount of time it will take a site to recover to the pre-fire state (Lentile et al. 2006, NWCG 2014). For example, a fire that kills all aboveground components of a shrub system may not be considered high severity if the shrub system typically recovers within a few years, as opposed to a conifer forest that could take many decades to recover. Second, severity assessments are often reported in broad categories (i.e., low, moderate, and high) and are generally ambiguous with respect to what the categories mean in terms of quantitative fire effects (e.g., the amount of tree mortality or change in canopy cover [CC]) (Lentile et al. 2006). Therefore, there is a clear need to develop a linkage between satellite-derived severity maps and actual fire effects on the ground. Hence, Miller et al. (2009a) described calibrations of one-year post-fire satellite data to tree mortality and change in CC for conifer forests.

In the 1990s, US land management agencies began using the normalized burn ratio (NBR) spectral index to derive severity maps because of the sensitivity of the Landsat bands used in the index were most sensitive to changes in pre-fire to post-fire vegetation (White et al. 1996, Miller and Yool 2002, Key and Benson 2006b). The NBR is formulated like the normalized difference vegetation index (NDVI) except the Landsat Thematic Mapper (TM) 2.08 µm to 2.35 µm mid-infrared (MIR) band is used in place of the red band, as follows:

$$NBR = \left( {{{NIR - MIR} \over {NIR + MIR}}} \right)$$

(1)

where NIR is the Landsat TM 0.76 µm to 0.90 µm near-infrared band (wavelengths for Landsat 8 are slightly different, but the similar bands can be integrated accordingly). The NIR wavelengths are primarily sensitive to chlorophyll, while the MIR wavelengths used in the NBR are primarily sensitive to water content, ash cover, and soil mineral content (Kokaly et al. 2007). The NBR ranges between −1 and 1 just like NDVI but, in common practice, NBR is scaled by 1000 to transform to integer format (Key and Benson 2006b).

Although first appearing in the formal literature in Miller and Yool (2002), Key and Benson were the first to subtract a post-fire NBR from a pre-fire NBR for mapping fires in an absolute change detection methodology (a differenced NBR [dNBR]) so that barren areas unchanged by fire would not appear as high severity (Key and Benson 2006b). However, dNBR must be individually calibrated for each assessment of fire severity because of disparities in vegetation types and density (Spanner et al. 1990, Miller and Yool 2002, Kokaly et al. 2007). Miller and Thode (2007) described a new spectral index, a relativized version of the differenced normalized burn ratio (RdNBR) that they claimed allowed for calibrations to field-measured variables to be applied to future fires without further field validation. Miller and Thode (2007) modified dNBR by dividing by a function of the pre-fire NBR:

$$RdNBR = \left( {{{dNBR} \over {\sqrt {Absolute\;\left( {{{PreFireNBR} \over {1000}}} \right)}}}} \right)$$

(2)

Miller and Thode (2007) published a calibration of RdNBR derived from one-year post-fire Landsat data (i.e., extended assessments [EA]) to the Composite Burn Index (CBI), a field based severity measure with values ranging from 0 to 3 (unchanged to high severity) that accounts for combined fire effects to surface fuels, understory, and upper canopy (Key and Benson 2006a). Consequently, it can be difficult to relate CBI values to a specific fire effect. Subsequently, Miller et al. (2009a) published calibrations of RdNBR EAs to two quantitative metrics: field-measured percent change in tree CC and basal area (BA). Estimates of all three metrics using RdNBR data are calculated as follows:

$$CBI = \left( {{1 \over {0{.}388}}} \right) \times \ln \left( {{{(RdNBR + 369{.}0)} \over {421{.}7}}} \right)$$

(3)

$$\begin{array}{*{20}c}{Percent\;BA\;change = \quad \quad \quad \quad} \\ {{{\left( {\sin \left( {{{(RdNBR - 166{.}5)} \over {389}}} \right)} \right)}^2} \times 100} \end{array}$$

(4)

$$\begin{array}{*{20}c}{Percent\;CC\;change = \quad \quad \quad \quad} \\ {{{\left( {\sin \left( {{{(RdNBR - 161{.}5)} \over {392{.}6}}} \right)} \right)}^2} \times 100} \end{array}$$

(5)

Miller et al. (2009a) also demonstrated that the calibrations produced fire severity classifications of similar accuracy in fires across a broad region of California that were not used in the original calibration process.

In 2006, the USDA Forest Service, Pacific Southwest Region, began developing methods to map severity to vegetation immediately post fire (i.e., initial assessments [IA] conducted within approximately 30 to 45 days of wildfire containment) to inform post-fire reforestation planning for the national forests in California. Their methods were later adopted by the USDA Forest Service’s Remote Sensing Applications Center (RSAC) under the Rapid Assessment of VeGetation (RAVG) Condition program to extend coverage to all USDA Forest Service lands nationwide (USDA 2007). The RAVG program produces severity map products using the EA calibrations published in Miller et al. (2009a); however, there is one basic difference. Although NBR is primarily sensitive to changes in chlorophyll, the MIR wavelengths employed in the NBR are also sensitive to ash cover, which decreases over time due to wind and rain (Kokaly et al. 2007, Woods and Balfour 2008). As a result, RdNBR values representing areas of complete mortality can differ between IAs and EAs. Therefore, new calibrations for IA post-fire images were necessary. However, there were not any immediate post-fire plot data available to derive new calibrations. The Pacific Southwest Region therefore modified the EA calibrations for CBI, and percent change in tree BA and CC to account for higher levels of ash cover that exist immediately post fire. In this paper, we first present the IA calibrations that were developed for California and were later adopted by RSAC for the national RAVG program. Second, we use EA post-fire ground-based measurements used in Miller et al. (2009a) to assess the accuracy of IAs and compare them with EAs. Finally, considering that fires in forested landscapes can transition from surface to crown fire within 30 m, equal to the width of a Landsat pixel (Safford et al. 2012), we also compare the accuracy of BA ground measurements <30 m and ≥30 m distant from the inside boundary of mapped severity category polygons to determine how plot location and scale of the satellite data impacts classification accuracy.

Methods

Study Area

The fires used in this study were located within the region formed by the Sierra Nevada Forest Plan Amendment (SNFPA) planning area (USDA 2004) plus two national forests in northwestern California (Figure 1, Table 1).

Figure 1

Fires used in this study.

Table 1 Fires in this study and number of plots sampled one year post fire.

The SNFPA planning area, which guides land and resource management on 50 000 km² of national forest land on 11 US national forests, not only includes the Sierra Nevada and its foothills but also the Warner Mountains, Modoc Plateau, White Mountains, Inyo Mountains, and portions of the southern Cascades. Climate is mediterranean type, with warm, dry summers and cool, wet winters; nearly all precipitation falls between October and April (Minnich 2007). Forest vegetation was diverse, with different dominant species and high variation in density and vertical structure. The fires in this study were all located in montane landscapes dominated by coniferous forest, with ponderosa pine (Pinus ponderosa Lawson & C. Lawson), Douglas-fir (Pseudotsuga menziesii [Mirb.] Franco), and hardwoods (mostly Quercus spp.) dominant at lower elevations; white fir (Abies concolor [Gord. & Glend.] Lindl. ex Hildebr.), incense cedar (Calocedrus decurrens [Torr.] Florin), sugar pine (Pinus lambertiana Douglas), ponderosa pine, and Douglas-fir at intermediate elevations; and Jeffrey pine (Pinus jeffreyi Balf.), lodgepole pine (Pinus contorta Loudon), and red fir (A. magnifica A. Murray bis) at the higher elevations. The amount of understory vegetation was largely dependent on stand density, with open stands having more understory vegetation.

Climate in northwestern California was also mediterranean, but somewhat moister on average than the SNFPA area, with higher spatial variability due to proximity to the Pacific Ocean and steep, complex topography (Skinner et al. 2006). While most of the dominant conifer species were shared with the Sierra Nevada and Cascade ranges, overall conifer diversity was higher with correspondingly greater vegetation diversity and complexity (Barbour et al. 2007). Midstory evergreen and deciduous hardwood trees were more abundant than in the Sierra Nevada (Sawyer and Thornburgh 1977).

Although total area burned per year on average increased at the end of the twentieth century, much of the study area had not burned since before the beginning of the twentieth century (Miller et al. 2009b, Miller et al. 2012b, Mallek et al. 2013, Safford and Van de Water 2014).

Severity Data

The severity data for fires used in this study came from a database maintained by the USDA Forest Service’s Pacific Southwest Region. The database contains fire severity data for most large wildfires since 1984 that have occurred at least partially on Forest Service lands in California. A majority of the RdNBR data used to produce the database was acquired from the Monitoring Trends in Burn Severity (MTBS; USDA-DOI 2005) and RAVG programs, but the database also contains many fires that were mapped by the Pacific Southwest Region. The MTBS protocols typically call for mapping fires in conifer forests using dNBR derived from a Landsat postfire image acquired during the summer of the calendar year after fire containment (Key and Benson 2006b, Eidenshink et al. 2007). Although categorical severity maps produced by MTBS are produced from dNBR, MTBS also provides the non-calibrated RdNBR index data for each fire. The RAVG program produces severity maps (CBI, BA loss, and CC loss) within the first 30 to 45 days after wildfire containment. Conforming to best practices for a change detection methodology RAVG, MTBS and the Pacific Southwest Region chose pre- and post-fire images with the best possible matching anniversary dates to minimize differences in sun angle, phenology, etc. (Singh 1989, Key and Benson 2006b). Before applying the calibrations to create categorical severity data, we applied a focal mean in a 3 × 3 pixel moving window to the RdNBR index data, which matched the 90-meter diameter of the field plots used to derive the calibrations, to reduce the number of single-pixel polygons in our database (Miller and Thode 2007). All severity data used in this study were derived from RdNBR, calculated from 30 m Landsat images, and calibrated to field measured CBI, percent change in BA, and percent change in CC.

Initial Assessment Calibrations

It is desirable to use ground reference data acquired near the same time as post-fire satellite data to train and assess the accuracy of derived classifications (Congalton and Green 1999, Jensen 2000). However, adequate postfire plot data were not available when we started to develop an IA methodology (circa 2006 to 2007). The only field data available in sufficient quantity were CBI and tree mortality data that had been collected one year post fire (Miller and Thode 2007, Miller et al. 2009a). Calibration of EA post-fire Landsat-derived RdNBR values to CBI had already been published, and a calibration to percent change in BA had been reported in a draft report (J. Miller and J. Fites, USDA Forest Service, Nevada City, California, USA, unpublished report; Miller and Thode 2007).

As discussed earlier, in the pre- and postfire change detection methodology we employ, wavelengths used in the RdNBR are primarily sensitive to changes in chlorophyll and ash cover (Key and Benson 2006b, Kokaly et al. 2007). However, additional tree mortality can occur for several years after a fire (Hood et al. 2010, van Mantgem et al. 2011). In addition, vegetation response (e.g., sprouting shrubs) in the first year post fire can result in higher chlorophyll levels in areas that experienced complete vegetation mortality (Crotteau et al. 2013). As we could not account for changes in chlorophyll due to delayed tree mortality or post-fire vegetative response using EA postfire tree mortality data, we felt that the best approach to developing calibrations for IAs was to adjust the EA calibrations to account only for differences in ash cover. To determine the average change in RdNBR due to changes in ash cover, we randomly selected areas with the following constraints: 1) they contain at least 1000 pixels representing unchanged conditions outside the fire perimeter and areas of complete tree mortality inside the perimeter, and 2) they were unaffected by post-fire management actions, delayed tree mortality, or vegetative response (e.g., sprouting shrubs). The IA and EA RdNBR values were then regressed using ordinary least squares (OLS) regression. The IA RdNBR values in severely burned areas are primarily a function of ash cover and soil mineral content (Kokaly et al. 2007). We used EA RdNBR values as the independent variable as a proxy for soil mineral content in the regression model. Differences between pre- and post-fire images are always removed before computing RdNBR by subtracting the mean of unchanged areas from dNBR before computing RdNBR. The RdNBR values for areas unaffected by fire or management actions are therefore typically normally distributed with a mean of zero as long as calendar dates of the pre- and post-fire images are similar (Key and Benson 2006b, Miller and Thode 2007). Consequently, we did not include the intercept term in the OLS regression model. The slopes of the regressions for a small representative set of fires were averaged and then incorporated as a linear scale factor into the EA post-fire calibrations. At the time (circa 2006 to 2007), Landsat images cost around $600 per scene and our funding for acquiring images was limited. In addition, through our experience in working with postfire satellite imagery, we knew that there can be a high degree of variability between fires, and sometimes within fires, in the amount of decline in ash cover during the first year post fire. Therefore it would require only a small set of fires to obtain a rough estimate (within one SD) for the mean of ash cover decline, and a larger set of fires would not result in a more robust estimate.

Accuracy Assessment

Plot data. For assessing the accuracy of the IA calibrations, we leveraged plot data gathered one year post fire that was used in Miller and Thode (2007) and Miller et al. (2009a) for which we had both initial and extended RdNBR-based severity assessments already entered in our severity database (Table 1). The plot data were collected by different teams and methods for ground-based assessment methods of CBI severity, and forest metrics were slightly different. As a result, plot size varied between years. Plots were circular with 90 m diameter for fires that occurred from 2001 to 2002, 60 m for 2004 fires, and 30 m for 2006 fires. Plots sampled in the SNFPA planning area were located at least 300 m apart on randomly placed transects. A stratified random procedure was used to generate potential plot locations for the 2006 fires in northwest California using a preliminary severity map. The CBI was not collected in fires that occurred during 2001, but was evaluated over the entire plot in subsequent years. Data collected to characterize fire effects on trees included status (live or dead), species, diameter at breast height (dbh), scorch height, crown base height (CBH), and percent of crown volume scorched. Tree data were collected for each tree in the 30 m diameter plots. The numbers of trees less than 10 cm in diameter were counted by species. In plots that were 60 m and 90 m in diameter, tree data were measured in four 10 m diameter subplots and tallied into 10-centimenter size classes. We estimated whether dead trees were killed by the fire or were already dead prior to the fire by presence or absence of dead needles as well as bark and wood consumption patterns. Live pre-fire trees are rarely consumed by fire (Pyne et al. 1996). But survival of pre-fire snags is dependent upon snag condition and fire intensity (Laudenslayer 1997, Skinner 2002). Pre- and post-fire BA was calculated from the diameters of trees thought to have been alive prior to the fire and alive after the fire, respectively. Pre- and post-fire CC was estimated using the Forest Vegetation Simulator (FVS; Dixon 2002). The FVS-derived estimates of individual tree crown cover assume that trees are healthy and unaffected by fire or disease. We therefore adjusted FVS-computed CC using equations for modeling crown shape for northern California conifer species (Biging and Wensel 1990). (See Miller et al. [2009a] for more complete details on the CC calculations.) As USDA Forest Service vegetation classification standards require 10 % tree CC for an area to be classified as forested, we only used plots in this study for which FVS indicated ≥10% pre-fire tree CC (Brohman and Bryant 2005).

Analyses. We evaluated classification accuracy using error matrices detailing producer’s, user’s, and overall accuracies and estimates of the Kappa statistic. Producer’s accuracy (1 — omission error) is an evaluation of when a plot is not mapped in the correct category (i.e., a Type II error). User’s accuracy (1 — commission error) is an evaluation of when a plot is mapped in the wrong category (i.e., a Type I error). Kappa is a measure of the difference between the actual agreement between reference data and classified data, and the chance agreement between the reference data and randomly classified data. Classifications can be statistically compared with Z-tests to determine whether classifications are significantly different (Congalton and Green 1999). We compared Kappa statistics of IAs and EAs, as well as Kappa statistics of plots that were <30 m and ≥30 m from mapped category boundaries of percent change in BA (Table 2).

Table 2 Number of plots by distance from edge of severity category polygon, severity category and assessment type. Lower case “a” and “b” denote initial and extended assessments.

We used a Generalized Linear Model (GLM) to determine whether plot measured values (CBI, and percent change in CC and BA) differed between assessment types. We also checked to see whether mapped severity categories differed within each assessment type. In the GLM, effects were therefore assessment type crossed with mapped severity category. The GLM was run twice, partitioning the interaction effects by assessment type and mapped severity category in separate runs. One advantage to GLMs is that they allow for response variables that are not normally distributed. However, there is still a requirement that model residuals be normally distributed (Nelder and Wedderburn 1972). The percent change data (BA and CC) did not effectively fit any of the standard distributions provided in GLM algorithms due to the high number of 0 % and 100 % values. Instead, we found that normality assumptions were best met by first transforming them using arcsine-square root. Therefore, a Gaussian distribution with an identity link function was used in the GLMs for all severity metrics. P-values were adjusted using the Tukey-Kramer method to account for multiple comparisons (Kramer 1956). Finally, we produced histograms of live tree BA in the highest BA mortality severity categories (≥75 % and ≥90 %) to illustrate how distance from the edge of mapped polygons (<30 m, ≥30 m, and all plots) affects accuracy.

Results

Initial Assessment Calibrations

The OLS regression slopes of initial to EA RdNBR values ranged between 1.069 and 1.193 (Table 3), resulting in a mean of 1.144 (SD = 0.053). The equations used to estimate CBI, percent change in BA, and percent change in CC from EA RdNBR values were modified as follows for IAs (Table 3: Miller et al. 2009a):

$$CBI = \left( {{1 \over {0{.}388}}} \right) \times \ln \left( {{{\left( {{{RdNBR} \over {1{.}144}} + 369{.}0} \right)} \over {421{.}7}}} \right)$$

(6)

$$\begin{array}{*{20}c}{Percent\;BA\;change = \quad \quad \quad \quad \;} \\ {{{\left( {\sin \left( {{{\left( {{{RdNBR} \over {1{.}144}} - 166{.}5} \right)} \over {389}}} \right)} \right)}^2} \times 100} \end{array}$$

(7)

$$\begin{array}{*{20}c}{Percent\;CC\;change = \quad \quad \quad \quad \;} \\ {{{\left( {sin \left( {{{\left( {{{RdNBR} \over {1{.}144}} - 161{.}0} \right)} \over {392{.}6}}} \right)} \right)}^2} \times 100} \end{array}$$

(8)

Table 3 Fires used to estimate an average change in high severity RdNBR values between initial and extended assessments due to a decrease in ash cover. Regression slopes from ordinary least squares regression without an intercept in the model (all regressions R² _adj ≥ 0.98, P < 0.001).

Accuracy Assessment

Initial vs. extended assessments. There were no significant differences between IA and EA Kappa statistics (Figure 2, all P ≥ 0.674). Comparing classifications of all plots, IA user’s and producer’s accuracies for the highest severity categories were greater than EAs, except for CBI (Table 4). However, there were no significant differences between mean plot values of IA and EA classification categories (Figure 3, P = 0.077 for BA ≥10 % and <25 %, all other P ≥ 0.143).

Figure 2

Kappa statistic (blue bars) and overall classification accuracy (hollow bars with black outlines) for initial and extended assessment error matrices of Composite Burn Index (CBI), percent change in canopy cover (CC), and basal area (BA) classifications. BA4 and BA7 refer to the four-category and seven-category basal area classifications. None of the comparisons of initial and extended assessment Kappa statistics are significantly different (all P > 0.674; Z ≥ 5.11).

Figure 3

Generalized Linear Model results comparing plot values in mapped categories between and within initial and extended assessments: A) Composite Burn Index (CBI), B) percent change in canopy cover (CC), C) four categories of percent change in basal area (BA4), and D) seven categories of percent change in basal area (BA7). Bars represent plot measured means within each severity category. Blue bars are initial assessments and gray bars are extended assessments. Bars labeled with different letters (small letters for initial assessments, capital letters for extended assessments) indicate P-values of comparisons between categories within each assessment are significantly different at P < 0.05. P-values were adjusted using the Tukey-Kramer method. There were not any significant differences in mean plot values between initial and extended assessments.

Table 4 User’s and producer’s accuracies for initial assessments and extended assessments for all plots and for plots that were <30 m and ≥30 m from the edge of the severity category polygon in basal area (BA) classifications.

User’s and producer’s accuracies were best for the highest severity categories, regardless of severity metric (Table 4). Accuracies for lower severity categories of the percent change in BA and CC indicated that classification of individual plots was not much better than random. Accuracies of the low and middle CBI categories were a little better than the percent change in BA and CC.

Regardless of assessment, differences between categories were significant for the four-category CBI and BA classifications (BA4; Figures 3A and 3C). When the number of categories increased, so did standard errors, resulting in differences between some adjacent categories not being significant (Figures 3B and 3D). Means of field-measured values in the lowest severity categories, regardless of assessment, were greater than what was expected in the mapped categories (Figure 3). Means of field values for all other categories generally fell within the category ranges except for the seven-category BA classification (BA7).

Distance from polygon boundary. Overall accuracy and Kappa values of plots ≥30 m from the mapped percent change in BA category boundary were higher when than when they were <30 m from the boundary (P < 0.001;

Figure 4). User’s and producer’s accuracies were highest, ranging from 89.9 % to 100 % for the highest (≥75 % or ≥90 %) severity categories when plots were ≥30 m from the polygon boundary (Table 4). In contrast, user’s and producer’s accuracies ranged from 38.6 % to 67.9 % when plots were <30 m from the polygon boundary. There were not any live trees in 81 % to 88 % of plots ≥30 m inside the highest severity polygons (EA ≥75 % BA change and IA ≥90 % BA change, respectively; Figure 5). In contrast, there were not any live trees in only 35 % to 62 % (EA ≥75 % BA change and IA ≥90 % BA change, respectively) of plots <30 m inside the highest severity polygons.

Figure 4

Kappa statistic (blue bars) and overall classification accuracy (hollow bars with black outlines) for error matrices using plots that were either <30 m or ≥30 m distant from the edge of mapped category boundaries. Kappa statistics for <30 m and ≥30 m error matrices are all significantly different at P < 0.001 (Z ≤ 0.42).

Figure 5

Frequency of live tree basal area in field plots mapped at the highest severity in the percent basal area (BA) change classifications. Frequencies are for plots that were <30 m and ≥30 m inside the highest severity polygon and all plots. Figures display field plots classified either ≥90 % or ≥75 % BA change by initial and extended assessments: A) initial assessment ≥90 % BA change, B) extended assessment ≥90 % BA change, C) initial assessment ≥75 % BA change, and D) extended assessment ≥75 % BA change.

Discussion

The modified EA calibrations that we used for IAs were based upon OLS regression of IA and EA RdNBR values from four fires. We only used four fires in part because of the available data, but we also assumed that a small sample would produce a mean value that would be within one SD of the mean of a larger sample because of the high variation in ash cover decline between fires. In order to confirm our assumption that a four-fire sample was adequate, we computed the mean of IA to EA regression slopes of all 12 fires (mean = 1.126, SD = 0.110, results not shown). The difference between the means from the four and twelve fire samples (0.018) was less than one SD of the four-fire sample (0.053), and a t-test of the difference between means was not significant (P = 0.38).

The IA calibrations derived using the four-fire sample adjustment factor produced classifications of similar accuracy to EA calibrations. Overall classification accuracies were similar and Kappa statistics were not significantly different between assessment types (Figure 2). Comparing category by category, there were not any significant differences in mean plot values between IAs and EAs (Figure 3). Additionally, in a separate analysis, Safford et al. (2015) found no significant difference in the percentage of high severity mapped by IAs and EAs within fires that occurred primarily in conifer forests on Forest Service managed lands in the Sierra Nevada (24.16 % vs. 23.99 %, IAs and EAs respectively; paired t-test P = 0.346, N = 53).

User’s and producer’s accuracies for low and middle severity categories were low for at least two reasons (Table 4). First, in most of the low and middle categories, there were more than twice as many plots <30 m from mapped category boundaries compared to the number of plots ≥30 m distant from boundaries (Table 2). It is therefore important to consider the registration accuracy of the satellite images and scale of the severity maps when using them for post-fire project planning. Plot location in relation to the boundary of the mapped severity polygons is important. Plots do not perfectly align with the 30 m satellite pixel, and as fire can transition from surface to crown within 30 m (Safford et al. 2012), fire effects can vary considerably within a pixel, resulting in lower accuracy for plots in the pixel adjacent to the edge of the polygon. The middle severity categories often occur in narrow bands around high severity patches. When the number of severity categories increases (i.e., BA4 vs. BA7), those bands decrease in width to only one to two pixels wide in many locations, leading to confusion between severity categories (Figure 3). Considering that pixels can be misregistered up to a pixel, it can also be impossible to identify the exact location of those moderate severity pixels on the ground. Second, mean plot values in the lowest severity categories were greater than indicated by the mapped categories (Figure 3). Kane et al. (2013) also detected tree mortality using LIDAR data in unchanged and low categories of maps calibrated to the CBI. All but two of the fires in our study occurred in areas that had not burned since at least the early 1900s. Numerous stand composition studies throughout our study area have shown that areas that experienced frequent low- to mixed-severity pre-settlement fires have contemporary stand structures distinguished by relatively small numbers of old trees (>300 yr) and abundant younger trees that established after fire suppression began (Vankat and Major 1978, Taylor and Skinner 2003, Leonzo and Keyes 2010, Scholl and Taylor 2010, Collins et al. 2011). Under- and mid-story trees can be killed in low intensity surface fire, which may not be detectable by a passive satellite (e.g., Landsat) under dense forest canopies (Stenback and Congalton 1990), leading to higher actual mortality measured in plots in the lower severity categories. Percent change in CC was based upon FVS calculations of pre- and post-fire live trees. The FVS estimates of CC were corrected for canopy overlap, but FVS uses random tree placement (Crookston and Stage 1999, Miller et al. 2009a). Therefore, the percent changes in CC may not represent actual CC as seen from overhead by a satellite, thereby also leading to higher than expected CC change in the lowest severity categories. As a result, it may be more appropriate to combine the 0% category (BA or CC) with the next higher category (Kolden et al. 2012). When the 0 % and >0 % and <25 % change categories are combined, user’s and producer’s accuracies increase from ≥26.0 % to ≥62.4 % (results not shown).

Accuracies of the low and middle CBI categories were a little better than the percent change in BA and CC metrics, perhaps because CBI is a composite measurement representing effects from surface fuels, understory vegetation, and the upper canopy (Key and Benson 2006a). However, the CBI protocol relies entirely upon ocular estimates, which can lead to considerable error (Korhonen et al. 2006).

Plot measured values we report for the lowest to mid-severity categories may not be representative of other vegetation types or of forests in other regions that are more or less productive, comprised of different species, or do not have a similar fire suppression history. However, it is likely that accuracies of the highest severity categories will be similar because our methodology is dependent upon relative change between pre- and post-fire images (i.e., stand replacement regardless of vegetation type is always 100 % change).

There are two major factors that can contribute to differences between IAs and EAs. First, with the time delay between IA and EA image dates, any additional tree mortality or resprouting or recovery of live vegetation can alter measured effects. Second, when fires are contained late in the calendar year, IA post-fire images may be acquired when the sun elevation angle is low. Low sun angles, especially after the fall equinox at latitudes in our study area or farther north, can affect image quality in two ways: 1) the sun does not fully illuminate north-facing slopes in steep terrain, and 2) it can be difficult even on level terrain to see through tree canopies to the understory (Holben and Justice 1980, Dymond and Qi 1997). When containment dates are late in the calendar year, a cloud-free post-fire image may not be available until the next spring due to cloud cover, thereby making an IA unfeasible. Without immediate post-fire plot data re-measured one year post fire, we cannot assess whether delayed mortality was a factor in our classification accuracy. Consequently, we attempted to model only change in ash cover when modifying the EA calibrations. Some of our IA post-fire images were late in the calendar year (Table 1), so we therefore checked to see whether the IA RdNBR values for our plots were affected by steep north-facing slopes, and none appeared to be affected by topographic shadows. Canopy shadowing may have been an issue as evidenced by lower IA accuracies of the lowest severity categories (Table 4), but there were not significant differences between assessment types (Figure 3).

From a management perspective, it is desirable for the highest severity categories to be accurate because that is where most potential ecosystem damage occurs and where most post-fire management actions occur. User’s and producer’s accuracies of the high severity category ranged from 74.5 % to 86.6 %. But, more than half (54 % to 67 %) of the plots in our study were <30 m from the boundary, between different mapped severity categories of percent change in BA (Table 2). When considering plots ≥30 m from the boundary, user’s and producer’s accuracies were ≥89.9 % (Table 4). More than 80 % of the plots ≥30 m from the boundary in the highest severity category had no live trees (Figure 5). As accuracy of IAs and EAs is similar, both appear to be adequate for producing broad-scale estimates and in identifying locations of complete tree mortality.

Currently, the calibrations presented in this manuscript are being utilized by the RAVG program for all initial post-fire assessments across the United States. We are confident that, for severely burned areas, the calibrations result in similar accuracies regardless of region and vegetation type. However, mortality in the low and moderate severity categories may differ because of different forest structures. When resources become available, regionally adapted models may need to be created where appropriate.