Reliability of cross-regional applications of global fire danger models: a Peruvian case study

Abstract

Background

Fire danger indexes (FDIs) are used as proxies for fire potential and are often developed for specific locations. For practical purposes, the extrapolation of the underlying calculations into novel locations is common, but it is generally uncertain if the relationships between FDIs and fire potential observed in the environment in which the index was developed are equally relevant in others. For example, although a topographically, ecologically, and climatologically complex country, f ire danger forecasts in Peru use a standard set of nationwide thresholds applied to the Fire Weather Index. In this study, we validate the underlying assumption that weather-fire relationships are spatially uniform within Peru by (1) making cross-regional comparisons of the statistical distributions of four FDIs—Burning Index, Energy Release Component, Fire Weather Index, and Keetch-Byram Drought Index, and (2) making cross-regional comparisons of the expected daily MODIS hotspot count percentiles conditioned on FDI values.

Results

Significant regional differences in the distributions of daily FDI values were observed in every pair of regions within Peru, and with the exception of a pair of regions within the Amazon, little data ($<90$ days) were necessary to detect these differences. After controlling for FDI values and seasonal and annual effects with regressions, differences in predicted hotspot percentiles were common, differing by as much as 47 percentage points. Across the pairs of regions, the magnitude of these differences tended to decrease as climatic similarity increased, but some counterexamples were also apparent.

Conclusions

The noticeable differences in the distributions of daily FDI values suggest that a standard set of breakpoints may produce unreliable inferences regarding fire potential. We also find that even if the climatic conditions were similar across Peru, the same FDI values in two locations can produce substantially differing predictions of wildfire activity. This suggests that other factors besides FDI values can strongly mediate wildfire activity and that better fire potential predictions could be produced if these factors are accounted for.

Résumé

Antecedentes

Los índices de peligro de incendio (FDIs) son usados como indicadores del peligro de incendio para lugares específicos. Para propósitos prácticos, es común la extrapolación de cálculos previos para lugares sin datos previos, pero éstos son generalmente inciertos si las relaciones entre los FDIs y el peligro potencial del fuego en el ambiente en el cual el índice fue desarrollado es igualmente relevante que en otros. Por ejemplo, aunque el Perú es un lugar topográfica, ecológica, y climatológicamente complejo, el pronóstico de peligro de incendios usado a nivel país se basa en un conjunto de umbrales standard aplicando el índice de Peligro de incendios (FWI). En este estudio validamos las suposiciones de que las relaciones entre el tiempo meteorológico y el fuego son espacialmente uniformes dentro del Perú mediante 1) una comparación regional cruzada de las distribuciones de cuatro FDIs - índice de Quema, Componente de Liberación de Energía, el Fire Weather Index (FWI), y el índice de sequía de Keetch-Byram- y 2) mediante comparaciones regionales cruzadas obtenidas mediante percentiles del sensor de puntos calientes de MODIS condicionados sobre valores de FDI.

Resultados

Diferencias regionales significativas en la distribución diaria de los valores de FDI fueron observados en cada par de regiones dentro del Perú y, con la excepción de regiones pares dentro de la Amazonía, muy pocos datos (< 90 días) fueron necesarios para detectar esas diferencias. Después de controlar valores de FDI, efectos estacionales y anuales mediante regresiones, fueron notables las diferencias entre valores predichos y los puntos calientes, difiriendo en hasta 47 puntos porcentuales. Entre los pares de regiones, la magnitud de esas diferencias tendió a decrecer a medida que la similitud climática aumentaba, aunque algunos ejemplos contradictorios fueron también aparentes.

Conclusiones

Las diferencias notables en la distribución de los valores de FDI sugieren que un conjunto standard de valores críticos puede producir inferencias inciertas relativas al peligro de incendio. Encontramos también que aún cuando las condiciones climáticas fueran similares a través de todo el Perú, los mismos valores de FDI en dos lugares diferentes pueden producir predicciones substancialmente diferentes en la actividad del fuego. Esto sugiere que otros factores más allá de los valores del FDI pueden mediar en la actividad del fuego, y que mejores predicciones potenciales pueden ser producidas si esos factores fuesen considerados.

Background

Fire danger indexes (FDIs) are quantities that can be estimated from meteorological data that communicate information regarding the future likelihood and magnitude of wildfire parameters—hereafter fire potential. FDIs were developed to support decision-making for various firefighting and land management activities. Although a large suite of drought and fuel moisture indexes are available (Zargar et al. 2011; Littell et al. 2016), only a subset are commonly used as FDIs by fire researchers and practitioners. The Keetch-Byram Drought Index (KBDI), Energy Release Component (ERC), Burning Index (BI), and Canadian Fire Weather Index (FWI) are four examples of commonly used FDIs. The KBDI was developed to predict forest fire activity in the Southeastern United States and is based on a water balance model of the upper soil layers (Littell et al. 2016; Keetch and Byram 1968). The ERC and BI are used in the USA to inform firefighting decisions (Jolly et al. 2015; Cullen et al. 2020) and are calculated from a complex equation of temperature, precipitation, wind, humidity, cloud cover, and sometimes fuel, topographic, and latitude data (Deeming et al. 1977; Bradshaw et al. 1984). Structured similarly to the BI (Fujioka et al. 2008), the FWI was developed in the 1970s and is the preferred choice of FDI in Canada (Van Wagner et al. 1974). Developing and validating FDIs is a highly technical, labor-intensive, and costly task that can be difficult to accomplish in more resource-limited contexts. Consequently, it is common practice to predict fire potential using FDIs that were originally developed for other geographic contexts.

For instance, KBDI was first developed to communicate fire potential in the Southeastern United States (Littell et al. 2016; Keetch and Byram 1968), but it has since been applied in other locations, including portions of the contiguous USA (Lorimer and Gough 1988; Carlson et al. 2002), Hawaii (Dolling et al. 2009), Australia (Hatton et al. 1988), Turkey (Varol and Ertuğrul 2016; Fujioka et al. 2008), the Mediterranean (Garcia-Prats et al. 2015), and Malaysia (Livingston 1974). FWI is widely used operationally across the globe (Vitolo et al. 2020) and has been applied in locations including Portugal (Carvalho et al. 2008), Greece (Dimitrakopoulos et al. 2011), China (Tian et al. 2011), the UK (Jong et al. 2016), Argentina (Cardenas et al. 2013), and Peru (SENAMHI 2018). In contrast, operational use of ERC and BI is largely limited to the USA, and few studies have experimented with the application of these indexes for fire potential prediction in novel geographic contexts (Van Wilgen 1984; Shmuel and Heifetz 2022). When used as fire potential proxies in global analyses, FDIs are implicitly applied in novel geographies, and BI (Jolly et al. 2015), KBDI (Gannon and Steinberg 2021; Liu et al. 2010), and FWI (Abatzoglou et al. 2019; Vitolo et al. 2019) have been used in such global fire potential analyses. Even locations that are perceived to be within the same geographic domain in which the FDI was developed for can have conditions that the FDI was never tested in. Environmental and anthropogenic conditions in which the FDIs were developed can be defined with multiple criteria (e.g., climate, vegetation, land management, season), and the real-world application of FDIs inevitably brings about a combination of conditions that were never considered in the original formulation of the FDI. As an example, although KBDI was developed to predict forest fires in the Southeastern United States (Littell et al. 2016), it assumes deep soil layers. KBDI might not then be expected to accurately reflect fire potential in a location with low fuel quantities (Keetch and Byram 1968), even if in the Southeastern United States. Given that many of the environmental and anthropogenic conditions that mediate fire potential—such as fuel type and quantity—vary in space and time (Newman et al. 2019), there are ubiquitous opportunities to unintentionally apply FDIs to conditions that differ from those that the index was developed for.

Even in these novel conditions, FDIs will often continue to be an adequate proxy for fire potential. For instance, KBDI is intended to measure deep soil moisture, but it is still reported to be a reasonable proxy for fire potential in the Amazon (Cavalcante et al. 2021), where fire typically burns in the shallow surface litter layer (Cochrane 2003; Ray et al. 2005). However, this robustness is not guaranteed and, without explicit validation, it can be unclear when and where the expected relationships between the FDIs and fire potential will breakdown (Regan et al. 2002; Hájek 2007). For instance, although KBDI is an important predictor of the annual number of wildfires in the Southeastern United States (Addington et al. 2015), the relationship has been shown to breakdown when used to predict fire size in the central USA (Krueger et al. 2017), and in general, an FDI’s predictive ability can breakdown even between locations that are spatially adjacent (Nogueira et al. 2017). Part of this breakdown can be explained by the fact that the statistical distribution of a FDI likely differs between regions. In other words, FDI amounts considered abnormal in one region may be significantly more common in another (Vitolo et al. 2019), and as a result, any presumed effect of a FDI on fire potential would not be consistently observed across locations. For this reason, absolute FDI quantities are often converted into percentiles so that the relative rarity of the fire weather conditions can be communicated (Jolly et al. 2019). Still, this scaling of FDI values cannot always account for other confounding non-meteorological factors that also mediate fire potential. For instance, rescaling FDIs cannot account for the fact that long-term precipitation can have differing effects on fire activity depending on the ecological context. In forested locations, low precipitation can increase fire potential since flammable biomass is abundant but rarely dry, whereas in non-forested locations low precipitation can decrease fire potential since flammable biomass is often dry but not always in sufficient quantity to carry large fires (Meyn et al. 2007). Human activities are another confounding factor to consider as they can influence fire-weather relationships through multiple means (Syphard et al. 2007; Rodrigues et al. 2018; Monjarás-Vega et al. 2020) and, in some cases, may be of greater importance than meteorological predictors (Syphard et al. 2017). Topographic factors can directly influence fire behaviors (Deeming et al. 1977; Bradshaw et al. 1984) and also mediate suppression effectiveness (Parisien et al. 2012; Silva et al. 2020). In addition to the spatial context, the relationship between FDIs and fire potential can sometimes breakdown if FDIs are applied outside specific temporal contexts. In China, for example, although KBDI is a strong predictor of seasonal fire counts and burned area, it is less useful than other fire proxies when predicting annual wildfire parameters (Zhao and Liu 2021). Similarly, in the Great Plains region, the predictive ability of some FDIs has been observed to be seasonally dependent (Krueger et al. 2015). Hence, FDI values are at best an incomplete proxy for fire potential, and these confounding factors might explain why wildfire parameters can differ in two distinct locations that report the same FDI values.

Within Peru, the FWI is the preferred choice of FDI for identifying locations with elevated fire activity. To support fire fighting and public safety decisions, the National Meteorology and Hydrology Service of Peru (SENAMHI) applies a standard set of thresholds (SENAMHI 2018) to gridded FWI data to classify each location into one of five fire danger categories (Eq. 1).

$$\begin{aligned} d(FWI)= \left\{ \begin{array}{ll} \text {Low}, &{} \text {if } FWI\le 6\\ \text {Moderate}, &{} \text {if } 6< FWI\le 12\\ \text {High}, &{} \text {if } 12< FWI\le 18\\ \text {Very-high}, &{} \text {if } 18< FWI\le 24\\ \text {Extreme}, &{} \text {otherwise} \end{array}\right. . \end{aligned}$$

(1)

Peru is also climatically and topographically variable. Dry lowland forests exist in the northwestern coastal portions of the country and a desert-arid subtropical climate southward along the coast. The Andes mountains dissect the country along a North-South transect. The Andes mountains are topographically complex and predominately characterized by grassland and shrubland vegetation, with very small and isolated high Andean forests. The eastern portion of the country is part of the Amazon rainforest and is characterized by high humidity and warm temperatures with varying topographic complexity depending on the proximity to the Andes (SENAMHI 2021). Given the issues highlighted in the previous paragraph, it is not obvious that the standard set of thresholds applied to FWI will adequately capture fire potential in all regions. In other words, it is unclear whether the statistical distributions of FWI are sufficiently uniform across the country to use a standard set of thresholds to classify fire danger. Additionally, it is unclear whether the response of wildfires to changes in FWI is uniform across space and time.

In this study, we will empirically test both the assumptions of this fire danger classification model using a set of three statistical analyses. Firstly, we will compare descriptive statistics obtained from daily averages of four FDIs across bioclimatic geographic units in Peru. Secondly, we will use these data to determine the level of statistical similarity in the distributions of daily-average FDI values between the same bioclimatic geographic units. Thirdly, we will describe the relative sensitivity of wildfire activity to changes in FDI values between the same bioclimatic geographic units.

Methods

Data and preprocessing

Hourly ERA5-land reanalysis data for 2003–2020 were obtained from the European Centre for Medium-Range Weather Forecasts (ECMWF) using the CDS Python API (Muñoz-Sabater et al. 2021). These data first were summarized to daily values and then were used to derive gridded FDIs for four globally relevant indices: the US Burning Index (BI) and Energy Release Component (ERC) (Bradshaw et al. 1984), Fire Weather Index (FWI) (Van Wagner and Pickett 1985), and Keetch-Byram Drought Index (KBDI) (Keetch and Byram 1968). BI and ERC were computed using Fuel Model G for spatial consistency (Jolly et al. 2015). FWI was computed using latitude corrections for the tropics (Lawson and Armitage 2008). For each FDI, a daily average time series was calculated from the gridded data for six zones. These zones were derived by blending Peruvian administrative boundaries (departments) with climate zones defined by SENAMHI (SENAMHI 2021) to create regions that were generally similar in terms of climate, vegetation, and topography. The administrative boundaries were determined as a priority factor following the official decision-making structure governing the country. These clusters of departments were verified in consultation with local experts. Additionally, MODIS hotspot data for 2003–2020 are used to produce a daily time series of hotspot counts for each of the six zones (Fig. 1).

Fig. 1

Regions and wildfire activity. Map of the biogeographical units considered in this study with 2003–2020 MODIS hotspot detects overlaid, and probability mass functions of MODIS hotspot occurrence. Biogeographical units include the Bosque Seco (dry deciduous forest), Selva Alta (humid upland tropical forest), Selva Baja (humid lowland tropical forest), Transicion (Andean rainforest ecotone), Sierra (Andean vegetation), and Arido (arid vegetation)

Summary statistics and power analysis

The empirical cumulative distribution function was calculated using each of the 24 time series (4 FDIs × 6 regions). The empirical cumulative distribution function was used to find the $\text{25th}$, $\text{50th}$, and $75\text{th}$ percentiles, which are used to estimate the central tendency (median) and the dispersion (interquartile range). Additionally, for a given FDI and pair of regions, the similarity of the two distributions is measured using a model credibility index (Lindsay and Liu 2009). We define the model credibility index, $N_{50}$, as the minimum sample size such that a two-sample Kolmogorov-Smirnov test is expected to correctly reject the null hypothesis (at an $\alpha =0.01$ significance level) with 0.5 probability. In other words, the model credibility index communicates how much data are required until known differences in the two probability distributions have a 50-50 chance of being detected. If the distributions of the FDIs are very different in two different regions, then the model credibility index will be low. If the distributions are similar, then the model credibility index will be high.¹ We estimate the model credibility index via resampling, where the statistical power of a two-sample Kolmogorov-Smirnov is estimated from 1000 random samples of size n for each $n \in {10,11,12,\cdots ,400}$.

Comparison of MODIS hotspots to FDIs

For each of the six region, four generalized linear models were fit assuming a log-link and a Poisson distribution (Eq. 2).

$$\begin{aligned} E[Y|FDI,A,S]=f(FDI,A,S)=\text {exp}\{\beta _3FDI + \beta _2A+ \beta _1S + \beta _0 \}. \end{aligned}$$

(2)

Here, the random variable (Y) represents the daily total MODIS hotspot counts and the expected value is assumed to be a function (f) of three covariates (daily average FDI and two dummy variables for annual and seasonal effects). Each of the four seasons was defined using sets of three consecutive months (e.g., December–February, March–May, June–August, September–November). The daily total MODIS hotspot counts were also used to estimate the empirical cumulative distribution function (g) (Eq. 3). The composite of the conditional expected value of hotspot counts produced from the regression and the empirical cumulative distribution functions (h) was then calculated as a proxy for the expected relative fire potential conditional on season, year, and FDI (Eq. 4).

$$\begin{aligned} g(y;Y)=\frac{1}{n}\sum \mathbbm {1}_{y<Y_i}; \end{aligned}$$

(3)

$$\begin{aligned} h(FDI,A,S,Y)=g \circ f. \end{aligned}$$

(4)

Note here y represents arbitrary hotspot value and Y is our sample of observed daily hotspot values. The difference in h—the percentile of the predicted hotspot counts—was calculated for all ${6 \atopwithdelims ()2} = 15$ possible combinations of regions (Fig. 1) to assess the relative fire potential conditional on days with identical fire weather. The maximum possible absolute difference in h, across all seasons, was recorded for each pair of regions to measure the maximum dissimilarity of predicted wildfire activity when FDI values are identical. The ability of the models to recreate historic trends in MODIS hotspot counts was represented using percent error as a performance statistic, which was estimated using resubstitution (Rafało 2021). The scatterplot of the model credibility index and the maximum absolute difference in h was produced for each FDI to identify the similarity of regions in terms of climate and fire-weather relationships. Moreover, the Pearson correlation coefficient of these two quantities was estimated to determine the degree to which cross-regional climatic similarity implied similar fire-weather responses. All calculations were performed using the R programming language (Computing et al. 2013).

Results

Descriptive statistics

Descriptive statistics calculated for each FDI revealed where pairs of biogeographical regions had similar fire weather. The largest differences were apparent between the tropical rainforest regions (Selva Alta and Selva Baja) and the dry coastal regions (Bosque Seco and Arido), with the median FWI, BI, and ERC being noticeably lower in the tropical rainforests than in the dry coastal regions. The median FWI was particularly sensitive to changes in location, and this value differed by as much as a factor of 60, ranging from 0.45 in the Selva Alta to 27.79 in the Bosque Seco. Even when two regions had similar median FDI values, the probability distributions could differ in terms of other statistics. With values of 32.95 and 34.69 respectively, the median ERC value was fairly similar in the Arido and Bosque Seco regions, but the interquartile range was nearly twice as high in the former (22.27) than in the latter (11.61). Likewise, the median FWI value was fairly similar in the Transicion (2.09) and Sierra (2.49) regions, but the interquartile ranges were 2.97 and 7.64, respectively. The smallest regional differences were observed in tropical forest regions. In the Selva Alta and Selva Baja, the median and interquartile range of FWI, BI, and ERC were similar, although noticeable differences in the median and interquartile range of KBDI were apparent (Fig. 2, Table 1).

Fig. 2

Cumulative distribution functions. Regional empirical cumulative distribution functions for a FWI, b KBDI, c BI, and d ERC. The gray line identifies the median of each distribution and the thicker portion of the curve identifies the interquartile range of each distribution

Table 1 Regional daily average fire danger index quartiles

Power analysis

The apparent differences observed from the graphical inspection of empirical cumulative distribution curves and comparisons of descriptive statistics were confirmed by the application of two-sample Kolmogorov-Smirnov tests. For each FDI and pair of regions, in every case, the null hypothesis of a common probability distribution generating both samples was rejected. Because it was known a priori that these probability distributions should be different, the model credibility index was used to gauge how well each pair of probability distributions approximates one another. Differences in the distributions usually required little data to detect. At an $\alpha =0.01$ significance level, a sample size of 30 was sufficient to conclude the data were not generated from the same probability distribution for most FDIs and pairs of regions (Table 2). In the tropical forests, where FDI values tend to be similarly distributed, more data were usually needed. Specifically, in the Selva Alta and Selva Baja for the FWI, ERC, and BI, model credibility index values ranged from 180 to 301. On the other hand, differences in the distributions of KBDI were easily detectable in the Selva Alta-Selva Baja, with sample sizes of 30 usually being sufficiently large enough to correctly reject the null hypothesis in the majority of cases. Lesser, but still notable, statistical similarities were occasionally observed in other pairs of regions for some FDIs, including the Bosque Seco-Arido, Selva Alta-Transicion, Selva Alta-Arido, Selva Baja-Transicion, Selva Baja-Sierra, and Transicion-Sierra (Fig. 3, Table 2).

Fig. 3

Power analysis. Simulated power curve for each of the 15 pairs of biogeographical units. The sample size is shown on the x-axis and the probability of correctly rejecting the null hypothesis at an 0.01 significance level. Each point of the power curve is estimated from 1000 bootstrap samples from each region’s empirical cumulative distribution function

Table 2 Estimated model credibility index by region pair and fire danger index

Comparison of MODIS hotspots to FDIs

The generalized linear models well-approximated historical hotspot activity with low levels of bias. For most regions, the absolute mean percent error between model predictions and hotspot count observations was less than 18%, although FWI-based models noticeably underestimated hotspot counts in the Bosque Seco and Sierra regions relative to other regions, and KBDI-based models underestimated hotspot counts in the Selva Baja (Table 3).

Table 3 Mean percentage error estimated via resubstitution

The composite (Eq. 4) of regressions between historical FDIs and hotspot counts (Eq. 2) and empirical distribution functions of hotspots (Eq. 3) suggest that the relative wildfire activity levels can vary drastically in regions with the identical FDI values (Fig. 4, Supplementary materials S1, S2 and S3). On a typical day from June to August 2020, the largest percentile difference would be expected between the Selva Baja and Sierra regions at a KBDI value of approximately 171.56, resulting in a difference of nearly 41 points. On a typical day from September to November 2020, the largest percentile difference would be expected between the Selva Baja and Arido regions at a FWI value of approximately 1.89, resulting in a difference of nearly 44 points. On a typical day from December to February 2020, the largest percentile difference would be expected between the Sierra and Arido regions at a KBDI value of approximately 151.07, resulting in a difference of nearly 45 points. Lastly, on a typical day from March to May 2020, the largest percentile difference would be expected between the Transcion and Arido regions at a BI value of approximately 12.72, resulting in a difference of nearly 47 points.

Fig. 4

Relative fire potential gradient. Difference in predicted hotspot percentile estimates between pairs of biogeographical units (y-axis) across a gradient of daily average FDI values (x-axis) during June–August 2020

Fire-weather relationship similarity

The Pearson correlation coefficient for the model credibility index and maximum absolute difference in h was negative for each FDI, suggesting that as the distribution of fire weather between two regions became more similar, the smaller the maximum difference in expected MODIS hotspot detect percentiles becomes. The Selva Alta and Selva Baja were usually highly climatically similar relative to the other pairs of regions and likely disproportionately effected these statistics. Moreover, there was still a large amount of variability in this overall trend. Although the Selva Alta and Selva Baja had similarly distributed FDIs, the predicted hotspot percentiles could still differ by nearly 20 points in identical fire weather conditions. Conversely, some regions were climatically dissimilar but had similar fire potential under the identical fire weather conditions. For example, despite the Bosque Seco-Selva Alta having noticeably different KBDI distributions (Fig. 2, Tables 1 and 2), the maximum difference in MODIS hotspot percentile under identical KBDI conditions was approximately 10 points (Fig. 5).

Fig. 5

Two-factor regional similarity. Scatterplot of model credibility index and maximum absolute differences in predicted hotspot count percentiles. Pearson correlation coefficients are reported in the upper left

Discussion

Summary statistics and power analysis

Each pair of daily FDI time series was generated from distinct locations with unique climates, vegetation types, topography, and anthropogenic influences. It is then unsurprising that, given enough data, differences in the statistical distributions of FDIs were detectable in the summary statistics and power analysis. In some contexts, the differences in the distributions, though statistically significant, are not of practical significance (Daniel 1977). For instance, in the Selva Alta and Selva Baja, randomly generated FDI data in the Selva Alta and Selva Baja are likely statistically indistinguishable in sample sizes that are less than a few months in duration (Fig. 3). Given that the large majority of hotspot detects occur over only a couple months’ time, it is likely that the statistics that are derived from FDI data will be similar in the Selva Alta and Selva Baja. This means that the output of simple fire danger models (Jolly et al. 2019) is likely to be similarly distributed in both regions, and the frequency with which the original SENAMHI fire danger model (Eq. 1) classifies days into low, moderate, high, very high, and extreme categories is likely to be similar in the Selva Baja and Selva Alta.

Although the differences in the distribution of FDIs were sometimes small enough to be ignored, more often it was the case that there existed detectable differences that made extrapolation of results across regions problematic. The differences in the distributions of FDIs were most noticeable between the interior tropical rainforests and the coastal desert-arid regions (Fig. 3). FDI values in the lower quartile of the Bosque Seco region could correspond to values in the upper extremes of the distribution of the Selva Alta region (Fig. 2, Table 1), and little data are required to detect differences in the pair of statistical distributions (Table 2). Consequently, the output of a simple fire danger model with standard thresholds is likely to be distributed much differently in both regions. If the fire danger model were optimized with data from a region with lower average FDI values, and predictions were then produced for a new region with higher average FDI values, then the model is likely to have an inflated false positive rate in the new region. On the other hand, if the fire danger model were instead optimized with data from a region with higher average FDI values, and predictions were then produced for a new region with lower average FDI values, then the model would likely have an inflated false negative rate in the new region.

It is important to note that for a given pair of regions, strong statistical similarity in the distributions of one FDI does not necessarily imply strong statistical similarity in another FDI. For example, although the distribution of ERC, BI, and FWI is very similar in the Selva Alta and Selva Baja (Table 2, Fig. 2), the distribution of KBDI is not similar. Conversely, in the Selva Alta and Transicion region, the distribution of KBDI is similar, whereas ERC, BI, and FWI are much less so (Table 2). These contrasts across FDIs can help identify pairs of regions that share common meteorological processes. Although to some extent all four FDIs are based on fuel moisture, they also include or omit certain variables in the calculation of the indexes. KBDI is measured solely using temperature and precipitation data (Keetch and Byram 1968), whereas ERC also includes information about solar radiation and relative humidity (Bradshaw et al. 1984; Deeming et al. 1977). Similarly, BI and FWI include components that increase the index in windy conditions, which are components not included in KBDI and ERC (Littell et al. 2016; Tian et al. 2011; Bradshaw et al. 1984; Deeming et al. 1977). We might then expect that pairs of regions (e.g., Selva Alta and Selva Baja) that have similar fuel moisture patterns but dissimilar wind patterns to have fairly low model credibility index values for BI and FWI, but higher model credibility index values for KBDI and ERC. We might also expect pairs of regions (e.g., Selva Alta-Transicion) that have similarly distributed temperature and precipitation but dissimilar distributions of other meteorological variables might have fairly high model credibility for KBDI but lower model credibility for FWI, BI, and ERC.

Comparison of MODIS hotspots to FDIs

Hitherto in our discussion of the results, we ignored the possibility that fire potential may vary between regions with similarly distributed FDIs. In general, we found that the more climatically similar two regions are, the more similar relative fire potential will be under identical meteorological conditions (Fig. 5). However, this does not mean that one can simply just assume that because a FDI-based model has high performance in one location that it would always also do well in another climatically similar one. Indeed, even though the Selva Alta and Selva Baja have similar statistical distributions of BI, FWI, and ERC, MODIS hotspot count percentiles are still expected to differ by as much as 18–27 percentage points under identical fire weather conditions (Fig. 5). Moreover, as was the case in the Bosque Seco-Selva Alta regions, approximately similar estimates of fire potential can also be produced from FDI-based models that were developed in climatically dissimilar places (Fig. 5). One potential explanation for these exceptions is that the effects of FDIs sometimes play a subordinate role in fire potential in comparison to other non-meteorological factors. If we hypothetically assumed that fire potential was largely independent of fire weather, as might be expected in a region with a great deal of anthropogenic influence (Syphard et al. 2017), then it would be possible for the MODIS hotspot detection percentiles to be close in both regions for any arbitrary shared FDI value. Given that it is the dry coastal regions—the Bosque Seco and Arido regions—with the largest anthropogenic influence, this explanation is at least somewhat plausible, but there are a suite of other competing non-meteorological factors that could also mediate fire potential here as well. It is worth noting that in some contexts, this assumption of geographic portability of fire-weather relationships could be adequate, even in climatically dissimilar regions. For instance, if FDI values are extremely high/low, then the conclusion that fire potential is high/low is still likely to be reasonable regardless of which region’s perspective you are considering (Fig. 4). We observed that it was mostly at intermediate FDI values, where there was a greater risk associated with extrapolating fire potential relationships observed in one location to another.

Applying FDI-based fire potential models to regions that they were not developed for may appear as a contrived modeling exercise, but our analysis illustrates the risks associated with assuming that the relationship between an FDI and wildfire activity is spatially uniform. This assumption may arise explicitly when fire potential models that work well in one location are assumed to also work well in another. Doubtless, the fire classification model in Peru is informative of fire danger in at least some locations, but it is also likely to produce misleading risk assessments in others. Beyond this specific case, we can also see that the assumption of geographic portability can also arise implicitly if spatial comparisons of fire potential are derived using FDI values only (Vitolo et al. 2019) or if fire potential is interpreted at a spatial scale that is different from that of the FDI products (Regan et al. 2002). In each of these cases, it is assumed that the conditional probability distribution of a wildfire parameter is the same across regions. However, because the fire potential can noticeably differ even in regions that are climatically similar, we can see that this assumption is problematic. Given that climatic similarity is no guarantee that fire potential will respond the same to identical fire weather conditions, we can see that although normalizing FDIs is a commonly recommended method for interpreting fire potential from FDIs across regions (Hall et al. 2003; Heinsch et al. 2009), bias can nevertheless remain. Ideally then, fire potential predictions should be locally calibrated using the statistical distributions of FDIs and a relevant wildfire parameter (e.g., MODIS hotspot counts) so that fire potential can be explicitly described in terms of a relevant wildfire parameter. However, climatic similarity (model credibility index) was at least weakly correlated similar fire potential responses (h), so the chances of producing grossly misleading fire potential forecasts from models produced in other locations with similar climates are likely small. Still, given the occasional exceptions to this trend, fire potential forecasts should, at a minimum, avoid drawing overconfident conclusions solely on the basis of FDIs and consider presenting important contextual information alongside FDI estimates.

Future research

As mentioned in the previous subsection, non-meteorological factors such as vegetation (Syphard et al. 2018; Meyn et al. 2007), anthropogenic effects (Syphard et al. 2007, 2017; Rodrigues et al. 2018; Monjarás-Vega et al. 2020), topography (Parisien et al. 2012; Silva et al. 2020), and temporal autocorrelation of fire activity (Vega-Nieva et al. 2018) are also important for gauging fire potential. Given the potential for these factors to be of greater importance than meteorological factors (Syphard et al. 2017), future work should seek to include these factors into fire risk predictions and assess their regional portability. For instance, anthropogenic influences have been observed to reduce the importance of climate on wildfire activity in Mediterranean California (Syphard et al. 2017), and it is worth testing if this relationship (1) is observed in other locations and (2) can explain the apparent robust relationship between FDIs and fire potential in climatically dissimilar locations like the Bosque Seco and Selva Alta. There are also a large set of meteorological variables, beyond those explored in this study, that have the potential to influence fire (Zargar et al. 2011; Littell et al. 2016; Vega-Nieva et al. 2018; Sismanoglu and Setzer 2005; Vega-Nieva et al. 2019). Identifying regional differences in the distribution of these variables, as well as in their presumed effect on fire activity, can further advance our understanding on when and where relationships between weather and fire can be reliably extrapolated. Repeating this analysis using a large suite of alternative FDIs may identify some that are more regionally robust than those examined in this study, so that accurate predictions of fire potential can be produced with little need for regional calibration. For at least two reasons, it is also important to see how the substitution of other wildfire parameters can influence the results of this analysis. Firstly, patterns and trends observed in one wildfire parameter may not be observed in others (Doerr and Santín 2016), and secondly, the practical relevance of a given wildfire parameter can vary depending on the specific decision-maker or constituency being considered (Podschwit and Cullen 2020). As an example, it is possible that daily MODIS hotspot counts are a relevant proxy for assessing smoke impacts of fire, but are not also a relevant proxy for firefighting effectiveness. The latter application may instead require the consideration of another wildfire parameter that correlates with FDIs in a way that drastically differs from daily MODIS hotspot counts. Simple rescalings—for example transforming MODIS hotspot counts into per-unit area rates—might also be used to produce alternative wildfire parameters that can be used to understand the sensitivity of FDIs to substitutions of fire potential proxies. Many defensible spatial aggregations different from those presented in this study (e.g., ecoregions) could have been proposed, which may lead to differing results than those presented in this study (Dark and Bram 2007), and the sensitivity of these results to varying spatial aggregations is another area of research that should be considered.

Conclusions

The extrapolation of relationships between fire danger indexes and fire potential observed in one location into others is an often necessary assumption when modeling fire risk. This assumption of geographic portability of fire-weather relationships is observed in the existing fire danger classification model in Peru, which uses the same critical values to classify low, moderate, high, very high, and extreme fire danger across biogeographical regions. However, this analysis has shown that the assumption of geographic portability of fire-weather relationships is often unrealistic for at least two reasons. Firstly, spatial variability in the distribution of fire weather can make it difficult to establish standard rules classifying fire danger. Secondly, the response of wildfire parameters to fire weather can vary spatially even in climatically similar regions. In Peru specifically, we observed large differences in the statistical distributions of the four FDIs, with the largest differences being between regions in the tropical rainforests and the coastal desert-arid subtropical regions. Moreover, the counterfactual hotspot analysis demonstrated that under identical fire weather conditions the predicted MODIS hotspot detect percentiles can substantially differ depending on which part of the country one is located. Finally, while these methods were demonstrated specifically for Peru, the methods can be applied to any area of the world to explore the similarities and differences in fire weather between regions, and the results can inform the development of more robust fire prediction systems.