Assessing the role of short-term weather forecasts in fire manager tactical decision-making: a choice experiment

Abstract

Background

Weather plays an integral role in fire management due to the direct and indirect effects it has on fire behavior. However, fire managers may not use all information available to them during the decision-making process, instead utilizing mental shortcuts that can bias decision-making. Thus, it is important to evaluate if (and how) fire managers use information like weather forecasts when making tactical decisions. We explore USDA Forest Service fire manager confidence in relative humidity, precipitation, and wind models. We then use a choice experiment where key weather attributes were varied to explore how sensitive fire managers were to changes in specific weather variables when choosing to directly or indirectly attack a fire that is transitioning to extended attack.

Results

Respondents were less confident in the accuracy of wind and precipitation forecasts than relative humidity or weather forecasts more generally. The influence of weather information on the decision depended on the framing used in the choice experiment; specifically, whether respondents were told the initial strategy had been to directly or indirectly attack the fire. Across conditions, fire managers generally preferred to indirectly attack the fire. Decisions about the tactics to apply going forward were more sensitive to time in season when the fire was occurring and wind and precipitation forecasts than to other attributes.

Conclusions

The results have implications for the design of decision support tools developed to support fire management. Results suggest how fire managers’ use of fire weather information to evaluate forecast conditions and adjust future management decisions may vary depending on the management decision already in place. If fire weather-based decision support tools are to support the use of the best available information to make fire management decisions, careful attention may be needed to debias any effect of prior decisions. For example, decision support tools may encourage users to “consider the opposite,” i.e., consider if they would react differently if different initial decision with similar conditions were in place. The results also highlight the potential importance of either improving wind and precipitation forecast models or improving confidence in existing models.

Resumen

Antecedentes

El tiempo meteorológico juega un rol integral en el manejo del fuego debido a los efectos directos e indirectos que tiene en el comportamiento del fuego. Desde luego, los responsables del combate y manejo del fuego pueden no usar toda la información de que disponen durante el proceso de toma de decisiones operativas, utilizando en cambio atajos mentales que pueden sesgar los procesos de decisión. Por lo tanto, es importante evaluar si (y cómo), los responsables del manejo del fuego usan la información, como por ejemplo los pronósticos meteorológicos, cuando toman decisiones tácticas. Exploramos la confianza de los responsables de manejo del fuego del Servicio Forestal de Departamento de Agricultura de los Estados Unidos (USDA) sobre los datos de humedad, precipitación y modelos de viento. Usamos luego una alternativa de experimento donde los atributos clave fueron variando para explorar cuan sensibles se mostraban los responsables del combate o manejo del fuego ante cambios en variables meteorológicas específicas al momento de elegir atacar directa o indirectamente un incendio, cuando este estaba en un estado transicional hacia un ataque extendido.

Resultados

Los que respondieron estaban menos confiados en la exactitud de los pronósticos de viento y precipitación que en la humedad o en los pronósticos de manera general. La influencia de la información meteorológica en la decisión dependió del encuadre usado en el experimento; de manera específica, si los que respondieron habían sido alertados sobre si la estrategia inicial había sido atacar el incendio de manera directa o indirecta. En todas las condiciones, los responsables del manejo del fuego generalmente prefirieron atacar indirectamente el fuego. Las decisiones sobre las tácticas a aplicar hacia adelante fueron más sensibles al período de la estación en la cual el fuego estaba ocurriendo y también al pronóstico del viento y precipitación más que en otros atributos.

Conclusiones

Los resultados tienen implicancias para el diseño de herramientas desarrolladas para sustentar la toma de decisiones en el manejo del fuego. Los resultados sugieren cómo los responsables del manejo del fuego usan la información meteorológica para evaluar las condiciones pronosticadas y ajustar las futuras decisiones de manejo que pueden depender de decisiones ya tomadas. Si las herramientas de decisión basadas en pronósticos meteorológicos indican apoyar el uso de la mejor información disponible para decidir acciones de manejo del fuego, una cuidadosa atención debe ser necesaria para corregir cualquier efecto de decisiones anteriores. Por ejemplo, las herramientas de decisión podrían recomendar a los usuarios “considerar lo opuesto” (i.e. considerar si ellos podrían reaccionar de manera diferente si las decisiones iniciales con las mismas condiciones prevalecieran. Los resultados también subrayan la importancia potencial de, o mejorar los modelos de pronósticos de viento y precipitación, o mejorar la confianza en los modelos existentes.

Abbreviations

ERC: Energy release component

FMO: Fire management officer

HRRR: High Resolution Rapid Refresh

IMET: Incident meteorologist

IMT: Incident management team

NDFD: National Digital Forecast Database

POD: Potential operational delineation

USDA: United States Department of Agriculture

WFDSS: Wildland Fire Decision Support System

Background

Purpose

Reintroducing fire to the landscape and transforming the fire management paradigm away from aggressive and costly suppression towards thoughtful risk management requires shifts in practice at both tactical and strategic levels of fire management from initial attack in the earliest stages of a fire to mop-up activities as fires end. As fires progress, each decision will shape and constrain opportunities for future decisions. For example, nearly 88% of all fires in the USA from 1992 to 2018 were contained during initial attack and kept small (<10 acres) (Short 2021), but defaulting to containment and suppression goals during initial attack shapes the decision space of future fire managers by contributing to landscape conditions that increase the long-term risk of catastrophic, uncontrollable wildfires (Calkin et al. 2015).

Fostering risk management also requires consideration of how decisions are made under risk and uncertainty. Decision-making with risk involves choices where the exact outcome is unknown, but the possible outcomes that could occur, and their probability of occurring, are known. In comparison, decision-making under uncertainty involves making decisions when possible outcomes are not known or their probability of occurring is not known. Fire managers work in uncertain and time-pressured environments and may not have the time or ability to deliberatively consider all information available to them. When people have insufficient time or resources to process information, they often rely on heuristics. Heuristics are simple rules and guidelines, or mental shortcuts meant to simplify and speed up decision-making by emphasizing some information while ignoring other information (Gigerenzer and Gaissmaier 2011; Simon 1956). Heuristics can be adaptive as they can enable decision-makers to make acceptably accurate decisions more quickly in some cases (Kahneman and Klein 2009; Gigerenzer 2008). However, heuristics can also systematically lead people astray from relevant information that could support more effective decision-making. Thus, heuristics can bias decision-making. There are a number of known biases that are regularly evident in decision-making. Among these, some common biases relevant for fire management include the availability bias, where one over-estimates the probability of events they have recently experienced occurring again in the future, and the phenomenon of anchoring-and-adjustment, where decision-makers anchor to initial information they receive and insufficiently adjust their beliefs in response to new information (Tversky and Kahneman 1974; Maguire and Albright 2005).

Even experts with years of training and experience in a particular decision context are prone to cognitive biases. A growing body of research has specifically examined the decisions of fire managers. These studies find that fire managers are subject to many of the same biases as others. In particular, research has found that fire managers appear risk-seeking (i.e., more willing to accept risky alternatives than alternatives with a fixed outcome) when choices are framed as the opportunity to minimize losses, but risk averse when the same outcomes are framed as maximizing gains (Wilson et al. 2011); this apparent contradiction in decisions based on how they are described was first recognized by Kahneman and Tversky and Kahneman (1974). Fire managers are also influenced by framing with regard to personnel safety; when information is framed in an affectively rich way, fire managers are more sensitive to personnel exposure than when information is presented analytically (Hand et al. 2015). Moreover, fire managers work in conditions of risk and uncertainty that may lead them to exhibit myopia, or excessive discounting of future outcomes in favor of short-term gains (Maguire and Albright 2005). Finally, fire managers display non-linear probability weighting; managers are more sensitive to changes in the probability of success over moderate probabilities than low or high probabilities (Wibbenmeyer et al. 2012). While several decision support tools have been developed to support thoughtful and deliberative decision-making (e.g., Calkin et al. 2011), the mere presence of additional information may not lead to more defensible or risk-informed decisions (Drews et al. 2015; Rapp et al. 2020; Noble and Paveglio 2020). The purpose of this paper is to contribute to this growing body of literature by exploring how fire managers use weather information in tactical decision-making. We focus on a key decision point for large fire management, the transition from initial to extended attack. Specifically, we examine how fire managers use weather forecasts when deciding whether to directly or indirectly attack a fire 48 h into an event. We chose this decision context because the transition from initial to extended attack marks an increase in complexity and is frequently accompanied by new personnel arriving on the incident.

Literature review: tactical decision-making and fire weather

On large fires that escape initial attack, the Incident Command System provides the framework for who has strategic and tactical decision-making authority. Under this system, line officers (e.g., local agency personnel with decision-making authority) work with incident commanders to establish the strategic objectives for the fire. Strategic objectives pertain to the overarching strategy of the fire, such as whether it will be managed for suppression, resource benefit, or both and where (Taber et al. 2013). In comparison, tactical decisions are made by the incident management team (IMT) and pertain to on-the-ground decisions about how the fire will be managed to achieve the strategic objectives. For example, the line officer might identify protecting a high-value watershed as a strategic objective, leaving the IMT to make the specific tactical decisions about what resources to deploy where to protect the watershed. While strategic objectives fundamentally inform the decision space of the IMT, the tactical decisions of the IMT will also shape the final fire outcomes. For example, on fires strategically managed for either suppression or fire use, the IMT may decide to attack the fire directly (constructing a fire line along the active fire perimeter) or indirectly (constructing the line away from the perimeter and possibly conducting burnout operations).

Tactical decisions about how a fire is managed are not well understood in terms of both how they are made and how they influence fire outcomes. Specifically, while fire size, weather, and landscape characteristics are related to the ultimate financial cost of a wildfire, it is unclear how they influence tactical decisions like resource ordering and deployment (Hand et al. 2017). Furthermore, it is difficult to quantify and evaluate how effective certain resources are at containing and controlling fires (Thompson et al. 2017b). Without understanding resource effectiveness, it is difficult to evaluate, for example, whether dozers or hand crews would be more efficient for a given area (Plucinski 2019). Given these conditions, it is difficult to calculate efficient tactical alternatives or provide tools to aid managers as they select between alternative tactical actions (Dunn et al. 2017). Several decision support tools have been developed to aid strategic decision-making (Calkin et al. 2011; Thompson et al. 2017a; Thompson and Calkin 2011), including the Wildland Fire Decision Support System (WFDSS) (NIFC 2019) and more recently potential wildland fire operational delineations (PODs) (Thompson et al. 2016). These tools can provide tactically relevant information, but have some limitations. For example, PODs do not provide dynamic information over the course of large fire events (O’Connor, Thompson, and Rodríguez y Silva 2016). Similarly, WFDSS cannot help fire managers evaluate and compare the outcomes of different alternative tactical actions. However, the fire behavior and weather information it provides can inform the feasibility and likelihood of operational tactic success (Rapp et al. 2020).

Indeed, weather information is critical to the likely success of wildland fire operations because weather, along with fuels and terrain, is one of the primary drivers of fire behavior (Countryman 1972). Weather information is so important to wildland fire fighting operations, and it is listed as the first of 10 Standard Fire Orders (systematically organized rules applied to all fires) that are taught to every firefighter on their first day of training: “Keep informed on fire weather conditions and forecasts.” Fire behavior characteristics, such as flame length, fire intensity, and spread rates, determine how safely firefighters can directly engage a wildfire and these factors may be used heuristically to guide choices of fire suppression tactics such as direct or indirect attack (Andrews et al. 2011). Weather conditions that promote rapid spread and high intensity can prevent initial firefighting resources from containing a new fire (Arienti et al. 2006), leading to a fire that spreads out of control for extended periods of time. Low relative humidity, strong near-surface winds, an unstable atmosphere, and severe drought can promote extreme fire behavior and make wildfires difficult to control (Werth et al. 2011; Tedim et al. 2018). These extreme fires often burn more area or cost more to suppress than fires that do not occur during extreme fire weather (Fernandes et al. 2016; Finney et al. 2009; Hand et al. 2017).

Firefighters and fire managers are accustomed to assimilating fire weather information in a range of forms and from a variety of sources. This information can impact decisions made at a variety of temporal scales and across a range of administrative levels. For example, weather information, often transformed into fire danger indices such as the Energy Release Component (Jolly et al. 2019), can be used to support pre-incident planning such as seasonal staffing, open burning restrictions, and public awareness activities aimed at preventing human-caused wildfires. This weather-derived information is also used to inform “run cards” that determine the numbers and types of firefighting resources dispatched during initial attack when fires do occur (Schlobohm and Brain 2002). During initial attack, fire managers monitor local weather conditions and request spot weather forecasts specific to their location (e.g., Wall et al. 2017) to assess the potential for rapid changes in fire behavior that could affect their safety or that could impact local communities or valued resources. If fires are not contained through local management efforts and IMTs are assigned, local weather forecasts are often produced on-site by an Incident Meteorologist (IMET) and forecast information is provided daily as part of the Incident Action Plan. For large fires that escape initial attack, fire managers are required to create a Published Decision through the Wildland Fire Decision Support System, which uses weather forecasts to inform fire spread and behavior projections produced by the Wildland Fire Decision Support System (Noonan-Wright et al. 2011). A Relative Risk Assessment (RRA) is a component of these published decisions, and these are developed collaboratively using observations, models, and data and are intended to capture the risk/reward basis of each decision and they are updated throughout the duration of the event as conditions change (Noonan-Wright and Seielstad 2021). The resulting information is used to minimize firefighter risk, maximize likelihood of containment success, and protect communities and infrastructure. As these examples illustrate, the use of weather information is ingrained into the wildland fire system. In fact, evidence suggests IMTs order resources less based on previous fire activity and more based on weather forecasts and their projected impact on fire behavior (Bayham et al. 2020). Given the importance of weather information to fire management decisions, it is critical to understand how weather data are used to ensure the best possible information is available for time-sensitive, tactical decision-making across a range of administrative levels and temporal scales.

While the provision of weather information is critical, the presence or availability of information does not guarantee its use by the IMT (Rapp et al. 2020; Drews et al. 2015; McLennan et al. 2006; Noble and Paveglio 2020). Existing decision support tools are intended to help fire managers engage in deliberative, stepwise decision-making (Zimmerman 2012); however, in practice, fire managers may still use the information provided by these tools, including weather forecasts, in a heuristic way. Specifically, fire managers may use weather information to recognize patterns and assess the extent to which a situation is similar to their previous experience (Drews et al. 2015; Lipshitz et al. 2002; Klein 2008). Furthermore, fire managers are susceptible to cognitive biases like framing effects, where the presentation of information influences preferences (Wilson et al. 2011; Hand et al. 2015). IMT personnel may feel pressure from the line officer, agency, or the public in general to manage fires in ways they may not believe is ideal. For example, spending more money and using tactics they believe are ineffective but the public wants to see, such as the ineffective use of aviation resources during periods of extreme weather (Canton-Thompson et al. 2008; Calkin et al. 2012). These challenges with using weather information are not unexpected or an inherently negative evaluation of fire managers, who operate in an environment categorized by considerable uncertainty, high risk, and multiple constraints (Kahneman and Klein 2009; Thompson et al. 2017b). However, it does ultimately suggest weather information may not be interpreted consistently across decision-makers or contexts.

Because weather is an important source of dynamic information available to IMTs during fire events, this study focuses on how fire weather informs fire manager tactical decision-making. Questions consider which pieces of weather information are used by fire managers and how they are used. We ask these questions specifically as they relate to the tactical decision to attack a fire directly or indirectly in the transition from initial to extended attack.

Methods

Subjects

The data presented here come from a web-based survey sent to federal fire managers working for the United States Department of Agriculture (USDA) Forest Service. For this survey, we specifically targeted fire management officers (FMOs) (i.e., assistant fire management officers, forest fire management officers, etc.). To be an FMO, individuals need several years of operational firefighting experience and hold qualifications to serve as division supervisors, operations section chiefs, or incident commanders on IMTs. We developed our initial list of FMOs from internal email lists (n = 239) and augmented and corrected this list by contacting individual Forest Supervisors to check that our list was up to date for their forests (n = 708). After removing invalid emails, we had a final list of 669 potential respondents. Surveys were conducted over Sawtooth, a web-based survey and choice experiment platform. Two hundred forty-three respondents or 36% responded. After removing respondents who did not complete the choice experiment, the final sample included 182 respondents for an adjusted response rate of 27%. This response rate is in line with previous online surveys of federal fire managers, with response rates varying from 25 to 50% (e.g., Hand et al. 2015; Wibbenmeyer et al. 2012; Wilson et al. 2011).

Descriptive measures

The survey included questions about how long participants had worked in fire and in their current job as well as their gender, education, and what role they served as most frequently on IMTs. Additionally, to control for other potential influences on decision-making beyond fire weather information, we asked respondents whether they perceive direct or indirect attack as riskier for firefighter safety using a bipolar 5-point scale ranging from “direct attack is much riskier” to “indirect attack is much riskier” where the middle point equals indifference. Additionally, we measured respondent confidence in weather models, specifically how frequently they believe wind, precipitation, relative humidity (RH), and general weather forecasts are accurate on a 4-point ordinal scale. To compare confidence across weather models, we conducted post hoc pairwise t-tests with Bonferroni adjustments for multiple comparisons.

Choice experiment rationale and description

The survey also included an embedded choice experiment. Choice experiments, or conjoint analyses, are frequently used to elicit consumer preferences. In natural resource management literature, these tools are often used to measure the willingness to pay for ecosystem services. Discrete choice experiments assume that people choosing between alternatives maximize their personal utility. In the context of fire management, maximizing personal utility is considered to align with maximizing optimal fire management outcomes (Calkin et al. 2012). Choice experiments highlight which attributes are the most important factors influencing decision-maker choices. Thus, choice experiments allow us to examine how different levels of a given attribute, such as the varying probability of wetting rain, influence which tactics fire managers believe are best for a fire. Choice experiments also allow comparisons of the relative importance of different attributes, for example, whether fire managers are more sensitive to changes in precipitation or changes in wind speed when making tactical decisions.

Before beginning the choice experiment, we provided all respondents with the same description of an ongoing wildfire event (see Fig. 1 for a full description). We designed this introduction to be ambiguous such that it was not immediately clear whether direct or indirect attack was safer or more likely to succeed, and both direct and indirect attacks were politically feasible and acceptable based on existing policy. There are substantial challenges in designing a wildfire scenario that is realistic in light of the real-world complexity associated with such decisions. Some simplification is required given the limitations posed by experimental research; however, we sought to develop a context for their later decision that included or controlled for the primary variables that influence decisions about direct versus indirect attack so we could assess the unique effect of weather information. To develop the background context or introduction as well as the critical attributes and levels to include, we sought feedback from several USDA Forest Service scientists with extensive experience working on these issues. We also conducted a focus group with FMOs from one USDA Forest Service region where scenarios were reviewed, discussed, and subsequently adjusted.

Fig. 1

Choice experiment scenario introduction

After reading the introduction, respondents were asked on a scale of 1 (strongly prefer direct attack) to 5 (strongly prefer indirect attack) to what extent they believed direct or indirect attack was preferable given the information provided. Respondents were then randomly assigned to one of two conditions (Table 1); in the first condition (n = 103), respondents were told the initial attack team had decided to indirectly attack the fire in the first 48 h. Now that the respondent was arriving on the scene, they would choose whether to stick with the indirect attack or switch from indirect to direct attack. In the second condition (n = 79), respondents were told the initial team had decided to directly attack the fire in the first 48 h, and now that the respondent was arriving on the scene, they would choose whether to stick with the direct attack or switch to the indirect attack. In both conditions, we asked respondents to what extent they agreed with the initial attack team’s decision on a scale of 1 (strongly disagree) to 5 (strongly disagree).

Table 1 Description of each experimental condition

Respondents were then presented with nine choice sets or nine opportunities to select a scenario where they would switch the strategy or indicate a preference for remaining with the current strategy. Each choice set contained three scenarios that varied across five attributes: energy release component (ERC), time in season, forecasted precipitation, forecasted relative humidity, and forecasted wind. The three weather attributes (precipitation, RH, and wind) were used to generate a map of potential fire spread for each scenario and this map was also provided to respondents. In each of the choice sets, respondents were instructed to examine the three scenarios and select the one for which they would most support switching the strategy, or to indicate they would continue with the initial team’s strategy (labeled as “NONE” as they would not switch the strategy in any of the scenarios). All attributes and possible levels are summarized in Table 2. An example of a choice set respondents could see is provided in Fig. 2. After the choice set, respondents were provided an open-ended box where they could describe what factors were the most important in influencing their decision. We coded open-ended responses and calculated how frequently each factor was mentioned across the two conditions.

Table 2 Summary of attributes and levels in the choice experiment

Fig. 2

Example choice set

Choice experiment analysis

We used Sawtooth Software SSI Web to conduct our online discrete choice experiment and determine both the necessary number of choice sets given the number of attributes and levels, as well as how the attribute levels would be assigned to each scenario within the choice set. Many choice experiments use a fixed orthogonal design to limit the total number of choice sets respondents must see to calculate unbiased coefficients. We used Sawtooth’s balanced overlap method of randomized design. This method allows some pairs of attributes to co-occur, reducing the number of choice sets a respondent must complete without significant loss in reliability (Sawtooth Software 2017). In balanced overlap designs, respondents see each level of any given attribute an approximately equal number of times. Each level of each attribute may not appear an equal number of times, especially if attributes vary by how many levels they have (e.g., our precipitation attribute has two levels and our relative humidity attribute has three levels).

Choice experiments use probabilistic modeling to separate the overall utility of an alternative U_j into the observable factors V_j of a given alternative A and the unobservable factors ε_j. Here, we are calculating the utility of each specific scenario in the choice set as the alternative. This random utility model is represented by the following equation.

$$ {U}_j={V}_j(A)+{\varepsilon}_j $$

We estimated our random utility model using hierarchical Bayesian (HB) analysis. HB analysis has two levels. The upper level assumes individual vectors of part-worths are described by a multivariate normal distribution. The lower level assumes that the probability that an individual selects a given scenario can be described by a logit model, where the utility of each scenario is the sum of the part-worths of its attribute levels (Johnson 2000). HB iteratively calculates individual part-worths and average utilities for the sample to examine how respondents differ from sample averages. After thousands of calculations, the solution converges. Calculations after convergence are averaged to get part-worth utility coefficients. We report on two measures, the average part-worth utilities for each level of each attribute and the overall importance of each attribute. For each attribute, the sum of the part-worths of each level is zero. Consequently, negative part-worths do not necessarily reflect negative utility, but rather a smaller utility than positive part-worths. Overall importance is a study-specific measure of how important a given attribute is compared to other attributes in the choice experiment. It is influenced by the range of the part-worth utilities of the attribute; the larger the range of part-worths for a given attribute, the more important it is. To calculate importance, the relative range of part-worth utility for each attribute for each respondent is calculated as a percent of the total range across attributes and then averaged across respondents (Orme 2010). Thus, the importance measures of all attributes sum to 100%, and measures of importance are ratio-scaled, which is to say an attribute with an importance of 50% is twice as important as an attribute with an importance of 25%. Therefore, in interpreting the results, the higher the utility score of a particular scenario, the more likely respondents are to switch from the default or initial attack decision.

Results

Respondent demographics

Respondents were very experienced in fire management, on average serving 8 years in their current position and 24 years in fire management overall. Respondents’ roles varied: 32% indicated they most frequently served as division supervisors, 26% as incident commanders (types 1–3), 18% as operations section chiefs, and 24% in other roles, e.g., technical specialists, safety officers, and task force leaders. Most of our respondents (88%) identified as male and a majority (69%) had at least a bachelor’s degree.

Risk perception and forecast reliability

The majority of respondents (56%) believed direct and indirect attacks were equally risky for firefighter safety (Table 3). Respondents tended to have moderate to high confidence in all weather forecasts, indicating that forecasts were reliable 51–75% of the time (Table 3). However, we did find that average confidence differed significantly across models (df = 3, F = 16.003, p < .001). Pairwise t-tests indicate that respondents have lower confidence in precipitation and wind forecasts than relative humidity or weather forecasts in general (p < .05) (Table 4).

Table 3 Descriptive statistics of agency pressure, risk perception, and forecast reliability

Table 4 Pairwise comparisons of average reliability of weather forecasts with Bonferroni correction

Choice experiment introduction and initial attack decision

While we intended the initial choice experiment introduction to be ambiguous such that respondents would not automatically prefer direct or indirect attack, two-thirds (68%) of respondents believed the indirect attack was at least somewhat more preferable than direct attack after reading the introduction (x̄ = 3.73). Later, when judging how much they agreed with the initial team’s decision to either directly or indirectly attack the fire, respondents were more supportive of the initial team’s decision to indirectly attack (t = 4.8, p < .001). Specifically, respondents who were told the initial team indirectly attacked the fire tended to agree with the initial team’s decision (x̄ = 4.03), while respondents who were told the initial team directly attacked the fire neither agreed nor disagreed with the initial team’s decision (x̄ = 3.20).

Choice experiment outcomes

Condition 1: Switching from indirect to direct attack

In this first condition, respondents chose whether to switch from indirect to direct attack. Seasonality was the most important attribute influencing this decision (average importance score = 37.40; see Table 5), indicated by the large range in the utility scores with being early in the season having the highest part-worth utility and being late in the season having the lowest part-worth utility of all attributes and levels. The highest part-worth utility for early in the season indicates that respondents have a stronger preference to switch to direct attack when it is early versus middle or late in the season (Table 6). Wind was the second most important attribute (average importance score = 19.31 indicating it is approximately half as important as seasonality), with respondents preferring to switch to direct attack when the forecasted wind was low (described as slightly windy) compared to when the forecasted wind was high (described as very windy). Precipitation was the third most important attribute (average importance = 18.74). Respondents have a stronger preference for switching to direct attack when wetting rain is forecasted compared to no rain in the forecast. ERC was the fourth most important attribute (average importance = 16.15). Interestingly, respondents did not have clear linear preferences with regard to ERC. Specifically, the highest value ERC, ERC trending upwards toward 90%, had the highest part-worth utility, but the medium value for ERC, ERC stable around 80%, had the lowest part-worth utility. Relative humidity was the least important attribute (average importance = 8.41), with respondents preferring to switch to direct attack when RH was high (described as humid) compared to when forecasted RH was low (described as dry).

Table 5 Importance of each attribute across choice experiments

Table 6 Utility of each attribute across choice experiments

These results indicate that the ideal conditions for switching to direct attack would be a fire with wetting rain, where conditions were humid and slightly windy and early in the season and with ERC trending towards 90% (sum of all part-worth utilities for that scenario = 180.04). This combination of weather factors suggests moderate fire behavior early in the fire season. Additionally, respondents indicated that they preferred to indirectly attack all three fires in 48% of all choice sets. The NONE scenario (meaning respondents would not switch to direct attack in any of the described cases) had a relatively high utility score (average part-worth utility = 154.9) and only some combinations of attributes led to scenarios that were viewed as preferable to indirect attack. For example, the scenario with the ideal conditions described above had a greater utility than the NONE scenario. However, if the same scenario occurred in the middle of the season or late in the season, it would not be preferable to the NONE scenario. Put another way, respondents were only willing to switch to direct attack for some combinations of attributes early in the season; otherwise, they preferred to continue with the indirect attack.

Condition 2: Switching from direct to indirect attack

Unlike the first condition, precipitation was the most important attribute (average importance = 31.46; see Table 5) when deciding whether to switch from direct to indirect attack. Specifically, a weather forecast with no chance of wetting rain had the highest utility while a forecast of wetting rain had the lowest utility, indicating a preference to switch to indirect attack when there was no rain in the forecast. Seasonality was the second most important attribute (average importance = 23.15), where consistent with responses in the other condition, respondents preferred to switch to indirect attack later in the season. ERC was the third most important attribute (average importance = 21.97), primarily driven by the relatively low utility of ERC trending towards 60% compared to ERC stable at 80% or trending towards 90%. Wind was the fourth most important attribute (average importance = 12.45). For wind, respondents did not have clear linear preferences. Specifically, the highest value for forecasted wind (i.e., very windy) had the highest part-worth utility, but the medium value for forecasted wind (i.e., windy) had the lowest part-worth utility. This does not indicate a clear trend for when respondents preferred to switch to indirect attack. Relative humidity was the least important attribute (average importance = 10.97). Respondents preferred to switch to indirect attack when forecasted RH was low (i.e., dry) compared to when forecasted RH was high (i.e., humid).

These results indicate that the ideal conditions to switch to indirect attack would be a fire with no rain forecasted, low humidity, and high wind, late in the season with ERC trending towards 90% (sum of all part-worth utilities for that scenario = 147.52). This combination of weather and fuel factors suggests extreme fire behavior, with a higher chance of a season-ending event on the horizon. In 92% of the choice sets, respondents chose to switch to indirect attack for at least one of the described scenarios. The NONE scenario had a low utility (part-worth utility = −68.69); thus, respondents preferred to stay with direct attack for only a few limited combinations of attributes. For example, respondents preferred the NONE option or preferred to stick with the direct attack when there was forecasted rain and conditions were described as humid and windy and early in the season with the lowest ERC (sum of all part-worth utilities = −156.24 or the lowest-utility scenario). However, if the same scenario were presented but without rain forecasted, we would predict respondents would prefer to switch to indirect attack. Put another way, respondents were only willing to continue with direct attack for some scenarios where wetting rain was forecasted; otherwise, they preferred to switch to indirect attack.

For some scenarios, respondents preferred the NONE alternative in both conditions, whether the decision was to switch to direct or indirect attack (Table 7). For example, for some scenarios where wetting rain was forecasted and it was not early in the season, respondents in both conditions preferred to continue with the initial strategy. It is unclear why fire managers preferred the default in these cases. For example, it may be that the relative gain in utility was not believed to be worth the cost of changing tactics, or it may be that the fire managers did not have a preferred tactic in those circumstances and defaulted to the previous team’s tactics.

Table 7 Example choice sets

Open-ended comments

Ninety-five respondents who evaluated whether to switch to direct attack and 69 respondents who evaluated whether to switch to indirect attack provided open-ended comments, ranging from a couple of words to paragraphs on what factors were most important to them in their decision. Regardless of condition, the most frequently mentioned factor was seasonality. Respondents then mentioned wind and ERC across conditions at similar rates, but consistent with the choice experiment, more respondents mentioned rain when switching to indirect attack versus when switching to direct attack (Table 8).

Table 8 Summary of most frequently mentioned codes for respondents’ open-ended answers on the most important decision factors in the choice experiment

Discussion

Our results have several implications for how weather information and forecast models can be communicated more effectively to support tactical decision-making. We consider two implications in greater detail. First, our results highlight the importance of considering how information is used in light of the potentially heuristic decision strategies of fire managers. Tools will be more effective when designed with the decision strategies of fire managers in mind, either by supporting heuristic-based decision-making or by debiasing and encouraging more deliberative decision-making. Second, our results point to possible areas of improvement for weather forecast models that might improve confidence. Wind and precipitation forecasts merit particular attention, either by improving model accuracy directly, improving confidence in existing models, or both.

Supporting heuristic versus deliberative decision-making

Weather information can be an important determinant of tactical decision-making and success in wildfire management (Rapp et al. 2020; Countryman 1972). However, our results highlight that weather information may not be used or interpreted consistently across decision-makers. Rather, what information fire managers use and what they learn from it depends on the context; weather information does not exist in a vacuum. This is consistent with the concept of preference construction, i.e., the phenomenon where decision-makers do not have pre-defined, immutable preferences going into the decision-making process. Instead, decision-makers form their preferences “on the spot” in response to cues that are available throughout the decision process. As a result, preferences are not revealed but rather constructed (Slovic 1995; Gregory et al. 2012). Specifically, our results show that the relative importance of a given piece of weather information may depend on prior decisions. For example, wind was the most important piece of weather information when switching to direct attack, but wind was less important when switching to indirect attack. The tactical decision made previously influenced how weather informed future tactical preferences. Similarly, we saw some situations where the tendency was to stick with the status quo, regardless of what the status quo decision was, which may suggest when the best decision is ambiguous, fire managers lean on previous decisions (Wilson et al. 2011). This is not necessarily a maladaptive or inefficient decision-making strategy if there are non-negligible costs to switching tactics, but the benefits of switching are unclear or uncertain.

While it is not clear from these results alone why the initial attack tactics shaped fire manager preferences, there are several theoretical explanations to consider. In the context of this experiment and in decision-making in the field, fire managers may be interpreting information holistically or comparing it to previous experience rather than integrating and weighing information through a deliberative process (Drews et al. 2015; Klein 2008). In that case, the initial team’s decision is a piece of information in and of itself, as respondents compare the current scenario to previous experience where the initial attack team either directly or indirectly engaged the fire.

Furthermore, individual pieces of information may not be considered separately but rather in light of each other. Indeed, in the context of weather, this is likely an adaptive and appropriate strategy where weather factors can be more than the sum of their parts and reach critical thresholds for extreme fire behavior (Young et al. 2019). Although examining interaction effects or non-linear thresholds was outside the scope of our study, it is worth exploring in the future to understand not only how weather components physically interact to create fire behavior, but how fire managers combine pieces of weather information to infer expected fire behavior and how this may influence their tactical decisions. For example, while more extreme projected fire behavior is related to fire managers ordering more resources, certain weather situations such as extremely high winds may pose unique risks or challenges that factor into tactical decision-making (Bayham et al. 2020).

The decision strategy that a fire manager chooses to use can make decision support and the provision of critical information more or less difficult. For example, decision-makers can use compensatory or non-compensatory decision strategies. Non-compensatory strategies do not deal directly with tradeoffs across attributes of a decision, while compensatory strategies do. A non-compensatory strategy would consider each attribute separately (e.g., if rain is forecasted, directly attack the fire; otherwise, consider the wind forecast), whereas a compensatory strategy would consider each attribute in combination (e.g., consider the precipitation and wind forecast information in light of each other when deciding on a preferred strategy). Non-compensatory strategies may be more common and are challenging to address through utility-maximizing decision support tools (i.e., decision support tools that assume decision-makers are utility-maximizers and, therefore, seek to calculate the maximum utility of each possible alternative with the assumption that the highest utility alternative is the best or most preferred). For example, a fire manager may use the Trade-off Analysis Exercise risk management tool to clarify and consider tradeoffs between risks to firefighters, the public, and identified values for several potential courses of action (Schultz et al. 2021). A compensatory decision-maker is willing to make tradeoffs between acceptable levels of risk across different values while a non-compensatory decision-maker seeks to minimize risk to one value, regardless of how much that may put other values at risk. Because the non-compensatory decision-maker is not seeking to maximize utility, but rather maximize the value of one attribute (e.g., minimize risk to a particular value, only attack directly if it is raining, etc.), utility-maximizing decision support tools may be less useful (Retief et al. 2013; Payne et al. 1993). Indeed, utility-maximizing decision support tools may be the least trusted where they are the most needed, for decisions that include painful or undesirable tradeoffs in which decision-makers have an incentive to ignore or deny the tradeoffs and make non-compensatory decisions (Beattie and Barlas 2001).

Because fire managers must make time-constrained decisions with considerable risk and uncertainty (Thompson et al. 2017b), it may be important to consider fire managers as adaptive decision-makers in the context of tactical decision-making. Adaptive decision-makers must make tradeoffs between accuracy and effort when choosing decision strategies (Payne et al. 1993). In other words, they might choose a more effortful strategy (i.e., a compensatory and tradeoff-focused strategy) to ensure a more accurate decision when the stakes are high but may do the opposite when the stakes are low. As their goals shift over the course of a fire, the same piece of information may be used in more deliberative or heuristic ways as the need for accuracy versus effort shifts. For example, when a fire first ignites and the probability of containment is high, precipitation forecasts may be used heuristically to quickly determine what resources should be sent to respond to an ignition. Later during an extended attack when it is clear the fire will not be easily contained, precipitation may be just one piece of information weighed against a host of other factors (e.g., current wind conditions, resources available, etc.). Furthermore, even in the context of one decision, fire managers may shift between decision-making strategies over time. This may occur when the decision context is uncertain and a different strategy seems more appropriate as information about possible alternatives is uncovered (Mintz et al. 1997; Mintz 2004).

These results have important implications for the design and evaluation of decision support tools for operational personnel. Understanding the impact of decision support tools on fire outcomes is difficult because the information these tools provide is only one consideration among many for fire managers (Canton-Thompson et al. 2008; Rapp et al. 2020). During pre-fire planning, decision support tools can help decision-makers make more informed and defensible decisions as they consider information in a collaborative and deliberative setting (Thompson et al. 2020). Successful decision support prior to a fire igniting may improve tactical decisions and outcomes in two ways. First, it can clarify objectives and goals for an area including what role fire may play on that landscape should one ignite. Second, it can provide insight into the relative ease of containment of a fire based on the climate, topography, and fuels (O’Connor, Thompson, and Rodríguez y Silva 2016; Wei et al. 2018). However, during a wildfire event, tactical decisions made in response to changing conditions may be more time-constrained and decision-makers may have fewer resources to dedicate to systematic decision-making or the type of compensatory decision-making intended to be supported by most existing tools.

While previous researchers have highlighted the types of information necessary for an operations-focused decision support tool (Dunn et al. 2017), results here emphasize that decision support tools should be designed and evaluated with the decision strategies used by fire managers in mind. For example, fire managers may consider some weather information more deliberatively or heuristically based on how it influences fire behavior. Weather conditions have both a direct and indirect impact on wildfires. For example, wind speed directly influences fire behavior by providing additional oxygen to the combustion zone and also by improving convective heat transfer to un-burned fuel ahead of the flaming front; therefore, increases in wind speed directly cause fires to spread faster and with higher intensity (Werth et al. 2011). Thus, all else equal, information on wind forecasts may be easier to analyze deliberatively given its incremental and direct effect on fire behavior. In comparison, other weather variations, such as temperature, humidity, and rainfall, indirectly influence fire behavior by their effects on fuel moisture content. Fire spread is determined by a simple energy balance: heat is used to either raise the temperature of adjacent fuels or it is used to evaporate water within that fuel. Variations in weather can either wet or dry fuels depending on the gradient between the fuel and air in the boundary layer around the fuel and these fuel moisture fluctuations can slow or accelerate fire spread. Thus, humidity and temperature have an indirect and incremental effect on fire behavior and may be neglected as information when making decisions rapidly. In comparison, rainfall has the strongest and most direct impact on these fuel moisture variations because it can quickly saturate fuels as well as leave additional water on the surface of the fuels. The strong influence of precipitation on fire behavior leads to a discrete and relatively concrete reduction in fire behavior, making it a useful indicator for heuristic-based decision-making, while wind, temperature, and humidity variations are more incremental and gradual.

With that in mind, decision support tools can be designed to support compensatory or non-compensatory decision-making depending on how they frame and provide weather information. Importantly, fire managers cannot be neatly demarcated as either compensatory or non-compensatory, but rather, fire managers likely change and adapt their decision-making strategy depending on the importance of the decision and the time constraints they face. Thus, it is may be helpful to provide decision-makers with a variety of tools or sources of information they can choose from based on their capacity to make deliberative versus heuristic decisions. For example, tools for compensatory decision-makers should seek to simplify and summarize information while tools for non-compensatory decision-makers should seek to reduce the arbitrariness of cutoff levels or decision thresholds (e.g., at what change of precipitation do fire managers act as if it will rain, at what ERC do fire managers switch to direct attack) (Cook 1993). In the case of previous decisions having undue influence on future planning, or the effects of anchoring to previous strategies and insufficiently adapting to new weather information, decision support tools should incorporate things like “consider the opposite” (Hirt and Markman 1995). To consider the opposite, decision support tools ask the decision-maker to consider if their decision would change with a different status quo in place, and if so, why, as a means of balancing out any effect of a particular preexisting strategy.

Improving confidence in and use of fire weather forecasts

Our results also provide insight into what conditions fire managers find most appropriate for direct and indirect attacks. Broadly speaking, fire managers preferred to directly attack fires occurring early in the season with mild fire behavior but preferred indirect attack on fires occurring late in the season with extreme fire behavior. For some fires occurring in the early or middle of the fire season where it is not raining, fire managers prefer to continue with the status quo, regardless of what it is. Fire managers were more sensitive to wind when switching to direct attack and more sensitive to precipitation when switching to indirect attack. Although the importance of different pieces of weather information varied in their influence on decision-making depending on the prior decision, our results still point to several practical needs when it comes to improving the weather information available to support decisions.

First, wind and precipitation were the most important pieces of weather information for decision-making yet respondents expressed lower confidence in the reliability of wind and precipitation forecasts. Thus, we suggest prioritizing efforts to improve the forecast accuracy where possible for these variables and increase confidence in the resulting forecast as appropriate. Typical fire weather forecasts are derived from the National Digital Forecast Database (NDFD) which are produced continuously across the USA by the US National Weather Service (Glahn and Ruth 2003). A recent study has shown that the NDFD consistently underpredicts windspeeds when the winds are stronger than about 4 m/s (~9 mi/h) (Page et al. 2018). Winds are particularly difficult to forecast due in part to local terrain influences, and extensive work is ongoing to improve wind forecasts in complex terrain. Models that downscale wind forecasts to correct for terrain influences, such as WindNinja (Wagenbrenner et al. 2016), show promise in improving local-scale wind forecasts.

Quantitative precipitation forecasts provided to wildland fire decision-makers are commonly derived from the NDFD, and they are often modified by forecasters prior to issuance. However, investigators are continually exploring ways to improve precipitation forecast skill and spatial resolution using models such as the High Resolution Rapid Refresh (HRRR) (Benjamin et al. 2016). Continual improvements to the HRRR model physics and data assimilation show promise in improving precipitation forecasts over the next 18 h (Bytheway et al. 2017). This interval generally conforms to an operational period for wildland fire operations. Other improvements to precipitation forecasts, such as ensembling, can provide uncertainty estimates of forecasts that may also be useful for decision-makers. Ultimately, given the importance of precipitation forecasts on decision-making, any efforts to improve skill or characterize uncertainty in precipitation forecasting will likely influence wildfire outcomes.

That said, improving model accuracy may not be sufficient on its own. While a certain threshold of accuracy and quality is necessary for model forecasts to have value to decision-makers, model quality is multi-faceted and not the same as the utility of a model to decision-makers (Murphy 1993). While it would be reasonable to expect some correlation between accuracy and confidence in wind and precipitation forecasts, it is not a given that improvements in forecast accuracy will automatically lead to increased confidence. Thus, distinct efforts may be necessary to improve confidence in the models. These efforts could be informed by better understanding what aspects of the model lead stakeholders to use or ignore the resulting forecast. For example, in some cases, personnel may be resistant to using models due to cultural ideas surrounding technology and models (Rapp et al. 2020; Noble and Paveglio 2020). In those cases, it may be more fruitful or even necessary to focus on changing how users relate using models and being competent at their jobs. In other cases, stakeholders and users may be disconnected from the development process for models, and communicating or demonstrating improvements may be helpful. In other cases still, the problem may not be with the models, but the perceived competence of the modelers (Noble and Paveglio 2020; Rapp et al. 2020). In these instances, investing additional resources and attention towards training modelers and establishing relationships between modelers and end users may contribute to improving confidence in the resulting forecasts.

Seasonality was the most important non-weather-related attribute across conditions, with roughly half of respondents explicitly highlighting it as an important decision criterion in the open-ended response. Across both conditions, respondents preferred direct attack early in the season and indirect attack later. Although direct and indirect attacks can be used on all fires regardless of the over-arching strategy, this preference appears borne out by the data which suggests that a greater proportion of fires are managed for suppression early in the season while the proportion being managed for other reasons increases later in the season (Young et al. 2020). In terms of tactics and strategy, the decision space of fire managers is likely larger later in the season as seasonal changes associated with the onset of autumn are likely to aid containment and reduce the severity of fire behavior. Additionally, fire managers may be able to justify using more resources to manage or indirectly attack a fire later in the season because these resources are less likely to be needed on a later fire during the same fire season. A key follow-up question is therefore how does weather information interpretation change over time? For example, while wind may be an important driver of fire manager decision-making regardless of the time of year, the interpretation of precipitation may depend on the time in season, where precipitation earlier in the season may have less of an impact (or indeed may make fires worse through lightning strikes) but late-season precipitation may signal a season-ending event. Similarly, weather information may vary in importance over the course of a fire event. This work examines a pivotal moment in fire management, when fires transition from initial to extended attack, but other key decision points are worth considering, such as the decision to manage for resource benefit or suppression. Indeed, as more forests utilize pre-identified operation control points, it will be important to understand how weather shapes which control points are selected and what tactics are used. It is worth exploring in greater detail how fire managers personally understand and estimate wind, rainfall, humidity, and other fire behavior drivers and thus how information on these drivers influences the perception of fire behavior over the course of events and seasons (i.e., to what extent do fire managers’ mental models of the effect of fire weather conditions on fire behavior mimic actual fire behavior model predictions?).

Conclusion

Considerable effort has been made to support risk-based strategic decision-making for fire managers. To that end, many tools exist to provide information and structure decision-making. While some of those tools can be used at the tactical level, tactical decision-makers may rely on and use different sources of information, especially weather. Weather plays a critical role in fire behavior and subsequently an important role in tactical decision-making and success. Understanding how fire managers use weather information to make tactical decisions is key to providing effective decision support. While weather information does indeed influence decisions, this information is not consumed in a vacuum; fire managers interpret it in light of previous tactics made by other actors. When designing operational decision support tools, it will be important to consider not only what information fire managers seek out and use, but how they use it, as decision strategies, deliberative, heuristic, or otherwise, will shape tactical decisions and their consequences. Consequently, rather than simply providing information, decision support tools should also actively debias for things like insufficient adjustment to new information, for example by encouraging fire managers to imagine how their decisions might be different under a different status quo. Additionally, our results suggest opportunities for improvement and further study. Between wind, precipitation, and humidity forecasts, wind and precipitation were the biggest driver behind switching operational tactics, yet fire managers were less confident in wind and precipitation forecasts than weather forecasts in general. Improving or communicating forecast reliability may facilitate more risk-informed outcomes on fires by influencing what information fire managers attend to and how much weight they give that information.