Impacts of Phased-Array Radar Data on Forecaster Performance during Severe Hail and Wind Events KATIE A. BOWDEN Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, Norman, Oklahoma PAMELA L. HEINSELMAN NOAA/OAR/National Severe Storms Laboratory, Norman, Oklahoma DARREL M. KINGFIELD Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, and NOAA/OAR/National Severe Storms Laboratory, Norman, Oklahoma RICK P. THOMAS School of Psychology, Georgia Institute of Technology, Atlanta, Georgia (Manuscript received 29 August 2014, in final form 8 December 2014) ABSTRACT The ongoing Phased Array Radar Innovative Sensing Experiment (PARISE) investigates the impacts of higher-temporal-resolution radar data on the warning decision process of NWS forecasters. Twelve NWS forecasters participated in the 2013 PARISE and were assigned to either a control (5-min updates) or an experimental (1-min updates) group. Participants worked two case studies in simulated real time. The first case presented a marginally severe hail event, and the second case presented a severe hail and wind event. While working each event, participants made decisions regarding the detection, identification, and re- identification of severe weather. These three levels compose what has now been termed the compound warning decision process. Decisions were verified with respect to the three levels of the compound warning decision process and the experimental group obtained a lower mean false alarm ratio than the control group throughout both cases. The experimental group also obtained a higher mean probability of detection than the control group throughout the first case and at the detection level in the second case. Statistical significance ( p value 5 0.0252) was established for the difference in median lead times obtained by the experimental (21.5 min) and control (17.3 min) groups. A confidence-based assessment was used to categorize decisions into four types: doubtful, uninformed, misinformed, and mastery. Although mastery (i.e., confident and correct) decisions formed the largest category in both groups, the experimental group had a larger proportion of mastery decisions, possibly because of their enhanced ability to observe and track individual storm char- acteristics through the use of 1-min updates. 1. Introduction During warning operations, weather forecasters rely heavily on radar technology to observe and monitor potentially severe thunderstorms (Andra et al. 2002). The National Weather Service (NWS) currently utilizes a network of 158 Weather Surveillance Radar-1988 Dopplers (WSR-88Ds) that are located across the United States (Whiton et al. 1998). Given that the WSR- 88D was initially designed with a projected lifetime of 20 yr (Zrnic et al. 2007), continuous upgrades are re- quired to maintain its functionality (e.g., Saffle et al. 2009; Crum et al. 2013). However, eventually the WSR- 88D network will have to be replaced. A replacement candidate under consideration is phased-array radar (PAR; Zrnic et al. 2007). To explore the suitability of PAR for weather observation, a phased-array antenna was loaned to the NOAA/National Severe Storms Corresponding author address: Katie Bowden, 120 David L. Boren Blvd., Norman, OK 73072. E-mail: [email protected]APRIL 2015 BOWDEN ET AL. 389 DOI: 10.1175/WAF-D-14-00101.1 Ó 2015 American Meteorological Society Unauthenticated | Downloaded 02/17/22 08:19 AM UTC
16
Embed
Impacts of Phased-Array Radar Data on Forecaster ...
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Impacts of Phased-Array Radar Data on Forecaster Performance during SevereHail and Wind Events
KATIE A. BOWDEN
Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, Norman, Oklahoma
PAMELA L. HEINSELMAN
NOAA/OAR/National Severe Storms Laboratory, Norman, Oklahoma
DARREL M. KINGFIELD
Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, and NOAA/OAR/National
Severe Storms Laboratory, Norman, Oklahoma
RICK P. THOMAS
School of Psychology, Georgia Institute of Technology, Atlanta, Georgia
(Manuscript received 29 August 2014, in final form 8 December 2014)
ABSTRACT
The ongoing Phased Array Radar Innovative Sensing Experiment (PARISE) investigates the impacts of
higher-temporal-resolution radar data on the warning decision process of NWS forecasters. Twelve NWS
forecasters participated in the 2013 PARISE and were assigned to either a control (5-min updates) or an
experimental (1-min updates) group. Participants worked two case studies in simulated real time. The first
case presented a marginally severe hail event, and the second case presented a severe hail and wind event.
While working each event, participants made decisions regarding the detection, identification, and re-
identification of severe weather. These three levels compose what has now been termed the compound
warning decision process. Decisions were verified with respect to the three levels of the compound warning
decision process and the experimental group obtained a lower mean false alarm ratio than the control group
throughout both cases. The experimental group also obtained a higher mean probability of detection than the
control group throughout the first case and at the detection level in the second case. Statistical significance
( p value 5 0.0252) was established for the difference in median lead times obtained by the experimental
(21.5min) and control (17.3min) groups. A confidence-based assessment was used to categorize decisions
into four types: doubtful, uninformed, misinformed, and mastery. Although mastery (i.e., confident and
correct) decisions formed the largest category in both groups, the experimental group had a larger proportion
of mastery decisions, possibly because of their enhanced ability to observe and track individual storm char-
acteristics through the use of 1-min updates.
1. Introduction
During warning operations, weather forecasters rely
heavily on radar technology to observe and monitor
potentially severe thunderstorms (Andra et al. 2002).
The National Weather Service (NWS) currently utilizes
a network of 158 Weather Surveillance Radar-1988
Dopplers (WSR-88Ds) that are located across the
United States (Whiton et al. 1998). Given that theWSR-
88D was initially designed with a projected lifetime of
20 yr (Zrni�c et al. 2007), continuous upgrades are re-
quired to maintain its functionality (e.g., Saffle et al.
2009; Crum et al. 2013). However, eventually the WSR-
88D network will have to be replaced. A replacement
candidate under consideration is phased-array radar
(PAR; Zrni�c et al. 2007). To explore the suitability of
PAR for weather observation, a phased-array antenna
was loaned to the NOAA/National Severe Storms
Corresponding author address: Katie Bowden, 120 David L.
Laboratory (Forsyth et al. 2005) in Norman, Oklahoma
by the U.S. Navy. A key characteristic of this PAR is its
capability to provide volume updates in less than 1min
(Heinselman and Torres 2011).
When exploring future replacement technologies to
the WSR-88D, an important consideration is forecaster
needs. In a survey conducted by LaDue et al. (2010),
forecasters expressed a need for higher-temporal-
resolution radar data during rapidly evolving weather
events. In particular, forecasters reported that the 4–6-min
updates provided by the WSR-88D are insufficient for
observing radar precursor signatures of thunderstorms
such as downbursts (LaDue et al. 2010). Fujita and
Wakimoto (1983) define a downburst as, ‘‘A strong
downdraft which induces an outburst of damaging winds
on or near the ground.’’ Radar precursor signatures,
such as a descending high-reflectivity core and strong
midlevel convergence, can be used to identify storms
capable of producing a downburst (e.g., Roberts and
Wilson 1989; Campbell and Isaminger 1990). Such pre-
cursor signatures, however, can evolve too quickly for
trends to be sampled sufficiently by theWSR-88D. Such
limitations may result in delayed warnings and therefore
reduced lead time or, worse, missed events. These lim-
itations are of concern because downbursts can produce
damaging winds at the surface, presenting a threat to life
and property. Therefore, for improvement in warning
operations, a future radar system should be capable of
sampling the atmosphere on a shorter time scale, which
PAR can provide.
Heinselman et al. (2008) examined the weather sur-
veillance capabilities of the PAR during severe weather
events. In particular, microburst precursor signatures
observed by the PAR were compared to those observed
by the WSR-88D. During a 13-min observation period
when a storm was sampled by both radars, the PAR and
WSR-88D collected 23 and 3.5 volume scans, respec-
tively. The considerably faster PAR sampling resulted in
an improved ability to observe and track microburst
precursor signatures, prior to the detection of divergent
outflow at the lowest scans. Additionally, Heinselman
et al. (2008) analyzed a hailstorm observed by PAR.
Although a comparison to the WSR-88D was not
available, the development of radar features indicative
of a hail threat (e.g., bounded weak-echo region and
three-body scatter spike) were clearly visible in PAR
data as the storm quickly evolved. These findings by
Heinselman et al. (2008) suggest that the use of PAR
data could provide forecasters with the ability to detect
impending severe weather earlier, which in turn may
provide the public with longer warning lead times.
The Phased Array Radar Innovative Sensing Exper-
iment (PARISE) was designed to assess the impacts of
higher-temporal-resolution radar data on the warning
decision process of forecasters (Heinselman et al. 2012;
Heinselman and LaDue 2013). The work of PARISE is
critical to ensuring that the implementation of PAR
technology would be beneficial to the NWS. The 2010
and 2012 PARISE focused on low-end tornado events
(Heinselman et al. 2012; Heinselman and LaDue 2013).
Both experiments reported enhanced forecaster per-
formance with the use of 1-min radar updates compared
to forecasters using traditional 5-min radar updates, as
demonstrated through warnings issued with longer tor-
nado lead times. The purpose of this study was to extend
the work of PARISE to include severe hail and wind
events, with a focus on downbursts (see section 3b for
the NWS definition of severe). Based on the findings of
Heinselman et al. (2012) and Heinselman and LaDue
(2013), we hypothesized that during such events, rapidly
updating radar data would positively impact the warning
decision process of NWS forecasters. To assess this hy-
pothesis, data collection focused on both quantitative
and qualitative aspects of the forecaster warning de-
cision process. In particular, details of warning products
were recorded so that forecaster performance could be
assessed from a verification standpoint. The data col-
lected revealed that the warning decision process com-
prised three key decision stages. For this reason,
verification was assessed with regard to what has been
termed the compound warning decision process, which
recognizes that forecasters detect, identify, and re-
identify severe weather (see section 3a). Additionally,
confidence ratings were obtained each time a forecaster
made a key decision, along with reasoning for each
confidence rating. Through the use of a confidence-
based assessment, these ratings were analyzed to ad-
dress the question of whether increasing the temporal
availability of radar data leads to better decisions. Spe-
cifically, decisions were categorized into four types:
doubtful, uninformed, misinformed, and mastery. The
reasoning for each confidence rating provides insight
into why each decision type occurred, and whether the
temporal resolution of radar data played a role.
2. Methods
a. Experimental design
From two NWS Weather Forecast Offices (WFOs),
12 forecasters were recruited to participate in the 2013
PARISE. The two WFOs were located in the NWS’s
Southern and Eastern Regions, and therefore given the
climatology of these regions, the 12 forecasters would
have experienced working severe hail and wind events
(Kelly et al. 1985). During each of the six experiment
weeks, one forecaster from each WFO visited Norman,
390 WEATHER AND FORECAST ING VOLUME 30
Unauthenticated | Downloaded 02/17/22 08:19 AM UTC
Oklahoma. The experiment adopted a two-independent-
group design, where each week forecasters were as-
signed to either a control or an experimental group.
The volume update time acted as the independent var-
iable, where the control group received 5-min updates
from temporally degraded PAR data, and the experi-
mental group received 1-min updates from full-
temporal-resolution PAR data.
To ensure balanced groups in terms of knowledge and
experience, matched random assignment was in-
corporated into the experiment design. Matching was
accomplished through an online survey that was issued
to participants prior to the experiment. Participants’
experience was measured by the number of years they
had worked in the NWS (Table 1, columns 1 and 3).
Although experience is important with respect to the
amount of exposure one has had in their work envi-
ronment, experience does not imply expertise. As de-
scribed by Jacoby et al. (1986), experience and expertise
are ‘‘conceptually orthogonal,’’ with a distinguishing
factor being that expertise is achieved through acquiring
a ‘‘qualitatively higher level of either knowledge or
skill.’’ Therefore, to assess aspects of forecaster exper-
tise relevant to this study, knowledge was measured
through four questions regarding familiarity (Table 1,
columns 2 and 3), understanding, knowledge of pre-
cursors, and training with respect to downburst events
(Table 1, columns 4–7). For knowledge, the three
questions requiring qualitative responses were com-
pared to criteria that were based on downburst con-
ceptual models (e.g., Atkins and Wakimoto 1991).
Based on their survey responses, all participants were
assigned an experience and knowledge score ranging
between 1 and 5 (Fig. 1). The experience score was
based on the single experience question, whereas the
knowledge score was generated by averaging the points
obtained from the four knowledge questions. Among
the participants, experience was spread fairly evenly,
and knowledge was clustered around the medium range
TABLE 1. Criteria for points assigned to questions from the preexperimental online survey. Columns 1 and 3 refer to how experience
scores were assigned, and columns 2–7 refer to how knowledge scores were assigned. In column 2, a scale from 1 to 10 is used (where 1
indicates no familiarity and 10 indicates extensive familiarity).
Experience
(yr) Familiarity Points
Understanding
of a downburst
Precursors for
forecasting a downburst Training Points
#5 1 and 2 1 Definition Suspended core Distance Learning
Operations Course
and Advanced
Warning Operations
Course
Assign one point for
each topic discussed
within the question
category; total of five
points for each category
#10 3 and 4 2 Wet and dry variety
recognized
Midaltitude radial
convergence
Seasonal familiarization
training
#15 5 and 6 3 Description of
soundings
Storm-top
divergence
Other courses (e.g.,
online/workshops)
#20 7 and 8 4 Thermodynamic
and dynamic
mechanisms
Environment
assessment
Exposure to
literature/current
forecasting techniques
.20 9 and 10 5 Demonstration of
an understanding
beyond that of a
typical responder
Demonstration of
an understanding
beyond that of a
typical responder
Personal experience
(e.g., storm chasing)
FIG. 1. Experience and knowledge scores for each participant are
given. The group assignment of each participant was based on the
control and experimental group combinations that yielded the
smallest Mahalanobis distance. Participants assigned to the control
or experimental groups are assigned open or filled circles,
respectively.
APRIL 2015 BOWDEN ET AL . 391
Unauthenticated | Downloaded 02/17/22 08:19 AM UTC
(Fig. 1). For all possible group combinations, the
Mahalanobis distance was computed to assess the sim-
ilarity between groups by using experience and knowl-
edge scores as variables (McLachlan 1999). The smallest
distance represented the greatest similarity between
groups, which therefore determined the group assign-
ment for each participant (Fig. 1).
Although efforts were made to match groups, the
limitations associated with the applied methodology
should be acknowledged. A limitation that arose fol-
lowing the distribution of the survey was that partici-
pants may not have always interpreted the questions
correctly, leading to discussions on tangential topics. For
example, participants were asked to explain their un-
derstanding of a downburst. Although most participants
perceived this question as intended (Table 1, column 4),
some responses focused on the type of damage observed
from downbursts. In addition, the amount of time and
effort that participants invested into the survey was
likely variable. For these reasons, it is possible that
survey responses did not provide a complete represen-
tation of participants’ knowledge. However, despite
this possibility, the consistent assessment of survey re-
sponses and the use of a similarity metric provided
a means to objectively match groups.
b. Case studies
The National Weather Radar Testbed located in
Norman, Oklahoma, is home to an S-band PAR that is
being evaluated and tested for weather applications.
Given that the PAR is a single flat-panel array, data
collection is limited to a 908 sector at any one time.
PAR’s electronic beam steering means that it operates
with a nonconformal beamwidth increasing from 1.58 to2.18 as the beam is steered from boresight to6458 (Zrni�cet al. 2007). Additionally, the electronic beam steering
allows the atmosphere to be scanned noncontiguously,
enabling weather-focused observations, which further
reduce the volume update time to less than 1min
(Heinselman and Torres 2011; Torres et al. 2012).
Based on the following criteria, two cases from ar-
chived PAR data were selected for the 2013 PARISE
(Table 2). First, the cases needed to be long enough to
allow participants to settle into their roles and demon-
strate their warning decision processes as the weather
evolved. Second, severe hail and/or wind reports needed
to be associated with the event, preferably toward the
end of the case to give participants an opportunity to
interrogate the storms beforehand and make warning
decisions as necessary. Third, for consistent low-level
sampling of the weather event, the PAR data needed to
be uninterrupted and within a range of 100 km from
the radar.
Case 1 presented multicell clusters of storms that oc-
curred at 0134–0210 UTC 20 April 2012 (Figs. 2a,b;
Table 2). This marginally severe (i.e., at or slightly
greater than the severe criteria) hail event was observed
by the PAR using an enhanced volume coverage pattern
(VCP) 12 strategy. Specifically, this VCP scanned 19
elevation angles ranging between 0.518 and 52.908. Al-
though only one severe hail report occurred during case
time, an additional six hail reports were associated with
the same storm 1h after case end time.
Case 2 included multicellular storms with some rota-
tion that were sampled by PAR at 2053–2139 UTC 16
July 2009 (Figs. 2c,d; Table 2). PAR collected data using
a VCP that was composed of 14 elevation angles ranging
between 0.518 and 38.808. Both severe hail and wind
events were reported and associated with a downburst
event that occurred in central Oklahoma. During case
time, there was one severe wind and two severe hail
reports. Within the hour after case end time, an addi-
tional 16 reports of severe hail and wind events were
associated with the same storm.
All storm reports were obtained from StormData, which
is logged in the NWS Performance Management System
(https://verification.nws.noaa.gov/). Because the spatial
and temporal accuracy of Storm Data is limited (e.g., Witt
et al. 1998; Trapp et al. 2006), it was important to ensure
consistency between the location and timing of storm re-
ports with the radar data. Additionally, weather reports
obtained during the Severe Hazards Analysis and Verifi-
cation Experiment (SHAVE; Ortega et al. 2009) were
examined to validate confidence in the storms that did not
produce severe weather. Both SHAVE and Storm Data
were in agreement with storms classified as null events.
The occurrence of both severe and nonsevere storms
during the cases provided a realistic scenario whereby
participants were challenged to differentiate between
storms that would and would not produce severe weather.
c. Working the cases
Before working each case, participants viewed
a weather briefing video that was prepared by J. Ladue
TABLE 2. Descriptions of cases 1 and 2.
Case 1 Case 2
Time and
date
0134–0210 UTC
20 Apr 2012
2053–2139 UTC 16 Jul 2009
Event type Multicell, severe hail Multicell, severe hail
and wind
Storm
reports
0209 UTC, 1-in.
hail
2135 UTC, 1.75-in. hail;
2135 UTC, estimated
gust 56-kt wind; and
2138 UTC, 1.75-in. hail
VCP 19 elevations,
0.518–52.90814 elevations, 0.518–38.808
392 WEATHER AND FORECAST ING VOLUME 30
Unauthenticated | Downloaded 02/17/22 08:19 AM UTC