A multi-sensor physically based weather/non-weather radar echo classifier using polarimetric and environmental data in a real-time
national system
Lin TangCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,
Norman, Oklahoma
Jian Zhang NOAA/OAR National Severe Storms Laboratory, Norman, Oklahoma
Carrie LangstonCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,
Norman, Oklahoma
John KrauseCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,
Norman, Oklahoma
Kenneth HowardNOAA/OAR National Severe Storms Laboratory, Norman, Oklahoma
Valliappa LakshmananCooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma,
Norman, Oklahoma
________________________________________________________________________________________________Corresponding author address: Lin Tang, CIMMS, 120 David Boren Dr., Norman, OK 73072 Email: [email protected]
1
Abstract
Polarimetric radar observations provide information regarding the hydrometeor
shape, size, and phase as well as an improved skill in differentiating radar echoes
of hydrometeors from those of non-hydrometeors. In this work, a multi-sensor
physically based algorithm is designed to classify weather/non-weather radar
echoes. The algorithm uses reflectivity, correlation coefficient, and temperature
sounding data to quality control the reflectivity data by applying a set of explicit
meteorological rules. The proposed methodology are tested with a large number
of real data cases across different geographical regions and seasons and showed
a high accuracy (Heidke Skill Score of 0.83) in segregating weather and non-
weather echoes. Comparing to other quality control methodologies using all
polarimetric variables, this algorithm’s advantage is in its simplicity, effectiveness
and computational efficiency. The current methodology is also demonstrated in a
real-time multi-radar and multi-sensor national mosaic system.
1. Introduction
A significant challenge in radar-derived quantitative precipitation estimation
(QPE) is the separation of precipitation from non hydro-meteorological returns.
Additionally, there are errors from uncertainties associated with where the radar is
sampling precipitation (i.e., the issue of non-uniform vertical profiles of reflectivity,
see Koistinen 1991; Kitchen et al. 1994; Andrieu and Creutin 1995; Smyth and
Illingworth 1998; Krajewski and Vignal 2001; Germann and Joss 2002) and the
conversion of radar reflectivity into rain rate (i.e., the issue of Z-R relationships, see
Marshall et al. 1955; Carbone and Nelson 1978; Rosenfeld et al. 1993; White et al.
2003).
An initial step for accurate radar QPE is the identification and removal of
non-precipitation contaminants including the electromagnetic interference with
2
transmitters (e.g., sun strobes), anomalous propagation (AP) of the radar beam due
to specific atmospheric temperatures or moisture gradients, beam blockage due to
topography (Bebbington et al., 2007), the biological echoes (birds, bats, insects),
power returns from manmade objects on the ground (Bachmann and Tracy, 2009)
, and sea clutter (Gray and Larsen, 2004). Manual quality control (QC) of radar
reflectivity data has been practiced at commercial weather companies and in
operations such as the National Weather Services River Forecast Centers. However,
manual quality control is resource expensive and prohibitive for timely warnings of
severe weather and flash floods on a local to national scale. To address the timely
need for quality radar information, an automated approach is required.
The national mosaic and multi-sensor QPE (NMQ) system is a real-time multi-
radar, multi-sensor system (Zhang et al. 2011). The system integrates 140+ U.S.
Weather Surveillance Radar-1988 Doppler (WSR-88D) radars and 30+ Canadian
weather radars to generate a three-dimensional reflectivity mosaic and suite of
severe weather and QPE products over the conterminous US (CONUS) domain at
1km spatial resolution every 5 minutes. Applying QC to the reflectivity is critical in
assuring the accurate depiction of storms and reliability of precipitation estimates.
The WSR-88D radar network was comprised of single-polarization radars before
2011. The reflectivity QC methodology used in the NMQ was a neural network
approach (QCNN) based on reflectivity, velocity and spectrum width, as well as
atmospheric environmental data (Zhang et al. 2004; Lakshmanan et al. 2007; 2010)
. Evaluations of the single-polarization radar QC in NMQ indicated that the scheme
was effective at removing a significant amount of non-precipitation returns under
various scenarios. However, biological clutter during the peak migratory seasons
of spring and fall remains a major issue, especially during periods when storms
develop and or move into regions of ‘biological blooms’. These ‘blooms’ are not
fully censored by the QCNN because the magnitude, vertical depth, and local texture
of insects/birds/bats returns in single-polarization radar fields are often very
similar to widespread, shallow stratiform rain or snow (Lakshmanan et al., 2010)
. The biological blooms can cause erroneous precipitation in the QPE products.
3
Conversely, light rain and snow echoes were sometimes removed as a result of
minimizing the biological blooms. Additional challenges are present when removing
AP from the ground and sea surface at far ranges. While AP echoes usually have zero
velocities, WSR-88D velocity field does not extend beyond 230km when reflectivity
data extends up to 460km, therefore, AP echoes at far ranges can be confused with
small convective storms in the reflectivity field.
Commencing in 2011, the WSR-88D network has been upgraded with dual
polarimetric capability. Polarimetric radar observations provide additional
information regarding the hydrometeor shape, size, and phase and an improved
skill in differentiating radar echoes of hydrometeors from those of non-
hydrometeors. The hydrometeor classification algorithms (HCAs) with varying
levels of complexity have been developed for radars of different wavelengths
(e.g., Liu and Chandrasekar 2000; Zrnic et al. 2001; Lim et al. 2005; Gourley et al.
2007; Marzano et al. 2008; Park et al. 2009, and etc.) and were shown, by multiple
investigators, to perform well.
Park et al. 2009 developed a fuzzy logic HCA that classifies WSR-88D radar
echoes into 11 different hydrometeor and non-hydrometeor categories, and the
algorithm has been implemented as part of the National Weather Service
operational radar product baseline. By taking the output of this sophisticated
classification scheme and removing the non-meteorological echoes, a dual-
polarization (DP) radar QC algorithm could be easily implemented. However, the
fuzzy logic HCA was applied on a pixel-by-pixel basis. Due to non-exclusive
membership functions in the fuzzy logics for different echo types, individual pixels
associated with biological or other non-hydrometeors could be incorrectly
identified as precipitation and vice versa (e.g., Liu & Chandra 2000). For instance,
there are cases that the HCA misclassifies clutter echoes as category “BD” (big drop)
(see, Figure1) or “UK” (unknown). The HCA also had difficulties in identifying severe
AP or sea clutter especially when the AP is beyond the range where the beam
supposedly intersects the melting layer. Figure2 provides an example of severe sea
clutter observed by WSR-88D KMHX at 23:58:34 UTC on 29 June 2012. As shown in
4
Figure 2(b), the HCA is able to classify most of the clutter from the sea surface
except the portion at far ranges. Therefore further refinements are needed for the
operational WSR-88D HCA algorithm to more accurately segregate hydrometeor
and non-hydrometeor echoes. Further, the operational HCA involves all
polarimetric variables, i.e., reflectivity Z, velocity V, spectrum width SPW,
correlation coefficient RhoHV, differential reflectivity Zdr, and differential phase of
propagation PhiDP. The HCA requires a pre-processing and quality assurance of
each of the variables before the fuzzy logic computations. The computational cost
(memory and CPU) for running such algorithms in real-time can be relatively high,
which poses a challenge for processing a national network of more than 140 radars.
For applications that do not require detailed classifications of different hydrometeor
species, such a sophisticated HCA might be an excessive option given that a simpler
scheme may be sufficient.
Figure1: (a) the original base reflectivity field at the elevation angle (EA) of 0.5 degrees from radar KEWX; (b) the corresponding hydrometeor field derived from HCA at the EA of 0.5 degree at 07:19:46 UTC on 06 May 2012.
Figure2: (a) the original base reflectivity field from radar KMHX at the elevation angle of 0.5 degrees; (b) the corresponding hydrometeor field derived from HCA; (c) the associated RhoHV field. The fields are observed at 23:58:34 UTC on 29 June 2012.
5
Lakshmanan et al. (2013) used a neural network approach (“dpQCNN”
hereafter) to segregate weather and non-weather echoes using all polarimetric
variables and showed a good Heidke Skill Score (HSS) of 0.8 (Lakshmanan et al.,
2013). Due to the statistical scoring process and the “black box” character of a
neural network model, it is not easily determined which variables are the most
important contributors to a particular output, and it may have limited ability to
explicitly identify possible causal relationships (Tu 1996). It requires all of the six
polarimetric radar moments and the computational requirements (especially the
memory) can be relatively high. Given the distinctly different characteristics of
weather vs. non-weather echoes in the correlation coefficient (ρhv or RhoHV) field
and in the 3D structure of reflectivity and atmospheric thermal dynamic fields, there
is potential for a simple and physically based method to segregate weather and non-
weather echoes.
In the current study, we explore a physically based dual-polarization radar
QC (“dpQC”) approach that segregates weather and non-weather echoes using
correlation coefficient, reflectivity, atmospheric sounding data, and a set of
explicit meteorological principles. With the reduced number of input variables,
computations related to the calibration or quality assurance of various polarimetric
radar variables (e.g., Zdr and Kdp) are reduced and the possible sources of error
lessened. The scheme was tested on the CONUS domain in real-time for several
months including the peak biological migrating season in the spring. The results
show that the dpQC scheme was very effective in removing biological echoes
while retaining the light precipitation. It performs equally well at segregating
weather and non-weather echoes as the operational WSR-88D HCA scheme and
the Lakshmanan (2013) scheme but with significantly less memory and CPU
requirements.
2. The Multi-Sensor Physically Based Dual-Polarization Radar QC Method
6
The following section provides details of the dpQC methodology with each
component being presented using real data examples individually, and then
collaboratively.
2.1 The basic RhoHV filter
Correlation coefficient (RhoHV) is one of three DP base moments. It
measures the consistency of the horizontal and vertical returned power and phase
for each pulse. It also provides information on the quality of the dual-polarization
data fields and implies information on the nature of the scatters. RhoHV is
expressed as a number between 0 and 1: generally pure rain or pure snow are
associated with high values close to 1; low RhoHV values, alternatively, represent a
spectrum of scatters that are likely non-meteorological. Using a simple RhoHV
thresholding approach, the clutter with low coherent signals can be censored.
Figure 3 shows an example of a basic RhoHV filter where all echoes with RhoHV ≤
0.90 are removed. Figure 3(a) is the base reflectivity map from the radar KEWX at
05:51:39 UTC on 6 May 2012. With the biological echoes (“blooms”) mixed with the
storm, the conventional QCNN algorithm failed to separate the precipitation
information from the non-precipitation as shown in Figure 3(b). While the power
returns from birds and insects can be as strong as meteorological scatterers, the
correlation coefficient of the former is much lower than that of raindrops (Figure
3(d)). There were some unphysical high RhoHV values (greater than 1) at the edge
of precipitation areas and in blooms, which were associated with very low signal-to-
noise ratios and will be discussed later in this paper. With a simple RhoHV
threshold, the QC’ed reflectivity field (Figure 3(c)) is nearly free of the biological
bloom contamination. This example shows the remarkable effectiveness of a simple
RhoHV filter when separating weather and non-weather echoes in cases of a heavily
mixed precipitation and biological echoes that are normally impossible for single
polarization radar based QC approaches.
7
Figure3: (a). The raw reflectivity field from radar KEWX at 05:51:39 UTC on 06 May 2012; (b) the reflectivity field after the process of QCNN; (c) the QC’ed reflectivity field with the method of QC_DT; (d) the RhoHV field. All images are from 0.5-deg tilt.
Figure 4(a) is a histogram of RhoHV values in a volume scan observed by KARX
on 26 June 2012 at 19:33 UTC. An experienced radar meteorologist reviewed the
data, and manually separated meteorological or non-meteorological echoes. A
histogram of RhoHV values was then constructed for each of the two echo types and
then normalized. The histograms (Figure 4(a)) showed that non-meteorological
pixels (gray bars) had a wide spread value of RhoHVs between 0.3 and 0.95, while
the meteorological echoes (black) are generally associated with RhoHVs greater
than 0.95. However, this RhoHV feature is not persistent across every weather
situation. For a convective storm observed by radar KVNX, at 07:36 UTC on 20 May
2011, the histograms (see Figure 4(b)) of weather and non-weather echoes are
overlapping, with the clutter RhoHV values spreading across the whole range of
0-1. Apparently there were power returns from non-meteorological objects with
high correlation between the horizontal and vertical directions; at the same time,
some portions of the weather echoes were associated with low cross-correlation
8
coefficients smaller than 0.80. In this case and likely others, a fixed RhoHV threshold
would lead to false alarms or misses in clutter detection. Therefore additional steps
are devised to further screen the clutter associated with high RhoHV values and vice
versa.
(a) (b)Figure4: The histogram plots of the RhoHV values of the power returns from the non-meteorological (gray) and meteorological targets (black) observed by radars (a) KARX at 19:33 UTC on 26 June 2012 and (b) KVNX at 07:36 UTC on 20 May 2011, respectively.
2.2 Handling Exceptions to the Simple RhoHV Filter
a). Low RhoHVs in hail
Due to the complex shape of large hailstones, the horizontal and vertical radar
pulses behave differently resulting in low RhoHV (0.80-0.90) in hail regions of
convective storms (e.g., the circled areas in Figure 5(b)). However, hailstorms are
generally associated with intense horizontal (conventional) reflectivity (> 45-
50dBZ) and tall storm echo tops (> 8km). By adjusting the RhoHV filter threshold
in regions with the aforementioned characteristics, the hail cores can be protected
from the filtering. Figure 5(c) shows the QC’ed reflectivity field with the adjusted
RhoHV filter, where the biological echoes close to the radar are completely removed
while weather echoes including the hail are successfully retained.
9
Figure5: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC on a PPI scope at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KAMA at the 02:50:30 UTC on 01 May 2012. The circled areas point out the hail echoes.
b). Low RhoHVs associated with non-uniform beam filling
Non-uniform beam filling (NBF) occurs when significant gradients of hydrometeor
properties (concentration, size, shape, or phase) occur within a pulse volume, e.g.,
near the edge of hail cores in convective storms. These gradients can occur in
either the horizontal or vertical. The simple RhoHV filter can incorrectly remove
precipitation echoes in regions with severe NBF, as shown in a hailstorm example
in Figure 6. The circled area in Figure 6(b) indicates a low RhoHV region due to
the NBF near a strong hail core. The simple RhoHV threshold could have removed
the echoes in this area. However, an examination of high echo tops in this region
indicated that the echoes were associated with storms. Therefore the echo-top
height information was applied to redeem those precipitation echoes with low
cross-correlation coefficients, as demonstrated in Figure 6(c).
10
Figure6: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KVNX at the 03:37:13 UTC on 01 May 2012. The white-circled areas outline regions of severe NBF.
c). Low RhoHVs in wet snow aggregations
Due to their mixed phases, echoes from the wet snow aggregates could have low
RhoHV values (<0.95). Figure 7 is a case of low correlation coefficient observed in
the melting layer during a winter precipitation event. To mitigate the effects from
low correlation coefficient in wet snow aggregations, the temperature sounding
data is employed in the dpQC algorithm. Using the approximate zero-degree height,
the algorithm searches for a ring of low RhoHV (~0.85-0.95), which is associated
with the melting layer. Along each of the azimuth radial direction, if the RhoHV
drop and rise are detected, the threshold of RhoHV filter is adjusted lower for the
melting hydrometeors. As shown in Figure 7(c), the dpQC was able to retain the
weak weather echoes. Retaining weak echoes in a cold environment is critical
for accurate precipitation estimations in cool seasons and for the identification of
potential icing hazards for aviation.
11
Figure7: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC on a PPI scope at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KCLE at the 16:24:29 UTC on 26 January 2012.
d). High RhoHVs in severe AP phenomena
Severe AP is occasionally associated with highly correlated power returns from
horizontal and vertical directions, where RhoHV values can be 0.95 or higher
(Figure 8(b)). The application of a simple RhoHV filter is not able to remove AP in
these situations. To address this issue, the RhoHV filter is followed by a vertical
gradient check (Zhang et al. 2004), which checks the vertical structure of the
echoes. If the strong power returns are only observed in the lowest tilt without
a consistency in the vertical, then the echoes are most likely associated with non-
precipitation scatterers. The vertical gradient test is dependent on the volume
coverage pattern (VCP) and the severity of the anomalous propagation. It is not
effective when the beam is so severely super refracted that large areas of strong
AP echoes are observed on the 2nd (3rd for the VCP 12 or 212) or even higher tilts.
Further, it cannot be applied at very far ranges because the height difference
between the lowest two tilts is too large, and a shallow precipitation cloud may only
be observed in the first tilt. Figure 8 provides an example where the gradient check
successfully censored AP echoes associated with high RhoHV values.
12
Figure8: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC at the elevation angle of 0.5 degrees. The precipitation event is observed from radar KOUN at the 02:08:28 UTC on 11 May 2010. The white-circled areas outline the region of severe AP.
e). High RhoHVs in weak echoes
When the signal-to-noise (SNR) ratio is low, RhoHV is often noisy in appearance
with varying magnitudes. RhoHV values may be greater than 1 for the weak signals
associated with large noise, according to estimate calculations of the co-polar
correlation coefficient (Melnikov and Zrnić, 2007). The biased high value RhoHV
(>1) appears in both meteorological and non-meteorological areas when the signal
to noise ratio is low (Figure 9(b)). To clean the clutter left from the basic RhoHV
filtering, further steps are required to differentiate the biased RhoHV of clutter
contamination from the precipitation echoes. The gradient of echo power along the
vertical height is computed for the data bins of biased RhoHV in the regions close
to the radar at low tilts. Vertical discontinuity is considered a characteristic of the
clutter. At far ranges or high tilts, the pixels of biased RhoHV are first segmented
into groups. The groups located closer to clutter areas (0<RhoHV<0.80) than
precipitation regions (0.95<RhoHV<1.0) are identified as clutter and removed.
Otherwise, they are considered as weak signal from the meteorological targets
(Figure 9 (c)) and are retained.
13
Figure9: The (a) raw reflectivity (b) RhoHV and (c) reflectivity field after QC at the elevation angle of 0.5 degrees. The data is obtained from radar KTLX at the 12:31:51 UTC on 08 March 2013.
f). The clutter at far ranges from radar site and other limitations of RhoHV
Due to the range difference between the polarimetric variables such as RhoHV
(298 km) and reflectivity field (460 km) of the WSR-88D radars, the RhoHV filter is
not able to handle the clutter further than 298 km away from the radars. To address
this issue, a simple image processing technique is used to remove sun strobes and
electronic interferences at ranges greater than 298 km. Different from others,
these two types of echoes often appear as isolated radial spikes originating from
the radar site and ending at the edge of the coverage. When portions of such spikes
are removed within the RhoHV coverage umbrella (due to low RhoHV values), the
rest of the spikes are also removed [see Figure 10 (a, b, c)]. This technique is not
effective on the AP ground or sea clutter at far distances (see Figure 10 (d, e, f)) due
to the fact that those clutter lack of the specific spike geometry. AP at far ranges
does not pose a significant problem for the inland WSR-88D radars because they
are relatively densely distributed with an average spacing of less than 250km. As a
result, inland data beyond 298km are not used in NMQ. Identification of AP echoes
beyond 298km for coastal radars remains an issue and requires additional data
sources (e.g., satellite).
14
Figure10: The (a) (d) raw reflectivity (b) (e) RhoHV and (c) (f) reflectivity field after QC at the elevation angle of 0.5 degrees. The fields are observed from radar KICX (a, b, c) at the 01:31:47 UTC on 11 Mar. 2013 and KJAX (d, e, f) at the 08:43:14 UTC on 23 September 2012.
2.3 Overview of the dpQC algorithm
Based on the detailed case studies presented above, a physically based multi-
sensor dpQC algorithm was assembled consisting of the following seven steps:
2.3.1 Clear air filter
The clear air filter is the first step for the clutter screening in the dpQC
algorithm. When the surface temperature is high (>10°C) and the radar uses a clear
air mode (i.e., VCP 31 or 32), all echoes in the whole volume scan are removed. This
simple step effectively removes biological echoes in the warm season and largely
reduces computations down stream.
2.3.2 Basic RhoHV filter
15
In the basic RhoHV filter, four different thresholds are applied according
to various scenarios predefined through reflectivity intensities and echo top
heights. The filter threshold is automatically adjusted for the situations when low
correlation coefficient values are found in the precipitation events, as discussed in
sections 2.2(a-c).
2.3.3 RhoHV texture filter
The RhoHV texture filter follows to censor weak echoes associated with noisy
local RhoHV texture. The estimate of the RhoHV texture is obtained by averaging
RhoHV data along the radial using a 1-km running-average window and subtracting
the smoothed estimates of RhoHV from the original values. The root mean square
value of the residuals is calculated as the local RhoHV texture (Park et al. 2009). The
texture characterizes the magnitude of the small-scale fluctuations of RhoHV along
the radial. The echo is censored as clutter when its RhoHV texture is larger than a
threshold. After two steps of Basic RhoHV and RhoHV texture filtering, most of non-
weather echoes with the RhoHV less than 0.90 are removed, and the clutter pixels
with high RhoHV values (i.e., those in overlapping region of the histograms in Figure
4) are further processed.
2.3.4 Spike filter
The spike filter was designed to handle spikes from electronic interferences and
sun strobes that extend beyond the maximum range of the RhoHV field (298 km), as
discussed in section 2.2(f).
2.3.5 Speckle filter
A speckle filter follows to remove isolated dots of echoes in the reflectivity fields.
2.3.6 Spatial analyze
After the speckle filter, a couple of spatial analyses are performed to handle the
non-weather echoes with high RhoHV values [i.e., those discussed in sections 2.2(d-
e)]. In regions close to the radar, a vertical gradient test is employed to check the
vertical continuity of any strong echoes that exist in the lowest tilts. If the echo
intensities decrease with height dramatically, then they are identified as clutter (AP)
. In regions far away from the radar, high RhoHV bins with low SNRs are classified
16
as weather or non-weather depending on their proximities to the nearest storm or
clutter centers.
2.3.6 Final clean up
In the final clean up, a median filter is applied to remove residual speckles and to
fill in small voids within weather echoes where sparse low RhoHV values are found
among blocks of meteorological targets.
In the dpQC algorithm, each step can remove clutter echoes when the stringent
criteria are met, leaving less data to process for the next step. This design effectively
increased the computational efficiency. The environmental data is used to remove
clear air and biological echoes in warm season and to help delineating areas of
mixed hydrometeor phases in the basic RhoHV filter. Figure 11 illustrates an
overview flowchart of the dpQC algorithm.
17
Figure11: Flow chart of the dpQC algorithm.
3. Evaluation of the dpQC Methodology
The dpQC algorithm was developed across 25 cases that represented various
weather and non-weather echo regimes from different regions of the US. There
were total of 138-hour of data from 29 polarimetric WSR-88D radars, as shown in
Table 1. The designed method is tested and evaluated in its QC performance and
computational resource.
Table 1: List of cases used for the development and tuning of dpQC.
18
Events Radar Time and date Event summary1 KCCX
(40.92ON, 78.00OW)0600-1100 UTC 11 Dec 2012 Winter storm (weak echoes)
2 KFFC (33.36ON, 84.57OW)
1100-1400 UTC 16 Nov 2012 Winter storm (weak echoes)
3 KJGX (32.68ON, 83.35OW)
1600-2200 UTC 17 Sep 2012 BI + convective
4 KMPX (44.85ON, 93.57OW)
0000-0200 UTC 10 Dec 2012 Winter storm (weak echoes)
5 KMHX (34.78ON, 76.88OW)
1800-2300 UTC 29 Jun 2012 AP + sea clutter
6 KOUN (35.24ON, 97.46OW)
2000 UTC 10 May 2012 –0800 UTC 11 May 2012
BI + AP + convective
7 KAMA (35.23 ON, 101.7 OW)
0100-0500 UTC 1 May 2012 BI + AP + hails + NBF
KVNX (36.74 ON, 98.13 OW)KDDC (37.76 ON, 99.97 OW)
BI + ground clutter (WF) + Hail + NBF
KICT (37.65 ON, 97.44 OW)
BI + Hail + NBF
8 KAMX (25.61 ON, 80.41 OW)
1600-UTC 18 May 20120800 UTC 19 May 2012
Chaff + convective
KBYX (24.60 ON, 81.70 OW)
Chaff + convective
9 KARX (43.83 ON, 91.19 OW)
1700-2100 UTC 26 Jun 2012 BI + stratiform
10 KCLE (41.41 ON, 81.86 OW)
1400 UTC 07 Feb 2012-0500 UTC 08 Feb 2012
Winter storm (weak echoes) + EI
11 KEWX (29.70 ON, 98.03 OW)
0200-1000 06 May 2012 BI + AP + EI + convective
12 KHTX (34.93 ON, 86.08 OW)
0000-0200 UTC 11 Aug 2012 BI + Hail + NBF
13 KLTX (33.99 ON, 78.43 OW)
1500-1700 UTC 17 Sep 2012 BI + convective
14 KMRX (36.17 ON, 83.40 OW)
2100-2300 UTC 17 Sep 2012 BI + Hail
15 KPBZ (40.53 ON, 80.22 OW)
1100-2300 UTC 11 Feb 2012 Winter storm (weak echoes)
16 KPDT (45.69 ON, 118.9 OW)
0900-1100 UTC 20 Jul 2012 AP + stratiform
17 KYUX (32.50 ON, 114.7 OW)
0900-1100 UTC 20 Jul 2012 AP + EI
18 KABR (45.46 ON, 98.41 OW)
0800 UTC 11 Aug 2012 Ground clutter + stratiform
19 KFSD (43.59 ON, 96.73 OW)
0000 UTC 04 Aug 2012 Hail + NBF
19
20 KJAX (30.48 ON, 81.70 OW)
0800 UTC 23 Sep 2012 BI + sea clutter
21 KNQA (35.34 ON, 89.87 OW)
1000 UTC 21 Sep 2012 AP + stratiform
22 KLOT (41.60 ON, 88.08 OW)
2000 UTC 04 Aug 2012 Hail + convective
23 KRLX (38.31 ON, 81.72 OW)
0800 UTC 01 Oct 2012 BI + AP +stratiform
24 KSGF (37.24 ON, 93.40 OW)
0600 UTC 29 Feb 2012 Hail + NBF
25 KRIW (43.07 ON, 108.5 OW)
1300 UTC 25 May 2012 Ground clutter + stratiform
Total 29 radars 138 h 25 events
Note: the abbreviations in the table are AP: anomalous propagation; EI: electromagnetic interference; BI: biological clutter; NBF: Non-uniform beam filling; WF: wind farm.
The dpQC algorithm was implemented on 122 radars in real-time in the NMQ
system (http://nmq.ou.edu/qvs-2012.html) since 19:25:00 UTC on 05 Dec 2012.
Figure 12 shows several examples (e.g., bloom, light precipitation, chaff) of the real-
time national composite reflectivity fields before and after the dpQC. In Figure
12 (a) and (b), the power returns from non-meteorological chaff (outlined with
white circles) cannot be censored with the single-polarization radar QC due to their
similarities to precipitation echoes in the reflectivity field. On the other hand, the
aluminum-coated glass fibers (chaff) are not as correlated in the horizontal and
vertical directions as meteorological objects, therefore the dpQC was able to identify
and remove the clutter (Figure 12(c)). Real time tests also showed the advantages
of the dpQC over the single-polarization QC in removing blooms while retaining
light precipitation (regions outlined with the white ovals in Figs. 12d, e, and f).
Such advantages are a result of distinctly different RhoHV features associated with
blooms (biological echoes) and with light precipitation.
20
Figure12: (a), (d) the raw composite reflectivity before any process of quality control; (b), (e) the composite reflectivity after the quality control of QCNN that is currently used in the NMQ system; (c), (f) the composite reflectivity field after process of dpQC. The images are cropped from the real-time NMQ system.
The dpQC and dpQCNN (Lakshmanan et al. 2013) algorithms were evaluated
on a set of independent cases (Table 2) and the two algorithms performed very
similarly with the HSS scores of 0.83 and 0.79, respectively. However, due to the
simplicity of the dpQC algorithm, it took much less computational resources when
compared with the more sophisticated dpQCNN scheme. Based on the test of
282 radar volume scans on a workstation with 4 quad-core Intel® Xeon® E5520
2.27GHz processors and 12GB DDR3 RAM, the average CPU for processing one radar
volume scan is 49.55s for dpQCNN and 3.15s for dpQC and the average RAM was
172MB for dpQCNN and 83MB for dpQC.
Table 2: Testing cases for evaluating the QC algorithm.
Events Radar Time and date Event summary1 KMHX (34.78ON, 76.88OW) 01 UTC 27 Aug 2011 Hurricane2 KMHX (34.78ON, 76.88OW) 19 UTC 27 Aug 2011 Hurricane3 KOUN (35.24ON, 97.46OW) 05 UTC 11 May 2010 AP + BI4 KOUN (35.24ON, 97.46OW) 11 UTC 19 May 2010 Supercell + BI5 KAMA (35.23 ON, 101.7 OW) 14 UTC 11 Apr 2012 Rain + BI6 KAMA (35.23 ON, 101.7 OW) 12 UTC 12 Apr 2012 Rain + BI7 KAMA (35.23 ON, 101.7 OW) 18 UTC 15 Feb 2013 Rain+ GC
21
8 KVNX (36.74 ON, 98.13 OW) 09 UTC 20 May 2012 Storm + BI9 KVNX (36.74 ON, 98.13 OW) 23 UTC 29 May 2012 ST+BI
10 KVNX (36.74 ON, 98.13 OW) 11 UTC 29 Jan 2013 Storms11 KBMX (33.17 ON, 86.77 OW) 11 UTC 30 Jan 2013 Storms12 KCLE (41.41 ON, 81.86 OW) 17 UTC 15 Feb 2013 Snow13 KDMX (41.73 ON, 93.72 OW) 03 UTC 29 Jan 2013 Convective14 KLVX (37.98 ON, 85.94 OW) 16 UTC 15 Feb 2013 Snow15 KTLH (30.40 ON, 84.33 OW) 11 UTC 26 Feb 2013 Squall line and clutter16 KTLX (35.33 ON, 97.28 OW) 23 UTC 09 Mar 2013 Squall line and clutter
Note: the abbreviations in the table are AP: anomalous propagation; BI: biological clutter; ST: sun strobe; GC: ground clutter.
Table 3: Summary of computational testing.
VCP Radar and Location Time and date
(UTC)
dpQCNN dpQCCPU
(Min: sec)RAM(MB)
CPU(Min: sec)
RAM(MB)
12 KBBX(39.50 ON, 121.63 OW)
1200-1500 04 Apr 2013
47:37 216 2:22 79
212 KBMX(33.17 ON, 86.77 OW)
1700-2000 24 Apr 2013
39:58 211 2:45 105
31 KBUF(42.95 ON, 78.74 OW)
1000-1300 13 Apr 2013
5:36 119 0:37 77
121 KCLE(41.41 ON, 81.86 OW)
0900-1200 19 Apr 2013
22:32 157 1:39 77
11 KCLX(32.66 ON, 81.04 OW)
1700-2000 04 Apr 2013
34:37 210 1:56 88
32 KFWS(32.57 ON, 97.30 OW)
0100-0400 15 Apr 2013
6:26 114 0:38 75
21 KLOT(41.60ON, 88.08 OW)
1000-1300 08 Apr 2013
22:08 157 1:49 86
221 KLRX(41.60ON, 88.08 OW)
0300-0600 20 Apr 2013
16:51 156 1:07 77
211 KTLX(35.33ON, 97.28 OW)
0700-1000 13 Apr 2013
37:09 205 1:54 84
Total 282 vol scans
232:54 14:47
Average Per volume scan
49.55s 172 3.15s 83
4. Conclusions
A multi-sensor physically based weather/non-weather echo classifier was
developed using polarimetric radar observations and atmospheric environmental
22
data. The algorithm applied a set of explicit meteorological principles that segregate
weather and non-weather echoes in the radar reflectivity field based on correlation
coefficient and 3D reflectivity and temperature structure. The new algorithm was
evaluated using 16 independent events and showed a high accuracy (Heidke Skill
Score of 0.83) in segregating weather and non-weather echoes. When compared
with a more sophisticated QC algorithm that uses all polarimetric variables and a
neural network approach, the dpQC showed a similar HSS score (0.83 vs. 0.79), but
with much less CPU (3.15 vs. 49.55 s) and RAM (83 vs. 172 MB) usages for process
a volume scan data on average. Due to the simplicity and transparency of the
algorithm, it is easy to implement and maintain in an operational environment with
a large national radar network.
5. References
Andrieu, H. and J.-D. Creutin, 1995: Identification of vertical profiles of radar reflectivity for hydrological applications using an inverse method. Part I: Formulation. J. Appl. Meteorol. 34, 225– 239.
Bachmann, S.M., and M. D. Tracy, 2009: Data driven adaptive identification and suppression of ground clutter for weather radar, 25th Conference on International Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, 11.B3: Phoenix, AZ.
Bebbington, D., S. Rae, J. Bech, B. Codina, and M. Picanyol, 2007: Modeling of weather radar echoes from anomalous propagation using a hybrid parabolic equation method and NWP model data. Natural Hazards and Earth System Sciences, Vol. 7, pp.391–398, ISSN 1561-8633
Carbone, R. E., and L. D. Nelson, 1978: The evolution of raindrop spectra in warm-based convective storms as observed and numerically modeled. J. Atmos. Sci., 35, 2302–2314.
Germann, U, and J. Joss, 2002: Mesobeta profiles to extrapolate radar precipitation measurements above the Alps to the ground level. J. Appl. Meteorol. 41, 542–557.
Gourley, J. J., P. Tabary, and J. Parent-du-Chatelet, 2007: A fuzzy logic algorithm for the separation of precipitating from nonprecipitating echoes using polarimetric radar observations. J. Atmos. Oceanic Technol., Vol. 24, pp. 1439–1451, ISSN 1520-0426
Gray, W., and H. Larsen, 2005, Radar rainfall estimation in the new zealand context, Atmos. Sci. Let. 6: 31-34.
23
Kitchen, M., R. Brown, and A. G. Davies, 1994: Real- time correction of weather radar data for the effects of bright band, range and orographic growth in widespread precipitation. Quart. J. Roy. Meteor. Soc., 120, 1231–1254.
Koistinen, J., 1991: Operational correction of radar rainfall errors due to the vertical reflectivity profile. Preprints, 25th Int. Conf. on Radar Meteorology, Paris, France, Amer. Meteor. Soc., 91–94.
Krajewski, W. F., B. Vignal, 2001: Evaluation of Anomalous Propagation Echo Detection in WSR-88D Data: A Large Sample Case Study. J. Atmos. Oceanic Technol., 18, 807–814.
Lakshmanan, V., A. Fritz, T. Smith, K. Hondl, and G. J. Stumpf, 2007: An automated technique to quality control radar reflectivity data. J. Appl. Meteor. Climatol., 46, 288–305.
Lakshmanan, V., J. Zhang, and K. Howard, 2010: A technique to censor biological echoes in weather radar images. J. Appl. Meteor. Climatol., 49, 453–462.
Lakshmanan, V., C. Christopher, J. Krause, and L. Tang 2013: Quality control of weather radar data using polarimetric variables. J. Atoms Oceanic Technol., submitted.
Lim, S., V. Chandrasekar, and V. N. Bringi, 2005: Hydrometeor classification system using dual-polarization radar measurements: Model improvements and in situ verification. IEEE Trans. Geosci. Remote Sens., 43, 792–801.
Liu, H., and V. Chandrasekar, 2000: Classification of hydrometeors based on polarimetric radar measurements: Development of fuzzy logic and neuro-fuzzy systems, and in situ verification. J. Atmos. Oceanic Technol., 17, 140–164.
Marshall, J. S., W. McK. Palmer, The distributions of raindrop with size, J. Meteorol., 5, 165–166, 1948.
Marzano, F., D. Scaranari, M. Montopoli, and G. Vulpiani, 2008: Supervised classification and estimation of hydrometeors from C-band dual-polarized radars: A Bayesian approach. IEEE Trans. Geosci. Remote Sens., 46, 85–98.
Melnikov, V. M., D. S. Zrnić, 2007: Autocorrelation and Cross-Correlation Estimators of Polarimetric Variables. J. Atmos. Oceanic Technol., 24, 1337–1350.
Park, H. S., A. V. Ryzhkov, D. S. Zrnic, and K.-E. Kim, 2009: The hydrometeor classification algorithm for the polarimetric WSR-88D: Description and application to an MCS. Wea. Forecasting, 24, 730–748.
Rosenfeld, D., D. B. Wolff, and D. Atlas, 1993: General probability-matched relations between radar reflectivity and rain rate. J. Appl. Meteor., 32, 50–72.
Smyth T.J., A. J. Illingworth, 1998: Radar estimates of rainfall rates at the ground in bright band and non-bright band events. Quart J Royal Meteorol Soc 124:2417–2434
24
Tu, J. V., 1996: Advantage and disadvantage of using artificial neural networks versus logistic regression for predicting medial outcomes, J Clin Epidemiol ,49, No. 11, pp. 1225-1231, 1996
White, A. B., P. J. Neiman, F. M. Ralph, D. E. Kingsmill, and P. O. G. Persson, 2003: Coastal orographic rainfall processes observed by radar during the California Land-Falling Jets Experiment. J. Hydrometeor., 4, 264–282.
Zhang, J., S. Wang, and B. Clarke, 2004: WSR-88D reflectivity quality control using horizontal and vertical reflectivity structure. 11th Conf. on Aviation, Range, and Aerospace Meteorology, 5.4, Hyannis, MA.
Zhang, J, and Coauthors, 2011: National Mosaic and Multi-sensor QPE (NMQ) system: Description, results, and future plans. Bull. Amer. Meteor. Soc., 92, 1321–1338.
Zrnic, D. S., and A. V. Ryzhkov , J. M. Straka, Y. Liu, and J. Vivekanandan, 2001: Testing a procedure for the automatic classification of hydrometeor types. J. Atmos. Oceanic Technol., 18, 892–913.
25