SHEF* Primer *Standard Hydrometeorological Exchange Format

SHEF* PrimerSHEF* Primer*Standard Hydrometeorological Exchange *Standard Hydrometeorological Exchange

FormatFormat

Mike CallahanMike Callahan

WFO LouisvilleWFO Louisville

September 29, 2009September 29, 2009

ObjectivesObjectives

Know the three different types of SHEF Know the three different types of SHEF reports and when they are usedreports and when they are used

Understand the various elements in a Understand the various elements in a SHEF reportSHEF report

Understand the different descriptors in Understand the different descriptors in a SHEF data elementa SHEF data element

Know how to encode/decode SHEF Know how to encode/decode SHEF reportsreports

The SHEF ManualThe SHEF Manual

The SHEF Manual Version 2.2 (NWSM 10-The SHEF Manual Version 2.2 (NWSM 10-944) is the ultimate source of all 944) is the ultimate source of all information about SHEFinformation about SHEF

Every office should have a copy of this Every office should have a copy of this manualmanual

It is available online in PDF format at It is available online in PDF format at www.nws.noaa.gov/directiveswww.nws.noaa.gov/directives

This primer refers to it constantly, get it This primer refers to it constantly, get it and be prepared to use itand be prepared to use it

The Three Type of ReportsThe Three Type of Reports

The .E format is used to encode an The .E format is used to encode an EVENLY spaced sequence of data EVENLY spaced sequence of data from a SINGLE locationfrom a SINGLE location

The .B format is used to encode a The .B format is used to encode a collective of SIMILAR data from a collective of SIMILAR data from a GROUP of locationsGROUP of locations

The .A format is used for everything The .A format is used for everything elseelse

A Simple ExampleA Simple ExampleWe want to encode this data:We want to encode this data:

The river gage reading at Deputy IN was The river gage reading at Deputy IN was 13.45 at 7:00 AM EST on 4/1/200913.45 at 7:00 AM EST on 4/1/2009

Since we have a single piece of data for Since we have a single piece of data for one location we will use the .A format.one location we will use the .A format.

We need:We need: The location ID: The location ID: DEPI3DEPI3 The date and time of the data: The date and time of the data: 20090401 E 20090401 E DH0700DH0700

The type and value of the data: The type and value of the data: HG 13.45HG 13.45

.A Reports.A Reports

.A reports are the simplest SHEF report.A reports are the simplest SHEF report These reports can encode any type of data but may These reports can encode any type of data but may

not be the most efficient choicenot be the most efficient choice Use for a set of non-repeating or random sequence of Use for a set of non-repeating or random sequence of

data for a single locationdata for a single location Split into two sections, positional fields and a data Split into two sections, positional fields and a data

stringstring Using the data from the previous slide, lets generate Using the data from the previous slide, lets generate

our first SHEF report:our first SHEF report:

.A DEPI3 20090401 E DH07/HG 13.45positional fields data string

.A Positional Fields.A Positional Fields

.A DEPI3 20090401 E

Format Specifier

Location ID

Date

Time Zone

Format SpecifierFormat Specifier

Specifies the report typeSpecifies the report type Only choices are Only choices are .A.A, , .B.B, or , or .E.E Optionally followed by an Optionally followed by an RR for a revision or a for a revision or a

number for a continuationnumber for a continuation Data stored in the AWIPS database as Data stored in the AWIPS database as

missing will be overwritten regardless of the missing will be overwritten regardless of the revision coderevision code

The revision code must be present to The revision code must be present to overwrite data non-missing already in the overwrite data non-missing already in the AWIPS databaseAWIPS database

When revising, make sure the date and time When revising, make sure the date and time of the new data exactly matches the old dataof the new data exactly matches the old data

Location IDLocation ID

An approved ID listed in the NWSLI An approved ID listed in the NWSLI databasedatabase

It is possible to encode data from non-ID It is possible to encode data from non-ID locations using stranger reportslocations using stranger reports Stranger reports use the latitude and longitude Stranger reports use the latitude and longitude

as an IDas an ID XlatlongXlatlong: North latitude, West longitude: North latitude, West longitude

Ex. Ex. X4511001X4511001 = 45.1 N latitude, 100.1 W longitude = 45.1 N latitude, 100.1 W longitude Ex. Ex. X3090988X3090988 = 30.9 N latitude, 98.8 W longitude = 30.9 N latitude, 98.8 W longitude

Be sure to include the leading zero for longitudes less Be sure to include the leading zero for longitudes less than 100 degreesthan 100 degrees

YlatlongYlatlong: North latitude, East longitude: North latitude, East longitude Ex. Ex. Y5030202Y5030202 = 50.3 N latitude, 20.2 E longitude = 50.3 N latitude, 20.2 E longitude

DateDate

See SHEF Manual, Section 4.1.4See SHEF Manual, Section 4.1.4 mmdd:mmdd: Month and Day Month and Day

If year is missing, the decoder will look 6 months in the If year is missing, the decoder will look 6 months in the future and 6 months in the past to assign the year future and 6 months in the past to assign the year nearest to the current datenearest to the current date

Ex: Ex: 0401, 12250401, 1225 yymmdd:yymmdd: Year, month, and day Year, month, and day

If a two digit year is used, the decoder will look 10 years If a two digit year is used, the decoder will look 10 years in the future and 90 years in the past to assign the in the future and 90 years in the past to assign the centurycentury

Ex: Ex: 090915, 990210090915, 990210 yyyymmdd:yyyymmdd: 4 digit year, month, and day 4 digit year, month, and day

Ex: Ex: 20091010, 1995041520091010, 19950415

Time ZoneTime Zone

See SHEF manual, Table 8See SHEF manual, Table 8 Common codes are:Common codes are:

Z:Z: UTC (GMT) UTC (GMT) defaultdefault E:E: Eastern Local Time (follows shift between standard Eastern Local Time (follows shift between standard

and daylight saving time)and daylight saving time) C:C: Central Local Time Central Local Time M:M: Mountain Local Time Mountain Local Time P:P: Pacific Local Time Pacific Local Time L:L: Alaskan Local Time Alaskan Local Time H:H: Hawaiian Local Time Hawaiian Local Time Followed by a Followed by a DD to force daylight time or to force daylight time or SS to force to force

standard timestandard time

Quick Quiz: .A Positional Quick Quiz: .A Positional FieldsFields

Grab a pen and write down your answersGrab a pen and write down your answers

1. List the elements of the positional 1. List the elements of the positional fieldsfields

2. What format specifier is used to 2. What format specifier is used to correct a previous .A reportcorrect a previous .A report

3. Encode the date for July 4, 20093. Encode the date for July 4, 2009 4. Encode Mountain Daylight Time4. Encode Mountain Daylight Time

Answers to .A Positional Answers to .A Positional FieldsFields

1. Format specifier, Location ID, 1. Format specifier, Location ID, Date, Time ZoneDate, Time Zone

2. 2. .AR.AR 3. 3. 2009070420090704 or or 090704090704 4. 4. MDMD

Data StringData StringDH07/HG 13.45

Time

Parameter Code

Value

All elements in the data string must be All elements in the data string must be separated by slashes (/) expect for the separated by slashes (/) expect for the parameter code and data pairparameter code and data pair

TimeTime

See SHEF Manual, Table 9 See SHEF Manual, Table 9 DefaultDefault is is 12:00:0012:00:00 for Z time zone, for Z time zone, 00:00:0000:00:00 for for

all others, all others, missing time defaults are rarely used missing time defaults are rarely used and could lead to some confusion and could lead to some confusion

DHhhmmss DHhhmmss == Data hour, minutes, seconds Data hour, minutes, seconds mmmm and and ssss defaultdefault to to 0000

DDddhhmmss DDddhhmmss == Data day, hours, minutes, seconds Data day, hours, minutes, seconds Use this if the day is not the same as the day in the Use this if the day is not the same as the day in the

positional codepositional code DMmmddhhmmss DMmmddhhmmss == Data month, day, hours, Data month, day, hours,

minutes, secondminutes, second DYyyyymmddhhmmss DYyyyymmddhhmmss == Data year, month, day, Data year, month, day,

hours, minutes, secondhours, minutes, second

Time RelativeTime Relative

DRtDRt±±xxxx Adds or subtracts Adds or subtracts xxxx units of time units of time tt

to/from the last to/from the last explicitexplicit time time t valuest values

S:S: seconds, seconds, N:N: minutes, minutes, H:H: hours, hours, D:D: days, days, M:M: months, months, Y:Y: years, years, E:E: end of month end of month

Ex:Ex: DRN+60 DRN+60 oror DRH+1 DRH+1 = add an hour= add an hour DRD-1DRD-1 = subtract one day = subtract one day

Date Relative End of MonthDate Relative End of Month

DREDRE±±xx Allows you to encode end of the month values Allows you to encode end of the month values

without knowing how many days are in the monthwithout knowing how many days are in the month x:x: the number of months the number of months DRE+1DRE+1 = the last day of the next month = the last day of the next month DRE+2DRE+2 = the last day of the next month = the last day of the next month Ex: encode spring (Mar., Apr., May) monthly Ex: encode spring (Mar., Apr., May) monthly

precipitation for ALTP1precipitation for ALTP1 .A ALTP1 0331 E DH07/PPM 4.46/DRE+1 .A1 .A ALTP1 0331 E DH07/PPM 4.46/DRE+1 .A1 3.56/DRE+2/6.803.56/DRE+2/6.80

Parameter CodesParameter Codes All codes except All codes except PEPE have a default have a default

PEDTSEP

Physical Element

Duration

Type

Source

Extremum

Probability

Physical ElementPhysical Element See SHEF Manual, Table 1 & Appendix GSee SHEF Manual, Table 1 & Appendix G First Letter Category, Second Letter Type, First Letter Category, Second Letter Type,

no defaultno default Categories:Categories:

A-CA-C: agricultural, : agricultural, EE: evaporation, : evaporation, FF: fish,: fish, GG: ground, : ground, HH: height, : height, II: ice, : ice, LL: lake,: lake, MM: moisture & fire/fuel, : moisture & fire/fuel, NN: dam & gate,: dam & gate, PP: pressure & precipitation, : pressure & precipitation, QQ: discharge,: discharge, RR: radiation, : radiation, SS: snow, : snow, TT: temperature,: temperature, UU: wind, : wind, VV: electrical, : electrical, WW: water quality,: water quality, XX: weather, : weather, YY: other : other

Physical Element (con’t)Physical Element (con’t)

Some common codesSome common codes HG:HG: river stage height river stage height HP:HP: pool elevation pool elevation HT:HT: tailwater elevation tailwater elevation PC:PC: accumulated precipitation accumulated precipitation PP:PP: incremental precipitation incremental precipitation SD:SD: snow depth snow depth SF:SF: snowfall snowfall SW:SW: snow water equivalent snow water equivalent QT:QT: total discharge total discharge TA:TA: air temperature air temperature

Special Physical ElementsSpecial Physical Elements Some physical elements are complex and have Some physical elements are complex and have

their own tables in the SHEF Manualtheir own tables in the SHEF Manual IR:IR: ice report, Table 14 ice report, Table 14 SR:SR: snow report, Table 15 snow report, Table 15 GR:GR: frost report, Table 16 frost report, Table 16 PT:PT: precipitation type, Table 17 precipitation type, Table 17 GS:GS: ground state, Table 18 ground state, Table 18 HI:HI: river trend, Table 19 river trend, Table 19 AF:AF: surface frost intensity, Table 20 surface frost intensity, Table 20 AM:AM: surface dew intensity, Table 21 surface dew intensity, Table 21 PM:PM: precipitation probability, Table 22 precipitation probability, Table 22 XW:XW: present weather, Appendix C present weather, Appendix C XP:XP: past weather, Appendix D past weather, Appendix D

DurationDuration See SHEF Manual, Table 3See SHEF Manual, Table 3 Some common codes:Some common codes:

I:I: instantaneous, instantaneous, defaultdefault for most PE codes for most PE codes D:D: 1 day, 1 day, defaultdefault for for PPPP, , SFSF, & others, see Table 7, & others, see Table 7 E:E: 5 minutes 5 minutes G:G: 10 minutes 10 minutes C:C: 15 minutes 15 minutes H:H: 1 hour 1 hour T:T: 3 hours 3 hours Q:Q: 6 hours 6 hours K:K: 12 hours 12 hours P:P: total since the previous 7AM total since the previous 7AM V:V: other, duration defined using other, duration defined using DVDV code code

Special DurationsSpecial Durations

If the duration you want is not listed in If the duration you want is not listed in Table 3, use the Table 3, use the DVtxxDVtxx code: code:

tt values values SS: seconds, : seconds, NN: minutes, : minutes, HH: hours, : hours, DD: days,: days, MM: months, : months, YY: years: years ZZ: reset duration back to default: reset duration back to default

Ex:Ex: DVH2/SFVDVH2/SFV = 2 hour snowfall = 2 hour snowfall DVN45/PPVDVN45/PPV = 45 minute precipitation = 45 minute precipitation

Be Careful with P DurationsBe Careful with P Durations

C DH07/PPP 0.35C DH07/PPP 0.35 24 hours of precipitation, 7:00AM yesterday to 7:00AM today24 hours of precipitation, 7:00AM yesterday to 7:00AM today

C DH06/PPP 0.50C DH06/PPP 0.50 23 hours of precipitation, 7:00AM yesterday to 6:00AM today23 hours of precipitation, 7:00AM yesterday to 6:00AM today

C DH0730/PPP 1.20C DH0730/PPP 1.20 30 minutes of precipitation, 7:00AM today to 7:30AM today!30 minutes of precipitation, 7:00AM today to 7:30AM today!

C DH0730/PPD 1.20C DH0730/PPD 1.20 24 hours of precipitation, 7:30AM yesterday to 7:30AM 24 hours of precipitation, 7:30AM yesterday to 7:30AM

today.today. PPPPPP is can be used for non-7AM observer reports if is can be used for non-7AM observer reports if

the observer dumps the gage at 7AMthe observer dumps the gage at 7AM Use Use PPDPPD for morning reports for morning reports

TypeType

See SHEF Manual, Table 4See SHEF Manual, Table 4 R:R: Reading (observed), Reading (observed), defaultdefault C:C: Contingency Contingency F:F: Forecast Forecast H:H: Historical Historical M:M: Model Model P:P: Non-forecast Processed Non-forecast Processed

SourceSource

Meaning changes with Type code, see Meaning changes with Type code, see SHEF Manual, Table 4SHEF Manual, Table 4

Some common Type & Source code pairsSome common Type & Source code pairs RZ:RZ: non-specified (observer) reading, non-specified (observer) reading, defaultdefault RG:RG: DCP reading DCP reading RP:RP: phone reading (LARC, CR10, etc.) phone reading (LARC, CR10, etc.) RR:RR: radio reading (ALERT, IFLOWS, etc.) radio reading (ALERT, IFLOWS, etc.) FZ:FZ: non-specified forecast, default for F type non-specified forecast, default for F type FF:FF: forecast including QPF forecast including QPF

ExtremumExtremum

See SHEF Manual, Table 5See SHEF Manual, Table 5 Some common codesSome common codes

Z:Z: no extremum, no extremum, defaultdefault N:N: minimum of day minimum of day H:H: minimum of 6 hours minimum of 6 hours P:P: minimum of 12 hours minimum of 12 hours X:X: maximum of day maximum of day R:R: maximum of 6 hours maximum of 6 hours Y:Y: maximum of 12 hours maximum of 12 hours

ProbabilityProbability

See SHEF Manual, Table 6See SHEF Manual, Table 6 Some common codesSome common codes

Z:Z: no probability, no probability, defaultdefault A:A: .2% .2% 1:1: 10% 10% G:G: 25% 25% 3:3: 30% 30% 5:5: 50% 50% H:H: 75% 75% 8:8: 80% 80% Y:Y: 99.8% 99.8% M:M: mean expected value mean expected value

Send CodesSend Codes

Many parameter code combinations are so Many parameter code combinations are so common that a new PE has been createdcommon that a new PE has been created

See SHEF Manual, Table 2See SHEF Manual, Table 2 Some common codesSome common codes

TNTN TAIRZNZTAIRZNZ = = minimum air temperature of the dayminimum air temperature of the day TXTX TAIRZXZTAIRZXZ = = maximum air temperature of the maximum air temperature of the

dayday HNHN HGIRZNZHGIRZNZ = = minimum gage height of the dayminimum gage height of the day HXHX HGIRZXZHGIRZXZ = = maximum gage height of the daymaximum gage height of the day PFPF PPTCFZZPPTCFZZ = = 3 hour flash flood guidance3 hour flash flood guidance SFSF SFDRZZZSFDRZZZ = 24 = 24 hour snowfallhour snowfall

ValuesValues The data valueThe data value

Trace amounts are encoded as Trace amounts are encoded as TT or or tt, stored , stored as .001as .001

Missing data are encoded as Missing data are encoded as MM, , mm, , MMMM, , mmmm, , ++, , --, or , or -9999-9999

For precipitation data: values missing a decimal For precipitation data: values missing a decimal will be divided by 100, thus will be divided by 100, thus PP 1PP 1 is stored as 0.01! is stored as 0.01! This feature is obscure and may lead to confusion.This feature is obscure and may lead to confusion.

Data can be followed by a qualifierData can be followed by a qualifier See SHEF Manual, Table 10See SHEF Manual, Table 10 Some common codesSome common codes

E:E: estimated, estimated, Q:Q: questionable, questionable, F:F: flagged flagged R:R: rejected, rejected, M:M: manual edit, manual edit, B:B: bad bad

Ex: Ex: PP 5.07QPP 5.07Q

Retained CommentsRetained Comments

It is possible to attach comments to data It is possible to attach comments to data values that will be storedvalues that will be stored

Follow the data value with the comment Follow the data value with the comment enclosed in single or double quotesenclosed in single or double quotes

The comment should not exceed 80 The comment should not exceed 80 character in lengthcharacter in length

Ex: we want to note that the temperature Ex: we want to note that the temperature was from a backup thermometerwas from a backup thermometer TA 88 "backup thermometer"TA 88 "backup thermometer"

Quick Quiz: .A Data StringsQuick Quiz: .A Data Strings

5. List the elements in data strings5. List the elements in data strings 6. Encode 12:00 UTC6. Encode 12:00 UTC 7. Encode 12 hour new snowfall7. Encode 12 hour new snowfall 8. Encode an estimated river stage at 8. Encode an estimated river stage at

18:15 UTC of 3 feet18:15 UTC of 3 feet

Answers: .A Data StringsAnswers: .A Data Strings

5. Time, parameter code, value5. Time, parameter code, value 6. 6. Z DH1200Z DH1200 or or Z DH12Z DH12 or or DH1200DH1200 or or DH12DH12 or nothing or nothing

7. 7. SFKSFK 8. 8. DH1815/HG 3.0EDH1815/HG 3.0E

Vector Physical ElementsVector Physical Elements

Some Physical Elements have paired values, Some Physical Elements have paired values, see SHEF Manual section 7.4.6see SHEF Manual section 7.4.6

Lets look at Lets look at TV: TV: temperature in vegetated temperature in vegetated soil at depthsoil at depth The paired values are the depth and the The paired values are the depth and the

temperaturetemperature The two values are encoded like this: The two values are encoded like this: dd.ttttdd.tttt, ,

where where dddd is the depth in inches and is the depth in inches and tttttttt is the is the temperature in degrees and tenths in temperature in degrees and tenths in °F°F

Ex:Ex: 12.062112.0621 = = 62.1 °F at 12 inches depth62.1 °F at 12 inches depth 5.4505.450 = = 450.0 °F at 5 inches depth (meant 450.0 °F at 5 inches depth (meant 5.0455.045)) -1.0022-1.0022 = = -2.2 °F at 1 inch depth -2.2 °F at 1 inch depth

Back to the Simple ExampleBack to the Simple Example

The river gage reading at Deputy IN was The river gage reading at Deputy IN was 13.45 at 7:00 AM EST on 4/1/200513.45 at 7:00 AM EST on 4/1/2005

Data time: Data time: 20090401 E DH070020090401 E DH0700 We could leave off the century or year in the dateWe could leave off the century or year in the date We will use We will use DHDH since the data day is the same as since the data day is the same as

the positional field daythe positional field day We could leave off the minutesWe could leave off the minutes

River gage reading is River gage reading is HGIRZZZ 13.45HGIRZZZ 13.45 We could leave off We could leave off IRZZZIRZZZ since they are the since they are the

defaultdefault

Putting the Report TogetherPutting the Report Together

The data elements are separated by a The data elements are separated by a // To continue the report on the next line use To continue the report on the next line use .An, .An,

you don’t need slashes at the end or beginning of you don’t need slashes at the end or beginning of a linea line .A DEPI3 0401 E DH07/HG 13.45.A DEPI3 0401 E DH07/HG 13.45.A1 PPD 0.15/TA 45/TX 60/TN 30.A1 PPD 0.15/TA 45/TX 60/TN 30

To correct a bad report use .ARTo correct a bad report use .AR .AR DEPI3 0401 E DH07/HG 12.45.AR DEPI3 0401 E DH07/HG 12.45

Each station is on a different lineEach station is on a different line .A ABEK2 0401 E DH07/PPD 0.25.A ABEK2 0401 E DH07/PPD 0.25 .A BNNK2 0401 E DH07/PPD 0.34.A BNNK2 0401 E DH07/PPD 0.34

.A DEPI3 20090401 E DH07/HG 13.45

Other Data String ElementsOther Data String Elements

DCccyymmddhhnn:DCccyymmddhhnn: Creation DateCreation Date DefaultsDefaults to same as observation time to same as observation time Required for river forecastsRequired for river forecasts Similar to Date codesSimilar to Date codes

DUa:DUa: Units CodeUnits Code DUEDUE = = English units, English units, defaultdefault DUSDUS = = Standard unitsStandard units

DQa:DQa: Qualifier CodeQualifier Code Same as data qualifier but applies qualifier to Same as data qualifier but applies qualifier to

ALL data ALL data

Internal CommentsInternal Comments

Allowed anywhere in reportAllowed anywhere in report Colon (:) turns comments onColon (:) turns comments on End of line or another colon turns comments End of line or another colon turns comments

offoff SHEF decoder ignores them and they are not SHEF decoder ignores them and they are not

storedstored Ex:Ex:

.A ABYV1 1123 DH11/PPD 0.23 : Albany VT.A ABYV1 1123 DH11/PPD 0.23 : Albany VT .A BELV1 : Belle VT : 1123 DH11/PPD 0.12.A BELV1 : Belle VT : 1123 DH11/PPD 0.12

Quick Quiz: .A ReportsQuick Quiz: .A Reports

9. Encode data AAWC1 for 5/20 at 08:30 PDT: 9. Encode data AAWC1 for 5/20 at 08:30 PDT: 24 hour pcpn 1.23, 24 hour max temp 70 24 hour pcpn 1.23, 24 hour max temp 70 °F, °F, 24 hour min temp 50 °F24 hour min temp 50 °F

10. At BLWW1, a DCP, the river crested on 7/4 10. At BLWW1, a DCP, the river crested on 7/4 at 34.5 feet at 1623Z, correcting an earlier at 34.5 feet at 1623Z, correcting an earlier incorrect crest of 34.2 feetincorrect crest of 34.2 feet

11. On 6/1 at 1500 EDT, you get a 11. On 6/1 at 1500 EDT, you get a questionable report of 5 inches of rain in the questionable report of 5 inches of rain in the last 12 hours at latitude 40.3° and longitude last 12 hours at latitude 40.3° and longitude 105.0° 105.0°

Quick Quiz: .A Reports Quick Quiz: .A Reports (con’t)(con’t)

12. BNNM8, an ALERT gage, collects 3.45 12. BNNM8, an ALERT gage, collects 3.45 inches over the last 5 hours ending 8/9 at inches over the last 5 hours ending 8/9 at 13Z.13Z.

13. LARC river readings at 6Z on 10/12: 13. LARC river readings at 6Z on 10/12: BSNK2 5.67, CYCK2 12.34, FFTK2 10.78BSNK2 5.67, CYCK2 12.34, FFTK2 10.78

14. Hourly river readings for PTTP1 on 8/9 14. Hourly river readings for PTTP1 on 8/9 starting at 18Z: 9.45, 9.47, 9.50, 9.53starting at 18Z: 9.45, 9.47, 9.50, 9.53

Answers: .A ReportsAnswers: .A Reports

9.9. .A AAWC1 0520 PD DH0830/PP 1.23/TX 70.A AAWC1 0520 PD DH0830/PP 1.23/TX 70.A1 TN 50.A1 TN 50

PPPP defaults to defaults to PPDRZZZPPDRZZZ, , TXTX defaults to defaults to TAIRZXZTAIRZXZ, , TNTN defaults to defaults to TAIRZNZTAIRZNZ

10.10. .AR BLWW1 0704 DH1623/HGIRGX 34.5.AR BLWW1 0704 DH1623/HGIRGX 34.5 Must use Must use .AR.AR to correct a bad report to correct a bad report Don’t need Don’t need ZZ time zone since this is the default time zone since this is the default Cannot use Cannot use HXHX since the send code translates since the send code translates

to to HGIHGIRZRZXZXZ 11.11. .A X4031050 0601 ED DH15/PPK 5.0Q.A X4031050 0601 ED DH15/PPK 5.0Q

Be sure to include the Be sure to include the QQ qualifier qualifier

Answers: .A Reports (con’t)Answers: .A Reports (con’t)

12.12. .A BNNM8 0809 DH13/DVH5/PPVRR 3.45.A BNNM8 0809 DH13/DVH5/PPVRR 3.45

Must use DVH05/PPVRR since there is no Must use DVH05/PPVRR since there is no duration code for 5 hoursduration code for 5 hours

13.13. .A BSNK2 1012 DH06/HGIRP 5.67.A BSNK2 1012 DH06/HGIRP 5.67.A CYCK2 1012 DH06/HGIRP 12.34.A CYCK2 1012 DH06/HGIRP 12.34.A FFTK2 1012 DH06/HGIRP 10.78.A FFTK2 1012 DH06/HGIRP 10.78

There is a better way to encode this dataThere is a better way to encode this data 14.14. .A PPTP1 0809 DH18/HG 9.45/DRH+1/HG 9.47.A PPTP1 0809 DH18/HG 9.45/DRH+1/HG 9.47

.A1 DRH+2/HG 9.50/DRH+3/HG 9.53.A1 DRH+2/HG 9.50/DRH+3/HG 9.53 Good use of the time relative codeGood use of the time relative code There is also a better way to encode this dataThere is also a better way to encode this data

The .B FormatThe .B Format

Problem 13 shows that the Problem 13 shows that the .A.A format format can get redundant for collectives can get redundant for collectives with the same data and timewith the same data and time

This is why the This is why the .B.B format exists format exists The repeated information is gathered The repeated information is gathered

into a header and is separated from into a header and is separated from the data the data

.B Reports.B Reports

.B.B reports consist of three parts reports consist of three parts Header, which describes the dataHeader, which describes the data Body, which contains the dataBody, which contains the data Terminator, Terminator, .END.END

.B LMK 1012 DH06/HGIRPBSNK2 5.67CYCK2 12.34FFTK2 10.78.END

Header

Body

Terminator

.B Header.B Header

Similar to the .A formatSimilar to the .A format Positional fields are the same except that Positional fields are the same except that

the location ID is replaced with the the location ID is replaced with the message source IDmessage source ID

The data string is replaced with the The data string is replaced with the parameter control string parameter control string

.B LMK 1012 DH06/HGIRPpositional fields parameter control

string

.B Parameter Control String.B Parameter Control String

Similar to the .A data string without Similar to the .A data string without the datathe data

Possible elements are the same as .A Possible elements are the same as .A data stringdata string DHDH: data time, : data time, DCDC: creation time: creation time DUDU: units, : units, DQDQ: qualifier, : qualifier, DVDV: duration: duration

Parameter codes are listed together Parameter codes are listed together separated by slashes separated by slashes

.B Body.B Body

Each line in the body:Each line in the body: Location IDLocation ID One or more spacesOne or more spaces The data for that location separated by The data for that location separated by

slashesslashes

BSNK2 5.67CYCK2 12.34FFTK2 10.78

.B Terminator.B Terminator

Required after the bodyRequired after the body Other SHEF reports may followOther SHEF reports may follow

.END

Another .B ExampleAnother .B Example

Encode the following collection:Encode the following collection:LocationLocation ZtimeZtime PcpnPcpn Max TempMax Temp Min TempMin TempATDK1ATDK1 12001200 1.231.23 7878 4545BELK1BELK1 12001200 MM 6767 4343CBDK1CBDK1 12001200 0.560.56DSOK1DSOK1 11001100 1.561.56 7575 4040ECLK1ECLK1 12001200 0.890.89 7878 MM

The data is similar but differs:The data is similar but differs: BELK1 pcpn report is missingBELK1 pcpn report is missing CBDK1 does not have a thermometerCBDK1 does not have a thermometer DSOK1 time was at 11Z not 12ZDSOK1 time was at 11Z not 12Z ECLK1 minimum temp is missingECLK1 minimum temp is missing

First: Create the HeaderFirst: Create the Header

.B TOP 0614 DH12/PPD/TX/TN

Format Specifier

Message Source

Date

Time

Parameter Codes

Next: Add the BodyNext: Add the Body

The order of the data must match The order of the data must match the order of the parameter codesthe order of the parameter codes

Encode M for data that usually exists Encode M for data that usually exists but is missingbut is missing

Encode nothing for data that does Encode nothing for data that does not existnot exist

Overriding the date and time in the Overriding the date and time in the body applies only to the current linebody applies only to the current line

Each Line of the BodyEach Line of the Body

ATDK1 is straight forwardATDK1 is straight forward ATDK1 1.23/78/45ATDK1 1.23/78/45

BELK1 rain report is missingBELK1 rain report is missing BELK1 M/67/43BELK1 M/67/43

CBDK1 does not have temperaturesCBDK1 does not have temperatures CBDK1 0.56//CBDK1 0.56// Don’t use M because the data never exists, but must Don’t use M because the data never exists, but must

include two slashes, a space between the slashes is not include two slashes, a space between the slashes is not requiredrequired

DSOK1 time was 11Z, not 12ZDSOK1 time was 11Z, not 12Z DSOK1 DH11/1.56/75/40DSOK1 DH11/1.56/75/40 Time is overridden for this line onlyTime is overridden for this line only

ECLK1 minimum temperature was missingECLK1 minimum temperature was missing ECLK1 0.89/78/MECLK1 0.89/78/M

The Complete .B BodyThe Complete .B Body

There are other options with .B There are other options with .B bodies as we will see laterbodies as we will see later

ATDK1 1.23/78/45BELK1 M/67/43CBDK1 0.56//DSOK1 DH11/1.56/75/40ECLK1 0.89/78/M

Finish: Add the TerminatorFinish: Add the Terminator

Will generate an error if it is missingWill generate an error if it is missing

.END

The Complete .B ReportThe Complete .B Report

.B TOP 0614 DH12/PPD/TX/TNATDK1 1.23/78/45BELK1 M/67/43CBDK1 0.56//DSOK1 DH11/1.56/75/40ECLK1 0.89/78/M.END

Commented BodyCommented Body

Use spaces and comments to make data Use spaces and comments to make data easier to readeasier to read

The SHEF decoder ignores these additionsThe SHEF decoder ignores these additions DON”T USE MORE THAN 14 CONSECTUTIVE DON”T USE MORE THAN 14 CONSECTUTIVE

SPACES OR THE SHEF DECODER WILL STOP!! SPACES OR THE SHEF DECODER WILL STOP!!

:LOC PCPN MAX MINATDK1 1.23/ 78/ 45 : ATWOOD BELK1 M/ 67/ 43 : BELOITCBDK1 0.56/ / : CAMBRIDGEDSOK1 DH11/1.56/ 75/ 40 : DE SOTOECLK1 0.89/ 78/ M : ELK CITY

Packed BodyPacked Body

Multiple reports on the same line, Multiple reports on the same line, separated by commasseparated by commas

More difficult to read but more compactMore difficult to read but more compact The SHEF decoder does not careThe SHEF decoder does not care

ATDK1 1.23/78/45, BELK1 M/67/43CBDK1 0.56//, DSOK1 DH11/1.56/75/40ECLK1 0.89/78/M

Quick Quiz: .B ReportsQuick Quiz: .B Reports

15. A group of LARCS from RLX report this 15. A group of LARCS from RLX report this data at 7/15 18Z; ABRW2 stage 3.21, pcpn data at 7/15 18Z; ABRW2 stage 3.21, pcpn accumulator 7.7, BEMW2 upper gage 12.3, accumulator 7.7, BEMW2 upper gage 12.3, lower gage 9.5; CFXW2 stage 1.00 lower gage 9.5; CFXW2 stage 1.00 (questionable), accumulator pcpn 8.3; (questionable), accumulator pcpn 8.3; DLYW2 accumulator pcpn 5.6DLYW2 accumulator pcpn 5.6

16. PCKI2, a DCP in LOT’s area reports this 16. PCKI2, a DCP in LOT’s area reports this stage data on 9/13; 13Z 5.67, 14Z 5.80, stage data on 9/13; 13Z 5.67, 14Z 5.80, 15Z 6.20, 16Z 6.45, 17Z 7.02, 18Z 7.23 15Z 6.20, 16Z 6.45, 17Z 7.02, 18Z 7.23

Answers: .B ReportsAnswers: .B Reports 15.15. .B RLX 0715 DH18/PCIRP/HGIRP/HPIRP/HTIRP.B RLX 0715 DH18/PCIRP/HGIRP/HPIRP/HTIRP

ABRW2 7.7/3.21//ABRW2 7.7/3.21// BEMW2 //12.3/9.5BEMW2 //12.3/9.5 CFXW2 8.3/1.00Q//CFXW2 8.3/1.00Q// DLYW2 5.6///DLYW2 5.6/// .END.END

16.16. .B LOT 0913 DH13/HGIRG .B LOT 0913 DH13/HGIRG

PCKI2 5.67, PCKI2 DH14/5.80, PCKI2 PCKI2 5.67, PCKI2 DH14/5.80, PCKI2 DH15/6.20DH15/6.20

PCKI2 DH16/6.45, PCKI2 DH17/7.02,PCKI2 DH16/6.45, PCKI2 DH17/7.02, PCKI2 DH18/7.23PCKI2 DH18/7.23 .END.END

The .E FormatThe .E Format

Both problems 14 and 16 show Both problems 14 and 16 show that that .A.A or or .B.B reports can still get reports can still get redundant if encode a sequence of redundant if encode a sequence of data from the same locationdata from the same location

This is why the This is why the .E.E format exists format exists .E.E SHEF reports are used for an SHEF reports are used for an

evenly spaced sequence of data from evenly spaced sequence of data from one location one location

.E Reports.E Reports

.E reports consist of two parts.E reports consist of two parts Positional fields, same as .A reportsPositional fields, same as .A reports Data string, similar to .A reports with the Data string, similar to .A reports with the

addition of data interval element, addition of data interval element, DIDI

.E PCKI2 0913 DH13/HGIRG/DIH1/5.67/5.80/6.20.E PCKI2 0913 DH13/HGIRG/DIH1/5.67/5.80/6.20

.E1 6.45/7.02/7.23.E1 6.45/7.02/7.23

Positional fields

data string

Data IntervalData Interval

See SHEF Manual, Table 12See SHEF Manual, Table 12 DItDIt±±xxxx

tt: : the same as the date codethe same as the date code ±±: : the increment/decrement flag, default is the increment/decrement flag, default is

incrementincrement xxxx: : the size of the intervalthe size of the interval

Ex:Ex: DIH1DIH1 or or DIN60DIN60 = one hour increment = one hour increment DIH-1DIH-1 or or DIN-60DIN-60 = one hour decrement = one hour decrement DIN-15DIN-15 = 15 minute decrement = 15 minute decrement DID+1DID+1 = 1 day increment = 1 day increment DIM2DIM2 = 2 month increment = 2 month increment

Time Interval End of MonthTime Interval End of Month

DIE1: End of month data, user does DIE1: End of month data, user does not have to worry about the different not have to worry about the different number of days in each monthnumber of days in each month

Ex: (previous example) encode Ex: (previous example) encode spring monthly precipitation for spring monthly precipitation for ALTP1ALTP1 .E ALTP1 0331 E DH07/PPM/DIE1/4.46/3.56/6.80.E ALTP1 0331 E DH07/PPM/DIE1/4.46/3.56/6.80

ValuesValues

The first value is the at the specified The first value is the at the specified timetime

The other values follow, separated by The other values follow, separated by slashesslashes

Values move forward in time with Values move forward in time with DIt+xxDIt+xx or just or just DItxDItx data intervals data intervals

Values move backwards in time withValues move backwards in time with DIt-xxDIt-xx data intervals data intervals

More .E ExamplesMore .E Examples

IFLOWS rainfall data from BALT1 on IFLOWS rainfall data from BALT1 on 2/26 was 1800Z 0, 1745Z 0, 1730Z 2/26 was 1800Z 0, 1745Z 0, 1730Z 0.04, 1715 M, 1700Z 0.080.04, 1715 M, 1700Z 0.08 .E BALT1 0226 DH18/PPCRP/DIN-15/0/0.04/M/0.08.E BALT1 0226 DH18/PPCRP/DIN-15/0/0.04/M/0.08

River forecast issued at 4/4 14Z for River forecast issued at 4/4 14Z for MLPK2 at 12Z for the next three MLPK2 at 12Z for the next three days: 12.4, 13.4, 15.6days: 12.4, 13.4, 15.6 .E MLPK2 0405 DH12/DC0504041400/HGIFF/DID1/12.4.E MLPK2 0405 DH12/DC0504041400/HGIFF/DID1/12.4.E1 13.4/15.6.E1 13.4/15.6

Quick Quiz: .E ReportsQuick Quiz: .E Reports

14. (again) Hourly river readings for PTTP1 14. (again) Hourly river readings for PTTP1 on 8/9 starting at 18Z: 9.45, 9.47, 9.50, on 8/9 starting at 18Z: 9.45, 9.47, 9.50, 9.539.53

17. Hourly rainfall at YCTA3 for the last 5 17. Hourly rainfall at YCTA3 for the last 5 hours ending at 00Z on 9/1 was all zeros.hours ending at 00Z on 9/1 was all zeros.

18. The 3 hour hydrograph for HMAC2, a 18. The 3 hour hydrograph for HMAC2, a DCP, starting on 11/30 00Z was 12.3, 14.5, DCP, starting on 11/30 00Z was 12.3, 14.5, 17.8, 22.3, 24.5, 24.3; the crest was 25.1 17.8, 22.3, 24.5, 24.3; the crest was 25.1 at 11/30 1430Zat 11/30 1430Z

Answers: .E ReportsAnswers: .E Reports

14. 14. .E PTTP1 0809 DH18/HG/DIH1/9.45/9.47/9.50.E PTTP1 0809 DH18/HG/DIH1/9.45/9.47/9.50

.E1 9.53.E1 9.53

17. 17. .E YCTA3 0901 DH00/PPH/DIN-1/0/0/0/0/0.E YCTA3 0901 DH00/PPH/DIN-1/0/0/0/0/0

18. 18. .E HMAC2 1130 DH00/HGIRG/DIH3/12.3/14.5 .E HMAC2 1130 DH00/HGIRG/DIH3/12.3/14.5 .E1 17.8/22.3/24.5/24.3 .E1 17.8/22.3/24.5/24.3

.A HMAC2 1130 DH1430/HGIRGX 25.1.A HMAC2 1130 DH1430/HGIRGX 25.1 We must use .A to encode the crestWe must use .A to encode the crest

SHEF DecodingSHEF Decoding

Once you know how to encode SHEF, Once you know how to encode SHEF, decoding is easydecoding is easy

Try to decode the following…Try to decode the following…

Quick Quiz: DecodingQuick Quiz: Decoding

19.19. .A TELI3 0803 E DH18/PP 0/TA 70/TX 85/TN 62.A TELI3 0803 E DH18/PP 0/TA 70/TX 85/TN 62

20. 20. .BR LMK 1123 DH12/PPD/HP/HT/QT.BR LMK 1123 DH12/PPD/HP/HT/QT

GRLK2 0/676.41/596.78/1.030GRLK2 0/676.41/596.78/1.030

NOLK2 0.50/495.78/M/0.103NOLK2 0.50/495.78/M/0.103

.END.END 21. 21. .E DURW3 0712 DH18/DC0507121515/HGIFF/DIH06.E DURW3 0712 DH18/DC0507121515/HGIFF/DIH06

8.3/7.8/7.4/7.28.3/7.8/7.4/7.2 22. 22. .A MELO1 0912 DH18/HPIRG 15.3/HTIRG 20.3.A MELO1 0912 DH18/HPIRG 15.3/HTIRG 20.3

TWIRG 75.0/TAIRG 82TWIRG 75.0/TAIRG 82

Answers: DecodingAnswers: Decoding

19. Aug. 3, 18 EDT for TELI319. Aug. 3, 18 EDT for TELI3 24 hour precipitation 0, current temperature 24 hour precipitation 0, current temperature

70 70 °F, 24 hour maximum temperature 85 °F, °F, 24 hour maximum temperature 85 °F, 24 hour minimum temperature 62 °F24 hour minimum temperature 62 °F

20. Collective from LMK for Nov. 23, 12Z20. Collective from LMK for Nov. 23, 12Z GRLK2: 24 hour precip 0, pool elevation 676.41 GRLK2: 24 hour precip 0, pool elevation 676.41

feet, tailwater elevation 596.78 feet, total flow feet, tailwater elevation 596.78 feet, total flow 1.030 kcfs1.030 kcfs

NOLK2: 24 hour precip 0.50 inches, pool NOLK2: 24 hour precip 0.50 inches, pool 495.78, missing tailwater, total flow .103 kcfs495.78, missing tailwater, total flow .103 kcfs

Answers: DecodingAnswers: Decoding

21. Forecast gage height including QPF for 21. Forecast gage height including QPF for DURW3 created on Jul. 12 at 15:15ZDURW3 created on Jul. 12 at 15:15Z 07/12 18Z 8.3 feet, 07/13 00Z 7.8, 06Z 7.4, 07/12 18Z 8.3 feet, 07/13 00Z 7.8, 06Z 7.4,

12Z 7.212Z 7.2 22. DCP report from MELO1 for Sep. 12 22. DCP report from MELO1 for Sep. 12

18Z18Z Pool 15.3, tailwater 20.3, water temp 75, air Pool 15.3, tailwater 20.3, water temp 75, air

temp 82temp 82

In ConclusionIn Conclusion

There are three different types of SHEF reports There are three different types of SHEF reports (.A, .B, and .E) which are used for different types (.A, .B, and .E) which are used for different types of dataof data

You can encode any data in .A but collectives are You can encode any data in .A but collectives are easier using .B and evenly spaced data intervals easier using .B and evenly spaced data intervals are easier using .Eare easier using .E

Practice makes perfect, try encoding and Practice makes perfect, try encoding and decoding different types of datadecoding different types of data

You can send test SHEF messages only to your You can send test SHEF messages only to your own AWIPS database by creating the message in own AWIPS database by creating the message in a text window and using File/Export to File; a text window and using File/Export to File; /data/fxa/ispan/hydro/test/data/fxa/ispan/hydro/test

The SHEF Manual is the standard resource for all The SHEF Manual is the standard resource for all SHEF questions, get to know itSHEF questions, get to know it