Uncertainty of Weekly Nitrate-Nitrogen Forecasts Using Artificial Neural Networks

s State

predicterature,the same

rmationntropy oftions forsed with

kly data

Uncertainty of Weekly Nitrate-Nitrogen Forecasts UsingArtificial Neural Networks

Momcilo Markus, M.ASCE1; Christina W.-S. Tsai, M.ASCE2; and Misganaw Demissie, M.ASCE3

Abstract: Nonpoint source pollution affects the quality of numerous watersheds in the Midwestern United States. The IllinoiWater Survey conducted this study to~1! assess the potential of artificial neural networks~ANNs! in forecasting weekly nitrate-nitrogen~nitrate-N! concentration; and~2! evaluate the uncertainty associated with those forecasts. Three ANN models were applied toweekly nitrate-N concentrations in the Sangamon River near Decatur, Illinois, based on past weekly precipitation, air tempdischarge, and past nitrate-N concentrations. Those ANN models were more accurate than the linear regression models havinginputs and output. Uncertainty of the ANN models was further expressed through the entropy principle, as defined in the infotheory. Using several inputs in an ANN-based forecasting model reduced the uncertainty expressed through the marginal eweekly nitrate-N concentrations. The uncertainty of predictions was expressed as conditional entropy of future nitrate concentragiven past precipitation, temperature, discharge, and nitrate-N concentration. In general, the uncertainty of predictions decreamodel complexity. Including additional input variables produced more accurate predictions. However, using the previous wee~week t21! did not reduce the uncertainty in the predictions of future nitrate concentrations~week t11! based on current weekly data~week t!.

DOI: 10.1061/~ASCE!0733-9372~2003!129:3~267!

CE Database keywords: Nitrates; Entropy; Neural networks; Uncertainty; Forecasting; Nonpoint pollution; Watersheds.

ofac-au

geeda

thrsd

ngri-

ro-teofnds

n-o-e-

bed-thatutsdif-

eis.o-tos

the

ng-orthissix

ged

,

,

cl:

s.i

te9

3-

Introduction

Many communities in the midwestern United States dependsurface water sources for their drinking water and have beening chronic water quality problems related to nonpoint sourpollution. Nitrate contamination of drinking water in central Illinois is one of the most critical and difficult problems. Severwater supply utilities occasionally have exceeded the maximcontaminant level~MCL! for nitrate-nitrogen~nitrate-N!, whichthe United States Environmental Protection Agency~EPA! set at10 mg/L ~EPA 1991!. The Illinois Environmental ProtectionAgency is enforcing the drinking water regulations by requirinwater supply utilities and municipalities to develop plans to rduce nitrate-N concentrations below the MCL within a specifiperiod of time. As a result, several utilities are evaluating alterntive solutions. An effective solution necessitates identifying important parameters that control nitrate-N concentrations inwater and developing a procedure that uses these parameteaccurately predict nitrate-N concentration under different con

1Hydrologist, Illinois State Water Survey, 2204 Griffith Dr.Champaign, IL 61820. E-mail: [email protected]

2Hydrologist, Illinois State Water Survey, 2204 Griffith Dr.Champaign, IL 61820. E-mail: [email protected]

3Section Head, Illinois State Water Survey, Watershed ScienSection, 2204 Griffith Dr., Champaign, IL 61820. [email protected]

Note. Associate Editor: Carl. F. Cerco. Discussion open until Augu1, 2003. Separate discussions must be submitted for individual papersextend the closing date by one month, a written request must be filed wthe ASCE Managing Editor. The manuscript for this paper was submitfor review and possible publication on July 24, 2001; approved on May2002. This paper is part of theJournal of Environmental Engineering,Vol. 129, No. 3, March 1, 2003. ©ASCE, ISSN 0733-9372/2003/267–274/$18.00.

JOURN

nc-e

lm

-

--eto

i-

tions. Such predictions are important in water supply planniand in reducing nitrate-N concentration during the critical peods.

The traditional modeling approach applies watershed hydlogic and water-quality models calibrated and verified to simulawater quantity and quality. Those models require preparationextensive input data sets and a time-consuming calibration averification process that is often too expensive for small utilitieand municipalities.

The artificial neural network~ANN! approach could poten-tially provide an inexpensive alternative to using traditional coceptual modeling to predict water quality under different hydrlogic and meteorological conditions. ANNs, an alternativmethodology for dealing with mathematically intractable problems, can capture the important features and subfeatures emded in a large data set to produce predictable outputs. Modelssuccessfully capture the governing relationships between inpand outputs then can be used to predict water quality underferent conditions.

This paper describes the application of ANNs to predict thnitrate-N concentration in the Lake Decatur watershed in IllinoThe Illinois State Water Survey has collected hydrologic, meterological, and water-quality data in the Lake Decatur watershedidentify pollution sources and monitor nitrate concentration~Keefer and Demissie 2000!. A weekly time increment was cho-sen to test several ANN models for the studied watershed, andmodels were evaluated using root mean square error~RMSE! ofprediction, bias and entropy.

This approach does not address the origins of nitrates, loterm changes in nitrate concentration, climate variations,changes in land use. Such analyses are beyond the scope ofpaper. The method described here uses the approximatelyyears of observed hydrologic and meteorological data avera

e

tTothd,

AL OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH 2003 / 267

being

e

wnle:tedhe

Fig. 1. An artificial neural network withM input nodes, one hiddenlayer with h nodes, andN output nodes

eklythatsee-

plededas

andxit

ne

g anoff93;

is-st-ofs

the

ts,net-theleicaloNN

dy

ed.l

theothhe

rrorfterisningbeo be

s

k ofm

unsy tooutthers,ri-atat bea-

l ofi-

heotp-se

ch

268 / JOURNAL OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH 2

Based on the ANNs, continuous real-valued functions canapproximated in a certain interval within a desired accuracy usa sufficient number of sigmoids~Cybenko 1989; Ito 1991!. Asimple ANN approximation~Salas et al. 2000! for n inputs, x1

throughxn , and one output,f (x), can be presented as

f ~x!5(j 51

h

a js~v j 1x11v j 2x21¯1v jnxn2b j !1g (2)

whereF~•!5sigmoid defined by Eq.~1!, anda j , b j , v j i , andg5model parameters. Indexi represents theith input, andj is ahidden node. Symbolh represents the number of nodes in thhidden layer.

The process of updating model parameters based on knosystem inputs and outputs is based on the following principModel parameters are modified iteratively such that the compuoutput differs as little as possible from the observed one. Tmethod must be applied iteratively until convergence is reachApproximation error is minimized by adapting the ANN modeparameters area j i , b j i , v j i , andg. The cross validation tech-

nique~Sheedvash 1992; Weigend and Gernshfeld 1994! that per-forms the training procedure on a part of a time series, whileremainder of the time series serves for testing, was used. Btraining and testing errors are recorded for each iteration. Ttraining error generally decreases steadily, while the testing einitially decreases over time and then starts to increase areaching its minimum. After the iteration at which testing errorat its minimum, it is likely that the neural network is trained othe noise of the training data set. Therefore, the minimum testerror indicates the iteration at which the procedure shouldterminated and at which the parameter values are assumed tfinal.

ANNs were found to be a robust tool for modeling variouhydrologic processes~ASCE Task Committee 2000b!. The back-propagation method has fast convergence, but there is a rispremature convergence to local optima. Although Ito’s theoredefines the existence of a desired optimum~Ito 1991!, that opti-mum may not be unique, and reaching it may need several rwith randomly assumed parameters each time. ANNs are easuse, flexible, efficient, and do not require much knowledge abthe hydrologic system. There is no scientific method to defineANN architecture, such as the number of inputs, hidden layeand nodes; it is rather empirically based and may differ for vaous researchers. Although ANN architecture, optimal training dsets, adaptive learning, extrapolation, and other issues musexplored further, numerous successful hydrologic ANN applictions have been reported. This study explores the potentiaANNs for use in weekly nitrate forecasting, a real-world engneering problem.

Entropy

As defined in information theory, entropy is a measure of tuncertainty of a particular outcome in a random process. Linfo~1957! demonstrated that informational correlation in physical aplications is invariant under transformations, which is not the cawith the ordinary correlation. Amorocho and Espildora~1973!and Valdes et al.~1975! were among the first to introduce thisnew concept in hydrology, strongly influencing future researdirections. Harmancioglu et al.~1986! compared correlation-

weekly and aggregated at a watershed scale to predict wefluctuations within the period of record. It was also assumedthe models developed here could be applied within the foreable future.

Artificial Neural Networks

ANNs are ‘‘massively parallel interconnected networks of simelements and their hierarchical organizations which are intento interact with the objects of the real world in the same waybiological nervous systems do’’~Kohonen 1988!. The nodes areclassified as input, hidden, and output nodes.Input nodesreceivedata from sources external to the network,hidden nodessend andreceive data only to and from other nodes in the network,output nodesproduce data generated by the network, which ethe system~Fig. 1!. The number of hidden layers equal to ooccurs in most hydrologic applications.

There have been various applications of ANNs in evaluatinrange of water resources problems. In numerous rainfall-rustudies~Lachtermacher and Fuller 1993; Zhu and Fujita 19Markus et al. 1995; Tokar and Markus 2000!, the ANNs com-pared favorably with traditional models. ANNs also had proming results in water-quality modeling, including salinity forecaing ~Maier and Dandy 1996!, agriculture chemical assessmentrural wells~Ray and Klindworth 1996!, and predicting pesticideleaching through the soil~Starrett et al. 1996!. A comprehensiveanalysis of the role of ANNs in hydrology was undertaken byAmerican Society of Civil Engineers~ASCE! Task Committee onArtificial Neural Networks in Hydrology~ASCE Task Committee2000a, b! to serve as an introduction to ANNs for hydrologisand also to summarize the merits and limitations of neuralworks. Among rare attempts to quantify the uncertainties inANN models, Guan et al.~1997! used a neural network exampin a framework for uncertainty assessment of an ecologmodel. Also, Salas et al.~2000! ran a Monte Carlo experiment tquantify the parameter uncertainty associated with an Amodel.

Salas et al.~2000! defined a simple sigmoid used in this stuas

s~x!52

11e2x215

12e2x

11e2x(1)

003

r besed

-

:

lele

ned

on

dgeari-

py

-

rmedod-totadataof

atudulylowh o

ageh-rateudy

Al-ve

rre-ra-s

lag-est,to

therre-

andandennifi-w--

son

atchd,zero

ela-on-in-andag-all,rossaxi-

m-dels

-

ate

n-pre-turerentintothe

ingns--N

r

t,

il-for

based and entropy-based measures of information transfetween variables. Entropy-based techniques also have been uvarious studies for streamgaging network design~Husain 1989;Yang and Burn 1994! and water-quality monitoring network design ~Harmancioglu and Alpaslan 1992!.

Entropy ~Yang and Burn 1994! can be expressed as follows

H~X!52E f ~x!log@ f ~x!#dx (3)

where f (x) represents a probability density function of variabX. EntropyH(X) is also called the marginal entropy of a singvariableX.

Similar to Eq.~3!, uncertainty of two variables,X andY, canbe described by joint entropy:

H~X,Y!52E E f ~x,y!log@ f ~x,y!#dxdy (4)

Here, f (x,y) represents the joint probability density functioof variablesX andY. The joint and marginal entropies are relat

H~X,Y!5H~X!1H~Y!2T~X,Y! (5)

where T(X,Y) or transinformation represents the informatitransferred fromX to Y.

The uncertainty ofY, givenX, denoted asH(YuX), is equal to

H~YuX!5H~Y!2T~X,Y! (6)

If X and Y are independent,T(X,Y)50, the uncertainty ofYafter obtainingX is the same as the original~marginal! uncertaintyof Y. If the variables are perfectly correlated, then the knowleof one variable gives complete information about the other vable. In that caseT(X,Y)5H(Y).

The marginal entropy of the predictant,H(Y), is reducedthrough a model by introducing various inputs. Model entropredictions can be computed as a conditional entropyH(YuY),whereY represents the model forecast, andY represents the observed data.

Model Application to the Sangamon Rivernear Decatur, Illinois

The assessment of weekly ANN-based forecasts was perfoon the Sangamon River~Fig. 2! using observed hydrologic anmeteorological data for January 1994–April 1999. Various mels were tested using average weekly air temperature andweekly precipitation data observed at Decatur; dischargefrom the Monticello gaging station, routed to just upstreamLake Decatur using the PACE model~Durgunoglu et al. 1987!;and nitrate-N concentration time series observed at the DecNorth water treatment plant~Fig. 3!. Data analysis demonstratethat a high-flow season, typically between December and Jcoincides with a high-nitrate concentration season. High andseasons of nitrate-N concentration are so distinct that eacthem can be analyzed separately~Fig. 4!. Naturally, the high-concentration season is of greater importance in water manment. An arbitrary cutoff level of 2.5 mg/L was set for higconcentration season. Data points at which the nitconcentration was less than 2.5 mg/L were not used in this st

Preliminary Data Analysis

Table 1 presents the high-nitrate season correlation matrix.though one might expect that temperature data do not ha

JOURNA

-in

d

l

r

,

f

-

.

a

direct effect on nitrate-N concentration, Table 1 shows that colation coefficient~R! between simultaneous nitrate-N concenttions and air temperature,R50.445, exceed that of simultaneounitrate-N and discharge,R50.277, or precipitation,R50.128.Results for a lead time of one week also are presented. Theone autocorrelation for nitrate-N concentration was the highR50.823, while all other correlations in Table 1 were moderatelow. The data for a lead time equal to two weeks~Table 2! dem-onstrated a decrease in autocorrelation for nitrate-N, whilecross correlations remained similar to those of the lag-one colation matrix. The correlation between lag-zero dischargenitrate-N concentration was smaller than those of lag-onelag-two data. The lag-zero correlation coefficient betwenitrate-N concentration and discharge was equal to 0.277, sigcantly less than the lag-zero correlation before removing loseason data from the model,R50.446. The lag-one autocorrelation for nitrate-N was 0.943 before introducing the high-seaapproach, and 0.823 afterwards.

Although the high-nitrate and high-discharge seasons mfairly closely ~Fig. 3! and visually appear to be correlatenitrate-N, and discharge are not strongly correlated; the lag-correlation is only 0.277. Lag-one (R50.401) and lag-two (R50.394) correlation coefficients exceeded the lag-zero corrtion, which indicates that one of the causes for high nitrate ccentrations may be found in the high flows of previous timetervals. Correlations between nitrate-N concentrationprevious precipitation do not contradict this hypothesis. The lzero correlation between precipitation and nitrate-N is smwhile the lag-one and lag-two correlations are greater. The ccorrelation between nitrate-N and temperature reaches its mmum for lag-zero and decreases with lag time from 0.455~lagzero! to 0.249~lag two!.

Artificial Neural Network-Based Models

The accuracy of ANNs even with a minimum number of paraeters can be comparable and even higher than in linear mo~Markus et al. 1995; Markus and Salas 1998!. A simple ANNforecasting model based on Eq.~2! demonstrates how model uncertainty varies with the number of inputs and time steps.

Three ANN models were tested for one-step weekly nitrprediction using one input~N1!, four inputs~NQPT1!, and seveninputs ~NQPT2!. Various combinations of previous nitrate cocentration, air temperature, precipitation, and discharge resented model inputs. The models have only one output, the funitrate concentration. All models assumed that data in the curinterval t and previous intervals were known, while future datainterval t11 were predicted based on the known past. Priorusing ANN models, data were standardized by subtractingmean and dividing by the standard deviation. After simulatANN model output in the standard domain, data were traformed back to the original domain. Average weekly nitrateconcentration were expressed in milligrams per liter~mg/L!, themean weekly discharge in cubic meters per second~cms!, totalweekly precipitation in centimeters~cm!, and average weekly aitemperature in degrees Celsius~°C!.

Model N1 was the simplest model with only one inpunitrate-N concentration in weekt, N(t), and one output, nitrate-Nconcentration in weekt11, N(t11). Previous concentrationN(t) was expected to explain a significant portion of the variabity in N(t11) because the lag-one autocorrelation coefficientseriesN(t) was 0.823. The predicted nitrate concentrationN(t11) is expressed based on Eq.~2!:

L OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH 2003 / 269

Fig. 2. Location of the Sangamon River and Lake Decatur

the

o,

270 / JOURNAL OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH

Fig. 3. Weekly precipitation~P!, nitrate-nitrogen~N!, discharge~Q!,and air temperature~T! data, January 1994–April 1999, MonticellSangamon River

N~ t11!5(j 51

h

a js~v jN~ t !2b j !1g (7)

Model NQPT1 used nitrate-N in weekt, N(t), discharge,Q(t),precipitation,P(t), and temperature,T(t) to predictN(t11). Amodified version of Eq.~2! describes the dependence betweenoutput and the inputs

N~ t11!5(j 51

h

a js~v j 1N~ t !1v j 2Q~ t !1v j 3P~ t !

1v j 4T~ t !2b j !1g (8)

Model NQPT2 usedN(t), Q(t), Q(t21), P(t), P(t21), T(t),andT(t21) to predictN(t11).

N~ t11!5(j 51

h

a js~v j 1N~ t !1v j 2Q~ t !1v j 3Q~ t21!1v j 4P~ t !

1v j 5P~ t21!1v j 6T~ t !1v j 7T~ t21!2b j !1g (9)

2003

999

nting

d

th. Tosie

dyrs

res

seacet

opt

5

sedrrod

eanofhaddataalleron-the-cy inadimi-

ini-age.ingch

fore-o bekly

con-

gthyim-

a-

anu

JOURNA

al

is

r

.

-

t-

.i-

rin

standard domain, original domain, and as a fraction of the mvalue ~Table 3!. For example, the last number in the first rowTable 3, 0.006, means that in the testing stage model N1average bias equal to 0.6% of the mean value for the entireset. Table 3 also indicates that model NQPT1 produced smRMSE than the other two models. The training RMSE demstrated an increase in accuracy with model complexity. Noneless, it appears that the model NQPT2 had some redundaninputs. Model NQPT2 was more complex than NQPT1, but happroximately equal RMSE. The forecast bias demonstrated slar behavior in the training stage, but model NQPT2 had mmum bias compared to the other two models in the testing stHowever, all forecast bias values were relatively small, rangfrom 0.1 to 0.6% of the mean. The RMSE values were mularger, between 14.6 and 17.3% of the mean. Based on thecast RMSE and forecast bias, the model NQPT1 appears tmore accurate than the other two models for predicting weenitrate concentrations. Fig. 6 compares observed nitrate-Ncentration and predicted using model NQPT1.

Comparison of Artificial Neural Network and LinearRegression

While conceptual models require extensive data sets and lenmodel calibrations, linear regression models and ANNs offer s

on

t994

Table 1. Correlation Matrix Containing Nitrate-Nitrogen Concentrtion ~N! in Week t11 and Nitrate Concentration, Discharge~Q!,Precipitation ~P!, and Air Temperature~T! in Week t, SangamonRiver near Decatur during High-Concentration Periods between Jary 1994 and April 1999

N(t11) N(t) Q(t) P(t) T(t)

N(t11) 1.000 0.823 0.401 0.234 0.325N(t) 0.823 1.000 0.277 0.128 0.445Q(t) 0.401 0.277 1.000 0.498 0.232P(t) 0.234 0.128 0.498 1.000 0.229T(t) 0.325 0.445 0.232 0.229 1.000

-

Table 2. Correlation Matrix between Nitrate-Nitrogen Concentrati~N! in Week t11 and Nitrate Concentration, Discharge~Q!, Precipi-tation ~P!, and Temperature~T! in Week t21, Sangamon River aDecatur during High-Concentration Periods between January 1and April 1999

N(t11) N(t21) Q(t21) P(t21) T(t21)

N(t11) 1.000 0.643 0.394 0.293 0.249N(t21) 0.643 1.000 0.267 0.132 0.446Q(t21) 0.394 0.267 1.000 0.500 0.230P(t21) 0.293 0.132 0.500 1.000 0.229T(t21) 0.249 0.446 0.230 0.229 1.000

To better represent the data set and to reduce the poteeffects of temporal variability within the six-year data set, trainidata included the first quarter of observed data~January 1994–March 1995! and the last quarter of observed data~December1997–April 1999!. Testing data included the remaining perio~April 1995-December 1997!. Although other training/testingcombinations could have been tested, it was assumed thatcombination was appropriate for the purposes of this paperminimize the number of parameters and potentially ensure eaand faster computational convergence, the parameterh was as-sumed to be equal to one in all ANN models used in this stuThis simple ANN model having minimum number of parametealso permitted more appropriate comparison with linear regsion.

The cross validation method was used on the training datato estimate model parameters and simultaneously monitor thecuracy of the training model applied to the testing data sChanges in the testing errors shown in Fig. 5 suggested anmum number of iterations of 60~models N1 and NQPT1!, and135~model NQPT2!. Using a relatively large learning rate of 0.2resulted in a faster convergence.

The forecast accuracy of the ANN models is expresthrough RMSE and absolute value of the average forecast e~bias!. Results in both training and testing phases are presente

L OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH 2003 / 271

T2

sionndendi-tan

tosiondeh in

ear

ith-the

NN

neating

the

re-ed

4

6

5

9

1

sing5–

272 / JOURNAL OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH

rFig. 6. Observed and predicted weekly nitrate concentrations umodel NQPT1 for high-nitrate concentration periods, April 199December 1997

ye ofthat

pler, and faster solutions. ANNs are more flexible than regresmodels and require less prior knowledge of the system ustudy. Unlike ANNs, the regression theory imposes strict cotions for error statistics, such as normal distribution and cons~homoscedastic! variance. On the other hand, it is difficultinterpret a physical meaning of the ANN parameters. Regrescoefficients can reveal useful information about the system ustudy, but there are no established techniques to obtain sucformation from the ANN parameters.

The aforementioned ANN models were compared with linregression models. The one-input ANN model~N1! was com-pared with two one-input linear regression models, with and wout the intercept term, LR1 and LR2 respectively. Similarly,four-input ~NQPT1! and the seven-input~NQPT2! models werecompared with corresponding linear regression models. The Aand regression models were compared using RMSE~Fig. 7!. Onaverage, the ANN models outperformed the corresponding liregression models by 4.42% in training, and 3.30% in tesstage.

Uncertainty of Artificial Neutral Network Using Entropy

The aforementioned ANN models also were validated usingentropy measures of uncertainty. A discrete version of Eq.~3! wasused to compute entropy for various models~Press et al. 1995!:

on

lrk

t

nr-

r

H~X!5(k51

K

p~xk!log1

p~xk!(10)

wherek denotes a discrete data interval for variableX, xk is anoutcome corresponding to intervalk, andp(xk) is the probabilityof xk . The probabilityp(xk) is based on the empirical frequencof variableX. The entropy is expressed in bits because the basthe logarithm was assumed to be equal to 2. It was assumedvariableX has a finite number of possible outcomes~K!. A dis-crete version of Eq.~4! ~Press et al. 1995! was used to calculatejoint entropy

H~X,Y!5(k51

K

(l 51

L

p~xk,yl !log1

p~xk,yl !(11)

Fig. 7. Comparison of artificial neural network and linear regressimodels with intercept~LR1! and without intercept~LR2!, for one-input artificial neural network model N1, four-input artificial neuranetwork model NQPT1, and seven-input artificial neural netwomodel NQPT2

Table 3. A Comparison of Model Uncertainties Expressed by Focast RMSE and Absolute Bias, for Artificial Neural Network-BasModels N1, NQPT1, and NQPT2

Forecast parameterA–trainingB–testing N1 NQPT1 NQPT2

RMSE in standard domain A 0.521 0.477 0.47B 0.560 0.529 0.535

RMSE in original domain A 1.005 0.920 0.915B 1.081 1.022 1.033

RMSE relative to mean A 0.161 0.147 0.14B 0.173 0.164 0.165

Absolute bias in standard domain A 0.006 0.003 0.00B 0.018 0.010 0.009

Absolute bias in original domain A 0.011 0.006 0.00B 0.035 0.020 0.018

Absolute bias relative to mean A 0.002 0.001 0.00B 0.006 0.003 0.002

2003

ata

l-

n

d

e.

2o

ininctev

ahef fnisc

mino

m

scufi-trungc

de

intoge-of

en-e

gnd-ectic-ne-edos-e

-

ob--,

lye

cu-eded

ar-

d

JOURN

te

.f

tytyor.lsi-ri-stu-s.

e

rei-

or-

ofe-res

er-l

NQPT2 appears more sensitive to changes in the number of dintervals in both the training and testing phases.

Summary and Conclusions

Accurate nitrate-N concentration predictions are importantwater supply management. ANNs, an inexpensive alternativetraditional conceptual modeling, were applied in forecastinweekly nitrate-N concentration in the Sangamon River near Dcatur, Ill. Data analysis suggested that high and low seasonsnitrate-N concentrations closely coincide with high- and low-flowseasons, and the research focused on the high nitrate-N conctration season. The initial results of a simple ANN model werencouraging.

Preliminary analysis indicated that during training and testinstages, the ANN models were more accurate than the correspoing linear regression models. More complex ANN architecturcould potentially provide more insight into the basic cause–efferelationships and possibly produce more accurate weekly predtions. Besides the traditional measures, bias and RMSE, the ostep weekly nitrate-N concentration forecasts were also validatusing marginal and conditional entropies. Dividing the data inthigh- and low-flow data reduced the marginal entropy of the hitorical data. Adding various model inputs further reduced thforecast uncertainty.

Among the three ANN models used for weekly nitrate concentration forecasting~N1, NQPT1, and NQPT2!, model NQPT1 wasthe most accurate. The entropy-based validation test was anjective method for quantifying the uncertainties of all three models. Each entropy of the model was highly correlated with RMSEbut not with forecast bias. Moreover, the biases were relativesmall compared to RMSE and were less critical in comparing thperformance of each model. It is concluded that reasonably acrate weekly nitrate concentration predictions can be achievusing simple ANNs, and the results can be adequately validatusing marginal and conditional entropies.

Acknowledgments

This article was based on the data collection and research ptially supported by the City of Decatur, Ill. and the Water QualityStrategic Research Initiative of the Illinois Council on Food anAgricultural Research~C-FAR! at the University of Illinois atUrbana–Champaign.

Table 4. Summary of Model Uncertainties Expressed by ConditionaEntropies for Models N1, NQPT1, and NQPT2, and Marginal Entropy, for Various Number of Intervals~Nint!

Stage N1 NQPT1 NQPT2 Marginal

Conditional entropyfor Nint55

Training 0.981 0.903 0.923 1.414

Testing 0.990 0.950 0.962 1.430

Conditional entropyfor Nint510

Training 1.298 1.268 1.271 2.035

Testing 1.362 1.237 1.228 2.041

Conditional entropyfor Nint515

Training 1.437 1.427 1.450 2.399

Testing 1.575 1.441 1.477 2.440

s

wherek denotes a discrete data interval for variableX, l denotes adiscrete data interval for variableY, p(xk, yl) is the probability ofan outcome corresponding to intervalk for X and intervall for Y,K, and L are the numbers of possible outcomes forX and Y,respectively. It was assumed thatK5L. Transinformation wascomputed using Eq.~5!, and conditional entropy was calculateusing Eq.~6!.

To assess the sensitivity of entropy to the number of discrintervals ~K!, the values ofK between 1 and 20 were exploredFig. 8 presents resulting marginal~unconditional! entropy andconditional entropies for ANN models N1, NQPT1, and NQPTFig. 8 shows how the uncertainty changes with the numberintervals. The top line in Fig. 8 represents the testing uncertaof weekly high-season nitrate-N concentrations. This uncertaassumes no knowledge of previous observations of any prediOther lines represent the conditional entropy of various modpredicting nitrate concentration in the testing stage, when preous concentrations and other hydrologic and meteorological vables are known. The line representing the model N1, the higamong the conditional entropies, represents the uncertainty oture nitrate-N concentrations, given past nitrate-N concentratioIt is evident that the conditional uncertainty of model N1smaller than the marginal unconditional entropy. The differenbetween the marginal entropy~top line, Fig. 8! and the entropygiven previous nitrate concentrations~second line from the top,Fig. 8! represents the amount of information conveyed froN(t21) to N(t). It also represents the reduction of uncertaintyone-step nitrate predictions based on the model N1. The mcomplex models NQPT1 and NQPT2, in general, performed silarly and had smaller uncertainty than the modelN1. The patternseems to be more regular and less noisy for seven intervalless. A larger number of intervals provides noisier results, partilarly for model NQPT2. Although the model NQPT2 had signicant noise, and in some cases, it is entropy exceeded thamodel N1, the relative proportions between the uncertaintiesmained relatively constant. Table 4 presents the entropy measof uncertainty for 5, 10, and 15 discrete intervals for both trainiand testing data. On average, model N1 had the highest untainty and NQPT1 and NQPT2 performance was similar. Mo

AL OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH 2003 / 273

f

l

ate

ura

.’’

oloe-

e

m-

f

on.ter-

s

P.t-ge

er,

-o

01,m-

on-

Notation

The following symbols are used in this paper:f (x) 5 artificial neural network approximation;

H(X) 5 marginal entropy ofX;H(X,Y) 5 joint entropy ofX andY;H(XuY) 5 conditional entropy ofX, givenY;

h 5 number of hidden nodes in neural network;K,L 5 number of discrete intervals in computing

entropy;N(t) 5 average nitrate concentration in mg/L in weekt;

N(t11) 5 predicted nitrate concentration for weekt11;P(t) 5 total weekly precipitation in cm in weekt;Q(t) 5 mean weekly discharge in m3/s in weekt;T(t) 5 mean weekly temperature in °C in timet;

T(X,Y) 5 transinformation betweenX andY;t 5 time interval in weeks;

a 5 artifical neural network parameter;b 5 artificial neural network parameter;g 5 artificial neural network parameter;l 5 parameter of sigmoid;

s~x! 5 sigmoid function; andv 5 artificial neural network parameter.

References

American Society of Civil Engineers~ASCE!, Task Committee onApplication of Artificial Neural Networks in Hydrology.~2000a!.‘‘Artificial neural networks in hydrology. I: Preliminary concepts.’’J.Hydrologic Eng.,5~2!, 115–123.

American Society of Civil Engineers~ASCE!, Task Committee onApplication of Artificial Neural Networks in Hydrology.~2000b!.‘‘Artificial neural networks in hydrology. II: Hydrologic applica-tions.’’ J. Hydrologic Eng.,5~2!, 124–137.

Amorocho, J., and Espildora, B.~1973!. ‘‘Entropy in the assessment othe uncertainty of hydrologic systems and models.’’Water Resour.Res.,9~6!, 1511–1522.

Cybenko, G.~1989!. ‘‘Approximation by superposition of a sigmoidafunction.’’ Math. Control, Signals, Syst.,2, 303.

Durgunoglu, A., Knapp, H. V., and Changnon, S. A.~1987!. PACE Wa-tershed Model (PWM): Volume 1, Model Development, Illinois StWater Survey Contract Rep. 437, Champaign, Ill.

Environmental Protection Agency~EPA!. ~1991!. ‘‘National primarydrinking water regulations; final rule.’’40 CFR Parts 141, 142, and143.Federal Regulation 56~20!, 3526–97.

Guan, B. T., Gertner, G. Z., and Parysov, P.~1997!. ‘‘A framework foruncertainty assessment of mechanistic forest growth models: A nenetwork example.’’Ecol. Modell.,98, 47–58.

Harmancioglu, N. B., and Alpaslan, N.~1992!. ‘‘Water quality monitoringnetwork design: a problem of multi-objective decision makingAWRA Water Resour. Bull.,28~1!, 179–191.

Harmancioglu, N. B., V. Yevjevich, and J. T. B. Obeysekera~1986!.‘‘Measures of information transfer between variables.’’Proc., 4th Int.Hydrol. Symp., Multivariate Analysis of Hydrologic Processes, H. W.Shen et al., eds., Colorado State University Press, Fort Collins, C

Husain, T. ~1989!. ‘‘Hydrologic uncertainty measure and network dsign.’’ AWRA Water Resour. Bull.,25~3!, 527–534.

Ito, Y. ~1991!. ‘‘Approximation of functions on a compact set by finit

274 / JOURNAL OF ENVIRONMENTAL ENGINEERING © ASCE / MARCH

l

.

sums of a sigmoid function without scaling.’’Neural Networks,4,817–826.

Keefer, L., and Demissie M.~2000!. ‘‘Watershed Monitoring for the LakeDecatur Watershed 1998–1999.’’ Illinois State Water Survey, Chapaign, Ill.

Kohonen, T.~1988!. ‘‘An introduction to neural computing.’’Neural Net-works,1, 3–16.

Lachtermacher, G., and Fuller, J. D.~1993!. ‘‘Backpropagation neuralnetworks in hydrology time series forecasting.’’Stochastic and statis-tical methods in hydrology and environmental engineering, Proc., Int.Conf. in Honor of Professor T. E. Unny, K. W. Hipel, eds., Univ. oWaterloo, Waterloo, Ont., Canada, 3, 229–242.

Linfoot, E. H. ~1957!. ‘‘An informational measure of correlation.’’Inf.Control, 1, 85–89.

Maier, H., and Dandy, G.~1996!. ‘‘The use of artificial neural networksfor the prediction of water quality parameters.’’Water Resour. Res.,32~4!, 1013–1022.

Markus, M., and Salas, J. D.~1998!. ‘‘Comparing methods for forecastingtotal summer flows on tributaries of the Colorado–Big Thompssystem.’’Proc., 1st Federal Interagency Hydrology Modeling Conf,Las Vegas, prepared by the subcommittee on Hydrology of the Inagency Advisory Committee on Water Data.

Markus, M., Salas, J. D., and Shin, H.~1995!. ‘‘Predicting streamflowsbased on neural networks.’’Proc., 1st Int. Conf. on Water ResourceEngineering, San Antonio, Tex., ASCE, New York.

Press, W. H., Teukolsky, S. A. Vetterling, W. T., and Flannery, B.~1995!. Numerical recipes in Fortran 77, The art of scientific compuing, 2nd Ed., Chap. 14, Statistical Description of Data, CambridUniversity Press, New York, 626–630.

Ray, C., and Klindworth, K. K.~1996!. ‘‘Use of artificial neural networksin agriculture chemical assessment of rural private wells.’’Proc.,North American Water and Environmental Conf., ASCE, New York,1678–1692.

Salas, J. D., Markus, M., and Tokar, A. S.~2000!. ‘‘Streamflow forecast-ing based on artificial neural networks.’’Artificial neural networks inhydrology, R. S. Govindaraju and A. Ramachandra Rao, eds., KluwDordrecht, The Netherlands, 23–51.

Sheedvash, S.~1992!. ‘‘New approaches for fast learning and architectural variation in multi-layer neural networks.’’ PhD thesis, ColoradState Univ., Fort Collins, Colo.

Starrett, S. K., Najjar, Y. M., and Hill, J. C.~1996!. ‘‘Neural networkspredict pesticide leaching.’’Proc., North American Water and Envi-ronmental Conf., ASCE, New York, 1693–1698.

Tokar, S., and Markus, M.~2000!. ‘‘Precipitation-runoff modeling usingartificial neural networks and conceptual models.’’J. HydrologicEng.,5~2!, 156–161.

Valdes, J. B., Rodriguez-Iturbe, I., and Vicens, G. J.~1975!. ‘‘A Bayesianapproach to multivariate hydrologic synthesis.’’Ralph M. ParsonsLaboratory for Water Resources and Hydrodynamics, Rep. No. 2School of Engineering, Massachusetts Institute of Technology, Cabridge, Mass.

Weigend, A. S., and Gernshfeld, N. A., Eds.~1994!. ‘‘Time series predic-tion: forecasting the future and understanding the past.’’Santa FeInstitute Studies in the Sciences of Complexity, Proc., Vol. XV.Addison–Wesley, Reading, Mass.

Yang, Y., and Burn, D. H.~1994!. ‘‘An entropy approach to data collec-tion network design.’’J. Hydrol.,157, 307–324.

Zhu, M., and Fujita M.~1993!. ‘‘Application of neural networks to runoffforecast, stochastic and statistical methods in hydrology and envirmental engineering.’’Proc., Int. Conf. in Honor of Professor T. E.Unny,Univ. of Waterloo, Waterloo, Ont., Canada, 3.

2003