Volume 50, Number 4, 2009 ACTA TECHNICA NAPOCENSIS Electronics and Telecommunications ________________________________________________________________________________ Manuscript received on October 01, 2009; revised on November 10, 2009 15 FORECASTING BY NEURAL NETWORKS IN THE WAVELET DOMAIN Ion RĂILEAN, Sorin MOGA, Monica BORDA Technical University of Cluj-Napoca, 15 Constantin Daicoviciu Street,Cluj Napoca,400020,Romania Telecom Bretagne, Technopôle Brest-Iroise - CS 83818 - 29238 Brest Cedex 3 – France Email : [email protected]Abstract: This paper presents a forecasting method for time series. This method combines the wavelet analysis and several forecasting techniques such as Artificial Neural Networks (ANN), linear regression and random walk. The proposed method is tested using three real time series: the first contains historical data recorded during eight weeks from a WiMAX network and the other two are based on financial series. It is shown that AI with wavelet analysis can be an efficient and versatile approach in time series prediction for small periods time interval (up to 1 month). For long time interval, the best method used is Linear Regression technique. Also we compared the results obtained using various types of wavelets. The results show that Daubechies 1 (db1) and Reverse biorthogonal 1 (rbio1.1) give the best results. Key words: Time series, Wavelet transform, forecasting, Neural Networks I. INTRODUCTION Forecasting or prediction is the process of estimation in unknown situations, based on the analysis of some factors that are believed to influence the future values, or based on the study of the past data behavior over time, in order to take decisions. A model is the representation of reality as it is seen by individuals who want to use this model to analyze and understand the reality and based on this information, to make short-term to long-term forecasts. Modeling and forecasting have applications in domains such as marketing, finance, telecommunications or organizational behavior. Time-series forecasting is an important area of forecasting where the historical values are collected and analyzed in order to develop a model describing the behavior of the series. Next, this model is used to extrapolate the time series into the future where the measurements are not available. A time-series represent a set of historical data, measured typically at successive times, each data being associated to a value. Time-series are interesting because many business operations are represented through time-series. Modeling financial time-series is interesting and useful, with many applications. The sales history of a certain product for example represents a time-series to be forecasted. The sales forecasts are very useful in the economic domain because they are used to optimize the inventory levels. The prediction of foreign currency risk or stock market volatility is also of high interest. Network traffic prediction plays a fundamental role in characterizing network performance and it is of significant interests in many network applications, such as adaptive applications, admission control or network management. Models that accurately catch the statistical characteristics of actual traffic are useful for analysis and simulation, and they help us to understand the network dynamics and to design and control the network. The main idea of traffic forecasting is to precisely predict traffic in the future considering the measured traffic history. The choice of the prediction method is based on the prediction interval, prediction error and computational cost. In order to come with a suitable conclusion regarding what prediction technique to use for this scope, different types of forecasting methods have been studied. There are numerous existing models for time series forecasting, which can be grouped into four categories [1]: (1) case-based reasoning: is a means for solving a new problem by using or adapting solutions to old problems - its essence in analogy. The basic principle of CBR is that similar problems have similar solutions (2) rule based forecasting: its application depends upon features of the time series (3) statistical models: exploit historical data. They contain early traditional models such as the single regressive model, exponential smoothing, ARIMA model (4) based on soft computing models: such as neural networks and their amelioration or mixture with other methods Between all of the above forecasting models, ANNs have been shown to produce better results [2], [3]. The performance of the neural networks against a standard statistical time series predictor is presented in [8], where a comparison between ARIMA and ANNs has been shown. This fact has been demonstrated again in [4], [5], [6]. In [7], the advantage of the artificial neural networks over
13
Embed
FORECASTING BY NEURAL NETWORKS IN THE WAVELET DOMAIN
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Volume 50, Number 4, 2009 ACTA TECHNICA NAPOCENSIS
Abstract: This paper presents a forecasting method for time series. This method combines the wavelet analysis and several forecasting techniques such as Artificial Neural Networks (ANN), linear regression and random walk. The proposed method is tested using three real time series: the first contains historical data recorded during eight weeks from a WiMAX network and the other two are based on financial series. It is shown that AI with wavelet analysis can be an efficient and versatile approach in time series prediction for small periods time interval (up to 1 month). For long time interval, the best method used is Linear Regression technique. Also we compared the results obtained using various types of wavelets. The results show that Daubechies 1 (db1) and Reverse biorthogonal 1 (rbio1.1) give the best results. Key words: Time series, Wavelet transform, forecasting, Neural Networks
I. INTRODUCTION
Forecasting or prediction is the process of estimation in
unknown situations, based on the analysis of some factors
that are believed to influence the future values, or based on
the study of the past data behavior over time, in order to take
decisions. A model is the representation of reality as it is
seen by individuals who want to use this model to analyze
and understand the reality and based on this information, to
make short-term to long-term forecasts. Modeling and
forecasting have applications in domains such as marketing,
finance, telecommunications or organizational behavior.
Time-series forecasting is an important area of
forecasting where the historical values are collected and
analyzed in order to develop a model describing the
behavior of the series. Next, this model is used to
extrapolate the time series into the future where the
measurements are not available.
A time-series represent a set of historical data, measured
typically at successive times, each data being associated to a
value. Time-series are interesting because many business
operations are represented through time-series.
Modeling financial time-series is interesting and useful,
with many applications. The sales history of a certain
product for example represents a time-series to be
forecasted. The sales forecasts are very useful in the
economic domain because they are used to optimize the
inventory levels. The prediction of foreign currency risk or
stock market volatility is also of high interest.
Network traffic prediction plays a fundamental role in
characterizing network performance and it is of significant
interests in many network applications, such as adaptive
applications, admission control or network management.
Models that accurately catch the statistical characteristics of
actual traffic are useful for analysis and simulation, and they
help us to understand the network dynamics and to design
and control the network. The main idea of traffic forecasting
is to precisely predict traffic in the future considering the
measured traffic history. The choice of the prediction
method is based on the prediction interval, prediction error
and computational cost.
In order to come with a suitable conclusion regarding
what prediction technique to use for this scope, different
types of forecasting methods have been studied. There are
numerous existing models for time series forecasting, which
can be grouped into four categories [1]:
(1) case-based reasoning: is a means for solving a new
problem by using or adapting solutions to old problems - its
essence in analogy. The basic principle of CBR is that
similar problems have similar solutions
(2) rule based forecasting: its application depends upon
features of the time series
(3) statistical models: exploit historical data. They
contain early traditional models such as the single regressive
model, exponential smoothing, ARIMA model
(4) based on soft computing models: such as neural
networks and their amelioration or mixture with other
methods
Between all of the above forecasting models, ANNs have
been shown to produce better results [2], [3]. The
performance of the neural networks against a standard
statistical time series predictor is presented in [8], where a
comparison between ARIMA and ANNs has been shown.
This fact has been demonstrated again in [4], [5], [6]. In [7],
the advantage of the artificial neural networks over
Volume 50, Number 4, 2009 ACTA TECHNICA NAPOCENSIS
maximum level of decomposition cannot exceed 4, or 3
respectively. It results that we have to train and model 3, 4,
or 5 neural networks.
As it has been mentioned at the beginning of this section,
we are going to have a prediction for day and week ahead
forecasting.
a) Wavelet Transform for Day Forecasting
We had two approaches in day prediction. The first
method consists in selection of days which are similar to the
one we want to forecast. For example, if we want to predict
how the traffic on Wednesday is, then for processing we
take only the data from all Wednesdays during weeks 1 to 7:
results in the training phase that we have 6 days for future
ANNs inputs, and one day for ANNs targets. The advantage
in this technique, is that usually the user's behavior is
modeled during certain week days, but the disadvantage is
that the number of subscribers is always changing.
The second idea for day prediction, is to take into
consideration the entire information until the day we want to
be forecasted. Let's take the same example: prediction of
Wednesday from the last 8th week. For training our ANNs,
we used at theirs inputs the given level of wavelet
decomposition corresponding to the information from the
Monday, 1st week, until Tuesday from 7th week. The
targeting data consists of the wavelet decomposition
obtained from Wednesday 7th week. In testing phase, we
took at ANNs inputs the decomposition from Monday 2nd
week, till Tuesday 8th week. The obtained output after
inverse wavelet transform, was compared to the real traffic
from the 3rd day of the 8th week. The main difference
between this method and the previous one, is that we do
always know about the amount of traffic coming from new
subscribers.
b) Wavelet Transform for Week Forecasting
We predicted the traffic from the last 8th week. We used
the wavelet decomposition from the first 7 weeks for ANN
training (6 for ANN inputs, and 1 for ANN target), and the
wavelet transform from weeks 2 - 8 for testing (2 - 7 for
inputs, while the traffic from the last week is compared to
the one obtained after inverse wavelet transform from all the
details and approximation taken from ANNs outputs).
Design of the Neural Network
The 2 most important types of ANNs are Feed-Forward
Neural Networks, and Recurrent Neural Networks. Feed-
Forward ANNs were applied in our forecasting techniques,
because according to [33] ,[9], this model is relatively
accurate in forecasting, despite being quite simple and easy
to use. Anyway, during our tests, the recurrent network
forecast performance was lower than that of the feed-
forward model. It might be because of the fact that recurrent
networks pass the data from back to forward as well as from
forward to back, and can become “confused” or unstable.
Further designing of the ANN implies the establishment of
the number of layers, and the number of neurons in each
layer. In [34] is pointed out the fact that the choice of the
number of layers is made knowing that one hidden layer
network is able to approximate most of the nonlinear
functions demanded by practice. This fact has been observed
by us in earlier studies on Artificial Neural Networks. That's
why we chose a single hidden layer ANN. Concerning the
dimension of each neuron layer the situation is as follows:
input and output layers are imposed by the problem to be
solved, while the dimension of the hidden layer is essential
for efficiency of the network. Now, let's discuss our analysis
according to the two types of prediction used.
a) ANN for Day Forecasting
For the output number of neurons, taking into
consideration that we want to predict a single element (data
from one day, or data from one week), gives us the idea of
using just one neuron for output, which contains in case of
day prediction an array of length 96, 32, or 16 samples. In
case of week forecasting, we have 672, 224, or 112 values.
Regarding the number of neurons for the input layer, we
have several options. The first option was to make a
temporal synchronization during the whole day. One of the
methods to achieve this is by taking an entire day from hours
00:00, till 24:00 as each input of the ANNs. By this we
made sure that the morning, noon, and evening periods from
known data, were responsible for the same periods of the
day from the forecasting sequence. In this case, we have 6
neurons for input. Another option for choosing the information for our neurons is if we think about the behavior of each person. According to [35], [36] our life cycle is divided into 3 parts: 8 hours of sleep, 8 hours of work, and the rest 8 hours for rest. But there are some industries where the use of these hours is shifted. The same survey described in the mentioned articles noted that there was a range of different 12 hours systems: in companies as steel, chemicals, aluminum, oil, food, engineering. Based on these articles, we used 8, 12, and 4 hours shifting between the information taken for ANN's neurons. In this case, the numbers of input
neurons are 16, 11, or 31 respectively, by applying the next
formula:
(3)
An example of the data selection in 8 hours shifting is
presented in Figure 6.
b) ANN for Week Forecasting
In case of week prediction, we have also single neuron
output, and variable number of inputs. The difference is that
we will make a shifting not by hours, but by days.
We have implemented also some methods for several
weeks forecasting. The first method is similar to the
previous: one week ahead forecasting. The only difference is
that in this case we will take as a target 2 weeks (or more in
case if we have enough information), and consequently less
data for ANNs inputs.
Volume 50, Number 4, 2009 ACTA TECHNICA NAPOCENSIS