Amnestic neural network for classification

1

Amnesic Neural Network for Classification:

Application on Stock Trend Prediction*

Author: Qiang Ye, Bing Liang, Yijun Li Publication: ICSSSM 2005 Presenter: Yu-Hsiang Huang

2011.9.23

2

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Experiment Data Classification algorithm

Experiment Result

Outline

3

Artificial Neural Network models (ANN) Based on the neural structure of the brain The ANN learns from experience Critical step: network training

Two classes of predict stock market method Fundamental analysis

Macroeconomic data and basic financial status of company Technical analysis

History will repeat itself The correlation between price and volume reveals market behavior

Prediction By exploiting implications hidden in past trading activities By analyzing patterns and trends shown in price and volume chart

Introduction & Literature review

4

Stock price predictionTraditional assumption: customer behavior is consistent In real world: customer behavior change greatlyTraining data set may be time-variantDifficult to predict customer behavior from old data

Two strategies Select all data from different time: can’t represent current knowledge Select only the latest data: lose useful information hidden in data of

early time

Data selection in stock markets prediction will influence the training result

Introduction & Literature review

5

Amnesic Neural network (ANN*) model Introducing psychological notion of forgetting into

Back Propagation (BP) neural networkSolve the problem of cross-temporal data classificationEffectiveness data depends on timePresent data is more useful than old data Old data has less effect on training result, like gradually

forgetting

Introduction (cont.)

6

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Experiment Data Classification algorithm

Experiment Result

7

Artificial Neural Network Computational modeling tools for Modeling complex real-world

problems Capable of performing massively parallel computations for data

processing and knowledge representation The feed-forward-error-back-propagation learning algorithm is the most

famous procedure for training ANN

Back Propagation Neural Network Based on searching an error surface using gradient descent for points

with minimum errors Each iteration:

Forward activation to produce a solution Backward propagation of computed error to modify the weight

Methodology

8

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Experiment Data Classification algorithm

Experiment Result

9

BP network Model Input:

Output:

Weight: connects the node j in previous layer to the node k

Activation function:

Error signal: I(n) is a set of input Y(n) is a set of corresponding output D(n) is the expected output (given by training example)

Methodology (cont.)

10

Methodology (cont.)

Input vector

Input hidden output

output vector

weightbias

11

1. Square error:

2. Summation of square error:

3. Summation of square error for entire training data set:

Objective of learning

Find W to minimize , which is

Methodology (cont.)

12

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Experiment Data Classification algorithm

Experiment Result

13

Amnesic Neural Network Model

The effectiveness of data depends on the time.Present data is more useful than old data. Just like forgetting as time goes on.Assign different weight to data of different times ( new>old ).

Original square error:

Square error:

Methodology (cont.)

14

is forgetting function is forgetting coefficient is the benchmark time (current time or the time of newest data) is time of I(n)

Summation of square error for entire training data set:

Objective of learning

Find W to minimize , which is

Becomes to

Methodology (cont.)

15

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Classification algorithm Experiment Data

Experiment Result

16

Training Algorithm in ANN*Step1

Initialize weights to connections with random weight. Subject to Uniform Distribution(0,1)

Step2 Input I(n) = , d(n) is expected output of example

n.Step3

Compute output according to Equation 1, compute the output of neuron in each layer . = is initial input.

...... (1) is output of neuron j in the ith layer in nth round learning

Methodology (cont.)

17

Step4 Modify weight

1) Calculate δ backward For the node in output layer (1)For the node in hidden layer (2)

2) Modify the weight from output layer to previous layer

(3)

η is learning step.

Step5 n=n+1 go to Step2 until the training converges and system error

decrease below an acceptable threshold.

Methodology (cont.)

18

Methodology (cont.)

Step1

Initialize weights

Step2

Input I(n)

Step3

Compute output

Step4 Modify weight

19

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Experiment Data Classification algorithm

Experiment Result

20

Experiment dataAbout 900 stocks are selected with the data from 2001 to 200414 factors were considered

Stock price, major profit ratio EPS, interest cover ROE, receivables turnover Liability/asset ratio, asset turn-over Liquidity ratio, Liquid market value PE, tax profit growth PB

Methodology (cont.)

21

Percentage figure of price change for each stock

1000 records were selected for training the ANN*Other 500 records formed a testing sample

Methodology (cont.)

22

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Experiment Data Classification algorithm

Experiment Result

23

Classification algorithm

Input1) Patterns to be classified ;2) Neural Network Weight matrix W;

Output Class of X

Methodology (cont.)

24

Algorithm Step1

Input as an output of layer 0 Step2

Computes output in hidden layer Step3

Computes output in output layer Step4

Assigns output to , satisfying Step5

Return class label , satisfying

Methodology (cont.)

25

Introduction & Literature review Methodology

◦ BP Neural Network Model◦ Amnesic Neural Network Model

Training Algorithm Classification algorithm Experiment Data

Experiment Result

26

Parameters of ANN* Neural nodes in the input layer, 21 Neural nodes in the hidden layer, 25 Neural nodes in the output layer, 3 Training step (η), 0.01 Threshold of maximum training cycles, 10000 Threshold of minimum error rate, 0.005 Forgetting coefficient, test 10 different forgetting coefficients from 0~1 )

Experiment Result

27

Result

2. 1. 3.

The highest correct classification ratio on the test sample is achieved when

forgetting coefficient is set to 0.1

When forgetting coefficient is 0, the ANN* would be ordinary BP

The ANN* could do better than ordinary BP with careful selection of forgetting coefficient

Experiment Result (cont.)

better

BP ANNAmnesic ANN

28

This paper introduced the psychological notion of ‘forgetting’ into the BP neural network model established the Amnesic ANN model.

The aim of Amnesic ANN is to address the stochastic time variation problem in customer’s behaviors.

With carefully selected forgetting coefficients, the Amnesic ANN could perform better than the common ANN (BP) in stock price prediction

Further research should be done for the ratio of the right classified stocks is still very low in this experiment.

Conclusions

29

Thanks for your listening.

Q & A