Neural Networks

Neural Networks

Week 5

Applications

Predict the taste of Coors beer as a function of its chemical composition

What are Artificial Neural Networks?

Artificial Intelligence (AI) Technique Artificial Neural Networks (ANN) are

biologically inspired and attempt to build computer models that operate like a human brain These networks can “learn” from the data and

recognize patterns

Basic Concepts of Neural Networks

Biological and artificial neural networks Neurons

Cells (processing elements) of a biological or artificial neural network Nucleus

The central processing portion of a neuron Dendrite

The part of a biological neuron that provides inputs to the cell

Basic Concepts of Neural Networks

Biological and artificial neural networks Axon

An outgoing connection (i.e., terminal) from a biological neuron Synapse

The connection (where the weights are) between processing elements in a neural network

Basic Concepts of Neural Networks

Basic Concepts of Neural Networks

Relationship Between Biological and Artificial Neural Networks

Soma – Node Dendrites – Input Axon – Output

ANNs typically have much fewer neurons than humans

Basic Concepts of Neural Networks

Network structure (three layers) Input Intermediate (hidden layer) Output

Basic Concepts of Neural Networks

Basic Concepts of Neural Networks

Transformation function (activation function) maps the summation (combination) function onto a narrower

range ( 0 to 1 or -1 to 1) to determine whether or not an output is produced (neuron fires)

The transformation occurs before the output reaches the next level in the network

Sigmoid (logical activation) function: an S-shaped transfer function in the range of zero to one –exp(x)/(1-exp(x))

Threshold value is sometimes used instead of a transformation function

A hurdle value for the output of a neuron to trigger the next level of neurons. If an output value is smaller than the threshold value, it will not be passed to the next level of neurons

Neural Network Prediction Formula

predictionestimate

weightestimate

hidden unit

biasestimate

0

1

5-5

-1

tanh

...

activationfunction

...

Neural Network Binary Prediction Formula

0

1

5-5

-1

tanh

0 1

5

-5

logitlink function

...

Learning in ANN Learning algorithm

The training procedure used by an artificial neural network Supervised learning

A method of training artificial neural networks in which sample cases are shown to the network as input and the weights are adjusted to minimize the error in its outputs

Learning in ANN

Learning in ANN How a network learns

Backpropagation The best-known supervised learning algorithm in neural computing. Learning is done by comparing computed outputs to desired outputs of historical cases

Learning in ANN How a network learns

Procedure for a learning algorithm 1. Initialize weights with random values and set other parameters2. Read in the input vector and the desired output3. Compute the actual output via the calculations, working forward through the

layers4. Compute the error5. Change the weights by working backward from the output layer through the

hidden layers

Learning in ANN

Error calculation and weights At each hidden node and target node: compute:

Linear combination function: C = w0 + w1x1 +…+ wnxn

Logistic activation function: L = exp(C)/(1+exp(C)

At the target node compute Bernoulli error function: sum errors over all observations, where the error is -2 ln (L) if there is a response, or -2 ln (1 – L) if there is no response

In the first iteration, random weights are used

In subsequent iterations, the weights are changed by a small amount so that the error is reduced

The process continues until the weights cannot be reduced further

Classification using NN

prerequisite set of training pattern (many patterns)approach code the values divide set of training pattern into:

– training set– test set

build a network train the network using the training set check the network quality using the test set

real data

training p.

coded p.

training set test set

Business Applications of Artificial Neural Networks (ANN)

Many applications across all areas of business target customers (CRM) bank loan approval hiring stock purchase trading electricity approving loan applications fraud prevention predicting bankruptcy time series forecasting

Disadvantages of Neural Networks coefficients are not readily interpretable

end user must apply insight in interpretation

Questions!