Hub Queue Size Analyzer Implementing Neural Networks in practice
Feb 22, 2016
Hub Queue Size Analyzer
Implementing Neural Networks in practice
Task
• Provide an opportunity for DataArt employees to know queue size in DataArt Hub via PM
Choosing implementation method• Create a service which takes an image from camera in
DataArt Hub and performs image recognition• Additional module makes decision about queue size based
on recognition results• It was recommended to implement Neural Network for
image recognition because of following reasons:– We don’t need exact solution– Recognition error is not fatal– We have small amount (4) of possible queue states– Alternative solution (image analyzing with wavelets, analyzing
pixels etc.) is too long and expensive to implement
Introducing Neural Networks
computational model that is inspired by the structure and functional aspects of biological neural networks.
3 words about Neural Network structure
Neural Network Advantages– We program only structure of system, not behavior.
Structure is universal Structure is flexible
– We provide an image on the input and get recognition result on the output. Simple and fast recognition
– We do not care about algorithm for image analyzing No need of PhD
– We can reuse existing neural network in similar AND different tasks with minor changesReusability
– Parallel equations Fast & furious
How does it work
Every 5 seconds image is downloaded from web
How does it work
Image is converted to gray scale
How does it work
Image is compressed, auto leveled and transformed to double array
Auto leveling
How does it work
Brightness adjustment
Processed image is sent to input layer of network. If weights are correct on the output layer we get desired result (queue size).
Weights
How does it work
Threshold, integrationBrightness
adjustment
Threshold value and integration of result are implementedResult picture are created
How does it work
Threshold, integrationBrightness
adjustment
Uploading results on server
Implementation details
• All Neural Network logic was placed in class library for reusability. This library can be used in other projects
• For monitoring network condition and making extra training if needed administrative tool was developed.
Administrative tool
• Creating and training network• Retraining existing networks• Enlarging training set• Monitoring of correct work for all processes• Monitoring current network error and dependability of recognition result.• Saving and loading weights from file
Neural Network minuses
• Very careful approach to training set creation– Each pattern must be representative – Training set must cover all typical situations– large diversity of training for real-world operation
• We should always check what our network has learned
• Limited number of input, hidden and output nodes
• Large computations during training process
EXPLAINING PRINCIPLES OF NEURAL NETWORK
Part 2
How does NN work?
Couple words about sigmoid function
Bias nodeNo bias Bias
NN Training
Trial and error method– Initial weights have to be small enough
– Feed with sample data set (training set)– Get the output value– Use error (output minus target value) as a criteria
of success in the training algorithm– Change is small, number of iterations is big
Algorithm overview1. Initialise the network with small random weights
o maxWeight < 5.0 / (inputNodesNum * maxInputNodeValue)2. Repeat for each input pattern in the input collection
1. Present an input pattern to the input layer of the network.2. Get output values3. Calculate network’s summary error 4. Reduce the error by changing NN weights properly (back propagation)5. Propagate an error value back to each hidden neuron that is
proportional to their contribution of the network’s activation error. 6. Adjust the weights feeding each hidden neuron to reduce their
contribution of error for this input pattern. 3. Repeat step 2 until the network is suitably trained.
Back Propagation
• Is a way to reduce summary NN error and improve its performance
• “Blind paratrooper” method• 3 steps:
o o o
What can be customized in NN?
• Nodes number• Network error• Maximum number of training iterations• Learning rate
Where NN has been used already • Image/sound recognition• Stock market• Data classification• Medicine• Detecting credit card fraud.• Forecast engines• Geo routing systems• Aviation• NASA• Etc
Questions