Offline Handwritten Gurmukhi Word Recognition Using Deep Neural Networks 1 Neeraj Kumar and 2 Sheifali Gupta 1 Dept of Electronics & Communication Engineering, Chitkara University, Himachal Pradesh, India. [email protected]2 Dept of Electrical & Electronics Engineering, Chitkara University, Punjab, India. [email protected]Abstract In spite of the advances achieved in pattern recognition technology, Handwritten Gurmukhi Word Recognition (HGWR) is still a challenging problem due to the presence of many similar characters and excessive cursiveness in Gurmukhi handwriting. Even the finest presented recognizers don’t give reasonable performance for realistic applications. Also due to variations in handwriting styles and speed of writing it is very hard to recognize the handwritten Gurmukhi characters. A majority of the work has already been reported on the online handwritten scripts like English, Bangla etc. Now research is being shifted towards the recognition of offline handwritten scripts. In order to recognize the handwritten Gurmukhi word, we present a novel method based on deep neural networks which has recently shown exceptional performance in various pattern recognition and machine learning applications, but has not been endeavored for HGWR. We present Handwritten Gurumukhi Word recognition method based on LBP features, Directional Features & Geometric features. A total of 117 features have been extracted which will be utilized to recognize the individual character for further word recognition. Also to map the recognized Gurmukhi text with Devanagari a suitable mapping technique has also been implemented. The proposed system effectively segments the words into individual characters and then recognizes the individual character, concatenates them and maps the International Journal of Pure and Applied Mathematics Volume 119 No. 12 2018, 14749-14767 ISSN: 1314-3395 (on-line version) url: http://www.ijpam.eu Special Issue ijpam.eu 14749
20
Embed
Offline Handwritten Gurmukhi Word Recognition Using Deep ... · Offline Handwritten Gurmukhi Word Recognition Using Deep Neural Networks 1Neeraj Kumar and 2Sheifali Gupta 1Dep t of
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Offline Handwritten Gurmukhi Word
Recognition Using Deep Neural Networks 1Neeraj Kumar and
Abstract In spite of the advances achieved in pattern recognition technology,
Handwritten Gurmukhi Word Recognition (HGWR) is still a challenging
problem due to the presence of many similar characters and excessive
cursiveness in Gurmukhi handwriting. Even the finest presented
recognizers don’t give reasonable performance for realistic applications.
Also due to variations in handwriting styles and speed of writing it is very
hard to recognize the handwritten Gurmukhi characters. A majority of the
work has already been reported on the online handwritten scripts like
English, Bangla etc. Now research is being shifted towards the recognition
of offline handwritten scripts. In order to recognize the handwritten
Gurmukhi word, we present a novel method based on deep neural
networks which has recently shown exceptional performance in various
pattern recognition and machine learning applications, but has not been
endeavored for HGWR. We present Handwritten Gurumukhi Word
recognition method based on LBP features, Directional Features &
Geometric features. A total of 117 features have been extracted which will
be utilized to recognize the individual character for further word
recognition. Also to map the recognized Gurmukhi text with Devanagari a
suitable mapping technique has also been implemented. The proposed
system effectively segments the words into individual characters and then
recognizes the individual character, concatenates them and maps the
International Journal of Pure and Applied MathematicsVolume 119 No. 12 2018, 14749-14767ISSN: 1314-3395 (on-line version)url: http://www.ijpam.euSpecial Issue ijpam.eu
14749
Gurmukhi text into Devanagari text using suitable unicodes.
Keywords: Word Recognition, LBP features, Directional features,
Regional features, Deep neural networks, Mapping.
International Journal of Pure and Applied Mathematics Special Issue
14750
1. Introduction
Computers have an excessive influence on everyone in these days and process
almost all the important works of our lives electronically. Keeping in mind the
use of computers these days, there is a requirement of developing proficient,
uncomplicated and speedy methods for the transfer of data between humans and
computers. Document Analysis and Recognition (DAR) systems play a main
role in transfer of data between humans and computers. Character Recognition
system is an essential part of a document analysis and recognition system.
Character Recognition systems have been developed to recognize printed texts
and handwritten texts [1]. Printed character recognition system deals with the
recognition of printed characters whereas handwritten character recognition
system (HCRS) deals with the recognition of handwritten characters.
Handwritten character recognition system (HCRS) can be categorized into two
streams namely, online HCRS and offline HCRS. In online stream, user writes
on an electronic surface by the aid of a unique pen and during the writing
process, data is captured in terms of (x, y) coordinates. Offline HCRS converts
offline handwritten text into such a form which can be easily understood by the
machines. It involves the processing of documents containing scanned images
of a text written by a user, usually on a plain paper [2]. In this kind of systems,
characters are digitized to obtain 2 dimensional images. Developing a realistic
HCRS which is capable of retaining lofty recognition accuracy is still a very
challenging task. Moreover, recognition accuracy depends upon the input text
quality.
A number of devices, including personal digital assistant and tablet PCs are
available these days that can be used for data capturing. In these systems,
characters are captured as a sequence of strokes. Features are then extracted
from these strokes and strokes are recognized with the help of these features.
Generally, a post-processing module helps in forming the characters from the
stroke(s). India is a multilingual country. Multiple scripts are used among
billions of people, mainly based on their geographic location. Based on the
literature studied so far, it is found that the majority of the work is being done
on individual character recognition in different languages like English, Bangla,
Devanagari, Gujrati etc.
Now researchers are shifting towards the recognition of Gurmukhi script.
Gurmukhi script is one most widely used script especially in the north part of
the country. So there is a need to develop a recognition system for Gurmukhi
script that will reduce the complexity of the work done in Government offices,
public banks where majority of the work is done in Gurmukhi script. Research
in this script is currently limited to single character recognition only. So there is
a huge scope in the Gurmukhi script for researchers to take the work forward
from character level to the Word level. In this article an attempt has been made
to recognize the Gurmukhi words using combined feature extraction and deep
neural networks.
International Journal of Pure and Applied Mathematics Special Issue
14751
2. Gurmukhi Script
Normally, each Indian script is having several scrupulous set of laws for the
amalgamation of consonants, vowels and modifiers. Gurmukhi script is widely
used in North part of the India. Gurmukhi script is used for inscribing the
Punjabi language. It is derived from the old Punjabi word “Gurumukhi”, which
means “from the mouth of the Guru”. Gurmukhi script has basic 35 characters,
10 vowels and modifiers, 6 additional modified consonants. There is no
conception of upper or lowercase characters in Gurmukhi script. The writing
style of the Gurmukhi script is from left to right and top to bottom. The basic 35
characters of Gurmukhi script are shown in Fig 1:
Fig.1 Consonants in Gurmukhi Script
3. Literature Review
R. Sharma et al [1] projected two stages for the character recognition. In the
first phase unidentified strokes are recognized and in the second stage the author
has evaluated the characters with the help of strokes that are found in the first
stage. Using the elastic matching a maximum recognition rate of 90.08% is
achieved.
N. Kumar et al [3] presented a technique for effective character recognition
system based on deep neural networks. The authors extracted three kind of
features namely LBP features, directional features and geometric features in
order to correctly recognize the text. U. Garain et al [4] proposed a technique
which works on fuzzy analysis and authors developed an algorithm to segment
touching characters. M. Kumar et al [5] projected a scheme to recognize
Gurmukhi characters which is based on transition & diagonal features extraction
International Journal of Pure and Applied Mathematics Special Issue
14752
and the classifier used in the work is k-NN. To determine the k-nearest
neighbors, the authors calculated the Euclidian distance b/w test point and
reference point. Rajiv Kumar et al [6] showed how segmentation can be done in
character recognition of Gurmukhi script. According to R.K Sharma et al [7]
that with the help of various features like diagonal, directional & zoning & K-
NN, Bayesian classifiers the handwriting of writers can be compared. Narang et
al [8] projected an approach based on parts or technique suitable for scene
images. The author has taken the corner points which are served as parts and
these parts are found to make a part based model. N. Kumar et al [9] has
discussed a number of feature extraction techniques & classifiers like power arc
Where “LossFunction” is an error between model output and measured response
and “crossentropy” is used to calculate neural network performance with given
target and outputs
Step7: Now the encoders and Softmax layer is stacked to make a deep network
using the following command:
deepnet = stack(autoenc1,autoenc2,softnet);
Step8: Two auto-encoders and Softmax layer are stacked to form stacked
network. The first layer of a stacked auto encoder tends to learn first-order
features in the raw input .The second layer of a stacked auto encoder tends to
learn second order features corresponding to patterns in the appearance of first-
order features.
Step9: Stacked network is trained on input and target data and accuracy is
calculated using it.
4.3.2 Testing Phase:
For testing, around 3000 images have been taken which includes broken
characters and characters written with different writing styles. For this, the
image which is to be recognized is gone through various preprocessing steps
like conversion of image from rgb to gray, gray to binary dilation, edge
detection etc. After preprocessing, all the 117 features i.e. directional features,
LBP features and regional features are extracted from the character image as
already discussed in section 4.2. Now the extracted features are passed to the
trained network to classify the character image. The recognized character is
mapped with the Devanagari text with the help of a look up table. The image for
word segmentation has been depicted in figure 7(a).The GUI for recognizing the
characters has been depicted in fig.7 (b, c, d) .The Devanagari text can‟t be
displayed directly in the MATLAB so in order to map and display the text from
Gurmukhi to Devanagari a suitable look up table has been created as shown in
International Journal of Pure and Applied Mathematics Special Issue
14760
figure 8. For mapping the text from Gurmukhi to Devanagari suitable unicodes
have been used. For example the Unicode for the character “न” is 2344 & the
unicode for character “र ” is 2352.This has been depicted in figure 10(a). In the
same way, all the images have been tested and mapped & around 99.3%
accuracy has been achieved. In order to display the whole recognized word the
recognized characters are concatenated using suitable unicodes and the
recognized word is displayed in figure 10(b).The accuracy for character
recognition can be shown through confusion matrix .Confusion matrix is a table
which is frequently used to depict the performance of a classification (or
"classifier") on a set of test data. The confusion matrix for the proposed work is
depicted in results and discussions part (section 5).
Fig.7(a) Uploaded Word Image for Segmentation
Fig.7(b) GUI for recognizing handwritten text “ਨ”
International Journal of Pure and Applied Mathematics Special Issue
14761
Fig.7(c) GUI for recognizing handwritten text “ਰ ”
Fig.7(d) GUI for recognizing handwritten text “ਕ”
Fig. 8 Look Up Table for mapping Gurmukhi to Devanagari
International Journal of Pure and Applied Mathematics Special Issue
14762
5. Results & Discussion
In order to correctly recognize the word initially the word has been segmented.
The results for the word segmentation have been depicted in figure 3(section
4.1). After word segmentation the individual characters have been recognized
and then recognized characters are concatenated in order to recognize the whole
word. The data set used for character recognition is around 6000 character
samples. The database comprises of 600 images per character and ten characters
have been used, so a total of 600*10 character samples have been used for
database. The words to be segmented have been formed using the characters
shown in figure 8. Out of 6000 samples 3000 samples have been utilized in
training & 3000 samples have been used for testing purpose. The proposed
system is achieving an accuracy of 99.3% and this accuracy has been depicted
in confusion matrix as shown in fig 9. The mapping of Gurmukhi text with
Devanagari has been shown in figure 10(a) & 10(b).
Fig. 9 Confusion Matrix
Fig. 10(a) Mapping of individual character
Fig. 10(b) Word Recognition
International Journal of Pure and Applied Mathematics Special Issue
14763
The proposed technique is compared with the state of the art approaches for
Gurmukhi character recognition and is shown in figure 11 as following:
Fig.11 Comparison of Accuracy of proposed method with previous techniques
6. Conclusion & Future Scope
In this work, an efficient method for the recognition of offline handwritten
Gurmukhi words has been proposed. Initially the word has been segmented into
individual characters in order to recognize the whole word. The classifier used
in the work is deep neural network that has been trained with 117 features. In
the proposed work, three types of features, namely LBP features, Directional
features and regional features have been extracted. Using these three types of
features the character recognition accuracy has been considerably increased.
The proposed system is achieving an accuracy of 99.3%. Furthermore, the
present work is limited to the recognition of word without modifiers. This work
can be extended to further script recognition and in text to speech applications.
References
[1] Sharma A., Kumar R., Sharma R.K., Online Handwritten Gurmukhi Character Recognition Using Elastic Matching, Congress on Image and Signal Processing (2008), 391-396.
[2] Singh, G. and Sachan, M., Offline Gurmukhi script recognition using knowledge based approach & Multi-Layered Perceptron neural network. International Conference on Signal Processing, Computing and Control (ISPCC), (2015), 266-271).
[3] Kumar N., Gupta S., A Novel Handwritten Gurmukhi Character Recognition System Based on Deep Neural Networks,
86.00%
88.00%
90.00%
92.00%
94.00%
96.00%
98.00%
100.00%
R. Sharma et al[1]
M. Kumar et al[5]
M. Kumar et al [13]
A.Aggarwal et al [16]
Proposed Technique
Accuracy
Accuracy
International Journal of Pure and Applied Mathematics Special Issue
14764
International Journal of Pure and Applied Mathematics 117(21) (2017), 663-678.
[4] Garain U., Chaudhuri B.B., Segmentation of touching characters in printed Devnagari and Bangla scripts using fuzzy multifactorial analysis, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 32(4) (2002), 449-459.
[5] Kumar M., Jindal M.K., Sharma R.K., k-nearest neighbor based offline handwritten Gurmukhi character recognition, International Conference on Image Information Processing, Himachal Pradesh (2011), 1-4.
[6] Kumar R., Singh A., Detection and segmentation of lines and words in Gurmukhi handwritten text, IEEE 2nd International Advance Computing Conference (2010), 353-356.
[7] Kumar M., Jindal M.K., Sharma R.K., Classification of characters and grading writers in offline handwritten Gurmukhi script, International Conference on Image Information Processing, Himachal Pradesh (2011), 1-4.
[8] Narang V., Roy S., Murthy O.V.R., Hanmandlu M., Devanagari Character Recognition in Scene Images, 12th International Conference on Document Analysis and Recognition, Washington (2013), 902-906.
[9] Kumar N., Gupta S., Offline Handwritten Gurmukhi Character Recognition: A Review, International Journal of Software Engineering and Its Applications 10(5) (2016), 77-86.
[10] Kumar M., Sharma R.K., Jindal M.K., A Novel Hierarchical Technique for Offline Handwritten Gurmukhi Character Recognition, National Academy Science Letters 37(6) (2014), 567-572.
[11] Adwait Dixit, Yogesh Dandawate, Handwritten Devanagari Character Recognition using Wavelet Based Feature Extraction and Classification Scheme, INDICON, 2014.
[12] Arica N, Fatos T. Optical Character Recognition for Cursive Handwriting. IEEE Transactions on Pattern Analysis and Machine Intelligence. June 2002; 24(6):801-813.
[13] Kumar M., Sharma R.K., Jindal M.K., Efficient Feature Extraction Techniques for Offline Handwritten Gurmukhi Character Recognition, National Academy Science Letters 37(4) (2014), 381-391.
[14] Kumar, M., Sharma, R.K. and Jindal, M.K., A novel feature extraction technique for offline handwritten Gurmukhi character recognition. IETE Journal of Research 59(6) (2013) pp.687-691.
International Journal of Pure and Applied Mathematics Special Issue
14765
[15] Blumenstein M., Verma B., Basli H., A novel feature extraction technique for the recognition of segmented handwritten characters, Seventh International Conference on Document Analysis and Recognition (2003), 137-141.
[16] Aggarwal, A., Singh, K. and Singh, K., 2015. Use of gradient technique for extracting features from handwritten Gurmukhi characters and numerals. Procedia Computer Science, 46, pp.1716-1723.
[17] Kumar, N. and Gupta, S., An Efficient Offline Handwritten Gurmukhi Character Recognition System Based On Combined Feature Extraction & Different Classification Techniques. International Conference on Computing For Sustainable Global Development (INDIACom), (2018), 3243-3249).
International Journal of Pure and Applied Mathematics Special Issue