Seminar Report

A Seminar Report on

BAR CODE RECOGNITION IN COMPLEX SCENES BY CAMERA PHONES

Submitted in partial fulfillment of the requirementsfor the award of B.Tech Degree in

Electronics And Communication Dept.by

Jinesh R S

AUGUST 2010

ADI SHANKARA INSTITUTE OF ENGINEERING &TECHNOLOGY

(An ISO 9001 Certified Institution)VIDYA BHARATHI NAGAR, KALADY, KERALA

1

ADI SHANKARA INSTITUTE OF ENGINEERING &TECHNOLOGY

(An ISO 9001 Certified Institution)VIDYA BHARATHI NAGAR, KALADY, KERALA

CertificateCertified that this is the Bona fide Record of the SEMINAR entitled

“BAR CODE RECOGNITION IN COMPLEX SCENES BY CAMERA PHONES”submitted during the year 2010 in partial fulfillment for the award of Bachelor of Technology in Electronics & Communication engineering by the candidate with

Name: Jinesh R S Register No: 56028

Branch: Electronics and Communication

Staff In-charge H.O.DDate…………....

2

ACKNOWLEDGEMENT

It is a great pleasure to express my sincere gratitude to Dr.S.G.IYER, Principal, ASIET, for his guidance advice and encouragement throughout this endeavor. I'm also grateful for granting me all the facilities for the seminar presentation.

I'm greatly indebted to HOD, Electronics & Communication Dept., for the help and guidance at stages of my work.

I take this opportunity to extent my sincere gratitude and thanks to Ms. Anju, seminar guide for her guidance & help in connection with this seminar.

I wish to express my gratitude and sincere thanks to all those who helped me by giving their valuable advice and guidance.

3

ABSTRACT

Mobile phone with camera functions are capable of capturing image and processing

task. Mobile phone can use for capturing barcode with their cameras and decoding them with

software running on the phone. Barcode is a fast, easy and accurate automatic data collection

method. Bar code enables product to be tracked efficiently and acurately at speeds not possible

using manual data entry system. Quick response code is a 2D bar code which has been widely used

in industrial information tagging applications where high data capacity and robust error corrections

are required. In deal with the infuence by the different light conditions and noise a recognition

algorithm is used. First, we combine filter, rough location and binarization to erase noises and

reduse computation. Then, we propose an accurate location and orientation, skew correction with

the help of alignment pattern algorithm to build a bar code grid. Finally, we use error correction and

decoding process to generate the result.

4

5

LIST OF FIGURES

FIGURE 1………………………………………………………..4

FIGURE 2………………………………………………………..6

FIGURE 3……………………………………………………….14

FIGURE 4……………………………………………………….15

6

CHAPTER 1

INTRODUCTION

Modern mobile phones are high resolution colour displays, they support different

standards of wireless networking, and they have reasonable processing power. Although still

primarily used for voice communication, with the inclusion of digital cameras these devices have

become a potential platform for machine vision application such as bar code recognition.

In recent years, advanced technology has succeeded in continuously producing

smaller yet smarter devices.Now mobile phones can implement many new kinds of applications

such as taking photos, and movie shooting by using embedded camera devices. So an interesting

approach is capturing bar code with their cameras and decoding them withsoftware running on the

phoneIn August 2006, 82.4 percent of the respondents whohad camera phones with QR Code

(Quick Response Code) readers used camera phones with QR Code. But previous research work has

shown that recognition of 2D barcode in mobile phone is very difficult because of the high noise,

non-uniform illumination, skew distortion, low resolution and optical blur. It is very difficult to

robustly extract accurate features such as edges and peaks of the bars and spaces from the barcode

images taken by a camera phone.

Many new algorithms are presented for dealing with 1D bar code in complex

situation.Ohbuchi et al.presented an algorithm capable of the real-time recognition of barcodes on a

mobile phone.Sun et al.introduce an algorithm to analyze and correct the distorted image of QR

7

Code. The algorithm includes gray-scale image transformation, binary image, canny edge detection,

external contours finding, inverse perspective transformation and cell grids generating. In this

method, binary image and edge detection is very important for following decode. But it is very

difficult to binaries image and extract accurate edge in the blurred and damaged bar code images.

Here we describe a new image recognition algorithm which applied to capture image in various

light condition.

8

CHAPTER 2

WHY USE MOBILE PHONE

BAR CODE READERS

Too Expensive Scanners.

Have No Other Use.

Too Bulky.

Workers Have To Be Trained.

MOBILE PHONES

Affordable Prices And Common.

Have Various Uses.

Too Handy.

No Training.

9

CHAPTER 3

BAR CODE

There are two types of barcodes are present -One dimensional and two dimensional.

1D barcodes have low data capacity, so we commonly use 2D bar codes. There ar two standardised

2D barcodes-Data matrix and quick response(QR) code.

3.1 DATA MATRIX

Data matrix is one of the most well known 2D bar code standards. It is widely used

in the automotive, aerospace and computer manufacturing industries, for large data capacity

labelling, such as direct part marking and package marking.

Figure 1

It consist of a solid- line locator( the two solid line), a patterned-line locator( the two

alternating dark and light patterned lines), the inside area with encoded data in binary, and aquiet

10

zone( a blank area) surrounding the whole tag.

Its capacity is 2334 alphanumeric characters or 1556 8-bit ASCII characters,

encoding any data. It employs the Read- Solomon error correction to enable accurate reads even

when substential pats of the code are distorted.

11

3.2 QR CODE

It comprised of the following patterns: finder pattern, timing pattern format

information, alignment pattern, and data cell. The finder patterns located at the three corners of the

symbol intended to assist in easy location of its position, size and inclination. Three dark-light-dark

squers are overlapped in every finder pattern, and the dark- light ratio is 1:1:3:1:1. The timing

patterns provide the secondary information which can help us to locate the symbol, decide the

rotation direction of symbol and the width of module.The location of alignment pattern center is

important for correcting the deformed shape.

Figure 2

12

3.2.1 Features of QR code

1) High capacity encoding of data

QR code has high capacity encoding of data, its maximum symbol can encode7089 numeric data or

4296 alphanumeric data.

2) High spead reading

Adapted with ccd reading, it can recognize more qr code symbol per second than pdf417 symbol for

encoding same data capacity.

3) Chinese encoding capability

Chinese and Japanese characters are represented by a two byte compination in other two-

dimensional barcode. But in QR code there is a specific chinese mode, it can use 13 bits encoding a

chinese character.

4) Readable from any direction from 360 degree

13

CHAPTER 4

VARIOUS ALGORITHMS

Many new algorithms are presented for dealing with 1D bar code in complex situation.

InImage Pre-processing for Bar Code Detection in Mobile Devices , a de-noising and vign-

etting elimination is performed first, followed by the start and the ending points detection,

finally psf estimation and line de-blurring is adopted.

A statistical method is proposed in which, wavelet-base and knowledge-base approach is

used to locate the bar code area and segment the bar code character

Because of the feature of 2D bar code is different from 1D bar code, the above algorithm used in 2D

bar code is not well.

Ohbuchi et al. presented an algorithm capable of the real-time recognition of barcodes on a

mobile phone. The most important drawback of the method is the fact that it has been hand

tailored for one certain hardware device. It relies on access to a powerful but also very spe-

cific hardware element, which is not accessible by normal application developers. Besides

that, the algorithm relies on two specific conditions: First is prior to the code’s decoding, the

barcode’s position is detected using a spiral scanning algorithm that runs on the device’s sig-

nal processor. Second is this algorithm makes the assumption that the point in the middle of

the screen is located in the code. Thisassumption is not realistic in usually environment. So

the algorithm is difficult to be used in 2D bar code decoding.

14

An algorithm capable of recognizing the PDF 417 barcode is proposed in real time with a

mobile phone. This method includes three steps. The first step detects the code region using

the Otsu algorithm and the Least Square Method. The second method searches for the cut-

off rules with a scanning approach. In third step symbol characters are segmented from the

original image. So the successful binarization of the code areas during the first step is the

most key step in this algorithm. This is mean that the recognition result will rely on the ef-

fect of the Otsu method. But the Otsu method could be failed in complex lighting condi-

tions, including highlight spots, low contradistinction, nonhomogeneous lighting, and vari-

ous mixed conditions . Additionally, many constraints are added in this method, so the ap-

plication scope is decreased.

Sun et al. introduce an algorithm to analyze and correct the distorted image of QR Code.

The algorithm includes gray-scale image transformation, binary image, canny edge detec-

tion, external contours finding, inverse perspective transformation and cell grids generating.

In this method, binary image and edge detection is very important for following decode. But

it is very difficult to binaries image and extract accurate edge in the blurred and damaged

bar code images.

An efficient scheme for 2D bar codes decoding is proposed in [9], which specific for low

quality cameras. The algorithm includes Hough transformation, BCRL(location with hori-

zontal and vertical waveforms), grid vibration (Bayesian decision), and deformable model

(psf estimation) and decode. In this method, the recognition time will be cost more than

above because there is no binarization operation.

15

CHAPTER 5

RECOGNITION ALGORITHM

It consists of several steps, gray scale image conversion, filter and location,

binarization, and normalization the skew shape, error correction and decoding. The input is a color

image which is captured by mobile phone and the output is a decoding result.

5.1 GRAY CONVERSION

QR Code symbol is captured by mobile phone with camera, and images are captured

in RGB 24bit format by most phones, but QR Code symbol is a set of dark and light pixels. It is

needless to deal with color information and the gray image calculated quickly with little space, so

gray conversion is needed to do firstly.

16

5.2 FILTER AND LOCATION

Perform morphological closing on the gray-scale image from above step. This will

smooth the bar code and reduce the influence of noise. Before binarizing the image, a barcode area

in image is extracted that our approach can reduce computational power and increase the accurate

rate of binarizing bar code. There are several methods to detect code area, such as pattern

searching, texture segmentation, wavelet analysis etc. Compared with the background, the gray

change frequency of the code area is quite high, so we can easily get the rough bar code area in

frequency domain.

17

5.3 BINARISATION

Binarization is a necessary step in recognition bar code, and it is used in most

recognition algorithm . Selection of a proper binarization method is critical to the performance of

barcode recognition system. In binarizing an image, a simple and popular method is threshold.

Ohbuchi propose an improving global threshold method. First the image was divided into nine

blocks. Then the gray histogram is calculated for each block, and the gray value was sorted. The

middle value was choosing as the threshold of each block. Finally the smallest value of these

middle values is the global threshold of this whole image. This method is existed two drawbacks:

(1) the barcode symbol must be in the center of the captured image, otherwise the method will

failed. (2) This method results in excessive segmentation. Using a global threshold method, the

resulting binary image will be very bad, if an image was in variable lighting conditions. In this case,

a local threshold method performs better. The main problems with a local threshold method are hard

to set a right window size, eliminate the block effect, and reduce the execution time.

The bar code area is getting in previous step, so the non-uniform illumination has

little effect on bar code area comparing with the whole image. Among more than 20 threshold

methods, it is concluded that Otsu’s method is the best, which chooses the threshold that minimizes

within-group variance. So we used the Otsu’s technique as the threshold algorithm.

18

5.4 ACCURATE LOCATION AND ORIENTATION

After binarizing the code area, the accurate location and orientation is implemented.

There are three identical position detection patterns located at three of the four corners of QR Code.

Three dark-light-dark squares are overlapped in every finder pattern, and the dark-light ratio is

1:1:3:1:1. There is the off chance that similar graphic existed in barcode symbol. So the ratio area

should be quickly find. But when one of the finder patterns is partially dirty or damaged, we can use

the timing patterns. The timing patterns provide the secondary information which can help us to

locate the symbol, decide the rotation direction of symbol and the width of module. Rotated images

are handled after orientation in common. But the interpolation operation is used in rotating

operation. The amount of calculation is great and is not accurately. So we can skip this step,

modular distance offset algorithm was used without rotating symbol. After located the finder

patterns, the module width and height and the angle of rotation were known. We get the grid

moving along the line which connects the central of three finder patterns. This method avoids the

rotation and interpolation, increases the computing speed.

19

5.5 CORRECTION THE DEFORMED SHAPE

The input image has a deformed shape because of being captured from the embedded

camera device., An inverse projection transformation is used to normalize the code sharp. But the

specific feature of QR Code symbol is not used in this method, so the computation complex degree

algorithm is high. Different from the other 2D barcode, in order to correct the contorted QR Code

symbol, there are many alignment patterns in QR Code symbol. With the version increased, the

number of alignment patterns was added, as shown in Fig. When the version is 3, the number of

alignment patterns is 1, but when the version is 7, the number of alignment patterns is 6. Link the

central point of the alignment patterns and three position detected patterns, the small sampling grid

is formed. In a small sampling grid, distortion is little, so we combine the affine transformation and

the central coordinates of alignment pattern to correct the deformed shape.

Figure 3

The location of alignment patterns center is important for correcting the deformed shape.

In international standard, the location algorithm of alignment pattern center will be failed when the

bar code symbol is deformed, because the 70 percent of

20

estimate location of alignment pattern center is inside the alignment pattern, the other 30 percent is

outside of alignment pattern. So we proposed an eight-direction- estimation algorithm, it can

improve the accurate rate, and 96 percent of alignment pattern center was successful location.

Assume the point A is the estimation location, as shown in Fig, we can search the eight

direction of point A in order to find the area of approximately dark- light ratio is 1:1:1:1:1. An

additional condition is that the eight neighbor points of alignment pattern center are all black, so the

point C is omitted. Finally, we can get the point E is the alignment pattern center.

Figure 4

21

5.6 ERROR CORRECTION AND DECODE

When the finder patterns and alignment patterns were located successfully, the

segment is easy to do, and the grid is easy to generate. Then the pixels were getting to ready

decoding.

The error correction and decoding process is the last step of recognition barcode. It

employs the Reed- Solomon error correction to enable accurate reads even when substantial parts of

the code are distorted. Decoding is just the reverse of the encoding procedure and the decoding

steps can reference to international standard of QR Code.

22

CHAPTER 6

EXPERIMENTAL RESULTS

This application is implemented in our test hardware architecture. An Intel® XScale

processor of Motorola, Motorola A1200r is run at up to 312MHz. In addition to the processor core,

its associated caches and memory management units, including additional 512MB Micro-SD, and

camera interface (twenty thousand pixel CMOS sensors) and LCD controller etc.. The barcode im-

age processing algorithm is developed with windows mobile 5.0 and embedded in this mobile.

A sample set of 100 unconstrained QR Code barcode image were tested to verify the pro-

posed recognition methods. All original images are acquired by camera phone. The images captured

have a resolution of 640*480 pixels. The experiments were done in real conditions. In order to eval-

uate the validity of our method, these images were divided into four groups: the first group barcode

image was captured in normal lighting conditions; the second group image was captured with high-

lighted spots and background noise; the third group image was captured in low contrast or non-ho-

mogeneous lighting conditions; the fourth group image was captured with deforming barcode.

In order to confirm the effectiveness of the proposed recognition method, which including

filter location, binarization, accurate location and orientation of finder pattern, alignment patterns

location from eight directions, and decoding. We evaluated it with whole recognition performance.

For the evaluation, the barcode image was captured and decoded in mobile phone. The recognition

rate and executing time were compared in Table . The experimental results have shown that the bar-

code recognition rate is higher (the first group image), average 97%, and the recognition time is lit -

23

ter in the normal lighting conditions, about average 400ms. With the lighting conditions became

weak or uneven, the captured barcode image can be recognized slower and the recognition rate de-

creased. The four group is the deformed barcode image, so the recognition rate is the slowest, de-

code an image will cost average more 500ms. In a whole, the experiment verified the proposed

method is efficient, and higher recognition rate was obtained.

Compare of different group

Group The average of total

recognition time(ms)

Recognition rate

The first group 380 97.5%

The second grop 430 95%

The third group 470 91%

The four group 540 87%

24

CHAPTER 7

RECOGNITION TEST

In order to conform the effectiveness of the proposed recognition method, which

including multilevel thresholding algorithm, determination of the location and orientation of the

finder pattern, located alignment patterns from eight directions, and decoing,we evaluate it with the

whole recognition performance. For the evaluation, we captured more than 70 barcode image and

run recognition algorithm taken by our test hardware.The recognition result were compare with two

other techniques, the reference algorithm of QRcode standard, and Ohbuchi’s method. The

experimental result are summarised in table. Experiment have shown that over 98% of the captured

image can be successfully recognized with the automatic recognition method. Higher recognition

rate was obtained with our algorithmas compared with other methods.

Binarization time Total recognition

time(ms)

Recognition rate

Reference method of

standard

40 90 75%

Ohbuch’s method 20 60 83%

Our method 30 70 98%

25

CHAPTER 8

CONCLUSIONS AND FUTURE WORK

,

The barcode recognition system discussed a simple but unique mobile application

system that has a huge potential to be widely used due. This application facilitates consumers with

quick and easy way to verify product’s status by enabling them to capture the barcode images rather

than typing in the barcode number. In the future we will enhance our system to include more

applications that are related to consumer industry, not necessarily related with Halal products.

Usability and user-testing is to be carried out in the later stage. We will enhance our work to find the

best way to solve the problem of highly distorted and low resolution bar code images by improving

the work .

26

REFERENCE

H. Kato. T. Tan. 2D Barcodes for mobile phones”, 2005 2nd International Conference on

Mobile Technology, Applications and Systems, 15-17 Nov.8 pages. 2005.

Wang, Kongqiao, et al. “ Barcode reading from images capture by camera phone”, 2005 2nd

International Conference on Mobile Technology, Applications and Systems. 15-17

Nov.2005.

Yue Liu, Mingjun Liu. “ Automatic Recognition Algorithm of Quick Response Code Based

on Embedded System”, Proceedings of the Sixth International Conference on Intelligent

Systems Design and Applications(ISDA’06), Volume 02, Nov.2006,pp. 783-788.

“Decoding with Camera-Equipped Mobile Phones”,Proceedings of the Fifth Annual IEEE

International Conference on Pervasive Computing and Communications Workshops

(PerComW'07), 19-23, March,2007,pp.597–600

Hao Wang, Yanming Zou. “Camera Readable 2D Bar Codes Design and Decoding for Mo-

bile Phones”, IEEE International Conference on Image Processing, ICIP06,8-11 Oct.,

2006,pp.469-472.

P.K. Sahoo, S. Soltani, and A.K.C. Wong. “A survey of thresholding techniques”, Computer

Vision, Graphics, and Image Processing, vol. 41, issue 2, Feb., 1998,pp.233-260.

27