Ring-shaped barcodes Bachelor Thesis August 2015 Robin Brügger University of Applied Sciences and Arts Northwestern Switzerland FHNW [email protected]Supervisor: Prof. Dr. Christoph Stamm University of Applied Sciences and Arts Northwestern Switzerland FHNW [email protected]Customer: Paul Glendenning NANO 4 U, 6060 Sarnen [email protected]Abstract: Previous work has shown that a specialized, ring-shaped barcode format is preferable for certain applications. This thesis develops a new ring-shaped, two-dimensional barcode format. The encoder generates an image of a barcode containing the payload data. The decoder recognizes the barcode and reads the payload data. The decoder is capable of dealing with perspective distortion by correcting it prior to reading the payload. Barcodes with small defective areas can still be read thanks to ECC (Error-Correction-Codes).
36
Embed
Bachelor Thesis - · PDF file1.2 Integration and software ... 8 API and project structure ... Bachelor Thesis
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Ring-shaped barcodes Bachelor Thesis
August 2015
Robin Brügger University of Applied Sciences and Arts Northwestern Switzerland FHNW
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
4 / 36
1 Introduction
NANO 4 U develops counterfeit prevention solutions for products. For certain products and applica-
tions, a ring shaped 2D barcode is considered to be more efficient than a standard square 2D bar-
code.
1.1 Project goals and requirements
The goal of this project is to develop a ring-shaped barcode format. For the specific applications of
the barcode, NANO 4 U has the following requirements:
Payload: The usable payload of the barcode should be at least 25 characters, resulting in 25
bytes.
Size: Smallest module size of at least 0.2mm or 5 times the lateral resolution of the scanner.
The maximal outer diameter of the barcode is 8mm.
Form: Ring-shaped.
The project asks for the following software components:
Encoder: Generates a new barcode as an image. Takes the payload, desired module size, in-
ner and outer radius as arguments und creates a barcode accordingly. The encoder is de-
scribed in the chapter Encoder (Barcode generator).
Decoder: Detects a barcode in an image and is able to recover the original payload data. The
decoder is covered in detail in the chapter Recognition and decoding.
Tests: It is not possible within the scope of this project to evaluate the results of the ring-
shaped barcodes on real products. The robustness of the algorithm shall therefore be tested
using software to distort and add noise to the barcodes.
1.2 Integration and software
Because it allows for a platform-independent solution
and easy integration into other software systems, C++
was chosen as the programming language for this
project.
I selected OpenCV as computer vision library because
of its excellent documentation, vast functionality and
my good experiences with it.
1.3 Existing circular barcodes
ShotCode is currently the only existing circular 2D
barcode1. An example is shown in Illustration 2. It was
developed to share links and can be read with low
resolution mobile phone cameras or webcams. Today,
it has been replaced by QR-Codes.
1 Barcoding Connected (2009): SHOTCODE BARCODE: THE CIRCULAR 2D BARCODE FOR MOBILE TAGGING. http://blog.barcoding.com/2009/02/shotcode-barcode-the-circular-2d-barcode-for-mobile-tagging/ (Accessed August 5, 2015)
Illustration 1: Shotcode4
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
5 / 36
ShotCode is not suitable for this project for two reasons:
Limited payload
Depending on the source, a ShotCode is limited to 492 or 403 bits (5 to 6 bytes) of data. How-
ever, NANO 4 U requires about 25 bytes.
Centre is not empty
A ShotCode uses the area in the centre of the circle. This area is not available for use on the
end-user products and has to be left blank.
1.4 Schedule
The project started on 24 February 20154with the deadline on 15 August 2015. Due to the unfore-
seeable challenges of decoding a barcode, it was not possible to break the decoding process down
into individual steps and schedule them separately. Illustration 2 shows the planned and effective
project schedule.
2 Wikipedia (2013): ShotCode. https://de.wikipedia.org/wiki/ShotCode (Accessed June 7, 2015) 3 Wikipedia (2014): Shotcode. https://en.wikipedia.org/wiki/ShotCode (Accessed June 7, 2015) 4 Mudie, Stuart: Shotcode. Licensed under CC BY-SA 2.0 via Wikimedia Commons - https://commons.wikimedia.org/wiki/File:Shotcode.png#/media/File:Shotcode.png
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
6 / 36
Illustration 2: Planned and effective project schedule
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
7 / 36
The workload for the bachelor’s thesis should be approximately 360 hours, spread over the whole
project duration. Because of lectures, exams and special occasions such as the project week, the
scheduled work hours varied from week to week. Illustration 3 shows the planned and effective work
hours per week. During the course of the project, slightly fewer hours were invested than planned.
However, this was made up in the last two weeks of the project, which had been allocated as re-
serve.
Illustration 3: Planned and effective work hours
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
8 / 36
2 Ring-shaped barcode format
In this chapter the developed barcode
format is explained. Illustration 5 is an
original barcode as it is generated by
the encoder. Illustration 6 is the same
barcode, but areas for recognition, per-
spective correction and timing are high-
lighted. The coloured areas serve the
following purposes:
Yellow: Solid black ring for lo-
cating the barcode in an image.
Orange: Markers for a rough
perspective correction. They are
arranged in a perfect square.
Green: The startmarker defines
the beginning in the otherwise
continuous circle. It is also used
to determine the outer radius of
the barcode.
Red: The vertical zebra pattern
determines the vertical timing
of the rings (how far apart the
rings are).
Violet: The radial zebra pattern
determines the angles between
the modules. It is also used to
achieve an accurate perspective
correction, therefore the radial
module count is always divida-
ble by 8. Thus, it is possible to
select four modules of the zebra
pattern which lay in a perfect
square, facilitating the perspec-
tive correction calculation.
Blue: Area used for the payload.
The data area can be interpret-
ed as a curved table, addressing
modules via their x and y index.
Illustration 7 shows how the ad-
dressing of the payload area is
organised. To make use of the
bigger modules toward the outside of the barcode, these modules are filled with payload
first.
Illustration 4: Ring-shaped barcode generated by the encoder
Illustration 5: Ring-shaped barcode with special areas highlighted
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
9 / 36
Illustration 6: Addressing of the payload modules
2.1 Usable payload and limitations
The barcode used in Illustration 5 and Illustration 6 was generated following the size requirements
described in the chapter Project goals and requirements and is therefore considered realistic in the
application at hand. It has a maximum payload of 56 bytes. However, some of that is consumed by
the protocol described in the chapter Protocol.
The maximum payload the ring-shaped barcode supports is 234 bytes. This is due to constraints of
the payload protocol. The barcode design itself poses no upper bound for the payload size.
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
10 / 36
3 Encoder (Barcode generator)
The encoder generates barcodes as an image. It takes the payload and three sizing arguments as
input.
3.1 Arguments
The size and shape of the barcode is determined by
three basic parameters, measured in pixels. Illustra-
tion 8 visualizes the meaning of the parameters.
Minimal module size (red): The minimal
size of a single module. This is also the
thickness of the individual rings. Technical
name: moduleSizePixels.
Minimal radius (blue): Distance from the
centre of the barcode to the inner perspec-
tive markers. The perspective markers are
the smallest module-sized structure in the
barcode. Technical name: minRadiusPixels.
Maximum radius (orange): Distance from
the centre of the barcode to its outermost
point. Technical name: maxRadiusPixels.
The barcode used in Illustration 8 was generated
using moduleSizePixels=8, minRadiusPixels=80, maxRadiusPixels=160.
3.2 Implementation
Implementing the encoder was a straightforward process. It uses the OpenCV drawing functions,
particularly the circle() function which is able to draw circle sectors. However, OpenCV does not pro-
vide a function to draw ring segments. Therefore, the barcode is rendered from the outside to the
inside. Firstly, the modules in the outermost circle are drawn as circle sectors. Upon completion of
that ring a slightly smaller white circle is drawn, transforming the already drawn circle sectors to ring
sectors. This process is repeated until the barcode is completely rendered. Illustration 9 provides
pseudocode for this process.
Illustration 7: Basic barcode encoder parameters
val minRadiusPixels, maxRadiusPixels, moduleSizePixels
The vertical zebra pattern defines how many rings a
barcode has and how far apart they are spaced. In
Illustration 25 the vertical zebra pattern is highlighted
in red.
Input
Accurately perspective corrected image
Radial module count
Angle of the startmarker
Output
Number of rings
Radius of the innermost ring
Radius of the outermost ring
Direction of reading the barcode: clockwise or
anti-clockwise Illustration 24: The area of the vertical zebra pattern is highlighted in red
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
21 / 36
4.8.1 Algorithm
The vertical zebra pattern is always situated next to
the startmarker. Because the barcode has possibly
been mirrored, it could lie either to the right or to
the left of the startmarker. Illustration 9 depicts how
two lines are shot from the barcode centre through
the potential locations of the vertical zebra pattern.
To evaluate if the line goes through a zebra pattern
the alternating continuous stretches of black and
white pixels are measured. Because all rings are
equal in thickness, a perfect zebra patterns would
have X continuous black pixels followed by X contin-
uous white pixels followed by X continuous black
pixels again. If no continuous stretch of equally col-
oured of pixels significantly deviates in length from
the average, the vertical zebra pattern has been
found.
The vertical zebra pattern also determines the read-
ing direction of the barcode payload. If it is to the right of the startmarker, the reading direction is
clockwise.
4.9 Evaluation of the payload modules
This is the last step of the decoder pipeline. It reads the actual payload of the barcode.
Input
Perspective corrected input image
Centre of the barcode
Angle of the startmarker
Number of radial modules
Number of rings carrying payload data
Radius of the innermost and outermost
ring
Reading direction (clockwise or anti-
clockwise)
Output
Payload data of the barcode
In Illustration 27, the area of the barcode’s pay-
load is highlighted.
4.9.1 Grid
With the input, it is possible to draw a grid over
the barcode in the input image. Illustration 28
shows debug output where a grid was drawn
using the input parameters of this processing
stage. To keep the grid visualisation readable a
line was drawn only through every second radial
Illustration 25: Two test lines are shot form the centre of the barcode through the potential locations of the vertical zebra pattern
Illustration 26: The payload area is highlighted in blue
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
22 / 36
module. At every intersection of the grid a module shall be evaluated as either black or white.
4.9.2 Reading a module
A total of five pixels are sampled for each module in a cross-shaped pattern as shown in Illustration
28.
Illustration 28 Cross-shaped sampling pattern
Each pixel votes for the module to be either black or white according to its colour. Because the sam-
pling is performed on a binarized image and an uneven number of pixels are sampled, a module’s
colour is never ambiguous, it can always be classified as either black or white.
Illustration 27: Grid matching the barcode modules
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
23 / 36
5 Protocol
The payload of the barcode is structured according to a simple protocol as shown in Illustration 29:
Illustration 29: Payload protocol
Field Size in bytes Description
Length 1 Contains the length of the body including the space used for the error correction codes. Because this field has a fixed length of one byte, the maximum size of the body of a barcode is restricted to 255 bytes.
Data variable The data encoded in the barcode
ECC 20 20 bytes are always reserved for error correction
5.1 Error correction codes
The open source library Rscode10 was used in this project. It implements Reed-Solomon error correc-
tion. Rscode performs error correction on 8-bit blocks. The amount of error correction codes has
been set to 20 bytes because Rscode needs this as compile-time argument. 20 bytes of error correc-
tion codes allows to fix up to 10 faulty blocks of 8 bit each.
10 Minsky, Henry: Welcome to http://rscode.sourceforge.net/. http://rscode.sourceforge.net/ (Accessed August 5, 2015)
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
24 / 36
6 Tests
6.1 Tests using a webcam
During the development phase, most tests were done using a laptop webcam as scanner to read a
barcode shown on a mobile phone display. The scanning process is fast and reliable enough to cause
no inconvenience to the user, even under not ideal conditions such as a considerable amount of per-
spective distortion.
Because the results are dependent on the actions of the person holding the barcode, it is impossible
to reliably quantify the success rate and overall quality with this test method.
6.2 Batch testing robustness
To measure the quality and capabilities of the decoder in a reproducible way, a batch testing tool has
been developed. It generates a barcode, adds some defects to it and then checks if it can be correctly
decoded. To obtain reproducible results, each test is run several times.
6.2.1 Added defects
The test tool is capable of adding three types of defects to a barcode:
Perspective distortion
The perspective distortion can be controlled in percent. 0% means no distortion, with 100% it is pos-
sible that all four corners of the barcode image are mapped to a single point in the distorted image.
For examples, refer to Illustration 30.
Illustration 30: Perspective distortion added by the test tool. From left to right: 0% distortion to 50% distortion in 10% incre-ments.
Salt and pepper noise
This defect adds random black and white pixels to the image and is controlled by a percentage value.
At 100%, the image is fully covered by noise. For examples, see Illustration 31.
Illustration 31: Salt and pepper noise added by the test tool. From left to right: 0% to 20% noise in 10% increments.
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
25 / 36
Area destruction
This defect adds random ellipses to the barcode, obscuring some areas. It is controlled by a percent-
age value. At 100%, the cumulative area of all ellipses is equal to the image size. For examples, refer
to Illustration 32.
Illustration 32: Random ellipses obscuring parts of the image added by the test tool. From left to right: 0% coverage to 2% coverage in 1% steps.
6.2.2 Tests
All tests were conducted with three different barcode sizes:
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
29 / 36
Component Specification
Processor i7-3520M dual core, 2.9 Ghz
RAM 8 Gigabyte
Operating System Windows 7 64bit
Illustration 40 shows the encoding and decoding times using multiple barcode sizes. The encoding
process is about 10 times faster than decoding process. Not much time was spent during this project
on optimisations as performance was deemed fast enough. It is expected that there are some quite
low hanging fruits when it comes to performance optimisations.
Illustration 40: Encoding and decoding performance of multiple barcode sizes
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
30 / 36
7 Further work and improvements
7.1 Scanning of actual coded products
Because of the long lead time on producing real products with the barcode on them, it was clear
from the start of the project that tests with scans of real products is out of scope for this project.
Furthermore, it is not clear what kind of defects a barcode on a product would possibly have and
which areas are most susceptible to damage. In agreement with the client it has been assumed that
the smaller modules on the inside of the barcode are more likely to be unreadable. Data is therefore
preferentially placed in the bigger modules towards the outside of the barcode. It is possible that real
world experiences with the barcode on products call for changes to the barcode format.
7.2 ECC license
The ECC library Rscode12 that was used in this project is licensed under GPL (General Public Li-
cense)13. The GPL has a strong copyleft, requiring the whole project code to be licensed under GPL as
well. Because this is possibly not desirable for NANO 4 U, two versions of the project are delivered:
Version with ECC, licensed under GPL
Version without ECC, not specifically licensed
The reason a GPL-licensed library was used is was not possible to source a C++ ECC library licensed
under a liberal license. For the future, there are multiple options:
Obtain a commercial ECC library
Port the ECC library used in ZXing from Java to C++. ZXing is licensed under the quite liberal
Apache 2.0 license14.
7.3 Parametrizing the decoder
The only argument the decoder takes is the input image. All information it requires to decode the
barcode is extracted from the barcode itself. This includes the number of rings and the number of
modules in each ring.
It is conceivable that supplying information about the basic shape of the barcode to the decoder
could improve robustness, especially if parts of the barcode are unreadable. Possible arguments are
primarily the already mentioned number of rings and number of modules in a ring. Attempts to make
use of such arguments have been made during the course of this project. However, it proved to be
more complex than expected because of the complex decoder pipeline and dual-use barcode fea-
tures such as the radial zebra pattern which is used for perspective correction and also to determine
the number of modules in each ring.
7.4 Avoid data looking like a startmarker
There is a chance that the payload data in a barcode forms a pattern identical to the startmarker. If
two starmarker patterns are present, it is possible that the decoder selects the wrong one and there-
fore the payload cannot be read. The chance of a pattern equal to the startmarker occurring in a
barcode is dependent on the number of rings and the number of modules per ring. Due to the zebra
12 Minsky, Henry: Welcome to http://rscode.sourceforge.net/. http://rscode.sourceforge.net/ (Accessed August 5, 2015) 13 Wikipedia (2015): GNU General Public License. https://de.wikipedia.org/wiki/GNU_General_Public_License (Accessed August 5, 2015) 14 ZXing (2014): License Questions. https://github.com/zxing/zxing/wiki/License-Questions (Accessed August 5, 2015)
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
31 / 36
pattern around the outside of the barcode, a startmarker-lookalike can only occur at every second
angular module. The chance of data looking identical to the startmarker for each angle is:
𝑝𝑎𝑛𝑔𝑙𝑒 =1
23∗𝑟𝑖𝑛𝑔𝐶𝑜𝑢𝑛𝑡
It is likelier that a startmarker- lookalike appears next to the real startmarker because the white
space to the side of the startmarker is already there:
𝑃𝑔𝑙𝑜𝑏𝑎𝑙 =1
22∗𝑟𝑖𝑛𝑔𝐶𝑜𝑢𝑛𝑡
For a barcode that has 8 rings, the chance of a startmarker-lookalike is quite small (~1:50000). How-
ever, if the number of rings is reduced this quickly becomes a big problem.
A possible solution is to check the generated barcode for startmarker-lookalikes. If such a startmark-
er-lookalike is detected, a random bitmask is selected. To get rid of the startmarker, an XOR-
Operation is performed on the payload body and the bitmask. To be able to decode the barcode
again the used bitmask has to be encoded into the barcode payload as meta information.
7.5 Error detection
Although ECC has been implemented, its possibilities are limited. If too many modules have been
incorrectly sampled, ECC can no longer correct all errors and the payload read by the decoder is in-
correct. Currently, this goes undetected. To fix this, a checksum over the barcode’s payload could be
used.
7.6 Error correction in length field
Currently, the length field of the payload protocol is not protected by any error correction mecha-
nism because the ECC decoder itself needs to know the length of the ECC-protected payload. Due to
the lack of error correction, any reading error in the length field will lead to an unreadable barcode. A
possible solution to this problem would be transmitting the length field three times redundantly to
compensate for reading errors.
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
32 / 36
8 API and project structure
8.1 Project structure
The project workspace is structured in six subprojects:
Project Description
DecoderLib Library for decoding a barcode
EncoderLib Library for generating a barcode
DecoderVideoDemo Sample application using the decoder library. The webcam captures con-tinuous frames which are handed to the DecoderLib.
Encoder Sample application using the encoder library.
BatchTestTool Command line tool used to test the decoder. Uses the EncoderLib to gener-ate barcodes which are deformed and damaged in multiple ways before trying to decode them using the DecoderLib.
Rscode Third party ECC library (http://rscode.sourceforge.net/)
8.2 API
The EncoderLib and DecoderLib each provide a simple API.
8.2.1 EncoderLib
The EncoderLib exposes the method encode(). It creates a new barcode as an image:
Mat BarcodeEncoder::encode(BarcodeEncoder::EncoderConfiguration, StringDataProvider)
If the data provided is longer than the capacity, an exception is thrown. The EncoderConfiguration
determines the size and shape of the barcode by specifying:
Inner radius
Outer radius
Module size
It also provides a method to determine how many bytes of payload can be placed in the barcode with
that specific barcode configuration:
int BarcodeEncoder::EncoderConfiguration::capacityBytes()
8.2.2 DecoderLib
The Deocoderlib exposes the method decode():
vector<char> Decoder::decode(Mat M)
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
33 / 36
9 Personal reflection
It has been a very interesting but also challenging project. A ring-shaped barcode is without prece-
dent, which makes this project quite innovative. Developing the first ring-shaped barcode format also
meant going new ways in detecting the barcode.
Although I attended a C++ module during my studies, this project was the first time I used C++ for a
productive project. Working with C++ provides tremendous opportunities, but frankly has often
been frustrating for a newbie like me. Compiler errors are hard to interpret and I had my fair share of
linker problems while transitioning to a multi-project workspace. However, this is made up by C++’s
versatility, execution speed and the possibility to integrate it with basically any software on any op-
erating system.
Overall, I am happy with the outcome of the project, although time was tight towards the end. There
are still some small things that could be done to improve the decoding process. The decoder works
well when decoding a frame captured with the webcam, but it is difficult to anticipate the specific
challenges when the barcode is scanned from a real product. I reckon it is likely that some adjust-
ments are needed, but I feel confident that the overall concept will prove effective.
A special challenge of this project was that the documentation as well as the communication with the
customer was in English. I consider my command of English quite good, but it has proved that it is
much harder to express complex technical relationships in a language other than one’s mother
tongue.
9.1 Acknowledgments
I would like to express my gratitude to my supervisor, Prof. Dr. Christoph Stamm, who has guided me
through this project. We had a lot of interesting discussions which often sparked new and creative
ideas.
Furthermore, I would like to thank NANO 4 U and Paul Glendenning for providing me with this inter-
esting and challenging project.
Finally, thanks to Dr. Lorenz Frey, with whom I had an interesting debate about error correction
codes and their placement within the barcode.
Bachelor Thesis (IP6) Ring-shaped barcodes Robin Brügger
34 / 36
10 Sources
Barcoding Connected (2009): SHOTCODE BARCODE: THE CIRCULAR 2D BARCODE FOR MOBILE