Scanning Woes and War Stories

Toke Eskildsen [email protected] ELAG 2014 – Scanning woes and war stories - 1/28

Scanning woes and

war stories

ELAG 2014

Toke Eskildsen IT nerd (boss says “System Architect”)


State and University Library

Denmark

“Everything onlinein 2020”

- Vision


We would like plentiful, raw, visible, solid pixels


Zoom

Not like this! Like this!


Histogram

Reference


Adjust Color Levels


That's a nice scan!


A shame it was sharpened

Haloes around text indicates sharpening

Reference


But this one seems fine?


Sharpened and JPEG compressed

Square areas and localized noise indicates JPEG compression

Reference


Lossless!We Promise!

Lossless workflow


A chain is only as strong...

Lossless!We Promise!

JPEG


This one? Please?


JPEG 2000 compression

JPEG 2000 lossy compression signs are best learned from multiple examples

Reference


Burnout

Sharp spikes at either end of the histogram indicates burnout

Reference


Burnout - visualization


But we need the dark to read!


No you don't!

Visualisation of ALTO-OCR files: https://github.com/tokee/quack


¦ “. i N ¦ M i; sk s, 011 el en al vei dens -t oi - te kom ei ner milen t'i bi -i i. 1 1 id iiiip .il -ukkersvg, I i ; \\ . : i .i i mod! , 1 1 km ikui i ene i !i;i • 1. t.v U.ilisk kali! Dit ei l u 't eksi isk.a bet /.> . i : 1 1 'I I l'b.! : man 'lit le.l b der ledes at Lundbecks tulbpeie forsk mi I I i 1 ' kt ' '1 F.va Sti 11 iess m i m ! o! i i . ! i , v! I ; vende l.epemiddel • ¦ ! a a 1. 1 >!:' a ! t \\ p. ¦ 2 tliahvtcs ; n 'i i te '. oi st, ¦ Klm.sKe i ol sop I n slik kel s vpe pat len! er Di amerikanske 1 1 1 1 1 1 1 i , F! >A il pi ve";

ABBYY FineReader 10.5

Some software upgrades matter!Novo Nordisk, som er en af verdens største koncerner inden for behandling af sukkersyge, bliver nu mødt af konkurrence fra en ny dansk kant. Det er biotekselskabet Zea - land Pharmaceuticals, der ledes af Lundbecks tidligere forsk - ningsdirektør Eva Steiness, som fører et nyt lovende lægemiddel til behandling af type 2-diabetes frem til de første kliniske forsøg på sukkersyge-patienter. De amerikanske sundheds - myndigheder, FDA, har givet

ABBYY FineReader 11


Post-processing

Holes in the histogram indicates leveling / exposure / contrast


Beware: Post-processing + JPEG

Post-processing indicators becomes less distinct when the image is JPEG compressed

Reference


Notice the lines?

300DPI @ 20x enlargement 300DPI @ 19x enlargement

Eve

ry o

the

r lin

e


Time for scanner calibrationG E N E R A L C A M E R A S E T T I N G S:

Camera Model No.: P2-20-08K40Camera Serial No.: 11041074Camera Network ID: 0Network Message Mode: disabled

Firmware Design Rev.: 03-081-20017-01 Aug 29 2007DSP Design Rev.: 03-056-20013-00

SETTINGS FOR UNCALIBRATED MODE:

Analog Gain (dB): +0.0 +0.0Analog Offset: 634 630

SETTINGS FOR CALIBRATED MODE:

Analog Gain (dB): -0.4 -0.5Analog Offset: 624 630Digital Offset: 0 0Calibration Status: FPN [uncalibrated] PRNU [calibrated]

SETTINGS COMMON TO CALIBRATED AND UNCALIBRATED MODES:

System Gain: 0 0Background Subtract: 0 0

Pretrigger: 0Number Of Line Samples: 32Video Mode: calibratedData Mode: 0Exposure Mode: 4

SYNC Frequency: external (9398.09) HzExposure Time: external

End-Of-Line Sequence: onUpper Threshold: 240Lower Threshold: 15Region Of Interest: 0001 - 8192

OK>

Systematic alternating lines indicates that the scanner should be calibrated


Last one is tricky


Zoom 9000


New tool – Grid lines


Upscaling

I'm working on a small tool for detecting scaling. Very alpha: https://github.com/tokee/telltale

BibTekConf 2013 - Lucene/Solr samsøgning og skaleringToke Eskildsen [email protected] ELAG 2014 – Scanning woes and war stories - 28/28

Are your scans just fine?

Toke Eskildsen, Statsbibliotekethttp://en.statsbiblioteket.dk/newsdigi

http://[email protected]

@TokeEskildsen

Scanning Woes and War Stories

Presentations & Public Speaking