1 Tactile Graphics with a Voice 2015 Richard Ladner University of Washington
1
Tactile Graphics with a Voice
2015
Richard LadnerUniversity of Washington
2
Text Book Translation Problem
text
math
images
Tactile Perception
3Somatosensory system
4
Tactile Perception Limitations
• Resolution of human fingertip: 25 dpi
• Tactual field of perception is no bigger than the size of the fingertips of two hands
• Color information is replaced by texture information
• Visual bandwidth is 1,000,000 bits per second, tactile is 100 bits per second
Resolution Study
5
.35mm 1 mm 2 mm
• On average, blind participants could sense 1 mm separation but no smaller.
• On average, sighted participant could sense 1.5 mm separation but no smaller.
Blind Spatial Perception
• Tactile perception can lead to a “spatial” model in the brain.
Tactile Graphic
7
Tactile Graphics Kit
8
Tactile Graphics Printing
• Embossing Printer
• Thermoform
• Swell Paper
9
10
Tiger Embosser
• 20 dpi (raised dots per inch)• 7 height levels (only 3 or 4 are distinguishable)• Prints Braille text and
graphics• Prints dot patterns for
texture• Invented by a blind physicist,
John Gardner
Refreshable Tactile Display
11
12
Graphics
13
Graphic Translation<LocationInformation><NumLabels>16</NumLabels><Resolution>100.000000</Resolution><ScaleX>1.923077</ScaleX><ScaleY>1.953125</ScaleY>-<sd><x1>121</x1><y1>45</y1><x2>140</x2><y2>69</y2><Alignment>0</Alignment><Angle>3.141593</Angle></Label>
preprocesstext extract
cleanimage
originalscannedimage
puregraphic
textimage
locationfile
14
Graphic Translation<LocationInformation><NumLabels>16</NumLabels><Resolution>100.000000</Resolution><ScaleX>1.923077</ScaleX><ScaleY>1.953125</ScaleY>-<Label><x1>121</x1><y1>45</y1><x2>140</x2><y2>69</y2><Alignment>0</Alignment><Angle>3.141593</Angle></Label>
puregraphic
textimage
locationfile
y(0,20)x=1515105Ox510152020x+y=20(15,0)(15,5)
y(#0,#20)x.k#15#15#10#5Ox#5#10#15#20#20x+y.k#20(#15,#0)(#15,#5)
text Braille
15
16
Available Books
• Computer Architecture: A Quantitative Approach, 3rd Edition 25 minutes per figure (230 figures)
• Advanced Mathematical Concepts, Precalculus with Applications 6.3 minutes per figure (1,080 figures)
• An Intoduction to Modern Astrophysics 10.2 minutes per figure (467 figures)
• Discrete Mathematical Structures 8.8 minutes per figure (598 figures)
• Introduction to the Theory of Computation, 2nd Edition 13.3 minutes per figure (180 figures)
17
TGA Workflow
• Advantages– Much faster production– Batch processing instead of one figure at a
time– Much tedious work is avoided
• Disadvantages– May be of lower quality than custom
translation– A lot of technology needs to be mastered
18
One-offs vs. Mass Production
1916 WoodsDual Power
Model T1906 Reo
Audio-Tactile Graphics
19
Talking Touch Tablet
Digital Pen audio-tactile graphic
Tactile Graphics with a Voice: using QR codes to access text
in tactile graphics
• C.M. Baker, L.R. Milne, J. Scofield, C.L. Bennett, RE Ladner. Proceedings of the 16th international ACM SIGACCESS conference on Computers. ASSETS 2014.
20
21
Tactile Graphics with a Voice
System Use
22
23
Modes
Top
Silent Verbal Finger Pointing
Study
• 10 blind users
• 4/10 non-Braille
• 4 tasks
• 3 modes
• 6 sessions
24
Tasks
Line Parabola
Triangle Bar Chart
25
Time
26
F5, 640=2.268, p<.05
F2, 640=0.619,p=.5391
Time outs
27
Silent Verbal Finger Pointing
Line 97% 97% 93%
Parabola 93% 95% 95%
Triangle 88% 88% 90%
Bar Chart 73% 70% 79%
All Tasks 88% 88% 89%
Accuracy
28
• 4 preferred silent– To be honest, I use screen readers every day and I am so sick
of electronic noise.
• 4 preferred finger pointing– I like the concept of the finger pointing. I feel more confident that
since it looks for a finger it’s getting the right QR code if you have multiple on the same page.
• 2 preferred verbal– The other thing I like about Verbal mode is that every time I hear
a zero I think so I need to move it a little bit
Preference
29
In a study of 82 images from a pre-calculus textbook, on average Braille took up more space
– Braille was larger in 326 out of 626 labels• Braille labels: 327 sq. mm
• QR codes: 225 sq. mm
Braille comparison
30
Braille - Time
31
Braille - Accuracy
32
Questions
33