1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people PROJECT FINAL REPORT ONLINE PUBLIC VERSION (updated Oct 5 th 2017) Grant Agreement number: FP7 - 611621 Project acronym: BlindPAD Project title: Personal Assistive Device for BLIND and visually impaired people Funding Scheme: FP7-ICT-2013-10 Period covered: from Jan 2014 to May 2017 Name of the scientific representative of the project's co-ordinator, Title and Organisation: Dr. Luca Brayda Researcher Fondazione Istituto Italiano di Tecnologia Tel:+3901081 72 205 Fax: E-mail: [email protected]Project website address: http://www.blindpad.eu/
37
Embed
Personal Assistive Device for BLIND and visually …...1 BlindPAD - Final Report – online public version 30.09.2017 Personal Assistive Device for BLIND and visually Impaired people
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1 BlindPAD - Final Report – online public version 30.09.2017
Personal Assistive Device for BLIND and
visually Impaired people
PROJECT FINAL REPORT
ONLINE PUBLIC VERSION (updated Oct 5th 2017)
Grant Agreement number: FP7 - 611621
Project acronym: BlindPAD
Project title: Personal Assistive Device for BLIND and visually impaired people
Funding Scheme: FP7-ICT-2013-10
Period covered: from Jan 2014 to May 2017
Name of the scientific representative of the project's co-ordinator, Title and Organisation:
2 BlindPAD - Final Report – online public version 30.09.2017
The BlindPAD project For visually impaired people it is difficult to digitally get graphical contents increasingly conveyed
through sight. The sense of touch can potentially bridge the gap, as it is crucial -in absence of vision
-for understanding abstract concepts and acquiring information about the surroundings. Examples
are learning at school and developing mental maps in orientation and mobility daily tasks. However,
available touch screens have limited or no tactile feedback at all. The potential and the market of
tactile displays are largely unexploited, although there is a clear demand from users: these devices
need to become more versatile, cheaper, portable and socially acceptable. This project has made
graphical contents accessible through touch by building and field-testing a Personal Assistive Device
for BLIND and visually impaired people (BlindPAD). BlindPAD puts veridical touch-based information
into the hands of users, exploiting and enhancing their residual sensory abilities. The BlindPAD has
explored several technologies, compared in terms of actuation force, resolution, safety, power
consumption and reliability.
By adopting a user-centred approach within an accessible and usable ecosystem, we have assessed,
with serious games, how the BlindPAD can help visually impaired people in two paramount use
cases: touch-based learning of symbolic content at school age; orientation and mobility skills indoor.
We have shown that our programmable tactile display increases, in persons with sensory deficits,
the spatial working memory, the mathematical abilities, the capacity to find one’s own position in
an unknown space and spatial knowledge in general, beyond current rehabilitation protocols.
BlindPAD will be a personal, portable and cheap solution to improve knowledge and independence,
thus increasing chances of employment, of social inclusion and, ultimately, of a better quality of life.
3 BlindPAD - Final Report – online public version 30.09.2017
Innovation
Figure 1 What is BlindPAD: a system comprising three main components. Combining a haptic display, a software that delivers graphical content and a series of exercises has been never done before.
The output of the BlindPAD project that sums up the foreground is the virtuous and innovative
combination of three components (see Figure 1):
• a haptic display, that stimulates the sense of touch with 192 programmable moving pins
• a piece of software that allows to translate visual concepts into tactile representation
• a series of exercises about 1) geometry and 2) orientation that are implemented on the
haptic display through the tactile graphics software.
Together, these three elements are the unique novelty of the project, that no previous research or
industrial activity has led to such stage. We detail each piece in the following.
=
Haptic
display
Tactile graphics software
+
Rehabilitation exercises
+
4 BlindPAD - Final Report – online public version 30.09.2017
1 Executive summary Visually impaired persons, especially children, are locked out from digital information and
communication because technologies are heavily oriented to visual user interfaces and provide
increasing amounts of information graphically. Blind people need to use the residual sense of touch
to understand information, something already achieved for digital text – thanks to Braille – but not
at all achieved for graphics. Therefore, the need of achievable tactile technologies facilitating their
better inclusion into modern society and work becomes very important. Fabricating a tactile tablet
for blind persons has been a challenge for decades: the idea being that of transforming a visual
concept into a tactile representation is well known as ‘sensory substitution’. Much like ‘pixels’, one
digital image can be formed by a grid of small tactile pins (‘taxels’) that can be programmed to be
‘up’ or ‘down’, therefore forming a bas-relief that can be sensed and understood with the hands.
However, making a dense array of taxels with sufficient force and displacement to be easily felt by
any user, and that is also low power, fast and compact is a major technological challenge. Due to
the complexity of drive electronics or the lack of performance of actuators, none of previous
technological solutions were shown to be scalable, have sufficient performance, and be portable.
Here, we have successfully built and field-tested a system comprising a new tactile tablet, a software
translating images into tactile representations and a series of exercises that together make digital
graphical information accessible to blind and visually impaired persons. Our tactile display is 12 cm
x 15 cm in size, consisting of 192 electromagnetic, independent taxels. Each taxel is very fast in
changing state (up/down), allowing static and moving patterns to be displayed. Together with our
software that renders tactile images, we verified that the BlindPAD system is effective on blind and
visually impaired people, especially at the developmental age. We considered two paramount use
cases: learning mathematics and learning unknown spaces from maps: we have shown that
BlindPAD successfully trains visuo-spatial working memory, complex mental operations,
mathematical concepts and helps to picture maps of unknown rooms, supporting the ability to find
one’s own position in a real environment. Our results contributes both to the research field of
material science, as we have shown that it is possible to build arrangements of small element able
to deliver high forces but with small components, to the field of experimental psychology and
cognitive neuroscience, in which the potential abilities of persons with sensory deficits are still
debated and to a certain extent underestimated or at least unexploited; to the field of computer-
human interaction, since tactile feedback on non-flat screens has deserved little attention; in
general to the field of computer-assisted rehabilitation, where no standards exists and the access
to digital information can be a breakthrough in making rehabilitation practices semi-autonomous.
The BlindPAD system was built in three years from scratch: it now comprises a hardware tactile
device, a software running on most PC and tablets, and a series of exercises that stimulate with
touch the abilities linked to spatial working memory, spatial processing, logics and mathematical
reasoning. The system, as it is, can be used in rehabilitation centers, with potential very high impact
on rehabilitation practices. Our prototypes are rather easy to reproduce, therefore the technology
can be exploited by transferring it to a start-up. With a relatively low effort in terms of engineering,
it can become a product.
5 BlindPAD - Final Report – online public version 30.09.2017
2 Context and objectives The novelty of this proposal is the construction around users’ needs of the first, truly personalised
and portable tactile display able to deliver graphical content to hands and fingers with a veridical
three-dimensional tactile feedback. To learn graphical concepts, visually impaired people are
generally forced to use large sheets of swelled paper, bulky or heavy physical three-dimensional
models, large Braille bars to purposes they are not built for. In addition, none of the solutions used
today can be easily personalized. Similarly, no portable solution exists yet which delivers refreshable
tactile maps.
The project started from the idea to build a tactile surface as big as a tablet computer. To achieve
this, the approach was to put into competition several technologies. The strategy to investigate two
different promising technologies aimed to reduce the risk to achieve the target objective of a cost-
effective, light-weight tactile display. These two innovative technological approaches were based
on electro-active polymers and electromagnetic actuation: they shared many higher level
components (interface, addressing, etc.), and were designed from the start to allow scaling, in this
project, from hundreds to even thousands of taxels, with potential further up-scaling at relatively
low costs. Both relied on low-voltages, but used different physical actuation principles. By having
these two technologies, we significantly reduced risks with only a small increase in person/months.
After the first level of research on both technologies, the best suitable regarding portability and
usability was selected for further investigation at a defined decision point in the project. The final
adopted technology aims at helping blind and visually impaired persons in at least two fundamental
tasks of their lives. The first is education, because touchable graphical content will play a role similar
to what images do for sighted people in any learning activity involving abstractions and symbolic
representations, which are of paramount importance in the developing age. The second in
orientation and mobility (O&M), because touchable maps will help the development of cognitive
representation of the world, able to be updated, zoomed, and tailored to specific sensory needs.
Both scenarios are fundamental in social inclusion, since visually impaired young persons will be
potentially able to follow, at their fingertips, the same information visually presented to their
schoolmates on the blackboard, due to the portable nature of the device; adults, instead, will exploit
state-of-art geolocalization technologies and software to find and communicate with their sighted
fellows, either by building mental maps before leaving home, or directly carrying the device with
themselves.
To reach this goal, we decided that this portable stimulator might have the form factor of a tablet,
thus suitable for interaction with tactile objects using few fingers up to two hands. The
microactuated personal assistive device proposed, called BlindPAD (Personal Assistive Device for
BLIND and visually impaired subjects), allows the dynamic control of the tactile information by virtue
of an electronic board, wirelessly connectable to standard mobile devices, such as tablets and
smartphones. Graphical content, stored either locally or remotely, is then haptically rendered and
displayed. To enhance the usability of BlindPAD, the interaction mode can be extended with state-
of-the-art touch sensitive technology, allowing to read the fingers positions on top of the novel
surface. This approach opens the way for interactive applications making the information access for
visually impaired even more effective.
The specific objectives of the project are:
6 BlindPAD - Final Report – online public version 30.09.2017
Objective 1: to investigate two novel and competing tactile feedback technologies and to select the
most effective in terms of 1) usability, 2) portability, 3) scalability for large array size and 4) suitability
for low-cost mass production for the development of the proposed BlindPAD display. The first
objective is technological. The BlindPAD display is composed of three parts: a tactile interface, i.e.
a bi-dimensional matrix of mm-scale taxels, a connected electronic board driving the actuators and
interfacing with a computer or smart phone, as well as a state-of-the art touch sensitive technology.
In addition, the position of one or more fingers is tracked to allow interaction with the touchable
objects. Among the different technologies, the best one is chosen to build the final demonstrator.
The partners refine their technology given portability (power, mass, robustness) constraints. The
limits of scaling are studied to understand realistic taxel size given both processing and drive circuit
constraints. For large arrays, attention must be paid to achieve uniform properties over the entire
array (both to have the same force for all taxels, but also to allow simpler drive circuits by not
requiring taxel by taxel calibration). The assembly scheme is planned to be engineered for
manufacturability. Lower-cost manufacturing technologies are studied, in view of an eventual
technology transfer to manufacturing partners.
Table 1 Technological objectives of the project BlindPAD
Har
dw
are
Build a single taxel interface: only one pin, but with several technologies
Build a low resolution grid of such interfaces: arrange several pins together, at
increasing resolutions (row and columns): 2x2, 4x4
Build a high resolution grid of such interfaces: arrange many pins together, at increasing
resolutions (row and columns): 12x16, 32x24
Choose the most affordable technology and build a tablet-sized haptic display
Build the electronics that drive the haptic display
Make the display portable
Build a sensing surface, to detect the position of the fingers on the haptic display
Soft
war
e
Build a human-computer interface allowing to draw arbitrary images on a grid
Allow the software to receive input from both scientific software tools and
unexperienced users, and send the output to the tablet-sized haptic display
Objective 2: The second objective is social inclusion, obtained by empowering rehabilitation
programs with a tool to increase spatial abilities faster and closer to that of sighted persons. The
second objective is to test the BlindPAD in two use cases.
The first use case was learning of basic graphical content in mathematics (geometry), with symbolic
dictionaries (object shapes, iconic and visual conventional signs translated into touchable content).
It was demonstrated that certain exercises can be performed with a tactile tablet at least as well as
current rehabilitation techniques, which have the drawback of requiring constant attention from
7 BlindPAD - Final Report – online public version 30.09.2017
practitioners and a tremendous amount of time to prepare them (up to 50% of rehabilitators is
spent in preparing personalized stimuli for blind children). Such exercises include basic tasks that
are generally performed at the school age, where the need of graphical elements is stronger:
establishing which tactile symbols made by few raised pins can be well recognized, then recognizing
which tactile symbol among a set of distracting elements is correct, then memorizing one or more
spatial disposition of elementary tactile symbols, then performing the difficult task of mentally
rotating a shape (something that will be useful later on when studying a map), or finally estimating
distances among tactile symbols.
The second use case is Orientation and Mobility: visually impaired subjects are limited by the lack
of reference points: for every novel environment people must have a representation of space in
their mind (i.e. a mental map). The BlindPAD wants to offer a representation of simple top-view
maps of rooms. We answer to fundamental questions such as “Where am I?” and “How do I get
there?”. Here, BlindPAD is used as a complementary tool for O&M programs currently addressed to
deprived subjects who already developed geometrical skills necessary to process maps and
orientation instructions. Tests are intended to be measurable: way how well a tactile map can be
acquired and memorized? Can memorizing a tactile map be useful when exploring an unknown
space? Interaction with the real landmarks, previously touched on the BlindPAD, can be used to
check, with help from O&M experts and standard spatial tests, how well the mental map was
developed, therefore deriving how well the BlindPAD was able to display it.
Importantly, the serious games will be experienced not in labs but in contexts which will be familiar
for the subjects, and which will be comfortable and plausible for the end-users (e.g. rehabilitation
centres), thus emphasizing the peculiarity of the BlindPAD as a personal device.
Table 2 Cognitive objectives of the BlindPAD project
Per
cep
tio
n
and
co
gnit
ion
Establish minimum criteria to say that an array of pins is well perceivable
Compare the competing technologies and choose the best one in function of user needs
Compose a set of tactile images: a dictionary that can be used in serious games
Reh
abili
tati
on
Improve the learning of graphical concepts in blind and visually impaired people
Improve orientation and mobility skills in blind and visually impaired people
Comparing and complementing existing rehabilitation protocols with BlindPAD
8 BlindPAD - Final Report – online public version 30.09.2017
3 S&T results/foregrounds
3.1 From many to one technology: the BlindPAD EM prototype
In this section we list the two main technologies that showed an advancement in the project (Shape-
Memory Polymers and Electro Magnetic) and were patented, together with the electronic
components that were part of the prototypes (electronic board and sensing surface). The final
adopted technology was the Electromagnetic because it granted high forces, high displacement and
reliability.
3.1.1 The Shape-Memory Polymer Technology
Shape Memory PolyUrethanes (SMPUs) are one type of shape memory polymers. Their main feature
is their ability to lock a temporary shape and recover back the permanent one anytime upon simple
thermal actuation. Shape-change temperature can be chosen from 35 to 85 °C.
For BlindPAD, the objective is to benefit from their enormous change in Young's modulus with
temperature to intrinsically implement latching and to selectively actuate individual taxels in the
array. The proposed unique and novel concept, which enables larger scale manufacturing and faster
refresh rate, is to integrate a stretchable heater on each taxel and to use a common pneumatic
source to move all taxels (i.e. one pump drives all the taxels, but only those that were heated move).
There are two stable states when the membrane is cold: “UP” or “DOWN”.
We emphasize that the SMP technology allows to change the tactile pattern in a differential fashion:
only the taxels which change from one tactile image to the next are eventually physically modified
(i.e. heated).
9 BlindPAD - Final Report – online public version 30.09.2017
Figure 2: Pictures of each separated parts used to drive in the 32x24 SMP prototype. The hardware (actuators + pumps), the driver board and the software are show.
Figure 3: Picture of the 32x24 SMP prototype with the pin interface (from IIT) and the copper plate as it would be used for
perception tests to speed up the refresh time.
3.1.2 The ElectroMagnetic actuation
Electromagnetic (EM) actuation offers particularly appealing performance in terms of force,
deflection, bandwidth, scaling, integration, robustness and portability. This haptic display was
developed with low cost in mind, and consists of a matrix of small magnets between two printed
circuit boards (PCBs). We have successfully built a technology that exploits electromagnetic force to
move the single taxels of a tactile tablet.
4x4 arrays: The EM technology was then scaled to a 4x4 array, with further advancements: a
schematic view of a single taxel is presented in Figure 4a; and key assembly steps of the 4x4 EM
device are presented in Figure 4b to c (8mm case). The device consists of an array of EM-based
10 BlindPAD - Final Report – online public version 30.09.2017
vertical actuators under a 3D-printed pin array interface. The moving part of each EM actuator
consists of pot-magnet suspended between two elastomer membranes over a multilayer planar coil.
Only the surface of the magnet facing the coil is unshielded. In this way, the magnetic flux from the
pot-magnet assembly is restricted mostly to the region below the magnet. The top and bottom
membranes support the pot-magnets and act as restoring springs.
Final: 12x16 array. Then the EM prototype that was scaled up is a 12 cm x 15 cm size haptic display,
consisting of 192 electromagnetic (EM) independent taxels that move up and down to present
graphical information that is explored by the user using his/her sense of fine touch. Each taxel
changes state (up/down) in under 10 milliseconds, allowing static and moving patterns to be
displayed. This dramatic step forward on the EM taxel technology combines several innovations
regarding magnet shielding method, coil placement, soft magnetic material arrangement, and
operating concept. All these factors taken together, enable the fabrication of a dense array of
electromagnetic actuators that can a) latch in 2 stable positions, b) offer strong holding force, c)
have no cross-talk, d) be fast (few ms), e) give mm-scale displacement, f) be compact, g) be low-
power and scalable to very large arrays.
Figure 4 Diagram and key elements assembly of the 4x4 haptic display. a) Schematic view of a single taxel and the main actuation components. b) Photo of the 6-layer PCB containing the array of planar coils. The PCB is placed on an aluminium plate
supported by four standoffs. c) Top view of he magnetic layer. It is formed by the 16 moving pot-magnets, a perimeter line of fixed pot-magnets, the top and bottom elastomer membranes (not visible) and an acrylic holder. d) A 3D printed pin interface
completes the device as a final layer, and is what the user touches.
The final demonstrator implements a novel working principle: the bi-stable EM actuators. In the
sequence a single taxel switches from the “UP” state to the “DOWN” state and vice versa. When the
taxel is up the lateral shielded magnet is attracted by the closest latching plate, i.e. the top one. Due
11 BlindPAD - Final Report – online public version 30.09.2017
to this interaction between the magnet and the latching plate, the taxel remains in the up state
without any power consumption. To switch to down state, a short actuation pulse is applied to the
top and bottom coils, with the direction of the electrical current properly chosen so that the top coil
repels the magnet while the bottom coil attracts it. During the magnet way down, it’s interaction
with the bottom latching plate increases, up to the point where the actuator switches to the second
stable position. The transition takes only a few milliseconds, when the actuation current is on. Same
as in the up state, no electrical current is need to maintain the magnet in the down state. To switch
from the down to up taxel state, similar actuation pulse is applied but with the electrical current in
the inverse direction.
3.2 Everybody can draw tactile graphics: the PadDraw software
PadDraw is developed within the project and connected directly to the BlindPAD and haptic devices
(like Hyperbraille). As part of the negotiation process this software is discussed how to offer the
software. As part of the full service offer, PadDraw will be offered, supported and also training on
“How to use PadDraw for BlindPAD” are discussed. In another scenario a lightweight PadDraw will
be offered as free service with some basic API functionalities, to support educational training and a
creative vision exchange. The latest version of PadDraw runs on Windows and MacOS. Windows is
also available as tablet and mobile version. Therefore, PadDraw will also run on such device and can
be used in mobile scenarios. In PadDraw every object you can add to an image and every effect you
can apply to the image is basically a component. When a component is added to the image it is
stacked onto the other components that are already contained in the image. This way a component
can also be regarded as a layer inside the image.
Components are basically divided into two groups:
• Drawable components
• Effect components
Every component has two core properties with the following characteristics:
• ID: integer, unique, automatically set by PadDraw
• tag: string, may be assigned to multiple components, set by user1
The application mainly consists of four major modules describes below in more details:
• Tactile image composer module
• Tactile slideshow composer module
• Remote control module
• Hardware interaction module
1 Please note: tags can only be composed of alphanumerical characters (a - z, A -Z, 0 - 9). All other characters are forbidden!
12 BlindPAD - Final Report – online public version 30.09.2017
Figure 5 Main Window of PadDraw with Components Inspector and Undo
3.3 Testing BlindPAD prototypes
We have compared different prototypes of BlindPAD, using a common technological setup. The goal
was to choose the most suitable technology, to be scaled up in the last year of the project. The setup
consisted in a matrix of 16 tactile actuators, organized as a 4x4 matrix. Perception tests involved
participants of different age and kind of visual disability. We compared the technology based on
ElectroMagnetic taxels to that based on ShapeMemoryPolymers. Tests involved perception and
recognition of tactile images under various constraints related to tactile vocabulary size and
cognitive resources involved, as well as usability questionnaires. Tests reported similar results
between technologies with results very significantly higher than chance when people were asked to
identify one over 12 tactile symbols. Interestingly, some symbols were perceived better than others,
with higher performance on more reminiscent symbols.
Success rate in detection tasks exhibited performance higher than 95%. Success rate in
discrimination of different tactile images exhibited performance of 90%. Success rate in a more
challenging identification task of tactile images on a wide vocabulary exhibited performance of
around 70%, i.e. 8 times higher than chance.
Although EM and SMP are very different in terms of maximum force and displacement (EM having
more displacement and less force, SMP having less displacement and more force), both
technologies were effectively perceived by participants, with personal preferences for either
technology.
13 BlindPAD - Final Report – online public version 30.09.2017
3.4 Programmable tactile displays increase spatial abilities of visually impaired people
3.4.1 Case study 1: increasing mathematical abilities with BlindPAD
We involved populations of sighted, low vision and blind participants. As for tactile displays, we
adopted several technologies: paper-based raised line drawings (the state of the art in rehabilitation
centers), a commercial pin-array display (Hyperbraille) and our prototype of pin-array display
(BlindPAD).
A first study (see Figure 6) was motivated by the fact that blind people have severe issues in
manipulating and integrating visuospatial information, and in memorizing a high number of data
(see Figure 7) This is relevant for the success in different scholar disciplines (geometry, mathematics,
etc.) as well as some working careers (e.g. science). We verified whether visuospatial working
memory can be trained in sighted, blind and low-vision youngsters using pin-array tactile displays.
Our results suggest that visuospatial working memory can be successfully trained in visually
impaired youngsters at least in tasks characterized by a low cognitive load, with trainings lasting
about one hour, once per week for four weeks.
The results are part of the following publication (open access):
Leo, F., Cocchi, E., & Brayda, L. (2017). The Effect of Programmable Tactile Displays on Spatial
Learning Skills in Children and Adolescents of Different Visual Disability. IEEE Transactions on
Neural Systems and Rehabilitation Engineering, 25(7), 861-872.
Download the PDF here DOI: 10.1109/TNSRE.2016.2619742
Figure 6 A. Experimental setup with the Hyperbraille display on the left side and the PC running the PadDraw software on the right side. The picture shows an example of trial of the spatial memory task. B. An experimenter and a rehabilitation practitioner
giving instructions to a blind child. A rehabilitation practitioner was always present during the tests involving visually impaired youngsters.
14 BlindPAD - Final Report – online public version 30.09.2017
Figure 7 A. Schematic of successive events within a trial for Single-matrix task. A speech synthetized voice indicated to the participant that the targets appeared on the screen. After a presentation time of 15 seconds, a voice (not reported in the figure)
asked the participant to remove the fingers from the display. The targets disappeared and a voice asked the participant to indicate target locations. After each answer, the experimenter started a new trial. B. Schematic of successive events within a trial for Double-matrix task. Task events were similar to the Single-matrix task but two different matrices were displayed in sequence. A 2-s interstimulus interval interleaved matrices presentation. Participants were asked to report where were the
targets in the two matrices by replicating the original temporal sequence (i.e. 1st matrix first, then 2nd matrix). After each answer, the experimenter started a new trial.
A second study clarified if Braille-based displays can be used also to convey graphical information.
We determined an ideal set of graphical symbols that can be easily recognized, in function of their
shape and size. Symbols as small as 5 mm in size can be discriminated easily enough to be used in
tactile maps and diagrams. However, our confusion matrices showed that certain shapes of symbols
could not be used without the risk of mistakes. This was the basis for the following studies, but also
for those concerning Orientation and Mobility.On our final BlindPAD prototype (EM 12x16), one
taxel only could be easily discriminated (here a taxel is more than twice larger than a Braille dot).
The final study targets mathematical abilities using the prototype of BlindPAD: a four-week training
was implemented (similar to Study 1), this time with a distance discrimination task on two-
dimensional spatial dispositions.
15 BlindPAD - Final Report – online public version 30.09.2017
Figure 8: A blind volunteer testing the first approach of wireless EM haptic display at EPFL.
16 BlindPAD - Final Report – online public version 30.09.2017
4 The potential impact (including the socio-economic impact and the wider
societal implications of the project so far) and the main dissemination
activities and exploitation of results.
4.1 Impact on technology
4.1.1 New actuation systems for haptics and beyond
The BlindPAD project has cast technological results well beyond the state of the art: our solutions
are based on the latching principle, that is the technique of maintaining tactile information without
spending power to deliver the desired force and displacements of the tactile actuators, a crucial
aspects for portability and power consumption issues. An ideal surface, should elicit static pressure
(without vibration), so that mechanical energy would result from the surface shape (possibly with
an on/off actuation) combined with active finger movement and skin stretch (Konio et al., 2003,
Dargahi et al, 2004). The idea behind the Shape-Memory Polymer is smart enough to lead to
solution outside the haptic field. The idea behind the shielding of the pot magnets can be used for
tactile stimulations but also for other actuation systems where multiple actuators are desired (e.g.
robotics).
Our latest prototype presents the following technical specifications:
Figure 9: 12x16 haptic display (15 cm x 12.5 cm), and Raspberry Pi (top right). The user touches smooth plastic taxels, driven by
an array of 192 high speed actuators.
• 192 (i.e. 12 x 16) independently controlled taxels
• 10 millisecond refresh time per taxel (enables rapid update of static images)
• 8 mm pitch between taxels
• 0.8 mm vertical travel
• 200 mN holding force
• latches in both up and down states
• controlled by Raspberry Pi
17 BlindPAD - Final Report – online public version 30.09.2017
4.1.2 New information systems that translate visual information into tactile graphics
The BlindPAD specifically targets an education use case and a mobility use case, taken as two
representative examples where acquiring and manipulating graphical content is crucial. In fact,
being able to learn as much as others can do and the ability of travelling without barriers are at the
base of any sustainable social relationship. The BlindPAD aims at favouring communication between
visually impaired youngsters, their teachers and mates. At growing pedagogic interest in touch-
based interfaces, therefore relaxing the constraints on the way symbolic representations in scientific
subjects are taught today. Because content distributed over the internet, starting from educational
content which goes more and more paperless, is more and more involved with visual effects, this
project makes tactile rendering central to the information delivery process. The nature of touch
imposes touchable graphical content to be essential; it will foster research not to what is “nice” to
be depicted, but what is “necessary” to be displayed understood. Therefore, the choice of the final
resolution of the device (also driven by technological constraints) cannot be evaluated in absolute
terms, but relatively to what can be done with that specific resolution.
4.2 Main dissemination activities
In the BlindPAD project overall:
• We organized two scientific workshop dedicated to interaction and haptic technologies
• As of June 30th 2017, we published 9 journal papers, 10 conference/workshop papers, while
2 journals are under review and 5 other journal papers are in preparation
• We presented our results in 17 events (congresses, conferences, invited talks), of which 10
were national and 7 international
• We had a BlindPAD booth or presentations at 9 wide-public exhibitions, of which 4 were
national and 5 international. The exhibitions we chose reached a large audience both in the
scientific domain and in the general public domain. We estimated that at least 5000 people
have directly seen our presentations, or touched our demos, or took our flyers.
• In some of our dissemination activities we involved visually impaired persons to explain
children and adults what are their user needs in the modern society.
• The term “blindpad”, without spurious meanings, is indexed by Google as part of 13000 web
pages. Considering that the most famous tactile pin array ever existed – the Optacon – totals
up to 38000 hits, and that the state-of-art Hyperbraille pin array totals up to 4400 hits, we
estimate that this is a very good result.
• BlindPAD has an accessible website, which meets the requirements of W3C. The website
totalized 23000 visits in 3 years, i.e. more than 600 visits per month. Twitter and Facebook
sites were regularly updated.
18 BlindPAD - Final Report – online public version 30.09.2017
Figure 10 Main dissemination activities: Festival della Scienza 2014, 2015 and 2016, ICT Lisbon 2015, CHI 2017, World of Haptics 2017, Makerstown 2017. Our exhibition were targeting an audience of all ages and particularly attracted children because of the
‘game-like’ attitude of our demos.
Our dissemination was centered on a “verify, then show” philosophy. We favored venues where
people could practically touch our prototypes and check that what we announce is what actually
works. We adopted the strategy: patent, then publish, then announce. This can explain the
relatively low number of followers on Twitter and Facebook.
19 BlindPAD - Final Report – online public version 30.09.2017
5 List of beneficiaries
Participant no. Participant organization name Part.
short
name
Country
1 (Coordinator) Fondazione Istituto Italiano di Tecnologia IIT Italy
2 École Polytechnique Fédérale de Lausanne EPFL Switzerland
3 Geomobile GEO Germany
4 Istituto David Chiossone onlus CHIO Italy
5 Fundacja Instytut Rozwoju Regionalnego FIRR Poland
6 Ateknea Solutions ATEK Hungary
Figure 11 List of beneficiaries
6 Coordinator contact details
Company name: Fondazione Istituto Italiano di Tecnologia
25 BlindPAD - Final Report – online public version 30.09.2017
template A2: list of dissemination activities
NO. Type of activities
Main leader Title Date/Period
Place Type of audience
Size of audience
Countries addressed
1 Conference G. Bubak Parylene coated carbon nanotube actuators for tactile stimulation, International conference SPIE Electroactive Polymer Actuators and Devices
April 2015
San Diego, CA, USA
Scientific Community (Research), Industry, Medias
300 Worldwide
2 Workshop W. Kunkhornsup Study of Static Tactile Detection Threshold via Pneumatically Driven Polydimethylsiloxane Membrane
2014 Dresden, Germany
Scientific Community (Research), Industry,
100 Worldwide
3 Conference J. Zarate Design and optimization of microfabricated planar coils for tactile displays”, in Proceedings of International Conference of Micro and Nano Engineering
5 Conference F. Leo Improving Visuo-Spatial abilities in blind youngsters using programmable tactile displays”, in Cognitive Neuroscience Society meeting
2017 S. Francisco, USA
Scientific Community (Research),
200 Worldwide
6 Conference F. Leo BlindPAD: proposta di uno strumento per incrementare le abilità visuo-spaziali nei non vedenti’, XXV Congresso Nazionale Airipa
2016 Turin, ITaly Scientific Community (Research),
200 Italy
7 Workshop F. Leo Recalling graphical traits with programmable tactile displays improves spatial abilities in young visually impaired persons’, The European Workshop on Imagery and Cognition
June 6, 2016
Paris, France Scientific Community (Research)
1000 EU
26 BlindPAD - Final Report – online public version 30.09.2017
8 Conference J. Zarate Keep in Touch: Portable Haptic Display With 192 High Speed Taxels", CHI 2017
May 2017
Denver, USA Scientific Community (Research), Industry, Medias
3000 Worldwide
9 Worskhop L. Brayda Organization of Workshop on Multisensory Interaction and Assistive Technology, within International Conference of Tabletop and Surfaces 2015
Nov 2015
Madeira, Portugal
Scientific Community (Research), Industry, Medias
200 Worldwide
10 Workshop L. Brayda Organization of Workshop on Haptics Interfaces for Accessibility within World of Haptics Conference
June 2017
Furstenfeldbruck, Germany
Scientific Community (Research), Industry,
500 Worldwide
11 Workshop E. Cocchi Workshop on Complex Disabilities, Fondazione Mariani
12 Conference E. Cocchi National conference “La riabilitazione visiva in età evolutiva – centri italiani a confronto” (Visual rehabilitation in developmental age – comparison of Italian centers),