CSE Template
Touchless, touch-based and Augmented Reality-based interactions
with bacterial biofilm images Mohammadreza Hosseini, Arcot
Sowmyamhosseini, [email protected] Bednarz
[email protected]
Click to edit Presents Name
1
IntroductionImage review, manipulation within sterile
environments, maintaining boundaries between sterile and
non-sterile areas of work environment, are essential in biology
studiesRemote control and visualization of biomedical images can
reduce direct exposure of researchers to viruses and bacteria
Investigation on human user interfaces, via touchless touch- based
and Augmented Reality interfacesAlso study their usability through
series of user experiments
School of Computer Science and EngineeringPage #
2
Touchless system design and image visualization
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
Touch based system design
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
User Experience Design 10 participants selected randomly from
among scientists and employees in CSIRO to participate in the
experiments People from different backgrounds, nationalities and
gendersAt the beginning of the experiments every participant
introduced individually to the system by an expert. They could
observe the expert interacting with two systems They get enough
time to fill the survey formsThey are also requested to indicate
which interface felt more natural
School of Computer Science and EngineeringPage #
Survey form1: SUSSystem usability scale (SUS) is a ten-item
Linkert scale) with a weighted scoring range of 0-100, giving a
global view of system usability.To calculate the SUS score, the
score contributions from each item are summed. Each item score
contribution will range from 0 to 4. For items 1, 3, 5, 7, and 9,
the score contribution is the scale position minus 1. For items 2,
4, 6, 8 and 10, the contribution is five minus the scale position.
Multipling the sum of the scores by 2.5 provides the overall value
of SUS.The scores are then converted to a percentile rank using a
process called normalization. The SUS score percentile rank is
usually referred as school grade scale
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
Survey form2: Self assessment manikinSelf Assessment Manikin
(SAM) is a graphical figure to measure feelings of pleasure,
arousal and dominanceSAM displays each dimension with a graphical
character array along a continuous nine-point centre movement scale
For pleasure, SAM shows characters from smiley happy faces to
unhappy and sad faces For arousal, SAM displays figures that are
very excited with eye open down to sleepy and bored facesFor
dominance, a very large figure presenting feelings of strength and
being in control, and a small figure showing the feeling of being
controlled or submissive
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
majority of user initiate touch based interaction as an
excellent (A) choice for interaction
School of Computer Science and EngineeringPage #
all participants gave higher scores to touch-based system in
comparison to tuchless interaction systems.
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
people feel happier and more in control when they are
interacting with a touchbased interactive system, but the
excitement of using touchless is higher
School of Computer Science and EngineeringPage #
ConclusionParticipant observations during user experiments
reveal that moving the entire hand is not an ideal way to
communicate with the system. The feeling of tiredness that users
experience when using hand gesture explain why the touchless system
is less pleasurable compared with a touch-based system.working with
large images where information content is very high is the major
cause of tiredness.the feeling of not being in control of the
touchless interface is due to the delay between the hand movement
and pointer positioning on the screen. Every participant also
thought that an iPad is a more natural screen than a 3-meter
hemispherical dome.
School of Computer Science and EngineeringPage #
Augmented reality as a new form of interactionTo benefit from
the capacities of both touchless and touch-based unimodal
interaction. AR is introducedAR interaction can enhance
understanding of physical objects by addition of digital
information to captured video streams.
School of Computer Science and EngineeringPage #
Tracking in AR Major challenge for every AR application is
trackingTracking is about locating position of an object in the
video frame and aligning the virtual information with the user
field of viewTo detect the users field of view, it is assumed that
the handheld device camera direction is the same as the user field
of view By detecting features in images received through the camera
and matching them with images stored in database, the field of view
can be computed
School of Computer Science and EngineeringPage #
System Design
School of Computer Science and EngineeringPage #
Proposed Tracking
Feature matching between image from camera and stored image in
database is used to find the corresponding tapped bacterium in the
image stored in databaseCorresponding bacterium
School of Computer Science and EngineeringPage #
Information Retrieval
Information from the bacteria in database is extracted and
translated back to the position of corresponding bacterium on the
image from cameraCorresponding bacterium
School of Computer Science and EngineeringPage #
Dimension: Length, WidthThe selected bacterium highlighted and
information from database displayed on the screen of handheld
device
School of Computer Science and EngineeringPage #
School of Computer Science and EngineeringPage #
Feature matching for tracking: Comparing different feature
detectors and descriptors on accuracy and real-time
performanceFeature DetectorFeature DescriptorFast : Using a window
of 16 Pixels to classify whether a pixel is actually a cornerBrief
: A simple comparison of pixel pairs around feature points.ORB : A
FAST or Harris detector ORB: A BRIEF descriptor with some hints
about the key point orientation SIFT : Using Scale-Space Pyramid
and DoG to detect feature pointsSIFT: A 128 vector of orientation
histogram of pixels around the feature pointSURF: Using a
simplified version of Laplacian of GaussianSURF: Wavelet response
in horizontal and vertical direction
School of Computer Science and EngineeringPage #
Real-time PerformanceDifferent combination of feature detector
and descriptor used for designing the tracking algorithm.The
application runs for 30 seconds and frame frame rates was recorded
for each different combination.
School of Computer Science and EngineeringPage #
Real-Time PerformanceFast BriefFast SiftFast SurfOrb ORBSIFT
SIFTSURF SURF
School of Computer Science and EngineeringPage #
Real-Time performanceDetectorDescriptorFrame
rateFASTBRIEF30FASTSIFT2-5FASTSURF4ORBORB8-10SIFTSIFT1SURFSURF1
WINNER
School of Computer Science and EngineeringPage #
AccuracyThe application accuracy is estimated by measuring the
acceptable range of device rotation.The acceptable range is the
maximum rotation in every direction before the application loses
the bacterium position between two consecutive tapsThis is carried
out by comparing the positions extracted from inverse homography of
different matching methods with results from SURF matching inverse
homography method in different device orientations. The reason for
selecting SURF as the base model is because of its rotation and
scale invariant properties.
School of Computer Science and EngineeringPage #
AccuracyFeature DetectorFeature DescriptorAccuracyFASTBRIEF15
degree in each directionFASTSIFT45 degree in each
directionFASTSURF90 degree in each directionORBORB85 degree in each
direction
WINNER
School of Computer Science and EngineeringPage #
ConclusionThe AR application can run in 30 frame per second
using FAST/BRIEF feature detector and descriptorThe FAST/BRIEF
combination has limited acceptable device rotation, which drop
usability of the application. There is a trade-off between accuracy
and real-time performance for AR application in high-dense
environment Further research is necessary for developing more
accurate feature with real-time capability
School of Computer Science and EngineeringPage #
Pause the movieControl the pointerExtract and zoomed in Changing
video frameResume video play
Pause the movieControl the pointerZooming in and out
Strongly DisagreeStrongly Agree
1I am familiar with this kind of system
12345
2I found the system is complex
12345
3I thought the system was easy to use
12345
4I think that I would need some help to be able to use this
system
12345
5I found the various functions in this system were well
integrated
12345
6I thought there was too much inconsistency in this system
12345
7I imagine that most people would learn to use this system very
quickly
12345
8I found the system very cumbersome to use
12345
9I felt very confident using the system
12345
10I needed to learn a lot of things before using this system
12345
11Which screen felt more naturaldomeiPad
12Other Comments and recommendations: