1 Playful Toothbrush: Ubicomp Technology for Teaching Too th Brushing to Kindergarten Childre n CHI 2008, Florence Italy Yu-Chen Chang, Chao-Ju Huang, Hao Chu, Peggy Chi National Taiwan University Jin-Ling Lo, Nan-Yi Hsu, Hisn-Yen Wang, Ya-Lin Hsieh National Taiwan University Hospital
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
1
Playful Toothbrush: Ubicomp Technology for Teaching Tooth Brushing to Kindergarten Children
CHI 2008, Florence Italy
Yu-Chen Chang, Chao-Ju Huang, Hao Chu, Peggy ChiNational Taiwan University
• Child behavior training as important but challenging parental responsibility– Potty training, self-dressing, cleaning room, self-
feeding, tooth brushing (this work).
3
Persuade Children to Change Behaviors
• The most common form of parental persuasion is verbal persuasion.– “If you don’t brush your teeth properly and thoroughly, you are not
allowed to go to sleep”.• Not effective, why not?
– Verbal persuasion alone lacks proper incentive to motivate children
4
A Case Study: Child Tooth Brushing
• Target kindergarten children (5-6Y)
– Learn to care for their own oral hygiene– Average 5Y children brush only 1/4 of teeth (Rugg-Gunn)– A common scenario: candies → improper brushing → cavities → dent
al clinic
5
Play-based Occupational Therapy
• Pediatric occupational therapy (O.T.)– Leverage the desire of children to play to induce their behavior change
willingly. • Children may not like to brush teeth, but like to play.• Add playfulness (game) into the brushing activity
– Effective – full of toys in pediatric O.T. clinic.
6
Play-Based O.T. Limitations
(1) Children treated in clinics during regular office hours (NTU Hospital)– Many child behavior problems not observable to therapists
(2) Train general performance skill rather than specific functional skill– Hand dexterity skill vs. tooth brushing skill– Improvement in general performance skills may not translate into the
target functional skills
7
HCI/UBICOMP Opportunities
• Embed digital technology into a child naturalistic living environment– Sensing to observe child behavior anytime, anywhere– Game playing to influence child behavior anytime, anywhere
• Occupational therapist perspective: – From treatment clinic– To the child actual living environment (functional behaviors)
• HCI/UBICOMP perspective:– From sense and track behaviors – To engage children to change behaviors
• Also called Persuasive technology (by Fogg, King, and others)
8
Playful Toothbrush Goals
• (Child) Dislike brushing– Make brushing more attractive
• (Child) Habitual brushing but not properly– Teach proper brushing skill
• Not replace adults’ supervision– Extend their effectiveness– Used together can overcome young
children’s limited physical and cognitive abilities, such that they can successfully learn proper brushing
9
• Sensing– Camera vision to detect children’s
brushing actions– Brushing actions become game inputs
• Playing– An interactive game of brushing teeth– Start with a mirror image of dirty teeth.– Brushing own teeth maps to cleaning
the same virtual teeth in the mirror image.
• Demo video
Playful Toothbrush
Brush extension
Web camera for tracking brushing motions
Camera #2 to record videos for later human analysis
Brushing game
10
Play-based Occupational Therapy Model
• Playful toothbrush is a treatment program / play activity (three steps).
Volition Performance Habituation
Bring enough enjoyment to attract a child to participate in the target activity.
Ensure a child will have a successful experience. Set appropriate level of difficulty.
Apply reinforcement to reward good performance, so increase change of repeating desirable behavior. Enough times to become a habit.
11
Design Considerations
• Enjoyment– Bring enough fun to attract children to brushing– Associate brushing with game playing and having fun
• Engagement– Simple Interaction (Not all young children can learn to operate complex devices)
– Use their natural brushing actions as game inputs • Automation
– Help children learn tooth brushing skill and internalize as habit• In brushing case, internalizing means doing without much thinking)
– Advice adults not to interrupt children and their own motor planning, necessary for internalization of brushing motor skill
12
Playful Toothbrush Prototype (two main components)
Vision-based brushing tracker
Brushing game
13
Vision-based Brushing Tracker• Use one camera to detect brushing
– Recognize brushing of 24 different teeth surface areas (or granularity of 2 adjacent teeth surfaces)
• Mouth closed during brushing – Bristle-teeth contact area not visible to an
external camera• Brush extension
– Each of four faces has unique LED-pattern– Used as marker to assist camera-vision
recognition– Children told not to block brush extension from the camera
14
How to recognize a brushing stroke on a specific teeth surface area?
• Take an example (brushing the outer surface of frontal teeth)
• Three features determines this brushing stroke– Bristle rotated toward the face– Brush oriented parallel to the face– Back-and-forward motion vector (parallel to the face)
15
Computer Vision Technique
• Use the brush extension marker to reconstruct– Bristle rotation angle (x-axis)– Brush orientation angle (z-axis)– Brush back-n-forward motion vector
• Infer the brushing stroke and its target teeth area
x (θroll)
z (θyaw)
y
brush extension
16
Accuracy Test (vision-based brushing tracker)
• 13 kindergarten children (72-81 months)• Recorded 48 minutes of their brushing videos• Compare human-read brushing strokes (ground-truth) and
– Need not be perfect. Occasional error in games has little effect.
17
Childrenfalse negative rate
(frame) (%)false positive rate
(frame) (%)
1 602/6521 9.2% 47/483 9.7%
2 880/5958 14.8% 49/332 14.8%
3 331/5130 6.5% 209/1304 16%
4 617/5270 11.7% 20/380 5.3%
5 209/4813 4.3% 60/634 9.5%
6 90/4551 2% 0/80 0%
7 290/5520 5.3% 0/196 0%
8 579/4302 13.5% 206/1288 12.2%
9 239/7104 3.4% 23/688 3.3%
10 32/3532 0.9% 115/1298 8.6%
11 695/6847 10.2% 111/630 17.6%
12 118/4974 2.4% 0/168 0%
13 241/6493 3.7% 41/343 12.0%
Average 6.8% 8.4%
18
Brushing Game
• Start with a virtual mirror image of the children’s own uncleaned teeth– Grouped into 24 teeth surface areas
• “1” appears next to the first cleaning target– 7 layers of plaques were drawn, requiring 7
brushing strokes– Each brushing stroke removes one layer of p
laque– Combine audio feedback: Do-Re-Mi-Fa ..– No game response for brushing other teeth
areas
19
Brushing Game
• After 1st area is complete, “2” appears next to the 2nd target area.
• Applause at the game completion– Visual-audio feedbacks provide a sen
se of accomplishment • Enforce a brushing sequence (Stillm
an’s brushing method)– Upper right, upper left, the lower righ
t, and lower left • Two benefits of sequencing
– Ensure brushing all teeth– Repeat this sequence enough time ->
“do without thinking”
2
23
20
User Study
• Two questions guided our user study– How effective is the Playful Toothbrush in improving the
brushing skills of kindergarten children?– What aspects of brushing behaviors were affected by
Playful Toothbrush?
21
User Study Test Subjects
• 13 young children (72 – 81 months) from a NTUH-run kindergarten class– 5 girls and 8 boys
• Written informed consent from parents• Tooth brushing is required after meals or snacks
• Setup– Install our system at the kindergarten’s restroom– A video camcorder to record children’s brushing sessions
22
User Study Procedure (11 days spanned over 3 weeks)
• Pre-trial (one day)– Familiarize with the therapists and our system
• Pretest (two days)– Brushed with their own toothbrushes – Established a baseline behavior
• Training (five days)– Brushed with our playful toothbrush– A trained therapist helps children understand the brushing game
• Posttest (two days)– Brushed with their own toothbrushes– Measured behavior improvement from pretest
• One week follow up (one day)– Brushed with their own toothbrushes– Measured behavior retention after a week
23
User Study: Brushing Effect
• Teeth cleaning effectiveness– Red plaque disclosing dye before each
brushing session (dye attached the plaque)
– Oral exam (counted teeth surfaces with dye before and after brushing)
– Plaque index• number of teeth surfaces with plaque /
total number of teeth surfaces– Cleaning effect
• Reduction of Plaque Index from before to after brushing
24
User Study: Brushing Behavior
• By analyzing/coding brushing videos, measured 3 aspects of brushing behaviors:– Length of brushing time– Number of brushing strokes on each of 24 teeth areas– Total number of brushing strokes
25
User Study: Cleaning Effect Results
Before brushing Mean(SD)
After brushingMean(SD)
Cleaning effectMean (SD)
Pre-test
Day 3 0.69(0.25) 0.37(0.18) 0.32(0.21)
Training
Day 4 0.76(0.14) 0.09(0.10) 0.67(0.15)
Day 6 0.68(0.28) 0.04(0.05) 0.64(0.26)
Day 8 0.79(0.18) 0.11(0.10) 0.68(0.17)
Average 0.74(0.19) 0.08(0.06) 0.66(0.17)
Post-test
Day 9 0.83(0.11) 0.16(0.10) 0.67(0.13)
Day10 0.86(0.16) 0.15(0.08) 0.70(0.15)
Average 0.85(0.10) 0.16(0.07) 0.66(0.28)
Follow-up
Day 11 0.88(0.14) 0.18(0.10) 0.70(0.14)
26
Number of brushing strokes Mean(SD)
Number of unbrushed teeth areas Mean(SD)
Brushing time Mean(SD) (sec)
Pretest
Day 2 212.31(137.77) 11.31(5.25) 84(53)
Day 3 168.62(157.03) 13.46(5.35) 67(46)
Average 190.46(138.38) 12.39(4.75) 76(45)
Training
Average
Posttest
Day 9 281.69(120.66) 7.46(4.89) 137(41)
Day 10 214.31( 71.91) 9.46(5.12) 99(31)
Average 248.00(87.12) 8.46(4.82) 118(30)
Follow-up
Day 11 239.62(107.48) 8.31(5.07) 120(36)
User Study: Brushing Behavior Results
27
Result Summary
• (Pretest) Child subject failed to clean 37% of their teeth surfaces• After 5 training days, improvements in
– Teeth cleaning effectiveness– Number of brushing strokes– Length of brushing time– Coverage of brushed teeth areas
28
Conclusion
• Demonstrate a case study of applying HCI/UBICOMP technology in play-based occupational therapy
• For teaching young children proper tooth brushing, user study results were encouraging
• Other similar repetitive development tasks for young children (self-feeding, potty training, cleaning room, etc.) – This technique can make training attractive and simple for
adults/children.
29
Examples of Other Persuasive Technologies
30
• Sense to recognize behavior
– Weight sensor underneath the tray to sense eating actions
– Eating actions as game input• Play to engage behavior change
– Interactive games: coloring cartoon character or penguin fishing
Case Study: Playful Tray Encourage good eating habit in young children
31
• Sense to recognize behavior
– Tilt sensor to detect drinking actions– Drinking actions are game inputs
• Play to engage behavior change
– Game metaphor: hydrating/dehydrating body -> watering/drying a tree
Case Study: Mug-Tree Encourage healthy habit of drinking fluid regularly
32
• Sense to recognize dress & dressing color
– Camera and Computer Vision• Play to engage behavior change
– Playful explore & experiment with how different clothing color look on people
• CHI 2008 poster on Tuesday
Case Study: ChroMirror Persuade people to explore more colorful dressing
33
Thanks
Q & A
34
Lesson #4
• Mediate, not automate – Mediation: not to replace children’s efforts but make the
experience of performing more enjoyable for children– Mediation works better than automation in this case.
35
Lesson #1
• Unpredictable behaviors from young children – Make activity recognition difficult
• Randomly switch from left to right hands, swing brush wildly, head movement brushing the brush …
Observe child behaviors carefully before designing and programming the system
36
Lesson #2
• Activity grading– Children have different physical and cognitive capabilities
• Some learned fast (not challenging enough); some learned slowly (frustration)
– Different levels of challenges fitting to each child’s ability
“Play-based O.T. is about the experience of performance, or the fitness between the level of challenge in an activity
and a child’s physical and cognitive capabilities.”
37
Lesson #3
• Personalization & customization– Easily personalized and customized to environmental or
human factors (preferences of children, changing performance of a child, different deployment environments).
– Left hand vs. right hand – Lighting condition– Child height– Preferences about game characters
38
Implementation Details
• Toothbrush extension weight– Not too heavy for children to cause inconvenience in brushing
• Prevent child from holding the brush extension– Block the brush extension from the overhead camera– Solution: a protruding cap as separator
• Camera placement– Not too high such that camera cannot see the brush extension clearly.– Not too low such that children can switch the brush extension outside