The Current State of Wearable Computing (2010)

• 1. wearable computing Jesse Venticinque Ubiquitous ComputingJesse Venticinque | Wearable Computing

2. Wearable computing (wearcomp) Vision & characteristicsEveryday wearcomp Commercial applications Current research Challenges Research directions Wearcomp design Jesse Venticinque | Wearable Computing 3. Wearable computing (wearcomp) 4. Wearable computing (wearcomp) small,light, portable computer designed to be worn on the body integrated into the user's clothing or attached to the body like a wristband or pocketintegrated into everyday objects that are constantly worn on the body, like a cell phone or watchuncommon intersection of science, engineering, design, psychology&fashionJesse Venticinque | Wearable Computing 5. Wearcomp Vision Consistency: always on; Constancy: always there - persistent access to information - everyday continuous use, interact at any give time Situational Awareness: - sense and model context: environment, physical, mental - aware of wearers situation and preferences - outputs are appropriate, desirable, and socially gracefulAugment & mediate interactions with users environment - with information from the physical and virtual realmsJesse Venticinque | Wearable Computing 6. Wearcomp Vision Anticipation & availability: - make relevant information available before wearer asks for it Perceive the world from the first-person perspective - seeing what you see and hearing what you hearJesse Venticinque | Wearable Computing 7. Everyday WearcompJesse Venticinque | Wearable Computing 8. Everyday WearcompJesse Venticinque | Wearable Computing 9. Everyday WearcompJesse Venticinque | Wearable Computing 10. Everyday WearcompJesse Venticinque | Wearable Computing 11. Everyday WearcompJesse Venticinque | Wearable Computing 12. Everyday WearcompJesse Venticinque | Wearable Computing 13. Everyday WearcompJesse Venticinque | Wearable Computing 14. Everyday WearcompJesse Venticinque | Wearable Computing 15. Mobile Phones Most successful wearable computer to date Want (2008) - Determine Transportation Modes on Mobile Phones * Reddy et al., 2009 (UCLA) * classified stationary, walking, running, biking, & motorized travel with greater than 98% accuracy * GPS & accelerometers- Output: auditory, visual, tactile - Location Tracking - Context Awareness - Input / Output - Games & Entertainment - Augmented RealityJesse Venticinque | Wearable Computing 16. Commercial Wearcomp - BodyMediaJesse Venticinque | Wearable Computing 17. Jesse Venticinque | Wearable Computing 18. Commercial Wearcomp - ZyPad 19. Commercial Wearcomp - Fossil 20. Wearcomp in academia Augmented Reality Capture & Access Gesture Recognition Posture Recognition Activity Sensing Context Awareness Healthcare & Assistive TechnologiesJesse Venticinque | Wearable Computing 21. What's a Wearable? A person's computer should be worn, much as eyeglasses or clothing are worn, and interact with the user based on the context of the situation. With heads-up displays, unobtrusive input devices, personal wireless local area networks, and a host of other context sensing and communication tools, the wearable computer can act as an intelligent assistant, whether it be through a Remembrance Agent, augmented reality, or intellectual collectives.Jesse Venticinque | Wearable Computing 22. Jesse Venticinque | Wearable Computing 23. MIT media lab sixthsense a wearable gestural interface that augments the physical world around us with digital information and lets us use natural hand gestures to interact with that information.http://www.pranavmistry.com/projects/sixthse nse/ http://www.youtube.com/watch?v=nZVjUKAsao Jesse Venticinque | Wearable Computing 24. Augmented Reality Sensory-Augmented Computing: Wearing the Museums Guide (Schiele, 2001)Jesse Venticinque | Wearable Computing 25. Augmented Reality Direct Retinal Displays - University of Washington HIT lab Set of 3 colored lasers scan image directly onto retinal surface - Scary but it works - Very high contrast, all in focus, - Potential for very very high resolution - Has to be head mounted - Large field of view, greater than 120 degrees - Very low power consumption Commercial applications of the Virtual Retinal Display are being developed at Microvision Inc.Jesse Venticinque | Wearable Computing 26. Jesse Venticinque | Wearable Computing 27. Jesse Venticinque | Wearable Computing 28. Jesse Venticinque | Wearable Computing 29. Jesse Venticinque | Wearable Computing 30. Jesse Venticinque | Wearable Computing 31. Jesse Venticinque | Wearable Computing 32. Augmented Reality Contact Lens Displays; Bionic Eyesight - Babak A. Parviz - worn daily by more than a hundred million -- build into the lens a display based on an array of LED pixels-- use passive pixels that modulate incoming light rather than producing their own.--construct an image by changing their color and transparency in reaction to a light sourceJesse Venticinque | Wearable Computing 33. Capture & Access Perceptual Rembrance Agent: Audio visual associative memory systemJesse Venticinque | Wearable Computing 34. Capture & Access Recollection of past events - e.g., associating a video or audio clip of the communication with a business card - remembering day-to-day information, visual & audio snapshots - augments human memoryJesse Venticinque | Wearable Computing 35. Gesture Recognition Hambone (Deyle, Palinko, Poole, ThadStarner) - sound generated by hand and ankle movements travel to device via bone conduction -Gesture Watch: controls play & pause commands on mobile media players Pointing (Oakley, Sunwoo, Cho) - Investigate pointing performance (via fitts law)Jesse Venticinque | Wearable Computing 36. Gesture Recognition Enabling Always-Available Input with Muscle-Computer Interfaces T. Scott Saponas, Desney Tan, Dan Morris, RavinBalakrishnan, Jim Turner, James Landay- UIST '09 - Proceedings of the 22nd annual ACM Symposium on User Interface Software and Technology file:///Users/jessev/Documents/MCHI/SAUI/UIST%20Proceedings/proceedin gs/videos/file215.wmvJesse Venticinque | Wearable Computing 37. Gesture Recognition Abracadabra: Wireless, High-Precision, and Unpowered Finger Input for Very Small Mobile Devices -Chris Harrison, Scott Hudson - UIST '09 - Proceedings of the 22nd annual ACM Symposium on User Interface Software and Technology file:///Users/jessev/Documents/MCHI/S AUI/UIST%20Proceedings/proceeding s/videos/file142.movJesse Venticinque | Wearable Computing 38. Gesture Recognition Real-Time American Sign Language RecognitionJesse Venticinque | Wearable Computing 39. Posture Recognition Elastomer strain sensors - measures strain in clothing Sustained Logging and Discrimination of sleep postures with low-level wristworn sensors - sleep Postures using wrist-worn sensors - detect nightly sleep periods & postures - Mattmann et al.Jesse Venticinque | Wearable Computing 40. Healthcare applications MiThrilLiveNet: wearable real-time shiver monitor for soldiers - continuous accelerometer sensing - shivering correlates with core body temperature - ultimate application: real-time wearable hypothermia classification systemJesse Venticinque | Wearable Computing 41. Healthcare applications AMON: A Wearable computer for High Risk Patients - wrist worn medical monitor -frees patients from stationary monitoring equipment - continuously monitors and logs pulse, blood, oxygen saturation and temperature - Lukowicz et al. (2002)Jesse Venticinque | Wearable Computing 42. Activity Recognition Recognize an activity, adaptinterface and information - jogging / audio notification vs. - sitting / text notification Often uses machine learning algorithms - trade off between training effort & recognition rate - differing accuracy for users who werent in the training set and those who were - off the shelf activity recognition Spelmezan, Schanowski, &Borchers: Snowboard trainingJesse Venticinque | Wearable Computing 43. Activity Recognition Wireless Force Sensing Body Protectors for Martial Arts - accurately measure amounts of power delivered to the body - objective: support judges scoring - Chi, Song, Corbin : PARC & StanfordJesse Venticinque | Wearable Computing 44. Activity Recognition SwimMaster: A Wearable Assistant for Swimmer - improve swimmers technique - accelerometer sensors mounted on swimmers back - actuator devices: visual, acoustic &haptic feedback during swimming - measure Time, swimming velocity, # of strokes, distance per stroke Limitations: Participants: 12 intermediates, no experts Is this comfortable? Does the wearable affect swimming technique? Do users become better swimmers? Cant correct / teach while swimming Jesse Venticinque | Wearable Computing 45. Input Augmenting Spatial Awareness with Haptic Radar - head-mounted device; - alerts to user about nearby objects - proximity sensors mounted on headband w/ mobile phone vibrator motors. - stronger vibrations for closer by objects- experimenter smack blindfolded participant with foam ball - most participants moved in response; more than half totally avoided it - Cassinelli, Reynolfs, IshikawaJesse Venticinque | Wearable Computing 46. Output Tactual Displays &Haptic Interfaces: - Tan &Pentland, MIT (1997) - when audio/visual channels are heavily loaded - use existing tactile information conveyances: * tap on the shoulderJesse Venticinque | Wearable Computing 47. Wearcomp ChallengesJesse Venticinque | Wearable Computing 48. Wearcomp Challenges: Wearability How well the human body can support a deviceAssessing the Wearability of Wearable Computers - impact on energy expenditure, biomechanical effects on posture, movement, comfort - Knight et al. (2007) Electronic Textiles Smart Cloths, eTextiles - Leah Buechley - MIT Media Lab - combines fabric with electronics - perception of clothing not computers * greater acceptability for consumer applications Dynamic wearability extends that definition to include the human body in motion. Jesse Venticinque | Wearable Computing 49. Wearcomp Challenges: Power Considerations Perhaps the most limiting factor in mobile technology (Starner) Massoften determined more by power source than underlying electronics How to spread power among clothing?Rechargeablebatteries? Plug yourself in? Charge while moving? While acting? - act of placing shoes / jacket in closet establish wired or wireless connections - leverage natural human movement - energy expended in pressing buttons on a one-handed keyboard power the keyboard Jesse Venticinque | Wearable Computing 50. Wearcomp Challenges: Heat Dissipation Problem with high end laptopsUse the the arms pendulum like movement to dissipate heat while walking? Dissipate heat directly to user a greatly appreciated situation in the winter Write applications for wearable computers with heat dissipation in mind - Delay computing processes until it senses a cooler environment Active cooling technology Phase-change materials resistant to heatJesse Venticinque | Wearable Computing 51. Other Challenges Interoperability: communication standards & protocols - wireless communication from mobile devices to a fixed infrastructure Privacy: intimate experience, very-personal information - time spent on long coffee breaks? In the bathroom? - are others/strangers constantly wearing video cameras and microphones? Interface Design: - evaluative methods? - size & body placement, form factor - portability and usability - user attention - fashion, Aesthetics? Jesse Venticinque | Wearable Computing 52. Other Challenges From you: Affordability? Health Concerns?, Will it make us lazy? Toomuch data to sort through Too many devices & moving parts? Readability? Fragility?Authenticityor the realness of relationships Better way to present the same information? Bulkiness? Network Coverage & Infrastructure? Too intrusive? Latency? Lack of visual displaysIf life evolves with an emphasis on always being better, faster, smarter, richer...is there ever a way to reach peace with who we are? Is there something to say for aspiration over attainment? unless its approaching the coolness factor of sci-fi movies, these gadgets will not be adopted into the mainstream Jesse Venticinque | Wearable Computing 53. Future Research Directions 10th Annual IEEE Computer Society International Symposium on Wearable Computers: - Evaluation: testing wearable computing ideas under realistic conditions and solving problems both theoretically and practically - Context: Not just about form factor and computational capabilities, but about sensing where we are and what were doing with them - Usability: User interfaces &Wearability.Jesse Venticinque | Wearable Computing 54. Wearcomp: the role of design Whats the preferred future state? Rich Interaction Aesthetics of interaction The Design Process: Experience Prototyping Experience Simulation How should design contributions be evaluated? Cultural gaps? Important design issues for wearable computing? Jesse Venticinque | Wearable Computing 55. The role of design MIT: Seamless. Computational Couture a fashion event featuring innovative and experimental works in computational apparel design, interactive clothing, and technology-based fashion.http://www.youtube.com/watch?v=LoyNqCsksdgJesse Venticinque | Wearable Computing 56. the resulting partnership will think as no human brain has ever thought. - StarnerJesse Venticinque | Wearable Computing 57. Invisibility Cloak http://www.youtube.com/watch?v=JKPVQal851U The Gadget Show - The Critical List: Wearable Tech http://www.youtube.com/watch?v=VHp8XcSaRTs&feature=fvwJapanese Wearable Computing Systems http://www.youtube.com/watch?v=jAdZ7zZy5Gs Advances in Wearable Computing and Computer Vision. http://www.youtube.com/watch?v=jAdZ7zZy5GsJesse Venticinque | Wearable Computing 58. Healthcare applications Wearable medical training equipment (Lvquist&Kulcsr, 2007) - Interactive multimodal learning environment using haptics - Experience of haptic perceptions, intuitive interactionJesse Venticinque | Wearable Computing 59. Healthcare applications Electronic Multiuser Randomized Circuit Training (EMURCT) - Audible and visually prompts trainee to begin and end an exercise - Instructing the Trainees through their workout. - Motivate and keep trainees on track while working out. - Graves et al.Jesse Venticinque | Wearable Computing 60. Wearcomp Gaming Unmaksing Mister X : - Combines physical & virtual environments - Multiplayer, pplayers must find Mister X - Each player equipped with a sensing device & head mounted display - Sensor values (movement &phsyicalenviroment) for Mr. X displayed in each players display - Unmask Mr. X by checking if sensor values match physical enviroment and movement of another player -Mr. X: act in a way that avoids detection, attentionJesse Venticinque | Wearable Computing 61. Augmented Reality Video-Rate Localization in Multiple Maps for Wearable Augmented Reality - SLAM: Simultaneous Location and Mapping - Decouples tracking from mapping - Uses multiple maps is more scalable; disconnected maps of the world can cover a larger total than a single map could - Best paper at 2008 International Symposium on Wearable Computing at CMU - Castle, Georg Klein, David W MurrayJesse Venticinque | Wearable Computing 62. Wearcomp Challenges: Power Considerations Keystroke-Level Energy Model - Lu Luo& Daniel Siewiorek (2007) - predicts the user-time and system-energy consumption it will take to perform an interactive taskPowerPack System - Backpack based system with wireless power distributionJesse Venticinque | Wearable Computing