Google’s Project Tango Department of CSE 1 | Page 1. INTRODUCTION 3D models represent a 3D object using a collection of points in a given 3D space, connected by various entities such as curved surfaces, triangles, lines, etc. Being a collection of data which includes points and other information, 3D models can be created by hand, scanned (procedural modeling), or algorithmically. The "Project Tango" prototype is an Android smartphone- like device which tracks the 3D motion of particular device, and creates a 3D model of the environme nt around it. Project Tango was introduced by Google initially in early 2013, they described this as a Simultaneous Localization and Mapping (SLAM) system capable of operating in real-time on a phone. Google’s ATAP teamed up with a number of organizations to create Project Tango from this description. The team at Google’s Advanced Technology and Projects Group (ATAP) has been working with various Universities and Research labs to harvest ten years of research in Robotics and Computer Vision to concentrate that technology into a very unique mobile phone. We are physical being that live in a 3D world yet the mobile devices today assume that the physical world ends the boundaries of the screen. Project Tango’s goal is to give mobile devices a human-sca le understanding of space and motion. This project will help people interact with the environment in a fundamentally different way and using this technology we can prototype in a couple of hours something that would take us months or even years before because we did not have this technology readily available. Imagine having all this in a smartphone and see how things would change. The first product to emerge from Google's ATAP skunkworks group, [1] Project Tango was developed by a team led by computer scientist Johnny Lee, a core contributor to Microsoft's Kinect. In an interview in June 2015, Lee said, "We're developing the hardware and software technologies to help everything and everyone understand precisely where they are, anywhere." [2] This device runs Android and includes development APIs to provide alignment, position or location, and depth data to regular Android apps written in C/C++, Java as well as the Unity Game Engine (UGE). These early algorithms, prototypes, and APIs are still in active development. So, these are experimental devices and are intended only for the exploratory and adventurous are not a final shipping product. Project Tango technology gives a mobile device the ability to navigate the physical world similar to how we do as humans. Project Tango brings a new kind of spatial perception to the Android device platform by adding advanced computer vision, image processing, and special vision sensors. Project Tango is a prototype phone containing highly customized hardware and software designed to allow the phone to track its motion in full 3D in real-time. The sensors make over a quarter million 3D measurements every single second updating the position and rotation of the
This document is posted to help you gain knowledge. Please leave a comment to let me know what you think about it! Share it to your friends and learn new things together.
Transcript
Google’s Project Tango
Department of CSE 1 | P a g e
1. INTRODUCTION
3D models represent a 3D object using a collection of points in a given 3D space, connected by
various entities such as curved surfaces, triangles, lines, etc. Being a collection of data which
includes points and other information, 3D models can be created by hand, scanned (procedural
modeling), or algorithmically. The "Project Tango" prototype is an Android smartphone- like
device which tracks the 3D motion of particular device, and creates a 3D model of the environment
around it.
Project Tango was introduced by Google initially in early 2013, they described this as a
Simultaneous Localization and Mapping (SLAM) system capable of operating in real-time on a
phone. Google’s ATAP teamed up with a number of organizations to create Project Tango from
this description.
The team at Google’s Advanced Technology and Projects Group (ATAP) has been
working with various Universities and Research labs to harvest ten years of research in Robotics
and Computer Vision to concentrate that technology into a very unique mobile phone. We are
physical being that live in a 3D world yet the mobile devices today assume that the physical world
ends the boundaries of the screen. Project Tango’s goal is to give mobile devices a human-sca le
understanding of space and motion. This project will help people interact with the environment in
a fundamentally different way and using this technology we can prototype in a couple of hours
something that would take us months or even years before because we did not have this technology
readily available. Imagine having all this in a smartphone and see how things would change.
The first product to emerge from Google's ATAP skunkworks group,[1] Project Tango was
developed by a team led by computer scientist Johnny Lee, a core contributor
to Microsoft's Kinect. In an interview in June 2015, Lee said, "We're developing the hardware and
software technologies to help everything and everyone understand precisely where they are,
anywhere."[2]
This device runs Android and includes development APIs to provide alignment, position
or location, and depth data to regular Android apps written in C/C++, Java as well as the Unity
Game Engine (UGE). These early algorithms, prototypes, and APIs are still in active development.
So, these are experimental devices and are intended only for the exploratory and adventurous are
not a final shipping product.
Project Tango technology gives a mobile device the ability to navigate the physical world
similar to how we do as humans.
Project Tango brings a new kind of spatial perception to the Android device platform by
adding advanced computer vision, image processing, and special vision sensors.
Project Tango is a prototype phone containing highly customized hardware and software
designed to allow the phone to track its motion in full 3D in real-time. The sensors make over a
quarter million 3D measurements every single second updating the position and rotation of the
Tango wants to deconstruct reality, taking a quarter million 3D measurements each second to create a real-time 3D model that describes the physical depth of its surroundings.
The smartphone specs are
The above specs include Snapdragon 800 quad core CPU running up to 2.3 GHz per core, 2GB or
4GB of memory, an expandable 64GB or 128 of internal storage, and a nine axis
accelerometer/gyroscope/compass. There’s also a Mini-USB, a Micro-USB, and USB 3.0.
In addition to above specs Tango’s specs also include: a rear-facing four megapixe l
RGB/infrared camera, a 180-degree field-of-view fisheye rear-facing camera, a 120-degree field-
of-view front facing camera, and a 320 x 180 depth sensor – plus a vision processor with one
teraflop of computer power. Project Tango uses a 3000 mAh battery.
Google’s Project Tango
Department of CSE 9 | P a g e
4. HARDWARE
Project Tango is basically a camera and sensor array that happens to run on an Android phone.
The smartphone is equipped with a variety of cameras and vision sensors that provides a whole
new perspective on the world around it. The Tango smartphone can capture a wealth of data never
before available to application developers, including depth and object-tracking and instantaneous
3D mapping. And it is almost as powerful and as big as a typical smartphone. The Front View and
Back View of a Tango Phone is shown below.
It is same like some other phones but the phone is having variety of cameras and sensors that make
the 3D modelling of the environment possible.
Fig (6) Tango Phone Front View
The device tracks the 3D motion and creates a 3D model of the environment around it by using
the array of cameras and sensors. The phone emits pulses of infrared light from the IR projector
and records how it is reflected back allowing it to build a detailed depth map of the surrounding
space.
There are three cameras that capture a 120-degree wide-angle field of view. 3D camera captures
the 3D structure of a scene. Most cameras are 2D, meaning they are a projection of the scene onto
the camera's imaging plane; any depth information is lost. In contrast, a 3D camera also captures
the depth dimension (in addition to the standard 2D data).A rear-facing four megapixe l
RGB/infrared camera, a 180-degree field-of-view fisheye rear-facing camera, a 120-degree field-
Google’s Project Tango
Department of CSE 10 | P a g e
of-view front facing camera, and a 320 x 180 depth sensor are the components of the phone at the
rear end that works together to give the 3D structure of the scene.
Fig (7) Tango Phone Back View
Project Tango, which Google paired with sensors and cameras to give the smartphone the same
level of computer vision and tracking that formerly required much larger equipment. The phone is
equipped with a standard 4-megapixel camera paired with a special combination of RGB and IR
sensor and a lower-resolution image-tracking camera. These combos of image sensors give the
smartphone a similar perspective on the world, complete with 3-D awareness and a awareness of
depth. They supply information to Movidius custom Myriad 1 low-power computer-vis ion
processor, which can then process the data and feed it to apps through a set of APIs. The phone
also contains a Motion Tracking camera which is used to keep track of all the motions made by
the user. The motherboard which contains all of these components is shown below
Project Tango is different from other emerging 3D-sensing computer vision products, such as
Microsoft Hololens, in that it's designed to run on a standalone mobile phone or tablet and is chiefly
concerned with determining the device's position and orientation within the environment.
The software works by integrating three types of functionality:
7.1 Motion Tracking:
Motion tracking allows a device to understand position and orientation using
Project Tango's custom sensors. This gives you real-time information about the 3D motion of a device. Motion-tracking: using visual features of the environment, in combination
with accelerometer and gyroscope data, to closely track the device's movements in space. Project Tango’s core functionality is measuring movement through space and
understanding the area moved through. Google API’s provide the position and orientation
of the user’s device in full six degrees of freedom, referred to as its pose.
7.2 Area Learning: Using area learning, a Project Tango device can remember the visual features of
the area it is moving through and recognize when it sees those features again. These features can be saved in an Area Description File (ADF) to use again later.
Project Tango devices can use visual cues to help recognize the world around them. They can self-correct errors in motion tracking and relocalize in areas they've seen before. . With an ADF loaded, Project Tango devices gain a new feature called drift corrections or
improved motion tracking. Area learning is the way of storing environment data in a map that can be re-used
later, shared with other Project Tango devices, and enhanced with metadata such as notes, instructions, or points of interest
Fig (15) Area Learning
7.3 Depth Perception: Project Tango devices are equipped with integrated 3D sensors that measure the
distance from a device to objects in the real world. This configuration gives good depth at
a distance while balancing power requirements for infrared illumination and depth processing.
The depth data allows an application to understand the distance of visible objects to the device. By combining depth perception with motion tracking, you can also measure distance between points in an area that aren’t in the same fame.
Google’s Project Tango
Department of CSE 18 | P a g e
Project Tango devices are equipped with integrated 3D sensors that measure the distance from a device to objects in the real world. Current devices are designed to work best indoors
at moderate distances (0.5 to 4 meters). It may not be ideal for close range object scanning. Because the technology relies on viewing infrared light using the device's camera, there
are some situations where accurate depth perception is difficult. Areas lit with light sources high in IR like sunlight or incandescent bulbs, or objects that do not reflect IR light cannot be scanned well.
By combining depth perception with motion tracking, you can also measure distances between points in an area that aren't in the same frame.
Fig (16) Depth Perception
Together, these generate data about the device in "six degrees of freedom"
(3 axes of orientation plus 3 axes of motion) and detailed three-dimensional information about the
environment.
Applications on mobile devices use Project Tango's C and Java APIs to access this data in
real time. In addition, an API is also provided for integrating Project Tango with the Unity game
engine; this enables the rapid conversion or creation of games that allow the user to interact and
navigate in the game space by moving and rotating a Project Tango device in real space. These
APIs are documented on the Google developer website.