Obsessed with technology?
Subscribe to the latest tech news as well as exciting promotions from us and our partners!
By subscribing, you indicate that you have read & understood the SPH's Privacy Policy and PDPA Statement.
Event Coverage

Google's Project Tango today: Building towards a believable AR experience

By PY Ho - 3 Jul 2017

Project Tango: Improvements in Google's AR efforts

A virtual lion overlay on physical entities as demoed on the upcoming ASUS ZenFone AR smartphone.

We first covered Tango in depth when Google made Tango an official development effort back in I/O 2016. Back then, Project Tango was still considered a work in progress and in the early stages of proving its worth; in fact, AR was hardly mentioned in the I/O 2016 keynotes at all. Fast forward to this year’s I/O, Project Tango made an entrance in the first keynote when Google announced its new VR and AR products and services. With Google lending prominence in Tango this year, it is time to revisit how has Tango improved over the past year.


Google Daydream’s WorldSense uses Motion Tracking principles 

Last year, we explained how Project Tango uses visual-inertial odometry to track user’s motion and location without using GPS on a mobile device. In Google I/O 2017, the announced Standalone Daydream headset will utilize such motion tracking principles, adding motion into the user’s VR experience. Google terms their method of visual-inertial odometry as WorldSense

The tracking system offers 6 degrees-of-freedom, which means you’ll be able to move more naturally and freely as the headset is capable of taking into account all the possible ways of navigating through 3D space. In comparison, the original Daydream headset only tracks head tilt and not position.


Visual Positioning Service is evolved from Area Learning

Finding one's way round the store using Visual Positioning Service

Last year, when we tested the Area Learning feature on the old Tango Dev Kit, the app that used to learn the area crashed pretty often. Area learning was also not very tolerant to changes in environment; it fails when an object, such as a table, moves even slightly.

Within a year, Google has made improvements in the processing and storing of area descriptor files. It also made Tango more tolerant to changes in environment. Confident that their area learning algorithm is well refined, Google announced Visual Positioning Service (VPS), a service that uses area learning, overlays it in Google indoor maps, and shows the user where he or she is on the indoor map quickly and more accurately than just GPS. In Google's words, GPS is sufficient to get you to the store, but VPS has the potential to direct you to exactly what you're looking for in the store. For someone who is visually impaired, VPS in addition to auditory clues can be a powerful tool for self empowerment. Museums too could benefit from VPS by capitalizing on its accuracy for personal virtual tours and extra information. Likewise, there are many other possibilities.


Faster image processing and smoother 3D reconstruction

Top: 3D reconstruction in the early days. Bottom: Current state of 3D reconstruction

One of the core propositions of Project Tango was to  accurately map and recreate a 3D environment that the Tango enabled mobile device exists. At its infancy last year, the reconstruction was slow, clunky, ugly, and barely usable for any meaningful post processing. It was however used for accurate positional data capture rather than re-creating the scene. At this year's I/O, processing capability has vastly improved: construction is very fast, post-rendering is snappier, and the resulting 3D model looks almost like the environment it has captured. This is no doubt due to refinements in the Project Tango software stack, as well as the newer mobile processing hardware available today.


Occlusion stands in the way

Imagine putting a virtual cat in a real box through a Tango enabled mobile device. Now move away from the box. You should not see the cat anymore. In technical terms, the virtual cat has been occluded by the physical box.

However, using this sandbox test as a benchmark, we have yet to see applications that have integrated occlusion well within their respective AR apps. A good number of Tango apps at the moment do not incorporate occlusion - or at least did not incorporate them well enough. Occlusion is key to providing a full immersive AR experience because using the above sandbox test example, it would be extremely weird for anyone to see the virtual cat lying outside the box, instead of staying hidden. We see occlusion as a key benchmark in defining the success of an app's AR experience.