Paul Gilster: Microsoft's HoloLens shows potential of augmented reality

Microsoft’s new HoloLens system is creating a stir in the world of “augmented reality.” A headset that allows you to overlay the real world with virtual imagery from a computer, HoloLens is all about enhancing what is in front of you rather than making it disappear into a fully immersive ‘virtual reality’ environment. You’re still anchored in the here and now, but you have capabilities overlaid onto it that only a computer can provide.

Case in point: NASA is working with Microsoft on a system called OnSight, which uses the company’s HoloLens system to enhance what scientists see when they interact with Mars rovers, such as Curiosity. Instead of looking into a flat screen and issuing commands to make the rover inch forward or drill into a rock, scientists using the headset will be able to experience Mars first-hand, using imagery from the rover itself. The wearable technology is designed to give them the sense that they are standing and working on the Martian surface.

Walking on Mars

This is more than just gaming technology taken into the space program.

OnSight uses what Microsoft is referring to as “holography,” although I’m seeing some buzz speculating on exactly how this is being implemented. (The system is still in its early stages, and we don’t yet know what’s inside.) But by surrounding researchers with the actual image of Mars and a virtual environment stuffed with Curiosity data, HoloLens lets them “walk” around on the surface, exploring the terrain from all angles, selecting commands to “drill down” into deeper data.

Another scientist wearing the same gear will appear as a distinct figure, so that groups can work together in planning things like the next logical operations for the distant rover. Imagine moving entirely around a rock formation to scrutinize it in detail, consulting with team members about where Curiosity should move for a closer look, scouting the terrain for possible obstacles. Curiosity will begin using OnSight later this year in test mode, and if the system works as advertised, the technology may play a much larger role in the rover to be sent to Mars in 2020.

Immersive experience

We’ll have to see how HoloLens plays out. Meanwhile, the question I’m always asked about the space program – What happened to manned spaceflight after Apollo that has left us stranded in Earth orbit ever since? – may have an answer in augmented and virtual reality. We have unmanned robotic probes all over the solar system, with one of them closing on the largest asteroid, Ceres, right now and another slated to reach Pluto and its moon Charon in July.

Imagine if probes of the future, not just on Mars but on exotic locations such as Saturn’s moon Titan, or hellish Venus, or even strange, geyser-spewing Enceladus, were equipped with future versions of these technologies. What if you could enter into a controllably immersive experience of Titan so that you seemed to walk under its orange skies and explore its methane/ethane seas? Couple this with serious advances in bandwidth through laser communications and this experience could be made utterly lifelike. Is this kind of public component going to be a part of future deep space missions, to engage interest and generate support for exploration? If so, we may see interest in expanding human missions to do the things only human crews can do. This is a fantastic hybrid of cutting-edge gaming technology and state of the art data acquisition.

On a broader level, augmented reality has tremendous implications. Consider just this: Microsoft HoloStudio will allow modeling with HoloLens data. They’re calling it a “print preview” for 3D printing, where engineers can walk through a design from all angles before actually creating the product. Exactly where all this goes is as exciting a question as asking in 1980 what we would do with desktop computers. Augmented reality in one form or another is going to be huge.

Paul A. Gilster is the author of several books on technology. Reach him at gilster@mindspring.com.

This story was originally published January 31, 2015 at 8:00 PM with the headline "Paul Gilster: Microsoft's HoloLens shows potential of augmented reality."