Open AR: what's the point?

Like many other folks involved in augmented reality, I’d like to see the mobile AR community embrace open standards for AR experiences. And just to be clear, by “embrace” I mean “create and implement”. Now, I know this discussion is eventually going to take us into deep waters, but let’s just start off with the simplest possible thing. I’d like to see the mobile AR community agree on how it represents a point in space. If we could do that, we might be able to create some simple, public AR experiences that work across platforms and in the various competing AR browsers. And the positive example of one agreed open standard, arrived at by an open community process, might lead to additional good things. So let’s talk about points.

Geographic AR Points

Geographic AR systems like Layar, Geovector, Wikitude, Robotvision, Gamaray etc, use a spheroid-based coordinate system of latitude, longitude and (sometimes) altitude to specify the point locations of the observer and georeferenced content. POIs (points of interest) consisting of a single (lat,lon,alt) coordinate tuple plus various metadata, are commonly used to represent physical entities such as restaurants, monuments and attractions. Unfortunately even in this extremely simple case, there is no agreement on specifications for a single point in space. For example, if altitude is used, is it the height of the point above the topographic surface at that location, the height above the observer’s location, or the height above the WGS-84 reference ellipsoid approximating mean sea level, as a GPS would measure it? Does a point also have accuracy metrics? And what metadata are required or optional for each point?  Each of the companies mentioned above is doing something a bit different, and so are their upstream POI data providers. So far, and despite recent announcements, openness is not really happening yet.

3D AR Points

AR has its roots in computer graphics & vision technologies, and these approaches primarily use 3D cartesian (xyz) coordinate systems. A 3D model of a teapot might have a local xyz coordinate system; the teapot rests on a 3D model of a table which in turn has its own reference coordinate system; the observer of the scene has their own reference coordinate system; the screen that the scene is displayed on has its own 2D pixel coordinates, and a set of mathematical transformations (e.g., translation, scaling, rotation & projection) ties them all together. A 3D graphics scene is not inherently tied to any physical world reference point; in marker-based AR, the fiducial marker provides an anchor that binds the 3D augmented scene to a physical world location. However, the data structure for the scene’s location is entirely relative, which makes the location of 3D models fairly portable.

Simple Geo + 3D AR

Of course, one simple and obvious thing we want is to enable 3D graphics models to be placed in geographic locations. If we truly think open AR is important, we are going to want to agree on which kinds of coordinate systems to use. This is not a trivial question. Do we want the 3D model to be on a local or global coordinate system? A fixed position relative to the world and regardless of viewpoint, or always located relative to the observer? What if the model and the observer are on boats? What if the model is something like an entire city? Different choices for coordinate systems and schema will impact computational costs and accuracy. In Google Earth, KML allows use of static COLLADA models which are then imported/transformed to the GE geographic coodinate system. Planet9’s virtual cities have a single reference coordinate system for the entire city, and use UTM WGS-84 in order to keep their building models square. The Web3D Consortium’s X3D framework supports georeferencing models in geodetic, UTM and geocentric reference frames, appropriate for a variety of use cases. What approach(es) makes sense for mobile AR? Can we leverage & extend existing standards, or will we have to create new ones from the ground up?

Start simple, but start now

Okay, so clearly things can get messy, even for the simple case of specifying a point in space. And it is also clear that multiple constituencies are going to be very interested in the geographic and 3D graphic aspects of AR. I think it’s time to have serious discussions about open standards for mobile AR, starting with the basic question of representing POIs and static 3D objects. I realize it is hard for small, fast moving teams to spend precious energy on this kind of discussion, but to me it seems like a critical thing for the community to establish a common foundation for the mobile AR experience. Do you agree? If not, why not? If so, then where should this discussion happen and who should be involved? Perhaps the recently formed AR Consortium can play a role here? Maybe it is already happening somewhere?

I’m very interested in your thoughts on this topic. Please share in the comments below, link here from your own blog, or respond @genebecker. YMMV as always.

For further reading

* Augmented Reality Should Be Open by Joe Ludwig
* Augmented Reality: Open, Closed, Walled or What? by Robert Rice
* Wikitude API
* Layar API
* Gamaray formats
* Garmin GPX POI schema
* WGS-84
* A Discussion of Various Measures of Altitude
* GeoRSS
* W3C Geolocation API
* X3D
* CityGML

Tags: , , , ,


5 Responses to “Open AR: what's the point?”

  1. Tim Sears says:

    Great post, and you’re dead-on about how Geographic AR points are being displayed.

    This was something I took into quite a bit of consideration when creating Robotvision, in that I wanted to provide an element of depth to represent distance without decreasing the size of the points to keep it easy to read.

    The problem with my approach is that where the points are positioned on the y-axis is not directly proportional to the distance. Distance is a factor, but if you want to look across a broad radius, your viewport might wind up very large for the y-axis (looking up and down).

    It’s going to be interesting to see what user feedback is like regarding my approach and others as more browser hit the market. Perhaps it will be the users who dictate what usability and presentation conventions we adopt and standardize on for mobile AR experiences.

  2. Joe Ludwig says:

    This is certainly an area that needs some standardization. Pretty much everyone is using lat/lon, but even for those two, they aren’t all using exactly the same notion. Layar, for instance, requires that you pass these coordinates back after multiplying them by 10^6. Some use positive and negative numbers to indicate hemisphere, while others use NSEW.

    Altitude is in a similar state with respect to precision, and has the added complication of specifying the origin. I believe KML lets you specify that an altitude is relative to sea level (WGS-84, presumably), “ground level”, or absolute.

    At least nobody is using feet and inches for anything. :)

  3. Gene says:

    Tim, I’m looking forward to seeing Robotvision! Now that OS3.1 is here, it should be any day now, right?

    Joe, since you argued for open AR as analogous to the web’s openness, I’ll ask you what I’ve been asking other folks: can/should an AR system be built using RESTful design principles? See for example.

  4. Great post, and while AR is only an area that I have recently gotten into I think the proposition of open standards is an excellent one. The Augmented Reality space is very interesting right now, especially around the particular topic of GeoAR. It’s getting a lof of press and entrepreneurial interest (which I can understand), however, these two external influences are probably going to shift towards proprietary standards rather than open.

    I would definitely expect that whoever ends up with a dominant market share in the GeoAR browser space will probably end up with a defacto standard, that will then be defined as an open standard through a process of arbitration. Generally, that standard is not the most technically elegant, but will do the job for current and probably next generation of GeoAR applications.

    In parallel, the technically elegant revisions required to take the technology forward will be applied over a period of time to the standards and all willing parties will jump board (a few will go rogue, they always do).

    Still, I agree, the sooner intelligent people such as yourself start discussing these standard with others (and that includes knocking on the doors of companies such as Layar and Moblizy) the better.

    Thanks again for a great post.

  5. […] a follow up to my post on location standards for mobile augmented reality, I’ve been thinking about how an AR author would specify the location of virtual objects in […]