Archive for the ‘augmented reality’ Category

join me at the eComm firehose next week?

Wednesday, April 14th, 2010

Next week I’ll be at the Emerging Communications conference, eComm 2010 in San Francisco, and would love to connect if you are going. The organizers have pulled together an impressive roster of speakers in a rapid-fire single track format that will undoubtedly feel like a three day firehose of ideas. Of course I’m very interested in the AR-heavy Wednesday lineup, but am also looking forward to seeing folks like John Hagel, Ge Wang and Debbie Estrin. Ping me if you’re there, and keep an eye out for digital film geek @endurablegoods who will no doubt be wielding weapons of mass digitization.

If you’re in town, you may want to extend your 3-day fun pass by going to Reality Checked – What’s Next for Mobile Augmented Reality on Monday night, and the wonderful Dorkbot-SF on Wednesday night. I’m just sayin’.

augmented humanity + enspirited reality: AR panel at NAB2010

Wednesday, April 14th, 2010

Earlier this week I spoke on the Augmented Reality: Entertainment Meets Ubiquitous Computing panel at the National Association of Broadcasters conference.  In my intro talk I stated that AR should be thought of as the intersection of two separate trends: the augmentation of human capabilities through technology, and the digitization of the physical world of people, places and things. I named these trends ‘augmented humanity’ and ‘enspirited reality’. My slides for the panel (cc-by-nc):

My fellow panelists (photo) came from diverse perspectives and included Joe Garlington from Disney Imagineering, Bruno Uzzan from Total Immersion, Chetan Damani of Acrossair, and Rebecca Allen, director of Nokia’s Hollywood research lab; the estimable Seth Shapiro moderated the discussion. Garlington showed video clips from several of Disney’s mixed reality theme park projects, Bruno wowed the crowd with his familiar K’NEX demo, Chetan showed off one of the acrossair apps live on iPhone, and Rebecca Allen screened an extended clip from NRC’s Westwood Experience mobile storytelling project. Overall I think the discussion was well-received, even if the topic was a bit fast-forward for the largely broadcast-focused attendees.

will the HP Slate be a killer AR device?

Monday, April 5th, 2010
The HP Slate in live video mode

The HP Slate in live video mode

Augmented reality enthusiasts and developers got the shaft (again) from Apple when the iPad launched without an integrated camera, thus becoming a dead platform for AR purposes. Well it looks like the little  computer company on the other side of Hwy 280 might pull a little auggie magic out of their hat, just in time for the AR summer of love in Silicon Valley. HP has been teasing their forthcoming Slate for a few months, and they just posted another video clip that clearly shows live video from a forward-facing camera. We already know the slate will run Windows 7, and we have heard public rumblings about Android from various quarters, so it’s likely to be reasonably developer-friendly.

With the horsepower to run object recognition and tracking plus high quality 3D graphics, the Slate will definitely blur the line between webcam AR and mobile AR experiences. You know all those marker-based AR toys that feel so gimmicky when you have to use them in front of a PC with a webcam? I guarantee they are going to seem 1000% cooler when you pull out your Magic Internet Magnifying Glass and look through it into an alternate universe. And if the Slate ends up shipping with a GPS and digital compass, just watch all the mobile AR guys scrambling to learn Win7 and Silverlight. Oh yeah SLAR toolkit dude, better get a bigger server ;-)

augmentation overload

Friday, February 5th, 2010

Watching Keiichi Matsuda’s superb short video imagining a hyper-branded augmented reality environment, for some reason I was reminded of this classic bit of dystopia from PK Dick’s 1969 classic novel Ubik.

The door refused to open. It said, “Five cents, please.”

He searched his pockets. No more coins; nothing. “I’ll pay you tomorrow,” he told the door. Again he tried the knob. Again it remained locked tight. “What I pay you,” he informed it, “is in the nature of a gratuity; I don’t have to pay you.”

“I think otherwise,” the door said. “Look in the purchase contract you signed when you bought this conapt.”

In his desk drawer he found the contract; since signing it he had found it necessary to refer to the document many times. Sure enough; payment to his door for opening and shutting constituted a mandatory fee. Not a tip.

“You discover I’m right,” the door said. It sounded smug.

From the drawer beside the sink Joe Chip got a stainless steel knife; with it he began systematically to unscrew the bolt assembly of his apt’s money-gulping door.

“I’ll sue you,” the door said as the first screw fell out.

Joe Chip said, “I’ve never been sued by a door. But I guess I can live through it.”

Ubik, Philip K. Dick 1969

experience design for locative media & AR

Thursday, December 3rd, 2009

The next generation of mobile Augmented Reality applications will go well beyond simply overlaying points of interest, floating post-its and 3D models on the video display of your phone. Mobile AR is becoming a sophisticated medium for immersive games, situated storytelling, large-scale visualization, and artistic expression. The combination of physical presence, visual and audio media, sensor datastreams and social environments blended together with web services offers tremendous new creative possibilities. However, the design challenges of creating engaging, exciting and compelling experiences are quite significant. AR experience designers will draw fruitful inspiration and practical lessons from game design, 3D graphics, architecture and stagecraft, as well as the structure, linking, protocols and openness of the web.

Some of the best research to date on experience design for locative media experiences, was done at HP Labs as part of the Mobile Bristol collaboration. You might find these papers useful and applicable to AR design, as I have.

Technology Experiences: What Makes Them Compelling? Alison Kidd, 2000

Experience Design for Pervasive Computing, Richard Hull & Jo Reid

Experience Design Guidelines for Creating Situated Mediascapes, Reid et al, 2005

Magic Moments in Situated Mediascapes, Reid et al

we’re wired to augment our world and our selves

Tuesday, December 1st, 2009

Humans are driven to augment our reality, and to augment our own capabilities. It’s the way we’re wired; it’s what we do. From the earliest cave paintings to modern day urban graffiti, we overlay our world with expressions of our inner selves. Architecture, street signs, billboards, fashions — these are all visual and functional augmentations of the physical world. We extend our sensory and cognitive capabilities as well, creating tools that allow us to perceive phenomena beyond the ken of our normal senses — at extremes of scale, distance, time, frequency, complexity, sociality. “Augmented Reality” is simply the next technological framework for expressing the natural human propensity to augment our world and our selves.

fiyo on the bayou: junaio from metaio

Friday, October 30th, 2009

In the last week I’ve been beta testing junaio, Metaio’s new mobile augmented reality app for the iphone 3gs. junaio allows you to author mixed reality scenes by adding 3D models to physical world locations, experience AR scenes on location in live video overlay mode, and share your scenes with other people on junaio and Facebook. My thoughts here are based on a pre-release version that does not include broader functionality such as online scene editing through the website.

My overall impression is that junaio is a fun, interesting app that is very different from other mobile GPS+compass AR apps. Popular mobile AR apps like Layar, Wikitude, Robotvision, GeoVector and so on are mostly information centric, focused on finding and presenting geoweb and POI data about the world. In contrast, junaio is about personal augmentation of the world with visual models — it’s essentially a storytelling environment where users can express themselves through 3D world-building and share their creations with a social community.

A slight disturbance on the stanford quad

A slight disturbance on the stanford quad

I found the creative authoring process surprisingly absorbing and satisfying, much more so than experiencing the scenes through the live “magic lens” for example. I was also impressed at how much could be done with a few finger gestures on a tiny device out in the world. Here’s an example scene I created while driving up Alpine Road last night (definitely not a recommended authoring process!):

kids, don't try this at home

kids, don't try this at home

Although junaio is a unique and engaging application with great ambition, I will warn you that it suffers somewhat from its high aspirations. junaio proposes to make 3D environment authors of us all. It does a reasonable job of simplifying a complex process and making it possible on a mobile device, but as a result it takes shortcuts that reduce the effectiveness of the end experience. For example the live camera overlay mode was a disappointment, because you do not experience the 3D scene that the author intended. I understand the technical reasons for this — limited GPS accuracy, lack of true 3D placement of objects, lack of camera pose data, physical world changes, etc — but my expectations were implicitly set that my scenes would turn out exactly the way I created them. Also, the user interaction model for the app is still rough in places, and I think many people will find it confusing and difficult to learn.

Despite the significant limitations of this first release, I actually think I will continue to use junaio. I really enjoy its creative aspects, and I think there is a lot of potential in the social community interaction as well. It’s not going to be everyone’s cup of tea, but I see it as a lovely way to live a few extra seconds into the future. Looking forward to the final release in the next few days.

In case you’re wondering, “Fiyo on the Bayou” is a song by the great Neville Brothers. Got a bit of New Orleans on the brain today, or maybe I’m hoping for a nice model of animated flames that I can place out on the Bay. Okay, mostly I just like the way it rhymes with “junaio from metaio”. YMMV as always. Cheers all.

where’s the auggie? six methods for specifying location in mobile AR

Friday, October 23rd, 2009

As a follow up to my post on location standards for mobile augmented reality, I’ve been thinking about how an AR author would specify the location of virtual objects in the physical world. In my research I came up with six different ways of specifying location, and identified some interesting challenges for mobile AR design. This is still at the “thinking out loud” stage, so help me out if you don’t like the way I came in.

(Just to be clear, this discussion is focused on the representation of location, and doesn’t speak to interaction design of the authoring process. I plan to address that topic in a future writeup.)

A location specification can be direct or indirect. Direct location is specified in measurable quantitative terms such as spatial coordinates. Indirect location is described in terms of a relationship relative to some other entity, or as a semantic identifier of a spatial or geographic entity. Each has its charms and disillusionments.

Direct Location

1. Geospatial coordinates — The most common model for virtual object location in mobile AR is to specify a point as (longitude, latitude, altitude) in an Earth-based coordinate system. This point becomes the origin of a local coordinate system for the virtual object. This model is relatively well understood and in wide usage by most of the mobile AR systems, location based services, GPS points of interest (POIs), geoweb services etc.

The specific challenges for mobile AR include standardizing on data representations, supporting multiple coordinate reference systems (CRS), defining precision and accuracy, and supporting geometries beyond simple points. Hopefully, ongoing discussions around ARML, GeoAR and others will lead to reasonable convergence among the interested communities.

2. Alternative spatial coordinates — It’s not too hard to imagine cases where you want something other than a ground-based spatial coordinate system. For example, what if my object is a giant billboard in (non-geosynchronous) orbit around the Earth? A geodetic system like WGS-84 does you little good in this case, so you might want to use a geocentric model. The X3D architecture supports geocentric coordinates, for example. Better yet, what if my object is a virtual planet in orbit around the Sun? Earth coordinates will be quite unhelpful for this case, and I’m not aware of any systems that have heliocentric CRS support. Yet another interesting scenario involves indoor positioning systems which establish their own CRS on a local basis.

Challenges here include identifying alternative reference systems that should be supported for AR use cases beyond ground-based scenarios, and specifying the transformations between these frames of reference and other involved CRSes.

Indirect Location

3. Spatial entity identifiers — The names of geographic places — Heathrow Airport, New York City, Fujiyama, the Transamerica Building, Beverly Hills 90210  — are indirect specifications of location. So are unique identifiers such as Yahoo GeoPlanet’s WOEIDs. They are indirect because the physical coordinates they represent, the centroids and bounding boxes of their ground shapes, must be looked up in some reference database.

For AR, the opportunity is obviously to embrace the human-centric context of place names, and to leverage the large investment in geoweb services by Yahoo and others. There are many AR use cases where it would be desirable to augment an entire geographic place at once. The challenge is to define a representation for virtual objects that supports identifiers as locations, and provides for appropriate services to resolve IDs to geographic coordinates. Of course for these identifiers to have meaning, the model also needs to support geometries beyond simple points.

4. Client-relative location — Location relative to the client device, which presumably knows its geographic location, as a proxy for the location of the human user. This is the augmented pet scenario, maybe. Faithful digiRover the auggie doggie’s location is specified as a vector offset from my location, and follows me around as I move.

5. Viewport-relative location — In computer graphics, the viewport is basically defined by the size and resolution of the viewing display. An AR application might wish to locate a virtual object model at a specific point on the user’s display, regardless of where the “magic lens” is pointed. For example, I might “click” the QR code on a physical object and a related virtual 3D object model appears pinned to the center of my display, where I can play with it without having to hold my device aimed steadily in one place. The object’s location is specified as (x%, y%) of my device’s screen size. If you like, we could have a good discussion in the comments about whether this is a valid “location” or not.

6. Object-relative location — An important class of use cases that are easy to describe in language, and more difficult to represent in code. “Alice’s avatar is on a boat”. “Bob’s tweets float above his head as he walks by”. “Charlie is wearing a virtual hat”. “The virtual spider is on the virtual table, which is on the physical floor”.  In each case, a virtual object’s location is specified relative to another virtual or physical object. The location of the second object may also be relative to something else, so we need to be able to follow the nested chain of relative locations all the way down to ground-based coordinates (or fail to locate). Of course, the second object might also be an AR marker, an RFID, a 1D or 2D barcode or similar physical hyperlink. It might be an object that is identified and tracked by an image-based recognition system. It might further be a physical object that has embedded location capability and publishes its location through an API, as Alice’s boat might do.

Clearly object-relative location poses a host of challenges for mobile AR. Among these: defining an object model for virtual and physical objects that includes representations for identification and location; defining an extensible object identification and naming scheme; and defining a location scheme that allows for a variety of methods for resolving location specifiers, including nested constructs.

As I said, this is definitely “thinking out loud”, and I’d love to have your feedback in the comments below or via @genebecker. I want to acknowledge @tishshute and Thomas Wrobel, who have been leading the discussion on using Google Wave protocols as a communication & distribution mechanism for an open AR network. That collaboration stimulated these ideas, and you might find it stimulating to participate in the conversation, which appropriately is happening as a Wave. Let me know if you need a Wave invite, I have a few left for serious participants.

As always, YMMV. Peace.

iPhone augmented reality apps: how’s business?

Wednesday, September 30th, 2009

I was curious about how the various mobile AR, GPS+compass apps are faring, so I did a quick and dirty channel check in the App Store and compiled the following table. I think it’s pretty interesting data. A few caveats: this is a snapshot in time, ~7.30am PT on 30 Sept 2009, it’s for the US App Store only, it’s iPhone only, and the apps are listed in no particular order. If anyone wants to do a similar roundup for Android AR apps, that would be outstanding. Other countries (I’m waving at you, air-taggy Japan) welcome too, just drop your tidbits or links in the comments section.

Update 9/30: added direct links to the iTunes App Store

Application Company Category Ranking Price Rating # of Ratings
Fairy Trails Freeverse Games NR $1.99 5 stars 2 ratings
Robotvision Tim Sears Utilities #32 paid app $0.99 4 stars 48 ratings
World Surfer GeoVector Navigation #60 paid app $2.99 4.5 stars 12 ratings
PeakAR Salzburg Research Travel NR free 5 stars 1 ratings
Yelp (Monocle) Yelp Travel #4 free app free 3 stars 1183 ratings
Bionic Eye Presselite Navigation #7 paid app $0.99 1.5 stars 98 ratings
Bionic Eye France Presselite Navigation NR $0.99 0 stars 0 ratings
Bionic Eye Tokyo Presselite Navigation NR $0.99 0 stars 0 ratings
Bionic Eye UK Presselite Navigation NR $0.99 0 stars 0 ratings
Washington Metro Presselite Navigation #51 paid app $0.99 3 stars 20 ratings
Chicago L Rapid Transit Presselite Navigation #94 paid app $0.99 2.5 stars 15 ratings
Tokyo Underground Presselite Travel NR $0.99 3 stars 8 ratings
Metro Paris Subway Presselite Travel #21 paid app $0.99 4 stars 176 ratings
London Bus Presselite Travel NR $0.99 2 stars 8 ratings
Nearest Places Acrossair Reference #7 paid app $1.99 3.5 stars 28 ratings
Nearest Wiki Acrossair Education #29 paid app $1.99 3.5 stars 7 ratings
New York Nearest Subway Acrossair Travel #8 paid app $1.99 4.5 stars 6 ratings
PhotosAR Acrossair Photography #64 paid app $1.99 4 stars 3 ratings
San Francisco Nearest Transit Acrossair Travel #53 paid app $1.99 stars 0 ratings
Chicago Nearest Transit Acrossair Travel #52 paid app $1.99 5 stars 1 rating
Nearest Tube Acrossair Travel NR $1.99 0 stars 0 ratings
Tokyo Nearest Subway Acrossair Travel NR $1.99 0 stars 0 ratings
Paris Nearest Metro Acrossair Travel NR $1.99 0 stars 0 ratings
Barcelona Nearest Metro Acrossair Travel #7 paid app $1.99 0 stars 0 ratings
Madrid Nearest Metro Acrossair Travel #7 paid app $1.99 0 stars 0 ratings

you know it’s the future when there’s a futurist floating in your magic window

Monday, September 21st, 2009


That’s @askpang‘s photo, apparently. How perfectly appropriate, on so many levels. Well Alex, it looks like the end has arrived.

Bonus points for naming that app.

Update 9/24/09: OK yes it’s @u2elan‘s Robotvision, which dropped in the app store today. But it’s version 1.1, with secret Wikipedia goodness, w00tski!