This is nokia

augmented humanity + enspirited reality: AR panel at NAB2010

Wednesday, April 14th, 2010

Earlier this week I spoke on the Augmented Reality: Entertainment Meets Ubiquitous Computing panel at the National Association of Broadcasters conference.  In my intro talk I stated that AR should be thought of as the intersection of two separate trends: the augmentation of human capabilities through technology, and the digitization of the physical world of people, places and things. I named these trends ‘augmented humanity’ and ‘enspirited reality’. My slides for the panel (cc-by-nc):

My fellow panelists (photo) came from diverse perspectives and included Joe Garlington from Disney Imagineering, Bruno Uzzan from Total Immersion, Chetan Damani of Acrossair, and Rebecca Allen, director of Nokia’s Hollywood research lab; the estimable Seth Shapiro moderated the discussion. Garlington showed video clips from several of Disney’s mixed reality theme park projects, Bruno wowed the crowd with his familiar K’NEX demo, Chetan showed off one of the acrossair apps live on iPhone, and Rebecca Allen screened an extended clip from NRC’s Westwood Experience mobile storytelling project. Overall I think the discussion was well-received, even if the topic was a bit fast-forward for the largely broadcast-focused attendees.

augmenting Nokia’s reality: quick thoughts on Point & Find

Tuesday, August 25th, 2009

From the “Who asked you anyway?” department…

I hadn’t really looked at Nokia’s Point & Find concept before, even though it has been shown around at least since Ubicomp 2007 and the public beta has been running since May09. After last night’s Nokia AR talk I thought I would have a peek.

point-find

OK, it’s a beta with limited functionality, and undoubtedly there are plenty of enhancements to the system and the business plan going on behind closed doors, so take this with a large scoop of salt because I only know what’s been said in public. But if I was advising Nokia on Point & Find, I would start with this:

1. The business opportunity is in the service for building and managing the image database, and managing all of the associated meta-behaviors like hyperlinking, bookmarking, purchasing, search, customer intent discovery, collaborative recommendations and so on. By building this physical-to-digital gateway between individual users and companies and institutions, Nokia can accelerate the development of the connected world platform and profit nicely from it as well.

2. The business opportunity for this is NOT in the handset. That should actually be great news for Nokia, because their core hardware business is highly commoditized and that’s a cold reality their employees have to live every day. In the long run, selling commodity hardware is not going to be the best strategy for Nokia (yes, that includes stuff like windows-based netbooks).

3. Point & Find should go on the iPhone and Android platforms, as soon as possible. Because the opportunity is in growing profitable services, not in fighting the low margin handset wars. Because the current AR hype is a swell marketing tool, and right now mobile AR is all about iPhones and gPhones. Because if Nokia doesn’t own image-based physical hyperlinking on those phones, someone else will, and that means the strategic momentum will shift. I realize this is totally countercultural, maybe even unthinkable, but that’s why it would be such an interesting play.

4. Nokia should open up an API to its backend service, so that any developer can make services that tap into that (hypothetical?) database of millions of images of products, printed ads, books, artworks and on and on. Developer momentum advances the entire market, and solidifes the platform’s competitive position. This can be one of those big web squared opportunities, if played right.

How about you, have you tried Point & Find? If you were Nokia, what would you do?

Nokia’s approach to mobile augmented reality

Tuesday, August 25th, 2009

We had good AR-related fun at last night’s talk by Kari Pulli and Radek Grzeszczuk from Nokia Research, “Nokia Augmented Reality” hosted by SDForum. It was basically a survey of AR-related work done at Nokia in the last few years, with special emphasis on their research work in image-based recognition.

Kari presented an overview of several research projects, including:

MARA (Mobile Augmented Reality Applications) — A GPS+compass style AR prototype, also using accelerometers as part of the user interaction model. See also this Technology Review article from Nov06.

Image Space — “A user-created mirror world”, or less romantically, a social photo capture & sharing system, using GPS+compass to locate and orient photos you take and upload, and allowing you to browse others’ photos taken nearby.

Landmark-Based Pedestrian Navigation from Collections of Geotagged Photos — A bit hard to describe, best to have a scan of the research paper (pdf).

Point & Find — Mobile service that uses image-based recognition to tag and identify physical objects such as products, movie posters and buildings and provide links to relevant content and services. This is being incubated as a commercial service and is currently in public beta.

nokia-ar

Radek did a technical dive into their approach to image-based recognition, touching on a variety of image processing techniques and algorithms for efficiently extracting the salient geometric features from a photograph, and identifying exact matching images from a database of millions of images. The algorithms were sufficiently lightweight to run well on a smartphone-class processor, although matching against large image collections obviously requires a client-server partitioning. This work seems to be an important part of NRC’s approach to mobile AR, and Radek noted that their current research includes extending the approach to 3D geometry as well as extracting features from streaming images. Capturing a comprehensive database of images of items and structures in the world is one barrier they face, and they are considering ways to use existing collections like Google’s Street View as well as urban 3D geometry datasets such as being created by Earthmine. Another area for further work is matching images that do not contain strong, consistent geometric features; the algorithms described here are not useful for faces or trees, for example.

Update: DJ Cline was there and has pictures of the event, the slides and the demo.

Related links:
SDForum Virtual Worlds SIG
SDForum Emerging Technologies SIG
Gathering of the AR tribes: ISMAR09 in Orlando