Archive for the ‘augmented reality’ Category

I’m joining Layar

Monday, November 29th, 2010

layar-logoI have a bit of news: I’m joining Layar, the Dutch mobile AR company, as an Augmented Reality Strategist. \0/ In this new role I’ll be developing the creative ecosystem for the Layar platform, working with artists, developers, agencies, brands and media geeks to push the boundaries of AR experience design. As the first US-based team member, I’ll also be helping establish Layar’s Bay Area presence and growing the North American community of mobile AR enthusiasts. There’s more at the Layar blog.

Signing on with the Layar team feels like a natural evolution to me. I’ve worked at the intersection of digital media and physical reality for more than a decade, and mobile AR is one of the most important world-changing developments in that field. I believe that today’s mobile AR is the leading edge of an emerging new medium of expression and communication, a vision the Layar team shares. The chance to help shape something this important, working with a team of this caliber…well let’s just say it’s going to be fun ;-)

As for Lightning Laboratories, I’ll still be blogging here and sending out the occasional Connected World newsletter. However, the consulting part of the business will be offline for the foreseeable future. My social media practice will move over to TrendJammer, the new social consulting & analytics venture that my friend Risto Haukioja has launched, and for which I’m an advisor. If you’re interested in that sort of thing, there’s lots of fun going on at and @trendjamr on Twitter.

If you want to get in touch, the usual channels still apply: @genebecker, @ubistudio, gene at lightninglaboratories dot com, etc. For super-double-secret Layar business, I’m now gene at layar dot com. Now let’s get out there, people — we’ve got a new medium to build!

experiments in historical augmented reality

Sunday, November 14th, 2010

In collaboration with Adriano @Farano, I’ve been experimenting with creating historical experiences in augmented reality. Adriano’s on a Knight Fellowship at Stanford, and he’s seeking to push the boundaries of journalism using AR; my focus is developing new approaches to experience design for blended physical/digital storytelling, so our interests turn out to be nicely complementary. This is also perfectly aligned with the goals of @ubistudio, to explore ubiquitous media and the world-as-platform through hands-on learning and doing.

Adriano’s post about our first playtesting session, Rapid prototyping in Stanford’s Main Quad, included this image:

Arches on the Quad 1905

Taken from the interior of the Quad looking toward the Oval and Palm Drive, you can see that the photo aligns reasonably well with the real scene. Notably, the 1905 picture reveals a large arch in the background, which no longer stands today. We later found out this was Memorial Arch, which was severely damaged in the great 1906 earthquake and subsequently demolished.

In our second playtesting session, we continued to experiment with historical images of the Quad using Layar, Hoppala and my iPhone 3Gs as our testbed. Photos were courtesy of the Stanford Archives. This view is from the front entrance to the Quad near the Oval, looking back toward the Quad. Here you can see the aforementioned Memorial Arch in 1906, now showing heavy damage from the earthquake. The short square structure on the right in the present day view is actually the right base of the arch, now capped with Stanford’s signature red tile roof.

Memorial Arch after the 1906 earthquake

In this screencap, Arches on the Quad 1905 is showing as the currently selected POI, even though the photo is part of a different POI.

One of the more famous images from post-earthquake Stanford is this one, the statue of Louis Agassiz embedded in a walkway:

Statue of Louis Agassiz 1906

Although the image is scaled a bit too large to see the background well, you can make out that we are in front of Jordan Hall; the white statue mounted above the archway on the left is in fact the same one that is shown in the 1906 photo, nearly undamaged and restored to its original perch.

Finally we have this pairing of Memorial Church in 2010 with its 1906 image. In the photo, you can see the huge bell tower that once crowned Mem Chu; this was also later destroyed in the earthquake.

Memorial Church 1906

Each of these images conveys some idea of the potential we see in using AR to tell engaging stories about the world. The similarities and differences seen over the distance of a century are striking, and begin to approach what Reid et al defined as “magic moments” of connection between the virtual and the real [Magic moments in situated mediascapes, pdf]. However, there are many problematic aspects of today’s mobile AR experience that impose significant barriers to reaching those compelling moments. And so, the experiments continue…

augmented reality developers camp 2010

Wednesday, November 10th, 2010

ARDevCampThe Bay Area’s 2nd annual (?) Augmented Reality Developers Camp is officially on for Saturday Dec 4th 2010. This year’s event will be held in downtown San Francisco at GAFFTA, the Gray Area Foundation For The Arts. ARDevCamp is an open unconference organized by and for the local community of AR developers, artists and enthusiasts, with the participation and support of leading AR companies including Layar, Metaio, Qualcomm and FXPAL. I’m helping organize again this year, along with @chris23, @metaverseone, @anselm & @mikeliebhold. If you’re in the Bay Area and into augmented reality, you need to be there! More info on the wiki at

je mixe ce soir!

Wednesday, November 3rd, 2010

In honor of Facebook’s announcements today about making mobile more social, I’d like to remind you of this visionary portrayal of what it will be like when Facebook is truly mobile. Looks like we’ve got a long way to go.


That’s right AR fans, it’s the Toxic Avenger feat. Orelsan performing last summer’s monster Internet dance hit, N’Importe Comment. So slip on your mindglasses, turn up the bass in your earplants, and prepare to “Like” this french fratboy fantasy from the future. Watch carefully, because this is a precious, fleeting snapshot of the way our connected culture felt, circa mid-2010. Someday, cyborg anthropologists are going to have a field day with this thing. Je mixe ce soir!

augmented reality 4 poets

Thursday, October 21st, 2010

Earlier this month I attended THATCamp Bay Area, a 2-day head-on collision of scholars and practitioners in the humanities with a range of folks from the tech world. It was quite refreshing and challenging to (attempt to) wrap my mind around linguistics, environmental history, experimental poetics and art curation, just to name a few of the disciplines that were represented. Interestingly, I also discovered unexpected hidden connections that led back to the EVOKE Summit and forward to @ubistudio; more about these later perhaps.

My contribution to the fray was a session named “Augmented Reality 4 Poets”, a hands-on workshop on creating basic mobile AR using the Layar platform and Hoppala CMS service, no programming required. It worked out pretty well, and I wanted to share the materials here. I’ll likely reprise some of this in a session at ARDevCamp in December, and possibly at other future events. Anyway, here’s the tutorial. I’ve kept it simple on purpose — both Layar and Hoppala have additional capabilities you should take the time to explore. Also, you’ll see that for THATCamp I made the shared @ubistudio accounts available, but if you want to go through this on your own, you will need to sign up for a Layar developer account and a Hoppala login (it’s easy).

Mobile Augmented Reality for Non-Programmers
A Simple Tutorial for Layar and Hoppala

1. What you need to create your first mobile AR layer:

* A smartphone that supports the Layar AR browser. This means an iPhone 3GS or 4, or an equivalent Android device that has built-in GPS and compass.
* The Layar app, downloaded onto your device from the appropriate app store.
* A computer with web access.
* A developer account with Layar and a login at Hoppala. For this tutorial, you will use our shared ubistudio account. Later, you can request your own at

2. Get connected:

The ubistudio credentials we will be using today are: [redacted]

You should use these credentials to sign in at 3 places:

* The Hoppala website:
* The Layar developer website:
* The Layar app on your device

Because these are shared credentials, you will see other people’s layers in these environments [only true for the shared tutorial account]. PLEASE DON’T TOUCH ;-) There is no undo or undelete!

3. Get started:

Log into Hoppala. You should see the Dashboard, a simple list of layers with Titles, Names and POI URLs.

Hoppala dashboard

At the bottom right of the page, click “Add layer service” to create a new layer. A new line will be added to the list, with “Untitled” and “noname”. On the far right of that line, click the pencil icon and give your layer a new title and name. The name needs to be lowercase alphanumeric. Click the Save button.

Next, click on the name of your new layer. You should see a Google Map. Navigate to our location and zoom in.

Hoppala map view

To add a point of interest (POI), click “Add augment” at bottom right of the page. This will add a basic POI in the center of the map. You can drag it to the location you want.

To customize your new POI, click on it and a popup will launch. The popup has 5 tabs, and we’ll mostly care about the first 3. Each tab is a form we will use to enter data about the POI.

Hoppala POI menu (click for larger view)


* Title and description fields can be whatever you want. Footnote is not editable
* Image is the picture that is displayed for the POI’s information panel in the mobile app view. You can use one of the images already loaded, or you can upload your own from your computer.
* POI Icons are what show up in the AR view for basic POIs. Choose ‘default’, and select a Custom Icon from the drop down list. You can also upload your own.
* BE SURE TO CLICK THE SAVE BUTTON and wait for the confirmation.


* For basic POIs, don’t worry about this.
* For images or 3D models, select the appropriate Type.
* Use the pulldown menus to select a preloaded image or model. You can also upload your own. 3D models need to be in a custom Layar l3d format.
* BE SURE TO CLICK THE SAVE BUTTON and wait for the confirmation.


* In the Layar browser, you can have actions triggered from POIs. These can include going to a website, playing an audio or video, sending an email or text, and making a call.
* If you make changes, BE SURE TO CLICK THE SAVE BUTTON and wait for the confirmation.

You can add more POIs, or move on to testing the layer.

4. Testing your layer:

Log into the Layar developer site. You will see a table of existing layers.

Layar Developer Site (click for larger view)

To add your new layer, click the “Create a layer” button. You will see a popup form.

* Layer name must be exactly the same as the name you chose in Hoppala.
* Title can be a friendly name of your choosing.
* Layer type should be whichever type you have made.
* API endpoint URL is the URL for your layer copied from the Hoppala dashboard (the long ugly one).
* Short description is just some text.

Click Create layer and you should be done!

(There are more editing options, but you can safely ignore them for now).

Start up the Layar app on your mobile device. Be sure you are logged in to the developer account , or you will not see your unpublished test layer. Select “YOURS”, and then “TEST”. You should see several test layers, including your own [different versions of the Layar app may put the TEST listing in different places, so you may need to poke around a bit]. Select your layer and LAUNCH it. Now look for your POIs and see if they came out looking the way you had expected.

Congratulations, you are now an AR author!

@ubistudio project: mobile AR layer for 2010 01SJ biennial

Wednesday, September 15th, 2010

One of the goals of the @ubistudio is to actually do projects with new media technologies, not just talk about them. In that spirit, we made a mobile augmented reality experience for the 2010 01SJ Biennial that takes place this weekend, Sept 16-19, 2010.

It’s a fairly simple layer, developed on the Layar AR browser and featuring basic points of interest (POIs) for many of the public artworks and venues of the 01SJ festival. Here’s a screen shot of our layer in action on an iPhone 3Gs:


Among the many artworks featured are “Play Me I’m Yours“, ~20 street pianos created by artists Luke Jerram; Poetics of Dis-communication by Patrick Manning, and ZOROP by Ken Eklund and Annette Mees. You’ll also find the major venues and outdoor performances, to say nothing of the stops where you can catch the ZOROP Mexican Party Bus!

We submitted the layer to the Layar developer program, and it was approved earlier this week. If you’re at 01SJ and have a newer iPhone or Android phone, please check it out and let us know how you like it. You’ll need to download the free Layar app for your phone if you don’t have it already. Then just search for “01SJ” and you should be able to find it easily. All of the interesting points are in the downtown San Jose area, so if you’re not in that area you won’t see much ;-) If you have questions or feedback, ping us on Twitter: @ubistudio or just get involved by coming to our next Ubiquitous Media Studio meetup.

Beyond Augmented Reality: Ubiquitous Media

Saturday, June 19th, 2010

Here are the slides I presented during my talk at ARE2010, the first Augmented Reality Event on June 3, 2010 in Santa Clara. Many thanks to all who attended, asked questions and gave feedback. For interested Bay Area folks, I will be organizing some face to face gatherings of the Ubiquitous Media Studio to explore the ideas raised here. The first one will be in July; follow @ubistudio on Twitter for further details.

ARE2010: kicking off the augmented reality summer of love

Tuesday, May 18th, 2010

ARE2010 – the Augmented Reality Event – is just around the corner on June 2-3. In case you missed the memo, this is going to be an outstanding conference! I’ll be giving a deep dive talk on Experience Design for AR, expanding on what I presented at Web2Expo earlier this month. More importantly, there will be over 80 great speakers from the AR world, including keynotes by los luminarios Bruce Sterling, Will Wright, Jesse Schell and Blaise Aguera. Don’t miss this, seriously. And when you register, use this ARE2010 special discount code: E195 to get the full 2 days for just $195. It’s a freakin’ bargain, folks. Be there.


Experience Design for Mobile AR: my Web2Expo slides

Wednesday, May 5th, 2010

my talk on mobile AR experience design at Web2Expo

Tuesday, April 27th, 2010

I’m presenting a session at Web2Expo in San Francisco on May 4th, titled “Challenge, Drama & Social Engagement: Designing Mobile Augmented Reality Experiences“. Here’s the blurb:

Mobile augmented reality adds digital overlays and interactivity to the physical world using the sensors and display of your smartphone. Design of mobile AR experiences is complex and takes us well beyond the browser-based web. This session will give you a mix of practical knowledge and new ideas for creating AR experiences, drawing from web design, 3D graphics, games, architecture and stagecraft.

The next generation of mobile augmented reality applications will go well beyond simply overlaying points of interest, floating post-its and 3D models on the video display of your phone. Mobile AR is becoming a sophisticated medium for immersive games, situated storytelling, large-scale visualization, and artistic expression. The combination of physical presence, visual and audio media, sensor datastreams and social environments blended together with web services offers tremendous new creative possibilities. However, the design challenges of creating engaging, exciting and compelling experiences are quite significant.

Research on the design of technology-mediated experiences has shown that compelling experiences often involve a mixture of physical and mental challenge or self-expression, a sense of drama, sensory stimulation, and social interaction. These elements can give us a physical “buzz” by activating the release of adrenaline, endorphins and related neurochemicals.

Mobile AR puts us “where the action is”—in motion through the physical world, surrounded by other people, in a stimulating environment. AR applications additionally provide challenges, stories, information and communication. Factors that AR experience designers need to consider include:

  • Goals of the AR experience
  • Users’ cognitive model of the system
  • Physical environment and context of the experience
  • Social context of the experience
  • Design of interaction models and experience mechanics
  • Story, goals and outcomes
  • Immersion and flow
  • Design of visual and audio assets
  • Non-player characters (“AIs”)
  • Tracking and analytics
  • Technical capabilities and limitations of the AR system
  • Managing the production process (designing an AR experience has much in common with producing a movie on location)

Should be fun, ping me if you’re going to be at the conference!