Trevor F. Smith: Exterior

Webbish Augmented Reality

It seems to be my lot in life to point out when we (the community of people who spec and build tech) are ignoring the lessons of the web when feeling our way into new territory.

This post is a short note in webhead lingo about how the augmented reality (AR) community should stop inventing new protocols and, instead, embrace web techniques. I'm assuming that the reader understands why the web always wins, but please feel free to write this up in language that the AR community will grok.

Problem space:

In this post "augmented reality" is narrowly defined: A live video feed overlaid with spatially placed graphics which add information gathered via the Internet.

High level summary of Webbish AR:

Webbish augmented reality consists of decentralized services and clients which make use of the patterns and protocols of the web: HTML/CSS/Javascript and HTTP 1.1 provide the base on which techniques like Comet, Ajax are laid. Together with a few new technologies like WebGL, browser geolocation, and HTML5's audio and video elements, the web provides a complete platform for AR.

An example instantiation:

In this example, there are layers containing resources. An example layer might be named "Public art in the City of San Jose" and an example resource in this layer might be named "Big Stone Finger Sculpture" with metadata including a location in SJC, an artist's name, and a fiduciary marker ID.

AR layers have two components: 1.) a JSON-over-Comet based protocol for discovering and receiving updates for nearby digital resources and 2.) a RESTful api for serving the resources. Resources are referenced by URIs and are hypertext documents of types like HTML and Collada.

Initial discovery of layers is performed using mDNS (a la Bonjour in Safari), by visible marker, by hand entered URL, or by querying the GOOG for layers by content-type and location.

Display is handled by the web browser you also use for 2D browsing. In the same way that the browser can open "Top Sites" or "History" pages, the browser can open "Augmented Reality" pages. The most prominent part of an AR page is an HTML5 video element (showing a stream from your device's camera) overlaid by a canvas element using a WebGL context to render two and three dimensional resources. The javascript on the page manages the comet connections to the layers, the geolocation and orientation event handling, recognition of fiduciary markers, and canvas drawing.

Yes, all of the open questions in any other AR system still exist (e.g. How do I find and present metadata for both a statue on the ground and a plane flying overhead?) but the takeaway here is that the exploration to find those answers should happen on the web so that we don't have to then watch the web eat all of our hard work like it did with video and is currently doing with MMOGs.

Code bases of interest:

FriendFeed's Tornado httpd for non-blocking long lived Comet, Safari and Firefox's nightly builds of WebGL, the geolocation api, and their HTML5 based video and audio elements.

Final note: I'm fully aware that we can't build this without changing today's browsers. That said, there's nothing here that requires changes to today's web protocols and standards beyond what any half-asleep monkey can see will soon occur. Be the change you want.

Art Cloud

For the past few months I've been involved with an exciting project to bring open techniques and infrastructure to public art. I'm working with the design firm Gorbet & Banerjee, The City of San José Public Art Program, and the Mineta San José International Airport to make the new terminals a demonstration of how art programs can embrace digital tools to make the entire experience of art (and the net) more cohesive than has been seen to date.

In addition to the many players who are constructing the terminals, I'm starting to work with the first wave of artists with pieces in the airport. There are too many to name here, but here's a sample: Aaron Koblin, Deborah Aschheim, Maggie Orth, and the powerduo of Matt and Susan Gorbet.

When I was approached to work on this project I was amazed to find that they are taking a very progressive approach to development and infrastructure. Since I joined we've used agile methodologies to construct a common infrastructure for shared data and management which will underpin all of the works of art. This includes pulling in a lot of environmental data about the airport and surrounding regions (e.g. weather and flight status) as well as a comprehensive control interface for the airport staff to use while maintaining the art.

What I've coded so far:
- Django based web app for tracking art works, equipment, and artist info
- A machine health monitoring system which will run on everything from embedded systems up to massive graphics systems
- Hard power controls using the iBoot network controlled power strips
- A wiki system which the airport staff will use to persist organizational knowledge
- A RESTful api wrapping the USGS weather data
- A calendar system for planning events like art installation, opening, and closing

Upcoming work:
- Web based control of AudioBox64 sytems on the jet bridges
- Building Management Control integration for lighting systems near art works which use camera and video projectors
- RESTful api wrapping flight data like arrivals, departures, and delays

One really nice aspect of the project is that all of the code I create is licensed under the Apache 2 license, and most of it is already available in the ArtInfrastructure git repo. Once the project has been in use for a while I'll spend a bit of time polishing the documentation so that other public art projects can make use of it, but in the mean time I'm just having a ball seeing everyone's hard work come together.

In Which I Make a Tentacle Face

Sport a District 9 Tentacle Face - More DIY How To Projects

Why I Maintain Hope

My gut often tells me that individuals are smart but crowds in meatspace are... not. This is a wonderful counter-example in which Bobby McFerrin shows us that we're wired for beauty and the right leader can bring it forth.

World Science Festival 2009: Bobby McFerrin Demonstrates the Power of the Pentatonic Scale from World Science Festival on Vimeo.

LOVELAND: Buy an Inch in Detroit

I'm already in for 100 inches, which I'm certain will be worth much more than that after he launches.

Coworking

I've been working out of Office Nomads for the last two weeks and it's a huge step up from my recent stint of home office and desk surfing lifestyle.

Here's a quick video of the space:

BarCamp Seattle 2009

This weekend I took part in the second annual BarCamp Seattle, an unconference in which 300 people come primed with ideas to share and questions to explore. I like unconferences because I feel like I'm running at 4x clockspeed and I often get pulled into situational comedies like the lightning rants (I was the one ranting about lolcats) or sounding boards like the "Startup in a Box" session which Rob Eickmann and I pulled together after discovering that we've been ruminating on a lot of the same questions about how to smooth the mechanics of startup legal and financial initialization.

I was also wrangled into sponsoring the Sunday morning donuts and coffee, so I'm partly to blame for the 300 pairs of too-tight pants walking around Seattle this morning.

For a sense of the sort of people and events at a BarCamp, check out the bcs09 Flickr tag.

Twilleetio: Twilio + Twitter + Django fun

A few weeks ago I was evaluating phone integration options for a public art infrastructure gig when I ran across Twilio, a service for interactive telephone applications driven by your web app. They handle the phone esoterica and connect to your web API to determine what to say, when to request keypad input, and other such telephonic interactions.

I'm impressed with how they've greased the skids for new developers, and in about 10 minutes I had a Hello World phone app talking with my django based system. (Well, I called it "Talk Nerdy to Me" but let's pretend it was Hello World) It's clear that the Twilio team is in tune with what makes developers happy, because the docs and code examples made everything easy (like Sunday morning, not like Staples).

Then the other night I noticed that Twilio are giving away a netbook per week to people using their system, so I took a couple of hours to whip up a call-in system to record audio then tweet a link to said recording. I deployed the code to my EC2 cluster, pushed the code to github for public consumption, then submitted it to Twilio and called it a night.

A few days later, one of the Twilio developer evangelists pointed out that they have a new audio-to-text transcription service so like a crow drawn to a shiny bit of foil I added a transcription field to the models (thanks South!), added a new callback view in Django, and voila, twilleetio also tweets a bit of the transcription.

So with very little fuss I made a fun Twilio + Twitter + Django mashup and you can see the tweets at @twilleetio.

Hacking together these services with Django was neat, so it was a bonus to learn that the Twilio team dug my app and they're going to send this week's netbook my way.

Helping Others Read News Online

If you're a reader in the US then I don't need to tell you that the fourth estate is down for the count. As part of the technoratti, you might not feel the pain yet. You're skimming through hundreds of feeds in your RSS aggregator and you're bouncing around the web like a ferret on crack. However, you probably know a few holdouts who still read news on paper with their morning coffee.

Those people are suffering and it's your responsibility to help them. Their papers are shrinking in size and content, if not dying altogether, but when they look at the flock of available services out there for reading the news they prefer to stick with their withering pulp.

Enter Acrylic Times:

This is an aggregator which takes a few interaction techniques of newspapers and brings them to your laptop. So, if you have someone who needs a gentle push onto the web of news, try prepopulating Acrylic Times with feeds from the newspapers they read and then sit them down with their morning coffee and your laptop.

GonzoCamp

If you're a Seattle area news+tech weenie, I expect to see you at GonzoCamp:

In the spirit of a startup weekend, journalists, programmers and business development folks will gather in the name of news innovation. Using a barcamp-style unconference, the goal is to pitch ideas and work in teams to build a functioning prototype of a new digital destination or platform in one day.

Thanks to the University of Washington's Masters of Communication in Digital Media program, we have a great room at UW with wifi and whiteboards. Serra Media will provide coffee, lunch, and the first round at the Big Time Brewery to celebrate our accomplishments.

We're looking for tech-savvy journalists, programmers, designer/developer/UI people, entrepreneurs and some college students.

I don't know what we'll have at the end of one day, but these events are great for finding smart people.