IdentityMine and REZN8 host IxDA Surface Event

Last night IdentityMine (my employer) and REZN8, IdentityMine’s strategic design partner, had the distinct privilege of hosting an event at our Los Angeles office to showcase Microsoft Surface for IxDA, a community of Interaction Design experts. We discussed the opportunities and challenges for designers and developers that Surface presents. Andrew Whiddett [IdentityMine] led the half of the presentation dedicated to developer concerns while John O’Keefe [REZN8] lead the designer discussion. Both presenters were eloquent and informed since IdentityMine and REZN8 are among the top three Surface shops in the world. The event went tremendously well (and not just because of all the wine and hors d’oeuvres) so I decided to post a summary here.

A Little History

What is now called Microsoft Surface started in the early 2000s as a Microsoft research & development project called ‘Play Table’ and was originally envisioned as a new platform for gaming. Microsoft quickly realized that there were opportunities for other types of uses than just games so full-scale development went into creating Surface as a brand-new application platform. If you’ve seen ‘Minority Report’ you’ve seen some of where Surface computing is heading.

IdentityMine and REZN8 first got the Surface alpha in 2006 and began working with Microsoft and Harrah’s on Food and Drink applications. The result of these efforts have been recently released at the Rio Hotel and Casino in Las Vegas.

Bill Gates publicly announced demonstrated [edit, thanks Josh Santangelo] Surface at CES 2008. The applications he demonstrated there (Wine Bar and Snowboard, shown in the videos below) are IdentityMine and REZN8 applications.

Recently, Surface applications have also appeared in Starwood hotels and AT&T stores.

Fingers, Not Mice

The primary means of interaction with Surface is fingers, not a mouse. Unlike most other touchscreen computers, Surface recognizes multiple points of contact at the same time. This means applications can support multi-finger gestures. Gestures like pinch and spin, that have become familiar for iPhone users, are possible when the machine recognizes more than one point of contact at a time. In the future, touchscreen gestures will become as much a part of the collective consciousness as right-clicking with a mouse. This will lead to more sophisticated, yet natural interactions with multi-touch enabled applications.

While the iPhone was the first truly mass-market multi-touch device, it was in development at the same time as Surface. The similarity of the gestures between the two devices further underscores the validity of the interaction. Indeed, natural gesturing as a means of human-computer interaction has been a field of extensive study and research since the 1970s. One of the earliest pioneers in the field, Bill Buxton, is a Principal Researcher at Microsoft.

Gestures themselves are a fascinating topic. Imagine you have a user interface construct – an orb that can be dragged or flicked with one finger and resized with two fingers. Inside that orb you have a carousel that can be rotated horizontally with one finger. The items in that carousel can be ripped out of the carousel vertically with one finger. Now, with all those nested controls, when a user puts a finger on the control, how do you determine what he or she means? Drag it? Rotate the carousel? Rip an item out of the carousel? All good questions.

Finger interaction presents unique challenges as well. For example, Surface applications avoid the need to use a mouse or keyboard. Therefore new and novel methods of interacting with applications must arise. Instead of typing an address into a field to steer a web browser, we must invent a new method of browsing, perhaps one more like the internet William Gibson envisioned in Neuromancer.

Going Beyond Touch

What distinguishes Surface in the world of multi-touch computers is its vision system. Capacitor-based multi-touch devices like Apple iPhone, Jeff Han’s Perceptive Pixel device and JazzMutant Lemur use electrostatic resistance to “feel” touch. Surface uses a camera system to “see” touch. This means that Surface can “see” other objects too. In fact, Surface can see the shadow cast from a finger, not just the fingertip, and determine the user’s orientation to the table from his or her finger’s shadow!

Surface can recognize objects like encoded tags, credit cards, mobile phones, pretty much anything, and then interact with that object in a way that no other platform can. Surface is really like a “mixed-media” environment where applications can use a combination of real objects and virtual objects together. This opens Surface up to a whole range of possibilities that are simply impossible on other multi-touch computers and utterly unimaginable on ordinary comptuers.

Multi-User Shared-Space Applications

Today’s applications are principally designed around a single individual sitting at a single workstation. There are very few single-workstation-multi-user applications in the world. The most notable are single-console multiplayer games, and in most cases each user owns a portion of the screen real estate, like split-screen racing games, for example. Surface is radically different in that it’s designed explicitly for multiple users to interact at the same time.

There are three main types of multi-user interactions in Surface: a ‘host-guest’ scenario where a host drives the application and offers features to a single client, like a concierge application; a ‘collaborative’ scenario where multiple users are working together toward a common goal, like a restaurant ordering application; and a ‘discrete’ scenario where multiple users are working independently and each wants a piece of the screen real estate for themselves. Each scenario is different and each requires a different approach to user interaction design.

One of the constant challenges in multi-user applications, especially in the ‘discrete’ scenario, is real estate. There are a limited number of pixels available on the screen. When four people using an application at the same time each want a piece of the table, you can quickly run into situations where there is a shortage of workspace. Traditional applications are like cubicles – each user gets their own little space to do with it what he or she pleases without affecting the other users. Surface applications are more like a large worktable with people gathered around it, sharing the space. If one user has a messy work style, scattering his or her objects around the space willy-nilly, its likely to get in the way of the other users, much like real-world space. Managing each users objects in the environment is a fun challenge to address. How do you know if a particular object belongs to you or another person? Color-coding? Tagging? A way to collect all your stuff in neat piles in front of you?

Omnidirectional Applications

In traditional applications there is a clear sense of top and bottom, clear borders, most everything lines itself up along right angles. Now think of Surface, an environment where a user can approach it from any direction. How do you know which way to orient the text when there is no real “top?” When a user approaches the device they expect the application to be facing them, regardless of where they are. How do you handle that?

Envision an application where two users are sitting across from each other, working together. If one user flicks an object, say a photo or a document, over to the other user’s side of the table, should it orient itself to face that user? At what point should it orient itself? These types of questions are the challenges omnidirectionality present but at the same time they offer tremendous flexibility. Surface applications are fluid, organic and amorphous. Rectangular windows and grid systems just feel out of place in Surface computing.

Physics and Virtual Objects

When an object is represented on a regular monitor, and manipulated with a mouse and keyboard, no matter how realistically the object is rendered there user never feels like he or she is interacting with a virtual object. On a Surface computer an object feels more visceral. Manipulating objects in Surface with the pointing device we were all born with, users feel a deeper physical connection with the object, bringing their real-world physical experience to the application. Consequently, they expect the object to behave in a manner consistent with real-world physics. Suddenly, gravity, friction, inertia, mass and collision become critical features of your application where previously these features were only found in gaming.

Here is a scenario that can help illustrate the importance of physics in a Surface application: imagine an application where a user has a number of photos scattered across the table. They also have a virtual toolbox in front of them that they can use for photo editing and manipulation. The photos and the toolbox all have physical properties – they can be dragged, flicked and spun around the application. Now, a user flicks a photo toward the toolbox. What happens? Does the photo bounce off the toolbox? Does it bump the toolbox, moving in response to the impact? Does it slide under the toolbox as if it were on a lower layer? These are the types of physics questions a multi-touch physics-enabled application engender.

The Future of Surface

One of the comments I hear frequently about Surface is its form factor. There was is popular spoof video on YouTube talking about the future of computing being a “big-ass table.” Yes, it is a big-ass table. We all know that. Microsoft knows that. Think back to the mainframes of the 1950s. The computing power that used to occupy an entire room is now eclipsed by the computer in your pocket. It is inevitable that the form factor will get smaller, lighter and more flexible. Due to a non-disclosure agreement I can’t say too much about this topic but I will say this: the coffee table is just the beginning…

In Closing

Thinking about designing and developing for Surface raises a tremendous number of new questions about application design. The answers? That will be up to the community of User Interface designers and developers, as well as the users themselves, to decide.

Explore posts in the same categories: Design, Development, Industry, Technology

Tags: IdentityMine

You can comment below, or link to this permanent URL from your own site.