Andrew Losowsky

Eye life

07 January 2005. 6 comments. Inspired by .

A few weeks ago I wrote in the Guardian about a games console gizmo called EyeToy. The original piece is over here.

I'm happy with the piece, but unfortunately the word count of 800 wasn't quite enough to include the rest of what I wanted to say. Actually, it wasn't even half what I wanted to say, and having told the important story of the device and its background, I found myself having to wind the article up.

Maybe that was for the best, but recently, others have been pondering the future (and present) of interface design and human interaction, which hasn't helped get rid of the thing from my subconcious. So, rather than let my research go to waste, and to help me sleep at night, here's the follow-up piece that the Guardian never commissioned but that I've now written anyway. It's a bit geeky. And it's also in dire need of a good editor to trim it.

But then you probably guessed all that. Read the Guardian piece if you want some EyeToy background. And then press Start to continue for a 2000 word piece that I'm strangely pleased I've got off my chest.

"We are on the cusp of the interface revolution."

Richard Marks is hovering in mid-air.

"And we need one because graphics have improved incredibly in the last few years, much faster than user interaction. The interface is limiting the fun you can have in virtual worlds."

He swings out an arm and grabs a collection of gold rings. He circles his other arm and he - or rather, his on-screen avatar - performs a somersault. It's a live demonstration of AntiGrav, and he's having fun. AntiGrav is the first full-length game written for his creation, the Sony PlayStation2's EyeToy CamEra.

We'll return to AntiGrav later, but to recap, EyeToy relies on the most basic interface ever invented - the human body. Graphics may get photo realistic, but there's nothing real about bashing X to run faster, or clicking the mouse to jump. If your character needs to run faster, run faster. If it needs to jump, jump. The interface gap is suddenly made all but irrelevant. Look at the screen. You see you? That's you, that is.

"There are two main reasons why it works so well," says Gonzalo Frasca, research in computer games at the IT University of Copenhagen. "The first and obvious reason is that it is extremely easy to learn and it involves a very natural interface: body movements. But the most important reason for its popularity is that it is also a fun game to watch. People make a lot of goofy movements while playing it, so it is very enjoyable for non-participants.

"The newer EyeToy games allow you to record little videos. That's a great feature, and they remove the game context from the video so you can have a good laugh at yourself and at your friends. A similar thing happens with the dancing mat games, such as Dance Dance Revolution - it's very intuitive and fun to watch, although the movements can be a lot more aesthetically pleasant when performed by experienced players."

Aesthetically pleasing moves are a great thing for gaming. Watch people play racing games on a console. Some will tilt the controller when they hit a sharp curve. Others tilt themselves. Or maybe they just press down until their thumb goes white - and then swear loudly. Everyone plays differently and camera-based interfaces can make use of that. Instead of studying the manual and tapping complicated combos, you should be able to record and then replay your own special moves in a dancing or a fighting game. With bonus points for style.

"With these games, players can use their bodies to communicate, to express themselves," says Frasca. "In addition to your play style, you also have your body language style. For example, some EyeToy players will try to make minimal movements, while others enjoy doing more grotesque ones. Lots of people also enjoy doing movements that are not functional to the game itself, like spinning or jumping, just because it's a cool thing to do and they are aware that they have an audience."

The design of both hardware and software for a camera-based interface is a totally new challenge. Suddenly there is an importance not just on who is playing but exactly where they are doing it, and what the conditions are like in players' front rooms. Although Richard Marks arrived at Sony London with a concept and working demos of an EyeToy camera, the hard work was still to come.

"The one thing that he's not constrained by and we are is lighting," says Mike Haigh of Sony London. "He can create things in a fixed environment in his laboratory but we have to come up with things that work in every environment. We had to constrain Rick [Marks], or at least put constraints on his technology. People had seen his demos and asked 'why aren't you using that', but it's because of the technological constraints we had. We didn't want peple to plug in a camera that wouldn't work if a light was switched off."

At first attempt, they may not have got it right. Take Denmark, for example.

"One of the cultural characteristics of this country," says Gonzalo Frasca, "is that they pay a lot of attention to furniture design and to making sure that the living environments are cosy. This can be seen in many houses that have lights that are not very bright. I was renting an apartment when I bought my EyeToy and we started playing during the day. After a while, it got dark and there was simply not enough light in the room. So I gathered every single lamp in the house into my living room in order to keep playing."

And how about the Japanese market, that Sony pre-launch felt would be the key market for EyeToy, given that "peripherals don't sell in Europe"? The territory was thought to be so crucial to the game's success that Sony London hired a Japanese graphic artist to give Play its oriental cartoony feel - but things didn't turn out as they'd expected.

"We were all geared up in Japan," admits EyeToy creative director Ron Festajo. "But having been released, Europe is the largest market for the EyeToy, and Japan is selling at a slower rate."

"We've touched on a Japanese market but the game is not truly Japanese," says Mike Haigh. "A Japanese person can see right through that. I think it looks European to them. And in Japan, the environments tend to be a lot smaller. I also hear they're not as family-orientated as we are. The thing we all experience of the family at Christmas playing board games together or gathering round the TV, doesn't happen there as often.

"The size of the room, and the level of the TV - they're lower in Japan - also play a big part. We're doing something about that. We've developed a wideangle lens that will effectively make the room bigger so that you can see your full body."

As with any platform, an EyeToy game reflects the local market that has created it. AntiGrav, for instance, is a more traditional style of game - a futuristic hoverboard simulator - albeit with a less conventional control system. But to do a somersault, you don't actually somersault - instead, certain arm movements represent combos. Neat, but widening the gap again between physical action and virtual reaction.

"It needs gaming knowledge and spatial awareness to truly play it well," says Ron Festajo. "But it was created for an American market and the US understands how their market works best."

So, despite a universal interface of the body, the considerations of how, why and where people play are as culturally specific as ever. And there's more. Videogame controllers are at first unnatural for everyone, so every culture starts at the same point on the learning curve. But everyone already has body movements, and these differ across the globe. As gesture-based games become more complex, so must the games be able to adapt to the different movements people will be accustomed to - and also to what those movements signify in their cultures.

Menus themselves have to be thought of differently, too. Aaron Fothergill is lead coder at Strange Flavour, based in Newcastle. Together with his brother Adam, he created ToySight, a similar kind of experience to EyeToy: Play for the Apple Mac's iSight camera - though neither of them had played EyeToy when they started coding their game.

"It turns out that we used a very different method of scanning from EyeToy," says Aaron. "But I suspect we had the same problems in design. For one, you have to consider carefully the position of controls on the screen, as you have to design the ergonomics around where the player is physically standing and where their arms might accidentally pass through while travelling to another control."

Similarly, EyeToy's latest game, Kinetic, comes with a wide-angle lens add-on, in part to eliminate the interference of "your mum on the sofa drinking a cup of tea."

Another problem in game design is sheer exhaustion on the part of the player. If gesture-recognition as a platform is going to last, it needs to outgrow the instant, throwaway, EyeToy: Play-style minigames and create more in-depth playing experiences for hard-core gamers. But an in-depth game means long playing times, constantly moving your whole body for an hour or more at a time. Which is going to be utterly exhausting. Until that problem is solved, Logitech shares will nary wobble.

"Part of our design concept with ToySight was to try and create a few more long-term games," says Aaron Fothergill. "The Plank and The Owl and The Pussycat [an arm-flapping owl simulator/platform game] are designed to be more like traditional games with a new style of control, rather than simple games to show off the controls. We use the physicality of the game to extend what we can do with powerups. For instance, in The Plank, the main powerup is a 5 second time out to give yourself a chance to rest your arms."

Gaijin Entertainment, based in Russia, has similar ambitions for their forthcoming webcam-based game Flights of Fancy for the PC, where the player becomes an enormous fire-breathing dragon.

"The player can play for many hours without interruption," says Gaijin's Ivan "Nash" Kuznetsov. "Due to the alternation of different types of a gameplay every 5-10 minutes, the player keeps making different movements. In our experience, the player can have 2-3 hours of gameplay and not get tired. And it is more useful and more enjoyable than repetitively pressing buttons."

The main limitation of their game is that it requires an 18" plus monitor - otherwise you have to stand too close to the screen to make the movements successfully.

Flying and hoverboarding are all very well, but the question we all want to know the answer to is: when do we get to use our computers like Tom Cruise in Minority Report?

Richard Marks' answer is... already. He has The Clam - created unsurprisingly by a team of developers who had just seen Minority Report.

"Some of them played around with gloves, but I've never liked tech like gloves, " says Marks. "When you put it on after someone else has used it, it feels all sweaty and it takes away from the enjoyment.

"The Clam a single U-shaped squeezable piece of fabric you put in your hand and when you squeeze it, it changes in aspect ratio very fast. So you can use it as a mouse cursor. You squeeze to click and drag, and then let go to release. Because you can monitor the direction of the aspect ratio, you can also use it to rotate objects."

His lab has developed a simple photo storage/manipulation program to use The Clam with - and it works so simply, it seems almost too obvious.

Camera-based tech is moving on fast - among the goodies currently in the hidden volcano Marks lab are Z-buffer cameras that record depth using an infra-red beam, allowing objects to appear behind you as well as flat on the screen in front. But cameras aren't the only way forward. Haptics - or touch-based interfaces - are another way of using what we already know - our own instinctive movements and sensations - to control gameplay. Touch is one of Sony's four Interface Research Areas (the others being Inertial, Video and Audio - the EyeToy has an in-built microphone, by the way). Tilt-based gaming, through handheld games such as Wario Ware, are also becoming successful. And there's potentially much more within our grasp.

"If you look at mobile phones now," says Ron Festajo, "practically every one has a camera. You can take photos and use it as an input device. It's very exciting."

People understand cameras. And cameras open up all kinds of possibilities. The revolution is already upon us, comrades.

"Joysticks, keyboard and other manipulators have become outdated," says Nash Kuznetsov.

The only trouble is, we're still trying to figure out what will replace them.

(ends)