• Neil Schwarz
• 17 Dec 2010
• 9 Comments

We’re really excited to try out the new Kinect motion sensor for the XBox. We were interested in how the Kinect was used to navigate through menus so decided to run a usability test. In case you were wondering – this wasn’t just an excuse to play on the XBox for three days in the run up to Christmas!

What we did

Tested 3 games with 10 people – all Kinect newbies with varying video game experience

Kinect Adventures

Uses a mouse like cursor system, with timed confirmation to select.

Kinect Adventures menu

Sonic Free Riders

Uses a carousel system to scroll through options, drag an option to select.

Sonic Free Riders menu

Dance Central

Uses a scrolling list, with a swipe gesture to select.

Dance Central menu

Here’s what we found out…

User expectations

We let participants try to figure out how to use the Kinect. Here are some of the interactions they expected to be able to do:

Making a selection: All participants expected to make a selection by completing a positive gesture – such as a tap with their hand or finger. Read on, to see how close each games manufacturer came to meeting this expectation.

Pause: Most participants expected to hold one hand up to the sensor to pause game play. Some tried to use the sensor’s voice recognition by saying ‘XBox pause’. The actual pause gesture (hold your left arm at 45 degrees) didn’t feel natural, but was easy to recall after participants got used to it.

Skip video: Participants tried to use a swiping action to skip through unwanted video clips. It took several seconds of waving their arms for participants to find out that the Sonic Free Riders intro skips when one arm is raised.

Accuracy

Cursor control
Some participants commented that the experience of moving the hand cursor matched with moving a computer pointer with a mouse. The system tries to help your aim by snapping the cursor to target areas.

Participants were familiar with how the system worked, but struggled with accuracy – often overshooting due to a slight lag (sometimes amplified as the snap pulled the cursor into place).

Sometimes participants struggled to keep the cursor on the button until the countdown had completed. In general, they felt this could be overcome with experience.

Carousel selection
Sonic Free Riders had the most severe usability problem. The primary navigation method was a carousel of options. This meant all the options were on view, but they were not selectable until they were at the front of the carousel. Most participants tried to select an option in the background.

“I have no idea how to get the green guy – I was trying to grab it”

Accidental undos
Participants encountered further problems on Sonic when they tried to spin the carousel more than once. A diagonal swipe made the carousel spin in one direction, but when participants moved their arm back to repeat the motion the system considered this as the opposing gesture and spun the carousel back to its original position.

Proximity
Participants also found they repeatedly opened information panels on Sonic Free Riders by accident due to being too close to other buttons.

Using a single axis
Most participants were more accurate when making selections using the Dance Central menu because they were scrolling up and down through one axis rather than moving a cursor through two axis.

Selection was controlled by a loose swipe at the discretion of the participant, rather than dependent upon a count down timer. Participants were less concerned about making errors as selection was in their control.

Speed

Timed selection
Using a countdown selection method means that it takes a long time to make selections. Even once you are used to the system. Less error prone, but slow.

Unnecessary steps
Once participants began to understand how to operate the carousel, they found it slow to make a choice. They had to spin the carousel to the correct spot using repeat gestures, then they had to make an additional gesture to make a selection.

Positive gesture
Instinctively, participants wanted to make a positive gesture to make a selection. With Dance Central, the selection was dependent upon their swiping action. They were in control of how long it took to make a choice.

“This is good – when I did it, it took half a second. It felt like it was responsive”

Clear mapping
In Dance Central, the opposite gesture with the left hand gives the opposite system response, which gave a greater sense of control. It felt quicker to use both hands together. It also maps what users are used to with other systems like web browsers and DVD players, Right to move forward, left to move back.

Perceived responsiveness
The Dance Central menu felt more responsive and none of the participants complained that they felt any lag. They did not have to aim at a specific point, just move their arm to a general position.

The ‘old’ way is quicker
Most of the participants said they would prefer to use the controller to get through the menus, only wanting to use the sensor for actual game play.

“Comparing it to a controller it is quite a lot of work to scroll through the options”

System feedback

System feedback is critical to any successful interaction, but perhaps more so when the user is controlling the interaction without touch.

Mirroring user movements
The presence of the avatar mirroring your position in the Kinect Adventures menu provided strong feedback. Participants could clearly see themselves, so instantly understood how their movement would effect what was happening on the screen.

“It’s good that you can see a mirror image so you know where your hands are”

Participants were delighted when they noticed the avatar had copied involuntary actions, such as scratching their head, it made the menus fun.

Clear highlighting
Dance central’s menus made it very clear to participants that they were scrolling through the options by raising or lowering their right arm. A single option was highlighted at any time, so participants knew what they were about to select.

Context of use

We noticed a couple of issues that related to the context in which the sensor will be used.

What if I’m chatting to someone: In some instances whilst using Kinect Adventures, participants accidentally made selections whilst they were chatting to us, rather than facing the screen. Their hand position was on an active part of the screen so the system interpreted a gesture. The system moved them through several screens without them noticing. This problem could be compounded if the sensor is used in a social environment as intended.

Can’t read all the text: The system helpfully tells you which is the optimum distance to stand from the sensor. On a few occasions participants commented that the text on the 22” screen was too small for them to read at that distance.

Pausing takes too long: Participants felt the pause control significantly interrupted their game play because it took too long. By the time the pause was initiated the player had lost their position in a race or missed a target.

“The pause is really slow – if you were first place in a race, everyone would pass you”

I don’t want to stand up until I start playing: Participants wanted to be able to control the menus using the controller. This simply gives them a chance to sit down between games.

Conclusion and recommendations

We’ve loved playing with the Kinect. There’s no doubt that the game play is lots of fun. In game menus are a barrier to that fun. Kinect should allow players to move through menus quickly and compensate for inaccuracy.

We felt the Kinect would benefit from some standardised global controls – much like a controller uses the A button to select and the B button to move backwards. We also think it needs a more responsive pause gesture – one that doesn’t interfere with the user’s game play.

Most of our participants found the Dance Central menu to be more effective, more efficient and more satisfying to use. Here are our recommendations for designing a Kinect menu interface:

1. Allow users to make selections through positive gestures, rather than timed positions
2. Place options on a single axis to make them easier and quicker to select
3. Allow users to control menus with the game pad if they prefer
4. Use large easy to read text
5. Don’t make users scroll through options unnecessarily – it takes too long
6. Users will be distracted if used in a social setting – test your menus in a social context to see if they are prone to errors
7. Avoid the cursor metaphor, it’s not what gamers are used to seeing in game menus, and makes it harder to implement alternative joypad controls

About the author

Steve Cable

Steve loves taking user test results and putting them in to design solutions. He has a background in design and media that he puts in to practice when he creates prototypes to test with. Steve enjoys making Flash based computer games, that have bizarre storylines!
Email Steve, or call +44 (0)117 946 3930

---