- 13 Nov 2008
- 5 Comments
Traditionally, participants for usability tests have been segmented by demographic criteria, e.g. 40-50 years old, 50/50 female/male split with a household income of 50k. We think there is a better way to identify audience segments and ensure that truly representative people are included in user tests. cxpartners identify audience segments based on the tasks that people do, not how old they are or how much they earn.
What’s wrong with demographics for user testing?
Defining an audience segment by traditional demographics is fine for traditional media like magazines, newspapers and TV. The audience segments are well understood, but critically the audiences are passive, they consume the media. When people use internet sites, mobile phones, computer games or even just work with computers they are interacting with the media. Their interactions are not defined by their age or salary but by the things that they do.
Audience segmentation
What’s right with tasks for user testing?
I’ll give you an example of how we recently segmented an audience of people who want to book a holiday. A young couple are considering a holiday in Cuba, they’re not sure whether the hotel is what they want so they look for other people’s reviews of the hotel on a popular review site. At the same time a retired man is considering a cruise in the Caribbean, he’s not sure about what meals are included so he pops into a travel agent’s to ask. Traditional demographic segmentation would never grouped these individuals together. However they are both carrying out the same task of researching a holiday, so they’re both potential candidates for user testing.
Personas too
The great thing about segmenting an audience by task, is that it also provides all the material necessary to develop some really great personas. And just like the audience segments these personas are based on reality not imagination. So, rather than the typical persona material, e.g.
‘Sara-Jane, 37, two kids, drives a second-hand light blue hatchback with a dent in the rear bumper, doesn’t read novels but likes to read about celebrity gossip’.
You get task based personas that reflect the activities of real people, e.g.
Savvy Saver, regularly browses holiday and flight last minute deal websites, writes reviews of hotels and posts them to her favourite review website, reads e-newsletters to keep up to date with what.
Rather than creating a caricature of a single person a persona is created that reflects the broad and varying modes of behaviour that real people exhibit.
An interesting and good alternative view at looking at personas. Could the caricature also be defined by the knowledge and comfort that the person has with the internet? So that a segment may have the same task in mind, but will carry it out in different ways based on their knowledge of the internet.
Good reading and demographics or segments may not tell you about “real need” or “passion” to want to do the task in question. If you can find people who are truly motivated to do the task, it will yield better results.
If an interesting idea, but isn’t it the caricature type details that help to make the intangible idea of behaviour groups more tangible (as in essence personas are behaviour groups )?
Those details you refer too are not supposed to be taken literally they are to help those working on a project/site have a mental picture of an otherwise faceless customer. Yes it may seem corny at times but it has been proven to help make the behaviour groups more memorable.
If the persona is derived and communicated effectively they should be task based anyway(demographics are rarely brought into a persona as a defining factor). Do you not think that by removing the ‘fun’ parts of a persona you could be jeopardising their acceptance, as the “behaviour groups” end up being given a name which isn’t easy to relate too on an emotional level as it comes across as UX ‘jargon’, e.g. ‘Savvy Savers’?
Walt
There is definitely something interesting here, as we are finding out in our work!
I think that some online propositions are so broad in terms of demographics that it makes very little sense to use the “pen pictures” approach as marketing and business folk will always say “but what about *this* type of customer…?” This doubt can lead to a lack of credibility for personas as a design tool.
Nevertheless it has been tricky to rationalise the approach to the numbers of groups and to adequately prioritise their commercial importance for decision making purposes. Like I have said, this is a journey rather than a silver bullet type scenario – as a client team there needs to be a way to develop and build knowledge about using personas and about how our customers behave. It’s a process of refinement over time.
Perhaps the ultimate recruitment tool would be some way to link on-site analytics which flags up patterns of behaviour and provides a list of users from which to recruit. This would reduce the risk of the hit and miss nature of the telephone screener…
We have been talking about setting up an “expert panel” of users from which we can recruit.
@Mike
I think my experience is that the ‘fun’ parts of the persona can also prove to be their weakest spot. People tend to obsess about the persona photo and the other more easily assimilated bits of info and this steers people’s thinking away from ‘user behaviour’. It’s ironic because the behaviour is precisely what personas are designed to provoke thinking about!
However I think you have got something – the way in which you label the groups is important imo. If the labels are too clever-clever I think you easily lose people. This is an area which is about effective communication more than anything else. You need to make sure that you are bringing people with you in baby steps rather than drowning people in airy-fairy esoterica.
DJ
Given that typical usability tests are qualitative, asking what is the best way to segment somewhere between 6-20 people is a great question, Walt.
I find it most useful to deliberately recruit people with and without experience of the task in question. This provides a rich contrast between users: specifically, the needs of experts and the needs of learners. I have found this more productive when it comes to thinking about design issues than to be comparing, say, the differing needs of women or men; or 40-somethings and 50-somethings.
You can go further than this and segment people with differing levels of experience of the task in question: so go for some with lots of experience (e.g. daily), some with occasional experience (done it a few times in the last year). This provides more quality insights: people recall daily experiences with greater accuracy than those who are remembering what they did six months ago; people who have only done something a few times usually show what is memorable (and forgettable) about the experience of the task in question.
Leave a comment