Quant data leaves Consumer Duty blindspots

Financial services firms rely on quantitative (quant) data to understand their customers. It’s essential to meeting new regulatory obligations, but it’s not enough. While demographic and usage data guides sales and marketing, it reveals little about customers' perspectives, motivations, or vulnerabilities.

To embed the Consumer Duty firms will need to fill these blindspots. That's why qualitative (qual) data is also essential.

To bring it to life, consider these three short examples:

A firm asked us to design a digital debt collection journey. They had a clear idea about what they wanted – a good start!

We got quickly to work, confidently drawing on our years of user-centred design experience and established best practices. We developed a tone of voice that we agreed felt calm and supportive, yet firm.

But when we tested our prototype using qual research methods it was immediately clear that we'd got it wrong. This is what we heard from our first research participant:

We’d recruited participants who had experience of problem debt, and despite our best intentions the journey triggered negative memories and trauma. All it took was a few short words in an introductory email title.

Here's some context that we wouldn't have found in web analytics: debtors using the online journey needed to feel safe.

If the firm had launched the journey without the insight qual gave us it would have been a pointless exercise in IT delivery, because few among the target audience would have made it past the email. And its customers would have refused the help on offer and remained in problem debt.

But our qual methods allowed us to identify then meet that need. Using what we learnt, we re-wrote the copy and tested the prototype again. This time the response was completely different – people engaged with the help on offer. When the debtors felt respected their fear eased and trust began to build, making it possible to have a productive dialogue about repayment.

Many firms struggle to service experienced customers and ‘newbies’ in tandem. It's common to assume that one cohort needs more help than the other. But the customers at risk of harm aren't always the ones you think.

We helped a firm design online tools to help experts choose investments. The company had an analytical data model that identified sophisticated customers using data about their holdings.

We designed tools we thought they would find useful, then watched and listened as they tried using our early prototypes.

Things started going awry, but not for the reasons you might expect.

It wasn’t that the investors disliked our ideas – it was that they were much less sophisticated investors than anyone had thought.

Using qual interview techniques, we learned how they made decisions. We found that many had a poor understanding of investing. They were making decisions based on hunches, what their friends where doing, tips in newspapers, brand allegiances, family histories and a whole host of other unhelpful reasons.

The sophistication of their portfolio was not a reflection of their expertise. Most investors – even supposedly sophisticated ones – aren’t entirely rational agents. Their decisions are driven less by mathematical probabilities than emotional drivers.

The firm’s strategy had been built on flawed assumptions because of this blindspot.

On a less serious note, in 2021 a meme did the rounds comparing King Charles to Ozzy Osbourne. You might have seen it. It pokes fun at how poor demographic data can be in helping us to understanding people. If you missed it:

It’s a contrived example of course, but there’s an important point here: pre-defined, structured data formats can only ever tell us about what we’re already looking for. This can be useful, but it tells us nothing about what we didn’t look for.

This can lead to massive blindspots!

The theme running through these three stories is how important it is to deeply understand customers and their broader context, and how quant data alone isn’t up to the task:

• In the debt collection example, it was only through learning about people’s past experience and how that shaped their expectations, that we could design an effective solution.
• In the investor example, it was only by speaking to and observing the people who’d been identified as ‘sophisticated’ that we understood that they weren’t who they appeared to be.
• And in the case of Charles and Ozzy, we can safely assume they have very different needs, attitudes and behaviours but there’s no way to know that from the ‘known’ data.

So if quant is insufficient, how do we make sure we’re gathering the right data? By complementing it with qual data. And that means doing qual research.

In my next blog, I’ll address some common misunderstandings and questions about qualitative research, such as “Why have you only spoken to a handful of people?”

In the meantime, drop me a line to discuss!

tom.scott@cxpartners.co.uk