027 Combining Qualitative and Quantitative Research with Helen Willmot and Simon Clark

Helen Willmot is a UX Research Consultant at REO Digital. Simon Clark is Head of CRO at Evolved Search. In this episode they’ll tell you all you need to know about combining qualitative and quantitative research to optimize your website. Simon also talks about event trackboards, listen on to 

Links in this episode:

  • https://www.reodigital.com/
  • https://www.evolvedsearch.co.uk/
  • https://www.zaraz.com/ (the magical tool that fixes your third-party script slowness)

Episode Notes

When it comes to digital marketing, quantitative and qualitative research are not mutually exclusive. If anything, you should use them hand-in-hand. That’s because quantitative research deals with knowing how people interact with your business, while qualitative research is about why they do what they do.

In this episode of No Hacks Marketing, we are joined by Simon Clark, Head of CRO at Evolved Search, and Helen Willmot, UX Research Consultant at REO Digital, to talk about the following:

  • Why it makes sense to combine quantitative and qualitative research
  • Defining and knowing how to use an event trackboard
  • How to get a buy-in from stakeholder

Why Qualitative and Quantitative Research Doesn’t Work in a Vacuum

Quantitative research uses numbers to define what’s happening to the website. You can’t affect their behavior if you don’t know what they’re doing, and that’s the key. You need to start with measuring what’s happening.

Of course, you start with all the wrong metrics, like how many pageviews the website gets. It doesn’t matter. But when you get to conversion and all that stuff, that’s when it matters. 

I’ve started to think more about what people are doing on the website? Where are they going? What page are they visiting? And how long do they stay on the site?

In short, what are the users doing on the site? Which could then wrap that into a report and present it to the clients by giving them that data. For me, it’s the numbers behind the websites, which are then the customers on the websites.

Qualitative analysis, on the other hand, is very much about taking those numbers and it’s essentially the “why” behind the website. This matters because, without it, you’re playing a dangerous game of looking at numbers and just guessing at what it means.

We’re all biased people, and we all think we know our customers. But the more you research, the more you realize that the customers are always going to surprise you. Essentially, qualitative techniques tell you things why things are happening, and they can be used to provide extra layers on quantitative data. But they can also be used to find new opportunities and identify unmet customer needs.

The Risk of Using Only One Method

Have you ever had a situation at your job where you look at only the numbers from analytics and decide based on that? And then it doesn’t turn out to be true without knowing?

Qualitative research does have the ability to do the qualitative research right, and you probably are just blind to that over time. You’re missing out on that research and insight you get from that. You can’t have as many data points. We take a lot of our research approaches and optimization; they always go through a technique of triangulation to increase the validity of your findings.

All research techniques have limitations. So, the more technique you use, the more you overcome those limitations. The more you overcome those limitations, the more you overcome the biases.

Usability testing is a little bit more than we get that always sits between the two. But if we’re finding attitudes from customers, we don’t know how many customers those attitudes have without adding a quantitative layer on it.

So, when looking at the prioritization of findings from qualitative projects, I’ve docked into analytics and data and used it. If it doesn’t seem to be looking at this area and then I’ll back that up with 0.5% of users that I’m looking at in the other area.

What is an Event Trackboard?

The best way to explain what an event trackboard is, it’s a dashboard with lots of numbers. It’s a quantitative piece and, essentially, they’re measuring engagement and performance. This includes clicks and engagement with elements progression from one page to the next.

Homepage, product listing page, product detail page, and checkout or a kind of lead gen form. Get a day-to-day view for the entire month and split by device. Once you get all that information, you’ll find it incredibly useful in terms of what you’d use that for.

Realize what the problem is, and qualitative is a better way to get that answer, even if it’s not a hundred percent reliable because of the small samples. But just asking the question is going to give you the answer better than looking at some numbers.

There’s so much that these can be useful, but I tend to use them for things like clicks on CTA. My preferred tool is essentially the Google Analytics API, which then I can turn to Google spreadsheet. I’m sure there are a hundred other ways to do it, but that’s just the one I’m very familiar with where I can replicate it quickly and easily share it.

Approaching Qualitative Research

When you ask something directly, you sort of plant that seed in their user’s head. Ideally, we’re going to be doing several qualitative techniques, sometimes at the same time.

For example, if we’re looking at usability testing sessions, we’d be looking at users looking for a return policy. Are they thinking about that? Is it something they’re doing automatically? Do they mention it? If they don’t, we might say, do you have an outstanding question?

So, we’re asking vague questions because, as a researcher, we shouldn’t be tied to a particular outcome.

We do want to ask those non-leading questions. If you’re looking for a survey, you might give several multiple-choice options. For example, if you are asking a user, “what’s important to you when you’re purchasing?” You may include “return policy” as an option there.

My kind of preference when it comes to surveys is even to avoid doing that. To begin with, initially just go out and say what’s important to you about the process. Once we have an idea of responses, create a multiple-choice question about that. I think we should avoid asking questions about it directly, but we can ask about the themes that play into, such as hesitancy around the purchase.

There’s a clear value in doing proper research. You have a stakeholder who’s going to say, how do I know this will get me revenue? What’s your go-to ask? Do you have a template answer for that? How do you address those?

We hear the stats and businesses want to take a part of that, but to get a part of that, to drive those results, you need to do the full process.

The senior stakeholders can be under the deception that it’s going to be expensive and it’s going to be a blocker. UX researchers have to change that perception so it doesn’t have to take a long time. You can get insights within a few days if we’re using the online kind of user testing platforms.

If the stakeholder is reluctant, just say it doesn’t cost much. Once people see the value and insights coming out of the user research, they tend to get a little bit interested.

Setting Client Expectations

Make sure the stakeholders understand that this is a long game. We need to get the data around. We need to set up the tracking on those kinds of key page templates. That’s going to take time to roll in. But once it does, it’s going to be transformative because we’re going to have all this stuff on all our key page templates.


Don’t forget to subscribe to the No Hacks Marketing Podcast. And if you learned something new today, we would appreciate it if you can leave us a review on your favorite podcast platform.