Quantitative vs. Qualitative Research

Quantitative vs. Qualitative Research

One of the most important aspects of delivering a solid product is making sure a lot of research goes into the effort. Given the limited time and money a project has, it's essential to understand the aspects that should go into your product for the best possible end user experience. In this post, we'll be going over quantitative vs. qualitative research, including their differences and when to employ each kind of research.


Quantitative Research

Quantitative research includes A/B testing, fake door testing, and following patterns.

A/B testing tests changes on the page against the current design. For example, half of our customers would see a blue button while the other half would see our current grey button. If the blue button shows a higher click-through rate, we know that the change is an improvement.

Fake door testing is placing a button for a feature that hasn't been rolled out to see how many people click on it. This helps gauge interest and saves effort if the response is a small one.

Another useful quantitative research method is tracking key metrics and noticing patterns. If a certain page setup consistently sees high conversion, then it's a good idea to follow that page setup for future new pages.

The most important goal that quantitative research accomplishes is that it answers the what question.


Qualitative Research

Qualitative research includes customer interviews, focus groups, and other standard usability testing.

At my company our UX researchers conduct interviews with actual customers to go over an existing feature on the site. The researchers watch customers navigate the website and ask follow-up questions to track usage patterns and note any confusing parts of the site.

The UX researchers also hold focus groups to discuss potential new features to gauge interest or hear suggestions. Frequently the final feature hasn't been created yet, so they use prototypes or even images to simulate how the actual feature would function.

These interviews and focus groups are helpful because customers' feedback gives us a glimpse into how people are actually navigating our website and allows us to pinpoint current and potential problems.

The most important goal that qualitative research accomplishes is that it answers the why question. 


Again, we're just scratching the surface on user research and testing. Laura Klein has a very helpful article on Medium that goes over quantitative and qualitative research in more detail, including what to do when testing one variable versus multiple-variable changes to a website or feature.

The key thing to remember from this post is that quantitative research comes up with what the problem is, and qualitative research answers why this problem is occurring. It's important to have a mix of both types of research in order to best serve the customer's needs and deliver an amazing product.

 


Looking to level up your user research skills? You might want to check out our popular course: One Week PM. You’ll learn the fundamentals of product management, how to launch your own side project, and how to dominate product manager interviews.

Join 30,000+ Product People and Get a Free Copy of The PM Handbook and our Weekly Product Reads Newsletter

Subscribe to get:

  1. A free copy of the PM Handbook (60-page handbook featuring in-depth interviews with product managers at Google, Facebook, Twitter, and more)
  2. Weekly Product Reads (curated newsletter of weekly top product reads)
Powered by ConvertKit

Published in On the Job, Product Management, Product Manager, Skills

Tagged in

6 Responses to “Quantitative vs. Qualitative Research”

  1. […] a particular time and place for quantitative data, and there’s a particular time and place for qualitative […]

  2. […] like how you regularly update your understanding of your product’s user through qualitative and quantitative user research, you should be doing the same to yourself as a PM. How do you regularly capture feedback about your […]

  3. […] feature prioritization is that it’s not set in stone. As you begin building the product and gathering customer insights from tests or interviews, there may be instances where it makes more sense to pivot from the […]

  4. […] down a few problems you’ve noticed, talk through how you might speak with potential users to validate the problem. You should be as detailed as possible and explain how you might find these users, what questions […]

  5. […] conduct a quick sneak-peek for a new prototype or show feedback from a recent user test. The goal here is to inform executives on what kinds of questions the product team is tackling, and […]

  6. […] managers should play a key role in handling objections, because product managers have qualitative insights from user research, backed by quantitative product metrics and data across their entire existing […]