Quality Research Requires A Clearly Defined Audience

As a researcher, how many times have you gotten a request to, “Go do some user research” and been frustrated with the lack of specificity in that request?  Before you can conduct quality research, it must be designed, and designing a qualitative research project starts with defining the audience.  In this context, “audience” is the set of people who will participate in research sessions and can be defined with questions like:

• Whose opinion matters?
• Who is in the target audience for this product, app, or service?
• Who should I talk to?
• Who will be affected by the decisions we make?

There is a direct relationship between the appropriateness of the research participants and the quality of the research result.

Right People, Right Answers

In research, not everyone’s point of view is relevant.  If you’re talking to the right people, you can feel confident in basing product and design decisions on the research results.  If you’re not talking to the right people, the questions you ask, and the responses you get, are not going to be helpful in making decisions, and may, in fact, steer you and your team down an unproductive path.

The Relationship Between Quality Research Participants And Your Objective

Designing your research project not only requires defining your audience but also requires a clearly stated research objective; they go hand in hand.

Below are some examples of marrying participants and research objectives.

Research Objective: Does our new design improve the users experience with our product?
Audience:  Current product users

Research Objective: Why aren’t we retaining people?
Audience: Past customers to learn why they didn’t renew their subscription.

Research Objective: What types of mobile yoga classes appeal to moms?
Audience: Moms who do yoga.

Why Does Audience Matter?

Your audience matters because people are different and those differences affect the way they interact with your product or service.  They work in different contexts, with different roles and responsibilities.  They have different knowledge and different abilities They have different life experiences.  Some are just starting in a career, some see retirement on the horizon.  Some have decision-making authority, some have their fingers on the keyboard.
They have different orientations and preferences.  And on and on.

While audience is important to consider in all kinds of research to reduce bias, it is especially important in qualitative research because qualitative research prioritizes depth over sample size.  You’ll be talking to a small set of people and you’ll want those people to be as helpful as they can be.  Matching research participants to objectives is crucial.

Define Your Audience With Specific Criteria

In order to recruit participants, you have to establish more specific criteria to determine if someone is in your target audience.  Frequency, recency, duration, and skill level are often relevant.  Sometimes geography, job role, or company type matter.

In the example of Yoga Moms, you might ask:

• How frequently: do they need to participant in a class: multiple times a week, weekly, monthly, only once?
• How recently: did they attend a class in the last month, the last year?
• How often: did they start years ago, or just last week?  Are they still actively participating or did they stop at some point?
• For skill level: do they attend beginner, intermediate or advanced classes?  Or are they an instructor?
• On motherhood: does the age or number of the children matter?

When defining your research audience, a more specific criteria for Yoga Moms might be: A woman with at least one child under 5 who has participated in one or more studio yoga classes in the last month.

Or your criteria might be: A woman with children at home who watches yoga videos daily.

There is not a single set of criteria that defines an ideal research participant; the idea is to use the recruiting criteria to establish key characteristics that matter to your research objective.

For our Yoga Mom example, it’s easy to imagine that a mother with three children under the age of five has a very different life experience than a mother with adult children, even if they both attend the same Yoga class.  Whether those differences are important to you depends on your app or product or service.

When Your Defined Audience Is Hard To Find

The fact is that some audiences are hard to find.  It could be limited by geography if, for example, you’re in the US, but want to learn about European attitudes.  Or the target population is very small as, for example, people who have a rare disease.  And some audiences are hard to recruit even though there are many of them, as is the case with C-level executives or surgeons.  High profile professions are often not willing to spend their valuable time on a research project and are therefore hard to recruit.

The extent that someone else can stand-in, or proxy, for your ideal participant depends on your specific situation.

Sometimes, close enough is good enough.

If the research is general in nature, for example, website navigation, a stand-in might be fine if they have basic computer experience, even if they may struggle with some topic-specific terminology.

In the yoga example above, Dads who do yoga, women without children who do yoga, or mom’s who do Crossfit all might be suitable stand-ins for Yoga Moms.  In contrast, people leaving the bar down the street from the yoga studio probably would not be good stand-ins because they are unlikely to share the goals and concerns of a Yoga Mom.

If you do use stand-ins, be very cautious in basing decisions on the data, and clearly communicate they are not members of your target audience.  Any time you use a proxy, the new requirement is to consider their distance from the ideal target in the analysis of their feedback.

For example, If you end up talking to a Crossfit Mom instead of a Yoga Mom, it would not be surprising if she doesn’t recognize specialized yoga terminology, but if Yoga Mom doesn’t recognize that same terminology, it would be a red flag and something to include in your research findings.

But, proximity does not make someone an appropriate choice.

It can be tempting to skip recruiting and just talk the most convenient people.  Perhaps your co-worker in the next cube, or a random person on the street.

But these “distant proxy” users should not be your first choice, even though your office-mate may be more than willing to express an opinion.  Only in the most extreme situation, where your choice is either to use a convenient proxy or have no external input, is using a distant proxy a good choice.  In general you should only use distant proxies if you have tried and failed to recruit your ideal target participant and when some outside input is better than none.

And, intermediaries also don’t count.

Most companies have people who regularly talk with customers.  Both salespeople and Customer Service agents fit this profile.

Are they good targets for research participants?  In general, no.

While internal people can be a good source of background information, and building bridges across organizational silos is a great practice, they are not good research participants.  For one, you are getting information second- or third- hand, with the associated loss of clarity or potential for misunderstanding.  (Remember the telephone game…that can go on here!)

Additionally, both of these audiences have some built-in bias.  Salespeople interact primarily with buyers who, in the enterprise context, are often different from users.  The concern with Customer Service agents is an audience bias.  People who contact customer service do so because they have a problem.  Day in, day out Customer Service agents are dealing with people who have problems.  So the problems, while real and valid, will be over-represented in their experience and opinions.

Internal people in customer-facing roles may provide valuable information, but it’s wise to retain some healthy skepticism about their ability to speak for the users.  A research project should be designed to speak directly with the source.

Key Takeaways For A Quality Research Audience

Quality research starts with defining the correct audience.  Your results won’t be high quality unless you are talking to the right people.

Remember these key points about how to define your research audience:

• Research participants are good to the extent they match the key characteristics of your target audience, are relevant to the research question, and are able to communicate and share their thoughts clearly.
• Not everyone’s point of view is relevant.  Match research audience to your research objective.
• Clearly specify recruiting criteria to determine if someone is in your target audience. There should be no ambiguity.
• Sometimes close enough is good enough; just consider their differences when creating research findings.
• Gather background from internal customer-facing people, but don’t use them as research participants.