Where Agile Research Fits in your Research Toolkit

Agile research is a solution for researchers and business stakeholders who need data early and often to inform decisions that might otherwise be driven by opinion. Before adding this methodology to your toolkit, it’s important to understand when--and when not--to use an agile methodology.

Not Everything is Quantitative or Qualitative: Agile Adds a New Dimension to Research

As we learned earlier, the key to an effective agile research strategy is understanding when--and when not--to use an agile methodology. Agile research is one tool among many that are available to researchers. Before we can think about where agile fits in processes and toolkits, we need to start by defining the most basic delineation of research approaches. 

Traditionally, research data is thought of in terms of being either quantitative or qualitative. Here’s how we’ll define those terms:

• Quantitative data is measured in numerical terms. In other words, this data is chiefly valuable in its quantity: such as how many people respond to questions in a certain way. 
• Qualitative data provides insight into feelings and reactions that cannot easily be translated into numerical values. This data is chiefly valuable in its descriptive nature: how are people feeling about a certain topic. This is revealed by choices of language and tone.

Agile research may be considered quantitative, qualitative, both, or neither. Getting caught up in that distinction isn’t as important as focusing on one of the most critical dimensions of agile: time. This means that a key value of agile research is the timeframe in which it is delivered. Executed correctly, agile research is used early and often to generate a constant stream of data to inform decision points. Agile research techniques can generate a blend of quantitative and qualitative data that provide the necessary directional input at the right time to inform business decisions.

The thought experiment described earlier lays out the importance of time in generating data by comparing the time it would take to generate a detailed map with the immediate visual input a driver receives from making sure their windshield wipers are working properly. Every driver knows both kinds of information are needed to navigate their course, and they also know that it’s vital to know when to use each input.

Any research methodology has corresponding strengths and weaknesses that must be weighed when selecting the right approach.

The Differences Between Quantitative, Qualitative, and Agile Research

If we look at the traditional quantitative and qualitative binary, it may seem like it’s a simple process to decide what type of methodology to use. If numerical data is more important, a researcher likely chooses a quantitative approach. If the researcher needs to understand feelings or dive deep on a certain topic, that researcher likely chooses a qualitative approach.

In the world of research, especially in market research, quantitative approaches are most common because they deal with structured data. This structure comes from the types of inputs involved or questions asked in a study, which have a predefined and limited set of answer choices. This type of study can be conducted and analyzed at scale because the data is formatted in a way that is easily organized.  Raw data is readily transformed into charts and graphs. Because of the scale of the sample size in quantitative studies, researchers often apply stat testing or conduct rigorous comparative analyses when analyzing data.

Because of the amount of structure inherent to a quantitative study, there is a significant amount of effort put into the design of these studies. It requires a level of research expertise to design, execute, and analyze a quantitative study. Typically, these studies are finite and provide a robust understanding of the subject of the research, which may be quite broad or address several objectives or hypotheses. The nature of most quant studies means that they tend to be time consuming to design and collect data. 

The outputs of quantitative studies often:

• Take months to design, execute, organize, and analyze
• Incorporate data from several hundred or even thousands of responses
• Segment responses from the whole dataset into subsets for analysis and comparison
• Apply statistical analyses and rigorous analytical frameworks
• Visualize data in the form of charts and graphs

Knowing how many people respond to a predefined set of response choices is useful, but does not provide a full picture of people’s attitudes or context for why choices were made. It’s helpful to see that people select certain answers more than others, but researchers often need to understand why these people hold their opinions. That is a question that is most often answered through more in-depth, open-ended forms of research. 

This is where qualitative research enters the picture. Qualitative research takes a range of forms. It can be as simple as going to the nearest Starbucks and asking people to take a look at your website (sometimes called “cafe testing”). It can involve designing and moderating a focus group discussion. It can be focused on collecting and analyzing videos of participants. The outputs of a qualitative study are unstructured or semi-structured in nature, such as open-ended text responses, recorded interviews, and/or transcriptions.

Qualitative studies often require people to invest a significant amount of time to share their thoughts with a researcher. Additionally, the results of qualitative studies do not have a set structure in the form of numerical values, which means it takes more time to synthesize the results. This is not to say that there are not any tools to efficiently analyze qualitative data at scale. There are. However, they may not be fit for every purpose and qualitative studies still typically involve a smaller sample size than quantitative studies.

The outputs of qualitative studies often:

• Vary widely in terms of turn-around time to delivery of data, depending on the complexity of the research method (cafe testing can take a single afternoon, whereas a focus group might take months to coordinate, execute, and analyze)
• Rely on small sample sizes with highly targeted audiences
• Utilize non-statistically driven analyses and rely on researchers to interpret and make sense of the data
• Lends itself to a narrative presentation of data, focused on answering the “why”

There is an ever-growing need for research that falls between the rigorous quantitative research and traditional qualitative research. Agile research fills that gap.

Agile research provides guidance for making decisions that are too often informed by opinion, rather than data. It is research that is designed to be used early and often in the decision-making process, where the cost of making a misstep is outweighed by the cost of conducting the research.

Agile research studies are often used for:

• Early-stage discovery
• Idea, concept, or message testing
• Project or feature prioritization

The outputs of agile research studies typically:

• Are completed within days, not weeks
• Include feedback from a targeted set of participants
• Contain a combination of charts, graphs, and open-ended responses to inform a laser-focused decision point