It’s time for something better than traditional market research.
The quantitative/qualitative double whammy of long-form surveys and focus groups still has a place, especially in identifying and measuring quantifiable trends over very long periods of time, but the effort may no longer be worth it for many of the nonprofits, advocacy groups, and brands we work with at Blue State Digital. Besides being slow and expensive, traditional market research can’t always predict how people behave online.
On one hand, methodology has not kept pace with digital consumer trends (or the fact that our attention spans are now shorter than those of goldfish).
Researchers continue to write complex and lengthy questionnaires and expect participants to still give their full attention to the assessments. That’s the same as shooting all 20 hours of the Harry Potter movies and releasing them for audiences only interested in a 6-second Vine. It’s no wonder that Pew Research observed a significant drop in response rates over the past 20 years. Surveys with low response rates make it hard to fully represent audiences.
On the other hand, even the best research can’t get past the fact that people are terrible at predicting how they will feel about or want something in the future. In 2009, Walmart asked its customers, “Would you like the aisles to be less cluttered?” to better compete with the Target shopping experience. People responded yes, aisles were de-cluttered, and customer satisfaction soared—until they stopped shopping. Walmart experienced the longest slide in domestic same-store sales in its history. Unfortunately, it missed an important ethnographic note for a discount chain like Walmart: more clutter evokes more value.
So, what’s our point of view on research? How do we make smart, insightful decisions on behalf of our clients, based on a deep understanding of their audiences, without traditional market research? Here at Blue State Digital, we don’t respond to what people say they will do; we focus on what people actually do. Here’s a peek behind the curtain to our faster, better, stronger research process:
Understanding where we are today
We begin any research process with a thorough analysis of existing users and their actions. We gather data using a variety of sources, such as tracking systems like Quantcast to gathering demographics and interests of an email list through Facebook Audience Insights to heat mapping with Crazy Egg to complex goal or event measurement with Google Analytics. Tracking and analyzing all existing digital properties is imperative to understanding our audience behavior and optimizing website design, email copy, and user experience flow.
Talking to our users
Analytics helps us to understand aspects of our current audience demographics, usage, and platforms far better than through a survey, but it cannot reveal their true motivations, desires, or feelings. We do not learn why they behave the ways they do. As the Walmart declutter debacle illustrates, people are poor predictors of what they want, so we cannot ask outright what digital features they would use or to what messages they would respond. Instead, we try to figure out how they’re actually using digital tools and what’s standing in the way of completing an action. To get at the “why,” we enhance our behavioral data by directly speaking with members of our target audience via one-on-one or small group interviews and observing the journey our audiences take with a particular digital experience in their own unique environment. In this way, we can surface new ideas and features, identify points of confusion or delight, and guide hypotheses for future testing.
Instant in-market testing
Test. Test. Test. Every piece of creative that Blue State Digital crafts is mercilessly tested, in market rather than a fabricated setting, to make sure it drives whatever conversion our strategy recommends. We release multiple proposed campaign messages simultaneously (and cheaply) and seamlessly adjust designs (such as web layouts or email subject lines) to determine whether our strategy is working before we’ve depleted our media dollars and time. We don’t stop when we find a winner; a campaign is continuously optimized and updated as we track and analyze new data.
Tracking success
For some clients, it’s very easy to measure success, such as increased donations, a bigger email list, or lower cost-per-click advertising. What happens if they want to track how our efforts affect something more intangible, like how people perceive their brand? Machine learning algorithms (think: Spotify deciding how to build playlists) provide insights into how people feel about a particular topic, product, or brand similar to a brand tracking survey, but faster and often at a lower cost. The most exciting part of tracking success through machine learning is its potential for merging the qualitative aspects of user observation with the quantitative tracking of a data point over time. Internally, we are experimenting with metrics such as a “Sentiment Score” that attaches a measurable number to an ethereal feeling. By posting to social media or writing a blog entry, users demonstrate they feel something—either positive or negative—through the energy required to take that act, and that something is what we can measure.
A word of caution: machine learning algorithms are still in their infancy for use as rigorous research tools. While they help us to manage more information than previously possible, we’re just beginning to understand the best ways to teach computers how to categorize complex messages. Microsoft recently released a tool that tried to analyze age and gender based on an uploaded photo. Below is a photo of my wife and me. Don’t worry—she’s actually six months older than me. Clearly, the machines still have much to learn.
At the end of the day, a digital strategy is only as good as its research foundation. Through new data-driven and ethnographic tactics, we can build plans that change the way organizations interact with people online and more quickly adapt to the needs (both present and future) of our audiences.