As a UX researcher, I get questions from time to time about the art, science and practice of user research from people who do not fully grasp its fundamentals. And that’s understandable, because as researchers, we can’t expect everyone to be familiar with the inner workings of what we do and how we do it. I think of clients and stakeholders in particular who want to know the how and why of research design, sampling, recruiting, drawing conclusions, etc., in order to create a business case for spending a bunch of money on research, impact decision making, or maintain credibility within their organizations.
A few examples of the kinds of questions I’m referring to are: “What can you really hope to learn from talking with just 8 people?” and “Can you find out how much people will pay for this thing?” and “Won’t people change their behavior in the presence of others?” and “Why should I do ethnography? It costs too much money and takes too much time.” The list goes on.
Seasoned researchers have lots of experience with fielding such questions and explaining why research of all stripes is worth the investment. Three years into my career, I still grapple with these kinds of questions and hope to get better at answering them. I welcome moments for practice with a tiny bit of reservation and anxiety because I know they will help me improve, but also because they can be tough. I want to say the right thing, but how do I respond in a way that is informative, culturally competent/resonant, and validating of my work and existence as a researcher?
Questions: What They Tell You & How to Answer Them
I like to think of such questions as mini quizzes to see if I’m really being thoughtful about what I do for a living, the overall value of my work to others, and my ability to interpret what someone is asking for. For example, what is someone really conveying when he asks me why I’m suggesting one-on-one interviews when a survey could do just fine for answering a particular research question? With this or any other question, I feel successful if my response is solid and well-informed. I feel triumphant if I’m able to teach someone something new or shift someone’s thinking, even if just a bit.
It helps me to think of these questions as clear illustrators of culture – what people value, how they see the world, and the rules that govern everyday life. This is especially important in business. In the above examples, I can detect values like numbers, statistical significance, prediction, money, time, and reputation. It’s a rookie move to assume a client has the same understanding of research as I do; the easy way out is to brush it off as “typical client ignorance” and wonder, why don’t they get it?! It’s more difficult to take a brief pause and remind myself that not everyone thinks the way I do (you’d think that anthropologists would do this really well all the time, but it’s not true.) When it comes down to it, the same lens I apply in user research, when I’m learning from others about how they do and think about things, should be applied to pretty much any other interaction I have with anyone else, including those with clients.
Social interaction is largely about understanding and negotiating values. If you’re thoughtful about it, it’s also about uprooting assumptions and empathizing. If the first step to answering a question like the ones above is to understand/empathize with the point of view of the person asking, then the second is to craft a response that does two things:
1) It should provide information/answer the question (duh).
2) It should acknowledge the underlying values that are at the root of the question.
The goal isn’t just to answer a question willy nilly, but to understand about what the person asking it really cares about (their values and worldview). Then, you have a choice. Either you match your response with the values you identify and validate their worldview, or ask the person to consider an alternative (a new worldview with new values.)
Considering “Truth” and “Validity” When Selling the Value of Ethnography
Let’s venture down the challenging path of asking clients to consider alternatives. In a recent interview, business anthropologist and culture consultant Grant McCracken gives a good, succinct response to the question: “How do you convince your client of the value of research – ethnography, in particular?” Here’s his take:
“The most general proposition on behalf of ethnography is simply that, whether we like it or not, whether we know it or not, we end up the captives of our own corporate culture. I did a lot of work for Coca-Cola some years ago and would come back to Toronto after a couple of weeks there, and Coke would feel like the inevitable choice. I would see a kid ordering a Pepsi, and it would seem like an impossible choice for me. My world was now “Coca-Cola World.”
Ethnography works like a helicopter. It lifts you up out of your own sense of the world…and drops you into the world of the consumer, so that you can see them and yourself as they do.”
McCracken’s point is this: to really understand the people who are most affected by a product, service, decision, or policy, we need to talk with them, and not just in focus groups or through surveys (two of the most common forms of so-called “talking with”), but in the contexts of their real lives. While his answer is likely just a small slice of what he might actually say to a real client in a real conversation, McCracken is laying the foundation of a way of explaining the value of ethnography in terms of the client’s underlying goal, which is to get a “truthful” sense of the needs, preferences, and perspectives of customers or users.
I imagine that in real life, McCracken’s response would be a lot more complicated than the one he gave in this interview. Understanding a client’s goals is one thing; getting at their definition of “doing it the right way” is another. Depending on the client, he might spend a considerable amount of time identifying values and adapting his response accordingly. The above question about the benefits of ethnography is a great example of responding to these kinds of inquiries because it is a very different approach to understanding the world than many clients are used to (think focus groups, surveys and usability testing.)
If ethnography can lead to a new way of understanding, it therefore must also lead to a new definition of “truth” and validity. Asking someone to reconsider what is “true” and “valid” is asking for a lot – it essentially means a change in culture. It’s no wonder we (user researchers, ethnographers, anthropologists) have such a hard time convincing people of the value of ethnographic methods and why there is so much conversation about it in the practitioner community (e.g., here, here, and here.)
In her book, “Practical Ethnography: A Guide to Doing Ethnography in the Private Sector“, Sam Ladner (Senior User Researcher at Microsoft) points out something that researchers regularly face in managing clients, which is that many clients adopt a positivist worldview in which the validity of research is based on its ability to quantify and predict the future (e.g., how many people will buy this widget? how many people can we get to switch to our service?) On the other hand, many user researchers, especially those who come from the social sciences, employ an interpretivist stance in their work, which seeks understanding and explanation over quantification. Ladner writes:
“Positivists rule in the private sector. Interpretivists are few in number and small in influence. Most marketers, business strategists, and product managers don’t understand the interpretivist point of view, not because it is incorrect, but because it has an unfamiliar conception of truth” (24).
The interpretivist lens of ethnography focuses on things like meaning, social context, symbolism, identity and social interaction, data points that differ from (but can compliment) numbers and “hard facts” like conversion rates, income, appeal ratings, and frequency of use. Ladner argues that the path to success for ethnography in business relies on the researcher’s ability to understand the culture (norms, values, etc.) of an organization, how it differs from the fundamentals of one’s own culture (or that of a discipline or epistemology), and adapt a response that clients will ultimately find acceptable (whether or not this involves a shift in though on their behalf.) In other words, a successful researcher
“…sees and understands the gap between her interpretivist method (in this case, ethnography) and the positivist bent of her clients and stakeholders…Her job is to understand the positivst standpoint, empathize with it, and make her standpoint more intelligible to her clients” (43).
Understanding Values to Sell Value
If I were faced with the task of selling the value of ethnography, there are lots of points I could make. I could emphasize the depth of understanding to be gained from talking with 8 or 12 or 24 people rather than 1,000. I could explain the difference between going in with assumptions and testing a hypothesis (deductive) versus exploratory, inductive approaches in which findings are based on what you learn in the field. I could tout context and the benefit of getting outside of the research facility and into people’s homes, offices, vehicles, social groups, shopping places, etc., for a realistic sense of what people are actually doing. I might even take McCracken’s metaphor in a slightly different direction by saying it’s less like a helicopter and more like sky diving, in the sense that you get lifted up and out of your own way of thinking and being, and jump into a new world with some basic tools, an open mind, and a relaxed conviction.
While this is all well and good, for a more successful conversation, I would need to start with empathy and speak to how ethnography would fit into the culture of the client’s organization by, as Ladner puts it, “speaking the language of positivism.” I would need to bring McCracken’s response to the next level, from storytelling (as useful and wonderful as it is) to what will ultimately resonate with the client’s worldview; i.e., how my research would lead to useful information for business strategy, product design, branding, etc.
Ladner provides some great ideas for thinking about ways to do this in her chapter on managing ethnographic projects. For example, if the client values quantification, consider talking not just about the number of participants you recruit for a project, but every relevant data point (including participants, locations, photos, hours of observation, minutes of video, etc.) If they value time, propose a project plan that emphasizes key milestones instead of all of the nitty gritty details of what’s actually involved (e.g., if it helps, don’t bother mentioning the literature review, transcriptions, debriefing, analysis, etc.) Communicate in terms of structure by using a planned project outline, with the caveat that ethnography is inherently unstructured and often unpredictable. When it comes to reports, deliver results that are succinct and not overly lengthy.
I would add a couple more to this list. If significance and validity are important (as they typically are), it should help to talk about how such requisites are met differently through qualitative research. And thinking of “actionable insights” as one of the ultimate values in business, decision-oriented results should be the focus of any report or presentation; information the researcher finds theoretically interesting but isn’t really relevant to the client should be left out or put in an appendix.
The best thing about such an approach is that it involves researchers and clients meeting in the middle, making compromises based on what both sides value with a mutual goal in mind. I recognize that this is an ideal scenario because it means that both sides must be willing to listen, empathize (understand), acknowledge where each other is coming from, and possibly even adopt a new way of thinking to achieve the best result. It’s essentially the marrying of two very different worlds – business and research – the latter often originating from academic contexts driven by drastically different values, priorities, notions of time, etc.
If researchers want to do ethnography in a business context (or another non-academic context for that matter), then compromises will have to be made. They will need to sky dive into the client’s culture to see things the way the client sees them, armed with the tools and open perspective they are so good at employing in user research. Likewise, if businesses desire to conduct and benefit from ethnographic research, they too will have to stray from the norm. Thinking of McCracken’s Coke story, they need to step away from the captivity of their organizational culture where traditional research methods are the norm and consider other seemingly “impossible” choices.
Image credit: “Conversation” by Valery Kenski