Product teams can benefit tremendously from both experimentation and research, but it can be difficult to understand when to apply each approach. This session will run through a framework for deciding when to experiment and when to do research and help people understand how to apply it to their own work.
Potato, Potato: Building Connection Between Product and Design
Jane Davis: Head of UX Research and Content Design – Zapier
Keywords: tools vs goals, metrics, process, reframing, misalignment between product and design.
TL;DR: Jane has applied her proficiency in pattern recognition to identify a key misalignment between product and design – teams focusing on tools rather than goals, and often conflating the two. This can be solved through using a process of clearly delineating goals by first defining the metric you are trying to effect, then identifying the behaviours that drive it and reframing the discussion around what the user is trying to do. She walks through this process with a number of examples identifying where and how misalignments have potential to happen and how to reframe your conversations to resolve and avoid these.
Jane is giving this talk coz we’re all slowly killing her! [gif of Madeline Kahn from Blazing Saddles: Goddamit I’m exhausted]
Why? Not because you’re addicted to Gantt charts or design sprints, although you definitely are (and that’s fine, it’s ok to have hobbies ;-), but because for years Jane has watched product teams full of smart talented people spending weeks and even months failing to solve the problem they’re working on.
We can do better! Jane’s here to discuss how. She has spent a good chunk of the past ten years helping to align product and design. She’s pretty hot stuff! But it’s not her particular magical touch that’s the key – anybody can do this.
Ergo: Why is she, a researcher, playing this role in the first place? And how can she teach others to do it so she can focus on parts of her job that do require her particular magic.
Goal of this presentation: Walk us through the process of how to identify and resolve the misalignments between product and design, and give us a process for avoiding them.
Again, why is this talk being given by a researcher? What specifically about being a researcher has enabled her to align these functions?
After rejecting some initial hypotheses…
- maybe I AM magic?
- Everyone = bad at everything??
- Product:Design – natural enemies? (like bears: sharks)
She took a step back to treat how she would treat any other research problem because that is one thing she does specialize in: Pattern Recognition.
What patterns repeat? (In design/product misalignment)
- Vague, unspecified agendas – teams weren’t explicitly stating what decisions they were trying to make
- Having the same conversation over and over, reaching an assumed agreement but starting over again the next time they met
- Using specific solutions to stand in for the general idea or larger goal they were trying to convey
This is like saying We need to clean out the fridge when actually the whole kitchen smells bad. This distinction matters.
Cleaning out the fridge is ONE solution to the problem (=the kitchen smells). It’s the most logical solution, (as opposed to Let’s buy a new house or Let’s throw out the fridge, which may not be as logical but nonetheless are solutions, otherwise known as TOOLS.
Teams often discuss tools for solving the problem, rather than the actual problem. So the first two problems (vague agendas and repetitive conversations) are actually higher order functions of the third problem (using solutions to stand in for goals). This is the key. In almost all cases, people were discussing tools not goals.
Let’s move from the abstract to a concrete example: Jane was looking for a drill at home to put up the numbers on the front of her house. The drill = the tool. But the house numbers are also a tool. Putting up house numbers is not the end goal – house numbers are a tool. The goal is to enable people to find the house. But because of the way she framed the goal originally: I was looking for a drill to put up house numbers, it’s easy to assume that the goal was to put up the house numbers. This does not at first seem like a big problem.
Isn’t the important thing that she got the house numbers up? No! Let’s translate this to a product team’s work. We’ll use a growth example:
Let’s imagine your team are trying to get more users to activate within their first 30 days. Analytics show that those who watch the 4 min intro video all the way through activate at twice the rate of other users. So you create a flow designed to get more people to watch the video and to stick with it, then wait for the numbers to correlate. Except they don’t. In fact, activation goes down. Team starts arguing about what changes need to be made to the experience. Weeks are spent trying to figure out how to get folks to watch the video, but nothing works. Why?
Because the video is not what is driving the activation rate. If you dig deeper, those who activate are all people with the same use case, which is the same use case the video walks them through.
Watching the video (tool) has gotten conflated with activation (goal). So we wind up arguing about a specific tool instead of the goal that tool is enabling.
In the team’s mind, activation has been conflated with watching the video, rendering them unable to have the right conversation about it. Instead, they need to be talking about what causes users to activate in the first place, and how to drive that. But because they can’t have the right conversation they will simply waste time arguing over opinions rather than the right outcome which is the activation itself.
Applying this to the house numbers situation, the team is arguing over what colour the drill should be, instead of how to enable the delivery person to drop off your damn pizza!
The key to doing great work as a team is to have the right conversations. You need to be talking about something that’s actually a goal, not just a higher-order tool.
Distinguishing between tools and goals: If we were meeting in person, this is the part where Jane would get us all to participate in some token exercise then say whatever she had planned to say regardless of how that exercise went. But since we are virtual, let’s take ten seconds of awkward silence and imagine we’re doing it and then we can appreciate the advantages of a remote conference.
[Jane stares at the camera for 10 seconds]
Back to the talk! Given that higher order tools (like the house numbers or the video) aren’t always obvious as tools, how do we identify them?
By anchoring on the outcome we are trying to drive. (The real outcome!)
- Have you considered metrics?
- Let’s try metrics!
What clear, measurable outcome are you trying to effect? If you can’t answer this, you will struggle. No shared measurable outcome = no shared goal. Once you have a shared measurable outcome, you have your anchoring starting point.
Defining a metric = a shared outcome = a goal.
Metrics alone are not enough. You need to translate your outcome/goal into your users goal. Three step process: Define your metric; identify the behaviours that drive it; reframe the discussion around what the user is trying to do. This reframing is important – no user is going to say I’m trying to activate! while giving feedback on an on-boarding experience.
On the product side, we’re using activation as a proxy for allowing the user to do what they are trying to do with our product. ‘Activation’ can be a great shorthand, but can disconnect from what users are trying to do.
Users don’t want to activate, they want to use your product to do something specific. Activation is how we measure the organization’s work, but reframing it from the user perspective can get you out of a deadlock by reminding you what you are trying to do, namely: to get your users to engage in a specific behaviour or accomplish something specific.
Translate your metrics into what your users are actually trying to do.
Once you’ve re-routed yourself in this shared metric, now ask why a given idea might or might not help them achieve that goal. Look at the reasons underlying your users’ behaviour, not the behaviour itself. [Jane has added an annoying keynote animation so that you will remember this point!]
This step looks like asking why instead of what. The question wasn’t: What are users who are more likely to activate doing? (Because the answer to that – watching a video – is wrong.) The question is: Why does (watching the video) help a user successfully start using our product? This process is about figuring out how each tool contributes to our ultimate goal (activation, which is proxy for user success.)
Walking through the process: Why are users who watch this video more likely to activate? Possible answer: They got information here that they needed. Yes, but that information is also just a tool, so… Why does this info make activation more likely? Possible answer: It tells them how to do something specific with the product. But knowing how is not the specific goal, they want to DO the specific thing. Why does doing something specific with the product make them more likely to activate Possible answer: Because it’s showing them exactly how to do what they came here to do. Now you know what drives activation, you can talk about different viable tools.
Handy recap slide!
- Start the discussion by agreeing on the goal you’re trying to get to, as defined by your metrics.
- Define what that goal looks like from your users’ perspective.
- Go beyond what users do – focus on why they do it and how it helps them achieve their goal.
There you have the secret magical way you can stop driving your friendly neighbourhood researcher around the bend! Go forth, and set goals not tools!