Putting Research in its Place

These days we are spending less time fighting about whether or not our companies need to do user research, but we’re still far from settled on who should be doing the research and where they should be in the organisation. Should designers and product managers do all the research themselves or do you need a specialist team? Should research be embedded in product teams, or would a centralised team work better. And what is the right ratio of researchers to designers and developers? Well, it depends… just kidding. This talk is full of strong opinions.

Leisa Reichelt – Putting Research In Its Place

Leisa starts by talking about a potential tattoo… but she just can’t have the typography of the design. How do you pick a typeface to go on your body?!

Perspective is the user researcher’s super power.

To understand perspective we need to understand truth.

Let’s go back in time… to when the internet looked like NCSA Mosaic and people made things in Hypercard.

When Leisa went to uni they were heavily into postmodernism, it permeated all the subjects.

Postmodernism believes God has died and as a result we now have to construct our own truths. We could agree on some facts back in those pre-Trump days, eg. we could define a general definition of the colour red. Even if our precise idea of red differed slightly.

Trick photo - bird's shadow looks like a shark, but it's really just a duck!</a>

There are no facts, only interpretations. – Nietzche

“I think Nietsche would be absolutely loving politics right now…”

Other people have a view of the world that is very true to them, and how we feel about that is very important to be able to get along together.

William Perry’s schema of cognitive development:

  1. Dualism – Knowledge is received and not questioned. Students feel there is a “correct” answer to be learned.
  2. Multiplicity – There may be more than one solution to a problem, or no solution. Students recognise their opinion matters.
  3. Relativism – Knowledge is seen as contextual. Students evaluate viewpoints based on source and evidence, and even experts are subject to scrutiny.
  4. Commitment – Integration of knowledge from other sources with personal experience reflection, make a commitment to values that matter and learn to take responsibility for committed beliefs. Recognition that acquisition of knowledge is an ongoing activity.

So where do you fall on this scale? Where would you kids be? Where is your company on this scale? …and why should we care? How does this relate to user research?

It’s part of framing, how we define the problem we set out to solve. Don’t fall in love with a solution… but how do we fall in love with the right problem? How do we konw we are focusing on the real thing and not the shadow of the thing.

Example problem: what to do about the Medicare enrolment form, required for people such as newly-arrived immigrants or newborn children. Well the obvious thing is to digitise the form! But at the DTA they had to work to the DTA standard, meaning they had to ‘know all about the user’. Goverment forms have lots of complexity; and the people using them are also complex. They have a huge range of backgrounds and contexts.

If you need reminding that your customers/consumers/users are people you have bigger problems. … User is a good word because it clearly indicates what the relationship is all about. Our primary responsibility is to make something that someone can use. It’s about utility. – Russel Davies, Consumers, users, people, mammals

The best solution was not to digitise the form. When a baby is born, the hospital already collects all the information required to enrol in Medicare. So the better solution was to simply have it done automatically before they went home.

However the initial framing had been biased towards immigrants, not the more common case of someone having a baby. The framing was wrong, so the solution was wrong.

The answers you get depend on the questions you ask. If you ask people about the shadow, they’ll tell you about the shadow.

A classic case for Atlassian: “what is the thing you do most often in JIRA?” The problem is the assumption that JIRA is the centre of the user’s world. It’s not really how people think about it.

You could instead ask: “what are you doing most often when you come to use JIRA?” This puts the focus onto the user and what they are trying to do.

No research is neutral. No analysis is unbiased.

It is a complete fallacy that you can do research without a bias. You can just try to have as much awareness of that bias as possible; and which truth you choose to privilege. Know which duck you are dealing with. Where you are in the organisation impacts the questions you can ask.

Six models for situating user research:

  1. non existent (we need to be realistic that this happens)
  2. sporadically outsourced
  3. ad hoc, not specialised
  4. internal consultancy
  5. embedded in product teams
  6. centralised strategic

(Natalie Hanson’s stages of UX maturity)

There are really only two models you want to aspire to: embedded researchers, or centralised strategic.

So which one is right for you? Naturally to decide you make a 2×2 because, well, that’s what you do. This one is Transaction/System vs Optimisation/Innovation.

Applied examples:

  • UK Govt Digital Services – embedded approach worked really well, everyone was convinced. It was heavily driven by transactions that were quite difficult, but the overall shape of the transaction was quite linear.
  • Australia’s DTA – initially the work was very transactional, so they started with an embedded model. But over time they started to look at ‘super services’ that moved across multiple government systems, which is a very messy transaction. So they shifted to having a centralised team as well, to work on the cross-cutting issues.
  • Atlassian – started off with an affinity sort and decided to focus on impact and capability. For a large team Atlassian had very few researchers. So pooled into a central team.

Atlassian’s approach:

  • User centred
    • For Atlassian an example focus area is How do teams make decisions and prioritise work? This is a user-centred approach, not a product-centred approach.
  • Continuous research
    • They talk about hunches, a feeling or guess based on intuition rather than fact.
    • Over the insight lifecycle they take a hunch and research it, until there’s more certainty.
  • Mixed methods (quant/qual)
  • Decentralised product research

Just about every designer reacted that they agreed with this academically, but in reality it did make their life a lot harder. So they needed to help people a lot, as they grew their ability to do research. They also had a lot of challenges with user recruitment for testing.

The truth will set you free, but first it will piss you off. – Gloria Steinem

…and that might be the next tattoo.