What Can’t AI See? Ethnographic Research in a Post-Screen World

Okay. Good morning everyone. It's an interesting time in UX design research. Things are moving so fast and I'm going to I'm going to guess that in the room today many of you have either been affected by some of the restructures, impacted or know someone who's been impacted, or have had the difficult decision of having to go through your team members and see who's going to be on the list.

It's also a very exciting time. As Steve mentioned, we've got a lot of enablement happening with AI and other parts of technology that have made some of the processes that we do a bit faster. I'm doing design research for twenty years. I'm a director at Publicis Sapient, is part of Publicis Group, which is the largest marketing and communication company in the world.

So most of my time is either helping companies find the people that they need, so design researchers, researchers, engineers to do the thing that they know they need to do, or they might not know what to do and they come to us for assistance to help them understand and try to solve the problem. So the good news that there's three new opportunities for design research that private and public sector are looking for.

So I'm going to get to those opportunities a little bit later, But first, I want us to also think about where we're at at the moment. At the AI design conference earlier this year, Millie Schmidt from Atlassian described it as musical chairs. We're all kind of walking around and some of those chairs get taken out and we're scrambling for a seat.

And what that seat might say? It might say content designer, it might say content researcher, it might say UX researcher, it might say UX UI researcher, and we're trying to figure out where do we fit. So many of us are looking to extend our skills. We're thinking about how do I do what I know works and also make sure that I can future proof my job opportunities.

So how did we get here? Well, it started in a room. Computers were massive. They weren't things that you purchased. They were things that you went to. They were behind closed doors, and trained specialists were working with them. Then they got smaller, they moved into the home, moved into the office, and then suddenly, we went from this desk, this desktop, this computer, into something that non technical people are using all the time.

And to be able to have from a room to a desk to a pocket, somebody had to make that easier and intuitive. And that's us. That's design research. That's what we're doing. And so when we think about that transition, we've gone from specialists, highly trained, don't need training, don't need to be the the computer doesn't need to have all the descriptions and all of the intuitive built in and the patterns to now where everyone has it in their pocket and it's our job to make sure that it's easy.

But for all this time, we've been designing for the rectangle. We've taken all of this into pretty much a rectangle. We've tried to bring our environment into this space where people do their shopping, their social, their banking, and this is all changing. It's been a long road and we've looked at to try to get all of the world that we have into this rectangle.

We spent time where we've shadowed people, we followed them on their jobs, mechanics, engineers, electricians, doctors, people doing their shopping. We've thought about the user, who are they, what motivates them. We've considered the task, what is this thing that they need to do, and we've considered the context. Are they doing this on the train on their way to work, or they have a big screen in front of them?

We brought all of this research into, guidelines. The guidelines have informed design. The design has gone into design guides and style guides. The style guides have become design systems. The design systems are now built into components and the components live in the software. So all of this research and this this these insights, these personas, this understanding of what makes a great interview, our discussion guides, they can be pretty well found by LLMs.

LLMs, the the chatty g's of the world, the clogs, they're able to find our discussion guides and be able to use that. They're able to they're able to find, the way that we ask questions and what a good insight looks like and what a good quote looks like to be able to justify. Have we been so good at research that we've researched ourselves out of a job?

Let's have a look at where we're at. So we started with computers being adopted into the home, and then we have this huge uptick where we really needed UX researchers and design researchers to try to make things intuitive. We had the onset of Internet banking, then we had COVID where lots of companies were standing up delivery services and trying to make this e commerce a thing.

And we've had this shift now. We've had this drop off where we've had a lot of AI solutions in market where businesses are they're skipping research side. They're going straight from need to design and with Claude Coates and others cranking out designs sometimes that research step is being missed. But again, the good news is we are about to move into a new world. We're moving away from the rectangle and moving into spatial auditory experiences. It's estimated that Garner said that by 2030 over 40% of Internet browsing is going to be auditory.

Also, we've got the increase of of agents and agentic experiences, and a lot of the research that we're doing now is to help determine what can an agent automate, where do we need the human and where there's areas where it's either legislative or policy or inclusivity or there's cultural nuances where we absolutely must have a person doing that work. So there's a lot of areas that are increasing in the next ten to twenty years.

This is a project in The UK where they did a study working with people who have aphasia, so this is where it's difficult for people to remember the word at the time that they want to remember that word and difficulty sometimes with being able to understand language. There are some really positive results from this where the people were wearing these glasses with the AR overlays to help guide them in everyday tasks. And you can see here that there's a really positive feedback from people who now are able to go about their regular activity, pass as being someone just wearing some glasses, not special fitted device, and be able to have that extra assistance for them to do that recall at the moment in time with memory prompts and images to help them be able to translate and be independent in in their life.

Now I know what you're all thinking, what about all the perverts? And there was just an article this morning, around Brisbane about people wearing those glasses to go to the public pools demanding people. This frustrates me so much, and there's always a bad apple who's gonna destroy the barrel, whatever the phrase is.

But this is happening. There is a huge uptake in people using glasses, using kinesthetic and auditory experiences to try to make their lives easier, and the demand and the sales show that. So as we're thinking about how do we design for gestures and movements and body language and ways that people will be able to communicate yes, no, stop, go with their hands and with their body instead of just clicking with their fingers.

There's a lot of things we need to consider from a cultural perspective. So looking across cultures, there's many parts of the things that we do that vary. This in Japan means come here, in Western society often means go away. Nodding left to right in Western society often means no, and in Indian society means yes, agreeing.

So a lot of these little gestures have to be thought about if we're going to let people control the device and the digital experience with their body, their with their face, with the facial mapping, and with their the way that they hold their themselves. Okay.

So Erin Meyer wrote a book. So she interviewed thousands of people to try to understand the difference in different cultures and to think about what do we do when we're designing these digital experiences that need to be localized so that we make sure that we're talking to people in the right way. She also looked at time.

So thinking about some countries where it's a bit more flexible and more of a suggestion than an actual time and something we should be considering when we are trying to have moments where people are either waiting for a page to load or waiting for something to come in. Thinking about how long do we need for that to be. And in different local areas, they might be more accepting or they might be more frustrated.

So it's kind of a lot. If we suddenly have gone from designing for the rectangle and we feel like we've got it sorted and now we're being asked to design for people in their culture as they've been raised for an entirely different way of living and working in different parts of the world, who can we learn from?

So those three anthropologists that we'll touch on that have done some interesting field work that we can bring some of those methods, into our work. So Margaret Mead spent time in Samoa trying to understand different ways that gender roles were raised in society.

She also looked at how the nature nurture debate went and her belief was that how people are raised and the way they think about gender and the way they think about sexuality is not biology. It is how you're raised. But what she found by living there for years and years, it was difficult for her to be part of the society and also an observer. So she was one of the first anthropologists who took so much footage, photographs and video.

She took 25,000 photos and 20,000 feet of video that then she could look at afterwards to try to then decode some of those patterns and some of that behavior. It was difficult when she was in the room doing that. We've probably all had experience where you've been in a room and conducting an interview or shadowing or walking through an experience with someone and there's so many things going on.

It's hard for you to notice what are the things that are are really What are those patterns? What are those behavioral patterns? And not until you go into the video afterwards that you can make sense of it. The next one which I'm sure most of you are familiar with is Jane Goodall. So she was only 26 when she arrived to Tanzania in 1960 and when she first arrived, it was the first time anyone had done a project this length with chimpanzees and they were absolutely frightened of her.

It took her months and months of sitting quietly in the forest until she was able to get closer and observe them with their habits and their behaviors, things they did together and things that they did independently and after that time she was finally had her breakthrough, which is when she identified one of the chimpanzees grabbing a twig from a plant, stripping the leaves off, plunging it into a termite mound and to get the termites out to eat.

And that was the first time that anyone had documented that it's not just humans who make tools to use in their life. So that again, raised the question in science between what is human and what what are animals. And Sarah Pink, who's developed the area of ethnography and sensory ethnography, where she worked with people following them, shadowing them, being in their home with them while they did daily activities and one of the areas that she looked at was in laundry in Indonesia to try to understand these washing machines that had gone into households.

How are people actually using them? Just because they buy the washing machines doesn't mean that they're actually using them in the same way that we might. And she found that there's actually quite a lot of distrust for the washing machines and that there was a belief that the washing machines would destroy people's clothes especially things that have been screen printed.

So people were only using the washing machines for doing the spinning and the drawing and even the water. They were distrusting the water from the government and using well water to to wash the clothes because it had less chemicals. So some of these insights that don't come from just sitting in an interview in a in a room with someone and asking them about their activities, about actually shadowing while they're doing things, trying to prompt with questions and understand that behavior.

So what can we bring from these three anthropologists? So what we learned from from Margaret Mead is it's difficult to sometimes identifying patterns when you're in the environment. So we can use some of the AI computer vision to be able to look at patterns. From Jane, it was about once the the people or the subject starts ignoring us, that's when we get to understand what the real behavior is. And then from Sarah, going along with people as they're doing the activity as they do it to be able to prompt.

But I don't know about you all but I don't have two years to research a project. We often will have two weeks, two months, short time. Anthropologists, you know, would spend years working with people. So looking at AI and some of the enablements here, we can look at using some of the the decoding to be able to understand where patterns are happening.

But it's still up to the researcher to be able to undo that sense making To try to look at a pattern, we know it's happening, but we might not know why it's happening. So that helps us take all of this footage, the stuff that we wouldn't normally have time to do, to get down to the patterns, and then, you know, kind of I'm I'm doing an inverted double diamond. Let me do it this way.

This way down where we understand what we need to look at and then go wide again to be able to probe and to try to find out what are some of the things. I've taken a long time to get to those new opportunities. Okay. What are they? This is exciting stuff. Right? Okay. The first one that we're hearing interest from companies and government about is about the spatial UX researcher.

So this might be some of you who are really interested in in ethnography. You love that. You love traveling or you love being around different cultures. You love noticing the things that make us unique, that make us different. These skills are gonna be very in demand as we bring those into understand the nuance of culture. And what does this mean when we're thinking about designing haptics and gestures and ways for people to communicate what they want to do in the digital realm.

The next one is around agentic foresight researcher. Agents, am I right? Okay. Yes. There's things they can do, but we still need to have researchers figuring out and thinking about what alternative futures might exist when they are doing some of our work.

We've seen in the news where companies have been pretty full on and done slashing, cut lots of jobs, and then realized later, actually, we needed people to do some of that stuff. We've kind of messed up. So the agentic foresight researcher is there to assist businesses and government thinking about what are all of the front stage and back stage activities we do in this organization, where are the areas where things can be automated, and where do we need people specifically dedicated around legislative policy inclusivity, areas that we require a human in the loop.

And then what might this possible future look like? So we've been asked for researchers who can play out some of those new futures, can say this is what this might look like. And people who are very good at doing that those process maps, thinking about the different steps people go through, and people who have either maybe come from a BA background or services on background who are familiar with mapping out those different flows and the people that are involved.

And the last one is around change researchers. So again, we're having requests from government and from private businesses around people from a UX background and a UX researcher background, design background to be working with the change managers to help bring through those insights on the ground of what's going to happen for either tech adoption, for new ways of working, or for different types of roles that people have not yet started.

People who are in this role have normally come from either gone through a digital transformation or worked on a project that is around digital transformations. They know some of those common stages. They're often people who are very good at connecting and being able to understand, talking to a bunch of different stakeholders and understand this is what's in it for them.

This is what's in it for them. This is what's in it for them. And feed that back to the change plan. Okay. And the last one. So again, exciting stuff, new opportunities for UX and design research, and the different areas that sit within them, whether people have an area that they've really enjoyed and they wanna go deeper, extending those skills into the area of AR, VR, extending those skills into more of a business analysis. So more of a BA background and really enjoy that thinking about feedback loops and cybernetics around that, enjoying that logic and structure, or people who are more in the change space.

So being able to assist change managers and be able to assist some of those stakeholders through areas of complexity and change. Okay. It's an exciting time, and there's lots of new areas. Thank you very much.