Product designers have power. We come up with ideas, and ship them to the world. And sometimes, those ideas take on a life of their own…. To severely unintended consequences. Twitter was designed as a micro-messaging platform, and ended up influencing world politics. Freaked out yet? Maybe we should be.
In this talk, Nicola shares one tool we can use to get a super-scaled perspective on our designs and the potential impacts they could have on the world, and what we can do with this information once we have it. She will also talk through one example of how this looks in the real world.
Who’s homeless because of what I designed?
Nicola Rushton: Product Designer
Keywords: Idealist, critic, pragmatist, scale, risk mitigation, execution, social impact.
TL;DR:Nicola is inspired by Walt Disney’s method of including an idealist, a pragmatist, and a critic on every project. She walks us through a number of examples demonstrating how a simple thought experiment performed at the beginning of our design process can offer a useful framework for identifying potential risks and unintended consequences down the line, particularly for creatives who tend toward idealism. By putting our critic’s hat on and thinking through how even small or seemingly innocuous products or features might perform differently depending on scale, we can better identify what to timeline and prioritize from the outset in terms of execution. Taking this approach helps us to think through bigger questions around what social impacts our designs may end up having.
Nicola is really happy to be here today. She’d like to share a small framework to help ask big questions about design. It’s said that Walt Disney relied on three leaders to lead every project: The Idealist – who was there to dream big, bring imagination, believe in magic; The Pragmatist – to look at the realities, the constraints, and bring back the dream a little closer to the real world; The Critic – who was there to take a more skeptical perspective and ask the larger why of the project and all the what if outcomes – safety and risk assessment.
As a designer, Nicola tends toward the idealist way of thinking. But idealism is something that can be really helpful and feels fairly natural as both a perspective and a role for creatives. But Disney’s approach is a good reminder that it can also be helpful to put on the other two hats at times, considering we are creating technology that goes out into the world
If the realist sees the obstacles in the path and the idealist sees the beauty in the distance, both would be well served to travel together. – Simon Sinek
Nicola would like to share one simple tool we can add to our toolbox as people who are building technology, that comes a little from the critic point of view; something that Nicola has found helpful.
The tool is actually a question: A question we can ask ourselves, ask of our product ideas, of our new features or of our designs.
What would this look like if we scaled it to ULTRA SIZE? What she’s not talking about: What would a screen look like scaled to a billboard? What she is talking about: What this might look like if we scaled the audience, the user base, the impact, to ultra size?
Ex: Nicola has previously worked for a project with a property management startup (dealing with rentals, liaising between tenants/landlords/properties). At the time of the project, the agency had approx 1000 properties on the market. Any work done had an impact at that scale; 2000-3000 corresponding users between tenants/landlords/properties. Applying our question here: (How would this look if they scaled up to ultra size?) would mean to Nicola – How would this feature that I’m designing now for a scale of 2-3000 users look if all of the rental properties in Australia were managed in this startup’s portfolio? This would be a scale of approx 7 million. Leaving aside that this may not be a company goal or be within reason, the point is the thought experiment. Take the perspective of: What if we scaled this up to Uber size, to Twitter size, to Telstra size, to Facebook size?What might the impact look like then?
Taking this perspective on a new idea or feature that you’re looking at begins to make risks and abuse cases and unintended consequences really start to come into view.
Ex: Let’s say we’re looking at a project around the way that people apply for rental properties. Nicola has gone through this process as a user many times, and her experiences range from really bad to just ok to slightly better than just ok. Formats have ranged from broken, hard to navigate online forms to physical paper you have to take to the realtor whose office might not even be in the same suburb as the place you’re applying for, etc. Finding a home that you might love and want to know if you can live in is a really important process and it can be highly stressful for potential tenants.
So as designers and product people, we see this, a not so great user/product experience, and we want to make it better. How might we improve on this? How might we make it easier on tenants to apply for homes?
Let’s say we brainstorm and come up with the idea of a digital form that can be pre-filled with information pulled from a user’s already existing social media account. This is convenient for the user but if you mediate it through a site like will also let the landlord know whether you’re employed, and thus likely to be able to pay rent regularly and reliably. So this idea works for our key personas – let’s mock it up and do some user research. We show it to some users based in Sydney, get decent feedback, and decide to push it out. This would be a fairly normal process for a small, simple feature like this.
OR: Before we proceed, let’s apply our question: What might this look like if we blew it up to ultrasize? The audience, impact, user base? So we’ve gone from 1000 users to 7 million users, all applying for their rental properties through this fill-form with LinkedIn buttons.
This is where some of the risks and potential unintended consequences start to become apparent. Risk #1: Start with the obvious: Not everybody has a LinkedIn account. Ex:tradies, labourers, carers, medicine – these professions don’t require networking. This means there is a large segments of potential tenants who would be excluded from the application process via this format.
Even if we have an alternative route for the non-LinkedIn people, if the LinkedIn method is the primary one, and takes, say, three minutes per application but the alternative route takes twice that, at a scale of 7 million we just made it appreciably easier for the LinkedIn people to apply. This will skew us toward more white-collar, professional roles, so have we just made it easier for them to get housing over others? It’s already likely easier to get approved for rentals if you have a fancier sounding job title and higher salary. We now made it even more easy, and simultaneously added a barrier for those not in that subset toward getting a home.
Suppose our startup scaled and grew, (which is the goal of most businesses) – Would we look back at some point in the future and see how our seemingly minor design decisions had widened the disparity between socioeconomic groups across the country?
Risk #2: When we filled out the application using the LinkedIn profile, what data was sent through to the owner or property manager? Did we send their photograph? Did we send a link to their company profile? Did we say that the application was filled out using LinkedIn? Or: What would a world look like where landlords made assessments on who their tenants would be in part based on a picture of their face? And if so, how might that go wrong? How many landlords across the country might be prejudiced against something they might see in a photo? Particularly prejudices that tech workers in a diverse, large city may not have thought about? Or in the worst case scenario: Could it pose a risk for the tenant if the landlord selects based on a photograph, and this person has a key to your house…. See where we’re headed?
When you scale that impact up, you start to see that there are some really scary implications of these small design choices that will not occur to us when thinking at the smaller scale.
Anybody who has worked on software products, particularly larger products, has probably seen examples of how seemingly small decisions that occurred earlier or when scale was smaller and more manageable can become difficult or almost impossible to change down the line.
So what do we do with this tool?What’s the application? A: In the execution. it’s really about how we design and build that feature or idea. Not if, but how.
So in the LinkedIn example, it doesn’t mean we don’t allow that function, but it’s going to impact the execution of that feature. Firstly, let’s be sure the landlord doesn’t see the applicant’s photo. Then we ensure we don’t add a link to the LinkedIn profile because we don’t want them to get access to the picture or to the company profile. In fact, we’ll remove any identifier that lets the client know that the applicant applied through LinkedIn at all. Let the landlord weigh all applicants equally without offering any additional data points.
Even though we are back to our initial 1000 sample size, it’s still important at a small scale that we don’t build businesses that have a negative impact socially on the world around us.
What about people that don’t use LinkedIn at all? We need to let them apply in a way that feels just as easy and fast. We could add facebook. But applying our question again, at a scale of 7 million, many may not have facebook either. So we can add a third manual sign in option. What about people who don’t want to fill out a form online? And so on.
The question now becomes: What do we as a business timeline? And: What do we prioritze?
Do we spend time making sure the manual form is easy to fill out simply and quickly? Do we build into our process that paper forms are available on-site at open houses? Even if only a few people take them, do we still take time to make sure it’s possible?
At 1000 properties, it can feel less important to focus on these kinds of questions, but if there’s a risk that only one or two people might be locked out of a home because of a decision Nicola made as a designer, she’s not ok with that.
Asking this question and taking this perspective also impacts metrics, specifically what other metrics that we’re going to measure. Are we watching the time differences between users signing in via social profiles vs manually? Are we looking at the metrics of which of those two groups are successful in getting approved most often (and: do we care, as a business?) What do we do with that information? When we learn that LinkedIn users are 11% more likely to be approved for a lease, do we say: Great, we want more of that, let’s advertise on LinkedIn! Or, do we say: Let’s just look at our manual form and ensure that we’re not adding to that disadvantage through our user interfaces and business processes.
Not necessarily about doing this thought experiment on your future idea and looking at future examples and implications at ultra scale and seeing potential disaster and throwing the whole idea away, but about seeing those implications and making small tweaks in the execution today. Look at how we do things when it’s small, and manageable, and keep a sharp eye out for those potential consequences.
Remember to put our critic hat on and think it through without being clouded by idealism. The reality for a lot of businesses is that they do aim to scale. This is just one small example but was deliberately chosen to show how even one small, simple button scaled out can change things.
This comes down to our responsibilities as people working in technology. We are seeing today some really intense and scary unintended consequences across the world from tech companies. Ex: Facebook and Cambridge Analytica and the influence that may have had on global politics – antivax posts on fb promoted by the algorithm due to likes which end up causing measles outbreaks, Youtube algorithm potentially radicalizing people by recommending certain types of videos, the rise of the gig economy and the impact that’s had on people and society.
What’s really scary to Nicola is that these impacts on the world were accidental. Those features on those products across all those companies were made by people just like us. We know how they get built. Decisions are made in meetings just like ours, those features were scoped out in Jira, maybe they did analytics or user research.
At what point does: How might this impact the world? come into the process? Nicola thinks those companies didn’t ask. Or if they did, they asked too little too late. They all did scale up, and we’re now seeing those implications at scale.
This is why each of us needs to look at our processes, look at the companies we work within and ask: Where does the social impact we may have on the world come into our process? Thinking about this can feel overwhelming, but we are the tech industry and it’s our responsibility to try. Try to not contribute something like that to the world.
Nicola doesn’t have the answer, and there likely isn’t one single answer – it will be different for every company. But it’s really important that we’re thinking and talking about it and asking: How could this go wrong? And: What impact could it have? You all have your own processes, ways of working, and teams, and scales of impact. Nicola would love to hear if you talk about this, and if so, how? Nicola would love for this to be a conversation that we continue into our careers and into the future and hopes to have a chat in the forums that the Web Directions team have set up for us! @nicolarushton