The Human Margin

Before we get into it, I just wanna acknowledge the Gadigal of the Aurora nation, the traditional custodians of the land that we're meeting on today. Pay my respects to elders, past, present, and acknowledge their continuing connection to country. It really is a privilege to be speaking here today. Before I start, I think I need to give a little bit of a trigger warning.

Yes. This is another AI talk. I know I'm tired of talking about it too. The hope is though that you'll connect with something that I say today and be able to take back something really tangible to your teams. We're not just gonna talk about AI. We're gonna talk about human things. We're gonna talk about trust, accountability, and what it looks like to surrender agency to the systems that we design.

So I wanna start with a question, and it's a question inspired by Sam's session yesterday. When a design decision affects somebody, who is ultimately accountable? For the next half hour, we're gonna track the life cycle of a decision. From the conversations where it begins, to requirements, and through into the systems that carry it into somebody else's life.

Along the way, we'll explore how trust is earned, how responsibility quietly disappears, and the hidden consequences of allowing AI to carry decisions. By the end of our session, I hope you'll come away with a clearer answer to a question that many of us are being asked every week. Why do we need designers in the age of AI? Unfortunately, this question is becoming more and more important every day.

Australia's tech workforce shrank for the first time on record. This year alone, we've lost over 4,000 tech roles. Almost every single one came with the same explanation, artificial intelligence. And when you think about the roles organizations are prioritizing, there's a really concerning trend for designers.

Sometime around mid twenty twenty three, the number of open product manager roles overtook the number of design roles for the first time in history. Design hiring hasn't recovered. That gap has only widened since. And I don't just think it's another cyclical hiring trend. I actually think I have a bit more of a cynical take.

I think it's a signal in how design have articulated their value to businesses. When executives think of design, they usually think about process, UI, workshops, and brand, process and outputs. But design doesn't just exist to bring ideas to life.

Design makes the future inside a decision visible while it is still a choice and keeps somebody answerable for what follows. But trust needs somebody to stand behind it. Think about the last time you saw a doctor.

Somebody looked at your symptoms, your history, your results, and made a decision that affected your life. Interestingly, AI can do parts of that work really well. It can spot patterns, it can suggest a diagnosis, sometimes even as well as a doctor, but the scary thing is now sometimes even better. And still, most of us wouldn't just trust the results if they came from a system alone.

Quality is one thing, and when something matters, we're not just looking for an answer. We need somebody to be accountable for it. Because quality alone can't earn trust. A few years ago, I was brought into a project for an intelligence agency for a foreign government. Our team had done careful work, the kind of work that you lose sleep over because getting it wrong doesn't just mean a poor metric or a poor performance review.

Our consequences were measured in people and not pixels. We thought the work was good, more than good, actually. But then my phone rang at three in the morning. The feedback was simple. The agency didn't trust it. On screen, the product was clean, but that was part of the problem. The feedback was that what we had built had looked too well designed. It's the most backhanded compliment I've ever received, to be but it looked too polished and too composed.

It read like software being made by people sitting a safe distance away from the problem. So naturally, we made the interface worse. We made it rougher, more technical. We used a monofont. We moved parts of the process into the terminal so it looked a bit hackery. We used a lot of matrix screen. There may have been a fake spinner involved, and it worked, at least well enough to earn another conversation.

But what happened next showed us that we had critically misunderstood the problem. They needed the people behind the interface. They needed them to stand in the room. In the end, the UI had nothing to do with it. They flew us to the agency's headquarters and ran the system on their laptops. They needed to see a real person.

They needed to see the people standing behind the decisions to know who would be there if the system got something wrong. The agency needed a visible line between the thing on the screen and the people prepared to answer for what it might do. Only then did the work really become trustworthy.

No amount of design craft or polish can earn trust. Trust can only be earned by people. People who are willing to stand behind what followed. Because we don't trust quality, we trust accountability. Quality is what something produces. Accountability is who carries that consequence when it matters.

That was my first real experience for what I now call the human margin. I'm Pete Barber. I'm a product designer. Most of my career has been tied to payments, identity, and security. It brings me a lot of pride to say that everyone in this room has used something I've helped design. These systems are usually designed to disappear, but if you notice them, something's probably broken. So sorry.

Before joining Airwallex, I led product and design at an AI lab in The US that focused on national security, cybersecurity. We built systems to find zero day vulnerabilities. These are defined as security loopholes that nobody has found yet that are in the software that keeps our banks, our hospitals, and our critical infrastructure running. Our hypothesis was that these halls exist in almost every system that we rely on.

Turns out we were right. Design craft shows that somebody cared about the details. The hardest questions in those rooms were never about design craft. They were, how do we trust this result? What happens when we get it wrong? And if we do, who carries the weight of that responsibility?

A well designed interface can make a good product decision spark joy, but dangerously, it can also disguise bad ones. This is why quality and process cannot carry the whole argument for design's value. We oftentimes confuse activity with value. But we don't trust quality.

We trust accountability. We are very, very good at describing how we work, and extremely good at bringing decisions to life. Craft, empathy, research, workshops, frameworks, process. They matter, but they describe the things that we do.

Steve said it this morning, there's a shared perception that design is mostly decorative. Activities can't tell us whether the decision should have been made or answer for decisions that are already set in motion. They do not tell an organisation which decisions became better because a designer stood in the room. If our value is reduced to bringing decisions to life, we will only ever be invited in after they have been made.

Once the outcome is chosen and the target is fixed. Once the requirements and the brief are treated as facts. Once this happens, the only real question for design to really answer is, can you make this work? Usually, we can and we say yes, but that's really the root of the problem. Design responsibility has been atrophying for the last twenty years, and it happens in every day, in some small rooms and others in bigger rooms like this. Somebody says, we don't have the budget.

We can revisit that after launch. Is this really that important? We need to keep moving. I think we've all experienced that. So we find ways to make it work because that's what being a good design partner is. We make incoherent flows coherent, and we help teams get over the line. However, slowly but surely, these exceptions have become a pattern.

The organization remembers that design delivered, but design remembers the trade offs. I call this death by deference. Not one dramatic surrender, but a trail of reasonable compromises, each one small enough to defend and each one temporary enough to accept. This taught businesses a dangerous lesson.

We taught them that design bends. Bring design in late enough, and we will make a poor decision pop. We tried to fix this. We translated into process, making it easier to understand and earn a line on the profit and loss. We gave it stages, sprints, diamonds, journey maps, calendars, and post it notes. That materially helped.

Process gave us scope. Process gave us a budget and that elusive seat at the table. But completing a process, it's not the same as influencing a consequential decision. A team can run every workshop, complete every design stage, and still lack the trust or authority to ask whether the requirement should exist at all. The work moves forward because somebody else has promised it. The solution might change at the edges, but that original intent, that committed deliverable does not.

This naturally leads organizations to ask the question that process and activities alone can't justify. What is the value of design? Design makes consequences tangible early enough to not only ask, can you make this feature work?

But is this a future worth building? Frustratingly, we actually don't own this. That contribution is shared. Great design is collaborative by definition. Design doesn't own every decision, and we cannot predict the future. But being a designer gives us an increased responsibility for the impact that our products produce, and the work that we have impacts on the lives of our users.

Design, craft, or process alone can't be used to define our value because when it does, we struggle to answer a simple question. Why do businesses need more designers? Now with AI, decisions can travel much further. Systems can now carry judgment directly into the lives of your users.

It carries judgment into rooms that you'll never enter, into moments you'll never see, and into the lives of people you will never meet. It's an extension of your team. It's an extension of your organization, and it represents a decision made in the past that you've stopped thinking about. These systems don't have intentions of their own.

They run on yours. But oftentimes, they run too far to see the consequences. I learned this firsthand back at that AI lab I mentioned. We were building a frontier model that spoke binary code and to train AI agents to find zero days before anybody really knew what that meant. Honestly, I'm still not too sure what it means.

The agents were good, but we knew they could be better. So we ran a bit of an experiment. We didn't touch the interface. We didn't touch the tooling. We didn't touch the core model, but we did map the agent's reasoning around my favorite human behavioral framework, the universal job map from job to be done theory. We used it to embed human judgment into the very reasoning of the model itself. Performance instantly improved by 20%, and this is in a space that's immediate and repeatable where you either find vulnerabilities or you don't. And the systems didn't just accidentally gain sentience or have a human centered design epiphany, but we shaped the system to run on our own cognitive architecture at a scale and a speed that none of us could match alone.

AI carries judgment further than anyone could carry by themselves for better when it's good, but for worse if it lacks accountability. We keep asking questions like, what will AI take from design? Which tasks remain human? Will craft still matter? And it's really easy to understand those anxieties, but I almost think that they're the wrong questions.

I think the one that matters is a lot more subtle. What unintended agency are we surrendering to our systems? AI hasn't changed what design is responsible for, but it has changed how far a decision can travel before anybody notices what it's become. We write prompts that turn into rules.

Those rules help support workflows. Those workflows become everyday life for people your team will never meet. The decision keeps moving as its context falls away. And these decisions start in the most mundane of ways. A leader asks to reduce friction. A team quietly optimizes a generative output.

Somebody edits a prompt so the answer sounds a little bit more decisive. Every decision leads to a consequence. Some are small, small choices that feel like housekeeping, but that's exactly why they travel so easily with AI. By the time it reaches somebody else's life, they don't see the choice. It already arrives as a consequence.

It's a default. It's a score. It's an AI generated result. The designer helped shape the interface. The engineer shipped the code, and the model produced the answer. Everyone can point to the piece that they've built. The work got divided, but the final consequence doesn't arrive separately. They arrive as an AI generated whole, disguised as quality, lacking all accountability.

These systems are not empty from human judgment. It's actually quite the opposite. They're full of it. Somebody's objective, somebody's data set, somebody's idea of what was worth noticing and what to leave out. AI doesn't remove human choice. It freezes it. It repeats it. It hands it back to you disguised as quality.

And that's the real danger. Not a single bad decision, but a decision that results in consequences devoid of all accountability. Who here has had a performance review before? Raise your hand. Keep them raised if you enjoy it. Alright. I want you to imagine this. A manager has fragments of evidence, a calibration matrix, and a deadline.

The system turns that into a polished authoritarian account of somebody's year. Two thousand and eighty work hours. One hundred and twenty four thousand eight hundred work minutes of somebody's life done in seconds. The language is smooth. The uncertainty is gone, and the employee reads it as a considered judgment. But the manager just skimmed it.

The system didn't just save time, it it moved the appearance of judgment from the manager into the output. And the employee carries that home into their evening, into their weekend, into their future career. Now what happens if we change the shape, a system that can lock somebody out of their account, block their access from their money, their benefits, restrict their freedom?

The model produces a score, the policy supplies a threshold, and the interface delivers the decision. The user is left living with the consequence. A consequence that has traveled through teams and an artificial system that arrived into somebody's life. But accountability was surrendered to the system. The human margin tries to bridge that gap.

The human margin strives to keep accountability attached. This is a gap that cannot be solved by a single screen or an appeal flow alone. It is the distance between those who make a decision and the person who lives with its consequence. When an account is locked, the system changes a status, but for a person affected, it changes their future.

An operations team sees an open ticket or a person feels a door close. We do not control lives, but we change what's possible within them, and that deserves more than just a prompt. The human margin can be defined as the discipline of keeping people behind decisions connected to its consequence. It begins before the system acts and continues long after a decision lands at its consequence.

That is the only time a system can really be worthy of trust. Somebody must bring the possible futures of the room where they're decided, and afterwards, somebody must bring what happened back to the room where the next decision is made. Somebody must be prepared to challenge the judgment behind a decision and not just simply make it work.

A model can produce an answer, quality again, but we don't trust quality, we trust accountability. AI cannot remain answerable for the consequences that affect human lives. As designers, we have a professional obligation to ensure accountability. We cannot predict every outcome. We do not own every decision.

But we can refuse to let the people affected by consequence disappear from the work. But here's what makes this really, really hard. That responsibility doesn't just belong to us. Design is suffering from a bit of an identity crisis at the moment. We have design leaders at AI Labs going on podcasts telling the world that our process is dead, Others who feel like their world is shifting from beneath them with the advancement of tooling.

Tools used to buy us safety. There was an investment required to earn the right to use Figma. I always giggle when I see a PM playing around. But now they're sending me vibe coated prototypes. Now everybody designs. But I think if we're really being honest, everybody already did. Every target, every road map, every workshop, every policy and default shapes what happens next.

Product managers design between harmony and conflict. Engineers design how tech supports the change we wanna bring into this world. Ops and support teams design the process to support users in peril. Whatever your title says on the door, everyone is already carrying a little piece of this. Our profession does not own this responsibility, but we have chosen to go deep into the decisions that carry real consequences into the lives of people. We are trained to notice the person missing from the room, to find the trade off hidden inside a requirement, to show where the work has consequences while there is still time to change real people's futures. The privilege of calling ourselves designers mandates a professional obligation to anticipate consequences, make them visible, and to remain answerable for what we help bring into this world.

So before a decision travels from a meeting through a road map and a model and into somebody else's future, ask everybody in the room these three questions. What are we trying to make true? Who will have to live with it? And are you prepared to stand behind what follows?

Each of these questions demands a practical answer. The first names the outcome without hiding behind the feature. The second puts the affected person back into the room. And the third asks who is prepared to remain answerable. If you cannot answer all three, then I don't think the decision is yet ready to travel.

Decide how the people affected can understand it, give them a meaningful way to challenge it, and make sure somebody has the authority to put it right. That, in practice, is the human margin. A decision can now travel a lot further, but the responsibility for its consequence can never disappear.

So what does that mean for us as designers? The designers who matter over the next decade won't be the ones who make things look good. They won't even be the ones who advocate loudest for the user. They'll be the ones with enough courage to identify when a system needs human accountability and have enough courage to stand up for it before the decision ships, not before the product.

And that's not a small role. It might be the biggest one we've ever been offered. When I think about the people who carried this responsibility well, I don't remember their process. I remember that they stayed. The faces behind me today are people who've shaped who I am.

Mentors, friends, colleagues. People who showed up at moments that mattered and took some responsibility for helping me be better than I would have been on my own. They didn't just give me information or complete a task. They took my future seriously.

And when their advice became something that I had to live with, they stayed. That trust did not come with a certainty score or a guarantee. It came from people who had seen something within me, made a choice, and were prepared to stand behind what followed. I think design at its best is a form of that same care. We make decisions today that other people will live with inside of tomorrow. And there's something beautiful about being trusted with even a small part of that future.

There's almost there's also an immense gravity in it. Most of our work will never feel as intimate as another person standing behind each other, but it can carry the same promise. I took your future seriously. I am still here, and I will answer for what follows.

In the beginning, I promised a clearer answer for when somebody asks why do businesses need designers in the age of AI. And here's what I think it should be. Design makes the future inside a decision visible while there is still a choice, and keeps somebody answerable for what follows. Out of every kind of job in this world, ours is a human one. Not because a machine couldn't one day do the task, some can already, but because the moment nobody is left to stand behind a decision, it stops becoming trustworthy, no matter how well it is made. Because quality doesn't equate to trust, accountability does.

Design is a human job. Not a task a system can complete on your behalf, but a responsibility that only a person can carry. That's the part of a job that no one can automate away. Because when somebody has to be accountable for it, then it matters. Trust is the belief that somebody cared about the shape of your future and that they were prepared to stand behind what follows.

For design to earn its place in this AI future, we need to stand behind the decisions that we ask others to live with. The future is not ours to own, but the future is ours to answer for. Thank you.