Beyond Silicon Valley: Building AI Governance on the Fair Go Principle

Alright. Hey, everyone. For folks who don't know me, I am Aubrey. And as you can tell from my horrific accent, I am a recovering American. I'm so sorry. It's really embarrassing. I can apply for my passport next year. Please keep me. But I'm so excited to talk about beyond Silicon Valley. And for folks who don't know my history, I helped scale Atlassian from 1,000 to 4,000 folks from 2015 to 2020 and spent the last six years at Culture Amp leading kind of ESG and people operations and a bit of responsible technology.

So most of the time now, I work as an AI ethicist. I'm a director at the Ethics Center and I advise technology companies through my consultancy. And I want to talk about why Silicon Valley sucks. So I used to start my talk when I was the chief diversity officer at Atlassian with fuck diversity. And I was like, what?

And I was like, yeah, I'm a contrarian. So I spent time in Silicon Valley. And when I say I spent time in Silicon Valley, I mean I did business development for Palantir. I had a trauma reaction during the first episode of Silicon Valley on HBO. So when I tell you I know what this culture is like and how much sucks and is full of sociopaths, even those who don't mean to operate that way, I cannot say that more seriously. It is one of the reasons, besides the fascism, that I moved. So the reason I say fuck Silicon Valley is because the values that have got us to this point of global dominance are hitting a tipping point where they're becoming unsustainable.

Right? Like if anyone has looked at like the unit economics of the Anthropic IPO, you're like, why? This is imaginary. You have vendor contracts for revenue that you can't possibly have any hope of doing without automating all of human labor, which your technology is too dumb to do. It is unicorns, and retail investors are going to pay the price.

So I want to offer a different way, but first, let's start with what Silicon Valley is. Mark Zuckerberg is really famous for talking about move fast and break things. Underlying assumption is they're not my things, so it doesn't matter. And so this is a sociopathic way to build things. And with the technology we're now using, the stakes could not be higher. So what are Silicon Valley values?

First, a basic disregard for unit economics. One of the reasons I think that Australia punches far above its weight in the global world in terms of technology is down here, there's very little VC money, so you actually have to run a company that makes sense. Right? You don't just have to be like, oh, I'm burning $3 for every dollar I make.

Dario. You actually have to build a business that people want to buy stuff from you. So I think here we have the ability to think strategically about the way that we build businesses that not only incentivizes us to build technology that creates actual user value, but actually builds businesses that go the test of time. Like, I am waiting for the collapse right now.

I think LLMs will have their place, but there's not a lot of moat happening. And so I think we're going to see a very different market in twenty four months than we're seeing now. Technosolutionism. So everybody in Silicon Valley is an engineer who thinks that coding something will solve it. I think anyone in this room probably knows that AI is more of a human problem than a tech problem, if you've ever tried to do AI implementation.

And even in Silicon Valley, people are looking at the rising AI resistance as, oh, people just don't understand. No, bro. They do understand. And you telling them that they're not going to have a job in six months does not make them want to use OpenAI. So again, I think here in the conversations that I have with technologists, one of the reasons I came here to participate, is because I think there is just a higher level of intelligence about what solving problems takes, more discipline about where technology is incredible and also where human side of the equation is important.

Both are right in different contexts. Winner take all. Silicon Valley is absolutely ruthless, you are measured by the commas in your bank account. If anyone has ever lived in San Francisco, you'll find out that one of the first questions someone asks you is, what do you do? They're not being friendly. They're trying to figure out how much money you make every year and then deciding if they want to be your friend.

I'm not joking. And so this ruthless competition is based in American values. There is this weird belief of meritocracy in The US. If folks who don't know, the word meritocracy was invented by a British comedy troupe to make fun of Americans who thought they didn't have a class based system. It was a joke.

I think here we're a little bit more conscious that in fact, the Lord of the Flies is not an aspirational tome. And so the idea that winner take all actually leads to degraded systems over time. Now, you're not reading DOS Capital, but Marx did make this point a while ago. So fair go AI.

One of the reasons that I chose to move away from Silicon Valley to Australia is because I actually believed that Australian culture has more of the fundamental ingredients, knowing that it's not perfect, that make great technology in ways that can benefit the world. So I want to talk about what I think of as fair go AI. But first, I want to make a correction.

I do not believe that our values and fair go AI are the competitive advantage. They're the collaborative one. And I think this is important because one of the strengths of our industry is the diversity that's in it, noting that I worked on those issues and it's still a little bleak. But what are the values of fairgo principles?

First, equal opportunity. I think this should apply equally to who is participating in AI and who is considered in the development and deployment of it. It doesn't mean equality of outcomes, but it does mean equality to engage in a way that makes sense for that person. Equality of the ability to understand the choices in front of you and make those considered choices from the ability to say, fuck no, opt out, to say, hell yes all the way in.

And that there's going to be a spectrum of engagement, but people should have the opportunity to make that choice based on their own positions and values and not based on the fact that they don't look like Mike and Scott. Collective responsibility. It is amazing to move to a country where believing that people shouldn't get medical bankruptcy because they have cancer is not a far left position.

To be fair, I still have far less political positions. But in America, they're like, Oh, you broke your leg? Your fault. Why didn't you not do that? I guess you lose your house now. So we have built in and we have systems that show that collective responsibility works. Yes, I get the NDIS is expensive and Medicare is imperfect. Sure. But we have some basic values that say taking care of each other is a decent thing to do and most people, not Pauline Hanson, think that that's a good idea.

In America, I cannot tell you and politicians will not say it openly, but if you are poor in America, it is considered your fault and a moral failing. There is really no cultural value around the idea that hard stuff happens and we should help people out. I think here in Australia, that's a basic thing that most people across the political spectrum can agree is that there's some level of responsibility to the folks around Us.

Equitable access to wealth and security. I think this is really important. I think it's easy to misread fair go as strict egalitarianism, meaning everybody gets the same thing all the time. I actually don't subscribe to that. What I would say is what my characterization of Australia is is there is a floor at which everyone is entitled to stand, But hard work and, yes, some luck play into the fact that there is still a difference of opportunity.

So the idea is everyone's basic needs should be met. But yes, hard work can have outsized impact on your opportunity set or potentially even your financial well So there's variance, but minimum standard. So I want to talk about what this looks like to bring it into governing the AI lifecycle.

And we're going to talk about that in three stages. So predevelopment, fair data labor practices. So one of the least explored and acknowledged things about this is labeling data is a trash job for which historically frontier labs have put people in conditions of slave labor.

So I want to walk you through a day in a data laborer. So companies not only employ folks in vulnerable positions like Kenya and Venezuela, and increasingly, I have recently found out all of those DEI professionals who got laid off when Trump decided white supremacy was cool, a lot of them are doing data labor sitting in Missouri because they can't pay their bills.

So this is often thought of as a Global South problem. It is not. All those writers that got laid off in the writers' strike in LA are now doing annotations. Wired just had an article about this. But data laborers in places like Venezuela and Kenya are often specifically targeted for the politically precarious positions that they are in such that the companies that outsource that labor do not anticipate the implementation of fair labor laws. The way it also works is it sounds great, but what you saw if you look in that Wired article is that consultants, people with degrees, are lured in with the idea to make $150 an hour.

That is the initial offer. But as people become more dependent on the work system allocated, that hourly rate drops. The piecemeal rate drops for the data laboring. And also, there is a system by which that work is allocated that is fundamentally unjust. So the way it usually works is that there's a queue of tasks, and if you don't grab them, they disappear.

If you miss those tasks, you become deprioritized in terms of achieving high value work. That means that a lot of data laborers, especially in the Global South, are chained to their computers. They're up at all hours of the night because they cannot afford to miss the work that is increasingly being lower paid and is actually leading to the automation of the precarious work that they're already in. That is a design choice.

That is not inevitable. But I bring this up because I think most people might not even know that this is happening. Every Frontier Lab has relied on these data practices, but we do not have to. This also happens with folks who are doing RLHF at the post training stage as well, so you can think about providing fair wages for the labor that's happening.

Also considering that these folks are often under draconian NDAs and exposed to severe psychological harm for which there is no mental health support provided. Again, these are all design choices that we don't have to engage with. So I want to lay that out as in we're selecting the method of building AI, especially if we're doing something custom, we don't have to do this.

And we can ask. Usually, it's a consulting firm or a third party that's actually sourcing this. You can ask about their payment and labor practices. You can ask about breaks. If you're interested in that, come talk to me. I can help you. Consent forward data collection usage. So one of my research areas I'm a master's student at the University of Cambridge in AI ethics is looking at what actually consent based governance looks like.

How many people have read all of the t's and c's that you've clicked except to? They're totally fucking bullshit. Right? Like, one, physically you don't have time to read them if you wanted to. They're written in inscrutable legalese that even folks like me with a stupid number of degrees have no idea what it means. And it happens one time.

And then when they change the platform, they give you a push notification that you swipe away because you want to see what, I don't know, whoever you follow on Instagram is doing. Yeah, not cool, not good enough, not ethical, legal, not ethical. And so thinking about what does it look like with the data that we're using to train our models?

Are we scraping it from the internet and saying, well, you didn't tell us not to crawl it? That's not consent forward. And the law is shifting to make this both a legal and an ethical concern. Right? So remember the $1,500,000,000 settlement from Anthropic for scraping all of those books. Again, ick. Don't do it. But also, this is based on something that we'll talk about in the next, is that data collection is often done far beyond what is required to achieve the necessary utility of the model.

And I want to talk about that actually from an environmental perspective. So first of all, not all models actually need to be trained on the whole internet. I think there was this drunk on scaling theory that was going on at Frontier Labs, and it turns out that, whoops, transformer models aren't AGI. But the idea is that scaling is having diminishing returns, which I think anyone who understands physics would probably anticipate.

But the idea is we actually don't need bigger and bigger data sets always to produce better and better model outcomes. There's a really amazing lab that I love in Seattle called A2 that because of their funding, they are restricted by the money they have that they have to really carefully curate their data sets. This is great for a lot of reasons.

That financial constraint means that they're actually producing really, really high quality models. They're focused on AI for scientific research. But what it also means is that it is a more environmentally responsible way to do AI. So they do have utility benchmarks that they're trying to hit, but they use smaller, more intentionally curated and annotated data sets to achieve that utility with smaller training times and smaller data sets.

So that reduces because we know that with models, there's a debate about the environmental impact. Marginal inference is actually not the big part. It is often actually the training process that produces the greatest water and carbon impacts. And so you can save money on training and slow climate degradation at the same time.

You love a win win solution. But really thinking about a smart training strategy that is not about maximization. So everything in Silicon Valley is about growth and bigger. And sometimes that is the right thing, but it is not always. And so questioning that assumption and saying, what is sufficient can be really powerful because once you deploy that model, it also means that your inference costs are lower over time.

So your cost of doing business, once you actually deploy the product, actually goes down, which is really great for those unit economics that Andreessen Horowitz doesn't care about, but you should. So now I want to talk about development. So harm aware scenario analysis. One of the things that I've seen working in Silicon Valley and I think it's really easy to think that the reason that women and people of color and disabled people and queer people are underrepresented is because we're not capable.

But one of the fundamental problems besides equal opportunity is that when you have a homogenous room, the folks in the room often don't actually have the correct answer to the question, what could go wrong? And the reason is because it won't go wrong to them. And so it is really important to do scenario analysis with your technology. Now, of course, you want to do a risk assessment.

Depending on the use case of the technology, the potential for harm is vastly different. This is actually what I work with a lot of companies on is doing impact assessments and really starting from perspective of who is likely to be harmed if something goes wrong, what is the plan to mitigate the likelihood, and what is the procedure that exists to fix it when something inevitably gets fucked up.

So we know that this technology works in unknown ways, and so we're not aiming for perfection. But if there's no consideration of what could go wrong when there is absolutely a room of economic research to tell you something will, you are choosing to enable that harm. And we don't have to. So harm aware scenario analysis, thinking about different groups who are likely to experience these use cases or technologies differently and then designing for it. So in the work I do, we talk about equitable design because it turns out that when you pick the most vulnerable user or the stress case user and you design for them, everybody else also gets a better experience.

So traditional design theory often has an edge case, and that often gets mathematically sorted out because you go, actually, that's only for 10 people. Like, I don't have time to build that. But when you actually think of those 10 people as a stress case who are at the biggest risk of harm and design for that, the other 90% of people also get a better experience.

So you actually reject the trade off as a necessity by focusing on the marginal case. Again, cheaper, more ethical. At the same time, we love a win win. Guardrail development and safety testing as a tollgate. Don't put shit in the market you haven't tried to break. Right. Like, I don't know.

That just seems basic, but but the reality is that happens. So and now, like, that is certainly an ethical issue, but we're seeing rising legal cases about this. So the example that is probably on everyone's mind, if folks didn't see, I was surprised it was Florida caring about humans. But state of Florida has sued OpenAI for a failure to put safety guardrails, especially around suicidal ideation and violent intentions with its users specifically focusing on children.

A lot of its litigation is about children. And we're even seeing, if anyone has read BJ Fogg's Persuasive Technology, like the design manual of Silicon Valley, that's a lot of what the meta cases are about, right? Addictive technology that is harming children. And so really thinking about from that harm aware scenario analysis, what are the system level prompt guardrails that we're putting in?

What are the monitoring strategies that we could develop to identify when something's going off the rails, either because of problematic user behavior or just because of model drift due to a variety of factors. But we shouldn't be deploying something into the hands of users without considering that and building that into our implementation plan. How do we build a safe technology?

And yes, you are probably going to be running ahead of regulation, especially here, because ALBO seems to think, we'll see, is a strategy. It's not. And so really thinking about that safety testing, again, thinking about the fact that you are definitely going to miss stuff. That's not an excuse to miss the easy stuff. There's the knowable harms.

Try to mitigate those. And then come up with a solution for the unknowable potential harms. Data sheets and model cards. So really, really great paper by Tamit Gabrou about this. So making sure that you're actually able to describe your data provenance and the agenda that it has. So I always say every data set has an agenda. You need to know what it is, and you need to be able to explain it to your users.

Going back to that consent aware piece, letting people decide if they want to engage with your tool. And having detailed model cards are great. Again, would say Anthropic, much better than OpenAI. I'm not even sure Meta does safety testing, or at least their model cards do not suggest that they do. And then quickly, I'll go through deployment.

So the reality is that deployment is going to require some kind of continuous monitoring, probably for those preventable and knowable harms that you identified earlier in the development process. Accessible harm redress procedures. Things will go wrong. Let people tell you about them. And this needs to be accessible. So if I think about the free version of ChatGPT because I refuse to give Sam Allman my money, It's really hard to find the feedback button. That is a dark pattern.

It is best practice. That does not mean it's ethical or that you should do it. So make it easy for people to tell you what's going wrong. Yes. That is definitely useful just for overall product improvement, But when something is going catastrophically wrong, your users are your earliest warning signal, and you can fix it before you end up with a very, very expensive lawsuit.

Predefined kill switch thresholds. This is the spiciest part of the talk. Depending on your use case, this may not be relevant. There are certain use cases that are just not high stakes enough for humans. But for example, if you find out that your chatbot is teaching kids how to kill themselves, you probably shouldn't leave it that way. And so again, from that scenario analysis, thinking about what could go wrong and if something catastrophically does, what does pause look like to both take care of the humans but then also make sure that harm doesn't recur?

This is one your VCs will not like. Just warning you. There is like, capitalism is everything suggesting that this shouldn't happen. But thinking about that ahead of time is really helpful because I promise you in the moment, the economic pressure to not shut something down will be overwhelming towards any wonderful individual's ethical commitments. Right? So defining that ahead of time, writing it down, making everybody aware of that SOP, I know it doesn't sound that interesting, but I promise you it leads to better things.

And in a country where trust in AI is basically at a global low, being able to say you have these things could actually drive user adoption and trust in a way that I think we have a unique ability to design for because we do have a skeptical public. That means that we can build the types of things that if they work for Australians, higher trust people will also use them. So it is a global competitive strategic decision.

And last, before I get off the stage, I want to talk about a fair go tech business. So I really suggest everyone go on archive and look for a paper if you want a little existential dread called the AI Layoff Trap. Who's heard of the prisoner's dilemma? Yeah? Cool. So all these boards are like, automate your people, juice the p and l, get rid of 10% of your workforce.

That sounds brilliant to you as an individual business, but if everybody does that in about two years, we get Yeah. So stop trying to replace people with AI. It is the most unimaginative idea ever. Why not augment them? Why not create more customer value? Why not innovate? Right? There's so many cool things we could do and we're like, let's make people destitute.

I'm sorry, that's what it is. Every business pursues automation. So we all say we're going to automate away everything. And yes, I do think there is some work that should be automated. The tax and consumer base purchasing power goes to zero. Societal collapse. So I often like to think of Henry Ford in this moment who is a noted bigot and a capitalist and he had a great philosophy.

He did not pay people a living wage out of the goodness of his heart. He did it so they could buy his cars. And so I offer this because I think the current way that we're thinking about AI adoption is so short term limiting. Like, we're probably not sitting here in ASX listed companies, so we probably don't have to report to our shareholders on a quarterly basis.

But longer term time horizon thinking opens up so many possibilities about the way that we can apply these technologies and run our technology businesses in ways that are growth oriented, that do result in financial returns, but do that in a way that isn't the way that Silicon Valley is trying to do it. Because Silicon Valley says, lie, chase unicorns, IPO, I got mine, society collapses, but it's fine.

I have a bunker in Hawaii. They do. Or New Zealand. So all I have to say is please keep me, but I hope that we can agree that the way to build AI is the Australian way. Thank you so much.