Having your cake and eating it: An implementation guide for privacy with AI

Thanks very much. Does that sound work? Yeah. Interesting about the product thing. Last time I gave a talk for web directions was 2018. I was doing product. Gave a talk about product management for machine learning systems as it was then. Come a fair way since then. Still doing machine learning or AI. I'm talking about privacy for AI today.

I think people are very much aware of the issues that AI systems have with privacy. But I wanna give a bit of a talk about some of the technologies that we can use to address some of those concerns. Privacy tech in itself is a really interesting topic. I'm very happy to talk about that in a lot of depth, but I am really focusing on the AI topic here.

So I'm trying to keep it short as I can. 2016, we can no long 2026, we can no longer ignore privacy. Consumers care about it. If you go and ask a person who's a bit AI wary about AI, the first thing they ask is, I don't want them to have my data.

And it's not a security concern. Like they may talk about security as well because they don't really know how to express it, but these are legitimate companies using their data for things that are written in the terms of service. And that's actually the thing they're concerned about. And that's very different to, oh, I'm gonna have my data hacked.

So, yeah, as product people, engineers, leaders in technology companies, we want to be able to build feature rich, functional AI powered applications, but we also want to give these consumers privacy. And for once, there's some good news. You actually can have your cake and eat it now.

It's hard and it's not this like simple ticker box thing that you can sort of do now with some degree of security issues, conceding the Swiss cheese problem. But like in security, there are mature products that let you address a majority of known security issues. Privacy is not like that. There's a bunch of emerging technologies that you sort of have to stitch together yourself. And I'm gonna talk through some of those.

This is mostly about awareness today. I really just want people to know about these topics so that they know the questions to ask. And when you're building products, you can sort of say, I have considered this approach. So also six different technology groups that we I'm gonna talk through. And I'm gonna go quite fast because otherwise I won't get through them all, but happy to talk about them in more depth. I've used some of these in production at various levels, and it's been an interesting experience.

And I'll talk about the ones that I've used. I'm gonna start with differential privacy. So differential privacy is something that's quite widely, reasonably widely deployed now. So it's a way of adding noise to your data that makes it still statistically useful, but you can no longer identify individuals within that data. So it protects the individuals in a statistically useful dataset.

It's quite mature. So the US census has used it, in production for think it was the 2020 census they did it, because of re identification attacks that were possible in earlier censuses. There's some really interesting work you can look through that show how easily you can re identify single people from protected groups from census data because people because the techniques for reidentification have have improved so much for better and worse.

So this is one of the things that I do have some personal experience on. I'm at a place at the moment. We have a product called the Everyone Project. We do diversity reporting and screen industry across Europe and The US and Australia. And we have to be GDPR compliant, So that means that we cannot collect and keep personally identifiable data, let alone release it. And yet a lot of the things people want to report on are directly on those protected quantities, and that makes it really hard. So we have used differential privacy to be able to generate statistically valid reporting, but doesn't allow people to slice and dice the reports to find out the one gay black person that was working on a particular film.

So that's really useful, but you know, from personal experience, the utility loss is a real thing. It's written up there. People expect BI type reporting, so business intelligence type reporting from their reporting. Differential reporting, differential privacy based reporting isn't that. It's census level type reporting. You can do statistically valid reports, but you can't go and see that you have three people and then break that down by two.

It doesn't work like that. But knowing what it's for, it's pretty useful. Some of the places that it's used, US Census, everyone project, that's where I'm at the moment. Google g board training, which I'll talk about a little bit later as well. So federated learning. Federated learning's pretty well understood and fairly widely deployed these days. It's a way of training a machine learning model across multiple devices and keeping that data private to your own organization.

So one of the places that this is in production used in production now is health applications, where hospitals can train, or medical providers can train a local model on their own data, and then share that the updates to that model into a shared global model.

And that gives both the benefits of this global training, because you want the diversity, but also a locally locally private data. Yeah. So protects your local data from raw data leakage. For a lot of people, that's really what they want. They cannot release medical records, for example, and yet they want to be able to do this full life cycle thing to be able to trace the how how often a person is admitted into hospital for what, and see if there's patterns.

And if you can do that across multiple health care providers, can get those patents much better, but you can't release the data. Federated Learning lets you deal with this. It's not a perfect privacy thing. There are attacks that are possible if you don't if the central authority isn't trustworthy. So for example, this this central authority has their own global model. You send the updates of the global model to the central authority, and the central authority could do sort of gradient descent from random data to try and find out what data creates the updates. So those types of attacks are possible.

They're expensive, but if you're working in a regulated industry where you do trust a central provider, it does give you a way to to get a lot of the benefits that you want from these models. Federated Learning. Yeah. So again, Google g board. Google g board's the keyboard that Android uses, and it's also on iOS. It's quite an interesting model that they use because it is quite well architected. They do both federated learning, so you try it trains a miniature model on your own device, so it learns your your own typing patterns, but then it also sends updates to the the global model. So it can actually see when, you know, new emojis are coming into fashion.

But it uses differential privacy on that update, so it so Google can't track which individual person is using a particular word or a pin code or something like that a lot. So you'll see that type of pattern a lot, where systems are architected using multiple methods of privacy preservation in order to build a fully privacy preserving system.

So a lot like security, you have these Swiss cheese approaches, and you have to try and stop the holes. And it is a difficult field because the the techniques aren't that well understood. Multi party computation. So this is pretty interesting. It's not we're slowly ramping up the complexity here, and this is getting close to sort of miracle maths type technology.

We're not quite there yet, but so MPC, not the same as MCP. I'm pretty sure I'm gonna make that mistake some at some point during this talk. So multiparty computation, it's a way of doing cryptographic calculations on your own data, and not so you can avoid revealing that data to other parties, but it's not the same as federated data because you're not really training a model for learning.

So you do things like so you can unlock features based on provable attributes. So we'll talk about zero knowledge as well, which is a similar thing, but you can do things like making sure a person's of age. So some of the examples here. Yeah. Cookie replacement in ad tech.

That's a great one. Anti money laundering's probably the best example. Banks that can't release all their transactions to each other, but money launderers will try and do transactions below the compulsory reporting level across spread across a number of banks. If you can encode those transactions in a way that you can share without revealing those people, and only the other banks that have that same person in their system see the the system itself on the encrypted data notifies that this is a pattern that we've seen elsewhere, you can send this notification back to all the other banks to say, okay, this is a pattern that we're seeing elsewhere.

None of the banks can decrypt that data that's coming in. They can only see that this is a problematic data, but problematic record because of some other data that's come from somewhere. It's pretty interesting. I did a paper with my friend, Tobin, who's actually anthropic now, on private location sharing, which is it's pretty amazing.

You can do this thing where you encode your locations as strings, and then if you've both been in the same location, you can get a notification that you share that location in common. Same with similar interests. So if you're both interested in eating McDonald's, but you don't want to share that information with everyone because you're ashamed of it, you can encode it in this, and then you can other people that are also interested in eating at McDonald's will see that that you're in the same group.

Obviously, there are leakage issues around that, people fake encoding stuff, but there's some interesting product possibilities around that. Homomorphic encryption. So this is like it's kind of a miracle math thing. So I can encrypt data with my key and send it to you, and you can perform operations on that data without decrypting it, and send back the results still encrypted, and I can decrypt that result on my thing.

So to ground that in something that you can do, you can do private search. Search is a calculation. It's vector operations done on a data set. So I can send this is what Apple does for their location search. You can encode a landscape photo as a vector, send it to their central service.

It never that data is encrypted with a key that's on your phone. It performs in encrypted form a nearest neighbor match to find other landmarks that look similar, and sends back still encrypted, you're at this encrypted form, which you have to decrypt with your key to see where it is. So Apple cannot decode this in any way.

They can't see where you've been. They do the same thing with their phone spam detection thing, where like, I don't know if you've seen, but there's a thing that pops up that says potential spam caller. That's not because they're just sharing this number back to Apple. They're sharing encrypted form of that number back to Apple, which then forms this homomorphic encryption calculation on the number to see if it's been seen before and marked as spam.

It's kind of amazing. Look, it's it is amazing. It's the downside is the computation costs. It for fully homomorphic encryption, which is all mathematical operations as opposed to just addition. It's between a thousand and a million times slower than doing it on normal calculations, which is a lot. But computers are fast, and based on some other technologies like zero knowledge, there's a good potential that this will improve in the future.

Apple has done a lot of performance tuning on their stuff, so those numbers may be slightly out of date. I don't think they've published it. Look, it's one of my favorite technologies because the only thing you need to trust here is the maths. The other people cannot decrypt your your work. You can do a quantum safe key, so even post quantum, they still can't decrypt what you've done.

That's like just amazing. So yeah, look, that covers off everything there. I talked about the both of these examples. Apple probably is the group that's pushing this the most. Signal, the messaging app, has got a little bit of this in some of their PIN recovery stuff, but I'll talk about that later because they use some other technologies as well.

A lot of the time, you'll see Google and Apple mentioned here. They're probably the groups that have pushed these technologies the most. I like this stat because of the homomorphic thing, which just seems amazing. Google's g boards thing is another thing that come up a lot. I think because they have this unique mix of both very good engineers and very good researchers within their organization, They're the types of groups that have been pushing this.

Outside of that, you will see that a lot of this is done through various open source libraries that you sort of have to stitch together yourself, but you have to have the awareness that this is doable as well. So it's only within these, like, big tech that it's happening because of this big barrier that's around the availability and the knowledge.

Hoping to at least share some of the knowledge here. Zero knowledge proofs. So you can prove a claim. So I'm older than 18. I earn enough to service alone without revealing anything other than that. So that's a mathematical proof that you're 18. If you trust that maths is real, you can guarantee this person is 18. But you don't learn their birthday.

You don't learn their actual salary. The place this has been pushed the most is in crypto. Look, I have I worked at a place that was adjacent to crypto for a while. That was an experience. Yeah. It's like AI, but turned up to 20 with unicorns.

But for better or worse, they were willing to take bets on some out there technology, and zero knowledge is one of the things that put they've pushed along along a long way. So in the last five years, the speed of generating zero knowledge proofs has increased between a 10,000 times at least.

So that's made enormous differences to how possible it is to run. If you're in the blockchain world, there's full Ethereum compatible virtual machines that can run-in a zero knowledge proof way, which is quite impressive. So you can actually prove that you've done these transactions, financial transactions using Ethereum or Ethereum compatible stuff via these proofs rather than through the blockchain. Then the proofs get written back to the blockchain.

There's this whole layering system. But I think the best example is the EU age verification. Yeah. Unlike some other solutions, you don't have to share your identity documents with weird companies like Persona, and you don't have to share sorry. You share a single mathematical proof with the place that you're logging into that proves that you're 18.

They don't get any identity. They don't get this unique ID that they have to trace back. They just get a mathematically proven proof that's signed by a government issuing agency, whether it be your passport or whatever, that you are 18. And it doesn't say, your birth date is this. It says, are you 13 for for Instagram? Whatever it is, whatever the claim is, that's the only thing that they get. And that, like with privacy, minimizing the amount of uniquely identifying data is incredibly important because of these re identification attacks.

Trusted execution environments. So these are hardware devices that run within the CPUs of modern computers. They are signed by the manufacturer and Secoova from the rest of the operating system, so you can have fully isolated compute there that the person running the machine, cloud provider, can't read.

I did a bunch of work on this a while back. Look, they sound great. I think for me, they sort of failed to live up to the promise a little bit. You're relying on this test on this trust chain that's signed by a hardware vendor. I don't think that those trust chains are safe anymore. The US has shown that they're prepared to use the NSA to do commercial spying.

There's an Airbus case around that. They've shown that they'd be able they're prepared to use court orders to get decryption keys. There's BitLocker cases around that, Which means that these keys, because there's only a single key per CPU family, if it ever gets into the custody of anyone else, that whole CPU is gone. There's been enormous nut quality quantity of bugs, which makes them not as safe as they should be.

And finally, most of the implementations, as far as I know, all of the implementations aren't quantum safe, and probably can't be made quantum safe because of the speed. So as a supplement to other things, they're interesting, But on their own, unfortunately, I don't think they're quite as good as they sound. There's a few things that people are using them for.

In particular, signal use case is quite interesting. They split keys across a number of trusted execution environments to try and avoid any vulnerability in a single one. And then I believe they use homomorphic encryption to share that key thing. So you when this is when you're trying to recover your PIN, you can send in some recovery information.

Not sure how quite how it worked there, which is then homomorphically encrypted and spread across the the three separate trusted execution environments, which put the key back together and send it to you. And then most cloud providers also have a trusted execution environment you can use.

Finally, private models. Look, we're at an AI thing. This is probably a thing most people are interested in. Look, this is running your AI model yourself. It's great. It's great fun And in coming becoming increasingly useful. So I think we've seen this morning for all the artificial analysis thing, how close some of the Chinese labs and some of the American labs are getting to frontier.

And, you know, if you thought that AI was useful at the start of this year, you can get a free model that you can run on an expensive computer yourself. So the elephant in the room is, of course, that they're Chinese models. Whether or not that's a problem often depends on your circumstances.

I would love for there to be an Australian provider. I know there is a Melbourne company, Maincode, who's doing some training. I haven't seen I've seen they've done one small model. Haven't seen anything else about it. But based on other things, you sort of have to put up in order to prove that you're close to the close to something that's useful. If you're going down this road, you're gonna have to invest in hardware running a a close to frontier model.

So something like DeepSeek v four is between Sonnet and Opus levels of intelligence. You can run a quantized version, so less than perfect version of that on a 128 gigabyte MacBook, which is roughly $10,000 of hardware, and it'll be roughly usable for a single person. Within a business, spending a couple of $100,000, you can get something that can run reasonable workloads at somewhat sub frontier levels of of performance. Invest in your evaluations.

I've done a bunch of I've got a whole massive thing of hundreds of local models, and it's interesting where things sit. You sort of begin to understand that there is this The frontier models really genuinely are the best, but that gap does close, and your benchmarks will get saturated. If your benchmarks are what you're actually trying to do, perhaps that doesn't matter anymore. Look, fine tuning for local models is something that's pretty useful too. Few things we didn't cover.

Secure aggregation. I don't know a lot about secure aggregation, to be honest. It's a way of combining data that's different to DP. I've never looked at it. Data redaction, remove your PII before doing before using it. That's useful, but there can be re identification attacks done on that through other mechanisms as well. Wow, misspelled using. OpenAI's got a open source model that does this.

I've got a little thing in browser, like a model that runs in browser. It can do it as well. NVIDIA's got synthetic data tools. So create synthetic data for doing various things. In some cases, that's really useful for both for testing and for evaluation since they've got personas for a number of company countries.

So when to use what? Look, I won't go through that in detail, but almost all cases, it's a combination of technologies. I think you sort of have to start thinking about what problem you're kind of trying to solve, who you're trying to protect, and then you can look at these technologies to work out what to do. There's a bunch of links, a QR code that will give you this slide deck, because I did move pretty fast.

Yeah. And I'm gonna be around for the rest of the couple of next couple of days. So happy to talk about it more.