All right. Thank you. All right. It's great to be in Melbourne today. It's great to be back. I moved to Sydney for work, so it's good to be here. And yeah, I'm really excited to have the opportunity to speak here about what is, I think, a very simple, very generic pattern that everyone should be thinking about across all the different levels of complexity for different AI systems, whether it's a Kurting agent or anything up to a swarm of agents.

I think this is a really nice thing to keep in the back of your mind. Everyone is building larger systems using AI. Weeknd to weekend, I feel like I see different things. But still, despite that, I think we all kind of see that agents have this pattern of really doing amazing things but still not quite being as reliable as we might want.

We can see recruiting agents breaking features when you're making unrelated changes. We can see that they sometimes write poorly optimized queries for data. We can see that they sometimes produce exploitable cerd or security vulnerabilities. We see them miss different edge cases and products that weren't considered.

And I think this is really important because in order for us to be able to ship any of the output that we're getting from different AI systems, we need to be able to actually trust the results that they're producing and trust the systems that they're building. This month, we saw a pretty impressive feat from the BUN team.

Setting aside whether or not moving from Zig to Rust is the right decision for the project, I think it's an impressive engineering feat. Over the course of about eight days, they went from a stray commit that was seen online to having produced around a million lines of kerb I think it's actually around 700,000 lines of rust over the course of about a week, and through that also having that launch with passing their existing test suite.

And I think being able to see that engineering feats like this are achievable now is something that should highlight why we sometimes see this disparity between incredible things like this and then trivial mistakes still happening as well. So if we look at what a typical kerning agent workflow has historically looked like, you'd have some sort of IDE or some sort of CLI tool.

You send a prompt to an agent. The agent makes edits to some piece of software or some artifact, and that software or artifact produces some sort of output that you look at. And in control theory, this actually has this sort of structure has a name. This is actually called an open loop system. This is something that's used in manufacturing plants and things like that where you have this controller component that is responsible for making changes to some sort of system.

You could think of this like a plant that's producing different sorts of manufacturing outputs. And the key thing here is that the actions that this controller or in our case, kerting agents are making independent of what the software is actually outputting. So an open loop system is an automated system in which the output action is entirely independent of the plan, which is in our case often software, what that software outputs.

A classic example of another open loop system is a terster. When you're using a terster, you have one input. You get to change the dial. When you change that dial, you're changing how long the terster is going to terst for. And it doesn't matter whether that terst is burnt or completely blonde. The terster is just going to go for that amount of time.

There's no sort of observability about whether or not the terst is actually done yet and whether or not that system should stop. And the analogy here is that if you're just changing your prompts, crossing your fingers, and then firing off the agent again, you're going to have a similar sort of experience. And so the difference here of what's actually missing in this picture is that we really want to have some type of automated feedback that's provided. In control theory, again, this is called a closed loop system where we have this additional sensor component that looks at the outputs from the software, makes an evaluation at them, and then provides that as feedback back to the controller.

So really we want to be able to move to make sure that the systems that we're working with have this sensor component. If we don't have this sensor component, you're going to have a bad time. So this is a closed loop system. Classic example of a closed loop system is a plane. A pilot doesn't respond to every gust of winds while while a plane is flying. They set high level parameters and objectives for where the plane should be moving.

And the plane itself will respond to different changes in in the winds, in its earned in its earned actions, when it changes the control surfaces that will influence other sorts of things about its position. But the system can see the impacts of its earned actions and be able to correct for them. So we want the same sort of thing out of our own agents. We want to be able to define the criteria that the system should operate within and let the agent be able to observe the impact of its own actions and be able to correct itself.

So this is the first lesson. We'll be going through three of these. First lesson, the main takeaway here is that you want to be able to close your agentic loop, provide automated feedback to your agent by building a closed loop system. And this is not crazy idea. I think that there's been a lot of discussion on this already, but I think what we'll move to next is kind of looking at this in a little bit more detail.

If we go back to Bunn, the censor that I think most likely that they really used and leveraged was the unit test library that they already had for Bunn. You would have seen if you followed the story that they were passing about 99.8% of the tests or something else like that. Another censor that they had was this migration guide.

If you haven't seen this, I'd encourage you to go have a look. It's excellent document that describes a lot about the specifics of how to migrate different components from Rust to Zig. And I think there's also an interesting point here, which is typically AI is is kind of talked about as being perfect for for for greenfield projects. But I think one of the best things that you can get from existing code bases is the set of tests, interfaces, and software contracts that are in that system.

Each of these provide excellent points at which you can use an agent to be able to compare its outputs against thr's NURN constant objects and check whether or not it still upholds thirs. So there's a danger here. If you're only measuring just test correctness, you're going to still have the agent be able to correct and respond to changes though, Feature to feature, you'll see the agent sometimes break some tests.

It'll see that. It'll fix it. That's fine. But if you're not measuring some of the other aspects of your system as well, you'll also see theirs start to degrade. So this isn't good. Really, I think a great pattern that I've seen is some combination of one type of feedback that measures the correctness of a system and another type of feedback that measures the quality of that system or the performance. This gives a good balance between making sure that that system is still correct and performing giving valid outputs but also guides it towards a better, more performant outcome.

You wanna be able to keep it working correctly but also to have it improve against some other metric. Another example of this was a project that the Shopify CEO went through where they were able to improve the performance of a templating library in Ruby over the course of two days or something like that on quite a long standing CURT base as well, mind you.

They were able to generate a 53% speed up. This is across a bunch of different areas, things like the speed to be able to pause different templates, reducing the amount of memory used. And again, it's a combination of having unit tests to be able to check that the agent hasn't actually broken the piece of software, but then secondary pieces of feedback to be able to guide the agent in terms of this has improved the latency of pausing.

All this has reduced the total memory allocation. This led to effectively many different micro optimizations by letting the agent search and iteratively try different things and get feedback on whether or there's a way of working. So this is the second lesson. Try to give your agent an objective and use automated feedback to specify whether it's still behaving correctly and guide it towards better outcomes.

So with all that said, the biggest decision for you as an engineer to make is what sensor and what piece of feedback do you provide to your agents? And there's a lot of different options here and I think this is where, as engineers, we'll begin to spend more time. If you're the only person, if you're the only sensor giving feedback to your agent, either you're in the loop on every step and you're failing to be able to leverage some of the utility that you can get out of AI, Or if you take yourself out without adequate controls, you're gonna lose control of that system instead, and neither of these is scalable.

So let's talk through a couple of different ones. Everyone's seen a move towards more strongly typed languages, and I think one of the reasons for that is that the compilers in those languages are able to get are able to give more specific feedback about the types of mistakes that the agents might make. Rust and TypeScript are good examples, and strong type systems overall can help catch some of these classes of errors or be able to help the agent prevent them from happening in the first place.

Another very common example is visual feedback. Everyone that's seen Playwright explored last year in terms of being able to help the agent be able to see the changes it makes on UIs and things like that, assist with spatial understanding. And again, the goal here is to be able to remove yourself from this loop and be able to have the agent be able to see the changes that it makes.

Hooks and harnesses are another good one. I think that this is interesting in that compared to some of the others it could be very timely. You can also be very specific about only running hooks on specific types of actions that the agent makes and trying to make sure that you're giving very specific precise feedback to the agent.

Adversarial review is another one. So I think there's already been lots of examples of having things like Claude Curr generate a piece of software and then have Codex review it or vice versa. This is great as well for being able to find bigger picture problems rather than just structurally your output the wrong data or something else like that or your test fails.

But it can also help with finding architectural changes that maybe you should make. Differential testing is another one, sort of being able to compare the output of the system after the agent has modified it against some NURN good implementation, some reference implementation, or even just the previous implementation or something on main.

Being able to compare between those can help you catch more issues. An example of this is Anthropic had a project earlier this year where they tried to reimplement a C compiler from scratch. At some points, they used GCC as a known good reference implementation for that. Another example, another one where we use a combination of correctness and quality projects where we try to generate newer GPU kernels for specific types of optimizations.

And often, that's combined with quite heavy fuzzing or things like that in order to validate that that kernel actually outputs the correct result that matches whatever reference operation you're trying to move into a kernel. And another one here is property based testing.

So a of unit tests people initially learn to write are just example based where, for example, you might have a specific data structure that you're expecting or make very specific assertions about the contents of a result. Property based testing could be a lot more flexible and focus is more on trying to trying to define what verifiable or cheaply verifiable properties should exist about the output from a system.

We'll come back to this later as well because I think it ends up playing a big part in terms of making sure that these sorts of tests and feedback are also maintainable. And the last one, again, bit of a different sort of thing, complexity constraints. I think adding feedback about this system has gotten too complicated is very, very interesting in terms of having a more constrained search space. Giving a budget to an agent and letting it optimize within it is normally quite interesting.

A great example of this is Kapathi's auto research project where you look at what's the best LM that you can train within five minutes of compute. Another example is trying to constrain the amount of source code that is available to use to implement a particular feature. If you start to see cases where agents just get way too involved, creating way too big of a set of changes for a feature, this is a very interesting one to be able to just try to squash things down, get it to implement the call functionality fast and focus on that.

And the more of these types of feedback that you have, the more complicated and the harder the type of problem that you'll be able to solve. Here, the idea here is basically if you're the only piece of feedback involved in the system talking to the agent, you're only going to get to these lower complexity tasks. If you include tests, you're going get further.

If you include tests and tools, you're going to get further. And like I really encourage you to try to think about what the best types of feedback are for your system. So really the third lesson is be creative in thinking about how you can build feedback into your system. Finding the right level of diversity and finding the right level of feedback for your project is really important, and it helps you to build that higher leverage.

And so really I think engineering as an industry will swing back towards a lot of these much more rigid not necessarily rigid, but more formal approaches to verifying the performance of the system and the correctness. And a couple of caveats here. Like I said before, you don't want to end up in a case where maintaining the separate set of feedback ends up being more burdensome than the original system itself.

You want to try to be careful about the types of feedback that you select so that you're not baking yourself into specific sets of examples. And property based tests, like I mentioned, great example of this where you can avoid needing to buy into very specific brittle examples. And so yeah, that's takeaway.

The goal is to close the loop. One, give your agent automated feedback. Two, set an objective, correctness and quality. And three, be creative in building feedback. All right. Thank you.

I did say earlier there is no time for questions, but I'm going to ask my same question as always. What would you like to have covered that you didn't get a chance to?

I think being able to see how these sorts of lessons are applied where I work at Optiva. We trade millions of different products. We process petabytes of data a day. Fortunately, there's a great opportunity to see how this is applied at Optiva. If you go to Ryan's talk at 02:40 in the hallway, you can see how this is applied in practice at a quantitative trading company. Really encourage you to show up for that.