Constitutional Prompting: Making AI Coding Agents Reliable Without the Iteration Tax

Awesome. Great. Yeah. So I was saying, this is one of my favorite quotes recently on x. It says, you can outsource your thinking, but you cannot outsource your understanding. And this nicely captures the boundary of useful agentic work that is that is being produced today.

Right. As we all know, agents are extremely good at producing text, writing code, producing detailed plans and reviews, doing deep analysis, etcetera. But where they fail woofly short is when they have to inherit your understanding. And that's what this quote is articulating here.

The models are unaware of your team's judgment, your taste, the team standards. The argument here is that if that knowledge stays locked in your head, the agent is forced to fill in these defaults by stepping in. And in in a lot of cases, that's not what you want. So this talk will hopefully argue for a case to make that understanding more explicit and accessible to the models.

So in the session, we're going to talk about two simple but powerful ideas to solve for this. Let's go. So I wanna motivate this talk with a real life example. This is I apologize for the for the small text. This is a screenshot. I was told explicitly not to do live demos, so you have to bear with this.

So this is actually a real life workflow from rp1.run, which is an open source product that I maintain. So in this scenario, the workflow is actually taking PR as an input. So it takes a PR URL or a number as an input and produces this three d walk through map. So this is not a not a replacement for your p r reviews. This is more about orienting yourself before you jump in and and review the code.

So two things will stand out when you run this yourself for the first time and you can. First is that this is under the hood a fairly complex multi step, multi agent workflow that runs flawlessly, completely autonomously without any manual intervention after the after the first prompt. Second is that it consistently produces useful three walk throughs every single time. So the point I wanna make here is that the con this consistency that you see is not accidental.

And and this is the output we want from AI agents. We want the AI agents to produce work that is free of bogus claims and hallucinations. So let let's look at how this works under the hood. This is an extremely condensed version of that workflow that you just saw. This is what r p one uses.

But it serves as a good illustration to to make the point. As you can already see here, we we we enforce some constraints and analytical policies on the agents in this little prompt here. The policies instruct the agents to collect an index evidence and then generate that report based on that evidence, not based on what it saw just now.

So you might be wondering what levers can you pull in your own agentic stack to enforce policies like this. It turns out there's just one lever that you can pull. As you all know, there are three main components of any agentic workflow or or stack. There's a model, there's a harness, there's the prompts that you can control. The models are obviously extremely capital intensive.

No one's no one's really training these models except for the frontier labs. And we just consume them using the APIs. And harnesses are kind of similar in the sense not everyone here is gonna build a harness themselves. There's enough choice that you can pick from. So again, harnesses are also something that you don't control that often. It finally leaves us with prompts.

And again, with prompts, I will argue that there are two levels of control that you can exert. One is what I say is a level one control which is, you know, your regular prompt engineering techniques. You know, techniques like chain of thought chain of thought or a few short examples. And these are all great patterns, these are all techniques that we should all learn and we should definitely apply.

But I wanted to spend some some time talking about the level two control, which is essentially the policies that govern all of these prompts. Not just a not just a single prompt, but the policies that you can enforce across all your workflows. So the two techniques we're gonna talk about today, that one is called the constitutional prompting technique, which is essentially behavioral rules that you enforce on agents, which includes, you know, what actions are allowed, what's forbidden, what is an workflow contract, what is output contract certain.

And the second one is called epistemological prompting technique which essentially enforces analytical policies or rules which allow agents to you know gather evidence properly, you know, manage conflicting information or evaluating uncertainty, etcetera. Again, as I mentioned before, if we are not deliberate at this layer, the model or the harness will step in and enforce its defaults. And again, not something what we want all the time.

So every agent, as you know, is making a series of decisions while it's working. Right? It decides to often do things like what's allowed, what's not allowed, what's forbidden, which input counts as evidence, and when should I escalate or what's how should I interpret them? And what is the workflow order of operation?

Sometimes you you give it a few things it needs to do and it needs to figure out how to sequence them properly. And finally, the output shape. So these are all decisions that the agent does automatically. And and it's great. It does it. But you sometimes need need to see that control back to yourself. And and these these decisions have consequences, obviously, in your agent workflow.

And you don't see these decisions being made by the model. All you see is the consequences. And some of the consequences include things like the iteration tax, where you're spending multiple iterations explaining your point of view to the agent. Or you have a complex workflow, and then you end up with unreproducible runs or inconsistent outputs. So the key thing to take away here is that the agent is already using some form of constitution.

It's just not the one that you provided. So constitutional prompting is about making that very, very deliberate. So what does a constitutional prompt look like? On the left side, see it's not a lazy prompt. It's a fairly well written prompt. It says, just a security review on this PR.

It does have some steps that it's suggesting to do. But again, if you look look at it, it clearly tells you that it's it it looks like a request as opposed to a policy document. Right? And the agents are trained to look at it as a request, not as a policy. So what makes the the prompt on the right more powerful is the fact that, you know, there's a clear scope which you defined.

And there is authority in framing. There's also composition where the input and output are clearly clearly use a schema, which means you can chain these workflows together and have multiple a workflow execution happen. Again, the key point here is not to write complex instructions. That's not what I'm saying. But well constructed policies that you can pass on to agents.

And by doing this, the accountability shifts from the model's ability to follow instructions to how can we make make it follow better by using better policies. This is a real life example from the recent infamous Cloud Code leak that happened a couple of months ago.

And again, this is you know, this is this is a clear example of how they're using constitutional prompting. Again, there's a lot of stuff you can read on the Internet about how Cloud Code works internally. But as an example, you can see there's a clear activation function which is looking for, I think, user underscore type, is an environment variable. If it's not equal to ant, ant, I'm guessing is the demonym of Anthropic employees.

So if you're not an Anthropic employee, you cannot do all of these things. That's what the instruction means here. And it's very clear. There's a clear activation function, as opposed to saying, please don't run this. Okay. So, take a step back and let's look at what an idea of of a constitution looks like.

So, a complete constitution, in my opinion, has about five layers. There's clear rules and boundaries where you define who the agent is, what's what's off limits, when does a spec activate, etcetera. And there's clear input and output schemas, hopefully, where you're using enums and validations and failure conditions very clearly to the agent.

And then there is a body, which is the workflow itself, which has clear numbered phases of how the agent should sequence the work. And I often end up introducing state machines within that workflow using just something like a mermaid diagram, where the agent knows how to navigate the sequence of workflows that you're proposing. And and this is generally the layer that people skip.

They've spent spent a lot of time writing great roles and boundaries, but they forget to either specify a clear workflow or or the output contract. And that ends up being a thing that the agent usually misses and fills the gap with defaults and ends up drifting. So again, if you skip any of these steps, the model will step in and fill it fill in with defaults.

So I wanna switch gears here and talk about the second part of this prompting technique, is epistemological prompting. So a quick definition. Epistemology is essentially how we know what we know. It's more concretely, it's a branch of philosophy where we deal with theory of knowledge. So every claim that an agent makes usually has a theory behind it.

Right? So why is it justified? And usually that theory is invisible to us, because it comes from the model's own training data. Constitutional prompting, as we as we saw before, makes behavior very predictable, but it does not, however, automatically make reasoning or justification more predictable. And that's what we wanna tackle next.

So at every decision point, agent has to agent usually ask themselves. I have a new message versus this old instruction, which one wins? Or I just found this conflicting evidence. Should I update my beliefs? Or should I rationalize further? Should I research more? Should I escalate? There's some missing evidence.

Should I use my training data to fill in? Or should I, again, escalate to the user? So these are questions that the agent would, again, just by nature of its own training, fill in with defaults. So in facing decisions such as these, you want to you want to steer the model to your own epistemology.

Again, on a side side note, if you're interested in epistemology, I highly recommend this book by David Deutsch. It's called The Beginning of Infinity. It's it's an excellent read and full of ideas worth stealing. So, what is epistemological prompting? It's same as same pattern as constitutional prompting.

It's all about making implicit stuff more explicit to the agents. Right? This time, instead of specifying how to do things, we specify the agent's analytical framework, something that it can use to reason about evidence. I would be remiss if I didn't mention Matt Art's research here.

His work on epistemological prompting frames frames is that configuring an agent's analytical commitments, not just giving it task instructions. And in his paper, he argues how to use a few lens that you can apply to to the research agents to to get the optimal output.

A concrete example would be a positivist model, which is, in my opinion, the general model. If any model that you work with usually takes takes to tend a positivist approach, would just pick a database based on, you know, the best of the fast fastest benchmark. However, if you apply the pragmatic lens, the the model might actually kind of try to understand your team situation and figure out, given the team composition, can this even be safely run-in production, this this database, because you don't have the skills to run-in your team.

So, and hence, applying the right lens would usually change the outcome and sometimes dramatically. Okay. That was a lot of philosophy. And I do want to move on to some practical examples. This is this is again a condensed version of an actual research workflow in RP one. So in this con in this workflow, the agent reads the corpus, usually code and documentation from your repo, builds a position and reports it.

And and there are a bunch of epistemological prompting that's that's being added here. But the main the main distinction here is that I've taken a Bayesian stance here. So essentially, I've used the Bayesian thinking as as the main epistemological stance here to ensure that the models actually follow the evidence and they update their beliefs as opposed to just throwing them away.

And one of my favorite parts of this is to actually make the agents spell out what would overturn its own conclusion. So that makes it think more. That makes it, you know, find more evidence before it actually comes back with the report. Again, I think the the biggest the proof for this is that when we when we change the prompt to use a a Bayesian thinking, we suddenly saw 42% drop in false information being written into our knowledge base, which is which is actually the foundation of all the other workflows that we use.

So that was a big win. Again, tying it all together, there's there's usually two things that you wanna focus on. One is the constitution and second is epistemology. You can think of constitution as a body where the work is being done. The agents have agents are actually doing the work. Epistemology is is the mind or how it applies an analytical framework while it's working. And I want to leave you all with with this quote from Peter Noor.

It's it's it's basically says, code is artifact. So code is just a shadow of what is the actual program which lives inside the head of the engineer. And I think it's a beautiful code which kind of captures it captures our current current state of things. Agents only become useful when we can actually externalize the program that's in our head.

So externalizing this program into constitution epistemologies and useful workflows, and that's where I think the real leverage lies. So if any of this landed to you, on you, there are a couple of places you can go and explore more. Rp1.run, it's an open source Apache two point o licensed project.

It was built and it's heavily used at block, but it's not an official block project. It's an open source community project. The QR code will take you to Discord. Come out come and hang out with us. And, yeah, thank you for your time and attention.