Your Agent Doesn’t Like Your APIs

That's it. Okay. Thank you very much. So hello, everybody. My name is Mike. I'm an AI specialist with Amazon. I work for AWS. And I specialize in agents. So I work with agent framework teams and agent infrastructure teams. And I have built an open source library for the MCP Lambda handler, which is being downloaded about 25 to 30,000 times a month is being used in some production workflows for MCP.

And what I wanted to come here to talk about today is why your agents don't like your APIs and potentially what we can do about it. Let's see if my clicker works at least. Yes. Okay. I picked up some quotes and then I decided last minute not to say who said this, because it doesn't really matter. But we've been hearing this kind of thing today actually and in general. Just throw away, just throw data at the thing that is really good at dealing with data.

It's a good approach to this problem. If you're going to take a picture of this and try to de blur that, you won't actually see who's saying that. Is this my fault? Yes, it is. So that just says, a trusted person that we all like said this in a podcast once. Hundreds of tools is still a fairly safe bet, at least for Claude. Similar kind of thing.

This is what we're saying. It's all about spending as many tokens as we can. We all want as many tokens as we can. We should we all should be token maxing is is a statement which I hear a lot. Now, I don't necessarily disagree with any of this, but at the same time, I completely disagree with all of this.

And so this is just going to step into this just a little bit because there are there are two distinct ways that we use agents. And I've been building agents since slightly before we called them agents and agentic. It was all used and all that kind of stuff. I'd like to point out that there's two very distinct camps here, I think, anyway. That's agents we use versus the agents we build.

And when it's with agents we use, sure, go for it. Spend all the tokens. Do all kinds of things you can do to try to rip into your data and to make amazing application stacks and all the rest of it. But then when we're building agents, we really need to be very conscious about how many tokens we're using and about how we're using our agents to interact with data.

And so this slide I added very last minute, just after listening to everybody talking this morning. Yes, we are all about spending as many tokens as we can, some of us anyway, but not necessarily when we're building our own agents, when we're making agents for other people to use kind of makes sense. So I wanna introduce you to this.

This is an API and it's an API that I want to interact with. It's an API that I've made up for this session and I realize now that I've missed a trick here with the actual name of it. I get that, but it doesn't matter. This is a perfectly reasonable REST API that would pass many secure many reviews. You might not like it, but it's sort of the kind of thing that you'll find that you're working with, of course, that we're all working with this kind of API.

So I have this running right now on my laptop. And aside from all of those issues that I had with my laptop a second ago, I'm planning to do some live stuff. So let's see if that works. And this is the way that the API looks. It's a bit longer than configured on the screen. But it's, you know, it's a customer it's got customer information.

It's got invoice information. It's got payments. It's got all those bit different kinds of things. So this is a REST API and we wanna interact with this. We want our agents to be able to go and get stuff for us because we have this request. You'll notice I've changed the font to a business font because this is a business request where what is the status of the latest invoice for this particular customer?

Include full details, the amount, the due date and the description, etcetera, etcetera. Now, let's go and have a go at doing just that. And I'm sure this is all going to work fine. And it's going to be fine and it's going to be great. Let's see if I can can I scroll? Oh, I can. Yes. Excellent. Okay. So remember I was talking before about the agents that we use and the agents that we build.

So first of all, let's have a go at doing this with an agent that we use. So if you use a, you know, TUI agent for building code, I've got one here. I don't know if you can tell what it's called, but the developers of it decided to put it in really big letters at the top. So this is Kiro.

And so there's this works in a very similar way to all kinds of other kinds of CLI tools like Claude code and the rest of it. So let's so how do I do this then? How do we get to my oh, gosh. Oh, Okay. Here we go. That's going to be fun. And so I've got a prompt here that I'm going to give to that thing. I'm going to scroll through very quickly.

There we go. Okay. Cool. Okay. So this is essentially that natural language version of that business prompt that we had before. But I do just need do need to tell it about where my API is so that it can go and find it. So let's run that and see that run. So more pressing enter, less talking. Remind me of that as we go through this.

And so it seems to be kind of working. I think this is good. It's going to hiccup a little bit here because it is just a local API. Oh, by the way, on the left hand side, this is the the essentially the logs, if you like, of the API. So we can see that it's actually hitting something and doing something. That little hiccup is not something I'm very concerned about.

It's currently using the hotspot in my pocket because I couldn't find WiFi. I don't know if everybody else has managed to find it. But hopefully, it'll manage to get through here and and get the data. So this is just a a quick fast API that's running on my machine. The data is actually hosted up in the cloud somewhere because I can't help myself and that's what I do.

But there we go. So it's churning through it and it's going to go and get the it's going to get the answer for us. And so, yeah, it's managed to find this person. Sample data, don't, you know, read too much into this. But there is an enormous invoice there for coffee. But we're in Melbourne. Right? So that's that's that's I'm I'm reaching out to you guys.

I'm from Brisbane. We don't drink coffee up there. It's taking a little longer to to run than I was hoping, but that's again probably because of the the the WiFi and stuff. And so we've got an $18,000 bill for coffee. Excellent. So it's managed to do it. Right? So what did I do? You saw the prompt that I gave it.

So I pointed it at the API. I said, go figure it out. And what did it do? It burnt a whole ton of tokens to be able to go and figure it out with an agent that I'm using. Let's go and have a look at an agent that I am building. So let's quit out of this. Let's clear that out and then I'm gonna scroll all the way over here and get this thing because I don't like live typing.

You've seen how good I am at PowerPoint. Imagine what it would be like if I never mind. Okay. So if we're going to paste this in here, I'm going to run an agent. Now let me go and find where that agent code actually is so I can actually show you what's running here. So here we go. So this is the Kiro IDE.

It's the IDE version of the CLI we were just looking at. And here, I have I'm sure I'm looking at the right one. The agent that I'm just running in the background now, but for saving time, I'm gonna walk through what the code looks like. And so you can see that it's strands agents SDK. So it's an open source SDK for building agents.

It's pretty cool. I like it a lot. You can see that we've imported agent and tool. I've imported lots of other things as well because I want it to pretty print some stuff at the end just so that we can see what's happened. But I'm pointing it at my API And if I scroll down here, then I have a tool definition.

So tool definitions are very easy with all of these frameworks, just a bit of a decorator and I can say, okay, well, I've gonna have list customers, get customer, blah blah blah blah blah. For each one of my APIs, I've got a separate tool which is going to go and work with that particular API. It's a one to one mapping.

So what's gonna happen? A very similar kind of thing to the where we just used our command our Kiro CLI version. So if I just scroll down here, there's a bunch of stuff in here about making it look pretty and then you'll see why in a minute. But if I scroll down to here, here is the agent itself. And in the agent itself, you've got all of those different things listed.

So we just say, here are all my tools. Here's a system prompt for you. And then I'm actually basically hard coding this agent to just go and find the answer to that question that we were talking about just a second ago. Okay. Let me scroll all the way back over here and it's finished. Fantastic. And so you can see the pretty stuff that it's done at the end here.

And so it's managed to get the same answer out that this person owes a lot of money for coffee for some reason and we look at the amount of tokens that it's estimates that it's burnt on this. So around about 15,000 tokens burnt on doing just this. So if I'm using my my coding assistant agent, my Kira, my Claude code, my whatever it is to go and hammer on my API, fine. Okay.

I'm maybe I'm okay with that. But if I'm shipping this out to a 100,000, 200,000 people as part of a mass deployment of my agent, then this is awful. Right? So I'm spending way too many tokens. We need to slim this right down. Okay. Let's go back to here. Let's see what happens if I press that, And let's let's keep going. I'm going to make one of the most obvious comments that you ever see said today.

Agents are not human. But okay. But let's think about what humans do for a second when it comes to them using data. So humans, us, we don't use APIs. So why do we make an agent use them? But we don't use APIs. Right? We don't. Like, if I'm asking you that question, you would probably go to a dashboard.

Ultimately, as a developer, maybe I'm gonna use an SDK. Maybe I'm going to use a library, but I'm more likely to go and use a user interface to be able to work with that data. I'm gonna go to some kind of dashboard to work with that data. And agents want that too. Why do they want that? Well, agents wanna be just like us.

Now, why is that? I mean, under the hood of an agent is an LLM, and the large language model has been trained on natural language, our natural language, the natural language that we've come up with. So it thinks and behaves sort of like us. It wants to work with the same kind of the data in the same kind of way that we work with that data.

Can it work with the API? Yes, of course, it can. I'm sure you could as well if I gave you long enough and, you know, postman and whatever and you're just hammering away and you're getting the data results back, but you don't want to work like that and the agent in its in its happiest state, if you will, doesn't wanna work like that either.

It wants to be just like us. And so the tools that we provide it are essentially the agent's UI, the the the user interface for the agent. That's how it gets access to the world around it is through those tools. So I want to present the four principles. This is a very grandiose thing. I'm sure there are some more principles out there we could gather as well that we've seen, that I've seen, that we've collected together to say, okay, well, this is how we wanna think about it when we're designing tools for agents.

And we'll see this done in just a second. Some of these might be kind of obvious, but it's kinda good to sort of like frame it like this. So rather than having that massive, great, big, long list of all of our APIs, the APIs that were built probably ten years ago and felt good at the time, we want to surface these.

We want to give these to the agent in a much more coarse grained kind of way. This sort of steals my thunder for what I how I want to finish this presentation, but I saw this this morning when we're talking about being output or outcomes oriented rather than, you know, token token spending. So we want to be outcomes oriented, not operations oriented.

Get ourselves some coarse grained tools. And the definition of these tools is going to depend, of course, on what it is that we're looking and focusing the agent to do, but giving it access to be able to combine together different API calls so that it can become much more efficient. It's kind of obvious when I say it like this, but, you know, if you've had any kind of gray hairs like I do and you've thought about service oriented architecture and stuff, then I feel your pain as well.

But it's a similar kind of thing to that, right? So next thing, principle number two, is to flatten the arguments. Now this is really just about making sure that the agent has got access to as rich and as concise information about how to go and use the the tools that we're giving it. So just skipping to the what is good, clear, typed, constrained, annotated, described. So we've got the types there.

We've got the enums there, so it doesn't have to guess what parameters are sent. It's easy to be able to make a simple tool, throw it together, and give it to an LLM or give it to an agent. And it will figure it out, but you're paying for all of those tokens whilst it's figuring it out. No need to do that.

Let's go ahead and and put in these details here. And of course, we can use agents to write this stuff, but let's just make sure it's there. Something I missed from that, I suppose, is the description of the tool. And I'm sort of saying that's kind of table stakes at this point, like docstrings or however you need or can add in the description of your tool.

But don't forget, it's not just about that. All the information you can get into the context, this context engineering, super important, and there's lot of things you can leverage here. Something that I'm gonna point out here is the errors that you send back because not all calls to these tools are gonna be successful. But saying four zero four not found is not necessarily that useful for your agent, not just as it wouldn't be for you.

You know, you'd have to then go and look in the documentation or find some esoteric error message and then just Google it and then go, okay. They don't know what it is either. Whereas if you can actually pass more context back at all times, even during the error, not entirely sure that the one in great makes a lot of sense, but maybe it does in this particular business case. I don't know.

But giving information back to the large language model behind the agent. It's going to read all of this stuff and it's going to use this to figure out where to go next. So in a sense, the last one I'm just going to say here, curate ruthlessly. Make sure that at each stage, you're actually thinking that we're not pushing this too far.

So yes, I even I put a slide up before saying, you know, 100 tools into a tool server, not necessarily a problem. It depends on a few things. But if you are getting to more than 50 tools in a server, something's probably gone wrong somewhere, and you need to start thinking and refactoring and and pairing that down.

Doesn't mean you can't have another tool server. The number of parameters. The response size. Huge one, this one. This is actually something that we've learned a lot is when you're responding back to an agent call, you don't necessarily need to give it all of the data it asked for. You could, and there's a lot of power in saying back to the agent, yep, I did that.

You asked for that and I've done that. Not only have I done that, but all of the data you want is in this location. Go get it at a point when you want it in the future. So you don't have to stuff the context with that massive 200 gigabytes worth of data. Obviously, that wouldn't work. Okay. So let's look at that what that might look like in my strands agent world and I'm gonna try where where am I?

Yes. Okay. So let's go back over here and over here and here. So I have a fixed, hopefully, version of this agent. So let's do take a quick look at it. And again, it's a strands agent, so it's pretty simple code. And I've got my tool decorators on my tools here. So GET latest invoice status, much more descriptive, much more outcomes oriented, got a lot more description about what it is that I want my tool to do.

And of course, it's then going to go and potentially knock on the door of a couple of different API calls to be able to get this information back and then be able to add that context to the data when it brings that back. Same with find overdue invoices, same with these smaller number of tools. And then my agent itself is down here.

So similar kind of thing. Yes, I know I'm using an old model, by the way, but that's kind of even not the point. Still works and I've got a smaller number of these outcome oriented tools. So let's whiz to here and here and this is the last one. So let's see if this works. And come all the way back to here and let's clear this out. Has actually worked, hasn't it?

Oh, I said that before I ran it. Come on. Okay. So we're going to run the same thing again or a similar kind of thing again but this time that version that we just looked at with the same question going in there. And look, comes back with the answer straight away. What a surprise like I would have done it otherwise.

And we've spent about 50 tokens rather than 15,000 tokens. I know that if I'm shipping something to my customers and ultimately, we're going to have to figure out how much this thing's going to cost, this is the way that I want to do it. Okay. Let's go back to to wrap it up. Had my sign held up to me just a second ago.

Elephant in the room that I haven't talked about, MCP. Obviously, if I was actually going to ship this out at production scale like I'm talking about, then there's a lot of things that I need to do to this to this agent to get it to work at cloud scale. MCP, definitely the way I would go for my tool usage, and I would definitely be looking to put that into an MCP gateway with my authentication to be able to host this at scale in the cloud. Not a surprise to any of you, I'm guessing, if you're looking at my shirt.

At AWS, we do have Amazon Bedrock Agent Core, which is a massive collection of different components and pieces you can cherry pick from to be able to take your agents and then cloud scale them and take them out to your breakout success. One of them is gateway. I'm not going to jump into the console now, but if you go into the AWS console, this diagram is in there, so you can pick it apart.

Gateway is a fully managed MCP gateway service. So if you've got APIs, you could put them behind gateway and make them available to everybody. Should you do that? Well, hopefully, we've just been talking a lot about that. Maybe not. Maybe So put some Lambda functions there to be able to combine together your API calls. Bits and pieces all there.

Please go and come and see me later if you'd like to know more about any of these components. But other than that, I will move on. So we all should be token maxing. I kind of spoiled my own ending here. I think we should all be outcome maxing. I'm not even the first person to have said that today, and I know that's a contentious point if you go and Google it.

But in terms of the agents that we are building, we should definitely be outcome maxing. Just some QR codes for those who are aligned at the very end. If you are interested in playing around with Strands agents, it's free. It's open source. I love it. Then there's a QR code there, strandsagents.com. If you don't get it now, you could get it later.

It's very easy to find. If you're interested in Amazon Bedrock Agent Core, I'm not going to send you to the product page because that doesn't speak to us, does it? But if you go to that QR code, it's a GitHub page with sample code in there, which takes you through all of those different pieces and componentry and all the rest of it and including that gateway service.

If you'd like to carry on the conversation with me, this is my LinkedIn QR code. So this is the one you take a picture of. Yeah. So please feel free to connect with me. I'd love to talk to you more about this kind of stuff. And very finally, this is more or less the code that I just showed. If you want to sort of pick it apart and play with it yourself, there's a couple of differences in here, but it's all based off of this repository.

Thank you so much. It's been a pleasure. I look forward to talking to you again at the conference. Thank you.

Thank you, Mike. We're actually running sort of five or six minutes ahead of schedule. Can I yeah? So I was wondering, can we can we throw you under the bus for a few for some q and a's? Does anyone have any questions for Mike?

Yeah. Yeah. Yeah. I was watching the timer down there. It went red. So I was like, hey, that's why

I was Yeah, no, I think that the timer was fine, but I think I started you early. Does anyone have any questions for Mike at all while we got in?

Why? That's a question.

Someone.

Oh, come on. Some one person has just one question, even if it's a frivolous question. What's the meaning of life? Yes. We'll have to go and find out. We'll get clawed on that one. Yeah. Question here.

You're showing the design, the recommended tool design. And on the right, you had tools that had a lot of arguments. On the left, you had a lot of tools but without arguments. One of the principles that you've been sharing.

Okay. So this is on one of my slides? Okay. You picking holes in one of my slides? Yeah. I mean, I would. Yeah, yeah.

You're suggesting that the tools we have less tools but more arguments. Doesn't it lead to quality degradation if we try to call those tools? Because now arguments needs to be picked by LLM whereas before, there's no arguments needs to be picked by LLM. It just chooses the tool.

Yeah. So so it's a good point. It's a balancing point. Right? So I guess it's about outcome, defining what the outcome is that you want. Again, if if we are going in there and being very specific about what those arguments are going to be, then all of that stuff's well within the bounds of what an LLM is going to be able to manage inside of the agentic loop, right? So it's a balance.

If you end up with, you know, 50 different parameters going in, then all of a sudden, hey, this isn't working for us. But I think one of the one of the four up there was just looking at when we were talking about ruthlessly refactoring, making sure that you don't have too many arguments. So it's it's correct that it's a balance.

But, you know, the way that that API was designed was in a pre agentic world and it was pretty ugly intentionally to make the demo work. And so, yeah, it was it was more efficient in this particular case. Yeah. Yeah. If there's anybody else putting their hand up, I can't see you because of the lights. You have to shout.

But the question here.

What would you say is this quickest way to wrap the API calls to something that agents could consume easiest?

That's the quickest way to wrap API calls into something that an agent can consume most easily. So using an agent probably to help you do that for a coding assistant, why not? Let's be honest about it. So the thing is to then is to actually go in to look at what the outcomes are that you actually want to actually get. So there is a bit of sort of like business analysis kind of stuff going on here where you want to define in the same way that you would for what does a human want to see when they look at a dashboard of this data or what does the agent wanna see?

And what the agent wanna see is probably related to what a human wants to see because a human's gonna be using the interface probably, but not necessarily. So, yeah, doing a bit of that analysis, getting an agent on board is probably the way to do it. One thing I will mention as I have time and because this gets me paid is because it's that gateway product that we talked about. One of the things it has is the ability to do semantic tool lookup. So if you do have and in in a longer version of this presentation, I would say and demo that if you do have like hundreds of API calls and hundreds of things that you simply can't refactor or for whatever reason you can't, then you can put it into agent core gateway.

And then gateway has a feature on it which allows the agent to be able to do semantic lookup of those tools. So it's a tool discovery thing which sort of sits in between it and then having hundreds of tools in there, which is pretty cool.

Awesome. I think we had one more question Yeah. From the gentleman in white. I can't quite see. You might have to shout.

Yeah. So if if you're building an agent which is going to have some sort of business use case, then making those outcome oriented tools and then having natural language based around them, yes, I'm finding that's working. If if what it is is if what you're doing is I have a general purpose agent which just has to deal with everything in my life from my emails to my Slack to my API, then it will get there and you'll have to spend a bunch of tokens to get there.

This is all about writing an agent that I am going to deploy to the masses, and they may not even know they're using an agent at that point, I suppose. So, yeah, I'm finding natural language is going to be beneficial there, especially with the things like those error messages to sort of explain what is the current situation.

So it doesn't so it just eliminates the guesswork. It will get there anyway, but we're just trying to get it there quicker. Does that does that answer that? Does it make sense?

Yeah.

Okay. We can talk. Okay. Cool. Awesome.

And if anyone's got any other questions for Mike, catch him afterwards. He'll be hanging around. So one more time. Let's put your hands together for Mike Chambers.

Thank you very much. Thank you.