Are Your AI Agents Secure? Defending the Privileged Agent

Hi, everyone. Oh, is mic working? Oh, no? Oh, okay. Is it working? Okay. Cool. Hi, everyone. Thanks for being here. Before me getting into the topic, you know, I'm based in Tokyo, Japan, and I came from the Japan Tokyo, actually. Then on Monday, it was 30 degrees.

And here, it's like 10 degrees. So I felt freeze, frozen. Yeah. So anyways, let's get started. So I'm gonna talk about the how to defend in the privileged agent. So let me get started. So first first, I'm gonna talk about the problems, how agents typically get access today and why it tends to go sideways.

So AI agents as autonomous colleagues, you know, it they do jobs for you, you know, integrate integrating reasoning on the actions or scaling beyond the system boundaries, you know, sending emails, write code, exit code, book a meeting, something like that. So they are behaving the proxy for the users, which means that you giving out your users privilege for the agent.

Typically, you are gonna give the OAuth token for the users and give it to the agent so agent can call external APIs with the user's credentials. So but here's the challenge. If you're passing the user tokens directly, which means the GIP who user previous to the AI agents, and also the AI agents could be the impersonator for the user, So it's hard to distinguish whether user did actions or agent did actions.

So it's gonna be difficult to log in and audit things when it goes wrong. The second thing is if you give a token with the permissions with the scopes, usually it's static. So if you are giving the drive read permission to the agent, it can have the whole access to your drive.

So whenever we I ask, hey. Give me summary of the ex my expense report. The agent can technically read everything and could access to the financial record on your drive, which is going to be dangerous for you guys. And start one is more critical. So if you give the permissions or privilege or token with the permission to the agent, when the platform is compromised or bad prompt injection happens, the attacker can inherit the the tokens that our agent has, and they can use the token with the different purposes.

And, you know, this is not a shortage of threat. You know, we've been observing this happened in the last two years. I don't call exact examples, but you know of what they are. So this isn't just my analysis. WASP has been documenting these patterns and the risks, also mitigations.

And somehow, in previous talks, somebody talked about this report. So I'm not go I'm not going to detail about what they are, but I'm wanna focusing on the number six, excessive agency. So this is the risk that unexpected LM out of us into a real harm via excessive privilege. There are three types.

One is the excessive permissions, which agents hold a broader access, like I mentioned. You know, read and write when only read is necessary. The second is excessive functionality. The toolbox contains somehow dangerous tools. It happens if you have the, like, a super agent.

It can does everything just in case the agent has the tool called the delete user tool. Yeah. It's very dangerous for the agent. And last one is excessive autonomy. You know? Hey. I wanna let agent does for everything for me so you can do whatever you want without asking me for the permission. This is powerful.

This is very productive, but it cause problem for you. Also, Rust published the latest top tens back in December last year, I guess. This is a very new report, and here's one for the identity and the previous abuse.

So basically, it's talking about what I've been talking to you, you know, uncoated, trans spread spread of privilege, which covers the not only for the single agent but with the sub agents. So sub agents contain the permissions that it shouldn't have or agent to agent to com agent to agent communications kind of things.

And also talking about the independent IDs. So I said the agents are proxy of the user. No. You need to have in the independent ID for agents so that you can log what agent did and you can audit them. So this is the latest report that I saw.

So I've been talking about the risks and issues, but I want to talk about how to mitigate those risks by dynamically authorized agent actions and also including human in the loop thing. So whenever you are talking about authorization, it's going to be like access control. So are you gonna use RBAC?

This is a traditional approach. You have the group or users, roles with the permissions. It's very effective if it's application or API level of the management of the access. But the challenge is granularity. Like I said, you know, drive, read cannot really control that agent can only read certain documentations.

If you wanna make it happen, you need to add attributes, and you need to be facing the ex explosion of the roles. So I want to introduce a different approach with the authorization. It's called rebac, relationship based access control. Instead of the roles and the permissions, it holds relationship between the entity and object. So something like, does the agent have a sufficient relationship with a specific object?

And this is not embedded by somebody else about that, you know, for example, that Google uses this one inside of their product, and they published a paper a couple years ago called Google Zanzibar. If you are interested about the Google's paper, you can search for the paper, or you can go to the Zanzibar Academy.

And I'm gonna explain a little bit about the Reebok. So first, you're gonna build the authorization model. So top left. Now here's the doc documentation, has three relations, editor, viewer, owner. And you can you also hold the authorization authorization data as a tuples. Kim is the owner of the road map documentation, or Ben is editor of the road map road map documentation.

Carl is a bureau of the slides, something like that. Then at runtime, you can query the authorization engine to whether this user has relationship with a specific object kind of things. So the top down right is is Carl is a a bureau of that size. Answer is yes, which means the he can access to the documentation. With this, you can implement dynamic authorization inside the URL to the calling or the workflow.

One example is the inside of a log pipeline. Whenever user queries, hey, I want to want to get the project information, Usually, Rag retrieves data semantically. Then you can inject the RBAC authorization decision inside of this workflow so that whether this user has the relationship with these documentations as a viewer. If not, it's going to be filtering out.

So the data never gets to the model. Another examples. So AI agent wants to do something. You know? It calls tools or it calls MCP, MCP servers. Inside of the tool calling or inside of the, you know, MCP server business logic, you can inject VBAC authorization engine.

So for example, hey. I wanna have the documentation secret. I wanna read it. I know. Want to answer user's input. Then you can check user, in this case. Agent Alice has relationship as can read with the object document secret documentation. So in this case, you know, you can have the agent ID and the identity instead of the user identity.

So if you set up the authorization model with that agent, you can check whether agent has the access to the specific documentation. And depending on the answers, yes or no, it proceeds or it denies. So the point is AI agent doesn't have really permission or privilege at the beginning. Whatever it's called, it it it's it's calling the tools. You can check the permission, you know, on the fly.

With this, still, you need to have the somebody else to look at the agent's activities. So it's something like, yeah, I wanted to refund for 5,000 USD or whatever. Inside of the approval logic, you should have the ask for users' approval when the amount is exceeded today's threshold.

Let's say, yeah, over the $100. No. I need to have someone else to take a look instead of the agent does four things. So with this one with this, you know, even with the, you know, dynamic authorization at runtime, you can check with the human as a RAS defense.

So to, you know, the mitigate the risk of the excessive agency. But I wanna point one thing. You know, is asking every time okay for you guys? How many of you have used cloud code and wanted to use dangerously skip permissions?

I see smiley faces. You know? First five time, you know, I watched, you know, what they are asking for. The after five times, I don't care. It's just s o s o l. So or like a cloud then you just keep permissions or something like that. So, you know, you don't wanna ask users every time. Just you need to have a clear reason to ask for their permissions.

Yep. So I built a a short demo to demonstrate how dynamic authorization works. So let's get started. So I built a, you know, support ticket agent. Left side is the no safeguards. It's only asking it's it does actions what they do. And right side is the with the dynamic authorization.

And this is a multitenancy scenario. So I I'm logging to the SaaS product. So I wanna list the tickets available right now. Hey. Show me the old tickets that need attention. So it's trying to retrieve the list of the ticket that I have. Hey. Yeah. You get the list.

It's good. It's good. Wait. I'm from the Pixel Zero company, but I see the tickets from the comfort zone. It's clearly the the security breach. The right side is the only the pixelated tickets. The point is, you know, the without the safeguard, it can go every ticket goes to the model.

But with the dynamic authorization, it only goes to the ticket that a user has access to. Actually, in this case, agent has access to it. So like this one, you know, probably model can filter them out, but, you know, the point is you don't expose unnecessary data to the model. The next one is the deleting ticket, which is dangerously, you know, create a tool.

Left side is just laying in the data as it re requested. The right one is, you know, it's checking whether the agent has the capability to delete the ticket. So in this case, you know, they're checking the with with the VBAC authorization engine. Hey, agent, can can agent delete the ticket?

No. I don't have the demo in the video, but if you are behaving like as a user, it kinda actually did the tickets inside of this demo. And I wanna show the last thing is to going to refund the money. So, yeah, I wanna refund the $3.03 $3,500 to the customer because it's not working.

So with the, you know, no safeguard, it's just refund. So it's too much power by the agent. But the right side is, you know, I'm trapping this one with a threshold. Hey, it's over $100. Yeah. I need somebody from human to approve this one.

So, yeah, I'm gonna open up the yeah. Go ahead. Go ahead. Yeah. It it just hold the actions while, you know, they're waiting for the approval. Yeah. Go ahead. It's not, you know, live demo, so I need to wait. You know?

While I'm waiting okay. Okay. Yeah. Actually, that CEBA is triggered. CEBA is the client initiated back channel what is the author authentication? Authorization. Oh, I think authorization. Yeah. So this is getting the this is just a simulator. So the Carol is the manager of the Alice.

It's a real human, got the message. Hi. Alice wanted to refund this one. Is it okay? Yeah. It's okay. Then a manager approved the request. So with this dynamic authorization and also human in the group can safely guard the agent behavior.

And can actually, you can use model to guard everything, but it's undeterministic way. But this is deterministic way because you have the authorization model, authorization data to make decisions. So to the summary, here, I talked about AI agents or Autonomous colleagues. It's a good colleague for you, but their excessive privilege can lead devastating damage. Yeah.

Last summer, something happened. Yeah. Move from static permission to relationship based based dynamic authorization to ensure least privilege. Like I said, agent doesn't have the any privilege with the dynamic authorization. You can give that privilege at one time, and you can strip them out after the executing code I mean, executing tasks.

And for high risk operations, always include the human in the loop as the final safeguard when it's necessary and appropriate time. So today, I showed a demo with you know, VBAC, the, and it's actually built with the this product called OpenXGA.

It's open sourced VBAC engine that you can use for free. If you're interested, can check with the OpenXGA. And if you scan this QR code, this is this is a page that will actually discuss how to implement dynamic authorization with the OpenFGA. Also, we provide the other features, including this dynamic authorization for the AI agents and security.

And if you're interested in that, you can check with our page as well. That's a wrap. Thank you for coming today, and, you know, enjoy the rest of the conference. Thank you.

Quick question, if I may, while we're changing over, But you can unplug. Yeah. How much do we really trust the humans in the loop? Because we've seen situations where there's a human in the loop, but their context is so limited. And they and like you said, I'll just say yes because I've said yes to the last five that I actually checked.

Yeah. Oh, is Michael along? Okay. Yeah. So like I said, you shouldn't ask every time. And you should include the context as much as you can. That's the key. Otherwise, it's going to be ignored.