The Agent Did Its Job. That Was the Problem.

Last week an AI agent tried to join DN42, the hobbyist network where people run real BGP and DNS for the love of it. Its operator gave it a goal: scan the network, comprehensively. Then one more instruction. Proceed immediately, without delay.

So it did.

It reasoned, correctly, that comprehensive scanning wants real hardware. It reached into its operator’s AWS account and stood up five m8g.12xlarge instances. Forty-eight vCPUs each. Load balancers. Lambda. Then it ran the deployment again. AWS sent a bill for $6,531, later reduced to $1,894 on appeal.

The agent didn’t break. It didn’t hallucinate a fake API or spiral in a loop. It did exactly what it was told, intelligently. It even stopped to ask for confirmation along the way. The operator waved it through. “Without delay” was, after all, the instruction.

The writeup shot to the top of Hacker News, which tells you the nerve it hit.

I work on agent authorization at Arcade.dev. I see a version of this every week, and it almost never starts where people think it does. When an agent goes wrong in production, the model is rarely the thing that failed. The thing around it is.

You didn’t delegate a task. You delegated authority.

Here’s the move nobody clocks in the moment. The operator thought they were handing off work: scan a network. What they actually handed off was authority: the standing power to spend money and provision infrastructure, on a real account, with a card attached.

Those are not the same thing, and the gap between them is where the bill comes from.

“Scan DN42” sounds like a task. Underneath it sat a pile of decisions nobody made out loud:

• how much compute is too much compute
• what it’s allowed to spend
• what it can stand up, and how many times
• what needs a human, and what doesn’t

The agent couldn’t see those, because nobody wrote them down. So it did what every system does when judgment is missing: it filled the gap with the most capable thing on hand. Five of the biggest instances AWS would sell it.

The agent had too much identity, not too little

It’s tempting to look at this and decide the agent needed to be more of a citizen. Give it its own account. Its own logins. Its own seat in your IAM system, so you can finally govern “the agent” properly.

That’s the trap, and it’s backwards. JertLinc3522 already had its own standing access: keys it held directly, on a real account, with a credit card behind them. That is a non-human identity with a budget. It is the thing that made the $6,531 possible.

An agent is not a new kind of employee that needs its own permissions. An agent is an app. And an app acting for a person should only ever do what that person is allowed to do, narrowed further to what this specific job needs, checked on every action. Not “what can this agent do.” Not “what can this account do.” The intersection: what is this agent allowed to do, for this user, on this resource, right now.

The DN42 agent didn’t have too little identity. It had too much standing access, and nothing standing between it and the action.

A credential doesn’t know what it’s for

The AWS account that spun up that fleet had no idea it was supposed to be scanning a hobby network. Credentials don’t carry intent. They carry power. Hand an agent a key that can do anything, and “anything” is now one confident reasoning step away.

This is the uncomfortable part, because it moves the blame. It’s easier to call the agent reckless than to admit it was handed god-mode and a vague goal and told to hurry. The agent was the cheapest part of this story. The expensive part was the standing credential.

What would have caught it

Not a smarter model. Not a longer system prompt reminding it to be careful. The thing that stops this is boring and structural. Every action the agent takes passes through something that knows what it’s allowed to do, right now, for this task, on this resource.

That’s the whole question, and it’s the one nobody was asking when instance number one booted. Can this agent, acting for this user, do this thing, here, now? Ask it on every action and the story ends differently. The agent reaches for “launch five 48-vCPU instances,” the request meets a policy that knows a scan doesn’t need a data center, and the call never fires. No instance. No bill. A logged “denied” instead of a four-figure surprise.

You write that boundary once. It holds on every action, whether the agent is being careful or being told to hurry.

The honest version

Two things this is not, because you’ll think them, and you’d be right.

It’s not a spending alarm. A budget cap catches dollars after they’ve started leaving. Authorization catches the action before it spends them. You want both, but only one of them prevents the thing.

And it only works if the agent acts through it. Give an agent a raw shell and root keys and there’s nothing in the path to say no. That’s the real lesson of DN42, and it’s older than agents. Don’t hand something powerful a credential it can’t be trusted to wield, and then look away. Put a layer between the agent and the action. Decide, once, what’s allowed.

The next one will do its job too

JertLinc3522 is going to have a lot of company. As agents stop summarizing and start acting (closing tickets, moving money, provisioning infrastructure), every one of them raises the same question its operator never got to ask in time.

The agent did its job. The next one will too. The only thing you control is whether anything is checking its work when it does.