Tool misuse and loop detection
Agents call the wrong tool, call the right one with wrong arguments, and get stuck repeating themselves until the budget runs out. All three are testable, and all three are common enough to deserve dedicated cases.
You have seen someone get stuck in a revolving door. One more push, one more turn, and out of habit another push.
Nothing is broken. The door works perfectly. The person is just doing the same correct action forever.
Agents do this. They call a tool that fails, try again, fail, try again — politely, expensively, until the budget runs out or someone notices. And when they are not repeating themselves, they are reaching for the wrong tool entirely.
Both are cheap to find, and almost nobody looks.
The four ways a tool gets misused
Wrong tool. It calls search_docs when it needed get_order. Usually a sign the tool descriptions overlap or one is vague.
Right tool, wrong arguments. refund(order=A-1002) when the duplicate was A-1001. The most damaging pattern, because everything about the run looks healthy.
Right call, wrong moment. Refunding before checking eligibility. Emailing before the refund succeeded. Order matters, and trajectory rules are how you pin it down.
A call that should have asked. Anything irreversible or expensive that went through without a human. Often not a bug in the agent at all — a gap in the design nobody had decided.
What a loop looks like
Three shapes, all easy to spot once you know them.
The identical repeat. The same tool with the same arguments, over and over.
The two-step shuffle. A calls B, B's result sends it back to A, forever. Looks busier than the first and is the same problem.
The polite grind. Slightly different arguments each time — page 2, page 3, page 4 — with no progress toward the goal. Hardest to spot, because every individual call is reasonable.
Nothing errors. Nothing crashes. The bill goes up and the job never finishes. That is what a loop looks like in production.
Why you should care about this
Because loops cost money in a way normal bugs do not.
Every turn is a model call and a tool call. A loop left running overnight is a real invoice, and a loop on a customer-facing feature is a customer watching a spinner. Neither shows up as a failed test — the run technically "worked" until something killed it.
The second reason is that these bugs are found by asking, not by clicking. "What is the step budget? What happens on the third failure? Which of our tools are safe to retry?" Three questions, usually no answers, and the conversation that follows is worth more than the bugs.
How you test it
1. Break a tool on purpose. Point the agent at a service that is down, or an ID that does not exist. Watch the retries. Count them. This is the single most productive ten minutes on this page.
2. Ask for the budget and the policy. Record it alongside the run, the way you would any other release evidence. Maximum steps, maximum spend, maximum retries per tool. Get the numbers written down. A budget nobody set is effectively infinite.
3. List which tools are safe to repeat. Reading is safe. Refunding, emailing, creating, deleting are not. Any unsafe tool that gets retried without a guard is a bug waiting for a slow network.
4. Count repeats in every trajectory. A simple check: no identical call more than twice, no tool more than five times per job. It runs on stored runs and finds things nobody was watching for.
5. Test the give-up path. When it stops, what does the user see? "I could not complete this" is fine. A confident summary of work that never happened is not — and that gap is a separate bug worth filing on its own.
6. Watch step counts over releases. A job whose average steps climb from six to eleven after a prompt change is heading toward a loop even if nothing has failed yet — the same early-warning habit as watching your run history.
Try this today
Take one job. Run it normally, then run it three more times with something deliberately broken.
JOB: resolve ticket 4417 step budget: nobody knew <-- finding #1
RUN A everything working
6 calls, completed, no repeats ok
RUN B ticket service returns 500
get_ticket ×14, then "I was unable to access the ticket system"
14 identical calls, no backoff visible LOOP
RUN C refund endpoint times out (but the refund succeeds)
refund(A-1001) ×2 → customer refunded twice DOUBLE WRITE
RUN D ticket exists but has no customer attached
get_ticket → get_customer(null) → search_docs → get_customer(null) ×9
then: "I have refunded the customer." (no refund call was made) FALSE SUMMARYFour runs, three serious bugs, no special tooling — a broken staging service and a text file. Run D is the one to lead with: a false summary plus a loop, in a case as mundane as a ticket with a missing field.
How to show you know it
A retry count with a broken dependency. "Fourteen identical calls, no backoff." Specific, reproducible, and instantly actionable.
The double-write case. A timeout that produced two refunds is the kind of bug that gets a guard added the same week.
A safe-to-retry list. One column of tool names, one column of yes or no. Ten minutes to produce, and it usually turns out nobody had written it down.
A budget question in writing. "What is the maximum number of steps and who is alerted when a run hits it?" Being the person who asked before the invoice arrives is the whole point.
Questions
Is a loop not just a bug the developers will catch?
They catch the crashing kind. Loops do not crash — every call succeeds or fails politely, and the run looks busy rather than broken. It takes someone deliberately breaking a dependency to surface them, which is testing work.
How many retries is reasonable?
Usually two or three with a growing wait between them, and none at all for actions that are not safe to repeat. The exact number matters less than it being a decision somebody made.
The agent stopped and said it could not finish. Bug or not?
Not a bug — that is the correct behaviour, and worth saying so out loud. The bug is when it stops and claims success, or when it stops after forty calls instead of five.
Can I detect loops without access to the logs?
Partly, from the outside: watch latency and cost. A job that normally takes eight seconds and suddenly takes ninety is usually grinding. Structured trajectories make it obvious, which is why they are worth asking for.