In agent systems, instructions live across four surfaces — system prompt, tool schemas, tool results, and few-shot text — not one. Most prompt debugging still only looks at the first.
Prompt engineering used to mean one artifact: a block of text you handed to a model before its single reply. Agentic systems broke that model without anyone updating the term. An agent's behavior is now shaped by instructions scattered across at least four different surfaces — the system prompt, the tool schemas the model chooses between, the tool results that flow back mid-task, and whatever few-shot or error-recovery text got bolted on after something broke in testing. Most teams still write and debug these as if they were one prompt. That mismatch is where a lot of agent flakiness actually comes from.
This isn't a claim that agents need more prompting effort. It's that the effort needs to go to different places than single-turn prompt engineering trained people to look, and the failure modes are structurally different from a bad chat prompt.
In a single-turn chat completion, there's one instruction channel: the system prompt (plus whatever the user says). In a tool-calling agent loop, instructions enter the context window from at least four places, each with different authority and different lifespans:
User request
│
▼
System prompt (persona, policy, constraints) ─── set once
│
▼
Agent loop: model decides next action
│
├──► reads Tool schemas/descriptions ─── set once, re-read every turn
│
▼
Tool call
│
▼
Tool result (external, untrusted, freshly injected) ─── written by the tool, not the prompt author
│
▼
back into Agent loop ─── model must reconcile all four surfaces
│
▼
Final answer or next tool call
The picture below makes the asymmetry explicit: two of these surfaces are authored once and reviewed carefully, and two are generated live, outside the prompt author's control, on every single turn.
Two of those four surfaces — tool schemas and tool results — are exactly where most agent debugging time actually goes, and they're the two surfaces classic prompt-engineering advice says the least about.
The natural instinct when writing a tool schema is to describe the tool the way you'd document an API for another engineer: what it does, what parameters it takes. That's necessary but not sufficient, because the model isn't reading the description to understand the tool in the abstract — it's reading the description at decision time, to decide whether this tool is the right move right now, with incomplete information about what's actually going to happen if it calls it.
Compare two descriptions for the same function:
{
"name": "search_docs",
"description": "Searches documentation.",
"parameters": {
"query": { "type": "string" }
}
}
versus:
{
"name": "search_docs",
"description": "Full-text search over indexed product docs. Use this for factual lookups about existing features (config options, API parameters, error codes). Do NOT use for questions about unreleased features or roadmap — this index only contains published docs and will return empty results, not an error, if nothing matches. Returns up to 5 ranked snippets with source URLs; if you need more context on a snippet, follow up with fetch_doc_page using the returned URL.",
"parameters": {
"query": { "type": "string", "description": "Natural-language query, not a keyword list. Phrase as a question for best ranking." }
}
}
The first description will produce a model that calls the tool at plausible-sounding moments and mishandles the empty-result case by treating it as evidence rather than as a coverage gap. The second one preempts that specific failure because it tells the model what an empty result means, not just what the tool does. That's prompt engineering, applied to a 200-character field most teams write once and never revisit.
This matters more as the number of tools grows. With 3 tools, the model can often infer the right one from names alone. With 30, ambiguous descriptions compound: two tools that both "search" something, three that both "update" something, and the model is making a routing decision under exactly the kind of underspecification that a system prompt can't fix after the fact, because the system prompt doesn't know which tools exist to disambiguate between.
The third surface is the one classic prompt engineering has no framework for at all: content injected mid-task by something other than the prompt author. A tool result isn't just data — the model treats it as context that can shift its plan, and it will follow phrasing in a tool result the same way it follows phrasing in the system prompt, because at the token level there's no structural difference once it's in the context window.
This is adjacent to, but distinct from, the prompt-injection security problem of treating tool output as untrusted input. The failure mode here is more mundane and shows up even with fully trusted tools: an internal API that returns "status": "retry_later" with no further explanation gets interpreted inconsistently — sometimes the model retries immediately, sometimes it gives up, sometimes it fabricates a reason for the user. The fix isn't a security control, it's the same discipline as writing a good tool description: the result schema needs to say what the model should do with each outcome, not just what the outcome was. A tool that returns {"status": "retry_later", "retry_after_seconds": 30, "hint": "rate limited, do not retry more than twice"} removes an entire class of inconsistent agent behavior that no amount of system-prompt tuning would have caught, because the system prompt is written before the agent has ever seen that particular error.
Teams building against an external API for tool use — whether hand-rolled REST calls or something like Utilix's MCP server, which exposes its formatting and conversion tools to agents over MCP — get this almost for free if the API's error and success responses are self-describing. An endpoint that returns a bare 422 forces the agent to guess; one that returns a structured reason the model can act on turns a tool result into a working instruction instead of an ambiguous data point.
Single-turn prompt engineering leans heavily on few-shot examples: show the model three good input-output pairs, get a fourth that matches the pattern. In an agent loop, a "shot" isn't a pair — it's an entire trajectory: a sequence of tool calls, intermediate results, and recoveries from partial failure. Static few-shot trajectories in a system prompt teach the model to imitate a specific path through a specific environment state, which is exactly the thing that won't repeat. The environment state — what the search tool returns, whether the file exists, whether the API is rate-limited — is different on every real run.
What actually transfers is not the trajectory but the decision procedure behind it: what to check before acting, what counts as "done," what to do when a step returns something unexpected. That argues for writing agent system prompts less like a demonstration and more like a checklist or state machine — closer to an SOP than to a stylistic example. "Before editing a file, read it. Before calling an endpoint that mutates state, confirm the precondition holds. If a tool result contradicts an earlier assumption, re-verify before proceeding" generalizes across environment states in a way that a worked example doesn't.
None of this argues for spending less time on the system prompt — it argues for spending comparable time on surfaces that currently get none:
The common thread is that "the prompt" for an agent isn't a document anymore, it's a distributed contract across everything the model reads before it acts — and the parts of that contract nobody's reviewing tend to be exactly where the agent's behavior actually comes from.
Takeaway: if an agent behaves inconsistently and the system prompt looks fine, stop rereading the system prompt. Read the tool descriptions and the shape of the tool results instead — that's usually where the actual instructions the model is following are hiding.