Blog

Deep-dive guides on AI agents, agent orchestration, MCP, and developer tooling.

1 post found

LLM Engineering

KV Cache Reuse and the Hidden Latency Budget of Agent Loops

Agent latency is usually a prompt-construction problem in disguise: prefix caching, continuous batching, and tool-call structure determine how much of your context gets reprocessed on every turn.

July 2, 2026 · 8 min read

New posts by email

One email when a new post goes up. No other mail; unsubscribe in one click.

We store your email only to send new posts. See the privacy policy.