Blog

Deep-dive guides on AI agents, agent orchestration, MCP, and developer tooling.

2 posts found

LLM Engineering

vLLM, SGLang, or TensorRT-LLM: What Actually Differs When You Self-Host an LLM

vLLM, SGLang, and TensorRT-LLM all promise fast self-hosted inference, but they make different bets on KV cache management, structured output, and hardware lock-in. Here's what actually changes depending on your traffic pattern.

October 8, 2026 · 9 min read

LLM Engineering

KV Cache Reuse and the Hidden Latency Budget of Agent Loops

Agent latency is usually a prompt-construction problem in disguise: prefix caching, continuous batching, and tool-call structure determine how much of your context gets reprocessed on every turn.

July 2, 2026 · 8 min read

New posts by email

One email when a new post goes up. No other mail; unsubscribe in one click.

We store your email only to send new posts. See the privacy policy.