Blog

Deep-dive guides on AI agents, agent orchestration, MCP, and developer tooling.

10 posts found

LLM Engineering

LiteLLM vs. OpenRouter vs. Portkey: What an LLM Gateway Actually Does Between Your App and the Model

LiteLLM, OpenRouter, and Portkey all promise one API in front of every model provider. Here is how their routing, caching, and failover architectures actually differ once you look past the pitch.

October 27, 2026 · 9 min read

LLM Engineering

Prompt Versioning and Regression Testing: Why Prompts Need a CI Pipeline Too

A one-line system prompt tweak that 'fixes' one failure case can silently break three others. Here's how teams are starting to treat prompts as versioned, tested artifacts instead of strings in a config file.

October 14, 2026 · 9 min read

LLM Engineering

vLLM, SGLang, or TensorRT-LLM: What Actually Differs When You Self-Host an LLM

vLLM, SGLang, and TensorRT-LLM all promise fast self-hosted inference, but they make different bets on KV cache management, structured output, and hardware lock-in. Here's what actually changes depending on your traffic pattern.

October 8, 2026 · 9 min read

LLM Engineering

Claude Haiku 4.5 vs. GPT-5.4 Mini vs. Gemini 3.8 Flash: The Economics of Cheap Inference

The three labs' cheap-tier models now sit within a quarter-dollar of each other per million input tokens, so the real differences are in release cadence, cache economics, and how much of your "cheap" budget goes to invisible reasoning tokens.

October 5, 2026 · 6 min read

LLM Engineering

Fixed-Size, Semantic, or Late Chunking: Where RAG Retrieval Actually Breaks

Most RAG recall failures are decided at the chunking step, long before retrieval or reranking. Here is how fixed-size, semantic, and late chunking actually differ, and when each one silently loses the fact you need.

October 2, 2026 · 9 min read

LLM Engineering

How Constrained Decoding Actually Forces an LLM to Emit Valid JSON

Structured outputs and strict tool use aren't prompting tricks. They mask invalid tokens at the decoding layer so malformed JSON becomes mechanically impossible, not just less likely.

August 24, 2026 · 10 min read

LLM Engineering

Claude Sonnet 5 vs. GPT-5.6 vs. Gemini 3.1 Pro: What the Pricing Table Doesn't Tell You

A side-by-side look at pricing, context windows, and tiering across the three frontier model families as of August 2026, and why the spec sheet undersells how differently they actually behave in agent loops.

August 1, 2026 · 7 min read

LLM Engineering

Prompt Engineering for Agents Is a Different Discipline Than Prompt Engineering for Chat

In agent systems, instructions live across four surfaces (system prompt, tool schemas, tool results, and few-shot text), not one. Most prompt debugging still only looks at the first.

July 7, 2026 · 7 min read

LLM Engineering

Naive RAG, Agentic RAG, and GraphRAG: What Actually Changes Architecturally

RAG is not one architecture; it is three structurally different systems with different costs and failure modes. Here is what actually changes between naive, agentic, and graph-based retrieval, and how to pick without over-building.

July 5, 2026 · 7 min read

LLM Engineering

KV Cache Reuse and the Hidden Latency Budget of Agent Loops

Agent latency is usually a prompt-construction problem in disguise: prefix caching, continuous batching, and tool-call structure determine how much of your context gets reprocessed on every turn.

July 2, 2026 · 8 min read

New posts by email

One email when a new post goes up. No other mail; unsubscribe in one click.

We store your email only to send new posts. See the privacy policy.