AI coding assistants aren't one architecture — they're three separable design decisions (how context gets assembled, how edits get applied, how execution gets contained) that every tool from Claude Code to Cursor answers differently.
AI coding assistants get described as "an LLM that can edit files," which is true in the same way a search engine is "a computer that can look things up." The interesting part isn't the LLM — it's the three subsystems built around it that turn a model with a fixed context window into something that can act on a codebase with millions of lines it will never fully see: how context gets assembled, how edits get applied, and how execution gets contained. Every coding assistant on the market — Claude Code, Cursor, GitHub Copilot's agent mode, Aider, Windsurf — is a different set of answers to the same three problems, not a different core idea.
The naive fix for "the model needs to see the code" is a bigger context window. It doesn't work, for two separable reasons. First, cost and latency scale with tokens processed, and a 200K-token context re-sent (or even cached) on every turn of a multi-step task is slow and expensive compared to sending the 3,000 tokens actually relevant to the current step. Second — and this is the part context-window marketing glosses over — model attention degrades over long contexts even when the window technically fits everything. Needle-in-a-haystack evals show retrieval accuracy dropping in the middle of very long contexts, a pattern sometimes called "lost in the middle." A bigger window raises the ceiling; it doesn't fix the fact that a model reasons worse when 150K of its 200K tokens are irrelevant file contents it has to read past.
So production coding assistants don't try to fit the repo in context. They fit a query for the repo in context, run it against the filesystem, and feed back only what's relevant. There are three dominant strategies, and they trade off differently:
Agentic exploration (grep/glob-first). The assistant is given shell-like tools — grep, glob, list-directory, read-file — and decides at each step what to look at next, the same way an engineer opening an unfamiliar repo would grep -r for a symbol before reading the file that defines it. This is what Claude Code, Aider, and most terminal-based agents default to. It requires no pre-indexing, works correctly the instant a file changes on disk, and generalizes to any language or repo structure. The cost is turns: finding the right file can take three or four tool calls before the model has enough to act, and each round-trip adds latency.
Embedding-based retrieval (RAG-style). The codebase is chunked and embedded ahead of time; a query embedding is compared against the index and the top-k matches get pulled into context. This is common in IDE-integrated assistants that can afford to maintain a background index (Cursor's codebase-wide search, Sourcegraph Cody). It's fast per-query — one vector search instead of several tool round-trips — but the index goes stale the moment code changes, and semantic search reliably misses exact-match cases that grep gets right: rare error strings, config keys, deliberately obfuscated identifiers. The two failure modes are almost mirror images of each other, which is why several tools run both and merge results.
Full-repo-in-context. For genuinely small repos (a few thousand lines), some tools just paste the whole thing in, at which point the context problem doesn't really exist. This isn't a serious strategy for anything past toy-project scale, but it's worth naming because it's the implicit baseline every other strategy is optimizing away from.
The practical answer most serious tools converge on is "agentic exploration by default, with an optional embedding index as an accelerant" — because grep-first degrades gracefully (worst case: a few extra tool calls) while pure-embedding degrades silently (worst case: the model confidently edits the wrong file because the real one didn't score in the top-k).
Once a task needs real exploration — "find every place this deprecated function is called and migrate the callers" — dumping all of that exploration into the main conversation burns the context budget the actual editing work needs later. The fix several tools converged on independently is delegation: spawn a subagent with a narrow prompt, let it do the wide search in its own context window, and have it return only the distilled result (a list of files and line numbers, not the full grep transcript) to the parent. The parent's context stays small; the exploration cost is paid in a disposable side-context that gets thrown away when the subagent finishes. This is the same idea as a map-reduce fan-out, applied to protect a token budget instead of to parallelize wall-clock time — though it does both when the subagents run concurrently.
Once the model has decided what to change, it has to communicate the edit back to the system in some serializable format, and this turns out to be a surprisingly load-bearing design decision. There are three common formats:
Full file rewrite. The model outputs the entire new file contents. Simple to implement, trivially correct to apply (just overwrite), but expensive — regenerating an 800-line file to change one line wastes tokens and gives the model more surface area to introduce an unrelated regression by "helpfully" rewriting something it wasn't asked to touch.
Unified diff (patch format). The model outputs a git diff-style hunk with line numbers and context lines. Compact, and it's a format the model has seen a huge amount of in training data. The failure mode is that models are unreliable at line numbers specifically — off-by-one hunk headers are one of the most common LLM diff errors, because the model is tracking line numbers as tokens, not as a real buffer position. Tools that use this format usually apply the patch fuzzily (matching on context lines rather than trusting the header exactly) to survive small miscounts.
Search-and-replace blocks. The model outputs an exact snippet to find and the snippet to replace it with, with no line numbers at all:
<<<<<<< SEARCH
def calculate_total(items):
return sum(item.price for item in items)
=======
def calculate_total(items):
return sum(item.price * item.quantity for item in items)
>>>>>>> REPLACE
This sidesteps the line-number failure mode entirely — correctness only requires the search block to match verbatim, which the model can do by copying text it already has in context rather than by counting. The tradeoff is the opposite one: if the search block isn't unique in the file (the same three lines appear twice), the edit is ambiguous, and if whitespace doesn't match exactly, it silently fails to apply. Most modern assistants (Claude Code, Aider's default mode, Cursor's apply model) use search-and-replace as the primary format precisely because "the model got confused about which line number" was the dominant real-world failure mode in earlier diff-based tooling, and eliminating a whole class of error is worth the narrower ambiguity risk that replaces it.
The context-gathering and diff-generation problems are about getting the model good inputs and clean outputs. The third problem is different in kind: once the assistant can run a shell command, install a dependency, or call an external API, it's not just generating text anymore — it's taking actions with real side effects, on a machine, based on a plan the model itself constructed. That requires an authorization layer that's architecturally separate from the model's own judgment, for a reason that's easy to underweight: the model can't reliably distinguish an instruction from the user from an instruction that arrived embedded in something it read — a comment in a file, a webpage returned by a fetch tool, output from a command it ran. A permission system that asks "does this tool call match what the user actually authorized" catches that regardless of why the model decided to make the call.
In practice this shows up as three concrete mechanisms layered together: an allow/ask/deny policy evaluated per tool call (read operations often auto-approved, destructive ones like rm or a force-push always gated), a human-in-the-loop prompt for anything outside the policy, and sandboxing at the OS level — a container, a restricted shell, or an isolated git worktree — so that even an approved-but-wrong command has a limited blast radius. The permission gate and the sandbox aren't redundant: the gate stops the intentional but unauthorized action (the model deciding to push to main); the sandbox limits the unintentional one (a command that does more than the model expected, or a dependency install that reaches out to the network). Tools that skip one or the other tend to fail in exactly the way you'd predict — either constant interruptions for things a sandbox would have made safe anyway, or a sandbox with no policy that lets an agent quietly do something the user never approved, just inside a smaller blast radius.
The diagram below is the shape most of these assistants converge on, whatever their marketing name for each box:
Everything above describes a coding assistant operating on its own repo with built-in tools. The same architecture is what lets these assistants extend beyond the filesystem: Model Context Protocol (MCP) standardizes how an LLM discovers and calls tools that live outside the assistant's own binary — a database, a ticket tracker, a REST API — using the identical tool-call-then-permission-gate loop described above, just with the tool implementation living in a separate process. A coding assistant that can already reason about "call Read, then call Edit, then call Bash" doesn't need new architecture to reason about "call this MCP server's search tool" — it's the same interface with a different backend. This is part of why MCP adoption moved fast once it shipped: it didn't ask assistants to learn a new capability, it let them point their existing tool-calling loop at a much larger set of tools. A utility API exposed over MCP — say, a hosted set of validators and calculators an agent can call instead of re-implementing checksum or formatting logic inline — plugs into this same loop with zero special-casing on the assistant's side.
None of the three subsystems is solved in a way that makes the others unnecessary. Better retrieval doesn't fix an agent that still miscounts line numbers in a patch; a better diff format doesn't fix an agent that runs an unreviewed rm -rf because there's no permission gate. If you're evaluating or building on top of a coding assistant, the useful question isn't "how big is the context window" — it's how the tool answers each of these three questions independently: how does it decide what to read, what format does it use to propose changes, and what actually stops it from doing something you didn't ask for.