A JSON array requires seeing the closing bracket before any of it is valid — which is exactly what breaks when you stream millions of records. Newline-delimited JSON fixes that by making every line a complete, independent document.
Here's a mistake that looks fine in dev and falls over in production: an API endpoint streams records as a single JSON array — [, then one object per row, then ] — and a client reads the response with a normal JSON.parse() once the connection closes. It works at 500 rows. At 5 million rows, either the client runs out of memory buffering the whole array, or the connection drops halfway through and the client is left holding a truncated string that isn't valid JSON at all — not even partially. You can't recover the 4.9 million rows you already received, because the last thing you have is [{"id":1,...},{"id":2,... with no closing bracket, and standard JSON.parse refuses to touch it.
This is the exact failure mode NDJSON (newline-delimited JSON, also called JSON Lines or .jsonl) exists to avoid. The fix isn't a smarter parser — it's a format where a partial read is still useful.
A JSON array is one value. Structurally, [{"a":1},{"a":2},{"a":3}] is a single tree, and the JSON grammar requires you to see the matching ] before the document is well-formed. A parser can't hand you object {"a":1} and consider its job done for that chunk — technically it needs to keep tracking bracket depth and string escaping until the array closes, because a truncated document is not a smaller valid document, it's an invalid one.
NDJSON throws out the wrapping array entirely. Each line is a complete, independent JSON value:
{"id":1,"name":"Ada Lovelace","role":"engineer"}
{"id":2,"name":"Alan Turing","role":"engineer"}
{"id":3,"name":"Grace Hopper","role":"engineer"}
There's no top-level [, no trailing ], and critically, no commas joining the records. That last part matters more than it looks — commas are a synchronization problem. If you've ever hand-written a streaming JSON array and forgotten to omit the comma after the last element, you know the failure mode; NDJSON just deletes the entire class of bug by making every line self-terminating.
Reading NDJSON is a loop over lines, not a stateful parse:
const readline = require('node:readline');
const fs = require('node:fs');
const rl = readline.createInterface({ input: fs.createReadStream('events.ndjson') });
for await (const line of rl) {
if (!line.trim()) continue;
const record = JSON.parse(line);
await processRecord(record);
}
Each JSON.parse(line) call is O(1) with respect to the rest of the file — it has no idea how many lines came before or after it. Memory usage is bounded by the largest single line, not the total file size. If the stream dies after line 4,900,000, you've successfully processed 4,899,999 complete records and lost nothing but the one in flight. Compare that to the array version, where a full streaming solution requires an incremental tokenizer (libraries like stream-json or oboe.js exist specifically because JSON.parse can't do this) that tracks nesting depth, distinguishes structural commas from commas inside string values, and buffers partial tokens across chunk boundaries. It's not that streaming a JSON array is impossible — it's that it requires meaningfully more machinery to get the same guarantee NDJSON gives you for free: a line boundary is a safe place to make a decision.
This is also why NDJSON tolerates malformed input gracefully in a way arrays don't. One corrupted record in an NDJSON file is one JSON.parse throw you can catch, log, and skip — the other 4,999,999 lines are unaffected. One corrupted object in a JSON array — a stray control character, an unescaped quote — and the entire array fails to parse, because there's exactly one parse operation covering the whole thing.
This isn't a theoretical format. It shows up anywhere records need to be produced or consumed one at a time at scale:
.jsonl — one training example per line — for the same reason: the file gets streamed and validated line-by-line rather than loaded whole.datasets and most ML data pipelines default to JSON Lines for sharded, streamable training data.kubectl and Docker CLI streaming output, and Server-Sent Events payloads in LLM streaming APIs, follow the same one-JSON-value-per-chunk philosophy, even when the wire format technically wraps each chunk in an SSE data: line rather than a bare newline.The common thread: any producer that doesn't know in advance how many records it will emit, or any consumer that wants to start acting on record 1 before record 2 has even been generated, gravitates toward line-delimited output.
NDJSON isn't strictly better than a JSON array — it trades away a few things on purpose:
| JSON array | NDJSON | |
|---|---|---|
| Valid as a single JSON document | Yes — JSON.parse() the whole thing | No — the file itself isn't valid JSON, only each line is |
| Streamable with a standard parser | No — needs an incremental/SAX-style parser | Yes — plain line-by-line reading |
| Memory to parse fully | O(total size) | O(largest single line) |
| Partial-read usability | Unusable once truncated | Fully usable up to the last complete line |
| Per-record error isolation | One bad record fails the whole document | One bad line fails only that line |
| Pretty-printing / casual inspection | Natural (jq ., editors) | Needs line-aware tooling |
Top-level metadata (e.g. {"total": N, "items": [...]}) | Natural | Requires a side channel or a special first line |
That last row is the one people trip on: NDJSON has no place to put a total count, a next-page cursor, or schema version at the "document" level, because there is no document — just a sequence of independent values. Pagination metadata either goes in a separate response header, gets embedded as its own line (some APIs emit a final line like {"_meta":{"total":50000}} and consumers know to special-case it), or is handled out of band entirely. If you need one authoritative wrapper object around a small result set that a client will always load in full, a JSON array (or object) is simpler and more standard. NDJSON earns its complexity specifically at the point where "load it all before doing anything" stops being an option.
If you're producing or debugging line-delimited data — checking that every line parses independently, catching a stray record that breaks the stream, or converting between a JSON array and NDJSON — Utilix's NDJSON formatter validates and pretty-prints line-delimited JSON without needing to load a huge file into a text editor first.
Takeaway: reach for NDJSON the moment "how many records will there be" becomes an unknown at write time, or "what if the connection drops at record 4,900,000" becomes a question you actually have to answer. If the payload is small and bounded, a plain JSON array is simpler and more tooling-friendly — don't adopt NDJSON as a default, adopt it as the fix for a specific streaming problem.