The Read Tool

how our read tool saves billions of tokens vs claude code

command code is purpose-built for open models, so we optimize things other coding agents get to ignore. for the v1 release i rebuilt the read tool from scratch. it's now one of the most complicated, carefully engineered pieces of the system, and it saves billions of tokens a month. here's what we learned.

context: we wanted the read_file in command code to be the best among coding agents, then benchmarked it capability-by-capability against the nine other common harnesses: claude code, opencode, cline, kilo, codex, grok, hermes, pi, openclaw. most were open-source; claude code ships none, so its column came from feeding the live tool crafted files and watching what came back.

count the reads in any agent session. every edit starts with a read. every grep hit becomes a read. a plan step opens 3 files. a few hundred reads per session, ~50 million a month across command code.

you've seen the failure modes. it reads a file, learns nothing, reads it again. it reads a 5MB lockfile straight into context. it reads a minified bundle once and that junk sits in the window for every turn after.

napkin math:

i think the read_file tool is like a compiler that turns your filesystem into the model's context. every decision inside it is a token budget decision multiplied by fifty million times it's used every month.

and that's why coding agents feel expensive: the bill is mostly reads building context.

what "saves billions of tokens" means here: cost per successful read. claude code's read tool succeeds by spending more: more tokens per call, more turns per miss, and a model smart enough to fish the signal out of the noise. ours had to succeed by spending less, because our models can't paper over a sloppy read and our users care about the token bill.

ask claude code to read a 3,000-line file and it hands the model all 3,000 lines. ask it for a file with a 3,900-character minified line and it hands over the whole line. no window, no byte ceiling, no per-line clamp. i ran the probe twice because i didn't believe it the first time.

everybody ships a read tool in week one. readFile, slice by offset, return the string. first tool you write, last one you think about. ours ended up as dozens of modules with 98 tests, and it was the highest-leverage thing in v1.

a naive read and a harness engineered read are both "correct". they both work. the difference is that one of them quietly spends a fifth of your context window on bytes the model never needed, and occasionally deadlocks against your own write tool.

a few things i learned worth sharing:

1/ you need three ceilings, not one

and it's always the third one people skip.

every codebase keeps a small zoo of hostile files: the 80,000-line lockfile, the minified bundle that's technically one line, the log that never stops growing. each ceiling handles one animal file if you will.

the line window bounds an ordinary large file. the byte budget bounds a file whose lines are wide rather than many. the per-line clamp catches the case the other two miss: one minified line that sits comfortably inside the 2,000-line window and, on its own, eats the entire byte budget. you get back a single unusable mega-string that displaced everything the model actually needed to see.

drop any one ceiling and there's a shape of file that costs you the whole read. no log will ever show it, just a turn where the model got nothing and paid full price for it.

2/ the most expensive thing a tool can return is silence

the costliest failure is an ambiguous non-answer. an empty result string is indistinguishable, from inside the model, from a broken tool. so it re-reads. widens the window. tries a different path. burns three turns learning what one sentence could have told it.

so every dead end names its own recovery:

two details carry most of the value here. the resume offsets are precomputed, so the model never does pagination arithmetic (which it does in reasoning tokens you pay for, and gets wrong often enough to cost another round trip). and none of these carry an Error: prefix, so the tui doesn't paint them red and the model doesn't treat a fact about the world as a failure worth apologizing for.

(the byte-truncated case deliberately resumes ON the last line shown rather than the line after it, because that line got cut mid-content. an off-by-one in a resume hint is a silently corrupted read, which is the one bug class here that's worse than a wasted turn.)

returns silence

1. 01

   the read comes back empty.

2. 02

   the model can't tell an empty file from a bad offset from a broken tool, so it guesses and retries.

3 turns, no new information

returns the recovery

1. 01

   Note: offset 900 is beyond the end of the file (412 lines scanned). Retry with a smaller offset.

2. 02

   the model knows what happened and what to send instead.

1 turn, correct next call

3/ the bug that taught us the most was relational

and no input validation could ever have caught it.

read_file records what the model has SEEN of each file into a ledger: the content, the mtime at read time, and a flag for whether the view was partial. write_file consults it and refuses to overwrite a file you've only partly seen, because you'd silently destroy the part it never saw.

now compose that with the per-line clamp:

we hit it in the wild, on plan files during plan reviews and refinement. every field in every call was valid. the invariant that broke lived in the relationship between three tools that never call each other.

the fix was three-sided:

• write_file now accepts an overwrite whenever the ledger's recorded content matches the bytes on disk, even when the view was flagged partial. a clamped full read still records the exact raw bytes.
• a genuinely partial view gets its own accurate error, Only part of this file has been read, instead of the misleading has not been read yet, which had been sending models into tiny-window re-read loops.
• the unchanged-read dedup no longer stubs a from-line-1 re-read unless the ledger already holds a full view. the dedup check runs after that guard, because a dedup hit consumes its record.

shape invariants are checkable per field, and every schema you write already checks them. relational invariants across stateful tools are where the real bugs live, and you only find them by watching production traffic.

4/ a cache whose stale hit is catastrophic should expire itself on use

re-reading the same window of an unchanged file is pure waste: the content is already sitting in the conversation. so we return a short stub. fires only when mtime, size, and the exact (offset, limit) window all match.

but that stub points at an earlier tool result. what if compaction ate it? now the model has been told to refer to something it can no longer see. forever.

a dedup hit consumes its record. worst case is one wasted turn instead of an unbounded loop. cheap miss, catastrophic stale hit → self-expiring cache. that shape shows up all over a harness once you look for it. i settled with this design as it was a good enough tradeoff between complexity and risk, and it was the only one that did well in our benchmark.

sticky cache

1. 01

   the record survives every hit. if the referenced result is gone from context, the model is pointed at nothing, forever, and no retry escapes it.

unbounded loss

consume on hit

1. 01

   the record is dropped when it fires, so the natural retry gets real content and the loop resolves itself.

one turn, worst case

5/ filenames are adversarial and the model can't see why

macos names screenshots with a NARROW NO-BREAK SPACE before AM/PM. it stores filenames NFD-decomposed. finder renames turn ' into ’.

different byte strings. in a terminal, the same picture. the model reads the path off the screen, retypes it faithfully, gets "file not found", and no amount of reasoning recovers because the difference isn't rendered. you can burn an entire session on this and never learn anything.

so before failing we retry 7 candidate spellings: narrow space ↔ regular, NFD, NFC, straight ↔ curly quote, NFD+curly. each one re-checked against the workspace boundary, because a repair must never quietly become an escape hatch. then, and only then, "did you mean?": substring match plus a bounded levenshtein of 2, which is what catches AGENT.md → AGENTS.md where substring matching finds nothing. these are the most common super cheap open model problems we now repair, saving more tokens than silly token compression tricks.

when a failure is invisible to the model, retrying is the tool's job. the model would retry the same wrong bytes forever. this is what harness engineering is about: finding the invisible failure modes and fixing them in the tool so the model can focus on reasoning.

6/ my favourite bug lives at a chunk boundary

reads stream chunk by chunk instead of loading the file, so a 400MB line sitting BEFORE your window never accumulates. fine. but it turns out that if the line limit is hit EXACTLY at a chunk boundary, you're standing in a spot where the answer to "is there more file?" doesn't exist yet.

saying "more of the file remains" at that moment is a lie roughly half the time, and it's a lie that costs a turn every time it fires. so defer the decision to the next chunk instead of guessing. when you can't know yet, say nothing yet.

(also: don't break out of the for-await. it calls the iterator's return() and destroys the stream underneath you.)

7/ images attach for real

vision models get the actual image, compressed down a jpeg quality ladder (95 → 80 → 60 → 40 → 20). a 4K screenshot degrades instead of failing to attach. format detection sniffs magic bytes, never the extension: garbage in a .png must never reach the api, real webp must pass. we also gave vision to non-vision models using a VISION tool. so fun.

8/ downscaled images disclose their scale factor

without that line, every click coordinate computed off a screenshot is confidently wrong. nothing in the image says it was resized on the way in. and at the end of the day you're saving token costs.

9/ notebooks render as documents

raw .ipynb is json soup: base64 blobs, per-character source arrays. we return tagged cells, plots attached as images. any cell output over 10,000 chars becomes a jq pointer, so one dataframe dump can't eat the read budget. if you do a lot of data work in notebooks, you can now read the notebook without reading the entire dataframe. the model can still reason about the data, but it doesn't have to pay for it in tokens. major time savings for the user, and a major token savings for the model.

10/ boring formats get one-line answers

svg is text (it's xml, the model can edit it). binary returns its mime type, never garbage bytes. pdf gets a pdftotext hint for now; inline is on the list. and we have tools to read and parse different formats as needed, loaded on demand. the model can reason about the file without reading it all, and the user doesn't pay for it in tokens.

11/ numbering matches cat -n

1-indexed, prefix on every line. the model, your editor, and your stack traces agree on what "line 412" means. every resume offset and edit target depends on that.

12/ repair inputs, don't bounce them

10 aliases for file_path (filePath, absolutePath, target_file...) get repaired through the repair layer. numeric strings coerce via Number(), never parseInt: "2abc" is rejected, never silently read as 2. fractional offsets are rejected, never floored. a silently wrong window is worse than an error. our repair harness engineering shows up everywhere.

13/ some paths must never be opened

/dev/zero, /dev/urandom, /dev/stdin, /proc/<pid>/fd/* are refused by name before any i/o. no extension to check, and the workspace boundary won't save you when cwd is /. a read tool that hangs on /dev/zero is a denial of service you shipped yourself.

14/ hygiene you only notice when it bites

bom stripped. crlf normalized. byte-cap truncation binary-searches a utf-8 prefix so it never splits a codepoint. the dedup ships with a kill-switch env var, because every cache needs one.

15/ two read tools is one tool too many

we shipped read_multiple_files beside read_file for a year. bulk glob reads, per-file headers, gitignore, exclusions — a good tool. and it was a mistake, for a reason that has nothing to do with what it did.

every tool you advertise is rent. the schema ships on every request of every turn, whether the model calls it or not:

two near-identical read contracts, and the model pays for both to answer "which read do i want?" — a question we invented for it. small models get that question wrong. they reach for the bulk tool to read one file, or fire five single reads when one glob would do.

so read_file absorbed it. one tool, one contract:

1,685 → 1,147 tokens. ~540 back on every turn, on top of removing a decision the model was never good at.

the thing we didn't expect: models immediately used the merged shape better than the tools it replaced. asked to read a readme and a source tree, tencent hy3 and gemini both sent one call mixing a literal path with a glob — ["README.md", "src/**/*.ts"] → Read 4/4 files. neither old tool could express that. read_file took one path; read_multiple_files took only patterns.

the union type that broke an entire model family

first attempt declared file_path as ['string', 'array'] — one field, both shapes. clean json schema. every test passed.

gemini rejected the request:

not the tool call. the whole request, before a single token was generated. providers translate json-schema unions differently, and gemini's function-calling layer refuses an any_of that carries a sibling description. a union-typed parameter is a portability bug you cannot see from your own test suite.

so: two fields, each with one scalar type. file_path for one, paths for many. file_path stays required (that's what keeps the bare-string-root rescue and the missing-field correction working), and a paths-only call satisfies it through a schema-side requiredAlternatives hint that never touches the wire. there's now a test asserting no advertised property is union-typed, because that class of bug only surfaces in production.

what a merged read has to get right

folding two tools into one is not free. three things had to hold:

the retired name has to keep working. a model mid-conversation, or a cached tool list, still emits read_multiple_files. that name aliases to read_file before lookup, and its fields (include, targetDirectory, gitIgnore, defaultExclude) rename onto the merged schema. and the shapes that arrive are not the ones in the schema: by far the most common malformed call sends include as a bare string rather than an array. that used to land on an array-wrap repair, because include was a declared array field. on the merged schema it's an alias, so the wrap never fires — the tool has to tolerate the string deliberately rather than by luck. a merge is only as good as its worst inherited input shape.

the caps have to move. the bulk tool leaned on the runner's 25K-token output cap to spill anything huge. read_file is exempt from that cap, precisely because it self-bounds. inherit that exemption without moving the ceiling and one wide glob dumps a quarter-million tokens into the window. the aggregate cap now lives inside the read: ~100 KB, one tool-output budget, with the summary naming exactly how many matches were skipped.

silence still costs. models send limit with a multi-file read — rare, but real, and the old tool dropped it as an unknown field. the merged tool declares limit, so it arrives at a call that cannot honor it. the read now says so rather than quietly handing back whole files. same rule as section 2: the dead end names its own recovery.

and the badge follows the result, not the tool name:

where everyone else keeps their bulk read

the bulk-read rows live in the benchmark below, with everything else. the short version:

two of us put the bulk read inside the read tool. cline went furthest — read_files is its ONLY read tool, there is no singular read, and each entry carries its own start_line/end_line, so one call can window into five different files. that is strictly more expressive than what we shipped, and it is the obvious next thing to steal.

gemini cli (not in the table below, which predates it) keeps read_many_files as a second tool — exactly where we were last week. everyone else has no bulk read at all: every multi-file read is n calls, n round trips, n turns.

one row moved between drafting and publishing, which is the whole argument for reading source instead of prompt dumps: kilo code's legacy read_file took up to five files in one call, and half the internet's system-prompt dumps still show that shape. the shipping tool doesn't — kilo now vendors opencode's read, single filePath. a table built from dumps would have that row backwards.

so where does everyone else land?

the top of the table is basically solved. eight of ten harnesses have a line window (500 to 2,000) and a second ceiling (25K tokens to 128 KB). that part is common knowledge now.

Changelog

the bottom of the table is empty almost everywhere. across all ten:

bulk reads sit in the same tail. eight of ten harnesses make you spend a turn per file, which reads as fine in a demo and costs you a round trip every time a plan step opens four files. cline is the exception worth studying — it deleted the singular read entirely rather than shipping two tools, and gave each entry its own line range. we merged in the same direction and still owe them the per-file window.

the pattern: none of those rows show up in a demo. every one of them starts costing you in hour nine of a long session (remembering what the model has already seen, giving it a way back when a read misses, refusing to read /dev/zero). teams build them only after production forces it.

no one nowadays is sitting watching their agents run and fixing the coding agent's bad behaviors. most developers just select a harness on some random vibe in the first week and never look back. this harness is minimal, it must be the best, this harness is from the model maker, this must be the best. wrong! 🤦‍♂️

… whatever happened to the engineer in us?

claude code is the interesting column precisely because it's the incumbent: ledger, notebooks, vision, empty-file note, and then no window, no byte cap, no clamp, no resume offset, no streaming, no suggestion on a miss. that team just hasn't been forced yet, and it runs on models forgiving enough to absorb the waste.

we were forced. we run open models where a wasted turn is visible in the eval score the same day, which is the entire reason any of the above got built. constraint is a feature - it forces you to engineer the right solution instead of hoping the model will figure it out.

i hope this post helps you see the difference between a harness that just works and one that was engineered to work well.

you can try all of this yourself in Command Code (we're also going open source soon so you'll be able to read the code yourself). i'd love to share more deep dives on harness engineering of command code, let me know what y'all wanna read about.

Related