justintsus-experiment Public
# TSUS experiment: plain TypeScript against TypeScript-Ultra-StrictTwo Claude agents built the same Rebasely feature from the same brief. Agent A wrote plain strictTypeScript. Agent B wrote [TypeScript-Ultra-Strict](https://rebasely.com/justin/typescript-ultra-strict)(TSUS), a stricter dialect with no `null`, `undefined`, `as`, `!`, `any`, `class` or `throw`, plus`Option`, `Result` and an exhaustive `match`. A judge agent then scored both builds against a rubricwritten before either agent started.The feature: a GitHub-style search query language (`is:open label:bug author:@me sort:updated`) forRebasely's Issues and Pull requests lists. It parses the query, prints a canonical form, and turns itinto parameterized SQL for D1, scoped to the repository and organization.The write-up is on the Rebasely blog: https://rebasely.com/blog/strict-typescript-ai-code## How it was run1. One brief, two copies. [`brief/BRIEF-ts.md`](brief/BRIEF-ts.md) and [`brief/BRIEF-tsus.md`](brief/BRIEF-tsus.md) differ only in their "Language" section ([`brief/BRIEF.md`](brief/BRIEF.md) is the template). Each agent worked alone in its own git worktree of Rebasely at commit `903f762f`, on the same model (Claude Opus 5.5), with no access to the other's work or to the tests.2. 118 hidden tests ([`hidden-tests/`](hidden-tests)), written before the builds from the brief alone: parsing, canonical form and round trips, semantics on SQLite, dates, SQL injection and scope. `run.sh <worktree>` runs them against a build.3. A judge agent scored each build against [`brief/RUBRIC.md`](brief/RUBRIC.md): hidden tests (40), code review (30), UI (20), cost (10), minus security penalties. A gap under 5 points is a tie. The judge also fuzzed both builds against each other and reviewed the page wiring for security ([`judge/JUDGE.md`](judge/JUDGE.md), probes in [`judge/probes/`](judge/probes)).4. The harness recorded each agent's tokens, tool calls and wall time ([`usage.json`](usage.json)).The judge knew which build was which: `.tsus` files cannot be hidden. The rubric tells it not toreward or penalize the language itself.## Results| | A, plain TypeScript | B, TSUS ||---|---|---|| Hidden tests passed | 118 / 118 | 118 / 118 || Judge's total (of 100) | 98.0 | 95.9 || Code review (of 30) | 28.5 | 27.0 || UI (of 20) | 19.5 | 20.0 || Cost (of 10) | 10.0 | 8.86 || Security findings | 0 | 0 || Tokens | 277,747 | 290,102 (+4.4%) || Output tokens | 35.7k | 57.7k (+61%) || Tool calls | 117 | 150 (+28%) || Wall time | 25.9 min | 25.1 min || Search module lines | 626 | 1,033 (+65%) || `as` casts | 11 | 0 || New unit tests | 17 | 28 || TSUS diagnostics during the build | n/a | 0 in 8 check runs |On 20,400 random inputs plus about 300 targeted probes the two builds returned the same results,errors, canonical forms and SQLite rows. The only differences were outside the brief: B fails on arow whose `labels` column is not valid JSON (A guards it with `json_valid`), and B throws on`parseQuery(null)`.## VerdictA tie on quality, and plain TypeScript was cheaper. The TSUS build had no casts where the plainbuild had 11, all of them safe. TSUS caught nothing during the build: every check run came backclean, because the agent read the rules first and wrote inside them. In exchange it took 33 more toolcalls, 61% more output tokens and 65% more code, including a 51-line adapter between `Option` and`Result` and the plain-TypeScript page code. The build that shipped in Rebasely is A, with B'sfull-width search box.On this feature, TSUS did not measurably beat strict TypeScript for an AI builder. Where it did helpwas an earlier pilot that rewrote existing code (Rebasely's on-call watcher) in TSUS: following the`Option` and `match` rewrite turned up two real production bugs, described in the blog post.## What is here| Path | What ||---|---|| `brief/` | The template brief, the two briefs the agents got, and the judge's rubric || `hidden-tests/` | The 118 hidden tests, their SQLite fixture (`helpers.mjs`) and `run.sh`; `results/` has both runs || `build-a-ts/` | Build A's new files: `apps/web/src/search/*.ts`, `apps/web/src/ui/search.ts`, `test/unit/search/` || `build-b-tsus/` | Build B's new files: `apps/web/src/search/*.tsus` and `api.ts`, `lists.ts`, `ui/search.ts`, `test/unit/search/` || `judge/JUDGE.md` | The judge's full report and scores || `judge/probes/` | The judge's differential probes, page renderer and screenshot script, and the probe output || `judge/shots/` | Six of the judge's 72 screenshots || `usage.json` | Tokens, tool calls and milliseconds per agent, from the harness |The builds are excerpts: only the feature's new files, not the routes, pages and styles they changedin the rest of Rebasely, which is not open source. So they do not run on their own here. The hiddentests only need `apps/web/src/search/api.ts` of a build, and import it from `SEARCH_WORKTREE`;B's `.tsus` files need the TSUS compiler first. The judge's scripts take the two checkouts from`WORKTREE_A` and `WORKTREE_B`.## Links- TypeScript-Ultra-Strict: https://rebasely.com/justin/typescript-ultra-strict- The blog post: https://rebasely.com/blog/strict-typescript-ai-code## LicenseMIT, see [LICENSE](LICENSE). The two builds are excerpts of Rebasely source, published by its ownerfor this experiment.