justintsus-experiment Public
# Judge report: search query language, plain TypeScript (A) vs TSUS (B)- **A, plain TypeScript:** `.claude/worktrees/agent-a7895120081998e14`, commit 40d9e6fd- **B, TSUS:** `.claude/worktrees/agent-a66151373793240e4`, commit a69ca436- **Base:** 903f762f. Judged 2026-10-17.Probe scripts, renderers and raw output are in `judge/probes/`. The judge took 72 screenshots; six are in `judge/shots/`.## Verdict**It's a tie on quality, and A is the cheaper build. Ship A, but bring over B's search box layout and B's test fix first.**The two builds behave the same everywhere I could measure:- The hidden tests pass 118/118 for both.- On 20,400 random (input, target) pairs plus about 300 targeted adversarial probes, they return the same `ok`, the same error codes and spans, the same canonical form and the same rows on SQLite.- The page glue produces the same tab hrefs, counts, pager links and label-chip links.Only two behavioral differences turned up, and neither is in the brief's contract:1. **Malformed labels JSON.** If a stored `labels` value isn't valid JSON, B's `label:`, `-label:` and `no:label` queries fail in SQLite. A guards the column with `json_valid`. The base code had the same unguarded `json_each` that B uses.2. **Non-string input.** B's `parseQuery(null)` and `parseQuery(undefined)` throw. A coerces them with `String()`. The contract only covers strings.The totals are 98.0 for A and 95.9 for B. That gap is inside the rubric's 5-point tie band, and most of it comes from cost: A used 12k fewer tokens and 33 fewer tool calls.**What TSUS bought here:** zero casts in B's whole diff, against 11 in A. Every one of A's casts is safe today. The TSUS checker reported **zero diagnostics in all 8 of B's check runs**. It caught nothing, because the builder read the rules first and wrote inside them. Its value in this experiment was preventive, not corrective. No bug existed in A that TSUS would have stopped.**What TSUS cost:**| | A | B | B vs A ||---|---|---|---|| Tool calls | 117 | 150 | +28% || Total tokens | 277,747 | 290,102 | +4.4% || Output tokens | 35.7k | 57.7k | +61% || Search-module lines | 626 | 1,033 | +65% || Calls on TSUS docs and checks | 0 | 15 | |B also needed a 51-line adapter and a few awkward idioms to work around banned indexing. Wall clock was a wash: B took 25.1 min and A took 25.9 min.## Scores| Item | Max | A | B | Evidence ||---|---|---|---|---|| Hidden tests | 40 | **40.0** | **40.0** | 118/118 each (`ts.out`, `tsus.out`) || Correctness beyond tests | 8 | 8.0 | 7.5 | Identical on every probe. B fails on malformed `labels` JSON, which A guards. || Error handling | 6 | 5.5 | 5.5 | A: a dead catch-all in `parseQuery` would hide a parser bug as a whole-input `bad_value`. B: an unreachable `throw` in `ui/search.ts`, and it throws on non-string input. Neither catches D1 failures, same as base. Both log and survive a RepoDO hydrate failure. || Readability and structure | 6 | 5.5 | 5.0 | A is compact (6 files, 486 code lines) but has 5 casts and a closure that mutates the lexer index. B has real tagged unions and exhaustive `match`, but it's 65% longer, with ban workarounds like `parts.slice(0, 1).join("")` and a for-loop over a 0-or-1-element array. || Edge cases | 5 | 5.0 | 4.5 | Same limits, unicode and whitespace behavior. A guards labels JSON. B guards a null `closedAt` and `mergedAt` in rows. || Tests the builder wrote | 3 | 2.5 | 3.0 | A: 17 new tests, and `test/unit/issues.test.ts` still fails. That failure predates both builds (I checked on base 903f762f), but A edited the file and left it red. B: 28 new tests, and it fixed that assertion. || Fit with the codebase | 2 | 2.0 | 1.5 | A keeps the one-round-trip `DB.batch` of base and reuses `ISSUE_COLS`. B makes two D1 round trips (`Promise.all` of two statements), duplicates `ISSUE_COLS`, and puts page glue that imports `repoStub` and `log` (`lists.ts`) in `search/`. B uses typed `.all<R>()` instead of casts. || **Code review subtotal** | 30 | **28.5** | **27.0** | || UI: search runs the language | 6 | 6 | 6 | Same results for state, label, author, text, date and sort queries || UI: errors inline | 5 | 5 | 5 | Both: wavy underline plus a superscript number per span, message list, `aria-invalid`, no list shown, input kept || UI: syntax help | 3 | 3 | 3 | `<details>` on both lists, only that list's qualifiers, opens with Enter and without JS || UI: URL round trip | 3 | 3 | 3 | Canonical `q` in the box, tabs, pager and chips. Legacy `state=` and `label=` links work. || UI: design | 3 | 2.5 | 3 | A's box is 460px wide in the filter row, so the canonical query is cut off at 1280 px. B puts a full-width box above the tabs. No overflow at 390 px for either. No uppercase, gradients or em dashes in either. || **UI subtotal** | 20 | **19.5** | **20.0** | || Cost: tokens (6) | 6 | 6.00 | 5.74 | 277,747 vs 290,102 || Cost: tool calls (4) | 4 | 4.00 | 3.12 | 117 vs 150 || **Cost subtotal** | 10 | **10.0** | **8.86** | Per hidden test passed: 2,354 vs 2,458 tokens || Security penalties | | 0 | 0 | None found (see below) || **Total** | 100 | **98.0** | **95.9** | **Tie (within 5); A is cheaper** |## Preparation checks| Check | A | B ||---|---|---|| Commits on branch | 1 | 1 || Stray files committed | None | None. No generated `.js`/`.d.ts`, no migration, no edits to `scripts/tsus-web.mjs`, `tools/tsus`, wrangler or CI. || `npm run typecheck` | Pass | Pass || `npm run lint` | Pass | Pass || Builder's own search tests | 17/17 | 28/28 || `test/unit/issues.test.ts` | 2/3, red, inherited from base | 3/3 || TSUS constraint (B only) | n/a, no `.tsus` used | Honored. All logic is in 7 `.tsus` files. `api.ts` (51 lines) only converts `Result`/`Option` and wraps `edit` helpers. |## Code metrics (added code under `apps/web/src`, comments excluded)| | A | B ||---|---|---|| Source files added or changed | 14 | 15 || Search module files | 6 `.ts` | 7 `.tsus`, `api.ts`, `lists.ts` || Search module lines (total / non-blank, non-comment) | 626 / 486 | 1,033 / 776 || Lines added in src + css / in tests | 1,002 / 495 | 1,354 / 570 || New unit tests | 17 | 28 || `as` casts (excluding `as const`) | 11 | 0 || Non-null assertions `!` | 0 | 0 || `any` | 0 | 0 || `try`/`catch`/`throw` added | catch-all in `parseQuery`, `.catch` on hydrate | try/catch on hydrate, 1 unreachable `throw` in `ui/search.ts` |A's 11 casts are `as Key`, `as DateOp`, `as RelUnit`, `as StateTab`, `as StateValue`, `as IssueTab | null`, `as "open" | "closed" | "merged" | "all"`, `as IssueRow[]`, `as PullIndexRow[]` and 2 of `as { state: string; n: number }[]`. All are safe given the surrounding checks. The last four copy the base code's D1 pattern.## Extra probe resultsAll probes import each build's `apps/web/src/search/api.ts` in one Node 24 process and run SQL on the hidden-test fixture (`node:sqlite`). Scripts: `judge-probes/probe.mjs` and `judge-probes/show.mjs`. Raw output: `judge-probes/probe.out`.| Probe | A | B | Brief ||---|---|---|---|| **SQL injection.** 16 payloads × 2 targets in text, phrases, labels, author, sort, dates and `#n` (`'; DROP TABLE`, `") OR 1=1 --`, `?1 :name`, NUL, U+2028). Plus a hostile ctx: repoId `repo-1' OR 1=1 --`, orgId with `")`, viewerId `' OR 1=1 --`. | Inert. No ctx value appears in the SQL text. | Inert. No ctx value appears in the SQL text. | Both match || **Scope escapes.** 20,400 fuzz pairs, every returned row checked against ctx repo and org, ANON included | 0 leaks | 0 leaks | Both match || **Cross-org agent slug.** `author:fixer`: org-1's fixer is FIXER; org-2's fixer (FIXER2) authored issue 6 and PR 6 in this repo. | Issues [5], pulls [2], never 6. Under orgId org-2: [] | Same | Both match || `review-requested:fixer`, `-author:fixer` on pulls | [] and [6,1,4,3,5] | Same | Both match || **NULL under negation.** `-author:@me` on pulls (PR 1 authored by ME, PR 2 owned by ME, others have owner NULL) | [6,4,3,5] | Same | Match || `-author:BOB` on pulls (PR 4 authored by Bob, PR 6 owned by Bob) | [2,1,3,5] | Same | Match || `-author:@me` / `-review-requested:@me` signed out | Every row in scope | Same | Match || `author:@me` signed out | [] | [] | Match || **LIKE escaping.** `100%` → [2], `a_c` → [4], `%` → [2], `_` → [4], `\` → [6], `a\b` → [6], `a_b` on pulls → [4] | As listed | Same | Match. No wildcards leak. || ASCII-only case folding: `Ü` → [4], `ü` → [], `ÜNÏCODE` → [] | As listed | Same | Match (SQLite LIKE semantics) || **Unicode spans.** `"🐛` → unterminated 0..3. `a 🐛" b` → unexpected 2..5. `🐛:x` → free text. A lone surrogate `\uD83D` is free text. | UTF-16 offsets, as listed | Identical | Match || **Whitespace kinds.** NBSP, U+3000, U+2003 and U+FEFF separate terms. U+200B, U+0085 and U+180E do not (JS `\s`). | As listed | Same | Match || Error ordering. `-label:` / `-sort:` → `empty_value` before `negation_not_allowed`. `sort:x sort:y` → two `bad_value`, no conflict. `sort:created-asc sort:CREATED-ASC` → deduplicated, no conflict. | As listed | Same | Match || Dates (48 values × 2 fields × 2 targets). `*..*`, `<0d`, `<10000d`, `7d`, `=D`, `1969-12-31`, `2026-04-31`, `<٧d` (Arabic digit), `A..B` with A after B, `>D..E` → all `bad_date`. `<0007D` → `<7d`. `9999-12-31` accepted. | As listed | Same | Match || **Limits.** 1025 chars → `too_long` 0..1025 alone. 33 terms → `too_many_terms` from the 33rd term to the end, even when an unterminated quote follows. | As listed | Same | Match || **D1 100-bound-parameter limit, worst case.** Most params per term is 2 (text on issues; `author:` on pulls). 32 distinct heavy terms + scope = 66 params, + LIMIT/OFFSET = **68**. Counts query: 66. Hydrate `IN` list: at most 25. | 68 max | 68 max | Both under 100 || SQL text size, worst case | 3.6 KB | 4.5 KB | Fine || **Speed.** 10k random queries, parse + serialize + toSql, median of 5 runs | 13.5 ms | 18.8 ms | Both negligible || Worst-case 1024-char inputs (escaped quotes, 1024 `"`, 1023 `\`, 341 negations), µs per parse | 7 to 37 | 9 to 30 | Both linear, no blowup || **Never throws, string inputs.** 20,400 fuzz + targeted | 0 throws | 0 throws | Match || Non-string input (outside contract): `parseQuery(null)` / `parseQuery(undefined)` | `ok: true` (coerces) | **Throws TypeError** | Not required || `toSql` with `now: NaN` (outside contract) | Binds `null` | Binds `null` | n/a || **Malformed `labels` JSON row** (`'not json'`) with `label:bug`, `-label:bug`, `no:label` | Works; the row counts as no labels | **SQLite "malformed JSON"**, which would be a 500 on the page | Not in the brief. Base code behaves like B. || Round trip: `parseQuery(serialize(q))` re-serializes identically and gives identical SQL, on every accepted fuzz input | 0 violations | 0 violations | Match || Placeholders = params on every accepted fuzz input | 0 violations | 0 violations | Match |The only behavioral differences between A and B are the last-but-one two rows: non-string input and malformed labels JSON. On every input the brief defines, the builds agree.## Security review of the page wiring| Area | A | B ||---|---|---|| Permission | `issuesPage` and `pullsPage` still go through `pageOf` (prContext, `repo.read`) before any search. Unchanged. | Same || SQL in the glue | List, counts and pagination queries interpolate only fixed table and column lists plus `toSql` output. `LIMIT ? OFFSET ?` are bound. | Same. The table name comes from a literal union. || Scope on counts | Counts query is `toSql(withState(q, "all"))`, so it is scoped | Counts query is `toSql(withState(q, null))`, so it is scoped || XSS | The error echo, the message list `<code>` and `value=` all go through `html```, which escapes `& < > " '`. Payload `"><script>alert(1)</script> foo:<img onerror> label:"<b>x</b>"` is rendered inert (`issues-xss.html`). | Same, verified the same way || Open redirects | No new redirects. Every link is `searchHref(base, …)` with `base` the repo path and a `URLSearchParams`-encoded `q`. | Same || Pagination limits | `page` clamped to 1..400 (max OFFSET 9,975). 25 + 1 rows per page. | Same (`MAX_PAGE` 400) || RepoDO hydrate failure | `.catch` logs `pulls.list_hydrate_failed` and renders from `pull_index` | try/catch logs `search.pulls_hydrate_failed` and renders from `pull_index` || D1 failure | Not caught (500), same as base `listIssueViews` | Not caught (500), same as base || Agent slug resolution | Subquery `agents.org_id = ? AND slug = ?` with ctx.orgId bound | Same |No security findings for either build, so no penalties.## Findings by severity**High:** none in either build.**Medium:**- **A, UI:** The search box is 460px wide in the filter row, so at 1280 px the canonical query is cut off: `is:open label:"needs review" sort:upd`. For a query language whose canonical form is the point of the box, that hurts. (See `A-issues-query-1280-light.png` and `B-issues-query-1280-light.png`.)**Low:**- **B:** `label:`, `-label:` and `no:label` throw in SQLite if any row in scope has malformed `labels` JSON, which makes the list page return 500. The base code had the same weakness. A's `LABELS` CASE guard avoids it.- **B:** `parseQuery(null)` and `parseQuery(undefined)` throw. This is outside the string contract, but A tolerates it.- **B:** Two D1 round trips per list (`Promise.all` of `.all()` calls) where base and A use one `DB.batch`.- **B:** Page glue (`search/lists.ts`, which imports `repoStub` and `log`) lives in `search/` next to the pure modules, and `ISSUE_COLS` is duplicated instead of exported.- **A:** `test/unit/issues.test.ts` stays red. The failure is inherited from base, but A edited that test and didn't fix it. B did.- **A:** The catch-all `try/catch` in `parseQuery` is unreachable by design. If a bug ever made it reachable, it would report a whole-input `bad_value` instead of surfacing the bug.- **A:** 11 casts. All safe today, but none are checked.**Info:**- **B:** Hydrated pull rows show the RepoDO's live title and state. A shows `pull_index`'s state with the DO's comment count. Both are allowed.- **B:** Old `?label=` links with an invalid label show a parse error. A silently drops the label. Both are defensible.- **A:** On an invalid query, A shows a "Nothing listed" empty state with a Clear button. B shows the error card with a "Clear the search" link and no list card. A's guidance is slightly clearer. A also renders the whole message list in the error color; B uses body text with a red heading, which reads more easily.## UI checkI rendered both builds' pages with each build's own page glue (`repo/list-search.ts` in A, `search/lists.ts` in B) against the hidden fixture through a `node:sqlite` D1 shim and a mocked RepoDO, then wrapped them in the real `renderPage` layout. The fixture had 30 extra issues so a second page exists. Script: `judge-probes/render.mjs`.I screenshotted the pages with Playwright (Chromium) at 1280 and 390 px, in light and dark (`judge-probes/shots.mjs`). Pages covered:- issues: default, a query, page 2, a legacy `?state=closed&label=bug` link, an invalid query, an XSS payload- pulls: default, a query, an invalid query- the help popover on both listsWhat I checked:- **Links and counts:** every tab href, count, pager link and label-chip link is identical between the builds.- **Overflow:** no horizontal overflow at 390 px on any page for either build.- **Help:** `<details>` opens with Enter from keyboard focus, and also with JavaScript disabled.- **Help contents:** each list shows only its own qualifiers. Pulls never show `label:` or `no:`; issues never show `review-requested:`.- **Error marks:** the offending spans are underlined (wavy) and carry a number, so they don't rely on color alone. A adds a visually hidden "(problem n)"; B marks the echo `aria-hidden` and lists the terms in `<code>`. B also puts `role="alert"` on the error card and points `aria-describedby` at the help.- **DESIGN.md:** no uppercase text transforms, gradients or em dashes in the new UI. Inter body text and mono for queries in both.## What TSUS demonstrably caught or prevented, and what it cost**Caught during the build: nothing.** All 8 `tsus-web.mjs check --json` runs in B's transcript returned `{"ok": true, "diagnostics": []}`. B's only failures came from its own unit tests (2 parse expectations, 2 SQL expectations) and the repo's design test (a raw `z-index`), the same kinds of failures A hit.**Prevented, by construction:** unchecked narrowing. A narrows strings to unions with `as` after a runtime check. B has to prove the narrowing with a total `match`:```ts// A, parse.tsif (!Object.hasOwn(KEYS, key)) return fail("unknown_qualifier", ...);const k = key as Key;...const op = m[1] as DateOp;return { kind: "age", op, n, unit: rel[2].toLowerCase() as RelUnit };``````ts// B, parse.tsusconst keyOf = (k: string): Option<Key> => match (k) { "is" => Some<Key>("is"), "label" => Some<Key>("label"), ... _ => None, };```**A design both reached, by different routes: NULL-safe negation.** A wraps every negated term generically. B makes each positive form never NULL (`IS` instead of `=` on the nullable column). Both are correct; A's is one rule, and B's needs care per term:```ts// A, sql.tsparts.push("neg" in t && t.neg ? `NOT IFNULL((${c.sql}), 0)` : `(${c.sql})`);``````ts// B, sql.tsusUser(x) => Some(frag("(pull_index.author_id = ? OR pull_index.owner_id IS ?)", [x.id, x.id])),```**Where B cost more:**1. **Adapter code.** `api.ts` (51 lines) converts at every boundary. `null` becomes `None`, `Result` becomes `{ ok }`, and the edit helpers need `Option` wrappers:```ts// B, api.tsexport function toSql(query: SearchQuery, ctx: SqlCtx) { const r = sql(query, { repoId: ctx.repoId, orgId: ctx.orgId, viewer: ctx.viewerId === null ? None : Some(ctx.viewerId), now: ctx.now }); return { where: r.where, params: [...r.params], orderBy: r.orderBy };}export function tabOf(query: SearchQuery): ListState | null { const t = tabOfQuery(query); return t.isSome() ? t.value : null;}```2. **Ban workarounds.** These are where plain indexing would have been clearer:```ts// B, values.tsus: parts[0] and parts[1] without indexingconst parts = v.split("..");if (parts.length !== 2) return None;const from = endOf(parts.slice(0, 1).join(""))?;const to = endOf(parts.slice(1).join(""))?;``````ts// B, edit.tsus: "the one element, if exactly one"const only = states.length === 1 ? states.slice(0, 1) : [];for (const s of only) if (!s.negated) return Some(s.state);return None;``` Compare A:```ts// A, edit.tsconst [t] = s;return s.length === 1 && t.kind === "is" && !t.neg ? (t.value as StateTab) : null;```3. **Volume and calls:**| | A | B ||---|---|---|| Search module lines | 626 | 1,033 (+65%) || Output tokens | 35.7k | 57.7k (+61%) || Tool calls | 117 | 150 (+33) | About 15 of B's extra calls went to TSUS itself: 7 reads of `tools/tsus/AGENTS.md`, the runtime `.d.ts` and the templates, plus 8 `check` runs. The rest went to a longer UI preview session and a WIP-commit-then-soft-reset. The total token gap is small (+4.4%) because input and cache reads dominate.**Where TSUS did not cost more:** wall clock (25.1 vs 25.9 min) and behavior (identical). TSUS added no runtime overhead worth mentioning: 18.8 vs 13.5 ms per 10k queries, both far below a request's budget.## Ship recommendationShip **A**: same behavior, cheaper, one D1 round trip, tolerant of bad labels JSON. Before shipping, fix three things:1. Widen the search box so the canonical query is visible on desktop. B's full-width box above the tabs is the better layout; take it.2. Fix the inherited `aria-current="page">Issues<` assertion in `test/unit/issues.test.ts` (B's regex fix works).3. Optionally, drop the unreachable catch-all in `parseQuery`, or make it log.If the team prefers B, it's equally shippable. Add the `json_valid` guard on `labels` (or accept base's behavior) and batch the two D1 statements.## Notes on the comparison- **Hidden tests didn't separate the builds.** They pass 118/118 each, and my fuzzing found no disagreement on any input the brief defines. The two independent implementations agree with each other and with the hidden tests, which is good evidence the brief was unambiguous.- **Environment friction hit both builders.** The worktree guard refused compound `ln`/`git` commands 7 times for A and 9 times for B. The pre-existing red `issues.test.ts` affected both equally; only B spent calls on it, and only B fixed it.- **UI was judged from static renders** through each build's own glue and the real layout, not a live `wrangler dev` stack with sessions. Signed-in versus signed-out differences (New issue button, sign-in prompt) weren't exercised beyond the `canOpen` flag. Neither build changed them.