Rebasely Beta: data may be purged

justintsus-experiment Public

Markdown · 20.7 KiB · at 8e57616

Brief: a search query language for the Issues and Pull requests lists

You are building one feature in the Rebasely codebase: a GitHub-style search query language for the repository Issues list and the repository Pull requests list. Another engineer is building the same feature from the same brief in parallel; the two results are compared on hidden tests, code review and a UI check. Work alone, inside your own worktree, and do not look at other worktrees.

Setup

  • The repository is at the worktree path you were given (a git worktree of the Rebasely checkout). Work and commit only there, on your worktree branch.

  • Do not push, deploy, or run any wrangler deploy, migration against a remote database, or release script. Commit your work on your worktree branch; the judge reads the branch.

  • Node resolves dependencies from the main checkout's node_modules, so a worktree needs its own links to the workspace packages before anything runs. From the worktree root:

    Shell
    mkdir -p node_modules/@duragit node_modules/@tsus
    for p in actions:apps/actions cli:packages/cli deadman:apps/deadman shared:packages/shared ssh:apps/ssh web:apps/web; do
      ln -sfn "../../${p#*:}" "node_modules/@duragit/${p%%:*}"
    done
    ln -sfn ../../packages/git node_modules/@duragit/git
    ln -sfn ../../tools/tsus/packages/runtime node_modules/@tsus/runtime

    (If your shell tool refuses a command, run the ln lines one at a time.)

  • Read DESIGN.md (UI rules: one accent color, Inter 450, no decorative stripes, gradients, emoji, uppercase kickers or em dashes in UI copy) before touching pages.

  • Checks that must pass before you finish: npm run typecheck, npm run lint, and the unit tests you add (node --test test/unit/search/*.test.ts or wherever you put them). The full npm run test:unit is slow; run at least the suites near what you change.

  • Do not read anything outside your worktree and your own scratchpad directory. In particular do not read anything else under /private/tmp (other sessions' scratchpads), other worktrees under .claude/worktrees, the main checkout's working tree, or other copies of any tool. Everything you need is in the worktree.

What exists today

  • Issues live in D1, table issues (apps/web/migrations/0020_issues.sql): repo_id, number, org_id, author_id, title, body, state ('open'|'closed'), labels (a JSON array of lowercase strings, default '[]'), comments, created_at, updated_at, closed_at, closed_by (times are epoch milliseconds). The list query is listIssueViews in apps/web/src/issues/service.ts; the page is issuesPage in apps/web/src/repo/issues-page.ts, rendered by issuesListBody in apps/web/src/ui/issues.ts, which already has a small search form (q, titles or #number).
  • Pull requests live in each repository's RepoDO (apps/web/src/do/repo/pulls.ts, prList), with a D1 copy kept by the DO's outbox: table pull_index (apps/web/migrations/0013_pulls.sql): repo_id, number, org_id, author_id, owner_id (the agent's owner for an agent's PR, else NULL), state ('open'|'closed'|'merged'), draft (0|1), title, base_ref, head_ref, created_at, updated_at, merged_at, closed_at, and pull_review_requests (repo_id, number, user_id, requested_at). The page is pullsPage in apps/web/src/repo/pulls-page.ts, rendered by pullsListBody in apps/web/src/ui/pulls.ts.
  • People have no login handle: a user is a UUID (users.id). Agents are users too, with a slug per organization (agents (user_id, org_id, slug, owner_user_id, ...), slug ^[a-z0-9][a-z0-9-]{0,38}$, AGENT_SLUG in packages/shared/agents.ts). Labels follow cleanLabels in packages/shared/issues.ts: lowercase, ^[a-z0-9][a-z0-9 ._-]*$, at most 32 characters.
  • There are no assignees, no milestones and no labels on pull requests.
  • The routes already decide repo.read before the list runs (prContext), so a repository the viewer cannot read never reaches your code. Your SQL still scopes every row to the repository and its organization (see "SQL contract"), so nothing a person types can widen it.

Language

Write the parser, the canonical serializer and the SQL evaluator in TypeScript-Ultra-Strict (.tsus files) in apps/web/src/search/ (as many files as you like). tsus is TypeScript with the escape hatches removed: no null, undefined, class, var or throw; Option and Result from @tsus/runtime; a typed match macro checked for exhaustiveness at compile time; a ? operator for early return; and 13 further bans (any, as, !, enum, ==, default exports, parameter mutation, floating promises, unused Result, untagged error types and more). Read tools/tsus/AGENTS.md first; it is the language reference (the vendored compiler and runtime are in tools/tsus, pinned in tools/tsus/VERSION).

How it builds here (already wired, see scripts/tsus-web.mjs):

  • Every apps/web/src/**/x.tsus compiles to x.js, x.js.map and x.d.ts next to it (gitignored; never commit or edit them). The wrangler build, npm run typecheck and npm run test:unit run the compile first; run it yourself with npm run tsus:web.
  • One .tsus file imports another as "./y.tsus", and the runtime as "@tsus/runtime" and "@tsus/runtime/boundary". Plain .ts code imports a .tsus module as "./x.js" (the compiled file; tsc reads x.d.ts). Do not put x.ts next to x.tsus.
  • Your loop: node scripts/tsus-web.mjs check --json (the tsus rules plus a strict type check of the .tsus files; a TypeScript error comes back as TSUS200 with tsCode), node scripts/tsus-web.mjs fix (and fix --unsafe) for machine fixes, and node tools/tsus/packages/cli/bin/tsus.js explain <CODE> for any rule. Use these instead of tsus check apps/web or tsus fix apps/web: the stock command type-checks every .ts file under src/ without the Worker types and drowns your diagnostics in hundreds of false ones. npm run typecheck must pass too.

apps/web/src/search/api.ts is a thin plain .ts adapter over your .tsus modules: it converts Result and Option into the plain objects of the interface below ({ ok: true, query } or { ok: false, errors }, viewerId: string | null into whatever you use) and nothing else; the logic lives in .tsus. The routes and pages stay .ts and call the module through api.ts (or through the compiled .js modules directly). Treat a bad query as data: no exceptions, and the adapter never throws for any string input.

The module interface (what the hidden tests import)

Create apps/web/src/search/api.ts. The hidden tests import exactly this file, under Node 24 with type stripping (node --test, no bundler, no wrangler), after running npm run tsus:web. So api.ts and everything it imports must load in plain Node: relative imports and @duragit/shared/* are fine; no cloudflare:* modules, no env bindings, no D1 at import time.

TypeScript
export type Target = "issues" | "pulls";
export type SqlValue = string | number | null;

export type QueryErrorCode =
  | "too_long" | "too_many_terms" | "unterminated_quote" | "unexpected_quote"
  | "unknown_qualifier" | "unsupported_qualifier" | "empty_value" | "negation_not_allowed"
  | "bad_value" | "bad_date" | "conflict";

// start/end: UTF-16 code unit offsets into the input, end exclusive.
export type QueryError = { code: QueryErrorCode; message: string; start: number; end: number };

// The parsed query. Its shape is yours (an AST); the tests treat it as opaque and only pass it back
// to serializeQuery and toSql.
export type SearchQuery = /* your AST type */;

export function parseQuery(input: string, target: Target):
  { ok: true; query: SearchQuery } | { ok: false; errors: QueryError[] };

export function serializeQuery(query: SearchQuery): string;

export type SqlCtx = {
  repoId: string;          // the repository being listed
  orgId: string;           // its organization
  viewerId: string | null; // the signed-in user's id, null when signed out
  now: number;             // epoch milliseconds, for relative dates
};

// A boolean SQL expression and an ORDER BY list for the target's table, with every value bound.
export function toSql(query: SearchQuery, ctx: SqlCtx): { where: string; params: SqlValue[]; orderBy: string };

These are plain JavaScript values at the boundary (ok booleans, arrays, strings, numbers, null for a signed-out viewer) whatever you use inside. message is a short sentence for people (shown in the UI), never empty.

The query language

Terms

An input is a sequence of terms separated by runs of whitespace (any character JavaScript's \s matches, so a no-break space or an ideographic space separates terms too). Every term is ANDed with the others. There is no OR, no parentheses, no NOT keyword: OR is just a word.

Reading one term, starting at its first non-whitespace character:

  1. Negation. A - immediately followed by a non-whitespace character negates the term and is not part of it. A - alone (followed by whitespace or the end) is the free-text word -.
  2. Quoted value. A " starts a quoted value. Inside, \" is a " and \\ is a \; any other backslash is kept as it is. The next unescaped " ends it. No closing quote: error unterminated_quote spanning from the term's start to the end of the input, and parsing stops there. After the closing quote the term must end (whitespace or end of input), otherwise unexpected_quote.
  3. Bare text runs to the next whitespace. If it reaches a ", that is only allowed when the bare text so far is exactly a qualifier key and its colon (label:"needs review"): the quoted value is the qualifier's value. Any other " inside bare text is unexpected_quote. An unexpected_quote error spans from the term's start to the next whitespace (or the end), and parsing goes on after it.
  4. Kinds. A bare term matching ^[A-Za-z]+(-[A-Za-z]+)*: is a qualifier: the key (lowercased) is the part before the first colon, the value is everything after it (or the quoted value). A bare term matching ^#[1-9][0-9]{0,8}$ is a number. Anything else, and every quoted term, is free text. So 12:30, a1:b and :x are free text, http://x is the qualifier http (unknown), and "label:bug" is the phrase label:bug.

Qualifiers and values

Qualifier Targets Values Negation
is: both open, closed; for pulls also merged, draft allowed
label: issues a label name allowed
no: issues label not allowed
author: both a user reference allowed
review-requested: pulls a user reference allowed
sort: both created-desc, created-asc, updated-desc, updated-asc; for issues also comments-desc, comments-asc not allowed
created: updated: both a date filter not allowed
#123 both the number not allowed
free text, "phrases" both any non-empty text allowed

Values are case-insensitive: is:, no:, sort:, author: and review-requested: values are lowercased. A label value is trimmed, lowercased and has each internal whitespace run collapsed to one space; the result must match ^[a-z0-9][a-z0-9 ._-]*$ and be at most 32 characters. A user reference is @me, a UUID (^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$ after lowercasing; checked before the slug rule) or an agent slug (^[a-z0-9][a-z0-9-]{0,38}$). Anything else after @ is a bad_value.

Date filters

All dates are UTC calendar days, YYYY-MM-DD exactly (zero-padded, a real date, year 1970 to 9999). Let start(D) be D at 00:00:00.000Z in epoch ms and DAY = 86,400,000.

Value Matches a timestamp t when
D start(D) <= t < start(D) + DAY
>D t >= start(D) + DAY
>=D t >= start(D)
<D t < start(D)
<=D t < start(D) + DAY
A..B start(A) <= t < start(B) + DAY; A after B is bad_date
A..* t >= start(A)
*..B t < start(B) + DAY

Relative values compare age (now - t) with a duration N units, where N is 1 to 9999 (leading zeros allowed, dropped in the canonical form) and the unit is h (3,600,000 ms), d (86,400,000) or w (604,800,000), case-insensitive. With T = now - N * unit:

Value Reads as Matches when
<7d younger than 7 days t > T
<=7d at most 7 days old t >= T
>7d older than 7 days t < T
>=7d at least 7 days old t <= T

A relative value needs an operator (updated:7d is bad_date); ranges take absolute dates or * only (*..* is bad_date); = is not an operator. Anything else is bad_date.

Errors

Every invalid term produces exactly one error spanning the whole term, from its first character (including a leading -) to its last (including quotes); errors are listed in input order. When any error exists, parseQuery returns ok: false with all of them. For a qualifier term the checks run in this order and the first failure is the error:

  1. unknown_qualifier: the key is not in the table (for example assignee:, repo:, milestone:).
  2. unsupported_qualifier: the key exists but not for this target (label: and no: on pulls, review-requested: on issues).
  3. empty_value: nothing after the colon, or "".
  4. negation_not_allowed: a - on no:, sort:, created: or updated:.
  5. bad_value (or bad_date for created: and updated:): the value is not valid for the key and target, for example is:merged on issues or sort:comments-desc on pulls.

Other errors: a negated number (-#5) is negation_not_allowed; a free-text quoted value with no non-whitespace character ("", " ") is empty_value; a second sort: term with a different value is conflict (on the second term).

Limits, checked before anything else and reported alone (a single error, nothing else):

  • too_long: the input is longer than 1024 UTF-16 code units; the error spans the whole input (start 0, end the input's length).
  • too_many_terms: more than 32 terms, counted as written (before duplicates are removed, invalid terms included); the error spans from the 33rd term's start to the end of the input.

An empty or whitespace-only input is a valid, empty query.

Canonical form (serializeQuery)

The canonical form is what goes in the page URL (?q=), so the URL round-trips: for any valid input x, with c = serializeQuery(parseQuery(x).query), parseQuery(c) succeeds, serializeQuery of it is c again, and toSql gives the same where, params and orderBy for both. Rules:

  • Terms keep their input order, joined by single spaces. A term whose canonical text equals an earlier term's is dropped.
  • Keys are lowercase. Values are normalized as above (IS:Open is is:open, label:" Needs Review " is label:"needs review", author:@ME is author:@me, updated:<007D is updated:<7d). Dates are written as given (created:>=2026-10-01, created:2026-10-01..*).
  • Negated terms start with -.
  • A qualifier value is written bare when it matches ^[^\s"]+$, otherwise quoted.
  • A free-text value is written bare when it matches ^[^\s"]+$, does not start with -, does not match the qualifier pattern ^[A-Za-z]+(-[A-Za-z]+)*: and does not match the number pattern ^#[1-9][0-9]{0,8}$; otherwise quoted. So "bug" is written bug, "label:x" stays "label:x", and a lone - is written "-".
  • Quoting writes ", the value with each \ as \\ and each " as \", then ". Phrases keep their inner whitespace exactly.
  • The empty query is the empty string.

Semantics

Term Issues (issues) Pulls (pull_index)
is:open state = 'open' state = 'open' (drafts included)
is:closed state = 'closed' state is 'closed' or 'merged' (as on GitHub)
is:merged n/a state = 'merged'
is:draft n/a draft = 1, in any state
label:x x is in the labels JSON array n/a
no:label labels is an empty array n/a
author:u author_id is u author_id is u, or owner_id is u (an agent's PR shows under its owner, as the list does today)
review-requested:u n/a a pull_review_requests row for this PR with user_id u
#n number = n number = n
free text w w occurs in title or in body w occurs in title
created: updated: created_at / updated_at per the date tables same
  • A negated term matches exactly the rows (in scope) its positive form does not match. Watch NULLs: -author:u on pulls must match a PR whose owner_id is NULL and whose author is someone else.
  • User references: @me is ctx.viewerId; with no viewer, author:@me and review-requested:@me match nothing and their negations match everything. A UUID is that user id. A slug is the agent with that slug in ctx.orgId (agents.org_id); a slug of another organization's agent, or of no agent, matches nothing.
  • Free text and phrases match as substrings with SQLite LIKE semantics (ASCII case-insensitive, other characters exactly), with %, _ and \ in the text matched literally (escape them; use ESCAPE '\'). A phrase is one substring, inner spaces included; each bare word is its own term.
  • Two is: terms are ANDed like everything else: is:open is:closed matches nothing. No term is implied: the empty query matches every row in scope, in every state. (The page supplies is:open as its default; the evaluator does not.)
  • sort: picks the order; the default is created-desc. orderBy is <column> <ASC|DESC>, number <ASC|DESC> with the tie-break in the same direction: created_at, updated_at or comments.

SQL contract (toSql)

  • The result is used as SELECT ... FROM <table> WHERE ${where} AND <more conditions> ORDER BY ${orderBy} LIMIT ?, where <table> is issues for an issues query and pull_index for a pulls query, without an alias. where must keep its meaning when more conditions are ANDed after it (parenthesize any top-level OR). It may reference the table by name and use subqueries (json_each, pull_review_requests, agents).
  • Every value is bound. Nothing from the input or from ctx (ids included) is ever written into the SQL text; params carries them, in placeholder order. Use ? placeholders (?NNN numbered ones are allowed if you use them for every placeholder). The SQL text may contain only constant literals you wrote yourself ('open', '\' in ESCAPE '\', 0, 1).
  • Scope cannot be dropped. where always requires repo_id = ctx.repoId and org_id = ctx.orgId at its top level, ANDed with the query's conditions, for every query including the empty one. No term can widen the scope.
  • orderBy is built only from the fixed column names and directions above.
  • toSql itself never fails for a query parseQuery accepted for the same target.

The pages

  • Issues (/:org/:repo/issues) and Pull requests (/:org/:repo/pulls): the list is driven by ?q= in the query language. With no q, the page uses is:open (and still honors the old state= and label= links by turning them into terms). Run the search through parseQuery and toSql against D1 for both lists: issues from issues, pull requests from pull_index. (The RepoDO stays the source of truth for a PR's own page; pull_index may lag a few seconds, which is fine for a list. You may hydrate rows from the RepoDO if the list needs fields pull_index lacks, or render what pull_index has.) Keep 25 per page with a way to the next page that keeps q.
  • The search box shows the canonical form of the current query. Links the page makes (state tabs, next page, a label chip, "Clear") use canonical q values. The Open / Closed (and Merged) tabs replace the query's is: state term.
  • Errors inline: a query with errors renders the page (status 200) with the input kept as typed, each error's message under the box, and the offending spans marked in a way that does not rely on color alone. No results are listed for an invalid query.
  • Syntax help: a small popover or disclosure next to the box listing the qualifiers that apply to that list, with one example each, usable by keyboard and screen readers and without JavaScript (for example a <details> element).
  • Follow DESIGN.md and the existing list styles. No new JavaScript framework; a little progressive enhancement is fine.

Out of scope

  • /inbox, the /api/v1 list endpoints (keep their current q behavior), the MCP tools, saved searches, autocomplete, assignees and milestones, labels on pull requests.
  • Migrations: none are needed. Do not add one.

When you are done

Commit on your worktree branch with a message in the repository's style, and reply with: what you built, the files, how you checked it (commands and results), and anything you know is missing or wrong.