examples/live-notes Public
Map the snapshots bucket and write down how snapshots work
1 check passedAdds the snapshots component, the edges from the room and the Worker, a step in the edit flow and a new flow for reading a note's history. The snapshots concept and decision 0003 explain what is saved, when, and why it lives in R2 rather than in the room.
Changed files
11 files changed181 additions17 deletionsagainst bd4f325
- Changed.rebasely/architecture/flows/edit-broadcast.yaml+4−1
- Added.rebasely/architecture/flows/reading-history.yaml+13−0
- Changed.rebasely/architecture/system.yaml+16−3
- Changed.rebasely/architecture/views/editing.yaml+2−2
- Changed.rebasely/kb/concepts/conflict-strategy.md+5−1
- Changed.rebasely/kb/concepts/note-room.md+13−2
- Added.rebasely/kb/concepts/snapshots.md+65−0
- Added.rebasely/kb/decisions/0003-snapshots-in-r2.md+37−0
- Changed.rebasely/kb/guides/local-dev.md+14−2
- Changed.rebasely/kb/index.md+2−0
- ChangedREADME.md+10−6
Changed.rebasely/architecture/flows/edit-broadcast.yaml+4−1
| @@ -1,5 +1,7 @@ | ||
| title: An edit reaching other editors | ||
| -description: Editor A types while editor B has the same note open. The room puts the edit in order, saves it and passes it on. | ||
| +description: >- | ||
| + Editor A types while editor B has the same note open. The room puts the edit in order, saves it | ||
| + and passes it on. Every 100th version it also copies the note to R2. | ||
| actors: [editor-a, editor-b] | ||
| steps: | ||
| - { from: editor-a, to: note-room, label: "op: insert \"oat \" at 4, written against version 41" } | ||
| @@ -7,4 +9,5 @@ | ||
| - { from: ops, to: note-room, label: "the op as it applies now, which the room saves to SQLite" } | ||
| - { from: note-room, to: editor-a, label: "ack: version 44" } | ||
| - { from: note-room, to: editor-b, label: the transformed op at version 44 } | ||
| + - { from: note-room, to: snapshots, label: "every 100 versions: put the note's text, after the replies" } | ||
| - { from: editor-b, to: ops, label: "move the op past B's own unacknowledged edits, then apply it" } | ||
Added.rebasely/architecture/flows/reading-history.yaml+13−0
| @@ -0,0 +1,13 @@ | ||
| +title: Reading a note's history | ||
| +description: >- | ||
| + Someone wants to see what the groceries note said before today's edits. The Worker reads the | ||
| + snapshots from R2 under the room's object id, which it computes with idFromName, so the room | ||
| + is never woken. | ||
| +actors: [reader] | ||
| +steps: | ||
| + - { from: reader, to: api, label: "GET /notes/groceries/snapshots" } | ||
| + - { from: api, to: snapshots, label: "list snapshots/<idFromName(groceries)>/, 100 keys at a time" } | ||
| + - { from: api, to: reader, label: "versions 100, 200 and 300, when each was saved and its size" } | ||
| + - { from: reader, to: api, label: "GET /notes/groceries/snapshots/200" } | ||
| + - { from: api, to: snapshots, label: "get snapshots/<id>/000000000200.txt" } | ||
| + - { from: api, to: reader, label: "the text at version 200, as text/plain, cacheable for a year" } | ||
Changed.rebasely/architecture/system.yaml+16−3
| @@ -1,17 +1,30 @@ | ||
| components: | ||
| api: | ||
| kind: worker | ||
| - description: The Worker. Routes /notes/:id to that note's room and serves the editor page as static assets. | ||
| + description: >- | ||
| + The Worker. Routes /notes/:id to that note's room, serves the editor page as static assets, | ||
| + and reads a note's snapshots straight from R2 so history never wakes the room. | ||
| owns: [src/index.ts, src/routes.ts] | ||
| talks_to: | ||
| - { to: note-room, via: rpc } | ||
| + - { to: snapshots, via: r2 } | ||
| note-room: | ||
| kind: durable-object | ||
| - description: One per note, picked by name. Orders every edit, keeps the note and its op log in SQLite, and holds the editors' WebSockets. | ||
| + description: >- | ||
| + One per note, picked by name. Orders every edit, keeps the note and its op log in SQLite, | ||
| + holds the editors' WebSockets, and copies the note to R2 every 100 versions. | ||
| binding: NOTES | ||
| - owns: [src/note-room.ts, src/room.ts] | ||
| + owns: [src/note-room.ts, src/room.ts, src/snapshots.ts] | ||
| talks_to: | ||
| - { to: ops, via: import } | ||
| + - { to: snapshots, via: r2 } | ||
| + snapshots: | ||
| + kind: r2-bucket | ||
| + description: >- | ||
| + Bucket live-notes-snapshots. The note's text every 100 versions, at | ||
| + snapshots/<object id>/<version>.txt, kept after the op log has moved on. Written by the | ||
| + note's room, read by the Worker. | ||
| + binding: SNAPSHOTS | ||
| ops: | ||
| kind: library | ||
| description: Insert and delete operations, their transform, and the messages on the socket. The same code runs in the room and in the browser. | ||
Changed.rebasely/architecture/views/editing.yaml+2−2
| @@ -1,2 +1,2 @@ | ||
| -description: What takes part in an edit, from the browser to the room and back. | ||
| -components: [client, ops, note-room] | ||
| +description: What takes part in an edit, from the browser to the room and back, and where the room keeps copies. | ||
| +components: [client, ops, note-room, snapshots] | ||
Changed.rebasely/kb/concepts/conflict-strategy.md+5−1
| @@ -2,7 +2,7 @@ | ||
| id: conflict-strategy | ||
| type: concept | ||
| owner: justin | ||
| -reviewed: 2026-10-14 | ||
| +reviewed: 2026-10-16 | ||
| component: ops | ||
| describes: | ||
| - src/ops.ts | ||
| @@ -58,3 +58,7 @@ | ||
| take the note past 100,000 characters gets a snapshot back instead of an ack. The editor shows the snapshot and drops its pending edits. | ||
| In practice this happens after a long disconnect, and the page disables the textarea while it | ||
| is offline so nothing is typed into the void. | ||
| + | ||
| +Ops older than the log are gone, but the text is not: every 100th version of the note is kept in | ||
| +R2 ([[snapshots]]). Snapshots are not part of conflict handling; the room never transforms | ||
| +against them. | ||
Changed.rebasely/kb/concepts/note-room.md+13−2
| @@ -2,7 +2,7 @@ | ||
| id: note-room | ||
| type: concept | ||
| owner: justin | ||
| -reviewed: 2026-10-14 | ||
| +reviewed: 2026-10-16 | ||
| component: note-room | ||
| describes: | ||
| - src/note-room.ts | ||
| @@ -25,16 +25,24 @@ | ||
| ## Storage | ||
| -Two tables in the object's SQLite database: | ||
| +Three tables in the object's SQLite database: | ||
| - `note`: one row with the text and its version. `Room` caps the text at 100,000 UTF-16 code | ||
| units, so the row and every snapshot frame stay small. | ||
| - `ops`: the op log, keyed by the version each op produced, trimmed to the last 200. | ||
| +- `snapshot`: one row with the version of the last copy saved to R2. | ||
| `commit()` writes the new text, appends the op and trims the log inside one `transactionSync`, | ||
| so a note and its log never disagree. The room only updates its in-memory copy after the | ||
| write. | ||
| +## Snapshots | ||
| + | ||
| +Every 100 versions `Room` asks for the note to be copied to the `SNAPSHOTS` bucket. | ||
| +`webSocketMessage` sends the ack and the broadcast first, then writes the copy and records its | ||
| +version in `snapshot`. A failed write is logged and tried again on the next edit, and only one | ||
| +runs at a time. See [[snapshots]]. | ||
| + | ||
| ## Hibernation | ||
| Sockets are accepted with `ctx.acceptWebSocket`, the Hibernation API. When nobody has typed for | ||
| @@ -50,3 +58,6 @@ | ||
| `GET /notes/:id` calls the `read()` RPC method and returns `{ id, text, version }`. It reads the | ||
| room's in-memory copy, which is always the last committed state. | ||
| + | ||
| +`GET /notes/:id/snapshots` and `GET /notes/:id/snapshots/:version` do not reach the room at all: | ||
| +the Worker reads them from R2 under `idFromName(id)`, the same id the room saves them under. | ||
Added.rebasely/kb/concepts/snapshots.md+65−0
| @@ -0,0 +1,65 @@ | ||
| +--- | ||
| +id: snapshots | ||
| +type: concept | ||
| +owner: justin | ||
| +component: snapshots | ||
| +describes: | ||
| + - src/snapshots.ts | ||
| +reviewed: 2026-10-16 | ||
| +--- | ||
| +# Snapshots in R2 | ||
| + | ||
| +A room keeps only the latest text and its last 200 ops ([[note-room]]). Every 100 versions it | ||
| +also writes the note's text to the `SNAPSHOTS` bucket, so what a note said last week can still | ||
| +be read after the op log has moved on. [[0003-snapshots-in-r2]] has the reasoning. | ||
| + | ||
| +These are the same thing as the `snapshot` message on the socket ([[wire-protocol]]), the text | ||
| +at a version. The difference is that these are kept. | ||
| + | ||
| +## When the room saves one | ||
| + | ||
| +`Room.receive()` (`src/room.ts`) marks an outcome with `saveSnapshot` when the new version is at | ||
| +least `SNAPSHOT_EVERY` (100) past the last saved one. `NoteRoom` sends the ack and the broadcast | ||
| +first, then puts the text in R2, then records the version in its SQLite `snapshot` table. The | ||
| +editors never wait for R2. | ||
| + | ||
| +If the put fails, nothing is recorded, so the next edit is due again and tries once more. Only | ||
| +one save runs at a time per room; edits that arrive while one is in flight do not start another. | ||
| +A failed save is logged as `saving a snapshot failed` with the version. | ||
| + | ||
| +Edits that change nothing, and edits the room refuses, never trigger a save. | ||
| + | ||
| +## Keys | ||
| + | ||
| +``` | ||
| +snapshots/<object id>/<version, 12 digits>.txt | ||
| +snapshots/6f1c...e2/000000000300.txt | ||
| +``` | ||
| + | ||
| +The object id is `idFromName(noteId).toString()`. The room knows it as `ctx.id`, and the Worker | ||
| +can compute it from the URL, so both sides agree on the prefix without the room ever being told | ||
| +its name. The zero padding makes R2 list keys in version order. | ||
| + | ||
| +An object is never overwritten: a version is only saved once, and a note's text at a version | ||
| +never changes. | ||
| + | ||
| +## Reading them | ||
| + | ||
| +The Worker serves them from R2 without touching the room: | ||
| + | ||
| +``` | ||
| +GET /notes/groceries/snapshots | ||
| +{ "id": "groceries", "snapshots": [{ "version": 100, "saved_at": "2026-10-15T09:00:00.000Z", "size": 412 }], "next": null } | ||
| +GET /notes/groceries/snapshots?after=10000 | ||
| +GET /notes/groceries/snapshots/100 | ||
| +``` | ||
| + | ||
| +The list is oldest first, 100 at a time; pass `next` back as `after` for the following page. | ||
| +`saved_at` is when R2 stored the object. One snapshot comes back as `text/plain` with a year of | ||
| +`cache-control`, because it can never change. A version that was never saved is a 404. | ||
| + | ||
| +## Tests | ||
| + | ||
| +`test/snapshots.test.ts` covers when the room asks for a save, retries after a failed one, the | ||
| +recorded version surviving a wake from hibernation, and the key format, saving, reading and | ||
| +paging against an in-memory bucket (`test/helpers/memory-bucket.ts`). | ||
Added.rebasely/kb/decisions/0003-snapshots-in-r2.md+37−0
| @@ -0,0 +1,37 @@ | ||
| +--- | ||
| +id: 0003-snapshots-in-r2 | ||
| +type: decision | ||
| +owner: justin | ||
| +reviewed: 2026-10-16 | ||
| +status: accepted | ||
| +--- | ||
| +# 0003 Snapshots of every note in R2, every 100 versions | ||
| + | ||
| +## Context | ||
| + | ||
| +A room keeps one row with the current text and a log of the last 200 ops, which is all the | ||
| +sequencer needs ([[0001-single-sequencer-per-note]]). Anything older is gone. When someone | ||
| +deletes half a note by accident and notices an hour later, there is nothing to go back to. | ||
| + | ||
| +Keeping more in the Durable Object would work, but every note's history would then live inside | ||
| +its object: reading it means waking the room, and deleting the object deletes its history. | ||
| + | ||
| +## Decision | ||
| + | ||
| +Every 100 versions, the room writes the note's text to an R2 bucket, `live-notes-snapshots`, | ||
| +under `snapshots/<object id>/<version>.txt`. The write happens after the replies, and a failed | ||
| +write is retried on the next edit. The Worker reads the bucket directly for | ||
| +`GET /notes/:id/snapshots`. See [[snapshots]]. | ||
| + | ||
| +## Consequences | ||
| + | ||
| +- A note's history is readable without waking its room, and it is ordinary R2 objects that can | ||
| + be listed, copied or given a lifecycle rule. | ||
| +- History has 100 version steps. What happened between two snapshots is not kept once it leaves | ||
| + the op log. A version is roughly a word of typing, so a step is a sentence or two. | ||
| +- One R2 write per 100 edits per note. A note is at most 100,000 characters, so a thousand edits | ||
| + cost at most about a megabyte of storage, and a typical note far less. | ||
| +- Nothing deletes snapshots yet. When that matters, an R2 lifecycle rule on the `snapshots/` | ||
| + prefix is the place to do it, not the room. | ||
| +- Restoring from a snapshot is by hand for now: read the text, paste it into the editor. The | ||
| + paste is an ordinary edit, so everyone sees it and it is in the next snapshot. | ||
Changed.rebasely/kb/guides/local-dev.md+14−2
| @@ -2,7 +2,7 @@ | ||
| id: local-dev | ||
| type: guide | ||
| owner: justin | ||
| -reviewed: 2026-10-13 | ||
| +reviewed: 2026-10-16 | ||
| --- | ||
| # Running it locally | ||
| @@ -29,6 +29,18 @@ | ||
| curl http://localhost:8787/notes/groceries | ||
| ``` | ||
| +## Snapshots | ||
| + | ||
| +wrangler dev gives the Worker a local R2 bucket for `SNAPSHOTS`. Type a hundred edits into a | ||
| +note (pasting a word a hundred times works), then: | ||
| + | ||
| +```bash | ||
| +curl http://localhost:8787/notes/groceries/snapshots | ||
| +curl http://localhost:8787/notes/groceries/snapshots/100 | ||
| +``` | ||
| + | ||
| +See [[snapshots]] for what is saved and when. | ||
| + | ||
| ## Watching the protocol | ||
| The browser's developer tools show the frames on the socket under the Network tab, on the | ||
| @@ -37,4 +49,4 @@ | ||
| ## Starting over | ||
| -Delete `.wrangler/state` to forget every note. | ||
| +Delete `.wrangler/state` to forget every note and its snapshots. | ||
Changed.rebasely/kb/index.md+2−0
| @@ -8,11 +8,13 @@ | ||
| - [[conflict-strategy]]: how two people typing at once end up with the same text. | ||
| - [[note-room]]: the Durable Object behind a note, its storage, and hibernation. | ||
| - [[wire-protocol]]: the messages on the socket. | ||
| +- [[snapshots]]: the copy of every note in R2 every 100 versions, and how to read them. | ||
| ## Decisions | ||
| - [[0001-single-sequencer-per-note]]: the room is the only place edits are ordered. | ||
| - [[0002-drop-inserts-inside-concurrent-deletes]]: the one rule that keeps the transform small. | ||
| +- [[0003-snapshots-in-r2]]: history lives in R2, not in the room. | ||
| ## Guides | ||
ChangedREADME.md+10−6
| @@ -18,7 +18,8 @@ | ||
| stays where it was while others type. | ||
| The room keeps the note and its last 200 operations in the Durable Object's SQLite database | ||
| -and accepts sockets with the Hibernation API. A note nobody is typing in is evicted from | ||
| +and accepts sockets with the Hibernation API. Every 100 versions it also saves a copy of the | ||
| +text to R2, so older versions can still be read. A note nobody is typing in is evicted from | ||
| memory without closing anyone's connection, and wakes up on the next keystroke. | ||
| ## Running it locally | ||
| @@ -36,11 +37,13 @@ | ||
| ## API | ||
| -| Request | Response | | ||
| -| ---------------------- | --------------------------------------------------------------- | | ||
| -| `GET /` | the editor page; the part after `#` is the note id | | ||
| -| `GET /notes/:id` | the note as JSON: `{ "id", "text", "version" }` | | ||
| -| `GET /notes/:id/ws` | a WebSocket to the note's room | | ||
| +| Request | Response | | ||
| +| ----------------------------------- | -------------------------------------------------------- | | ||
| +| `GET /` | the editor page; the part after `#` is the note id | | ||
| +| `GET /notes/:id` | the note as JSON: `{ "id", "text", "version" }` | | ||
| +| `GET /notes/:id/ws` | a WebSocket to the note's room | | ||
| +| `GET /notes/:id/snapshots` | the note's saved versions, oldest first; `?after=` pages | | ||
| +| `GET /notes/:id/snapshots/:version` | the note's text at that version, as plain text | | ||
| Note ids are 1 to 64 letters, digits, dashes or underscores. On the socket, the editor sends | ||
| @@ -72,6 +75,7 @@ | ||
| routes.ts path matching for the Worker | ||
| note-room.ts the Durable Object: sockets, hibernation, SQLite | ||
| room.ts the sequencer, with storage behind an interface | ||
| + snapshots.ts copies of the note in R2: keys, saving, listing | ||
| ops.ts insert, delete, transform, compose, diff | ||
| protocol.ts the socket messages and their validation | ||
| client/ | ||
Checks
1 check passed| Status | Check | Reported by | Description | Details |
|---|---|---|---|---|
| Success | duragit/ci/test | Rebasely Actions | succeeded | Run |
Runs
| Status | Workflow | Event | Started |
|---|---|---|---|
| Success | ci | Push |