examples/MCP-tools Public
Record the rate limit decision and how the limiter works
1 check passedDecision 0004 says why the limit is exact and per IP, why a refusal is a tool result rather than a 429, and why the limiter fails open. The new rate-limits concept walks through the bucket and the Durable Object, and the README and client guide say what a limited client sees.
Changed files
7 files changed71 additions6 deletionsagainst dae155a
- Changed.rebasely/kb/concepts/http-transport.md+1−1
- Added.rebasely/kb/concepts/rate-limits.md+29−0
- Changed.rebasely/kb/concepts/tool-design.md+1−1
- Added.rebasely/kb/decisions/0004-per-client-rate-limit.md+29−0
- Changed.rebasely/kb/guides/connecting-a-client.md+2−1
- Changed.rebasely/kb/index.md+3−1
- ChangedREADME.md+6−2
Changed.rebasely/kb/concepts/http-transport.md+1−1
| @@ -34,7 +34,7 @@ | ||
| ## Rate limit | ||
| -`createServer` takes an `admit` hook, asked once per tool call after the arguments pass the input schema. The Worker answers it from the caller's `RateLimiter` object, named by the `cf-connecting-ip` header. A refusal comes back to the model as an `isError` result saying how many seconds to wait; `initialize` and `tools/list` are never charged. If the Durable Object call throws, the call is admitted and the error logged. | ||
| +`createServer` takes an `admit` hook, asked once per tool call after the arguments pass the input schema. The Worker answers it from the caller's `RateLimiter` object, named by the `cf-connecting-ip` header. A refusal comes back to the model as an `isError` result saying how many seconds to wait; `initialize` and `tools/list` are never charged. If the Durable Object call throws, the call is admitted and the error logged. [[rate-limits]] covers the bucket itself. | ||
| ## Usage counting | ||
Added.rebasely/kb/concepts/rate-limits.md+29−0
| @@ -0,0 +1,29 @@ | ||
| +--- | ||
| +id: rate-limits | ||
| +type: concept | ||
| +owner: justin | ||
| +component: rate-limiter | ||
| +describes: | ||
| + - src/rate-limiter/** | ||
| +reviewed: 2026-10-15 | ||
| +--- | ||
| +# Rate limits | ||
| + | ||
| +Each client gets a token bucket: 30 tool calls to start, one more every two seconds, never more than 30. [[0004-per-client-rate-limit]] has the reasoning. | ||
| + | ||
| +## Where the pieces are | ||
| + | ||
| +- `src/rate-limiter/bucket.ts`: the arithmetic. `take(bucket, now)` refills by the time elapsed, spends one token if there is one, and returns the new bucket and a decision: `{ ok: true, remaining }` or `{ ok: false, retryAfterSeconds }`. Pure, so `test/rate-limit.test.ts` drives it with a fake clock. | ||
| +- `src/rate-limiter/index.ts`: the `RateLimiter` Durable Object, bound as `RATE_LIMITER`. Its one RPC method, `take()`, loads the bucket from storage, runs the arithmetic, saves the result and returns the decision. | ||
| +- `src/worker.ts`: names the object by the caller's `cf-connecting-ip` and turns a refusal into the message the model sees. See [[http-transport]]. | ||
| + | ||
| +## Behavior worth knowing | ||
| + | ||
| +- A refused call does not spend anything, so a client that waits the time it was told always gets through. | ||
| +- A clock that steps backwards adds no tokens. | ||
| +- After each call the object sets an alarm for the moment its bucket would be full. If nothing else happens first, the alarm deletes the object's storage, and the next call starts from a fresh, full bucket, which is the same state. | ||
| +- Durable Objects handle one request at a time, so concurrent calls from one client cannot both spend the last token. | ||
| + | ||
| +## Changing the limit | ||
| + | ||
| +Edit `LIMIT` in `bucket.ts`. Buckets already stored keep their token count and refill toward the new capacity. Update the numbers in [[0004-per-client-rate-limit]] only by writing a new decision that supersedes it. | ||
Changed.rebasely/kb/concepts/tool-design.md+1−1
| @@ -39,7 +39,7 @@ | ||
| Unknown time zone "Mars/Olympus_Mons". Use an IANA name such as Europe/Paris, Asia/Tokyo or America/New_York. | ||
| ``` | ||
| -Arguments that fail the input schema never reach `run`; the SDK answers them with an `isError` result too. A call the `admit` hook refuses (the per-client rate limit) is answered the same way, with how long to wait, and the tool never runs. Any other exception is a bug: the server logs it and tells the model that retrying the same input will not help. | ||
| +Arguments that fail the input schema never reach `run`; the SDK answers them with an `isError` result too. A call the `admit` hook refuses (the per-client rate limit, [[rate-limits]]) is answered the same way, with how long to wait, and the tool never runs. Any other exception is a bug: the server logs it and tells the model that retrying the same input will not help. | ||
| ## Annotations | ||
Added.rebasely/kb/decisions/0004-per-client-rate-limit.md+29−0
| @@ -0,0 +1,29 @@ | ||
| +--- | ||
| +id: 0004-per-client-rate-limit | ||
| +type: decision | ||
| +owner: justin | ||
| +reviewed: 2026-10-15 | ||
| +status: accepted | ||
| +--- | ||
| +# 0004 Per-client rate limit in a Durable Object | ||
| + | ||
| +## Context | ||
| + | ||
| +The tools are cheap, but not free, and a model stuck in a loop will call one hundreds of times a minute. The usage counters in KV cannot stop that: they are approximate and eventually consistent by design ([[0003-usage-counts-in-kv]]). A limit has to be exact at the moment it is checked, which needs a single writer per client. | ||
| + | ||
| +Workers has a rate limiting binding, but its counters are per location and approximate, and it cannot tell us how long a refused client should wait. | ||
| + | ||
| +## Decision | ||
| + | ||
| +- One `RateLimiter` Durable Object per client, named by `cf-connecting-ip`. It holds a token bucket: 30 tool calls in a burst, refilling one token every two seconds. | ||
| +- Only `tools/call` spends tokens, and only after the arguments pass the input schema. `initialize` and `tools/list` are free. | ||
| +- A refused call returns an `isError` tool result that says how many seconds to wait, in line with [[0002-tool-errors-are-results]]. Not an HTTP 429: most clients surface a 429 as a transport failure the model never reads, or retry it at once. | ||
| +- When the bucket is full again the object deletes its storage from an alarm, so idle clients cost nothing. | ||
| +- If the Durable Object call fails, the tool call is admitted and the error logged. | ||
| + | ||
| +## Consequences | ||
| + | ||
| +- Every tool call now makes one RPC to a Durable Object before it runs. The object lives near the client's first call, so that hop is usually short. | ||
| +- Clients behind one NAT or proxy share a bucket. Without auth there is nothing better to key on; if the server ever takes tokens, the key should become the token. | ||
| +- Failing open means a broken limiter lets a loop through. We prefer that to failing every call when the limiter has a bad minute. | ||
| +- The stateless transport ([[0001-stateless-streamable-http]]) is unchanged: the limiter keeps state per client, not per session. | ||
Changed.rebasely/kb/guides/connecting-a-client.md+2−1
| @@ -2,7 +2,7 @@ | ||
| id: connecting-a-client | ||
| type: guide | ||
| owner: justin | ||
| -reviewed: 2026-10-13 | ||
| +reviewed: 2026-10-15 | ||
| --- | ||
| # Connecting a client | ||
| @@ -55,4 +55,5 @@ | ||
| - **406 Not Acceptable**: the client did not send both `application/json` and `text/event-stream` in `Accept`. | ||
| - **415 Unsupported Media Type**: the body was not sent as `application/json`. | ||
| - **405 on GET**: the client is trying to open a standalone SSE stream or resume a session. Pick a Streamable HTTP client mode that does not need one; older SSE-only clients are not supported. | ||
| +- **"Rate limit reached for this client"**: the client made more than 30 tool calls in a short burst. Wait the number of seconds the message gives; one call every two seconds is always allowed. Clients behind the same IP share the limit. | ||
| - **Tools missing**: check `GET /` answers, then run `tools/list` with curl to see what the server sends. | ||
Changed.rebasely/kb/index.md+3−1
| @@ -1,11 +1,12 @@ | ||
| # mcp-tools knowledge base | ||
| -mcp-tools is a small MCP server on Cloudflare Workers. Three read-only tools (`convert_units`, `time_in_zone`, `text_stats`) answer from their input alone; the Worker serves them over Streamable HTTP at `/mcp` and counts calls in KV. | ||
| +mcp-tools is a small MCP server on Cloudflare Workers. Three read-only tools (`convert_units`, `time_in_zone`, `text_stats`) answer from their input alone; the Worker serves them over Streamable HTTP at `/mcp`, limits each client's tool calls with a Durable Object and counts calls in KV. | ||
| ## How it works | ||
| - [[tool-design]]: what every tool has (name, description for the model, schemas, structured output) and how failures reach the model. | ||
| - [[http-transport]]: how a POST to `/mcp` becomes an MCP response, and what the Worker does around it. | ||
| +- [[rate-limits]]: the per-client token bucket and the Durable Object that holds it. | ||
| - [[usage-counts]]: what `/stats` counts, where, and how far to trust it. | ||
| ## Why it is this way | ||
| @@ -13,6 +14,7 @@ | ||
| - [[0001-stateless-streamable-http]]: a fresh server per request, no sessions. | ||
| - [[0002-tool-errors-are-results]]: failures come back as tool results the model can read. | ||
| - [[0003-usage-counts-in-kv]]: approximate counters in KV rather than a Durable Object. | ||
| +- [[0004-per-client-rate-limit]]: an exact limit per client IP, refused calls answered as tool results. | ||
| ## How to | ||
ChangedREADME.md+6−2
| @@ -2,7 +2,7 @@ | ||
| A small [Model Context Protocol](https://modelcontextprotocol.io) server on Cloudflare Workers. It gives models three exact, read-only tools for things they tend to get wrong: unit arithmetic, the current time somewhere else, and counting words. | ||
| -The server speaks Streamable HTTP at `/mcp`, holds no sessions, and counts tool calls per day in a KV namespace. | ||
| +The server speaks Streamable HTTP at `/mcp`, holds no sessions, limits each client to a burst of 30 tool calls with a Durable Object, and counts tool calls per day in a KV namespace. | ||
| ## Tools | ||
| @@ -109,13 +109,14 @@ | ||
| npm run dev # wrangler dev on http://localhost:8787 | ||
| ``` | ||
| -`npx wrangler deploy` deploys it. The `USAGE` namespace in `wrangler.jsonc` has no id, so the first deploy creates it. | ||
| +`npx wrangler deploy` deploys it. The `USAGE` namespace in `wrangler.jsonc` has no id, so the first deploy creates it; the `RateLimiter` class is created by the `v1` migration. | ||
| ## Design notes | ||
| - **Stateless.** Every POST gets a fresh `McpServer` and a `WebStandardStreamableHTTPServerTransport` with no session id and JSON responses. Any isolate can answer any request; nothing needs a Durable Object. | ||
| - **Tools are pure.** Each tool in `src/tools/` is a name, a description written for the model, zod input and output schemas, and a function from input to output. `src/server.ts` is the only file that knows about MCP. | ||
| - **Errors are results.** A tool throws `ToolError` with advice ("Use an IANA name such as Europe/Paris"), and the server returns it with `isError: true` so the model can read it and try again. | ||
| +- **A limit per client.** Each caller IP gets a `RateLimiter` Durable Object with a token bucket: 30 calls, then one every two seconds. A refused call is an `isError` result that says how long to wait, so the model backs off instead of the client seeing a 429. | ||
| - **Counting stays off the request path.** Usage writes go through `ctx.waitUntil`, after the response. The counts are approximate, which is fine for seeing what gets used. | ||
| The longer versions are in the knowledge base under `.rebasely/kb/`, and the component map is `.rebasely/architecture/system.yaml`. | ||
| @@ -128,6 +129,9 @@ | ||
| worker.ts routes: /mcp, /stats, CORS | ||
| server.ts createServer(): registers the tools with McpServer | ||
| usage.ts per-day counters in KV | ||
| + rate-limiter/ | ||
| + index.ts RateLimiter Durable Object | ||
| + bucket.ts token bucket arithmetic | ||
| tools/ | ||
| convert-units.ts | ||
| time-in-zone.ts | ||
Checks
1 check passed| Status | Check | Reported by | Description | Details |
|---|---|---|---|---|
| Success | duragit/ci/test | Rebasely Actions | succeeded | Run |
Runs
| Status | Workflow | Event | Started |
|---|---|---|---|
| Success | ci | Push |