Fouria Dispatcher
Wildcard DNS dispatch worker that routes *.fouria.io requests to the correct tenant worker.
Wildcard dispatch Worker for FourIA’s multi-tenant OpenClaw gateway platform. Routes all *.fouria.io traffic to the correct tenant worker, enforces Cloudflare Access auth per-tenant via JWT audience matching, and applies plan-level CPU/subrequest limits.
Goal
The dispatch Worker is the entry point for all tenant traffic. On every request it:
- Extracts the tenant slug from the hostname (
tenant-a.fouria.io→tenant-a) - Resolves the tenant — checks KV cache (TTL: 60s), falls back to a Supabase query
- Enforces auth — verifies the Cloudflare Access JWT signature. In production, non-ROOT callers must present a JWT
audthat resolves (via the Cloudflare API) to the requested hostname’s slug; Paso4 ROOT operators are exempt (platform wildcard AUD*.fouria.ioor membership of thePaso4 Root UsersAccess group). Staging is ROOT-only (platform AUD or ROOT group membership), except the tenant health endpoints (/api/health,/api/statusand their sub-paths): those path-scoped Access apps carry no role requirement — they admit any authenticated user and lerma’sFourIA Health Monitorservice token — so Dispatch lets a request that already passed Access through on those paths (every other path stays ROOT-only). - Applies the operational gate — rejects
draft/suspended/archivedand answers 503 forprovisioningand forfrozen/stopped(ROOT-exempt). A health-derivederrornever blocks (the container is loaded lazily). - Resolves the forwarded role —
X-FourIA-Rolecarries the lerma-assigned client role (client_user_instances). On adevinstance a client-level ADMIN (validatedclient_usersrow) is admitted even without an explicit dev-instance assignment, since admins are the ones developing there; BASE_USERs still need an explicit dev-instance assignment. - Builds billing metadata — queries
clients,credit_pools, andclient_userstables, encodes as base64 JSON inX-FourIA-Metadata - Dispatches — forwards the request via
DISPATCHER.get(workerName)with plan-level limits and billing context
Architecture
Request → *.fouria.io
│
├─► extractSlug() — parse tenant from hostname
│ OR apex (fouria.io) — resolve slug from Access JWT aud via Cloudflare API
│
├─► resolveTenant()
│ ├─► KV cache (slug:<tenant>) — 60s TTL
│ └─► Supabase query (clients + credit_pools + client_users)
│
├─► requireAccessAuth() — Cloudflare Access JWT (jose)
├─► resolveSlugByAud() — Cloudflare SDK (production hostname + apex)
│
├─► tenantRouteGate() — lifecycle + frozen/stopped (ROOT-exempt); never health
│
├─► buildBillingMetadata() → X-FourIA-Metadata base64 header
│
└─► DISPATCHER.get(workerName).fetch() — plan limits + billing contextData flow
The dispatch Worker resolves tenants by querying Supabase (shared Postgres backing lerma) using SUPABASE_URL + SUPABASE_SERVICE_ROLE_KEY. No HTTP dependency on the dashboard (lerma). The Supabase queries span three tables:
| Table | Columns used |
|---|---|
clients | id, slug, billing_plan, status |
credit_pools | lifetime_consumed, lifetime_purchased |
client_users | count by client_id (seats used) |
Apex tag routing
Requests to fouria.io (host === PLATFORM_DOMAIN) are resolved differently:
- Verify the Cloudflare Access JWT signature (
jose) - Extract the
audclaim from the JWT payload - Call the Cloudflare API (
cloudflareSDK) to list Access apps filtered byaud - Find the app whose
audmatches; read its domain ({slug}.fouria.io) to derive the slug - Dispatch to
tenant-{slug}in the production dispatch namespace
Staging tenants are NOT reachable via apex — they only use the hostname path (staging-{slug}.fouria.io).
Auth via Cloudflare SDK
JWT signature/JWKS verification uses jose (as before). Tenant-by-aud resolution uses the official cloudflare npm package (github.com/cloudflare/cloudflare-typescript). Per-tenant AUD is NOT persisted in the database anymore — the dispatch worker resolves it dynamically from the Cloudflare Access app configuration.
Staging Prefix
Staging tenants use the staging-{slug}.fouria.io pattern to avoid collisions with production tenants. The dispatch Worker handles both production ({slug}.fouria.io) and staging (staging-{slug}.fouria.io) subdomains on the same PLATFORM_DOMAIN. When extractSlug() encounters a staging- prefix, it strips the prefix and routes to the fouria-tenants-staging dispatch namespace with the un-prefixed slug.
| Subdomain pattern | Namespace | Example |
|---|---|---|
{slug}.fouria.io | fouria-tenants | acme.fouria.io → tenant-acme |
staging-{slug}.fouria.io | fouria-tenants-staging | staging-acme.fouria.io → tenant-acme |
Dashboard traffic (
dash.fouria.io,dev-dash.fouria.io) is served via bypass worker routes on the fouria.io zone — the Dispatch Worker does not intercept these hostnames.
Client environments and the routing gate
Routing has two independent axes (see Lerma.Instances.Naming):
| Axis | Encoded as | Example |
|---|---|---|
| Platform | staging- hostname prefix | staging-acme.fouria.io → tenant-acme |
| Client environment | -dev hostname / worker-name suffix | acme-dev.fouria.io → tenant-acme-dev |
resolveTenant() parses both axes: the -dev suffix selects the client’s dev
environment, the staging- prefix selects the staging platform, and the base
slug is looked up in clients with the instances.environment filter applied.
The operational gate before dispatch depends on the environment:
- production —
clients.statusgates routing:draft/suspended/archivedare rejected (404) andprovisioningreturns 503. A client left inerrorby the HealthMonitor is routable. - dev — the instance is the source of truth. A
devinstance is provisioned lazily while the client is stilldraft(a dev provision never drives the client status), soclients.status = draftmust NOT block it. Adevrequest routes when the instance exists and isrunning/upgrading/error; it is rejected when the instance is missing (404) orprovisioning(503).
frozen/stopped instances return 503 for non-ROOT callers (billing
protection), but a ROOT operator is exempt so they can always reach the
worker to revive it. Explicit operator revocations (clients.status = suspended/archived) block both environments.
A health-derived error status is never a routing gate: tenant containers
are loaded lazily by the User Worker, so a tenant the HealthMonitor could not
reach is not an outage, and blocking it would prevent operators from reviving
it. The gate runs after authentication so the ROOT exemption can be applied.
Role resolution on a dev instance
The forwarded role (X-FourIA-Role) resolves from the user’s client_users
row linked to the requested instance. A dev instance has an extra fallback:
when no per-instance role is found, a validated client-level ADMIN is
granted access — client admins develop on the dev instance and are typically
only assigned to the production instance. BASE_USERs are not granted dev
access this way; ROOT operators are unaffected (they resolve before the
fallback).
Separately, the tenant Worker lets a resolved ROOT operator view and
perform as the tenant’s ADMIN on a dev instance via the dashboard’s “View
as” switcher (a dev-only, strict-downgrade role preview — see
apps/fouria/docs/WORKER-DEVELOPMENT.md §RBAC).
That is a tenant-side concern; Dispatch still forwards the real ROOT role.
Periodic backup fan-out
The Worker’s cron trigger sweeps every tenant in TENANT_INDEX (slug: keys)
and forwards a backup control action to each (POST /control, header
X-FourIA-Control: backup). Deterministic per-tenant delays spread the fleet
across a jitter window so snapshots do not all start on the same wall-clock
minute.
Sweep and queue semantics
- Paginated sweep.
collectTenantSlugsfollows KVlist_complete/cursor, so every tenant is reached even past KV’s 1,000-key page (the previous single-page loop silently starved tenant #1,001 onward). Keys are deduped in aSet. - Queue fan-out. When the
CONTROL_QUEUEproducer is bound, the sweep sends one message per tenant ({ slug, requestedAt }) in batches of at most 100 — Cloudflare’ssendBatchlimit — instead of holding one subrequest per tenant in the cron invocation. The consumer (queue()insrc/index.ts) resolvesslug:<slug>against the live index at delivery time, so a tenant frozen or deleted after the sweep is skipped rather than woken. Missing/frozen/nameless entries are skipped;deliveredandskippedareacked,failedis retried. - Per-tenant jitter. Each queued message carries
delaySecondsderived deterministically from the slug (FNV-1a,0..CONTROL_FANOUT_JITTER_SECONDS, default 300 s). Determinism keeps a redelivered sweep on the same schedule; the offset is queue-only, so the direct fallback delivers immediately. - Bounded fallback. Without the queue binding (local dev or an environment that has not created it yet), the sweep delivers directly with a concurrency cap of 8, so one slow tenant cannot serialize the whole fleet. Delivery is idempotent: the tenant enqueues a durable snapshot job and the backup gate is interval/lock-based.
The tenant answers without ever starting the container: it reads
sandbox.isRuntimeActive() (a non-waking bridge call) and returns
{ ok: true, skipped: 'not-running' } for a sleeping instance. Only a live
instance proceeds to the interval/lock-gated scheduleBackupIfNeeded. This is
the backstop for tenants the lerma HealthMonitor does not poll, and because it
never wakes an idle container it does not defeat sleepAfter or restart billing.
Health probes themselves are observability-only and never write the backup
catalog.
Queue names: fouria-dispatch-control (local), -staging, -prod. The
deploy-dispatch workflow creates them idempotently (wrangler queues info … || wrangler queues create …) before deploying. Set
CONTROL_FANOUT_JITTER_SECONDS=0 to disable jitter, or another positive value
(capped at 86400) to widen/narrow the window.
Pre-replacement checkpoint
POST /_control/:slug/checkpoint forwards a synchronous, verified checkpoint
(the checkpoint control action) to the tenant worker with the same raised CPU
budget as backup (BACKUP_CONTROL_CPU_MS). The Lerma control plane calls it
immediately before an image rollout / container replacement so the state about
to be discarded is durably captured first. Unlike backup it is not
interval-gated and fails closed: the tenant worker only reports success once
the committed archive is restorable, and a failure propagates as a non-2xx so the
replacement is aborted. A sleeping/destroyed instance is reported as
skipped: not-running (never woken).
Sandbox v1 migration
POST /_control/:slug/migrate forwards the migrate control action to the
tenant worker with the same raised CPU budget as backup/checkpoint
(BACKUP_CONTROL_CPU_MS). The tenant worker only enqueues: it records a
durable job on its SandboxV1 Durable Object and answers immediately, so the
request never spans the archive round-trip (the conversion runs on the DO
alarm). The tenant worker is the authority for the migration (the staging-only
platform gate, the reserved client slug, and the conversion lease), so a
refusal arrives as 403/409 with its code preserved.
POST /_control/:slug/migration-status forwards the read-only
migration-status action (same route, no raised CPU budget, no tenant-cache
invalidation). It exists because migrate is an idempotent enqueue — it starts
work from absent/failed — so a poll that reused it would convert a tenant
just by asking. migration-status reads the durable state and can never enqueue.
Both echoes carry the durable conversion state, which the Lerma control plane renders:
| Tenant answer | Forwarded status | Echo |
|---|---|---|
{state: "queued" | "running"} | 202 | state |
{state: "committed", recordId} | 200 | state, recordId |
{state: "absent"} (status only) | 200 | state |
{state: "failed", error} | 502 (migrate) | state, tenantError |
{ok: false, code: "MIGRATION_…"} | 403/409 | code, tenantError (no invented state) |
A failed conversion is a durable state, not a transport error, so its state
is echoed on the failure path too — the control plane distinguishes “the
conversion broke” from “the request was refused” by the presence of state
versus code. Only POST /_control/:slug/migrate can start a conversion;
migration-status is safe to poll.
Environment Variables
Wrangler vars (plaintext)
| Variable | Staging | Production | Description |
|---|---|---|---|
PLATFORM_DOMAIN | fouria.io | fouria.io | Domain suffix for tenant slug extraction |
| (staging prefix) | staging-{slug} | (not used in production) | Staging tenants use staging-{slug}.fouria.io via dispatch |
CF_ACCESS_TEAM_DOMAIN | paso4.cloudflareaccess.com | paso4.cloudflareaccess.com | Cloudflare Access team domain for JWT auth |
SUPABASE_URL | (set per env) | (set per env) | Supabase project URL |
CF_ACCOUNT_ID | 35afea16440634aa2350331d2a736eec | 35afea16440634aa2350331d2a736eec | Cloudflare account ID (used by Cloudflare SDK) |
CONTROL_FANOUT_JITTER_SECONDS | (optional) | (optional) | Per-tenant fan-out delay window; default 300, 0 disables, max 86400 |
Wrangler secrets
| Secret | Description |
|---|---|
SUPABASE_SERVICE_ROLE_KEY | Supabase service role key (bypasses RLS) |
CF_ACCESS_PLATFORM_AUD | Cloudflare Access audience for Paso4 platform / staging auth (ROOT is also granted by Paso4 Root Users group membership) |
CF_ACCESS_API_TOKEN | Cloudflare API token (Access:Read) |
Bound resources
| Binding | Resource | Description |
|---|---|---|
TENANT_INDEX | KV namespace 081f07b1e0e54fb2abd1253626b05880 (shared) | Tenant cache (slug → TenantEntry) |
DISPATCHER | Dispatch namespace fouria-tenants (prod) / fouria-tenants-staging (staging) | Tenant worker dispatch |
CONTROL_QUEUE | Queue `fouria-dispatch-control[-staging | -prod]` |
Provisioning Environment Variables
All env vars are defined in wrangler.jsonc under vars (top-level for local) and env.staging.vars / env.production.vars. To add or update a variable:
1. Edit wrangler.jsonc
// workers/fouria-dispatch/wrangler.jsonc
"vars": {
"YOUR_VAR": "value",
},
"env": {
"staging": {
"vars": {
"YOUR_VAR": "staging-value",
},
},
"production": {
"vars": {
"YOUR_VAR": "production-value",
},
},
}
Important: vars are not inherited between environments. If a var is needed in production, it must be defined in
env.production.varsexplicitly.
2. Add the TypeScript type
Update DispatchEnv interface in src/index.ts:
export interface DispatchEnv {
// ...
YOUR_VAR: string;
}3. If the var is a secret (not plaintext)
Use wrangler secret put instead of wrangler.jsonc:
npx wrangler secret put YOUR_SECRET --env staging
npx wrangler secret put YOUR_SECRET --env production
Secrets are available as env.YOUR_SECRET at runtime, same as vars. They do not need to be declared in wrangler.jsonc but do need the TypeScript type in DispatchEnv.
Internal control-plane authentication
GET /_invalidate/:slug and POST /_control/:slug/:action require the Worker
secret INTERNAL_DISPATCH_INVALIDATE_TOKEN as Authorization: Bearer <token>.
The Worker compares SHA-256 digests in constant time and answers 403 when the
secret is unset or the credential does not match — there is no hardcoded
fallback. wrangler.jsonc lists it under secrets.required in every
environment, and deploy-dispatch.yml uploads it from GitHub
(INTERNAL_DISPATCH_INVALIDATE_TOKEN for production,
STAGING_INTERNAL_DISPATCH_INVALIDATE_TOKEN for staging) before deploying.
Lerma must send the matching value; deploy the Worker first in each environment.
4. Deploy
# Staging
bun run deploy:staging
# Production
bun run deployCloudflare API Token Permissions
This project requires a Cloudflare API token (CF_ACCESS_API_TOKEN) with these minimum permissions:
| Permission | Reason |
|---|---|
Access:Apps & Policies:Read | Resolve tenant slugs from Access JWT audience claims |
Access:Groups:Read | Resolve Paso4 Root Users group membership for ROOT operator detection |
The token is deployed as a worker secret and used at runtime for every request that goes through apex routing. All credentials are documented in docs/schemas/secrets-manifest.schema.json.
Supabase
The shared Postgres database (backing lerma). Key tables and columns used by dispatch:
clients
| Column | Type | Purpose |
|---|---|---|
id | uuid | Primary key (used to join credit_pools/client_users) |
slug | TEXT | Unique tenant identifier (from hostname) |
billing_plan | TEXT | Plan name (free, basic, pro, enterprise) |
status | TEXT | Client status. Production routes for active/production/error (a health error never blocks); a dev instance routes on its own instance status even when the client is draft |
credit_pools
| Column | Type | Purpose |
|---|---|---|
client_id | uuid | FK to clients.id |
lifetime_consumed | int8 | Credits consumed by this tenant |
lifetime_purchased | int8 | Credits purchased for this tenant |
client_users
| Column | Type | Purpose |
|---|---|---|
client_id | uuid | FK to clients.id (counted for seats) |
Local Development
bun install
bun run test # Vitest with @cloudflare/vitest-plugin
bun run typecheck
Tests use miniflare with ephemeral in-memory KV and mocked Supabase.