# FourIA — Environment Variables Reference

Complete reference of all environment variables used across FourIA Workers. Derived from `apps/fouria/src/gateway/env.ts` and `apps/fouria/.dev.vars.example`.

---

## Variable Classification

FourIA uses three classifications for configuration values:

| Classification      | Storage                                   | Binding Type  | Managed By                   | Examples                                    |
| ------------------- | ----------------------------------------- | ------------- | ---------------------------- | ------------------------------------------- |
| **Instance Secret** | CF encrypted `secret_text`                | `secret_text` | lerma provisioner            | Gateway token, CDP secret, AI keys, R2      |
| **Variable**        | `wrangler.jsonc` / WFP `plain_text`       | `plain_text`  | Provisioner / IaC            | DEV_MODE, URLs, account IDs, sandbox config |
| **Dynamic Secret**  | F4E1-encrypted R2 object in BACKUP_BUCKET | —             | End-user via integrations UI | Channel tokens, OAuth, custom integrations  |

### Instance Secrets

These are automatically generated and deployed by the `lerma` provisioner as `secret_text` bindings on each tenant User Worker. For the full inventory of required secrets, see the secrets manifest.

| Secret                   | Description                                                                                 |
| ------------------------ | ------------------------------------------------------------------------------------------- |
| `MOLTBOT_GATEWAY_TOKEN`  | Auto-generated gateway token for CLI auth. Mapped to `OPENCLAW_GATEWAY_TOKEN` in container. |
| `CF_ACCESS_PLATFORM_AUD` | Platform-level Access AUD (always present, from dispatch worker).                           |
| `CF_ACCESS_CLIENT_AUD`   | Per-tenant Access AUD (optional, provisioned by lerma).                                     |

> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.

### Variables (Non-Secret Config)

Operational configuration, safe to store in `wrangler.jsonc` or `.dev.vars`. Deployed as `plain_text` bindings by the provisioner.

| Variable                          | Container | Required | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
| --------------------------------- | --------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `DEV_MODE`                        | Yes†      | No       | Skips CF Access auth + bypasses device pairing. Local dev only.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `OPENCLAW_DEV_MODE`               | Yes       | No       | Explicit override for the dev-agents flag. Normally derived from the instance's own `-dev` hostname suffix (`WORKER_URL`), which enables the dev-only Fouria Builder/Q&A agents, client-doc materialization, the dev-only ROOT "view as" role preview, and the `opencode-credits-check` automation. Does **not** bypass Cloudflare Access auth. Falls back to `DEV_MODE` for local development.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `E2E_TEST_MODE`                   | Yes‡      | No       | Skips CF Access auth but keeps device pairing. For automated E2E.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `BILLABLE_EVENTS`                 | No        | No       | Opt-in for **billable-action emission on a dev instance** (the `billable-action` Analytics Engine rows + `BILLING_QUEUE` enqueue lerma reconciles into credits). Billable production and the **staging platform always emit** and cannot be opted out (staging mirrors production so the billing pipeline is exercised before release; its rows land in the staging Analytics Engine dataset). A dev environment is silent by default and emits only with `true` — useful for exercising billing end-to-end from a dev instance. Emission only: the `platform`/`environment` cost-row dimensions and the container sleep policy are unchanged, so an opted-in dev instance still auto-sleeps (and staging keeps its 15m idle timeout).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
| `DEBUG_ROUTES`                    | No        | No       | Enables `/debug/*` endpoints (ROOT role only). Off in production. lerma provisions dev instances with `DEBUG_ROUTES=true` by default; toggle per instance from the dashboard.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `DEMO_MODE`                       | No        | No       | Enables demo-specific UI behaviors (read-only, sample data).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `SANDBOX_RUNTIME`                 | No        | No       | Requested container runtime: `legacy` (default) or `v1`. It is **not** the authority for serving: on the trusted staging platform the runtime comes from the durable conversion intent, else from the fresh-v1 marker (`sandbox-migration/<slug>/runtime.json`, written by Lerma at provisioning). The flag only makes an unprovisioned tenant **fail closed**: a staging tenant with no intent and no marker resolves to `pending` (`503 SANDBOX_V1_CONVERSION_PENDING`) instead of falling back to legacy. See `apps/fouria/docs/WORKER-DEVELOPMENT.md` § "Provisioning an instance directly on Sandbox v1". The container idle timeout is **derived, not configured**: staging 15m / dev 2h / production 6h (the Sandbox SDK 1.0 platform cap), from the instance hostname — there is no `SANDBOX_SLEEP_AFTER` binding. **Only non-billable instances auto-sleep.** Billable production keeps the container awake with the 6h platform maximum (`setInactivityTimeout`) renewed by a recurring keep-alive alarm on the `SandboxV1` DO (`resolveInactivityPolicy` → `keepAlive: true`); it is stopped only by an explicit lerma freeze (`POST /control` → `sandbox.destroy()`). A stopped container is never woken by a snapshot. Waking a sleeping container is the `ActivityScheduler` DO alarm's job. Lerma provisions this binding per instance from `instances.sandbox_runtime` (`legacy` by default; `v1` only for a `dev` instance on the staging platform) and writes the marker as a hard provisioning step **before** the script deploy — see [`apps/lerma/README.md`](/llms/platform/apps/lerma/index.md#choosing-the-sandbox-runtime-when-deploying-an-instance). |
| `WORKER_PLATFORM`                 | No        | Yes      | Trusted, non-secret deployment platform (`local`, `staging`, `production`), provisioned by lerma from its own `cf_environment`. **Sandbox v1 migration is allowed only when this is exactly `staging`** (contract C1). Absent/empty/whitespace/`production`/`dev`/`local`/`STAGING` all refuse before any R2 write or DO/container addressing. It is never derived from the request `Host`, `WORKER_URL`, `X-FourIA-*` headers, `DEV_MODE`, or a slug prefix/suffix.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `SANDBOX_V1_CUTOVER`              | No        | No       | Per-tenant Sandbox v1 cutover intent (contract S1), default `false`. Off keeps the tenant on the legacy `Sandbox` class byte-identically; a staging-only migration flips it through the audited `/control` action. No production cutover is in scope.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `SANDBOX_V1_SNAPSHOT_INTERVAL_MS` | No        | No       | Idle-snapshot cadence for a converted (`SandboxV1`) tenant, in milliseconds (`0` disables it). The DO alarm commits a v1 record this often while the container is running, so the data-loss window is bounded even with no traffic/automations (SDK 1.0 has no eviction hook). Defaults to the configured `BACKUP_INTERVAL_MINUTES`, else 1h, and never exceeds **half** the instance's inactivity timeout (staging 7.5m, dev 1h; production 3h). It never wakes a stopped container and skips when the recovery point is already newer. Worker-only.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `CONTAINER_INSTANCE_TYPE`         | Yes       | Yes      | Provisioned container size (`lite` \| `basic` \| `standard-1`..`standard-4`), deployed by lerma from the instance type chosen at provisioning. A `durable_object` Sandbox v1 container selects its size in `ctx.container.start()` **on every boot and wake**, so this must carry the provisioned value; absent/invalid falls back to `standard-1` and logs `sandbox.v1.instance_type.invalid`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `SANDBOX_INSTANCE_TIMEOUT_MS`     | No        | No       | Bound for container VM provisioning (default: `20000`). Fail-fast detection of a degraded/off container.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `SANDBOX_PORT_TIMEOUT_MS`         | No        | No       | Bound for container port readiness (default: `90000`). Keep generous — gateway cold start is slow.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `SANDBOX_POLL_INTERVAL_MS`        | No        | No       | Poll interval for container readiness (default: `300`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `SANDBOX_LOG_LEVEL`               | No        | No       | Container log level: `debug`, `info`, `warn`, `error`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `SANDBOX_LOG_FORMAT`              | No        | No       | Container log format: `pretty` or `json`                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `WORKER_URL`                      | Yes       | No       | Public Worker URL (used for CDP and OTel endpoint)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `WORKER_NAME`                     | Yes       | No       | Worker name for config bucket key resolution                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `CF_AI_GATEWAY_ACCOUNT_ID`        | Yes       | Yes\*    | Cloudflare account ID for AI Gateway URL construction                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `CF_AI_GATEWAY_GATEWAY_ID`        | Yes       | Yes\*    | AI Gateway ID for URL construction                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `AI_GATEWAY_PROXY`                | No        | No       | Opt-in (`'true'`). Routes tenant inference through the sandbox → Worker proxy (`ai.internal`): the container holds a per-instance derived token instead of the account credential, and every request carries trusted `tenant_id`/`environment`/`platform` metadata for per-client spend attribution on a shared gateway pool. Enable only when the tenant's primary model resolves under `cloudflare-unified-billing/*`; while the official `cloudflare-ai-gateway` provider handles the turn it ignores the custom domain and the derived token is rejected. Lerma binds it per instance from `instances.ai_gateway_proxy` (instance page → **AI proxy**), including a binding-only re-deploy of already-running instances.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `CF_AI_GATEWAY_CUSTOM_DOMAIN`     | No        | No       | Optional operator opt-out. When set, the container talks to that custom domain directly with the real credential (no proxy, no tenant metadata). It must **not** be Access-protected: tenant containers carry no Cloudflare Access credentials.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `CF_ACCESS_TEAM_DOMAIN`           | No        | Yes\*    | Cloudflare Access team domain (e.g., `myteam.cloudflareaccess.com`)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `CLOUDFLARE_ACCOUNT_ID`           | Yes       | No       | Cloudflare account ID. Read by the Sandbox SDK for R2 presigned URL signing (backup/restore). Legacy `CF_ACCOUNT_ID` is deprecated (still mapped into the container by `buildEnvVars`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `BACKUP_BUCKET_NAME`              | Yes§      | No       | R2 backup bucket name (default: `fouria-data`). Required by the Sandbox SDK presigned-URL backup path; mapped to `R2_BUCKET_NAME` in container.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `BACKUP_MAX_VERSIONS`             | No        | No       | Max squashfs snapshot versions to retain (default: 3)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `BACKUP_INTERVAL_MINUTES`         | No        | No       | **Minimum** scheduled-snapshot cadence (default: 5). The effective interval is `max(this, 2 × the last snapshot's end-to-end duration)`, so a tenant whose snapshot outlasts the cadence is not snapshotted back-to-back (which starved the automation wake and held the R2 lease continuously). See `resolveEffectiveIntervalMs` in `src/persistence.ts`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `BACKUP_ARCHIVE_EXCLUDES`         | No        | No       | Feature flag (default: off). `'true'` passes `SNAPSHOT_EXCLUDES` (`.npm`, `*.log`, `*.tmp`, `.cache`, `workspace/*node_modules*`) to `mksquashfs`, shrinking the archive. **Off by default** because it is a fidelity trade: an archive restored without those paths no longer matches the filesystem a Tier 1 workspace snapshot restores, so the recovery tier stays a complete image until an operator opts in per instance. Worker-only.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `BACKUP_CHANGE_DETECTION`         | No        | No       | Change-aware gate (**default: on**). Skips a scheduled snapshot when the tenant's state has not changed since the last committed one — a snapshot re-uploads the whole archive, so an unnecessary one dominates the whole backup cost. Only an explicit `'false'`/`'0'`/`'off'` disables it. The scan fails closed to `changed`, so a broken scan costs an extra snapshot, never a missed one. Set on lerma; deployed to tenant Workers.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
| `BACKUP_CHANGE_WATCH`             | No        | No       | Feature flag (default: off). `'true'` runs the per-tenant BackupWatch loop. Set on lerma, which also binds `BACKUP_WATCH`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
| `BACKUP_CHANGE_CHECK_MINUTES`     | No        | No       | BackupWatch check cadence while armed (default: 2). Worker-only.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |     |     |
| `BACKUP_CHANGE_DEBOUNCE_MS`       | No        | No       | Quiet window before a still-changing tenant is snapshotted, in ms (default: 60000). Worker-only.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |     |
| `WORKSPACE_SNAPSHOT`              | No        | No       | Feature flag (default: off). `'true'` captures a Tier 1 container snapshot immediately before idle eviction and commits its handle to `workspace/snapshot.json`. Requires a container application with `scheduling_policy: "durable_object"`; on the default policy the API is unsupported and the attempt is recorded as such. Worker-only.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |
| `BACKUP_LOCAL_BUCKET`             | No        | No       | Set to `true` to store R2 backups in the Sandbox **local** binding bucket instead of remote R2 (needed for local dev without presigned-URL support). Explicit; falls back to `DEV_MODE` only when unset, so a real deployment — including the cloud E2E, which sets `DEV_MODE=true` merely to bypass Access — must set it to `false` or backups land somewhere the Worker cannot restore from.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `LEGACY_STAGE_TIMEOUT_MS`         | No        | No       | Bound (ms) for the legacy-state staging shell that runs before `sandbox.createBackup`; the stage is retried once on a transient timeout. Default: 600000 (10 min)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `ANALYTICS_DATASET`               | No        | No       | Analytics Engine dataset holding emitted `billable-action` rows, read back by `GET /api/admin/costs/v2/summary` for the dashboard "Usage & Billing" widget. Default: `fourai_cost_metrics`; the staging provisioner sets `fourai_cost_metrics_staging`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `ANALYTICS_ENGINE_BASE_URL`       | No        | No       | Overrides the Analytics Engine SQL API base URL for cost-summary queries (default `https://api.cloudflare.com/client/v4/accounts`). Local billing-workflow E2E only — points the query at a fake AE.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `ANALYTICS_ENGINE_API_TOKEN`      | No        | No       | **Optional** Analytics Engine SQL token (Account **Analytics:Read**) for the dashboard read-back only. Emission needs no credential (the Worker writes with its `ANALYTICS` binding) and Lerma no longer provisions this token; set it as an instance secret if a tenant should read its own rows back. No `CLOUDFLARE_API_TOKEN` fallback (that alias lacks AE read).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |

† `DEV_MODE` is a local-development switch (auth bypass). The dev-only agents surface is derived from a `dev` instance's `-dev` hostname (or an explicit `OPENCLAW_DEV_MODE` override) without bypassing auth.
‡ Mapped to `OPENCLAW_E2E_TEST_MODE` in container.
§ Mapped to `R2_BUCKET_NAME` in container.

**Mode behavior summary:**

| Mode            | CF Access Auth | Device Pairing | Debug Routes | Write API |
| --------------- | -------------- | -------------- | ------------ | --------- |
| Production      | Required       | Required       | Disabled     | RBAC      |
| `DEV_MODE`      | Bypassed       | Bypassed       | Configurable | ROOT      |
| `E2E_TEST_MODE` | Bypassed       | Required       | Configurable | ROOT      |
| `DEMO_MODE`     | Required       | Required       | Disabled     | Read-only |

---

## Dynamic Secrets (F4E1-Encrypted R2 Storage)

Channel tokens, tool OAuth credentials, and custom integration secrets are managed by end-users through the **integrations dashboard**. They are stored as F4E1-encrypted JSON objects in the `BACKUP_BUCKET` R2 bucket under the `secrets/<customerSlug>/` prefix.

**Secret categories:** `ai_provider`, `channel`, `integration`

The encryption uses the shared `BACKUP_ENCRYPTION_KEY` (same master key used for backup snapshots) with per-object HKDF-derived AES-256-GCM keys (domain-separated via `fouria-r2-secrets/v1`). The master key never leaves the Worker and is never passed to the container.

Unlike the old vault, there is no unlock/lock mechanism: secrets are always accessible when the key is configured. The `/lock` endpoint is a no-op kept for client compatibility. The WebSocket API (`/ws`) has been removed (returns 410 Gone).

> See `apps/fouria/src/services/secrets/R2SecretsStore.ts` for the full implementation.

---

## AI Provider Configuration

At least one provider must be configured. Priority: CF AI Gateway > Anthropic > OpenAI.

### CI-Only Secret Mapping

The CI workflow maps GitHub secrets (prefixed with `E2E_`) to their corresponding env variables. See `.github/workflows/test.yml` for the authoritative mapping.

| GitHub Secret                       | Maps To                         |
| ----------------------------------- | ------------------------------- |
| `E2E_CLOUDFLARE_AI_GATEWAY_API_KEY` | `CLOUDFLARE_AI_GATEWAY_API_KEY` |
| `E2E_CF_AI_GATEWAY_ACCOUNT_ID`      | `CF_AI_GATEWAY_ACCOUNT_ID`      |
| `E2E_CF_AI_GATEWAY_GATEWAY_ID`      | `CF_AI_GATEWAY_GATEWAY_ID`      |
| `E2E_TEARDOWN_API_TOKEN`            | `CLOUDFLARE_TEARDOWN_API_TOKEN` |
| `E2E_TELEGRAM_BOT_TOKEN`            | `E2E_TELEGRAM_BOT_TOKEN`        |

> There is no GitHub secret named `CLOUDFLARE_AI_GATEWAY_API_KEY`. Staging and
> production Lerma deploys (`.github/workflows/deploy-dashboard.yml`) both read
> `E2E_CLOUDFLARE_AI_GATEWAY_API_KEY` and deploy it as the
> `CLOUDFLARE_AI_GATEWAY_API_KEY` runtime env. Referencing a non-existent
> `secrets.CLOUDFLARE_AI_GATEWAY_API_KEY` expands to empty and ships a dashboard
> whose tenant provisioning fails with `:missing_ai_gateway_credential` — the
> deploy workflow fails fast when the secret is empty.

> `E2E_TEARDOWN_API_TOKEN` is used only by the E2E teardown for destructive cleanup (`wrangler delete` + R2 bucket empty/delete + container/Access/service-token deletes). It must carry Workers Scripts, Workers KV Storage, Workers R2 Storage, Cloudflare Containers, and Access (Apps + Service Tokens) **Edit** permissions. When unset, teardown falls back to `CLOUDFLARE_API_TOKEN` (`DEPLOY_WORKER_API_TOKEN` in CI).

> `E2E_TELEGRAM_BOT_TOKEN` is an optional real Telegram bot token (from
> `@BotFather`) for the Telegram webhook-ingress E2E (`q_telegram_webhook.txt`).
> The suite self-skips when it is unset. It is never passed to the container or
> used in production.
>
> **Local cloud E2E file.** `bun run secrets:generate` produces the gitignored
> `apps/fouria/.dev.vars.e2e` with the mandatory cloud E2E vars —
> `CLOUDFLARE_API_TOKEN` (from `LERMA_API_TOKEN`, since E2E provisions real
> Workers/Containers/R2/Access/KV resources), `CLOUDFLARE_ACCOUNT_ID`,
> `WORKERS_SUBDOMAIN`, `CF_ACCESS_TEAM_DOMAIN`, `R2_ACCESS_KEY_ID`,
> `R2_SECRET_ACCESS_KEY` — plus the AI Gateway trio and Tunnel token when set.
> Vars the registry does not track (`CLOUDFLARE_TEARDOWN_API_TOKEN`,
> `SUPABASE_*`, channel tokens, …) live in the file's preserved manual section.
>
> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.

### Runtime AI Credentials

| Variable                        | Class                | Container | Description                                                                                                                                                                                                                        |
| ------------------------------- | -------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CLOUDFLARE_AI_GATEWAY_API_KEY` | Platform Secret      | Yes       | Real AI-scoped Cloudflare credential supplied by Lerma. In proxy mode it stays in the **Worker**; the container receives a per-instance derived token (`HMAC(gateway token, "fouria-ai-proxy/v1")`) instead.                       |
| `CLOUDFLARE_API_TOKEN`          | Tenant Runtime Alias | Yes       | Same derived token in proxy mode (the unified-billing plugin sends it as the bearer). It only authenticates against the Worker's `ai.internal` handler, never against Cloudflare. With a custom-domain opt-out it is the real key. |

The unified-billing plugin always sends `CLOUDFLARE_API_TOKEN` as the bearer. With `AI_GATEWAY_PROXY=true` the Worker sets `CF_AI_GATEWAY_CUSTOM_DOMAIN=ai.internal`, so the plugin calls `https://ai.internal/compat/...`, which the sandbox intercepts and routes to `src/sandbox-ai-bridge.ts`. That handler authenticates the sandbox with the derived token, derives the tenant from the Worker env (never the request), adds `cf-aig-metadata` (`tenant_id`, `environment`, `platform`, plus the plugin's `openclaw_run_id`), forces `cf-aig-gateway-id` from the trusted binding, and forwards to `https://api.cloudflare.com/client/v4/accounts/{account}/ai/v1/...` with the real platform credential.

Without the proxy the container keeps the real credential and calls Cloudflare directly; AI Gateway log rows then carry no tenant metadata, so Lerma reports them as unattributed and falls back to the tenant-scoped OTel `ai-query` estimate instead of assigning a pool total to a client.

The proxy is what makes per-client spend attributable on a shared gateway pool (the account allows 20 gateways) and keeps the account-scoped credential out of the container. Attribution metadata is a reporting and budget dimension — Cloudflare exposes it to Analytics `metadataValue(key: ...)` grouping and spend-limit rules — not an authorization boundary; the proxy is the boundary. Tenant containers carry **no** Cloudflare Access credentials: `CF_ACCESS_TOKEN`/`CF_ACCESS_CLIENT_ID`/`CF_ACCESS_CLIENT_SECRET` were removed, because an Access-protected custom domain requires a per-tenant service token that still cannot produce `cf.user_id`.

`bun run secrets:validate` proves this credential by running a 2-token inference
through the account's gateway (`gateway.ai.cloudflare.com/{account}/{gateway}/workers-ai`),
so a wrong `CF_AI_GATEWAY_GATEWAY_ID` or `CF_AI_GATEWAY_ACCOUNT_ID` fails the
check too. Workers AI retires model ids on a schedule, and a retired id answers a
non-auth `4xx` (e.g. `required properties at '/' are 'model,fields'`, `Model has
been deprecated`) — so that class of rejection is re-checked with a model-free
gateway read: the credential is only reported broken when the read fails as well,
and a stale pinned model (`AI_GATEWAY_TEST_MODEL` in `scripts/validate-secrets.ts`)
is surfaced as a warning to refresh, instead of red-lighting the secrets run.

\* At least one provider must be configured.

---

## Authentication & Access Control

| Variable                 | Class           | Container | Required | Description                                                                 |
| ------------------------ | --------------- | --------- | -------- | --------------------------------------------------------------------------- |
| `MOLTBOT_GATEWAY_TOKEN`  | Instance Secret | Yes†      | Yes      | Gateway token for CLI auth. Auto-generated per tenant by lerma provisioner. |
| `CF_ACCESS_PLATFORM_AUD` | Instance Secret | No        | Yes*     | Platform-level Access AUD (always present, from dispatch worker).           |
| `CF_ACCESS_CLIENT_AUD`   | Instance Secret | No        | No       | Per-tenant Access AUD (optional, provisioned by lerma).                     |
| `CF_ACCESS_TEAM_DOMAIN`  | Variable        | No        | Yes*     | Cloudflare Access team domain (e.g., `myteam.cloudflareaccess.com`).        |

> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.

\* Required for production. Not required when `DEV_MODE=true` or `E2E_TEST_MODE=true`.
† Mapped to `OPENCLAW_GATEWAY_TOKEN` by `buildEnvVars()`.

---

## Persistence & Storage

| Variable                          | Class    | Container | Required    | Description                                                                                                                                                                                                                                                                                                                                                                                                          |
| --------------------------------- | -------- | --------- | ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CLOUDFLARE_ACCOUNT_ID`           | Variable | Yes       | No          | Cloudflare account ID. Read by the Sandbox SDK for R2 presigned URL signing (backup/restore). Legacy `CF_ACCOUNT_ID` is deprecated (still mapped into the container by `buildEnvVars`).                                                                                                                                                                                                                              |
| `BACKUP_BUCKET_NAME`              | Variable | Yes†      | No          | R2 backup bucket name (default: `fouria-data`). Required by the Sandbox SDK presigned-URL backup path alongside the `BACKUP_BUCKET` binding; mapped to `R2_BUCKET_NAME` in container.                                                                                                                                                                                                                                |
| `R2_ACCESS_KEY_ID`                | Secret   | No‡       | Yes         | **Deprecated for the container.** Worker binding only: required by the Sandbox SDK presigned-URL backup path. Resolved per-instance by the lerma provisioner (shared lerma credential or a generated per-instance token). No longer forwarded into the container — sandbox R2 access uses the `outboundByHost` bridges (`releases.r2.internal` / `docs.r2.internal` / `leads.r2.internal` / `software.r2.internal`). |
| `R2_SECRET_ACCESS_KEY`            | Secret   | No‡       | Yes         | R2 secret key, paired with `R2_ACCESS_KEY_ID`. Worker binding only (see above); not forwarded into the container.                                                                                                                                                                                                                                                                                                    |
| `BACKUP_MAX_VERSIONS`             | Variable | No        | No          | Max squashfs snapshot versions to retain (default: 3)                                                                                                                                                                                                                                                                                                                                                                |
| `BACKUP_INTERVAL_MINUTES`         | Variable | No        | No          | **Minimum** scheduled-snapshot cadence (default: 5). The effective interval stretches to at least twice the last observed snapshot duration so a slow tenant cannot snapshot back-to-back. See `resolveEffectiveIntervalMs` in `src/persistence.ts`.                                                                                                                                                                 |
| `SANDBOX_V1_SNAPSHOT_INTERVAL_MS` | Variable | No        | No          | Idle-snapshot cadence for a converted (`SandboxV1`) tenant, in ms (`0` disables). Defaults to the configured `BACKUP_INTERVAL_MINUTES`, else 1h, capped at half the inactivity timeout; never wakes a stopped container. See `resolveSnapshotIntervalMs` in `src/sandbox-v1.ts`.                                                                                                                                     |
| `SANDBOX_V1_ZERO_EXIT_GRACE_MS`   | Variable | No        | No          | Grace window (ms) after `start-openclaw.sh start` exits **0** before the gateway port opens (the "lost the container startup lock" path). After it, a boot with **no** gateway process anywhere fails fast instead of waiting the full cold-start budget; `0` disables the fast-fail. Default: 15000. See `resolveZeroExitGraceMs` in `src/sandbox-v1.ts`.                                                           |
| `BACKUP_LOCAL_BUCKET`             | Variable | No        | No          | Set to `true` to store backups in the Sandbox local binding bucket (local dev only). Explicit; falls back to `DEV_MODE` when unset. Real deployments (including the cloud E2E) set it to `false` so backups land in the remote `BACKUP_BUCKET`.                                                                                                                                                                      |
| `LEGACY_STAGE_TIMEOUT_MS`         | Variable | No        | No          | Bound (ms) for the legacy-state staging shell in `createSnapshot`; retried once on a transient timeout. Default: 600000 (10 min)                                                                                                                                                                                                                                                                                     |
| `BACKUP_ENCRYPTION_KEY`           | Secret   | No        | Recommended | Base64 32-byte AES-256 master key for client-side R2 backup encryption (F4E1). Worker-side only, never passed to the container. Generate: `openssl rand -base64 32`                                                                                                                                                                                                                                                  |
| `BACKUP_ENCRYPTION_REQUIRED`      | Variable | No        | No          | Set to `true` to fail closed (skip backup + error) when `BACKUP_ENCRYPTION_KEY` is missing (default: false, legacy plaintext + warning)                                                                                                                                                                                                                                                                              |

† Mapped to `R2_BUCKET_NAME` in container.
‡ Deprecated for the container: kept as a Worker binding for the Sandbox SDK presigned-URL backup path, never exposed inside the container. Sandbox R2 access goes through the `outboundByHost` bridges (`releases.r2.internal` / `docs.r2.internal` / `leads.r2.internal` / `software.r2.internal`).

> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.

---

## Browser Automation (CDP)

| Variable     | Class    | Container | Required | Description                                        |
| ------------ | -------- | --------- | -------- | -------------------------------------------------- |
| `WORKER_URL` | Variable | Yes       | No       | Public Worker URL (used for CDP and OTel endpoint) |

> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.

---

## Container Rescue (`/debug/rescue-container`)

Recovers a container stuck in Error mode (e.g. the `fouria:unstable-<sha>` image it references was deleted by the stale-image cleanup cron). The tool probes the container via the Sandbox SDK, re-points the application image to the expected tag through the Cloudflare Containers API (rollout), and destroys the stuck container so the platform provisions a fresh one. See `apps/fouria/src/diagnostics.ts`.

| Variable                   | Class          | Container | Required | Description                                                                                                                                                                                                                               |
| -------------------------- | -------------- | --------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CLOUDFLARE_API_TOKEN`     | Runtime Secret | Yes       | No       | AI Gateway REST Bearer token on tenant workers. On ROOT/ops deployments the same binding may also be used by `/debug/rescue-container` when it has **Containers:Edit**.                                                                   |
| `CONTAINER_APPLICATION_ID` | Variable       | No        | No       | Cloudflare Containers application id for this worker's Sandbox container. **Not provisioned by Lerma**: set it manually on a ROOT/ops deployment that needs `/debug/rescue-container` (a tenant's AI-scoped token lacks Containers:Edit). |
| `CONTAINER_IMAGE_TAG`      | Variable       | No        | No       | Expected container image tag (e.g. `unstable-<sha>`); used to derive the repair target `registry.cloudflare.com/<account>/fouria:<tag>`. Set manually with `CONTAINER_APPLICATION_ID`.                                                    |
| `CONTAINER_IMAGE_NAME`     | Variable       | No        | No       | Container image repo name for the tenant sandbox image (default: `fouria`).                                                                                                                                                               |

> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.

---

| Variable                              | Class           | Container | Required | Description                                                                                                                                                                                                                                                                                          |
| ------------------------------------- | --------------- | --------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Variable                              | Class           | Container | Required | Description                                                                                                                                                                                                                                                                                          |
| ------------------------------------- | --------------- | --------- | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------                                                                                                    |
| `OTEL_SHARED_SECRET`                  | Instance Secret | Yes       | Yes†     | Per-instance bearer for OTLP ingest authentication on the sandbox bridge, generated by Lerma at provisioning and stored on the instance row. Mandatory for the billing path.                                                                                                                         |
| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | Auto            | Yes       | Auto     | Auto-set to `http://otel.internal/v1/metrics` (virtual host resolved by the sandbox bridge)                                                                                                                                                                                                          |
| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`  | Auto            | Yes       | Auto     | Auto-set to `http://otel.internal/v1/traces` (virtual host resolved by the sandbox bridge)                                                                                                                                                                                                           |
| `OTEL_EXPORTER_OTLP_PROTOCOL`         | Auto            | Yes       | Auto     | Auto-set to `http/json` (never grpc/protobuf)                                                                                                                                                                                                                                                        |
| `OTEL_TRACES_ENABLED`                 | Variable        | Yes       | Yes†     | `'true'` enables `diagnostics.otel.traces` (default: disabled). Lerma provisions it: `true` for a billable production/staging instance, `false` for a client's non-billable dev instance. Without it the gateway emits no spans, so no billable action exists.                                       |
| `OTEL_SERVICE_NAME`                   | Auto            | Yes       | Auto     | Auto-set to customer slug or worker name                                                                                                                                                                                                                                                             |
| `ANALYTICS_ENGINE_API_TOKEN`          | Instance Secret | **No**    | No       | **Optional** Account Analytics:Read token for the dashboard read-back only. Not required for emission (the Worker writes with its `ANALYTICS` binding) and no longer provisioned by Lerma; set it as an instance secret if a tenant should read its own rows back. **Never** given to the container. |
| `PROMOTION_SOURCE_BUCKET`             | Binding         | **No**    | Auto‡    | Read-only R2 binding on a **production** instance to its sibling dev bucket. The private promotion transport (stage on dev, pull on boot). **Never** forwarded to the container.                                                                                                                     |

† Mandatory for the billing proof only: the bridge rejects an unauthenticated
export (`401`) and no billable traces are produced when either the secret is
unbound or traces are not enabled.

‡ Provisioned automatically for `environment: "production"`; dev instances do not
get it. Both buckets stay separate — production only reads the sibling's prefix.

The `OTEL_SHARED_SECRET` above authenticates the tenant Worker's OTLP ingest and
is forwarded to the container as `OTEL_EXPORTER_OTLP_HEADERS`. When it is unset
the bridge rejects every export and **no billable traces are produced**. It is
**generated per instance** by Lerma at provisioning, stored on the instance row
(`instances.otel_shared_secret`), and deployed as that tenant's `secret_text`
binding. It is never read from Lerma's environment: a leaked value is scoped to
one tenant, and rotating it is a redeploy of that tenant (the upgrade path
preserves the stored value; a legacy instance with no stored value gets a fresh
one on its next upgrade).

`ANALYTICS_ENGINE_API_TOKEN` is **not part of the emission path and is not
provisioned by Lerma**. The Worker writes every billable row with its `ANALYTICS`
binding (no credential), and the platform reconciles credits from its own
Analytics Engine credentials. The token only enables the optional per-tenant
dashboard read-back; when it is absent the summary reports `available: false` and
the SQL API is never called (there is no `CLOUDFLARE_API_TOKEN` fallback — that
alias lacks Analytics read and only produced `403`s).

> **The container never touches Analytics Engine.** Emission is the Worker's job:
> the container exports OTLP to the sandbox bridge (`http://otel.internal`) and
> the Worker writes the `billable-action` row with its `ANALYTICS` binding — no
> credential involved — and the platform reconciles credits from its own
> Analytics Engine credentials. The optional dashboard read-back (a per-instance
> `ANALYTICS_ENGINE_API_TOKEN`) never leaves the Worker. No Analytics Engine
> credential is ever forwarded into the sandbox, and no agent, skill, or manifest
> entry should try to reach the AE SQL API.

### Private transports (no public ingest or import route)

| Feature   | Transport                                                                                         | Authority                                          | Public route |
| --------- | ------------------------------------------------------------------------------------------------- | -------------------------------------------------- | ------------ |
| OTLP      | container → `http://otel.internal/v1/{traces,metrics}` → `TenantSandbox.outboundByHost`           | per-tenant `OTEL_SHARED_SECRET` (defense in depth) | none         |
| Promotion | dev writes `promotion/staged/*` to its own bucket; production reads via `PROMOTION_SOURCE_BUCKET` | R2 binding (tenant-scoped)                         | none         |

`/api/otel/*` and `/api/admin/releases/import` no longer exist. The OTLP route was
Worker-public but Access still redirected it, so the container now exports to a
virtual host that the SDK resolves inside the Worker: the traffic never leaves
the sandbox ↔ Worker control plane and Access never sees it. Promotion no longer
crosses the internet either: the dev instance stages the bundle in its own
bucket, and the production instance pulls it on boot (or on an explicit
`POST /api/admin/releases/apply`) under the shared persistence lease, skipping
while parked or while a maintenance window/restore is in flight.

---

## Billing API (Lerma, server-side only)

These are Lerma-dashboard environment variables — never deployed to tenant workers.

| Variable                    | Class          | Required | Description                                                                                                                                                                                                                                       |
| --------------------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `BILLING_INGEST_TOKEN`      | Runtime Secret | No       | Bearer token guarding the internal billing API: L2 ingest (`POST /api/credits/v1/ingest`) and L5 reconcile/usage (`POST /api/credits/v1/reconcile`, `GET /api/credits/v1/usage/:id`). Unset → every request is rejected (`401`).                  |
| `ANALYTICS_DATASET`         | Variable       | No       | Analytics Engine dataset holding billable rows (`fourai_cost_metrics` default; `fourai_cost_metrics_staging` when `CF_ENVIRONMENT=staging`).                                                                                                      |
| `ANALYTICS_ENGINE_BASE_URL` | Variable       | No       | Overrides the Analytics Engine SQL API base URL for `Lerma.BillingReconcile`/`Lerma.Analytics.CFAnalytics`. **Local billing-workflow E2E only** — points them at the local fake AE (`apps/lerma/test/e2e_billing_local/`). Unset everywhere else. |

**`BILLING_INGEST_TOKEN` permissions:** none. It is **not** a Cloudflare API token and needs **no Cloudflare scopes** — it is an opaque shared bearer secret. Generate it with `openssl rand -hex 32` and give the same value to every caller of the internal billing API (the billing queue consumer, operator tooling, the staging billing-trace e2e). The endpoints compare it in constant time and fail closed (`401`) when it is unset. Rotate by updating the value in the callers and in Lerma's environment; no scope changes are involved.

---

## Lerma Provisioning API Keys

Lerma supports both a platform-wide compatibility key and per-application keys:

| Name                           | Storage                    | Scope                      | Description                                                                                                                                                                                                                   |
| ------------------------------ | -------------------------- | -------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PROVISIONING_API_KEY`         | Lerma runtime secret only  | Platform-wide              | Optional global bearer for `/api/provisioning/*` and `/api/ai-gateways/*`. When unset, global-only API operations fail closed (`401`). **Not** a GitHub repository secret: it is never synced by `secrets:generate --github`. |
| `STAGING_PROVISIONING_API_KEY` | GitHub Actions secret only | Staging Fouria application | Used by `.github/workflows/promote-staging.yml` to upgrade the staging `client` tenant. It is not injected into Lerma's runtime.                                                                                              |

Per-application keys are generated, rotated, and revoked by ROOT users on
`/dashboard/applications/:id`. Lerma stores only the SHA-256 digest in the
application row and reveals a newly generated key once. Copy the staging
Fouria application's value to the GitHub secret `STAGING_PROVISIONING_API_KEY`;
never reuse it in production. Calls using an application key must send
`X-FourIA-Application: <application-slug>`, and Lerma filters the operation to
that application's instances. The platform-wide key, when configured, remains a
full-scope compatibility credential.

---

## Feature Flags

| Variable                  | Class    | Container | Required | Description                                                                                                                                                                                                                                       |
| ------------------------- | -------- | --------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `MOLTLAZY_FEATURE_FLAGS`  | Auto     | Yes       | Auto     | JSON feature flags for moltlazy config generation (auto-generated). Includes the `softwareProvisioning` key (Worker-side: reconcile the declared software manifest at boot).                                                                      |
| `WORKER_NAME`             | Variable | Yes       | No       | Worker name for config bucket key resolution.                                                                                                                                                                                                     |
| `STARTUP_LOG_VERBOSITY`   | Variable | Yes       | No       | Clean boot-log verbosity for the loading page: `steps` \| `detailed` (default) \| `verbose`.                                                                                                                                                      |
| `OPENCODE_API_KEY`        | Secret   | Yes       | Dev      | Platform OpenCode management credential. Lerma mints a restricted (`Models: Request`) key per client and binds **that** key to the client's dev instances; the management key needs `This account's API keys: Write`. Never used for API billing. |
| `MOLTLAZY_AGENTS`         | Variable | Yes       | Paso4    | Comma-separated premade agent roster to inject (moltlazy `MOLTLAZY_AGENTS`). Empty/unset means the platform default (all non-opt-in agents).                                                                                                      |
| `MOLTLAZY_PASO4_AGENTS`   | Variable | Yes       | Paso4    | `true` makes the Paso4-exclusive agent registry eligible (the opt-in `paso4-dev` agent). Set by lerma only on `scope: "paso4"` instances.                                                                                                         |
| `GITHUB_TOKEN`            | Secret   | Yes       | Paso4    | GitHub token for the `paso4-dev` agent's `gh` CLI. Deployed by the provisioner to Paso4-exclusive instances only; never present on client tenants.                                                                                                |
| `PASO4_TELEGRAM_GROUP_ID` | Variable | Yes       | Dev      | Paso4 general Telegram group id for platform alerts (global lerma env; default `-5442477312`).                                                                                                                                                    |
| `TELEGRAM_INGRESS_HOST`   | Variable | No        | No       | Public Telegram webhook ingress host (`hooks.fouria.io` / `hooks.dev.fouria.io`). The Dispatch Worker matches it and routes to the tenant Worker without an Access JWT.                                                                           |
| `TELEGRAM_INGRESS_QUEUE`  | Queue    | No        | Yes      | Cloudflare Queue (`fouria-telegram-ingress`) buffering Telegram updates that arrive while the container is cold; the consumer wakes/heals and forwards with at-least-once delivery.                                                               |
| `TELEGRAM_INGRESS`        | DO       | No        | Yes      | Per-tenant `TelegramIngress` Durable Object owning the fixed-window rate limiter for inbound webhook updates.                                                                                                                                     |

> **Telegram webhook ingress (#364).** Managed tenants switch Telegram to webhook mode. The webhook secret + bot token live in a Worker-readable F4E1 R2 registry (`telegram-ingress/<slug>/<account>`); moltlazy writes `channels.telegram.accounts.<account>.webhook*` from the `TELEGRAM_WEBHOOK_MANIFEST` env var the Worker injects. The Dispatch Worker adds `TELEGRAM_INGRESS_HOST` (its own ingress routing) and the tenant Worker adds the `TELEGRAM_INGRESS` DO + `TELEGRAM_INGRESS_QUEUE` queue bindings.

---

## Stripe Billing Integration

The tenant User Worker has no Stripe credential binding and no Stripe
checkout/portal route. Stripe billing runs in Lerma; the admin UI shows the
plan, limits, and usage it receives from the Dispatch Worker headers, but plan
changes and payment methods are managed in the customer dashboard.

| Variable        | Class         | Container | Required | Description                                                                                                                                                       |
| --------------- | ------------- | --------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `BILLING_QUEUE` | Queue binding | No        | Yes      | Shared Cloudflare Queue producer for canonical billable actions to Lerma (`fouria-billing` / `fouria-billing-staging`). Created/adopted by Lerma at provisioning. |
| `TASKS`         | Queue binding | No        | No       | Per-tenant task buffer (`fouria-tasks-<slug>[-dev]`). The tenant Worker is its own consumer; Lerma creates the queue and registers the script as consumer.        |

Billing data (plan tier, status, seats, credits, Stripe customer ID) is sourced from the Supabase database via the Dispatch Worker. The Worker does not persist billing state in Durable Objects.

## Communication Schema (Dispatch Worker → User Worker)

Billing metadata is passed from the Dispatch Worker to the User Worker via one of three resolution methods (in priority order):

| Priority      | Method                                       | Header / Source                             | Format                                                                               |
| ------------- | -------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------ |
| 1 (preferred) | Cloudflare for SaaS custom hostname metadata | `request.cf.hostMetadata`                   | JSON object — set via `PATCH /zones/:id/custom_hostnames/:id` with `custom_metadata` |
| 2             | Single metadata header                       | `X-FourIA-Metadata`                         | Base64url-encoded JSON of [`BillingMetadata`](#billingmetadata-schema)               |
| 3 (legacy)    | Individual headers                           | `X-FourIA-Plan`, `X-FourIA-SeatsUsed`, etc. | String values (deprecated)                                                           |

**Preferred approach (1):** Set `custom_metadata` on each customer's custom hostname via the Cloudflare API. The Dispatch Worker reads `request.cf.hostMetadata` and the User Worker reads it when `trusted_workers: true` is set on the dispatch namespace.

**Fallback approach (2):** The Dispatch Worker encodes the [`BillingMetadata`](#billingmetadata-schema) object as base64url JSON and sets it as the `X-FourIA-Metadata` header when forwarding the request.

### BillingMetadata Schema

```typescript
interface BillingMetadata {
  plan_tier: string; // "free" | "basic" | "pro" | "enterprise"
  plan_name: string; // Human-readable name
  billing_status: string; // "active" | "past_due" | "canceled" | "trialing" | "incomplete"
  seats_used: number; // Current seat count
  seats_max: number; // Plan seat limit
  credits_used: number; // Credits consumed this period
  credits_max: number; // Credit limit (0 = unlimited/pay-as-you-go)
  stripe_customer_id: string | null; // Stripe customer for payments
  period_end: string | null; // ISO 8601 — current billing period end
  trial_end: string | null; // ISO 8601 — trial end (if trialing)
}
```

### RBAC Role Resolution

The user's RBAC role is forwarded by the Dispatch Worker via the `X-FourIA-Role` header (one of `ROOT`, `ADMIN`, `BASE_USER`), when the Dispatch Worker can resolve it:

- **Paso4 ROOT users** are **always** `ROOT`. A user qualifies when their JWT `aud` is the platform AUD (the `*.fouria.io` wildcard Access application) **or** their email is a member of the shared `Paso4 Root Users` Access group. The group check matters because per-instance Access applications include the ROOT group policy (so ROOT operators are not locked out at `{slug}.fouria.io`); Cloudflare matches the most-specific application, so those tokens carry the per-instance AUD rather than the platform AUD. In staging this is the only permitted audience/identity.
- **Client users** (per-instance Access application AUDs) get their role only when `client_users.status = validated` and `client_user_instances` links them to the requested instance. Pending, rejected, production-only, or dev-only assignments do not cross environment boundaries.

The User Worker's `resolveCustomerRole` resolves the role in priority order:

1. JWT `aud` matches `CF_ACCESS_PLATFORM_AUD` → `ROOT`
2. `X-FourIA-Role` header (from the Dispatch Worker)
3. JWT `custom.role` claim (IdP-provided)
4. `LOCAL_ROLE_MAP` (local dev fallback)

---

## Dashboard Tenant Provisioning (`apps/lerma`)

Variables required by the Phoenix dashboard to create and decommission tenant User Workers.

### CF Access Local Development

Variables for local CF Access integration via cloudflared tunnel. See `apps/lerma/bin/setup-cf-access-local.sh` for one-time setup.

| Variable                         | Class           | Required | Description                                                                                                                                                                                   |
| -------------------------------- | --------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CF_ACCESS_TEAM_DOMAIN`          | Variable        | Yes*     | Cloudflare Access team domain (e.g., `paso4`). Used by lerma's dashboard auth and deployed to tenant workers.                                                                                 |
| `CF_ACCESS_AUD`                  | Variable        | Yes*     | Dashboard's own Access AUD (JWT audience for lerma's dashboard). Populated by the deploy flow from `CF_ACCESS_LERMA_AUD`/`STAGING_CF_ACCESS_LERMA_AUD`.                                       |
| `CF_ACCESS_PLATFORM_AUD`         | Variable        | Yes*     | Platform-level Access AUD (Paso4 Root app). Deployed to every tenant worker as the `CF_ACCESS_PLATFORM_AUD` secret so its auth middleware can validate JWTs forwarded by the Dispatch Worker. |
| `CF_ACCESS_SERVICE_TOKEN_ID`     | Instance Secret | No       | CF Access service token Client ID for local CI/CD (non-interactive).                                                                                                                          |
| `CF_ACCESS_SERVICE_TOKEN_SECRET` | Instance Secret | No       | CF Access service token Client Secret for local CI/CD.                                                                                                                                        |
| `CLOUDFLARED_TUNNEL_TOKEN`       | Instance Secret | No       | Tunnel token for `lerma-local` (routes to port 4001).                                                                                                                                         |
| `SKIP_AUTH`                      | Variable        | No       | Set to `"true"` to bypass CF Access JWT verification (escape hatch).                                                                                                                          |

### Lerma staging healthcheck tunnel

The staging Hetzner host runs a remotely managed Cloudflare Tunnel sidecar at
`lerma-health-dev.paso4.io`. Its ingress forwards only the minimal
`GET /health` response (`ok`) to Lerma; all other paths return 404. The existing
dashboard hostname and Access policies are unchanged. The tunnel does not replace
the existing staging A record or close its current origin ports.

| Variable                                 | Class            | Required | Description                                                                                                                                                  |
| ---------------------------------------- | ---------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `TF_VAR_healthcheck_tunnel_api_token`    | Local IaC Secret | Apply    | Dedicated Cloudflare token used only by the staging OpenTofu apply; requires Account Tunnel:Edit and paso4.io Zone DNS:Edit. Do not reuse `LERMA_API_TOKEN`. |
| `LERMA_STAGING_HEALTHCHECK_TUNNEL_TOKEN` | GitHub Secret    | Staging  | Remote-managed Tunnel run token passed through Ansible to the staging-only `cloudflared` sidecar. It is not passed to the Lerma Phoenix app.                 |

### Invitation Email

Lerma delivers invitation confirmation emails through Cloudflare Email Service over authenticated SMTP submission (`smtp.mx.cloudflare.net:465`, implicit TLS). It reuses the existing provisioning credential as the SMTP password, so no additional secret is required.

| Variable          | Class  | Required | Description                                                                                                                                                                                        |
| ----------------- | ------ | -------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `LERMA_API_TOKEN` | Secret | Yes      | Cloudflare API token. Must include the **Email Sending: Edit** permission; used as the SMTP password (username is the literal `api_token`). The `From` domain must be onboarded for Email Sending. |

Local development continues to use Swoosh's local mailbox adapter; tests use the Swoosh test adapter.

### Provisioning Configuration

| Variable                                    | Class    | Required | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| ------------------------------------------- | -------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `CLOUDFLARE_API_TOKEN`                      | ENV only | Yes      | API token with Workers:Edit, R2:Edit, Containers:Edit, and D1 edit permissions. **Containers:Edit is required** — the dashboard generates short-lived registry credentials (`POST /containers/registries/{host}/credentials`) to list versions, and uses R2:Edit to fetch the worker bundle from the `fouria-bundles` R2 bucket at provisioning time. Without it the calls fail with `403 Authentication error`.                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `CLOUDFLARE_ACCOUNT_ID`                     | Variable | Yes      | Paso4 Cloudflare account ID for provisioning calls. Must be set or the registry-health check reports it missing and registry-credentials calls fail with `Could not route to /accounts/containers/registries/... — object identifier is invalid` (the account ID is interpolated into the request path). On the fouria User Worker it is a plain var: the Sandbox SDK reads it for R2 presigned URL signing (backup/restore) — missing it throws `InvalidBackupConfigError` (`Backup requires R2 presigned URL credentials`). The dashboard health check also validates that `CLOUDFLARE_API_TOKEN` grants the required permission groups (`Workers Scripts Write`, `DNS Write`, `Access: Apps and Policies Write`, `Workers Containers Write`), flagging a missing one as "Limited access". Legacy `CF_ACCOUNT_ID` is deprecated. |
| `CONTAINER_REGISTRY_HOST`                   | Variable | No       | Cloudflare managed container registry host (default: `registry.cloudflare.com`). Mirrors `CLOUDFLARE_CONTAINER_REGISTRY` in wrangler.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `CONTAINER_IMAGE_NAME`                      | Variable | No       | Container image repo name for the tenant sandbox image (default: `fouria`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `CONTAINER_BUNDLE_R2_BUCKET`                | Variable | No       | R2 bucket that publishes the worker bundle `fouria.mjs` (default: `fouria-bundles`). CI uploads the bundle to `<version>/fouria.mjs` and the dashboard fetches it from R2 at provisioning time.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `FOURIA_DISPATCH_NAMESPACE`                 | Variable | No       | Workers for Platforms dispatch namespace name (default: `fouria-dispatch`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `FOURIA_R2_PUBLIC_ENDPOINT`                 | Variable | No       | R2 public endpoint template, e.g. `https://pub-{bucket_id}.r2.dev`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `ANALYTICS_DATASET`                         | Variable | No       | Analytics Engine dataset name (default: `fourai_cost_metrics`, staging: `fourai_cost_metrics_staging`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `CONTAINER_IMAGE`                           | Variable | No       | Container image reference for the tenant's container application (linked to the Sandbox DO namespace). Example: `registry.cloudflare.com/<account_id>/fouria:latest`. When unset, lerma derives it from the published bundle version as `registry.cloudflare.com/<account_id>/fouria:<image_tag>`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `CONTAINER_MAX_INSTANCES`                   | Variable | No       | Max container instances for tenant container apps (default: `3`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `DEV_INSTANCE_SSH_PUBLIC_KEY`               | Secret   | No       | Optional single-line `ssh-ed25519` **public** key Lerma applies to Cloudflare Container `authorized_keys` only for `dev` instances. It is never a Worker/container environment binding; keep the private key on the operator workstation. Access uses `wrangler containers ssh` (Cloudflare-authenticated; no public SSH port). Existing dev apps receive key changes on their next Lerma redeploy.                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `WORKER_OBSERVABILITY_ENABLED`              | Variable | No       | Enable Workers Observability (logs + traces) on deployed tenant User Workers (default: `true`). Mirrors `apps/fouria/wrangler.jsonc`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `WORKER_OBSERVABILITY_SAMPLING_RATE`        | Variable | No       | Head sampling rate for traces on tenant User Workers, `0..1` (default: `1`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `WORKER_OBSERVABILITY_LOGS`                 | Variable | No       | Enable logs on tenant User Workers (default: `true`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
| `WORKER_OBSERVABILITY_INVOCATION_LOGS`      | Variable | No       | Enable invocation logs on tenant User Workers (default: `true`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `WORKER_OBSERVABILITY_TRACES`               | Variable | No       | Enable traces on tenant User Workers (default: `true`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
| `WORKER_OBSERVABILITY_LOGS_DESTINATIONS`    | Variable | No       | Comma-separated account-level Cloudflare Observability **Logs** destination names (Observability → Destinations), referenced by name in the worker upload metadata. Empty → no export destination.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 |
| `WORKER_OBSERVABILITY_TRACES_DESTINATIONS`  | Variable | No       | Comma-separated account-level Cloudflare Observability **Traces** destination names. Empty → no export destination.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `WORKER_OBSERVABILITY_LOGS_SAMPLING_RATE`   | Variable | No       | Logs head sampling rate, `0..1`. Falls back to `WORKER_OBSERVABILITY_SAMPLING_RATE`. Use `0.1` for a 10% pilot.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `WORKER_OBSERVABILITY_TRACES_SAMPLING_RATE` | Variable | No       | Traces head sampling rate, `0..1`. Falls back to `WORKER_OBSERVABILITY_SAMPLING_RATE`. The uptime widget/API report this rate and treat "no events" as **unknown**, never downtime.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `WORKER_OBSERVABILITY_LOGS_PERSIST`         | Variable | No       | Keep the Cloudflare-side copy when an export destination is configured (default: `false` — Lerma is the destination of record).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
| `WORKER_OBSERVABILITY_TRACES_PERSIST`       | Variable | No       | Keep the Cloudflare-side trace copy when an export destination is configured (default: `false`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
| `CONTAINER_OBSERVABILITY_LOGS`              | Variable | No       | Enable log capture on the tenant's container application (default: `true`).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |
| `CONTAINER_REGIONS`                         | Variable | No       | Container application placement regions (deployment zone), comma-separated (default: `WEUR`). Valid values: `ENAM`, `WNAM`, `EEUR`, `WEUR`, `APAC`, `SAM`, `ME`, `OC`, `AFR`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
| `R2_TOKEN_STRATEGY`                         | Variable | No       | How lerma resolves the mandatory per-tenant R2 S3 backup credential: `auto` (default), `shared`, or `per_instance`. `auto` uses `R2_TOKEN_MAX_PER_ACCOUNT`: a known limit `<= 250` selects `shared`, `> 250` or unknown selects `per_instance`. `per_instance` mints a scoped R2 token per instance at provisioning. See `Lerma.Provisioning.R2Credentials`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                       |
| `R2_TOKEN_MAX_PER_ACCOUNT`                  | Variable | No       | Account API-token limit used by `:auto` (default: unknown → `:per_instance`). Cloudflare exposes no endpoint for the maximum, so pin a known value here if desired.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
| `R2_ACCESS_KEY_ID`                          | Secret   | Cond.    | Lerma-level shared R2 S3 access key, required when the effective strategy is `shared`. Kept as a Worker binding for the SDK presigned-URL backup path (never forwarded to the container — sandbox R2 access uses the `outboundByHost` bridges). Generate with `mix lerma.r2_token --all-buckets`.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
| `R2_SECRET_ACCESS_KEY`                      | Secret   | Cond.    | Lerma-level shared R2 S3 secret key, paired with `R2_ACCESS_KEY_ID`. Required when the effective strategy is `shared`; provisioning fails with `:missing_shared_r2_credentials` otherwise. Worker binding only (see above).                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |

### Observability destination ingestion (Lerma)

Cloudflare account-level **Observability → Destinations** push Workers Logs and Workers Traces to Lerma over OTLP/HTTP JSON. This is a **push** stream: it adds no per-tenant polling, does not replace `HealthMonitor`/billing, and never wakes a sleeping container. `HealthMonitor` continues to drive `instances.status`/`clients.status`; destination telemetry only supplies diagnostics and uptime statistics.

| Variable                        | Class          | Required | Description                                                                                                                                                                                                       |
| ------------------------------- | -------------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OTEL_INGEST_TOKEN`             | Runtime Secret | No       | Shared bearer secret configured as the destination's `Authorization: Bearer …` header. The ingest endpoints compare it in constant time and fail closed (`401`) when unset. Generate with `openssl rand -hex 32`. |
| `OTEL_INGEST_MAX_EVENTS`        | Variable       | No       | Maximum sanitized rows accepted per ingest request (default `1000`). The remainder is reported as `skipped`.                                                                                                      |
| `OTEL_INGEST_RETENTION_DAYS`    | Variable       | No       | Retention for stored destination telemetry (default `90`, matching PRIVACY.md).                                                                                                                                   |
| `OTEL_INGEST_PRUNE_INTERVAL_MS` | Variable       | No       | Retention sweep interval (default `86400000` / 24 h). A first sweep runs one minute after boot, then every interval.                                                                                              |

**Destination names per environment:**

| Environment | Logs destination         | Traces destination         | Lerma OTLP base                                 |
| ----------- | ------------------------ | -------------------------- | ----------------------------------------------- |
| staging     | `lerma-worker-logs`      | `lerma-worker-traces`      | `https://lerma-dev.paso4.io/api/ingest/otel/v1` |
| production  | `lerma-worker-logs-prod` | `lerma-worker-traces-prod` | `https://lerma.paso4.io/api/ingest/otel/v1`     |

`apps/fouria/wrangler.jsonc` references the staging names in `env.staging` and
the production names in `env.production`; Lerma's CD passes the same names to
each environment's tenant-worker metadata. The account-level destinations must
exist before a Worker that references them is deployed.

1. In **Observability → Destinations**, create the pair for the environment you
   are rolling out (staging first), pointing at that environment's Lerma host.
   Add the custom header `Authorization: Bearer <OTEL_INGEST_TOKEN>` to both.
   The API equivalent is:

   ```bash
   # staging values shown; for production use LERMA_HOST=lerma.paso4.io and
   # the `-prod` destination names.
   LERMA_HOST=lerma-dev.paso4.io
   LOGS_NAME=lerma-worker-logs
   TRACES_NAME=lerma-worker-traces

   for pair in "$LOGS_NAME opentelemetry-logs logs" "$TRACES_NAME opentelemetry-traces traces"; do
     set -- $pair
     curl -sS "https://api.cloudflare.com/client/v4/accounts/$CLOUDFLARE_ACCOUNT_ID/workers/observability/destinations" \
       -H "Authorization: Bearer $CF_OBSERVABILITY_TOKEN" -H 'Content-Type: application/json' \
       -d "{\"name\":\"$1\",\"enabled\":true,\"configuration\":{\"type\":\"logpush\",\"logpushDataset\":\"$2\",\"url\":\"https://$LERMA_HOST/api/ingest/otel/v1/$3\",\"headers\":{\"Authorization\":\"Bearer $OTEL_INGEST_TOKEN\"}}}"
   done
   ```

   Use a credential with **Workers Observability Write** (the standard
   `LERMA_API_TOKEN` may need that permission added). Cloudflare's destination
   preflight needs a live, decompressing receiver, so create the staging pair
   after staging Lerma is on the gzip-aware release, and the production pair
   after production Lerma is.

   The dashboard CD runs `mix lerma.observability.destinations --environment
   <env> --warn-only` before each deploy (staging and production). It lists the
   destinations, creates missing ones, and PATCHes existing ones so a rotated
   `OTEL_INGEST_TOKEN` propagates. It prefers `CF_OBSERVABILITY_TOKEN` and falls
   back to `LERMA_API_TOKEN`; either must carry **Workers Observability Write**.
   With `--warn-only`, a missing permission logs a warning instead of failing
   the deploy, so add that permission to the token (or set a dedicated
   `CF_OBSERVABILITY_TOKEN`) to make the rollout fully self-healing.

2. Set `OTEL_INGEST_TOKEN` on that Lerma deployment and set
   `WORKER_OBSERVABILITY_LOGS_DESTINATIONS` / `WORKER_OBSERVABILITY_TRACES_DESTINATIONS`
   to the environment's two names. The CD does this automatically (staging:
   un-suffixed names; production: `-prod` names).
3. Redeploy the tenant workers so their upload metadata references the
   destinations (new tenants inherit them automatically). Existing tenants pick
   them up on their next Lerma upgrade/redeploy.
4. Confirm delivery on the destination status (**Last: n minutes ago**) and query
   `GET /api/analytics/v1/uptime/tenant-<slug>?window=24h`.

The ingest endpoints accept **OTLP/HTTP JSON only** (Cloudflare gzips the body while keeping `Content-Type: application/json`, so `LermaWeb.Plugs.OtlpBodyReader` decompresses it before parsing) and answer `202` with `{accepted, skipped}` so Cloudflare does not retry-loop; binary protobuf is rejected with `415` and a hint. Ingestion is idempotent — the destinations deliver at least once, and a replayed batch adds no rows. Only allowlisted metadata is stored: log message text, stack traces, URLs/query strings, user agents, request/response bodies and DB query text are dropped before persistence.

### Health Monitoring

Lerma's `HealthMonitor` periodically probes each tenant's public `GET /api/health` endpoint and keeps `clients.status` and `instances.status` accurate. `/api/health` is the **unified** health endpoint (returns startup/readiness/liveness probes plus the canonical `status` and `processId`); `GET /api/status` is a legacy alias over the same snapshot. Both are exposed by every User Worker without auth and are made publicly reachable via path-scoped Cloudflare Access applications (`*.fouria.io/api/health`, `*.fouria.io/api/status`, see `apps/fouria/iac/access.tf`). Probing also cold-starts asleep containers (`ensureGateway`), so a stopped VM recovers on the next check — pass `?passive=1` (used by ROOT diagnostics) to probe without warming.

Each instance is probed at its **own environment hostname** (`acme.fouria.io` for production, `acme-dev.fouria.io` for dev).

**`up` means the gateway is running.** A probe is `up` only when the snapshot reports `status: "running"`. A `200` `status: "sleeping"` (an idle container whose gateway is not running) is **not** up: it records a down transition for the uptime widget and marks the instance `sleeping` (alive, but not running). A `503` (`stopped`) or a transport failure records a down transition but leaves the instance status untouched — health is **observability-only**, so a probe miss (no traffic, a slow cold start, or an Access challenge) never flips a tenant to `error` or takes it out of service. A sleeping/stopped probe also does **not** recover a client left in `error`; only a genuine `running` probe does.

An exact-host per-tenant Access app shadows the wildcard health app, so the provisioner (`Lerma.Cloudflare.AccessPolicy`) also creates a per-tenant **health application** (`{slug}.fouria.io/api/health`) that carries no role requirement (any authenticated user) and allows the health-monitor service token. The monitor authenticates with `HEALTH_MONITOR_ACCESS_CLIENT_ID` / `HEALTH_MONITOR_ACCESS_CLIENT_SECRET`.

The dashboard CD (`.github/workflows/deploy-dashboard.yml`) wires these credentials for both staging and production: it passes `HEALTH_MONITOR_ACCESS_CLIENT_ID`, `HEALTH_MONITOR_ACCESS_CLIENT_SECRET`, and `CF_ACCESS_HEALTH_MONITOR_TOKEN_ID` to Ansible as `health_monitor_access_client_id` / `health_monitor_access_client_secret` / `cf_access_health_monitor_token_id`. When they are omitted, the deployed Lerma probes go out unauthenticated and every Access-protected tenant answers `302` (an Access login redirect), which the probes surface as an Access challenge.

| Variable                                     | Class    | Required      | Description                                                                                                                                                                                                                                                                                                                                                                                                                             |
| -------------------------------------------- | -------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `HEALTH_MONITOR_ENABLED`                     | Variable | No            | Enable the periodic health monitor (default: `true`). Always disabled in test env.                                                                                                                                                                                                                                                                                                                                                      |
| `HEALTH_MONITOR_INTERVAL_MS`                 | Variable | No            | Poll interval in milliseconds (default: `120000`).                                                                                                                                                                                                                                                                                                                                                                                      |
| `HEALTH_MONITOR_TIMEOUT_MS`                  | Variable | No            | Per-probe HTTP timeout in milliseconds (default: `15000`).                                                                                                                                                                                                                                                                                                                                                                              |
| `HEALTH_MONITOR_PLATFORM_DOMAIN`             | Variable | No            | Domain the tenant workers are served under (default: `fouria.io`).                                                                                                                                                                                                                                                                                                                                                                      |
| `HEALTH_MONITOR_SLUG_PREFIX`                 | Variable | No            | Hostname prefix for staging tenants (default: `""`, staging auto-set to `staging-`).                                                                                                                                                                                                                                                                                                                                                    |
| `HEALTH_MONITOR_ACCESS_CLIENT_ID`            | Secret   | No            | Cloudflare Access service-token client ID sent on each probe (`cf-access-client-id`). Required for probing Access-protected tenants.                                                                                                                                                                                                                                                                                                    |
| `HEALTH_MONITOR_ACCESS_CLIENT_SECRET`        | Secret   | No            | Cloudflare Access service-token client secret (`cf-access-client-secret`).                                                                                                                                                                                                                                                                                                                                                              |
| `CF_ACCESS_HEALTH_MONITOR_TOKEN_ID`          | Variable | No            | Pins the service-token id granted access to each per-tenant health app. When empty, `AccessPolicy` resolves the `FourIA Health Monitor` token by name.                                                                                                                                                                                                                                                                                  |
| `INTERNAL_DISPATCH_INVALIDATE_TOKEN`         | Secret   | Yes           | Bearer Lerma sends on the Dispatch Worker's internal routes (`/_control/*`, `/_invalidate/*`), and the value the Worker compares (digest, constant time) for production. The Worker has **no hardcoded fallback** and fails closed (`403`) when unset or mismatched, so cache invalidation, freeze/unfreeze, and the pre-replacement checkpoint cannot run. Production only; staging uses `STAGING_INTERNAL_DISPATCH_INVALIDATE_TOKEN`. |
| `STAGING_INTERNAL_DISPATCH_INVALIDATE_TOKEN` | Secret   | Yes (staging) | Staging-only counterpart uploaded to `fouria-dispatch-staging` and passed to staging Lerma. Never reuse the production value. Roll out the Dispatch Worker before Lerma in each environment.                                                                                                                                                                                                                                            |

#### Dashboard connectivity checks (`StartupHealth`)

Lerma's `StartupHealth` runs external connectivity checks (Cloudflare API token validity/permissions and the container image registry) and caches the result in ETS. The result drives the dashboard connectivity banner (`/dashboard`) and the `/health/liveness` + `/health/readiness` probes. Checks run at startup **and then on a recurring interval** so a transient boot-time failure (e.g. a registry request timing out) clears itself once connectivity is restored: the banner updates live over PubSub and the readiness probe recovers, without restarting the app.

| Variable                     | Class    | Required | Description                                                                                                                                     |
| ---------------------------- | -------- | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------- |
| `STARTUP_HEALTH_INTERVAL_MS` | Variable | No       | Re-check interval in milliseconds (default: `300000`). Set to `0` to run the connectivity checks only once at startup (startup-only behaviour). |

#### Provisioning health probe (`HealthPoller`)

Health verification is **asynchronous and post-deployment**: once a provisioning
run (or upgrade) is finalized, the provisioner schedules a background task that
runs the **non-passive** `/api/health` probe (`HealthPoller`). It is **not** a
deployment step, so a slow container cold start (the Worker's boot budget is
`STARTUP_TIMEOUT_MS`, 300s) never blocks, times out, or rolls back a deployment.
A failed probe is **observability-only**: the downtime is recorded for the
instance uptime widget, but the instance status is left untouched (a probe miss
is not an outage). The poll succeeds (`:ok`) only when the snapshot reports
`status: "running"` — a `200` `status: "sleeping"` is not ready and keeps
retrying until the deadline. The background check is disabled with
`skip_health_poll: true`.

The probe targets the instance's **environment hostname** — the client slug plus
the `-dev` environment suffix for a dev instance, then the `HEALTH_MONITOR_SLUG_PREFIX`
platform prefix (`{prefix}{slug}[-dev].fouria.io`). The platform suffix (`-staging`)
is deliberately **not** part of the hostname; hostname construction must never
reuse the `routable_slug` used for Cloudflare resource names (R2/AI Gateway),
or the platform axis is applied twice.

Upgrade rollback never deletes the existing worker script / container
application, so a transient failure cannot destroy a running tenant. Fresh
provisions still clean up the resources created for the new tenant.

| Variable                  | Class    | Required | Description                                                                                                              |
| ------------------------- | -------- | -------- | ------------------------------------------------------------------------------------------------------------------------ |
| `HEALTH_PROBE_TIMEOUT_MS` | Variable | No       | Provisioning health-probe deadline in milliseconds (default: `360000`). Must exceed the Worker cold-start budget (300s). |
| `HEALTH_PROBE_BACKOFF_MS` | Variable | No       | Delay between provisioning health-probe attempts in milliseconds (default: `3000`).                                      |

### How Secrets Are Deployed

The provisioner deploys all instance secrets as **`secret_text` bindings** directly on the User Worker. These are encrypted at rest by Cloudflare and available at runtime as plain `string` values (not Secrets Store objects).

The provisioner no longer binds a Cloudflare Secrets Store placeholder on tenant Workers. Dynamic secrets use the F4E1-encrypted R2 store; instance secrets are **never** placed in a Secrets Store.

```elixir
# Per-tenant secrets (generated by lerma provisioner)
%{type: "secret_text", name: "MOLTBOT_GATEWAY_TOKEN", text: gateway_token}
%{type: "secret_text", name: "CDP_SECRET", text: derived_cdp_secret}  # HMAC of the gateway token
%{type: "secret_text", name: "OTEL_SHARED_SECRET", text: otel_secret}
%{type: "secret_text", name: "CF_ACCESS_PLATFORM_AUD", text: platform_aud}
%{type: "secret_text", name: "CF_ACCESS_CLIENT_AUD", text: tenant_aud}  # Optional

# Platform-level secrets (from Lerma ENV, optional)
%{type: "secret_text", name: "CLOUDFLARE_AI_GATEWAY_API_KEY", text: ai_gateway_key}
%{type: "secret_text", name: "CLOUDFLARE_API_TOKEN", text: ai_gateway_key}  # Tenant REST alias
# ... etc.
```

See `apps/lerma/lib/lerma/provisioning/tenant_provisioner.ex` for the authoritative binding list.

### Container-Backed Durable Objects

The tenant User Worker's `Sandbox` DO class is a container-backed Durable Object (it extends
`Container` from `@cloudflare/sandbox`). lerma deploys the worker into the dispatch namespace
with a `containers` array in the upload metadata (declaring `Sandbox`), which enables the
container runtime for that class — without it, Cloudflare throws
`Containers have not been enabled for this Durable Object class` when the worker calls into the
container (e.g. `/api/doctor`). lerma also creates/updates a container **application** for the
tenant (image + instance type + max instances) linked to the Sandbox DO namespace.

Container app image comes from `CONTAINER_IMAGE`, and the container instance type is derived
from the instance type chosen at provisioning time. When none is chosen it defaults to `basic`
(4 GB disk — `lite`'s 2 GB disk is too small to unpack the sandbox image).
`CONTAINER_MAX_INSTANCES` configures the max instances (see table above).

The tenant container application is pinned to the **WEUR** deployment zone by default
(`CONTAINER_REGIONS`) and container log capture is enabled (`CONTAINER_OBSERVABILITY_LOGS`).
Observability for the User Worker itself (logs + traces + invocation logs) mirrors the
`apps/fouria/wrangler.jsonc` block and is enabled by default
(`WORKER_OBSERVABILITY_*` vars above).

---

## Legacy Channel Secrets (E2E Testing Only)

> **Deprecated.** Channel secrets are now managed through the integrations dashboard (F4E1-encrypted R2 vault). These env vars are kept **only for E2E test fixtures**. Do not use in production.

> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.

---

## Internal Variables (Auto-Set)

These are automatically derived by the Worker and should never be manually configured:

| Variable                              | Source                                                                                                                                            |
| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- |
| `OPENCLAW_GATEWAY_TOKEN`              | Mapped from `MOLTBOT_GATEWAY_TOKEN`                                                                                                               |
| `OPENCLAW_DEV_MODE`                   | Derived from the `-dev` hostname (`WORKER_URL`) or an explicit `OPENCLAW_DEV_MODE` override; `DEV_MODE` for local development                     |
| `FOURIA_DOCS_CHANNEL`                 | Derived: `latest` on a production-platform production instance, `staging` on the staging platform or a `-dev` environment (`src/docs-channel.ts`) |
| `FOURIA_DOCS_BASE_URL`                | Auto: `https://fouria.io/<channel>` — the channel-matched docs site                                                                               |
| `FOURIA_DOCS_FALLBACK_URL`            | Auto: `https://fouria-docs.pages.dev/<channel>` — public mirror of the same channel                                                               |
| `FOURIA_VERSION`                      | Auto: the Worker bundle's component CalVer (`src/build-info.ts`), so agents can compare against the docs channel's `versions.json`                |
| `OPENCLAW_E2E_TEST_MODE`              | Mapped from `E2E_TEST_MODE`                                                                                                                       |
| `CUSTOMER_SLUG`                       | Set from tenant resolution                                                                                                                        |
| `OTEL_EXPORTER_OTLP_HEADERS`          | Auto: `Authorization=Bearer {OTEL_SHARED_SECRET}`                                                                                                 |
| `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` | Auto: `http://otel.internal/v1/metrics` (sandbox bridge)                                                                                          |
|                                       |
| `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`  | Auto: `http://otel.internal/v1/traces` (sandbox bridge)                                                                                           |
| `OTEL_TRACES_ENABLED`                 | Provisioned by Lerma (`true` for production/staging, `false` for dev); `'true'` enables `diagnostics.otel.traces`                                 |
| `MOLTLAZY_FEATURE_FLAGS`              | Set as per-customer binding (managed via FourIA API); the staging `wrangler.jsonc` env default enables `{"softwareProvisioning":true}`            |
| `MOLTLAZY_ANALYTICS_DATASET`          | Auto-set to tenant Analytics Engine dataset                                                                                                       |
| `CUSTOMER_ANALYTICS_DATASET`          | Auto-set to tenant Analytics Engine dataset                                                                                                       |
| `STARTUP_LOG_VERBOSITY`               | Passed through from Worker env when set (`detailed` default in container)                                                                         |
| `OPENCODE_API_KEY`                    | Per-client key minted by Lerma (`Lerma.Opencode`) from the platform management credential; bound only to the client's `dev` instances             |
| `MOLTLAZY_AGENTS`                     | Deployed by the provisioner when an instance has a selected agent roster; comma-separated premade agent ids                                       |
| `MOLTLAZY_PASO4_AGENTS`               | Deployed by the provisioner (`"true"`) only on Paso4-exclusive (`scope: "paso4"`) instances                                                       |
| `GITHUB_TOKEN`                        | Deployed by the provisioner to Paso4-exclusive instances only; authenticates the `paso4-dev` agent's `gh` CLI                                     |
| `PASO4_TELEGRAM_GROUP_ID`             | Deployed by the provisioner to `dev` instances (lerma env, default `-5442477312`); Telegram group for platform alerts                             |
| `OPENCLAW_CONFIG_PATH`                | Baked into image: `/root/openclaw/.openclaw/openclaw.json` — the REAL config path (see below)                                                     |
| `OPENCLAW_STATE_DIR`                  | Baked into image: `/root/openclaw/.openclaw` — the REAL state dir (see below)                                                                     |

> **Why `OPENCLAW_CONFIG_PATH` / `OPENCLAW_STATE_DIR` point at `/root/openclaw/.openclaw`:**
> `HOME=/home/openclaw` and `/home/openclaw/.openclaw` is a **symlink** to the real config dir at
> `/root/openclaw/.openclaw` (kept under the `/home/openclaw` snapshot root for R2 backups).
> OpenClaw 2026.8.2+ fs-safe refuses atomic file replaces when the immediate parent directory is a
> symlink ("Atomic replace parent must be a real directory"), so config/state writes through the
> `/home` symlink fail. These two env vars point OpenClaw at the real path for its own writes; the
> `/home/openclaw/.openclaw` symlink is preserved so backups and the Worker's `/home`-based config
> reads/writes keep working. Set in the Dockerfile (`apps/fouria/Dockerfile`), not per-tenant.

---

## Setting Variables

### Local Development (.dev.vars)

Copy from `.dev.vars.example`:

```bash
cp apps/fouria/.dev.vars.example apps/fouria/.dev.vars
# Edit .dev.vars with your values
# .dev.vars is gitignored — never commit it
```

In local dev, instance secrets are plain strings (simulating `secret_text` bindings). Dynamic secrets need `BACKUP_ENCRYPTION_KEY` configured with the F4E1 R2 store.

### Production Deployment

Instance secrets are deployed automatically by `lerma` when a tenant is provisioned. No manual `wrangler secret put` commands are needed.

Platform-level secrets should be set as environment variables in the dashboard's deployment:

```bash
CLOUDFLARE_AI_GATEWAY_API_KEY=...
OPENCODE_API_KEY=...   # management key, used to mint per-client dev keys
# ... etc.
```

### OpenTofu (IaC) Mapping

> **Secrets reference:** See [docs/schemas/secrets-manifest.schema.json](./docs/schemas/secrets-manifest.schema.json) for the canonical credential inventory.
