> ## Documentation Index > Fetch the complete documentation index at: https://docs.norafleet.ai/llms.txt > Use this file to discover all available pages before exploring further. # Architecture # Architecture > Canonical public overview of how Nora's control plane is wired together — frontends, API, queues, workers, backend adapters, and the runtime contract package. Nora is the self-hosted AI agent ops platform — an operator-facing control plane for OpenClaw, Hermes, and supported sandboxed runtimes. Three browser surfaces sit behind one nginx ingress, platform state lives in PostgreSQL, background work is coordinated through Redis and BullMQ, and runtimes are provisioned (or proxied) through backend adapters that share the runtime contract package in `agent-runtime/`. The public repo centers on a single control-plane host. Agent workloads can stay on the local Docker host, be placed onto GA Kubernetes targets, use an experimental operator-registered Remote Docker host over SSH, or opt into experimental Proxmox LXC placement without changing the core operator workflow. NemoClaw is an experimental sandbox profile layered onto supported targets; NemoClaw on Proxmox remains blocked. Operator dashboard — the surface this architecture renders to the user ## System map ```mermaid theme={null} theme={null} flowchart LR Browser["Browsers
public, operator, admin"] --> Nginx["nginx"] subgraph ControlPlane["Nora control plane"] Nginx --> Marketing["frontend-marketing
Next.js"] Nginx --> Dashboard["frontend-dashboard
Next.js"] Nginx --> Admin["admin-dashboard
Next.js"] Nginx --> API["backend-api
Express.js"] API --> Postgres["PostgreSQL"] API --> Redis["Redis + BullMQ"] Redis --> Provisioner["worker-provisioner"] Redis --> BackupWorker["worker-backup"] API --> Contracts["agent-runtime contracts
and backend catalog"] Provisioner --> Contracts end subgraph RuntimeInfra["Runtime execution layer"] Provisioner --> Adapters["Backend adapters
Docker / Remote Docker / k3s-k8s / Proxmox"] Adapters --> Runtime["Provisioned agent runtime"] API <--> Runtime end ``` ## Major components | Component | Repo surface | Role | | ------------------------ | ------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | Public and auth UI | `frontend-marketing/` | Landing pages, signup, login, and public entrypoint routes. | | Operator workspace | `frontend-dashboard/` | Deployments, migration/import, fleet operations, filesystem access, logs, settings, Agent Hub, alert rules, backups, and runtime interaction surfaces. | | Admin workspace | `admin-dashboard/` | Fleet-wide administration, moderation, audit, platform settings, release upgrades. | | Reverse proxy | `nginx.conf`, `nginx.public.conf`, `infra/` | Routes browser traffic to the correct UI or API surface and carries WebSocket / SSE traffic. | | Control-plane API | `backend-api/` | Auth, persistence, workspace RBAC, API keys, queue orchestration, monitoring, alert rules, Agent Hub logic, runtime coordination, and runtime proxy endpoints. | | Durable state | PostgreSQL 15 | Accounts, agents, workspaces, members, remote hosts and grants, API keys, deployments, migration drafts, snapshots, Agent Hub content, integrations, channels, metrics, events. | | Queue and worker handoff | Redis 7 + BullMQ | Carries deployment, ClawHub-install, Hermes-skill, backup, and alert-delivery jobs between the API and workers. | | Provisioning worker | `workers/provisioner/` | Resolves backend choice, injects bootstrap state, applies imported managed state, waits for readiness, persists status. Also runs the ClawHub-install, Hermes-skill, and alert-delivery workers. | | Backup worker | `workers/backup/` | Captures encrypted backup archives, runs scheduled backups, prunes expired backups. | | Runtime contract package | `agent-runtime/` | Shared runtime-side files, ports, endpoint conventions, bootstrap helpers, and backend metadata used by the API and worker. | ## Control plane ### Request routing ```mermaid theme={null} theme={null} flowchart TB Browser["Browser"] --> Nginx["nginx"] Nginx --> Marketing["frontend-marketing"] Nginx --> Dashboard["frontend-dashboard"] Nginx --> Admin["admin-dashboard"] Nginx --> API["backend-api"] Dashboard --> API Admin --> API Marketing --> API API --> Postgres["PostgreSQL"] API --> Redis["Redis + BullMQ"] API <--> Runtime["Runtime proxy surface"] ``` WebSocket upgrades are handled at the nginx layer; the Express app attaches WS handlers via `attachGatewayWS`. SSE chat endpoints (`/api/agents/*/gateway/chat`) have chunked transfer encoding disabled for real-time streaming. OAuth callbacks land at the marketing app before redirecting into backend-issued HttpOnly session cookies. ### API responsibilities `backend-api/server.ts` is the control-plane integration hub. It wires together: * Security middleware: helmet, CORS, rate limiting, correlation IDs, prototype-pollution-safe input validation * Auth middleware (session JWT + workspace-scoped API key bearer auth) * Workspace-scoped route families for agents, alert rules, API keys, channels, integrations, LLM providers, monitoring, Agent Hub, and workspace members * Skill catalogs for both runtime families: ClawHub browse and install for OpenClaw agents, Hermes skills browse/search/install served from a disk-cached copy of the aggregated Hermes skills index, and the instance-curated Hermes Skills Library of pinned registry refs surfaced first in skill pickers * Session-only Remote Docker registration for personal hosts plus workspace-based host grants * Admin-only routes for platform Remote Docker hosts, fleet operations, audit, moderation, release upgrades, and platform settings * Gateway proxy with SSRF guards (`gatewayProxy.ts` — exposes `createGatewayRouter` + `attachGatewayWS`) * Background telemetry (`backgroundTasks.ts`, `scheduler.ts`) and release-availability checks (`releaseInfo.ts`) ### State and queue boundaries | Service | What it stores or carries | | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | PostgreSQL | Users, user groups, workspaces, workspace members, personal/platform Remote Docker hosts and grants, API keys (HMAC-hashed), agents, deployments, migration drafts, snapshots, Agent Hub records, alert rules, backups, integrations, and events. | | Redis + BullMQ | Five queues: `deployments`, `clawhub-jobs`, `hermes-skills-jobs`, `backups`, `alert-deliveries`. Each has its own retry, backoff, and DLQ retention configured in `backend-api/redisQueue.ts`. | The API persists desired state first, then hands long-running work to a queue-backed worker. That keeps provisioning failures, retries, and delayed readiness out of the synchronous browser request path. ### Workspace RBAC Shared agents and other workspace resources use workspace RBAC. Personal Remote Docker hosts remain directly user-owned and may be granted into a workspace without transferring credentials or ownership. Platform hosts are owned by the control plane, managed by any platform admin, and may also grant access through workspaces, direct users, user groups, or all accounts. A workspace has a creator (`workspaces.user_id`) plus a row per member in `workspace_members` with one of four roles: | Role | Can read | Can edit | Can manage members | Can delete workspace | | -------- | -------- | -------- | ------------------ | -------------------- | | `viewer` | ✅ | ❌ | ❌ | ❌ | | `editor` | ✅ | ✅ | ❌ | ❌ | | `admin` | ✅ | ✅ | ✅ | ❌ | | `owner` | ✅ | ✅ | ✅ | ✅ | Permission checks read from `workspace_members.role`, never from `workspaces.user_id` directly. Invitations land in `workspace_invitations` with a hashed token signed by `NORA_WORKSPACE_INVITE_SECRET` (falls back to `JWT_SECRET`). For either host type, a workspace viewer can see the host and an editor or higher can deploy. Workspace access never permits connection or credential changes: the personal owner manages a personal host, while any platform admin manages a platform host. ### API keys and bearer auth Each workspace can mint scoped API keys (`/app/workspaces/:id/api-keys`). Tokens are bearer-only, prefixed `nora_`, hashed at rest with HMAC-SHA256, and carry a fixed scope set: `agents:read`, `agents:write`, `workspaces:read`, `monitoring:read`, `integrations:read`, `integrations:write`. Workspace mutation, member management, key issuance, and every `/remote-hosts` management operation stay on session auth. Migration upload, draft, and live-inspection routes are also session-only. An API key cannot mint another key, manage a Remote Docker host, or ask the control plane to inspect a container or network source. ## Migration and filesystem contract ### Migration flow Nora ships a control-plane-managed migration path for both `openclaw` and `hermes`: 1. The operator prepares a migration draft from one of: * An uploaded Nora migration bundle or legacy OpenClaw template JSON * A local live Docker source, available only to a platform admin on a self-hosted control plane 2. `backend-api/agentMigrations.ts` normalizes the imported data and stores an encrypted manifest in PostgreSQL. 3. The operator deploys a new Nora-managed agent using that draft. 4. The provisioning worker recreates the runtime under Nora control instead of adopting the original runtime in place. Nora does not bind to the legacy runtime as-is. It uses the draft as desired state for a fresh Nora-managed deployment. Live SSH pull is disabled until Nora can require independently verified host-key trust. Export a bundle on the source machine and upload it instead. Live Docker pull is privileged because it reads through the control plane's Docker socket; hosted deployments and non-admin users cannot invoke it. ### Import surface The current public import contract is intentionally scoped: * `openclaw`: agent files, workspace content, session memory, and provider material Nora can extract from supported source files * `hermes`: workspace content, model config, supported Hermes channel config, and provider environment material * Both families: supported Nora-managed state such as imported provider records, channel/integration wiring where available, and per-agent secret overrides Unsupported runtime-specific state is surfaced as draft warnings instead of being silently invented or applied. ### Export and live files * Nora-managed agents can be exported as `nora-migration-bundle/v1` bundles for recreation on another Nora control plane. * The agent detail Files tab reads the actual runtime filesystem through `backend-api/agentFiles.ts`. * Filesystem access is root-allowlisted: a writable workspace root and curated read-only system roots for inspection and download. * The browser never receives arbitrary host filesystem access; reads and writes are mediated through runtime-aware backend commands. ## Runtime provisioning ### Selection model Nora chooses a concrete backend through four layers of intent: | Layer | Current values | Meaning | | ---------------- | ----------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | | Runtime family | `openclaw`, `hermes` | Which operator contract the runtime satisfies. | | Deploy target | `docker`, `k8s`, `remote-docker`, `proxmox` | Which adapter family runs the workload. Docker and Kubernetes are GA; Remote Docker and Proxmox are experimental. `k3s` normalizes to `k8s`. | | Execution target | `docker`, `k8s:`, `remote:`, `proxmox` | The concrete destination recorded on the agent. Registered Kubernetes clusters and Remote Docker hosts require their exact id. | | Sandbox profile | `standard`, `nemoclaw` | Which isolation profile wraps the runtime. NemoClaw is experimental, OpenClaw-only, and blocked on Proxmox. | The worker resolves the final backend through shared metadata in `agent-runtime/lib/backendCatalog.ts`. Placement is explicit: selecting `remote:build-host` does not let Nora substitute another host. The port allocator prevents Nora-managed published-port collisions, but Remote Docker has no aggregate host-capacity admission, automatic placement, failover, drain, or rebalance. See [Provisioner backends](/configuration/provisioner-backends) for the supported combinations and their maturity. ### Provisioning lifecycle ```mermaid theme={null} theme={null} sequenceDiagram participant UI as Dashboard UI participant API as backend-api participant Queue as Redis/BullMQ participant Worker as worker-provisioner participant Catalog as backend catalog participant Adapter as backend adapter participant Runtime as agent runtime participant DB as PostgreSQL UI->>API: Submit deploy or redeploy request Note over UI,API: Optional migration draft may already exist API->>DB: Persist desired agent state API->>Queue: Enqueue deployment job Queue->>Worker: Deliver job Worker->>Catalog: Resolve backend, sandbox, runtime metadata Worker->>Adapter: Create or update runtime Worker->>Runtime: Inject bootstrap files and env Worker->>Runtime: Reapply imported managed state when present Worker->>Runtime: Wait for readiness Worker->>DB: Persist running, warning, or failed status API->>Runtime: Proxy gateway, exec, log, or dashboard traffic ``` ### Bootstrap contract The worker and backend adapters share one runtime bootstrap package from `agent-runtime/`: * Runtime library files are injected into the launched environment. * Template payload files are copied into the runtime workspace when present. * Runtime environment variables are assembled centrally. * Endpoint conventions stay shared across the worker and API. Shared ports: | Port | Purpose | | ------- | ------------------------------- | | `9090` | Nora runtime-side HTTP contract | | `18789` | OpenClaw gateway port | | `9119` | Hermes dashboard port | ### Backend adapter code sharing Compose mounts `./workers/provisioner/backends` into `backend-api` at `/app/backends`. Adapter code is **physically shared** between the API and the provisioner worker. When editing adapters, verify both consumers still work. ## Background work | Queue | Job type | Concurrency | Retry | Worker | What it does | | -------------------- | -------------------------------- | --------------------------------------- | --------------------------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------ | | `deployments` | `deploy-agent` | `DEPLOYMENT_WORKER_CONCURRENCY` (6) | 5 attempts, exponential 3 s base | `workers/provisioner/worker.ts` | Provisions or redeploys an agent runtime through the chosen backend adapter. | | `clawhub-jobs` | `install-skill` | 1 | 1 attempt (operator drives retries) | `workers/provisioner/worker.ts` | Runs `clawhub install` inside an OpenClaw agent and persists the installed-skill state. | | `hermes-skills-jobs` | `install-skill` / `delete-skill` | 1 | 1 attempt (operator drives retries) | `workers/provisioner/worker.ts` | Runs `hermes skills install`/`uninstall` inside a Hermes agent and persists the installed-skill state. | | `backups` | `run-backup` | `BACKUP_WORKER_CONCURRENCY` (2) | 2 attempts, exponential 5 s base | `workers/backup/worker.ts` | Captures, encrypts, and uploads an agent backup archive, then prunes expired backups. | | `alert-deliveries` | `deliver-webhook` | `ALERT_DELIVERY_WORKER_CONCURRENCY` (5) | `ALERT_DELIVERY_ATTEMPTS` (5), exponential 1 s base | `workers/provisioner/worker.ts` | Posts a webhook delivery for an alert rule. Each job is one (rule, channel) pair. | Failed jobs are retained on each queue for inspection. The DLQ surface for deployments is exposed via `getDLQJobs` / `retryDLQJob` in `backend-api/redisQueue.ts`. ## Trust boundaries * Browsers never talk directly to PostgreSQL or Redis. All browser traffic enters through nginx and reaches stateful services through the frontends or `backend-api/`. * `backend-api/` owns auth, persistence, queue orchestration, release metadata, and runtime-facing proxy routes. Frontends do not provision runtimes directly. * `backend-api/` also owns migration draft inspection/storage and all runtime file access mediation. Browser users do not receive direct host or container filesystem access. * `workers/provisioner/` and `workers/backup/` handle long-running infrastructure work outside the request path. They consume queued jobs and write the result back into control-plane state. * `agent-runtime/` defines the runtime-side contract used after launch. Control-plane code depends on that contract rather than embedding backend-specific assumptions everywhere. * External execution systems such as Docker, experimental Remote Docker over SSH, Kubernetes, experimental Proxmox LXC placement, and NVIDIA secure sandboxes are reached through backend adapters instead of directly from browser surfaces. * Remote-host SSH credentials are encrypted at rest and masked in API responses. Only the personal owner or a platform admin for the corresponding host type can replace them; all-account, direct-user, user-group, and workspace grants permit placement only and never reveal credentials. * Stored secrets — provider keys, integration credentials, OAuth tokens, backup encryption material, SMTP password — are AES-256-GCM encrypted with `ENCRYPTION_KEY`. API keys and Agent Hub installation keys are HMAC-hashed at rest, not encrypted. ## Deployment topologies Fleet view — control plane managing multiple agent placements | Topology | Ingress owner | Control plane placement | Agent placement | Best fit | | ----------------------------------- | ---------------------------------------------- | ----------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------ | | Local single-host | Nora nginx on `NGINX_HTTP_PORT` | One Docker Compose host | Local Docker by default | Evaluation, local proof, small self-hosted installs | | Public domain, Nora-managed ingress | Nora nginx on public ports | One Docker Compose host | Local Docker or supported external targets | Straightforward public self-hosting | | Public domain behind external proxy | Host or upstream proxy terminates and forwards | One Docker Compose host | Local Docker or supported external targets | Existing nginx, Cloudflare, or host-managed TLS setups | | External runtime targets | Same ingress as above | One Docker Compose host | Kubernetes GA; Remote Docker, NemoClaw, and Proxmox LXC experimental | Teams that need different runtime placement without changing operator workflow | The clearest public path today is one host running the control plane, with agent runtimes launched locally through Docker by default. Public-domain setups can either let Nora own public ingress directly or put an external reverse proxy in front of Nora's internal nginx. ## Current constraints * The public OSS path is primarily a single-host control plane. The repo does not currently claim a first-class HA or distributed control-plane deployment story. * OpenClaw is the default runtime family. Hermes is a narrower, deployment-first runtime path with a different operator contract. * Migration recreates runtimes under Nora control; it does not adopt a legacy runtime in place. * Hermes is a runtime family, not a backend id. Docker and Kubernetes are the current GA Hermes execution targets. Remote Docker Hermes support is experimental and currently requires the agent deployment API because the Hermes picker does not merge registered hosts. * Kubernetes (k3s/k8s) is a GA execution-target option for agents. Remote Docker, NemoClaw, and Proxmox LXC placement are experimental. * Remote Docker's manual connection result has no expiry or continuous health loop. Nora does not automatically move workloads when a remote host becomes stale, unreachable, or overloaded. ## Related Supported deploy targets, their maturity, and required configuration. Every variable Nora reads, grouped by subsystem. Workspace ownership, RBAC roles, member invitations, and Remote Docker grants. SSH, test scope, explicit placement, sharing, capacity limits, and safe retirement.