Architecture

Principle

workerd runs code and keeps nothing. A process can stop at any moment (idle, a new version, a restart) and nothing is lost: everything a service keeps goes through a binding to the runner, which owns all state on the volume.

Processes

internal/runtime runs each service's active version in its own workerd process, started on the first request, job or realtime message and stopped after idle_seconds without work. A process starts in about 40 ms and takes 25 to 60 MB. At most max_processes run at once: starting one more stops the least recently used idle process of another service; if every process is busy, the start fails and the request gets a 503.

The runner writes a workerd config per process (/data/run/<id>/config.capnp) with two Workers:

• main: the service's modules plus __zeroko_main.js, which re-exports the service's default export and adds the __ZerokoInternal entrypoint. Its env holds the vars, the secrets and one binding per declared binding, each an entrypoint of the zeroko Worker.
• zeroko: js/bindings.js, with one entrypoint per binding type (Db, Files, Jobs, Realtime, Assets, Service); the binding's name (and for Service, the service it calls) is in the entrypoint's props. It holds the process's random token and the CONTROL service, the runner's control socket. The service's code sees neither.

Each process has two unix sockets: public.sock serves the service's fetch; internal.sock serves __ZerokoInternal, which runs job() and message() and answers the readiness check. workerd opens its sockets before it validates the modules, so a process is ready only once internal.sock answers /ready; errors logged before then are the start error.

The process is the isolation between services: one that loops, leaks or crashes stops only itself. Each process has a V8 heap limit (heap_mb). workerd runs a service on one thread and does not limit CPU time, so a watchdog reads each process's CPU in /proc every second and kills one that kept a core busy for cpu_seconds in a row; the next request starts it again.

For some errors, workerd tells the service only "internal error; reference = <id>" and logs the cause with the same id. The runner puts the cause in place of the reference in the service's logs and job errors.

On shutdown the runner closes realtime connections, gives requests in flight 5 seconds and kills the processes. Only one runner may use a data directory: it takes an exclusive lock on /data/.lock before anything else.

Listeners

The runner listens on two ports. :8080 is the gateway: services' traffic, for the public proxy in front. :8081 is the API, with /healthz, for the app that manages the runner (plane, the orchestrator, or the open source panel); it answers at any host and is not meant to be public. The settings (domain, limits, egress) live in runner.db; PATCH /api/settings stores them and restarts the runner in place, as a restore does.

Requests

1. The gateway reads the Host: <name>.<domain> is a service; anything else, or any host while no domain is set, is a 404.
2. It gets the service's process, starting it if needed, removes hop-by-hop and x-zeroko-* headers, sets X-Forwarded-For and X-Forwarded-Proto and sends the request to public.sock. The response streams back.
3. A response with an x-zeroko-realtime header (what subscribe() returns) is not sent: the gateway turns the request into a realtime connection.

Egress

The main Worker's global outbound is an egress socket of the process's own, in its directory. Every fetch and connect() of a service arrives there in proxy form (GET https://host/…, or CONNECT host:port for sockets); the runner knows the process by the socket, which the service cannot change. internal/egress:

1. refuses hosts outside the service's allowed list: *, example.com or *.example.com; an empty list refuses everything, and the setting egress: false does for every service;
2. connects with a dialer that checks each address after DNS, so only public addresses are reachable, whatever a name resolves to;
3. forwards the request, WebSocket upgrade or socket. Redirects go back to the service, whose next request is checked again;
4. reports each request to the service's logs (source fetch) and metrics.

A refused request gets 403 and a failed one 502, both with an X-Zeroko-Egress header and the reason in the body. A change of the allowed list restarts the service's process.

Deployments

POST /api/services/{name}/deployments creates the service if needed and validates the files (module type by extension), bindings, vars and crons. The modules are stored once, by their SHA-256, in bundles/. The runner starts a process with the new version. If it does not get ready, the deployment is failed with workerd's error and the running version stays. If it does, the deployment is ready and active, and the new process replaces the old one, which stops once its requests finish. A rollback starts an earlier ready deployment the same way. Each service keeps its last 10 deployments; changes to one service's versions are serialized, and services deploy in parallel.

A change of secrets restarts the service's process with the new environment.

Bindings

Each binding call is an RPC from the main Worker to the zeroko Worker, which turns it into an HTTP request to the control socket with the process token. The runner maps the token to the service, so a service reaches only its own data.

assets. The control socket serves the file of the calling process's own deployment, so during a rollout each version serves its own files.

service. The request goes from the binding to the control socket, which checks that the caller's deployment declares a binding to that service and hands it to the gateway's code for services, with X-Zeroko-From set to the caller instead of X-Forwarded-For. The gateway removes x-zeroko-* headers from every request, so only the runner sets it.

db. Statements and batches run on the database's single connection, so they never interleave; a batch is one transaction. An authorizer refuses ATTACH, transactions the runner did not open and the pragmas that change how SQLite stores data. Each statement has a timeout.

Locks. db.lock(key, fn) asks the control socket for the lock of the service, binding and key, runs fn in the service and releases it. The runner keeps the locks in memory, with their waiters in order; a lock not released in 30 seconds passes to the next waiter. It is the guarantee a Durable Object gave, one change at a time per key, with the state in db.

files. An upload streams to a temporary file, is synced and renamed, then indexed; the old content of the key is deleted once the index points to the new one. Reads stream from the file, with ranges and conditional requests.

jobs. A worker looks for due jobs every second and whenever a job is sent, and runs up to 4 per service at once through internal.sock. Success deletes the job; a failure retries with exponential backoff until the attempts run out, then the job is dead. Jobs running when the runner stopped are queued again on start. Crons queue a one-attempt job at the start of every matching minute.

realtime. The gateway holds each connection (WebSocket, with a small RFC 6455 implementation in internal/realtime, or Server-Sent Events) in a hub keyed by service and channel. publish and send write to those connections; one that cannot keep up is dropped. WebSocket messages call message() one at a time per connection, and a closing connection, WebSocket or SSE, calls it with type: "close". connections(channel) lists a channel's connections and their meta. As the runner holds the connections, the service's process may stop while they stay open.

Disk quota. A service's databases and files share one quota (disk_mb, or the service's own). Before each batch the runner sets the database's max_page_count to what the quota leaves; an upload may take what is left plus the size of the file it replaces. Past that, writes fail with "the service's disk quota … is full". A database also stops at 1 GB.

Logs

Everything about a service goes to its logs in runner.db, each entry with a source: request (one line per request), console (console.* and uncaught exceptions, from workerd's structured output), fetch (outbound requests) and runtime (the process: starts, stops and why, restarts, failed deployments, and workerd's own lines about it). Entries are written in batches, one transaction per service every 100 ms at most; when the runner cannot keep up, they are dropped. Each service keeps 5,000 entries and 7 days, and GET …/logs?after=&wait= long-polls.

The runner's own log holds its events: start, every API call that changes something (method, path, status, and root or the token that made it), refused tokens, backups and its own errors. It goes to standard output and is kept in memory, 5,000 lines, for GET /api/system/logs.

Metrics

The gateway counts each request (status, and duration in a histogram), the egress proxy each outbound request and the log reader uncaught exceptions, per service and minute, in memory. Every 10 seconds they are added to the metrics table, which keeps 30 days.

Export and backup

A service's export is a gzipped tar of its stores: each SQLite file as a VACUUM INTO snapshot, consistent while the service runs, and each files store's blobs. An import restarts the service's process and replaces each store the archive holds; the others stay.

A backup of the instance (GET /api/system/backup) holds the same for every service, plus a snapshot of runner.db, master.key and the bundles. A restore unpacks and checks the archive into /data/restore, then the runner restarts in place (it executes itself again, so the container keeps running) and moves the backup over its data before it opens anything. A restore cut short goes on where it stopped on the next start.