platforms/devplace/pubsub_service.md

DevPlace Pub/Sub Websocket Bus

The database-free publish/subscribe bus at /pubsub: the websocket protocol and frame types, dotted topics with trailing wildcards, the service-lock-owner rule behind close code 4013, and the ephemeral best-effort delivery guarantees.

devplaceapihttp-apipubsubwebsocketmessagingreal-timefan-outephemeraltopics

The pub/sub service is a general-purpose, database-free publish/subscribe bus over websockets. It lets the frontend, background services, and Devii broadcast and receive messages on named topics without polling. It is mounted at /pubsub.

Audience: administrators, maintainers, and frontend developers.

It reuses the in-process hub pattern used elsewhere in the platform (the same shape as the job progress hub) and the service lock-owner gating used by Devii and the tools websockets. It stores nothing: there is no table, no message log, and no persistence.

How delivery works without a database

Messages are delivered entirely in memory. The hub keeps a map from topic pattern to the set of live websockets subscribed to it; publishing a message sends it to every matching socket and drops any socket that has gone away.

To make this correct across multiple web workers without a message broker or a database, the bus is served on the service lock owner only. The WS /pubsub/ws handler closes with code 4013 on any non-owner worker, and the browser client transparently reconnects until it lands on the owner. Because every subscriber converges on the one owner worker, an in-process fan-out reaches all of them. Background services, which already run on the lock owner, publish in-process.

This is a deliberate trade-off. The bus is ephemeral and best-effort: a publish reaches whoever is connected at that moment, and a server restart drops in-flight messages. It is built for live UI updates and coordination, not for durable messaging. If you need durability, persist the event yourself (for example with the database API) and publish a notification on the bus.

Topics and wildcards

A topic is a dotted name such as public.deploys or user.<uid>.inbox, matching ^[A-Za-z0-9_.-]{1,128}$. A subscription may use a trailing wildcard: subscribing to foo. receives foo and any foo.bar. The client dispatches wildcard matches locally as well, so a single subscription callback fires for every matching topic.

The websocket protocol

Connect to WS /pubsub/ws. The server first sends { "type": "ready", "actor": "<kind>" }. Then send frames:

{ "type": "subscribe",   "topic": "public.deploys" }
{ "type": "unsubscribe", "topic": "public.deploys" }
{ "type": "publish",     "topic": "public.deploys", "data": { "service": "web", "status": "ok" } }

The server replies with {"type":"subscribed"} / {"type":"unsubscribed"} / {"type":"ack","delivered":N}, or {"type":"error","message":"..."} when a topic is invalid or not allowed. A delivered message arrives as:

{ "type": "message", "topic": "public.deploys", "data": { ... }, "ts": "2026-..." }

Payloads are capped at 64 KB and are never evaluated; they are forwarded verbatim.

Who can do what

The connecting identity is resolved from an administrator session or the internal key, and then falls back to the session user or a guest. Authorization is checked on every frame:

  • Administrators and internal services may subscribe and publish to any topic.
  • Members may subscribe to public. and to their own user.<uid>. namespace, and may publish only to their own namespace. Publishing to public.* is restricted to administrators and internal services so a member cannot broadcast to everyone.
  • Guests get read-only access to public.* topics, and only when pubsuballowguests is enabled on the service config; otherwise the socket is closed.

Publishing from the backend and over HTTP

Background code on the lock owner publishes in-process:

from devplacepy.services import pubsub

await pubsub.publish("public.deploys", {"service": "web", "status": "ok"})

Administrators and internal services can also publish over plain HTTP, which is useful from scripts and from Devii:

  • POST /pubsub/publish with { "topic": "...", "data": ... } publishes a message and returns the delivered count. Because the bus lives on the lock owner, a request that lands on a non-owner worker returns 409 and should be retried.
  • GET /pubsub/topics lists the live topics and their subscriber counts.

Both are admin/internal only, and a publish is recorded in the audit log as pubsub.publish.

Live notification toasts and badge counts

The bus powers live notification toasts. NotificationRelayService runs on the lock owner, polls the notifications table for new rows, and publishes each one to the recipient's user.<uid>.notifications topic. The frontend (static/js/LiveNotifications.js, wired as app.liveNotifications) subscribes to that topic for the signed-in user and raises a click-through toast. Because the relay runs on the same lock owner where every subscriber converges, the in-process publish reaches the recipient, and because a notifications row exists only when the in-app channel is enabled, a toast fires exactly for the notifications that user has enabled. See Notification settings.

The same relay tick also publishes the recipient's fresh unread counts { notifications, messages } to user.<uid>.counts. A single publish point covers both new notifications and new direct messages, because a direct message also inserts a message-type notification row. The header badge controller (static/js/CounterManager.js, wired as app.counters) subscribes to that topic and updates the badges instantly, and keeps a 60-second poll of GET /notifications/counts only as a fallback that reconciles decrements on read and any missed message.

Live view relay (admin views without polling)

LiveViewRelayService is a second bridge on the lock owner that replaces the per-client HTTP polling on the admin live views with server push. On each tick it inspects the live subscriptions and, only for topics that currently have a subscriber, computes a fresh snapshot and publishes it. Each snapshot is the same JSON the matching admin endpoint already returns, so the page renders it unchanged. The views and their refresh cadence are:

  • container.list (4s) and project.<slug>.containers (3s): container instance lists.
  • container.<uid>.detail (4s) and container.<uid>.logs (3s): a single instance's status, metrics, schedules, and log tail.
  • fleet.bots (2s): the bot fleet monitor frames.
  • admin.services (5s) and admin.services.<name> (5s): background service status, logs, and metrics.
  • admin.ai-usage.<hours> (15s): the AI gateway usage analytics for the selected window.
  • admin.backups (8s): the backups dashboard (storage usage, backup archives, and schedules).

All of these are administrator-only by the topic rules above. The admin pages subscribe to their topic for live updates and keep a slow (15 to 30 second) HTTP poll as the initial load and as a fallback if the bus is unavailable. Computing a snapshot only when someone is subscribed means expensive views (such as AI usage analytics) cost nothing while no administrator is watching. To add a new admin live view, register one entry in the relay's view table that returns the endpoint's payload and have the page subscribe to the topic.

The backups view is the one case with a per-viewer field. The backup download link is restricted to the primary administrator, so the relay publishes the dashboard with downloads withheld rather than leak a download URL to every administrator subscribed to the shared topic. The backups page keeps its own download permission from its authoritative per-viewer poll and builds the download URL on the client, and the download route itself remains restricted to the primary administrator.

Frontend client

The browser client is created once as app.pubsub (static/js/PubSubClient.js). It connects lazily on first use, reconnects with backoff (and a fast retry on the 4013 owner bounce), and re-subscribes to all topics after a reconnect.

const off = app.pubsub.subscribe("public.deploys", (data, topic) => {
    app.toast.show(`${data.service}: ${data.status}`);
});

app.pubsub.publish("user." + myUid + ".ping", { at: Date.now() });

off(); // unsubscribe

Configuration

On /admin/services the Pub/Sub service exposes a single toggle, pubsuballowguests, which lets unauthenticated guests subscribe to public.* topics. Disabling the service closes the websocket with the standard "try again later" code. The service has no run loop; it exists to surface the toggle and to gate the websocket.

Security summary

  • The websocket requires a resolvable identity; guests are admitted only when explicitly enabled.
  • Every subscribe and publish is authorized per frame against the topic namespace rules.
  • HTTP publish and topic introspection are admin/internal only.
  • Topic names are validated and payloads are size-capped; nothing is persisted, so there is no data at rest.