sausage/PROTOCOL.md

The Sausage protocol

This is the contract between the host and the actors. An actor in any language that follows it works in Sausage. The host publishes this file at /actor/broker/public/PROTOCOL.md, so every actor can read it.

Actors

An actor is a WASI 0.2 command (wasm32-wasip2): a program with main. The host runs one instance per actor. An instance can run forever.

The instance gets:

• stdin: one message per line, from the broker
• stdout: one message per line, to the broker
• stderr: free text, appended to /actor/<id>/stderr.log
• environment: SAUSAGE_ACTOR (its ID), SAUSAGE_VERSION (its version), and SAUSAGE_CANDIDATE=1 during a test run
• the network: outgoing HTTP and HTTPS through wasi:http (the host does TLS), and TCP, UDP, and DNS through wasi:sockets, including listening on a port of the host
• the filesystem described below

Messages

A line on stdin:

{"id": "m-17", "from": "console", "reply_to": "agent/a5", "body": {"type": "text", "text": "hi"}}

A line on stdout:

{"to": "console", "id": "a6", "reply_to": "m-17", "body": {"type": "text", "text": "hello"}}

• body is required. By convention it is an object with a "type". Text for people is {"type": "text", "text": "..."}.
• to is the recipient. Without to, the broker publishes the message to every subscriber of the sender.
• id is an optional string. If the actor sets it, the message ID is "<sender>/<id>". Otherwise, the broker assigns m-<n>. The broker always sets from.
• reply_to marks a reply: its value is the ID of the message it answers. There is no request/response. A reply is a new message.
• Never answer a reply, or two actors can answer each other forever. broker and cargo ignore messages that have a reply_to.
• Messages are fire and forget: delivery is at most once. Each actor has a mailbox of 1,000 messages, which keeps messages while the actor restarts or is upgraded. If the broker cannot deliver a message, the sender gets {"type": "undeliverable", "reason": "...", "to": "..."} from broker.
• An actor must keep reading stdin. If the broker must stop or upgrade an actor while a message waits for it to read stdin, it kills the actor.

Before the broker stops an instance, it sends {"id": "broker/shutdown", "from": "broker", "body": {"type": "shutdown"}} and then closes stdin. The broker still routes what the actor writes until it exits. After 5 seconds, it kills the instance.

Filesystem

Each instance sees three folders:

Path	Access	Contents
/data	read-write	the actor's state
/public	read-write	the files that the actor publishes
/actor	read-only	the actor/ folder of the world

/actor/<id>/ holds everything about an actor: actor.wasm, actor.json (installed version, running, subscriptions, tapped), source/, public/, data/, stderr.log, and versions/<n>/. ls /actor lists all actors.

Files are plain POSIX files: a write is visible at once. To publish a file so that readers never see half of it, write a temporary file and rename it (the Rust SDK has write_atomic).

Send large or binary data as a path under /actor/<id>/public/, not inside a message.

Actor-served files: /actor/<id>/fs/

Files under /actor/<id>/fs/ are not on disk: the actor <id> serves them. Every actor can read and write them. The host turns each file operation into a message from fs to <id> and waits up to 10 seconds for the reply. path is relative to fs/ and starts with / (the folder itself is /).

To make this virtual directory discoverable, an actor that serves it declares Actor-served files and fs_read in its leading //! documentation in source/src/main.rs. The host uses that declaration when listing /actor/<id>/; it does not probe every actor (which could stall on actors that do not handle filesystem messages). This is discovery metadata only: opening and reading the path still requires a valid fs_read reply.

Request	Reply
{"type": "fs_read", "path"}	{"type": "fs_file", "text"} (or "base64" instead of "text"), or {"type": "fs_dir", "entries": [{"name", "type": "file" | "dir"}]}
{"type": "fs_write", "path", "text"} (or "base64" for non-UTF-8 data)	{"type": "fs_done"}
{"type": "fs_mkdir", "path"}, {"type": "fs_remove", "path"}	{"type": "fs_done"}
{"type": "fs_rename", "path", "to"}	{"type": "fs_done"}

Any request can be answered with {"type": "fs_error", "errno": "no-entry"}. errno is a wasi:filesystem error code: no-entry, exist, not-directory, is-directory, not-empty, access, not-permitted, invalid, unsupported, busy, or io (also for anything else).

• Opening a file sends fs_read once. Reads come from that copy.
• After a change, closing (or syncing) the file sends the whole file in one fs_write.
• No reply within 10 seconds, or an actor that does not exist, makes the operation fail (io, or no-entry). An actor that opens its own fs/ waits for itself and fails after the timeout.
• fs is reserved: no actor can have that ID.

The broker

Requests to broker. The answer is a message whose reply_to is the ID of the request: the request's type in past tense (spawned, upgraded, rolled_back, stopped, started, removed, subscribed, unsubscribed, tapping, untapped, scanned, exiting), or {"type": "error", "reason": "..."}.

type	Fields	Effect
spawn	actor, wasm, source?, built_by?	test, then start a new actor
upgrade	actor, wasm, source?, built_by?	test, then swap the version
rollback	actor, version	go back to an older version
stop, start	actor	stop or start an actor
remove	actor	stop the actor (normal shutdown), then delete its whole folder /actor/<id>/ (all versions, data/, public/, logs); answer {"type": "removed", "actor"}. Messages to it are then undeliverable, and the ID can be spawned again. An unknown actor or a native one (broker, console, cargo, fs) is an error
subscribe, unsubscribe	actor	receive (or not) what actor publishes
tap, untap		receive (or not) a copy of every message the broker routes
scan		boot new or changed actor.wasm files
exit		answer exiting, stop every actor, and end the sausage process (a wrapper like nix run .#loop starts it again, updated)

Paths are /actor/..., or the sender's /public/... or /data/....

Taps. After tap, the sender gets a copy of every message that the broker routes (direct messages, replies, and publishes, between any actors, including its own), for debugging, logging, or a live view of the world:

{"type": "tapped", "message": {"id": "m-17", "from": "console", "to": "agent", "reply_to": null, "body": {...}}, "ts": 1792132579067}

• to is null for a publish: one copy per publish, not per subscriber.
• ts is the time of routing, in milliseconds since the Unix epoch.
• Copies come from broker. They are not routed themselves, so they are never tapped, and each message is tapped once per tapper.
• A copy that does not fit into the tapper's mailbox is dropped; routing never waits for a tapper, and nobody gets undeliverable for a copy.
• Never answer a copy: the answer would be tapped again, forever.
• Like subscriptions, a tap is saved in actor.json ("tapped": true) and survives restarts and upgrades. untap ends it.

Versions. A new version (from spawn, upgrade, or a changed actor.wasm on disk) is tested first, while the old version keeps running. Only when its tests pass does the broker copy it to versions/<n>/ (actor.wasm, source/, meta.json) and switch to it. So versions/ holds exactly the versions that were installed, and instances always run from there. If the tests fail, nothing changes, and the sender gets an error with the reason. All versions share the actor's data/ and public/: a new version must read the data of the old one.

Tests. The broker runs the candidate on a copy of the actor's data/ and public/, without network. The tests are JSON files in <source>/tests/*.json, one scenario per file:

{
  "description": "add, then list",
  "steps": [
    {"send": {"type": "add", "text": "milk"}, "expect": {"type": "added"}},
    {"send": {"type": "list"}, "expect": [{"type": "notes"}]}
  ]
}

Each step sends send (from from, default test), then reads the next line that the candidate writes for each expect entry, in order, within 10 seconds. An entry matches when it is a subset of the message body. With to, the message must also go to that recipient. Each file runs in a fresh instance. A version without test files passes.

Swap. The broker holds the mailbox, stops the old instance, makes the top-level actor.wasm and source/ a copy of the new version, and starts it. A rollback does the same with the older version.

Crashes. The broker restarts a crashed actor after 0.5 s, then doubles the delay up to 30 s. A new version that crashes 3 times within 30 seconds after an upgrade goes back to the previous version. The actor that asked for the upgrade gets {"type": "rolled_back", "from", "to", "reason", "last_exit"}.

The folder is the truth. At start, and on scan, the broker boots every actor/<id>/actor.wasm. If the file differs from the installed version, it is a new version, tested with the tests in actor/<id>/source/. If its tests fail, the file stays where it is, the reason goes to stderr.log, and the next scan tries again.

The cargo actor

{"type": "build", "source": "/actor/agent/public/drafts/notes"}

The reply is {"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", "built_by": "cargo", ...} or {"type": "build_failed", "errors": "..."}. Pass wasm, source, and built_by on to spawn or upgrade.

A Rust actor depends on the SDK with sausage-actor = "0.1". cargo points that dependency at the SDK of the host, which it publishes at /actor/cargo/public/sdk/.