surma/sausage
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

PROTOCOL.md, DECISIONS.md, README.md, AGENTS.md for plugins, processes, and JSON-RPC

Browse Source
This commit is contained in:
surma
2026-10-22 23:50:24 +01:00
parent c901568f56
commit d1be22f653
4 changed files with 505 additions and 338 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+15 -15
View File
@@ -10,25 +10,25 @@ Keep it that way.
caller. No configuration that nobody asked for. caller. No configuration that nobody asked for.
3. **Readable.** A new reader must understand a file without the others. 3. **Readable.** A new reader must understand a file without the others.
Each module starts with a doc comment that says what it owns. Each module starts with a doc comment that says what it owns.
4. **Efficient** on the paths that run per message. 4. **Efficient** on the paths that run per call.
5. **Raw and minimal.** Sausage has no permissions, sandboxing beyond WASI, 5. **Raw and minimal.** Sausage has no permissions, sandboxing beyond WASI,
limits, or authentication. Do not add them to the core. Add nothing to limits, or authentication. Do not add them to the core. Add nothing to
the host that an actor (or the agent, by changing itself) could do. the host that a plugin (or the agent, by changing itself) could do.
## Layout ## Layout
- `cli/`: the host (`sausage` binary and library). - `cli/`: the host (`sausage` binary and library): the core, the native
- `lib.rs`: `Host`, which starts and stops a world. `cargo` plugin, the WASI test runner, and the console.
- `broker.rs`: routing, mailboxes, the version store, broker requests. - `vfs/`: `sausage-vfs`, the virtual filesystem for WASI guests. It knows
- `supervisor.rs`: runs one WASM actor (restarts, swaps, rollbacks). nothing about Sausage.
- `runtime.rs`: Wasmtime instances and what they get (mounts, network). - `interface/`: `sausage-interface`: OpenRPC documents, the compatibility
- `candidate.rs`: tests a new version. `cargo_actor.rs`: builds Rust. check, and the generator of client code.
- `world.rs`: the on-disk layout. `console.rs`: the terminal. - `sdk/`: `sausage-plugin`, the SDK for Rust plugins, and `sdk/macros/`, its
- `sdk/`: `sausage-actor`, the SDK for Rust actors. Never published: the macros. Never published: `cargo` points `sausage-plugin = "0.1"` at its
`cargo` actor points `sausage-actor = "0.1"` at its local copy. local copy.
- `agent/`: the agent actor. `sausage init` embeds and builds it. - `agent/`: the agent plugin. `sausage init` embeds and builds it.
- `examples/`: small actors. They are not workspace members. - `examples/`: small plugins. They are not workspace members.
- `PROTOCOL.md`: the contract between the host and the actors. Change it in - `PROTOCOL.md`: the contract between the core and the plugins. Change it in
the same commit as the behavior. `DECISIONS.md` says why; when the two the same commit as the behavior. `DECISIONS.md` says why; when the two
disagree, `PROTOCOL.md` and the code win. disagree, `PROTOCOL.md` and the code win.
@@ -36,7 +36,7 @@ Keep it that way.
```sh ```sh
nix flake check -L # all tests below; CI runs this on every push nix flake check -L # all tests below; CI runs this on every push
nix develop -c cargo test -p sausage -p sausage-actor -p agent nix develop -c cargo test --workspace
nix develop -c cargo build --release -p agent --target wasm32-wasip2 nix develop -c cargo build --release -p agent --target wasm32-wasip2
nix develop -c cargo clippy --workspace nix develop -c cargo clippy --workspace
``` ```
DECISIONS.md
+56 -40
View File
@@ -1,55 +1,71 @@
# Decisions # Decisions
Why Sausage is the way it is. `PROTOCOL.md` and the code own the details; Why Sausage is the way it is. `PROTOCOL.md` and the code own the details.
when this file disagrees with them, they win. When this file disagrees with them, they win.
## The shape ## The shape
- **Actors are WASI commands, not components with interfaces.** stdin, - **JSON-RPC, not component interfaces.** Every language can read and write
stdout, files, HTTP, and sockets are enough, and every language can do lines of JSON, and models know JSON-RPC from LSP and MCP. A call has a
them. result, like an LLM tool call.
- **Messages are JSON lines through a broker, fire and forget.** Like - **A process per call.** WASI has no threads, so processes give
Erlang: no acknowledgments, no request/response. A reply is a message with parallelism. A new version applies at the next call, and old processes
`reply_to`. Sausage is not a reliable message bus. retire after a grace period.
- **The broker accepts only `.wasm` files.** It knows no source language. - **Attached by default.** A process does not outlive its caller by
`cargo` is one compiler actor; others can add more languages. accident. A long-running process must ask for `detach`.
- **The world folder is the source of truth.** Everything about an actor is - **Services in one file.** Only the core starts and restarts them, so
in `actor/<id>/`. Copy a folder into another world, and it runs there. upgrades and singletons need no logic in plugins.
- **The agent is a normal WASM actor.** It can read and upgrade its own - **Files to inspect, calls to act.** This keeps the core API small, and the
source, and change how it does inference. agent's file tools are enough to explore the world.
- **The core accepts only `.wasm` files.** It knows no source language.
`cargo` is one way to get a `.wasm` file.
- **The world folder is the source of truth.** Copy a plugin folder into
another world, and it works there.
- **The agent is a normal plugin.** It can read and upgrade its own source,
and change how it does inference.
## Interfaces
- **Generated imports and interface checks, without version numbers.** A
model that translates a description by hand makes errors. A generator does
not. The check catches the most common error in a world that changes
itself: a change to one plugin breaks another one.
- **A small schema subset.** With Draft 7 and a fixed subset, our own check
can decide every case. It never has to guess.
- **Docs are part of the interface.** Other agents work from them, so the
SDK requires a doc comment on each method, type, and field.
- **The check covers calls, not state.** Old and new versions share
`/state`. A plugin keeps its state format compatible, or migrates it in
`rpc.retired`.
## Minimal on purpose ## Minimal on purpose
The harness does only what it needs to keep going. Everything else can be an The core does only what a plugin cannot do for itself. Everything else can be
actor, or a change that the agent makes to itself. a plugin, or a change that the agent makes to itself.
- **The host is not self-modifiable; actors are.** So the host keeps only - **No permissions, no limits, no hiding.** Any process can call, upgrade,
what an actor cannot do for itself: routing, running instances, versions, or stop any plugin, read and write every file, and use the network. The
tests before an install, and the rollback of a version that crash-loops. checks protect against mistakes, not against plugins. Locks are advisory.
Without that rollback, a broken self-upgrade would end the agent. - **Locks are in the core.** A plugin cannot see that another process died,
- **No permissions, no limits, no hiding.** Any actor can message, upgrade, so it cannot release that process's lock. The core can.
or stop any actor, read every folder, and use the network. People who want rules - **Our own VFS, in its own crate.** It works on Linux and macOS, it makes
can build them. `/proc` live, and plugin-served files need it later. Mounts now, overlays
- **No re-delivery and no state snapshots.** A crash loses the message, and later.
all versions of an actor share `data/` and `public/`. A rollback restores - **TCP endpoints for now.** They need no host code. VFS pipes can replace
code, not data. them later behind the same SDK API.
- **Simple scheduling for now.** An epoch tick every 10 milliseconds keeps
the core responsive and makes kills reliable.
- **The agent is dumb.** It has a short prompt, a few tools, an append-only - **The agent is dumb.** It has a short prompt, a few tools, an append-only
history that it sends in full, and no compaction. When the context is too history that it sends in full, and no compaction. When the context is too
large, it stops sending the oldest turns, so it never gets stuck. large, it stops sending the oldest turns, so it never gets stuck.
## Small choices ## Small choices
- **Plain POSIX files.** An earlier version staged every write until close. - **Plain POSIX files.** Programs like `tempfile`, git, and SQLite expect
That broke `tempfile`, git, and SQLite, which expect normal files. Atomic normal files. Atomic writes use a temporary file and `rename`.
publishing is the usual temporary file + `rename`. - **Tests run under WASI.** `cargo test` on the host cannot find failures
- **Test first, then switch.** A new version is tested where it lies. Only that occur only in WASI, for example a missing WASI function or a panic at
a version that passes goes to `versions/`, and the old one runs until start. The test runner uses the same Wasmtime and the same mounts as the
then. `versions/` keeps every version that was installed, so a rollback core.
can always go back. A file that fails stays where it was put. - **The SDK is not published.** `cargo` points `sausage-plugin = "0.1"` at
- **Tests are JSON transcripts** next to the source. They run on a copy of its own copy, which it also shows in `/plugins/cargo/sdk/`.
the actor's state, without network, so a test cannot send real requests.
- **`broker` and `cargo` never answer a reply,** so no two actors answer
each other forever. The agent starts a turn only for text that is not a
reply, for the same reason.
- **The SDK is not published.** `cargo` points `sausage-actor = "0.1"` at its
own copy, which it also shows in `/actor/cargo/public/sdk/`.
PROTOCOL.md
+391 -241
View File
@@ -1,275 +1,425 @@
# The Sausage protocol # The Sausage protocol
This is the contract between the host and the actors. An actor in any This is the contract between the core and the plugins. A plugin in any
language that follows it works in Sausage. The host publishes this file at language that follows it works in a Sausage world. The core shows this file
`/actor/broker/public/PROTOCOL.md`, so every actor can read it. at `/proc/core/PROTOCOL.md`. `DECISIONS.md` in the repository says why the
contract is the way it is.
## Actors Rust plugins use the SDK `sausage-plugin`. Its source is at
`/plugins/cargo/sdk/`, and its crate docs explain each item.
An actor is a WASI 0.2 command (`wasm32-wasip2`): a program with `main`. The ## 1. Terms
host runs one instance per actor. An instance can run forever.
The instance gets: | Term | Meaning |
- **stdin:** one message per line, from the broker
- **stdout:** one message per line, to the broker
- **stderr:** free text, appended to `/actor/<id>/stderr.log`
- **environment:** `SAUSAGE_ACTOR` (its ID), `SAUSAGE_VERSION` (its version),
and `SAUSAGE_CANDIDATE=1` during a test run
- **the network:** outgoing HTTP and HTTPS through `wasi:http` (the host does
TLS), and TCP, UDP, and DNS through `wasi:sockets`, including listening on
a port of the host
- **the filesystem** described below
## Messages
A line on **stdin**:
```json
{"id": "m-17", "from": "console", "reply_to": "agent/a5", "body": {"type": "text", "text": "hi"}}
```
A line on **stdout**:
```json
{"to": "console", "id": "a6", "reply_to": "m-17", "body": {"type": "text", "text": "hello"}}
```
- `body` is required. By convention it is an object with a `"type"`.
Text for people is `{"type": "text", "text": "..."}`.
- `to` is the recipient. Without `to`, the broker publishes the message to
every subscriber of the sender.
- `id` is an optional string. If the actor sets it, the message ID is `"<sender>/<id>"`.
Otherwise, the broker assigns `m-<n>`. The broker always sets `from`.
- `reply_to` marks a reply: its value is the ID of the message it answers.
There is no request/response. A reply is a new message.
- **Never answer a reply,** or two actors can answer each other forever.
`broker` and `cargo` ignore messages that have a `reply_to`.
- Messages are fire and forget: delivery is at most once. Each actor has a
mailbox of 1,000 messages, which keeps messages while the actor restarts
or is upgraded. If the broker cannot deliver a message, the sender gets
`{"type": "undeliverable", "reason": "...", "to": "..."}` from `broker`.
- An actor must keep reading stdin. If the broker must stop or upgrade an
actor while a message waits for it to read stdin, it kills the actor.
Before the broker stops an instance, it sends
`{"id": "broker/shutdown", "from": "broker", "body": {"type": "shutdown"}}`
and then closes stdin. The broker still routes what the actor writes until
it exits. After 5 seconds, it kills the instance.
## Filesystem
The world folder is the foundation of every instance's filesystem:
| Path | Access | Contents |
| --- | --- | --- |
| `/` | read-write by every actor | the whole world folder |
| `/self` | read-write | alias for the actor's own folder (`public/`, `data/`, `source/`, etc.) |
| `/data` | read-write | alias for the actor's state folder |
| `/public` | read-write | alias for the actor's published files |
| `/actor` | read-write by every actor | the `actor/` folder of the world |
Every actor can read and write every file in the world, including other actors'
`/actor/<id>/data/` and `/actor/<id>/public/`. Nothing there is private.
Creating a folder in the world makes it visible to running actors immediately:
`/config`, `/docs`, `/www`, and `/skills` need no new mounts or host restart.
New worlds include an empty `/config` for global configuration by convention;
existing worlds can simply create it. Keep secrets out of global configuration.
Candidate instances see the same writable world root. Their `/self` alias
points to the scratch actor folder; `/public` and `/data` still point to its
corresponding children, and network access remains off;
absolute world paths are not isolated from the live world.
`/actor/<id>/` holds everything about an actor: `actor.wasm`, `actor.json`
(installed version, running, subscriptions, tapped, watches), `source/`, `public/`, `data/`,
`stderr.log`, and `versions/<n>/`. `ls /actor` lists all actors.
Files are plain POSIX files: a write is visible at once. To publish a file
so that readers never see half of it, write a temporary file and `rename` it
(the Rust SDK has `write_atomic`).
Send large or binary data as a path under `/actor/<id>/public/`, not inside a
message.
### Actor-served files: `/actor/<id>/fs/`
Files under `/actor/<id>/fs/` are not on disk: the actor `<id>` serves them.
Every actor can read and write them. The host turns each file operation into a
message from `fs` to `<id>` and waits up to 10 seconds for the reply. `path`
is relative to `fs/` and starts with `/` (the folder itself is `/`).
To make this virtual directory discoverable, an actor that serves it declares
`Actor-served files` and `fs_read` in its leading `//!` documentation in
`source/src/main.rs`. The host uses that declaration when listing
`/actor/<id>/`; it does not probe every actor (which could stall on actors that
do not handle filesystem messages). This is discovery metadata only: opening
and reading the path still requires a valid `fs_read` reply.
| Request | Reply |
| --- | --- | | --- | --- |
| `{"type": "fs_read", "path"}` | `{"type": "fs_file", "text"}` (or `"base64"` instead of `"text"`), or `{"type": "fs_dir", "entries": [{"name", "type": "file" \| "dir"}]}` | | world | The folder that a `sausage` host runs. It is `/` for every process. |
| `{"type": "fs_write", "path", "text"}` (or `"base64"` for non-UTF-8 data) | `{"type": "fs_done"}` | | core | The host. It runs processes, routes calls, serves the virtual files, and keeps the journal. |
| `{"type": "fs_mkdir", "path"}`, `{"type": "fs_remove", "path"}` | `{"type": "fs_done"}` | | plugin | A folder `/plugins/<name>/` with a `plugin.wasm`. A plugin does not run by itself. |
| `{"type": "fs_rename", "path", "to"}` | `{"type": "fs_done"}` | | process | One run of a plugin version. Its ID is `<plugin>-<uuidv7>`, for example `notes-0197f3a2-7c1e-7d2b-9a41-5f0c2e8b1d34`. |
| call | A JSON-RPC request. `notes.add` calls the method `add` of the plugin `notes`. |
| interface | A set of methods. A plugin has one main interface. It can also declare endpoint interfaces. |
| endpoint | An interface that a running process serves on a TCP port. An endpoint file tells others where it is. |
| service | A call in `/config/boot.json`. The core starts it at boot and again after an upgrade of its plugin. |
| import | An interface of another plugin that a plugin uses. |
| provider, consumer | The plugin that has an interface, and a plugin that imports it. |
| journal | The central log of the world. |
Any request can be answered with `{"type": "fs_error", "errno": "no-entry"}`. ## 2. The world folder
`errno` is a `wasi:filesystem` error code: `no-entry`, `exist`,
`not-directory`, `is-directory`, `not-empty`, `access`, `not-permitted`,
`invalid`, `unsupported`, `busy`, or `io` (also for anything else).
- Opening a file sends `fs_read` once. Reads come from that copy. ```text
- After a change, closing (or syncing) the file sends the whole file in one <world>/
`fs_write`. plugins/<name>/
- No reply within 10 seconds, or an actor that does not exist, makes the plugin.wasm the current version
operation fail (`io`, or `no-entry`). An actor that opens its own `fs/` plugin.prev.wasm the version before, for `core.revert`
waits for itself and fails after the timeout. source/ the source. `cargo.build` builds it from here.
- `fs` is reserved: no actor can have that ID. It also sends `fs_changed` imports/<plugin>.json descriptions of the imported plugins
to watchers (see Watches). state/<plugin>/ persistent state of a plugin, by convention
state/core/journal.sqlite the journal
config/ global configuration
boot.json the services
agent.json API, base URL, model, and token of the agent
```
## The broker - Every process can read and write all of it. There are no permissions.
- Plugin names use `a-z`, `0-9`, `-`, and `_`. The name `core` is reserved.
- The world folder must not contain `proc`, `run`, or `self`. These names are
mount points (section 4). The host does not start a world that has them.
Requests to `broker`. The answer is a message whose `reply_to` is the ID of ## 3. Calls and processes
the request: the request's `type` in past tense (`spawned`, `upgraded`,
`rolled_back`, `stopped`, `started`, `removed`, `subscribed`, `unsubscribed`,
`tapping`, `untapped`, `watching`, `unwatched`, `exiting`), or `{"type": "error", "reason": "..."}`.
| `type` | Fields | Effect | ### 3.1 The wire protocol
- A plugin is a `wasi:cli/command` component for `wasm32-wasip2`.
- stdin and stdout carry JSON-RPC 2.0, one message per line.
- A line is a message only if it is a JSON object with `"jsonrpc": "2.0"`.
The core discards every other line on stdout. For the first discarded line
of a process, it writes one warning to the journal.
- Each line on stderr goes to the journal. Log to stderr.
- `params` is always an object. Batches are not supported.
- Write each message in one write, and write a newline before it. Then a
`print!` without a newline cannot damage a message.
### 3.2 Method names
- A method name has the form `<target>.<method>`. The core splits it at the
first `.`.
- `notes.add` starts a new process of `notes` and sends it the request `add`.
`notes.rpc.discover` sends it `rpc.discover`.
- `core.<method>` is a method of the core (section 9).
- Method names that start with `rpc.` are reserved: `rpc.discover`,
`rpc.retired`, and `rpc.endpoint`.
### 3.3 What a call does
1. The core gives the call a UUIDv7 and starts the process `<plugin>-<uuid>`
with the current version of the plugin.
2. The core writes the request to the stdin of the process. The request ID is
the process ID, and the method has no plugin prefix.
3. The process writes the response to stdout. The core sends the response to
the caller, with the caller's request ID.
4. If the process ends before it answers, the caller gets the error -32002.
Its `data` has the exit status and the last journal lines of the process.
Before the core decides this, it reads the stdout of the process to the
end.
- A process can write calls to stdout at any time, to the core and to
plugins. The responses come on stdin. A process can have many calls in
flight.
- The core keys each pending request by the process and the exact JSON ID.
The core drops a response with an unknown or late ID.
- A notification to `plugin.method` (without an ID) starts an attached
process. That process gets the notification without an ID, and nobody gets
a result.
### 3.4 Detached calls
`core.call {"method": "www.start", "params": {...}, "detach": true}` makes the
call and returns its result to the caller. The new process has no parent, so
it does not end with its caller.
Sent as a notification (without an ID), the same `core.call` starts the
process and forgets it.
### 3.5 Notifications to the caller
A process can tell its caller about progress. It sends the notification
`core.notify {"name": "text", "params": {...}}`. The core gives it to the
caller as `core.notify {"call": <ID of the caller's request>, "name": "text",
"params": {...}}`. After the call has ended, the core drops such
notifications.
So a process never needs to know who called it.
### 3.6 Lifetime
A process lives until `main` returns. A process that uses the SDK lives
while one of these runs:
- a handler of a call,
- a background task,
- an endpoint that listens.
A process has three states:
1. **running**
2. **stopping**: the core has closed stdin. EOF on stdin means "stop now".
- New calls of the process fail at once, and its pending calls fail.
- A stop hook can still write files and send notifications. It cannot
make calls, because no response can arrive.
- After 5 seconds, the core kills the process.
3. **ended**: the core releases the locks of the process (section 6) and
removes its folder in `/proc`.
Rules:
- `core.stop {pid}` starts the stopping state. `core.kill {pid}` kills the
process at once.
- When a process ends, the core stops each of its attached children. All
children stop in parallel, so a whole tree takes 5 seconds at most.
- A detached process has no parent. Only `core.stop`, `core.kill`, an
upgrade (section 3.7), and the end of the host stop it.
- At host shutdown, the core stops every process in parallel.
Note: on Unix, children outlive their parent. Here, an attached child ends
with its parent. Only `detach` makes a process independent.
### 3.7 Upgrades
A process is an instance of one plugin version. When a new version becomes
active (section 10.3):
1. New calls get the new version at once.
2. Each process of an older version gets a grace period to finish its work.
The default is 60 seconds. A plugin can declare its own value in
`rpc.discover` (`x-sausage-grace`). Then the core stops the process.
3. When the old process of a service has ended, the core starts the service
again with the new version.
4. When the last process of an older version has ended, the core starts a
process of the new version and calls `rpc.retired {"previous":
"<sha256 of the older file>"}`. A plugin can use this hook to migrate its
state, or to write a marker that its other new processes wait for. The
result goes to the journal.
During the grace period, an old process can call a method that a new
provider no longer has.
### 3.8 Environment and network
- `SAUSAGE_PID`: the process ID.
- `SAUSAGE_PLUGIN`: the plugin name.
- `SAUSAGE_PARENT`: the parent: `console`, `core`, or a process ID.
Every process gets the network: outgoing HTTP and HTTPS through `wasi:http`,
and TCP, UDP, and DNS through `wasi:sockets`.
A process that computes for a long time does not block other processes. The
core interrupts it every 10 milliseconds.
### 3.9 Errors
The standard JSON-RPC codes apply. The core adds these codes:
| Code | Meaning |
| --- | --- |
| -32001 | There is no such plugin. |
| -32002 | The process ended without an answer. `data` has the exit status and the last journal lines. |
| -32003 | The activation was rejected. `data` has the reasons. |
## 4. The filesystem
Each process gets one preopen, `/`. The core resolves each path through a
mount table:
| Path | What it is | Access |
| --- | --- | --- | | --- | --- | --- |
| `spawn` | `actor`, `wasm`, `source`?, `built_by`? | test, then start a new actor | | `/` | the world folder | read-write |
| `upgrade` | `actor`, `wasm`, `source`?, `built_by`? | test, then swap the version: a running actor is stopped (normal shutdown) and starts the new version; a stopped actor gets the new version and stays stopped, with its mailbox untouched, until `start`. The answer has `running` | | `/proc` | generated by the core | read-only |
| `rollback` | `actor`, `version` | go back to an older version; like `upgrade`, a stopped actor stays stopped. The answer has `running` | | `/self` | the folder `/proc/<own pid>/`, in memory | read-write |
| `stop`, `start` | `actor` | stop or start an actor | | `/run` | in memory, shared, empty at each host start | read-write |
| `remove` | `actor` | stop the actor (normal shutdown), then delete its whole folder `/actor/<id>/` (all versions, `data/`, `public/`, logs); answer `{"type": "removed", "actor"}`. Messages to it are then undeliverable, and the ID can be spawned again. An unknown actor or a native one (`broker`, `console`, `cargo`, `fs`) is an `error` |
| `subscribe`, `unsubscribe` | `actor` | receive (or not) what `actor` publishes |
| `tap`, `untap` | | receive (or not) a copy of every message the broker routes |
| `watch`, `unwatch` | `path` | receive (or not) `fs_changed` for every change of `path` or anything under it; answer `{"type": "watching" \| "unwatched", "path"}` with the path as an absolute world path |
| `exit` | | answer `exiting`, stop every actor, and end the `sausage` process with exit code 75 (a wrapper like `nix run .#loop` starts it again, updated) |
**Headless.** `sausage run --headless [dir]` (or `sausage --headless`, or `SAUSAGE_HEADLESS=1`) turns the console input off: the host does not read stdin, so end of input (`/dev/null`) does not stop the world. `nix run .#loop -- --headless <world>` passes it through the loop. It runs until SIGTERM or SIGINT (stops every actor, exit code 0) or a broker `exit` request (exit code 75, as above). Without the flag, EOF on stdin or `/quit` ends the host. A listing of `/` shows `proc`, `run`, and `self`. A rename across two of
these areas fails.
Paths are absolute world paths, or the sender's `/self/...`, `/public/...` or ```text
`/data/...` aliases. Native senders (such as the console) can also supply /proc/
absolute host paths; apart from `/actor` and these aliases, core/
these retain their host meaning. PROTOCOL.md this file
graph.json plugins, imports, and the result of each check
**Taps.** After `tap`, the sender gets a copy of every message that the graph.dot the same graph for Graphviz
broker routes (direct messages, replies, and publishes, between any actors, locks.json held locks and waiting processes
including its own), for debugging, logging, or a live view of the world: plugins/<name>/
discover.json the rpc.discover document of the current plugin.wasm
```json client.rs the Rust code that import!("<name>") generates
{"type": "tapped", "message": {"id": "m-17", "from": "console", "to": "agent", "reply_to": null, "body": {...}}, "ts": 1792132579067} <pid>/
info.json plugin, version, method, params, parent, detached, start time, state
log the journal lines of this process
<files> the files that the process writes to /self
``` ```
- `to` is `null` for a publish: one copy per publish, not per subscriber. Rules:
- `ts` is the time of routing, in milliseconds since the Unix epoch.
- Copies come from `broker`. They are not routed themselves, so they are
never tapped, and each message is tapped once per tapper.
- A copy that does not fit into the tapper's mailbox is dropped; routing
never waits for a tapper, and nobody gets `undeliverable` for a copy.
- Never answer a copy: the answer would be tapped again, forever.
- Like subscriptions, a tap is saved in `actor.json` (`"tapped": true`) and
survives restarts and upgrades. `untap` ends it.
**Watches.** The host watches the whole world folder with the OS file - A process writes its own folder only through `/self`. Writes through
watcher: every change inside it counts, whoever makes it (actors, a 9P mount, `/proc` always fail.
a shell or `git` on the host). Actor-served files (`/actor/<id>/fs/`) are not - In a process folder, the names `info.json`, `log`, and `fs` belong to the
on disk and do not count. After `watch {path}`, the sender gets for each core. A write to `/self/info.json` fails.
change of `path` or a path under it (`/actor/a` covers `/actor/a/x`, not - A read of a generated file returns a snapshot that ends. For example,
`/actor/ab`): `/proc/<pid>/log` returns the lines up to the time of the open.
- The folder of a process goes away when the process ends. Its files are in
memory. Large or persistent data belongs in `/state`.
- Do not send a `/self` path to another process. There it means a different
folder, like `localhost` on another machine.
Principle: **inspect through files, act through calls.**
## 5. Endpoints
An endpoint is a JSON-RPC interface that a running process serves over TCP,
for example the control interface of a web server.
- The process binds a free port on 127.0.0.1. When the listener is ready, it
writes an endpoint file in one atomic step (write a temporary file, then
rename it). The file has one line: the address, the owner process, and the
interface. Example: `tcp:127.0.0.1:41234 www-0197… control`.
- Usually the file is in `/self`. A service that others must find uses a
fixed path in `/run`, for example `/run/secrets`.
- A client reads the file and connects. Its first request is
`rpc.endpoint`, which returns `{"owner", "plugin", "interface"}`. If they
do not match the file, the client disconnects. So a stale file or a reused
port cannot connect a client to the wrong process.
- JSON-RPC runs on the connection, one message per line, in both directions.
Notifications of a handler (`core.notify`) go over the connection.
- A plugin declares its endpoint interfaces in `rpc.discover`
(`x-sausage-endpoints`).
## 6. Locks
- `core.lock {"path": "/state/notes/notes.json"}` returns `{"held": true}`
when the caller holds the lock.
- Each acquisition gets its own place in a FIFO queue. The same process
cannot take one lock twice: its second acquisition waits behind its first.
- With `"wait": false`, the call returns at once with `{"held": true}` or
`{"held": false}`.
- `core.unlock {path}` (a notification) releases the lock.
- When a process ends, the core releases all of its locks, also after a
crash or a kill. It waits until the last file operation of the process has
returned. So a write of a killed process cannot land after the next
process got the lock.
- Locks are advisory. The core does not block writes to the file: the path
is only a name.
- SQLite has no file locks in WASI. Hold the lock while the database is open.
## 7. The journal
- An SQLite file at `/state/core/journal.sqlite`. Only the core writes it.
- Each entry has a UUIDv7 (it sorts by time), the process ID, the plugin, the
stream (`stderr` or `core`), and the line.
- The core writes its own events too: start (with method and params), exit
status, stop, kill, upgrades, and the warning about discarded stdout.
- `core.log {plugin?, pid?, since?, until?, grep?, limit?}` queries the
journal. `/proc/<pid>/log` shows the lines of one running process.
## 8. Boot and services
At start, the core starts each service in `/config/boot.json`, in order:
```json ```json
{"type": "fs_changed", "path": "/actor/notes/public/list.md", "op": "write"} [
{"method": "secrets.serve", "params": {}},
{"method": "www.start", "params": {"port": 8080}}
]
``` ```
- `op` is `write`, `create`, `remove`, or `rename` (then `to` is the new - A service is a detached call with the parent `core`. The core waits for
path; a watcher of either path gets it). its result before it starts the next one. The result shows that the
- Changes of one path within about 100 ms become one event (a `write` after service is ready.
a `create` stays a `create`). - If a service fails, the core writes the error to the journal and continues.
- The host's temporary files (names starting with `.sausage-tmp-`) are - After an upgrade of the plugin of a service, the core starts the service
skipped; renaming one onto a file is a `write` of that file. again with the new version (section 3.7).
- Events come from `fs`. Like tap copies, they are not routed, a full - The core does not restart other detached processes.
mailbox drops them, and they never block the writer. Never answer them. - A singleton is a service with a fixed endpoint in `/run`. Only the core
- If the OS drops events, every watcher gets one starts it, so two instances cannot start at the same time.
`{"type": "fs_changed", "path": "/", "op": "rescan"}`: anything may
have changed.
- A watch is saved in `actor.json` (`"watches": [...]`) and survives
restarts and upgrades. `unwatch` with the same path ends it.
**Versions.** A new version (from `spawn`, `upgrade`, or a changed ## 9. The core API
`actor.wasm` on disk) is tested first, while the old version keeps running.
Only when its tests pass does the broker copy it to `versions/<n>/`
(`actor.wasm`, `source/`, `meta.json`) and switch to it. So `versions/` holds
exactly the versions that were installed, and instances always run from
there. If the tests fail, nothing changes, and the sender gets an `error`
with the reason. All versions share the actor's `data/` and `public/`: a
new version must read the data of the old one.
**Tests.** The broker runs the candidate on a copy of the actor's `data/` | Method | Params | Result |
and `public/`, without network. The tests are JSON files in | --- | --- | --- |
`<source>/tests/*.json`, one scenario per file: | `core.call` | `method`, `params`, `detach`? | the result of the method |
| `core.notify` | `name`, `params` | none (a notification) |
| `core.stop` | `pid` | `{}` when the process has ended |
| `core.kill` | `pid` | `{}` |
| `core.lock` | `path`, `wait`? | `{"held": bool}` |
| `core.unlock` | `path` | none (a notification) |
| `core.activate` | `plugin` and `wasm`, or `plugins: [{plugin, wasm}]`. `force`? | the new `rpc.discover` documents |
| `core.revert` | `plugin`, `force`? | like `core.activate` |
| `core.log` | filters (section 7) | journal entries |
| `core.exit` | none | `{}`. Then the host stops with exit code 75. |
```json Everything else is a file.
{
"description": "add, then list",
"steps": [
{"send": {"type": "add", "text": "milk"}, "expect": {"type": "added"}},
{"send": {"type": "list"}, "expect": [{"type": "notes"}]}
]
}
```
Each step sends `send` (from `from`, default `test`), then reads the next ## 10. Interfaces
line that the candidate writes for each `expect` entry, in order, within 10
seconds. An entry matches when it is a subset of the message body. With
`to`, the message must also go to that recipient. Each file runs in a fresh
instance. A version without test files passes.
**Swap.** The broker holds the mailbox, stops the old instance, makes the ### 10.1 `rpc.discover`
top-level `actor.wasm` and `source/` a copy of the new version, and starts
it. A stopped actor is not started: it keeps its mailbox and runs the new
version at the next `start`. A rollback does the same with the older version.
**Crashes.** The broker restarts a crashed actor after 0.5 s, then doubles Each plugin answers `rpc.discover` with an OpenRPC 1.2 document. The schemas
the delay up to 30 s. A new version that crashes 3 times within 30 seconds are JSON Schema Draft 7, as OpenRPC requires. Shared schemas are in
after an upgrade goes back to the previous version. The actor that asked for `components.schemas`, and `$ref`s point to `#/components/schemas/<Name>`.
the upgrade gets `{"type": "rolled_back", "from", "to", "reason", "last_exit"}`. Params are by name. The document can have these extension fields:
**The folder is the truth.** At start, the broker boots every - `x-sausage-params` for each method: a `$ref` to the schema of the params
`actor/<id>/actor.wasm`. Afterwards, the file watcher reports a new or object.
changed `actor/<id>/actor.wasm` (or a new folder `actor/<id>/`) once it is - `x-sausage-endpoints`: the endpoint interfaces by name, each with its
quiet for about 100 ms: a new file becomes a new actor, a changed one an `methods`.
upgrade, tested first with the tests in `actor/<id>/source/`. A file equal - `x-sausage-imports`: for each provider, the part of its document that the
to the installed version changes nothing. If its tests fail, the file stays plugin uses.
where it is, the reason goes to `stderr.log`, and the next change of the - `x-sausage-grace`: the grace period in seconds.
file tries again. When the OS drops events, the broker checks every actor
folder again, as at start.
## The `cargo` actor The schema of params describes what the provider can read. The schema of a
result describes what the provider writes. Descriptions are part of the
interface: other agents read them to use it.
```json The core starts a process only to call `rpc.discover`. Do not do expensive
{"type": "build", "source": "/actor/agent/public/drafts/notes"} work at start.
```
The build request may include `"skip_tests": true` to skip Rust unit tests. ### 10.2 Compatibility
By default, `cargo` first runs `cargo test` for the host target (not WASI),
so pure-logic unit tests in a `wasm32-wasip2` actor crate can run without a
WASI test runner. If tests fail or cannot compile, the build fails before the
WASI build. The reply carries `tests: {"passed": <count>, "failed": <count>,
"output": "<cargo test stdout and stderr>"}`; counts sum Cargo's test-result
summaries, and a test command/compilation failure without a summary counts as
one failure. With `skip_tests: true`, counts are zero and output says tests
were skipped.
The reply is `{"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", "built_by": "cargo", "tests": {...}, ...}` A provider can add methods, optional params, and result fields. For each
or `{"type": "build_failed", "errors": "...", "tests": {...}}`. Pass imported method:
`wasm`, `source`, and `built_by` on to `spawn` or `upgrade`.
Build scripts can compile C for `wasm32-wasip2`: the host sets 1. The provider has the method.
`CC_wasm32_wasip2` and `AR_wasm32_wasip2` to a WASI clang with wasi-libc, so 2. Each params value that the consumer can send is valid for the provider.
the `cc` crate (for example `rusqlite` with `bundled`) works. Its headers are 3. Each result that the provider can return is valid for the consumer.
WASI 0.1: C code that needs sockets does not build.
A Rust actor depends on the SDK with `sausage-actor = "0.1"`. `cargo` points - The same rules apply to endpoint interfaces.
that dependency at the SDK of the host, which it publishes at - There are no version numbers. This check replaces them.
`/actor/cargo/public/sdk/`. - The check covers a fixed subset of JSON Schema: objects with `properties`
and `required`, the primitive types, arrays, optional and nullable values,
string enums, integer ranges, and `$ref`. A construct outside the subset
fails the check.
- The check covers calls, not state. Old and new processes share the files in
`/state`, and `core.revert` restores only code. A plugin must keep its
state format compatible, or migrate it in `rpc.retired`.
### 10.3 Activation
`core.activate {"plugin": "notes", "wasm": "/state/cargo/builds/7/notes.wasm"}`:
1. The core copies the file. All further steps use this copy, even if the
file changes.
2. It starts the copy and calls `rpc.discover`.
3. It waits until it has the documents of all plugins (section 10.4).
4. It checks the imports of the new version against the current providers.
5. It checks each current consumer of the plugin against the new version.
6. If all checks pass, `plugin.wasm` becomes `plugin.prev.wasm`, and the copy
becomes `plugin.wasm`. Then the upgrade starts (section 3.7).
If a check fails, nothing changes, and the error -32003 lists each problem,
with a counterexample where possible.
- The core runs one activation at a time.
- `plugins: [...]` activates several plugins together. The checks then use
the new set.
- `force: true` skips steps 4 and 5.
- `core.revert {plugin}` activates `plugin.prev.wasm` in the same way, and
the current version becomes `plugin.prev.wasm`.
- A person can also copy a file to `plugin.wasm` by hand. That skips the
checks.
### 10.4 The graph
- The core keeps the `rpc.discover` documents of all plugins, by file hash.
At start, it gets them in the background.
- A file watcher on `/plugins/*/plugin.wasm` updates them, for example after
a copy by hand. `/proc/core/graph.json` then shows each broken import, and
the journal records it.
- An import of a plugin that does not exist is "missing".
## 11. Building and tests
The core has one native plugin: `cargo`. `cargo.build {"plugin": "notes"}`
builds `/plugins/notes/source/`:
1. It copies the source once. The tests and the build both use this copy.
2. It runs the tests under WASI.
3. It builds the release `.wasm` for `wasm32-wasip2`.
4. It returns `{"wasm": "/state/cargo/builds/<n>/notes.wasm", "tests":
{"passed", "failed", "output"}, "warnings"}`. If a step fails, it returns
an error with the compiler or test output.
- A call to `cargo.build` gets a process ID, a folder in `/proc`, and journal
entries, like other calls.
- The dependency `sausage-plugin = "0.1"` points to the SDK copy at
`/plugins/cargo/sdk/`.
- Build scripts can compile C: `CC_wasm32_wasip2` and `AR_wasm32_wasip2`
point to a WASI clang.
- The build does not activate the result. Call `core.activate` next.
Tests:
- Each test runs in its own instance, so a panic ends only its own test.
- The normal Rust test setup works: `#[test]`, `tests/*.rs`, name filters,
and `--ignored`. `#[should_panic]` cannot work, because WASI cannot catch
a panic.
- Each test gets an empty temporary world as `/`, the package folder at its
path, and its own `/proc`, `/self`, and `/run`.
- The test runner answers `core.lock`, `core.unlock`, `core.notify`, and
`core.log`. A call to a plugin fails: a test has no other plugins.
README.md
+43 -42
View File
@@ -2,11 +2,11 @@
You can see how the sausage is made. You can see how the sausage is made.
Sausage is a world of small programs ("actors") that you grow by talking to Sausage is a world of small WASI programs ("plugins") that you grow by
it. Each actor is a WASI program with stdin and stdout. Actors send each other talking to it. A plugin is a `.wasm` file with JSON-RPC methods. Each call
JSON messages, one per line, through a broker. The agent is an actor too: it to a plugin starts a new process. The agent is a plugin too: it writes new
writes new actors in Rust, builds them, tests them, starts them, and can plugins in Rust, builds them, tests them, activates them, and can change its
change its own code. own code.
## Start ## Start
@@ -29,8 +29,8 @@ nix run git+https://gitea.surma.technology/surma/sausage -- run # starts the
`init` asks for the API shape (`openai-responses` or `anthropic-messages`), `init` asks for the API shape (`openai-responses` or `anthropic-messages`),
the base URL, the model, and the token. It reads `SAUSAGE_API`, the base URL, the model, and the token. It reads `SAUSAGE_API`,
`SAUSAGE_BASE_URL`, `SAUSAGE_MODEL`, and `SAUSAGE_API_TOKEN` first. The `SAUSAGE_BASE_URL`, `SAUSAGE_MODEL`, and `SAUSAGE_API_TOKEN` first. The
values go to `world/actor/agent/data/config.json`. Both commands take another values go to `world/config/agent.json`. Both commands take another folder as
folder as an argument, for example `... -- run my-world`. an argument, for example `... -- run my-world`.
To keep a world on the newest version, run it in a loop: To keep a world on the newest version, run it in a loop:
@@ -39,60 +39,61 @@ nix run git+https://gitea.surma.technology/surma/sausage#loop # ./
nix run git+https://gitea.surma.technology/surma/sausage#loop -- my-world # an existing world nix run git+https://gitea.surma.technology/surma/sausage#loop -- my-world # an existing world
``` ```
The loop runs `nix run --refresh git+https://gitea.surma.technology/surma/sausage -- run my-world`. When the host crashes or a plugin calls `core.exit`, the loop fetches the
When the host crashes or an actor sends `@broker {"type":"exit"}`, the loop newest commit and starts it again. `quit` ends the loop. Set `SAUSAGE_FLAKE`
fetches the newest commit and starts it again. `/quit` ends the loop. Set to run another flake URL instead, for example your fork.
`SAUSAGE_FLAKE` to run another flake URL instead, for example your fork.
From a clone (`git clone https://gitea.surma.technology/surma/sausage`), the From a clone (`git clone https://gitea.surma.technology/surma/sausage`), the
same commands are `nix run`, `nix run -- init`, `nix run -- run` and same commands are `nix run`, `nix run -- init`, `nix run -- run`, and
`nix run .#loop`. `nix run .#loop`.
### Headless ### Headless
`--headless` (or `SAUSAGE_HEADLESS=1`) turns the console input off: the host `--headless` (or `SAUSAGE_HEADLESS=1`) turns the console input off. The host
never reads stdin, so end of input does not stop the world. It runs until runs until SIGTERM or SIGINT (exit code 0) or until a plugin calls
SIGTERM/SIGINT (clean stop, exit code 0) or a broker `exit` request (exit `core.exit` (exit code 75, so the loop starts the newest version):
code 75, so the loop starts the newest version). Create the world first
(`init`), then pass the flag through the loop:
```sh ```sh
nix run git+https://gitea.surma.technology/surma/sausage#loop -- --headless my-world nix run git+https://gitea.surma.technology/surma/sausage#loop -- --headless my-world
``` ```
In the console: ## The console
```text | Input | Effect |
write an actor "notes" that stores notes, and start it → text to the agent | --- | --- |
@notes {"type":"add","text":"milk"} → a JSON message to an actor | text | a message to the agent (`agent.chat`) |
@agent hello → text to an actor | `<plugin>.<method> {json}` | a call. The console prints the result. |
/quit | `ls [path]`, `cat <path>` | the filesystem as a process sees it, with `/proc` and `/run` |
``` | `methods <plugin>` | the methods of a plugin, with params and docs |
| `ps` | the running processes |
| `log [plugin or pid] [-n N]` | the journal |
| `activate <plugin> <wasm>`, `revert <plugin>` | activate a new version, or go back |
| `stop <pid>`, `kill <pid>` | stop a process |
| `help`, `quit` | |
## The world folder ## The world folder
The folder is the source of truth. Every actor lives in `world/actor/<id>/`:
```text ```text
actor.wasm the current version plugins/<name>/plugin.wasm the current version of a plugin
actor.json installed version, running, subscriptions plugins/<name>/source/ its source
source/ the source of the current version state/<plugin>/ the state of a plugin, by convention
public/ files that the actor publishes state/core/journal.sqlite the journal: stderr of all processes, and core events
data/ actor state; owner-writable, world-readable via /actor/<id>/data/ (not private) config/boot.json the services that start with the world
stderr.log what the actor and the broker write about it config/agent.json the configuration of the agent
versions/<n>/ every installed version: actor.wasm, source/, meta.json
``` ```
To add or replace an actor by hand, put an `actor.wasm` (and its `source/`, Each process also sees `/proc` (the processes and the state of the core),
with the tests in `source/tests/`) into its folder; the host watches the `/self` (its own folder in `/proc`), and `/run` (shared files in memory).
world folder and picks the file up once it stops changing. The broker tests
the file before it uses it. To replace a plugin by hand, copy a `.wasm` file to its `plugin.wasm`. That
If the tests fail, the old version keeps running, and `stderr.log` says why. skips the interface checks. `revert <plugin>` goes back to the version
before the last activation.
## More ## More
- [PROTOCOL.md](PROTOCOL.md): messages, the filesystem, the broker, `cargo`. - [PROTOCOL.md](PROTOCOL.md): calls, processes, the filesystem, locks,
Read this to write an actor in any language. interfaces, and `cargo`. Read this to write a plugin in any language.
- [`sdk/`](sdk/): `sausage-actor`, the SDK for Rust actors. - [DECISIONS.md](DECISIONS.md): why Sausage is the way it is.
- [`examples/notes`](examples/notes): a small actor with a test. - [`sdk/`](sdk/): `sausage-plugin`, the SDK for Rust plugins.
- [`examples/`](examples/): small plugins.
- [AGENTS.md](AGENTS.md): how to work on Sausage itself. - [AGENTS.md](AGENTS.md): how to work on Sausage itself.