PROPOSAL.md: results of the review, no mention of the old model
ci / check (push) Successful in 2m33s
ci / check (push) Successful in 2m33s
Upgrades retire old processes after a grace period, then call rpc.retired. Services live in boot.json. One preopen / with a mount table in the sausage-vfs crate, with guards for in-flight file operations. Draft 7 and our own compatibility check on a fixed subset. Imports register with inventory. Rules for liveness, stop states, routing, locks, activation, endpoints (rpc.endpoint handshake), boot, builds, and a 10 ms scheduling tick.
This commit is contained in:
+346
-168
@@ -4,17 +4,10 @@ Status: a proposal on the branch `plugins`. When we accept it, its content
|
||||
goes into `PROTOCOL.md` (the contract) and `DECISIONS.md` (the reasons).
|
||||
Then this file goes away.
|
||||
|
||||
## 1. Why
|
||||
## 1. The idea
|
||||
|
||||
Sausage was a world of actors: long-running programs that send each other
|
||||
fire-and-forget messages through a broker. Agents found this model hard.
|
||||
|
||||
- A reply was a new message. The agent had to match `reply_to` values, wait
|
||||
with `wait_seconds`, and sort "other messages".
|
||||
- Mailboxes, subscriptions, taps, watches, versions, and restarts were all
|
||||
part of the contract.
|
||||
|
||||
This proposal replaces the model with three ideas that models already know:
|
||||
Sausage is a world of small WASI programs that an agent grows by talking to
|
||||
it. The design uses three ideas that models already know:
|
||||
|
||||
1. **JSON-RPC calls.** A call has a result, like an LLM tool call. LSP and
|
||||
MCP use the same protocol.
|
||||
@@ -31,10 +24,11 @@ This proposal replaces the model with three ideas that models already know:
|
||||
| world | The folder that a `sausage` host runs. It is `/` for every process. |
|
||||
| core | The host. It runs processes, routes calls, serves the virtual files, and keeps the journal. |
|
||||
| plugin | A folder `/plugins/<name>/` with a `plugin.wasm`. A plugin does not run by itself. |
|
||||
| process | One run of a plugin. Its ID is `<plugin>-<uuidv7>`, for example `notes-0197f3a2-7c1e-7d2b-9a41-5f0c2e8b1d34`. |
|
||||
| process | One run of a plugin version. Its ID is `<plugin>-<uuidv7>`, for example `notes-0197f3a2-7c1e-7d2b-9a41-5f0c2e8b1d34`. |
|
||||
| call | A JSON-RPC request. `notes.add` calls the method `add` of the plugin `notes`. |
|
||||
| interface | A set of methods. A plugin has one main interface. It can also declare endpoint interfaces. |
|
||||
| endpoint | An interface that a running process serves on a TCP port. An endpoint file tells others where it is. |
|
||||
| service | A call in `/config/boot.json`. The core starts it at boot and again after an upgrade of its plugin. |
|
||||
| import | An interface of another plugin that a plugin uses. The SDK generates it from that plugin's description. |
|
||||
| provider, consumer | The plugin that has an interface, and a plugin that imports it. |
|
||||
| journal | The central log of the world. |
|
||||
@@ -51,23 +45,29 @@ This proposal replaces the model with three ideas that models already know:
|
||||
state/<plugin>/ persistent state of a plugin, by convention
|
||||
state/core/journal.sqlite the journal
|
||||
config/ global configuration
|
||||
boot.json calls that the core makes at start
|
||||
boot.json the services
|
||||
agent.json API, base URL, model, and token of the agent
|
||||
```
|
||||
|
||||
Every process can read and write all of it. There are no permissions.
|
||||
- Every process can read and write all of it. There are no permissions.
|
||||
- The world folder must not contain `proc`, `run`, or `self`. These names are
|
||||
mount points (section 5). The host does not start a world that has them.
|
||||
|
||||
## 4. Calls
|
||||
## 4. Calls and processes
|
||||
|
||||
### 4.1 The protocol
|
||||
|
||||
- JSON-RPC 2.0 on stdin and stdout, one message per line, as in MCP's stdio
|
||||
transport.
|
||||
- `params` is always an object. Batches are not supported.
|
||||
- A line on stdout that is not a JSON-RPC message goes to the journal. The
|
||||
core continues at the next line. The SDK writes a newline before each
|
||||
message, so a `print!` without a newline cannot damage a message.
|
||||
- A line is a message only if it is a JSON object with `"jsonrpc": "2.0"`.
|
||||
- The core discards every other line on stdout. For the first discarded line
|
||||
of a process, it writes one warning to the journal, without the content of
|
||||
the line. Logs go to stderr.
|
||||
- Each line on stderr goes to the journal.
|
||||
- `params` is always an object. Batches are not supported.
|
||||
- The SDK writes each message with one blocking write through std's stdout,
|
||||
and it writes a newline before each message. So a `print!` without a
|
||||
newline cannot damage a message.
|
||||
|
||||
### 4.2 Method names
|
||||
|
||||
@@ -76,25 +76,38 @@ Every process can read and write all of it. There are no permissions.
|
||||
- Plugin names use `a-z`, `0-9`, `-`, and `_`. The name `core` is reserved.
|
||||
- `notes.add` starts a new process of `notes` and sends it the request `add`.
|
||||
`notes.rpc.discover` sends it `rpc.discover`.
|
||||
- `core.<method>` is a method of the core (section 9).
|
||||
- `core.<method>` is a method of the core (section 10).
|
||||
- Method names that start with `rpc.` are reserved: `rpc.discover`,
|
||||
`rpc.retired`, and `rpc.endpoint`.
|
||||
|
||||
### 4.3 What a call does
|
||||
|
||||
1. The core gives the call a UUIDv7 and starts the process `<plugin>-<uuid>`.
|
||||
1. The core gives the call a UUIDv7 and starts the process `<plugin>-<uuid>`
|
||||
with the current version of the plugin.
|
||||
2. The core writes the request to the stdin of the process. The request ID is
|
||||
the process ID.
|
||||
3. The process writes the response to stdout. The core sends the response to
|
||||
the caller, with the caller's request ID.
|
||||
4. If the process ends before it answers, the caller gets the error -32002.
|
||||
Its `data` has the exit status and the last journal lines of the process.
|
||||
Before the core decides this, it reads the stdout of the process to the
|
||||
end.
|
||||
|
||||
A process can write calls to stdout at any time, to the core and to plugins.
|
||||
The responses come on stdin. A process can have many calls in flight.
|
||||
- A process can write calls to stdout at any time, to the core and to
|
||||
plugins. The responses come on stdin. A process can have many calls in
|
||||
flight.
|
||||
- The core keys each pending request by the process and the exact JSON ID.
|
||||
The core drops a response with an unknown or late ID. The SDK writes a note
|
||||
about such a response to stderr.
|
||||
- A notification to `plugin.method` (without an ID) starts an attached
|
||||
process. That process gets the notification without an ID, and nobody gets
|
||||
a result.
|
||||
|
||||
### 4.4 Detached calls
|
||||
|
||||
`core.call {"method": "www.start", "params": {...}, "detach": true}` makes the
|
||||
call and returns its result. The new process does not end with its caller.
|
||||
call and returns its result to the caller. The new process has no parent, so
|
||||
it does not end with its caller.
|
||||
|
||||
Sent as a notification (without an ID), the same `core.call` starts the
|
||||
process and forgets it.
|
||||
@@ -107,25 +120,65 @@ caller as `core.notify {"call": <ID of the caller's request>, "name": "text",
|
||||
"params": {...}}`. After the call has ended, the core drops such
|
||||
notifications.
|
||||
|
||||
So a process never needs to know who called it.
|
||||
So a process never needs to know who called it. Endpoint calls send their
|
||||
notifications over the endpoint connection (section 6).
|
||||
|
||||
### 4.6 Lifetime
|
||||
|
||||
- A process ends when `main` returns. The core keeps its stdin open until
|
||||
then.
|
||||
- EOF on stdin means "stop now".
|
||||
- `core.stop {pid}` closes stdin, waits 5 seconds, and then kills the
|
||||
process. `core.kill {pid}` kills it at once.
|
||||
- When a process ends, the core stops each of its attached children with
|
||||
`core.stop`.
|
||||
- A detached process has no parent. Only `core.stop`, `core.kill`, and the
|
||||
end of the host stop it.
|
||||
- At host shutdown, the core stops every process.
|
||||
A process lives while one of these runs:
|
||||
|
||||
- a handler of a call,
|
||||
- a background task,
|
||||
- an endpoint that listens.
|
||||
|
||||
The SDK's own reader of stdin does not count. When nothing of this runs, the
|
||||
SDK returns from `main`, and the process ends.
|
||||
|
||||
A process has three states:
|
||||
|
||||
1. **running**
|
||||
2. **stopping**: the core has closed stdin. EOF on stdin means "stop now".
|
||||
- New calls of the process fail at once, and its pending calls fail.
|
||||
- A stop hook can still write files and send notifications. It cannot
|
||||
make calls, because no response can arrive.
|
||||
- After 5 seconds, the core kills the process.
|
||||
3. **ended**: the core releases the locks of the process (section 7) and
|
||||
removes its folder in `/proc`.
|
||||
|
||||
Rules:
|
||||
|
||||
- `core.stop {pid}` starts the stopping state. `core.kill {pid}` kills the
|
||||
process at once.
|
||||
- When a process ends, the core stops each of its attached children. All
|
||||
children stop in parallel, so a whole tree takes 5 seconds at most.
|
||||
- A detached process has no parent. Only `core.stop`, `core.kill`, an
|
||||
upgrade (section 4.7), and the end of the host stop it.
|
||||
- At host shutdown, the core stops every process in parallel.
|
||||
|
||||
Note: on Unix, children outlive their parent. Here, an attached child ends
|
||||
with its parent. Only `detach` makes a process independent.
|
||||
|
||||
### 4.7 Environment and network
|
||||
### 4.7 Upgrades
|
||||
|
||||
A process is an instance of one plugin version. When a new version becomes
|
||||
active (section 11.3):
|
||||
|
||||
1. New calls get the new version at once.
|
||||
2. Each process of an older version gets a grace period to finish its work.
|
||||
The default is 60 seconds. A plugin can declare its own value in
|
||||
`rpc.discover` (`x-sausage-grace`). Then the core stops the process.
|
||||
3. When the old process of a service has ended, the core starts the service
|
||||
again with the new version.
|
||||
4. When the last process of an older version has ended, the core starts a
|
||||
process of the new version and calls `rpc.retired {"previous":
|
||||
"<sha256 of the older file>"}`. A plugin can use this hook to migrate its
|
||||
state, or to write a marker that its other new processes wait for. The
|
||||
result goes to the journal.
|
||||
|
||||
During the grace period, an old process can call a method that a new
|
||||
provider no longer has. The time is limited, so this risk is acceptable.
|
||||
|
||||
### 4.8 Environment and network
|
||||
|
||||
- `SAUSAGE_PID`: the process ID.
|
||||
- `SAUSAGE_PLUGIN`: the plugin name.
|
||||
@@ -134,7 +187,18 @@ with its parent. Only `detach` makes a process independent.
|
||||
Every process gets the network: outgoing HTTP and HTTPS through `wasi:http`,
|
||||
and TCP, UDP, and DNS through `wasi:sockets`.
|
||||
|
||||
### 4.8 Errors
|
||||
### 4.9 Scheduling
|
||||
|
||||
- Each process is a task on the core's multi-thread tokio runtime, with work
|
||||
stealing.
|
||||
- A scheduler thread advances Wasmtime's epoch every 10 milliseconds. At
|
||||
each tick, a process that computes yields to the runtime. So a busy process
|
||||
cannot block the core or the other processes.
|
||||
- `core.kill` aborts the task. It takes effect at the next tick at the latest.
|
||||
- Most processes wait for IO most of the time. A better scheduler can come
|
||||
later.
|
||||
|
||||
### 4.10 Errors
|
||||
|
||||
The standard JSON-RPC codes apply. The core adds these codes:
|
||||
|
||||
@@ -146,14 +210,18 @@ The standard JSON-RPC codes apply. The core adds these codes:
|
||||
|
||||
## 5. The filesystem
|
||||
|
||||
Each process gets four preopens:
|
||||
Each process gets one preopen, `/`. The VFS resolves each path through a
|
||||
mount table:
|
||||
|
||||
| Path | Backend | Access |
|
||||
| Path | Layer | Access |
|
||||
| --- | --- | --- |
|
||||
| `/` | the world folder | read-write |
|
||||
| `/proc` | virtual, served by the core | read-only |
|
||||
| `/self` | virtual: the folder `/proc/<own pid>/` | read-write |
|
||||
| `/run` | virtual, shared, in memory, empty at each host start | read-write |
|
||||
| `/` | host: the world folder | read-write |
|
||||
| `/proc` | generated by the core | read-only |
|
||||
| `/self` | memory: the folder `/proc/<own pid>/` | read-write |
|
||||
| `/run` | memory, shared, empty at each host start | read-write |
|
||||
|
||||
A listing of `/` shows `proc`, `run`, and `self`. A path from any folder
|
||||
descriptor goes through the same mount table.
|
||||
|
||||
```text
|
||||
/proc/
|
||||
@@ -166,8 +234,8 @@ Each process gets four preopens:
|
||||
discover.json the rpc.discover document of the current plugin.wasm
|
||||
client.rs the Rust code that import!("<name>") generates
|
||||
<pid>/
|
||||
info.json plugin, method, params, parent, detached, start time, state
|
||||
log the journal lines of this process, live
|
||||
info.json plugin, version, method, params, parent, detached, start time, state
|
||||
log the journal lines of this process
|
||||
<files> the files that the process writes to /self
|
||||
```
|
||||
|
||||
@@ -177,6 +245,8 @@ Rules:
|
||||
`/proc` always fail.
|
||||
- In a process folder, the names `info.json`, `log`, and `fs` belong to the
|
||||
core. A write to `/self/info.json` fails.
|
||||
- A read of a generated file returns a snapshot that ends. For example,
|
||||
`/proc/<pid>/log` returns the lines up to the time of the open.
|
||||
- The folder of a process goes away when the process ends. Its files are in
|
||||
memory. Large or persistent data belongs in `/state`.
|
||||
- Do not send a `/self` path to another process. There it means a different
|
||||
@@ -185,27 +255,67 @@ Rules:
|
||||
Principle: **inspect through files, act through calls.** A file read is
|
||||
cheaper than a call, and every language and tool can read a file.
|
||||
|
||||
### 5.1 The VFS crate
|
||||
|
||||
The VFS is a crate of its own, `sausage-vfs`, so that it can become public
|
||||
later. It knows nothing about Sausage.
|
||||
|
||||
- One trait with the operations of a filesystem: stat, list, open, read and
|
||||
write at an offset, set size, create a folder, remove, and rename.
|
||||
- The layers:
|
||||
- **host:** a real folder, sandboxed with `cap-std`. Each file operation
|
||||
runs in a blocking task. That task holds a guard of its process until
|
||||
the system call has returned, even if the process was killed.
|
||||
- **memory:** files and folders in memory.
|
||||
- **generated:** files that a function makes at each open.
|
||||
- **mounts:** sends each path to the right layer. Each process has its own
|
||||
mount table, which shares most layers with the other processes.
|
||||
- An adapter implements `wasi:filesystem` for Wasmtime on top of the trait.
|
||||
It replaces the filesystem of `wasmtime-wasi` completely.
|
||||
- Not now: overlays (a union of two layers). Deletes and renames across
|
||||
layers need whiteouts and copy-up. The trait leaves room for overlays.
|
||||
- Tests:
|
||||
1. Unit tests for each layer.
|
||||
2. Differential tests: `proptest` generates sequences of operations. They
|
||||
run on the host layer and on `std::fs` in a temporary folder, and the
|
||||
results must be equal.
|
||||
3. A model test for the memory layer.
|
||||
4. WASI tests: Rust guests use `std::fs` through the adapter. `rusqlite`
|
||||
and `tempfile` must work.
|
||||
|
||||
## 6. Endpoints
|
||||
|
||||
- `listen(path, interface)` binds a free port on 127.0.0.1 and writes an
|
||||
endpoint file at `path`. The file has one line: the address, the owner, and
|
||||
the interface. Example: `tcp:127.0.0.1:41234 www-0197… control`.
|
||||
- Usually the path is in `/self`. A singleton uses a fixed path in `/run`,
|
||||
for example `/run/secrets`.
|
||||
- `connect(path)` reads the file. It checks that `/proc/<owner>` exists and
|
||||
that the interface is the expected one. Then it connects. So a stale file
|
||||
- `listen(path, interface)` binds a free port on 127.0.0.1. When the listener
|
||||
is ready, it writes an endpoint file at `path`, in one atomic step. The
|
||||
file has one line: the address, the owner, and the interface. Example:
|
||||
`tcp:127.0.0.1:41234 www-0197… control`.
|
||||
- Usually the path is in `/self`. A service that others must find uses a
|
||||
fixed path in `/run`, for example `/run/secrets`.
|
||||
- `connect(path)` reads the file and connects. Its first request is the
|
||||
built-in `rpc.endpoint`, which returns the owner, the plugin, and the
|
||||
interface. If they do not match the file, `connect` fails. So a stale file
|
||||
or a reused port cannot connect a client to the wrong process.
|
||||
- JSON-RPC runs on the connection, one message per line.
|
||||
- `listen` accepts only interfaces that the plugin declares (section 10.1).
|
||||
- If the file does not exist yet, `connect` waits a short time, for example
|
||||
while the services start.
|
||||
- JSON-RPC runs on the connection, one message per line. In an endpoint
|
||||
handler, `ctx.notify` sends the notification over the connection.
|
||||
- `listen` accepts only interfaces that the plugin declares (section 11.1).
|
||||
|
||||
## 7. Locks
|
||||
|
||||
- `core.lock {"path": "/state/notes/notes.json"}` returns when the process
|
||||
holds the lock. Processes that wait get the lock in FIFO order.
|
||||
- With `"wait": false`, it returns at once with `{"held": true}` or
|
||||
- `core.lock {"path": "/state/notes/notes.json"}` returns when the caller
|
||||
holds the lock.
|
||||
- Each acquisition gets its own place in a FIFO queue. The same process
|
||||
cannot take one lock twice: its second acquisition waits behind its first.
|
||||
- With `"wait": false`, the call returns at once with `{"held": true}` or
|
||||
`{"held": false}`.
|
||||
- `core.unlock {path}` releases the lock. When a process ends, the core
|
||||
releases all of its locks, also after a crash.
|
||||
- `core.unlock {path}` releases the lock. The SDK sends it as a notification
|
||||
when a guard drops. If the future of an acquisition was dropped, the SDK
|
||||
releases the lock when the grant arrives.
|
||||
- When a process ends, the core releases all of its locks, also after a
|
||||
crash or a kill. It waits until the last file operation of the process has
|
||||
returned (section 5.1). So a write of a killed process cannot land after
|
||||
the next process got the lock.
|
||||
- Locks are advisory. The core does not block writes to the file: the path
|
||||
is only a name.
|
||||
- SQLite has no file locks in WASI. Hold the lock while the database is open.
|
||||
@@ -214,14 +324,34 @@ cheaper than a call, and every language and tool can read a file.
|
||||
|
||||
- An SQLite file at `/state/core/journal.sqlite`. Only the core writes it.
|
||||
- Each entry has a UUIDv7 (it sorts by time), the process ID, the plugin, the
|
||||
stream (`stderr`, `stdout`, or `core`), and the line.
|
||||
stream (`stderr` or `core`), and the line.
|
||||
- The core writes its own events too: start (with method and params), exit
|
||||
status, stop, and kill.
|
||||
status, stop, kill, upgrades, and the warning about discarded stdout.
|
||||
- `core.log {plugin?, pid?, since?, until?, grep?, limit?}` queries the
|
||||
journal. `/proc/<pid>/log` shows the lines of one running process.
|
||||
- Later: structured logging, like `tracing`.
|
||||
|
||||
## 9. The core API
|
||||
## 9. Boot and services
|
||||
|
||||
At start, the core starts each service in `/config/boot.json`, in order:
|
||||
|
||||
```json
|
||||
[
|
||||
{"method": "secrets.serve", "params": {}},
|
||||
{"method": "www.start", "params": {"port": 8080}}
|
||||
]
|
||||
```
|
||||
|
||||
- A service is a detached call. The core waits for its result before it
|
||||
starts the next one. The result shows that the service is ready.
|
||||
- If a service fails, the core writes the error to the journal and continues.
|
||||
- After an upgrade of the plugin of a service, the core starts the service
|
||||
again with the new version (section 4.7).
|
||||
- The core does not restart other detached processes.
|
||||
- A singleton is a service with a fixed endpoint in `/run`. Only the core
|
||||
starts it, so two instances cannot start at the same time.
|
||||
|
||||
## 10. The core API
|
||||
|
||||
| Method | Params | Result |
|
||||
| --- | --- | --- |
|
||||
@@ -230,7 +360,7 @@ cheaper than a call, and every language and tool can read a file.
|
||||
| `core.stop` | `pid` | `{}` when the process has ended |
|
||||
| `core.kill` | `pid` | `{}` |
|
||||
| `core.lock` | `path`, `wait`? | `{"held": bool}` |
|
||||
| `core.unlock` | `path` | `{}` |
|
||||
| `core.unlock` | `path` | none (a notification) |
|
||||
| `core.activate` | `plugin` and `wasm`, or `plugins: [{plugin, wasm}]`. `force`? | the new `rpc.discover` documents |
|
||||
| `core.revert` | `plugin`, `force`? | like `core.activate` |
|
||||
| `core.log` | filters (section 8) | journal entries |
|
||||
@@ -238,18 +368,26 @@ cheaper than a call, and every language and tool can read a file.
|
||||
|
||||
Everything else is a file.
|
||||
|
||||
## 10. Interfaces
|
||||
## 11. Interfaces
|
||||
|
||||
### 10.1 `rpc.discover`
|
||||
### 11.1 `rpc.discover`
|
||||
|
||||
Each plugin answers `rpc.discover` with an OpenRPC document. The SDK
|
||||
generates it from the code, with all doc comments. It has two extension
|
||||
generates it from the code, with all doc comments. The schemas are JSON
|
||||
Schema Draft 7, as OpenRPC requires. The document has these extension
|
||||
fields:
|
||||
|
||||
- `x-sausage-params` for each method: a `$ref` to the schema of the params
|
||||
struct. So the name and the docs of the struct are not lost.
|
||||
- `x-sausage-endpoints`: the endpoint interfaces by name, each with its
|
||||
methods.
|
||||
- `x-sausage-imports`: the methods that the plugin uses from each provider,
|
||||
with their schemas.
|
||||
- `x-sausage-grace`: the grace period in seconds, if the plugin declares one.
|
||||
|
||||
The schema of params describes what the provider can deserialize. The schema
|
||||
of a result describes what the provider serializes. (Schemars generates both
|
||||
kinds. They differ, for example for fields with `#[serde(default)]`.)
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -262,14 +400,16 @@ fields:
|
||||
"params": [{"name": "port", "required": true,
|
||||
"description": "The TCP port to listen on, on all interfaces.",
|
||||
"schema": {"type": "integer", "minimum": 0, "maximum": 65535}}],
|
||||
"result": {"name": "pid", "schema": {"type": "string"}}
|
||||
"result": {"name": "pid", "schema": {"type": "string"}},
|
||||
"x-sausage-params": {"$ref": "#/components/schemas/StartParams"}
|
||||
}],
|
||||
"components": {"schemas": {"StartParams": {"…": "…"}}},
|
||||
"x-sausage-endpoints": {"control": {"methods": [{"name": "stats", "…": "…"}]}},
|
||||
"x-sausage-imports": {"notes": {"methods": [{"name": "add", "…": "…"}]}}
|
||||
}
|
||||
```
|
||||
|
||||
### 10.2 Compatibility
|
||||
### 11.2 Compatibility
|
||||
|
||||
A provider can add methods, optional params, and result fields. For each
|
||||
imported method:
|
||||
@@ -278,22 +418,29 @@ imported method:
|
||||
2. Each params value that the consumer can send is valid for the provider.
|
||||
3. Each result that the provider can return is valid for the consumer.
|
||||
|
||||
The same rules apply to endpoint interfaces. There are no version numbers.
|
||||
This check replaces them.
|
||||
- The same rules apply to endpoint interfaces.
|
||||
- There are no version numbers. This check replaces them.
|
||||
- The check is our own. It covers a fixed subset of JSON Schema: objects with
|
||||
`properties` and `required`, the primitive types, arrays, optional and
|
||||
nullable values, string enums, and `$ref`. `#[schema]` gives a compile
|
||||
error for a type outside this subset. So the check can decide every case.
|
||||
- The check covers calls, not state. Old and new processes share the files in
|
||||
`/state`, and `core.revert` restores only code. A plugin must keep its
|
||||
state format compatible, or migrate it in `rpc.retired`.
|
||||
|
||||
The check uses `jsoncompat` if it is good enough. Otherwise we write our own
|
||||
check for the subset of JSON Schema that `schemars` writes. An answer that
|
||||
the check cannot prove counts as a failure.
|
||||
|
||||
### 10.3 Activation
|
||||
### 11.3 Activation
|
||||
|
||||
`core.activate {"plugin": "notes", "wasm": "/state/cargo/builds/7/notes.wasm"}`:
|
||||
|
||||
1. The core starts the file and calls `rpc.discover`.
|
||||
2. It checks the imports of the new version against the current providers.
|
||||
3. It checks each current consumer of the plugin against the new version.
|
||||
4. If all checks pass, `plugin.wasm` becomes `plugin.prev.wasm`, and the
|
||||
file becomes `plugin.wasm`.
|
||||
1. The core copies the file. All further steps use this copy, even if the
|
||||
file changes.
|
||||
2. It starts the copy and calls `rpc.discover`.
|
||||
3. It waits until it has the documents of all plugins (section 11.4). A
|
||||
missing document never counts as "no consumers".
|
||||
4. It checks the imports of the new version against the current providers.
|
||||
5. It checks each current consumer of the plugin against the new version.
|
||||
6. If all checks pass, `plugin.wasm` becomes `plugin.prev.wasm`, and the copy
|
||||
becomes `plugin.wasm`. Then the upgrade starts (section 4.7).
|
||||
|
||||
If a check fails, nothing changes. The error lists each problem, with a
|
||||
counterexample where possible:
|
||||
@@ -306,17 +453,17 @@ notes was not activated: 1 consumer breaks.
|
||||
Update agent first, or activate both together.
|
||||
```
|
||||
|
||||
- The core runs one activation at a time.
|
||||
- `plugins: [...]` activates several plugins together. The checks then use
|
||||
the new set.
|
||||
- `force: true` skips steps 2 and 3.
|
||||
the new set. The core writes all files under temporary names first, and
|
||||
then renames them.
|
||||
- `force: true` skips steps 4 and 5.
|
||||
- `core.revert {plugin}` activates `plugin.prev.wasm` in the same way, and
|
||||
the current version becomes `plugin.prev.wasm`.
|
||||
- Running processes keep their version until they end. New calls get the new
|
||||
version.
|
||||
- A person can also copy a file to `plugin.wasm` by hand. That skips the
|
||||
checks (section 10.4).
|
||||
checks (section 11.4).
|
||||
|
||||
### 10.4 The graph
|
||||
### 11.4 The graph
|
||||
|
||||
- The core keeps the `rpc.discover` documents of all plugins in memory, by
|
||||
file hash. At start, it gets them in the background.
|
||||
@@ -326,15 +473,17 @@ Update agent first, or activate both together.
|
||||
- An import of a plugin that does not exist is "missing".
|
||||
- Untyped calls (`call("x.y", ...)`) are not in the graph.
|
||||
|
||||
## 11. Building and tests
|
||||
## 12. Building and tests
|
||||
|
||||
`cargo.build {"plugin": "notes"}` is a native method of the host. A call
|
||||
still gets a process ID, a folder in `/proc`, and journal entries, like other
|
||||
calls. The method does these steps:
|
||||
|
||||
1. It runs the tests of `/plugins/notes/source/` under WASI.
|
||||
2. It builds the release `.wasm` for `wasm32-wasip2`.
|
||||
3. It returns `{"wasm": "/state/cargo/builds/<n>/notes.wasm", "tests":
|
||||
1. It copies `/plugins/notes/source/` once. The tests and the build both use
|
||||
this copy.
|
||||
2. It runs the tests under WASI.
|
||||
3. It builds the release `.wasm` for `wasm32-wasip2`.
|
||||
4. It returns `{"wasm": "/state/cargo/builds/<n>/notes.wasm", "tests":
|
||||
{"passed", "failed", "output"}, "warnings"}`. If a step fails, it returns
|
||||
an error with the compiler or test output.
|
||||
|
||||
@@ -345,6 +494,8 @@ calls. The method does these steps:
|
||||
- The warnings include unused imported methods. Remove them from
|
||||
`imports/<plugin>.json`, so that fewer changes of the provider break the
|
||||
plugin.
|
||||
- A kill of the call stops the whole process group of cargo, with rustc, the
|
||||
build scripts, and the tests.
|
||||
|
||||
Tests:
|
||||
|
||||
@@ -356,40 +507,33 @@ Tests:
|
||||
and `--ignored`. Async tests use `#[sausage_plugin::test]`, like
|
||||
`#[tokio::test]`. `#[should_panic]` cannot work, because WASI cannot catch
|
||||
a panic.
|
||||
- Each test gets an empty temporary world as `/`, the package folder as its
|
||||
current folder, the variables that cargo sets, and its own `/proc`,
|
||||
`/self`, and `/run`.
|
||||
- Each test gets its own mount table: an empty temporary world as `/`, the
|
||||
package folder (so that fixture files work), the variables that cargo sets,
|
||||
and its own `/proc`, `/self`, and `/run`.
|
||||
- The runner answers `core.lock`, `core.unlock`, `core.notify`, and
|
||||
`core.log`. A call to a plugin fails with a clear error: a test has no
|
||||
other plugins.
|
||||
|
||||
## 12. Boot
|
||||
|
||||
At start, the core makes each call in `/config/boot.json`, detached:
|
||||
|
||||
```json
|
||||
[{"method": "www.start", "params": {"port": 8080}}]
|
||||
```
|
||||
|
||||
## 13. The SDK
|
||||
|
||||
Two crates: `sausage-plugin`, and `sausage-plugin-macros`, which
|
||||
`sausage-plugin` re-exports. The SDK also re-exports `serde` and `schemars`
|
||||
for the macros.
|
||||
`sausage-plugin` re-exports. A plugin depends only on `sausage-plugin`. The
|
||||
SDK also re-exports `serde` and `schemars` for the macros.
|
||||
|
||||
| Item | Purpose |
|
||||
| --- | --- |
|
||||
| `#[rpc]` | Makes an `impl` an interface. `#[rpc(endpoints(name = Type))]` declares endpoint interfaces. |
|
||||
| `#[schema]` | For each type that a method takes or returns: adds the serde and schemars derives. |
|
||||
| `#[rpc]` | Makes an `impl` an interface. On the main interface: `#[rpc(endpoints(name = Type), grace_seconds = N)]`. |
|
||||
| `#[on_retired]` | Marks the method that answers `rpc.retired`. |
|
||||
| `#[schema]` | For each type that a method takes or returns: adds the serde and schemars derives, and checks the subset of section 11.2. |
|
||||
| `run(T)` | In `main`: serves the main interface on stdin and stdout. |
|
||||
| `call(method, params)` | An untyped call. Returns a JSON value. |
|
||||
| `.detach()`, `.on_notify(f)` | Options of a call. |
|
||||
| `Ctx` | An optional second argument of a method: `ctx.notify(name, params)`. |
|
||||
| `Ctx` | An optional second argument of a method: `ctx.notify(name, params)`, `ctx.exit_after_response()`. |
|
||||
| `lock(path)` | A lock guard. `.try_now()` does not wait. `.keep()` holds the lock until the process ends. |
|
||||
| `listen(path, value)`, `connect(path)` | Endpoints. |
|
||||
| `background(future)` | Runs a task in this process. The process lives while a task runs. |
|
||||
| `import!("name")` | Generates `mod name` from `imports/name.json`. |
|
||||
| `state_dir()`, `write_atomic(path, bytes)`, `pid()`, `endpoint_owner(path)`, `exit_after_response()` | Helpers. |
|
||||
| `state_dir()`, `write_atomic(path, bytes)`, `pid()` | Helpers. |
|
||||
| `#[sausage_plugin::test]` | An async test. A plain `#[test]` also works. |
|
||||
|
||||
Rules that the compiler enforces:
|
||||
@@ -402,9 +546,18 @@ Rules that the compiler enforces:
|
||||
Other agents read these docs to use the interface. Write them like good
|
||||
Rust documentation.
|
||||
|
||||
Method docs become `summary` (the first paragraph) and `description` in
|
||||
OpenRPC. Type and field docs become `description` in the JSON Schema. The
|
||||
generated imports contain them again as doc comments.
|
||||
Other rules:
|
||||
|
||||
- Method docs become `summary` (the first paragraph) and `description` in
|
||||
OpenRPC. Type and field docs become `description` in the JSON Schema. The
|
||||
generated imports contain them again as doc comments.
|
||||
- Each `import!` registers its document in a static list with the
|
||||
`inventory` crate. `run` reads the list for `rpc.discover`. `import!` uses
|
||||
`include_str!`, so cargo rebuilds the plugin when the file changes.
|
||||
- The value that `run` receives must be cheap to create, because the core
|
||||
also starts a plugin only for `rpc.discover`. Open files and databases in
|
||||
the methods.
|
||||
- Log with `eprintln!`. stdout belongs to the protocol.
|
||||
|
||||
### 13.1 A plugin
|
||||
|
||||
@@ -506,13 +659,14 @@ mod tests {
|
||||
```rust
|
||||
//! A web server for the files in `/state/www/`.
|
||||
//!
|
||||
//! `start` starts the server in its own process and returns at once. Call it
|
||||
//! detached, or the server ends with its caller. The running server serves
|
||||
//! the endpoint `control` at `/proc/<pid>/control`.
|
||||
//! `start` starts the server in its own process and returns at once. Run it
|
||||
//! as a service, or call it detached, or the server ends with its caller.
|
||||
//! The running server serves the endpoint `control` at
|
||||
//! `/proc/<pid>/control`.
|
||||
|
||||
use std::{cell::Cell, rc::Rc};
|
||||
|
||||
use sausage_plugin::{Result, background, listen, pid, rpc, schema};
|
||||
use sausage_plugin::{Ctx, Result, background, listen, pid, rpc, schema};
|
||||
use wstd::net::TcpListener;
|
||||
|
||||
/// The www plugin.
|
||||
@@ -525,7 +679,7 @@ struct StartParams {
|
||||
port: u16,
|
||||
}
|
||||
|
||||
#[rpc(endpoints(control = Control))]
|
||||
#[rpc(endpoints(control = Control), grace_seconds = 5)]
|
||||
impl Www {
|
||||
/// Start the web server and return the ID of its process.
|
||||
///
|
||||
@@ -554,8 +708,8 @@ impl Control {
|
||||
}
|
||||
|
||||
/// Stop the server. The process ends after this response.
|
||||
fn stop(&self) -> Result<()> {
|
||||
sausage_plugin::exit_after_response();
|
||||
fn stop(&self, ctx: &Ctx) -> Result<()> {
|
||||
ctx.exit_after_response();
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
@@ -574,20 +728,37 @@ The same type can serve stdin and an endpoint. The example uses two types
|
||||
for a reason: `www.stats` through the core would start a fresh process, and
|
||||
that process would answer 0.
|
||||
|
||||
### 13.3 A singleton
|
||||
### 13.3 A service with a fixed endpoint
|
||||
|
||||
```rust
|
||||
/// Start the secrets service, or return the ID of the one that runs.
|
||||
async fn start(&self) -> Result<String> {
|
||||
let Some(lock) = lock("/run/secrets").try_now().await? else {
|
||||
return sausage_plugin::endpoint_owner("/run/secrets"); // another instance runs
|
||||
};
|
||||
lock.keep(); // hold the lock until this process ends
|
||||
listen("/run/secrets", Secrets::open()?).await?;
|
||||
Ok(pid())
|
||||
//! Secrets: keeps secrets in memory, for other plugins.
|
||||
//!
|
||||
//! Run it as a service: `{"method": "secrets.serve", "params": {}}` in
|
||||
//! `/config/boot.json`. Clients connect to the endpoint `vault` at
|
||||
//! `/run/secrets`.
|
||||
|
||||
use std::rc::Rc;
|
||||
|
||||
use sausage_plugin::{Result, listen, rpc};
|
||||
|
||||
/// The secrets plugin.
|
||||
struct Secrets;
|
||||
|
||||
#[rpc(endpoints(vault = Vault))]
|
||||
impl Secrets {
|
||||
/// Open the vault and serve it at `/run/secrets`.
|
||||
///
|
||||
/// Returns when the endpoint is ready. The process then lives while the
|
||||
/// endpoint listens.
|
||||
async fn serve(&self) -> Result<()> {
|
||||
listen("/run/secrets", Rc::new(Vault::open()?)).await
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
A client imports `secrets` and calls
|
||||
`secrets::vault::connect("/run/secrets").await?`.
|
||||
|
||||
### 13.4 Imports
|
||||
|
||||
To use another plugin, an agent does these steps:
|
||||
@@ -617,8 +788,8 @@ async fn example() -> Result<u64> {
|
||||
|
||||
- The params of each method are one struct. The fields have the same names as
|
||||
the JSON params.
|
||||
- Nested types take their names from `$defs`. When the provider uses
|
||||
`schemars`, those are the names of its Rust types: `AddParams` stays
|
||||
- Types take their names from the schemas. When the provider uses
|
||||
`#[schema]`, those are the names of its Rust types: `AddParams` stays
|
||||
`AddParams`.
|
||||
- The generated items are private to the crate, so rustc warns about each
|
||||
unused method.
|
||||
@@ -630,15 +801,19 @@ async fn example() -> Result<u64> {
|
||||
`text`.
|
||||
- The agent takes a lock on `/state/agent/history.jsonl`. So a second message
|
||||
waits until the first turn has ended.
|
||||
- The agent declares a long grace period (for example 10 minutes), so that a
|
||||
turn can finish after an upgrade. A new version applies at the next
|
||||
`chat`.
|
||||
- The tools: `call` (method, params, detach), `read_file`, `write_file`,
|
||||
`list_dir`, and `copy_dir`.
|
||||
- The configuration is in `/config/agent.json`.
|
||||
- The prompt stays short. It says where the contract is
|
||||
(`/proc/core/PROTOCOL.md`), where the SDK is (`/plugins/cargo/sdk/`), how
|
||||
to build and activate, and where the agent's own source is
|
||||
(`/plugins/agent/source/`).
|
||||
- A new version of the agent applies at the next `chat`. The current turn
|
||||
ends with the old code.
|
||||
- The prompt stays short. It says:
|
||||
- where the contract is (`/proc/core/PROTOCOL.md`),
|
||||
- where the SDK is (`/plugins/cargo/sdk/`),
|
||||
- how to build and activate a plugin,
|
||||
- where the agent's own source is (`/plugins/agent/source/`),
|
||||
- that logs go to stderr,
|
||||
- that the interface check does not cover state.
|
||||
|
||||
## 15. The console
|
||||
|
||||
@@ -663,75 +838,78 @@ async fn example() -> Result<u64> {
|
||||
|
||||
## 16. Caches
|
||||
|
||||
- The compiled code of each `.wasm` file is cached in
|
||||
`$XDG_CACHE_HOME/sausage/`, or `~/.cache/sausage/` without that variable.
|
||||
`--cache-dir` changes the folder.
|
||||
- The key is the SHA-256 of the file and Wasmtime's engine hash. So a
|
||||
Wasmtime upgrade never loads old machine code.
|
||||
- Wasmtime has a compile cache of its own. If it supports components, we use
|
||||
it. If not, our own cache needs about 20 lines.
|
||||
- Wasmtime's own compile cache stores the compiled code of each `.wasm` file.
|
||||
It supports components.
|
||||
- The folder is `$XDG_CACHE_HOME/sausage/`, or `~/.cache/sausage/` without
|
||||
that variable. `--cache-dir` changes the folder.
|
||||
|
||||
## 17. What goes away
|
||||
|
||||
The broker, mailboxes, publish and subscribe, taps, `watch` and
|
||||
`fs_changed`, actor-served files, `versions/`, the JSON transcript tests,
|
||||
restarts and the rollback after a crash loop, `/actor`, `/data`, `/public`,
|
||||
the `sausage-actor` SDK, and the automatic install of a changed
|
||||
`actor.wasm`. There is no compatibility with old worlds.
|
||||
|
||||
## 18. Decisions
|
||||
## 17. Decisions
|
||||
|
||||
- **JSON-RPC, not component interfaces.** Every language can read and write
|
||||
lines of JSON, and models know JSON-RPC from LSP and MCP.
|
||||
- **A process per call.** We get parallelism without threads. A new version
|
||||
applies at the next call, so the core needs no restarts and no rollback.
|
||||
- **A process per call.** We get parallelism without threads in WASI. A new
|
||||
version applies at the next call, and old processes retire after a grace
|
||||
period.
|
||||
- **Attached by default.** A process does not outlive its caller by
|
||||
accident.
|
||||
- **Services in one file.** Only the core starts and restarts them, so
|
||||
upgrades and singletons need no logic in plugins.
|
||||
- **Files to inspect, calls to act.** This keeps the core API small, and the
|
||||
agent's file tools are enough to explore the world.
|
||||
- **Generated imports and interface checks instead of versions.** A model
|
||||
- **Generated imports and interface checks, without version numbers.** A model
|
||||
that translates a description by hand makes errors. A generator does not.
|
||||
The check catches the most common error in a world that changes itself: a
|
||||
change to one plugin breaks another one.
|
||||
- **A small schema subset.** With Draft 7 and a fixed subset, our own check
|
||||
can decide every case.
|
||||
- **Docs are part of the interface.** Other agents work from them, so the
|
||||
compiler requires them.
|
||||
- **No permissions.** The checks protect against mistakes, not against
|
||||
plugins.
|
||||
- **Our own VFS.** It works on Linux and macOS, and plugin-served files need
|
||||
it later.
|
||||
plugins. Locks are advisory.
|
||||
- **Our own VFS, in its own crate.** It works on Linux and macOS, it makes
|
||||
`/proc` live, and plugin-served files need it later. Mounts now, overlays
|
||||
later.
|
||||
- **TCP endpoints for now.** They need no host code. VFS pipes can replace
|
||||
them later behind the same SDK API.
|
||||
- **Simple scheduling for now.** An epoch tick every 10 milliseconds keeps
|
||||
the core responsive and makes kills reliable.
|
||||
|
||||
## 19. Later
|
||||
## 18. Later
|
||||
|
||||
- Plugin-served files at `/proc/<pid>/fs/`, for example for an FTP plugin. A
|
||||
request gets a timeout of 10 seconds, because a process that reads its own
|
||||
served files would wait for itself forever.
|
||||
- Overlays in the VFS.
|
||||
- VFS pipes for endpoints, instead of TCP ports.
|
||||
- Subscriptions to file changes.
|
||||
- Structured logging.
|
||||
- A secrets service.
|
||||
- Cancellation of calls.
|
||||
- A better scheduler.
|
||||
|
||||
## 20. Plan
|
||||
## 19. Plan
|
||||
|
||||
The work happens on the branch `plugins`. Each step includes its tests.
|
||||
|
||||
First, three checks of risks:
|
||||
Already confirmed:
|
||||
|
||||
1. The round trip of `typify`, schemars 1.x, and `jsoncompat` on real
|
||||
documents. The result decides between `jsoncompat` and our own check.
|
||||
2. Virtual preopens in `wasmtime-wasi`: add `/proc`, `/self`, and `/run` to
|
||||
the preopen list, and send each descriptor under them to our backend.
|
||||
3. Whether Wasmtime's compile cache supports components.
|
||||
- `inventory` works on `wasm32-wasip2`.
|
||||
- Wasmtime's compile cache supports components.
|
||||
- C code builds for `wasm32-wasip2`.
|
||||
|
||||
Checks of risks first:
|
||||
|
||||
1. An end-to-end pair: a provider and a consumer with a generated import,
|
||||
through the real transport. This includes `typify` with Draft 7 from
|
||||
schemars, and our compatibility check on real documents.
|
||||
2. Fixture paths in tests under the runner.
|
||||
|
||||
Then the steps:
|
||||
|
||||
1. `PROTOCOL.md` and `DECISIONS.md` from this proposal.
|
||||
2. The core: a process per call, JSON-RPC routing, the process tree, locks,
|
||||
the journal, and the compile cache.
|
||||
3. The VFS: `/proc`, `/self`, `/run`, and `/proc/core/`.
|
||||
2. `sausage-vfs`: the trait, the layers, the Wasmtime adapter, and its tests.
|
||||
3. The core: a process per call, JSON-RPC routing, the process tree, upgrades,
|
||||
services, locks, the journal, scheduling, and the compile cache.
|
||||
4. The SDK and its macros.
|
||||
5. `cargo.build`, the test runner, activation, and the graph.
|
||||
6. The agent, the console, and boot.
|
||||
7. The removal of the old code.
|
||||
6. The agent and the console.
|
||||
|
||||
Reference in New Issue
Block a user