sausage-plugin: the crate docs are the manual for agents; remove PROPOSAL.md
Its content lives in PROTOCOL.md (the contract), DECISIONS.md (the reasons), and the SDK docs (how to write a plugin).
This commit is contained in:
-922
@@ -1,922 +0,0 @@
|
|||||||
# Proposal: plugins, processes, and JSON-RPC
|
|
||||||
|
|
||||||
Status: a proposal on the branch `plugins`. When we accept it, its content
|
|
||||||
goes into `PROTOCOL.md` (the contract) and `DECISIONS.md` (the reasons).
|
|
||||||
Then this file goes away.
|
|
||||||
|
|
||||||
## 1. The idea
|
|
||||||
|
|
||||||
Sausage is a world of small WASI programs that an agent grows by talking to
|
|
||||||
it. The design uses three ideas that models already know:
|
|
||||||
|
|
||||||
1. **JSON-RPC calls.** A call has a result, like an LLM tool call. LSP and
|
|
||||||
MCP use the same protocol.
|
|
||||||
2. **Processes.** Each call to a plugin starts a new WASI process. WASI has no
|
|
||||||
threads, so processes are the unit of parallelism. Calls are asynchronous,
|
|
||||||
so one process can make many calls at the same time.
|
|
||||||
3. **Files to inspect, calls to act.** The core shows its state as virtual
|
|
||||||
files in `/proc`, like Plan 9 and Linux. Each change of state is a call.
|
|
||||||
|
|
||||||
## 2. Terms
|
|
||||||
|
|
||||||
| Term | Meaning |
|
|
||||||
| --- | --- |
|
|
||||||
| world | The folder that a `sausage` host runs. It is `/` for every process. |
|
|
||||||
| core | The host. It runs processes, routes calls, serves the virtual files, and keeps the journal. |
|
|
||||||
| plugin | A folder `/plugins/<name>/` with a `plugin.wasm`. A plugin does not run by itself. |
|
|
||||||
| process | One run of a plugin version. Its ID is `<plugin>-<uuidv7>`, for example `notes-0197f3a2-7c1e-7d2b-9a41-5f0c2e8b1d34`. |
|
|
||||||
| call | A JSON-RPC request. `notes.add` calls the method `add` of the plugin `notes`. |
|
|
||||||
| interface | A set of methods. A plugin has one main interface. It can also declare endpoint interfaces. |
|
|
||||||
| endpoint | An interface that a running process serves on a TCP port. An endpoint file tells others where it is. |
|
|
||||||
| service | A call in `/config/boot.json`. The core starts it at boot and again after an upgrade of its plugin. |
|
|
||||||
| import | An interface of another plugin that a plugin uses. The SDK generates it from that plugin's description. |
|
|
||||||
| provider, consumer | The plugin that has an interface, and a plugin that imports it. |
|
|
||||||
| journal | The central log of the world. |
|
|
||||||
|
|
||||||
## 3. The world folder
|
|
||||||
|
|
||||||
```text
|
|
||||||
<world>/
|
|
||||||
plugins/<name>/
|
|
||||||
plugin.wasm the current version
|
|
||||||
plugin.prev.wasm the version before, for `core.revert`
|
|
||||||
source/ the source. `cargo.build` builds it from here.
|
|
||||||
imports/<plugin>.json descriptions of the imported plugins
|
|
||||||
state/<plugin>/ persistent state of a plugin, by convention
|
|
||||||
state/core/journal.sqlite the journal
|
|
||||||
config/ global configuration
|
|
||||||
boot.json the services
|
|
||||||
agent.json API, base URL, model, and token of the agent
|
|
||||||
```
|
|
||||||
|
|
||||||
- Every process can read and write all of it. There are no permissions.
|
|
||||||
- The world folder must not contain `proc`, `run`, or `self`. These names are
|
|
||||||
mount points (section 5). The host does not start a world that has them.
|
|
||||||
|
|
||||||
## 4. Calls and processes
|
|
||||||
|
|
||||||
### 4.1 The protocol
|
|
||||||
|
|
||||||
- JSON-RPC 2.0 on stdin and stdout, one message per line, as in MCP's stdio
|
|
||||||
transport.
|
|
||||||
- A line is a message only if it is a JSON object with `"jsonrpc": "2.0"`.
|
|
||||||
- The core discards every other line on stdout. For the first discarded line
|
|
||||||
of a process, it writes one warning to the journal, without the content of
|
|
||||||
the line. Logs go to stderr.
|
|
||||||
- Each line on stderr goes to the journal.
|
|
||||||
- `params` is always an object. Batches are not supported.
|
|
||||||
- The SDK writes each message with one blocking write through std's stdout,
|
|
||||||
and it writes a newline before each message. So a `print!` without a
|
|
||||||
newline cannot damage a message.
|
|
||||||
|
|
||||||
### 4.2 Method names
|
|
||||||
|
|
||||||
- A method name has the form `<target>.<method>`. The core splits it at the
|
|
||||||
first `.`.
|
|
||||||
- Plugin names use `a-z`, `0-9`, `-`, and `_`. The name `core` is reserved.
|
|
||||||
- `notes.add` starts a new process of `notes` and sends it the request `add`.
|
|
||||||
`notes.rpc.discover` sends it `rpc.discover`.
|
|
||||||
- `core.<method>` is a method of the core (section 10).
|
|
||||||
- Method names that start with `rpc.` are reserved: `rpc.discover`,
|
|
||||||
`rpc.retired`, and `rpc.endpoint`.
|
|
||||||
|
|
||||||
### 4.3 What a call does
|
|
||||||
|
|
||||||
1. The core gives the call a UUIDv7 and starts the process `<plugin>-<uuid>`
|
|
||||||
with the current version of the plugin.
|
|
||||||
2. The core writes the request to the stdin of the process. The request ID is
|
|
||||||
the process ID.
|
|
||||||
3. The process writes the response to stdout. The core sends the response to
|
|
||||||
the caller, with the caller's request ID.
|
|
||||||
4. If the process ends before it answers, the caller gets the error -32002.
|
|
||||||
Its `data` has the exit status and the last journal lines of the process.
|
|
||||||
Before the core decides this, it reads the stdout of the process to the
|
|
||||||
end.
|
|
||||||
|
|
||||||
- A process can write calls to stdout at any time, to the core and to
|
|
||||||
plugins. The responses come on stdin. A process can have many calls in
|
|
||||||
flight.
|
|
||||||
- The core keys each pending request by the process and the exact JSON ID.
|
|
||||||
The core drops a response with an unknown or late ID. The SDK writes a note
|
|
||||||
about such a response to stderr.
|
|
||||||
- A notification to `plugin.method` (without an ID) starts an attached
|
|
||||||
process. That process gets the notification without an ID, and nobody gets
|
|
||||||
a result.
|
|
||||||
|
|
||||||
### 4.4 Detached calls
|
|
||||||
|
|
||||||
`core.call {"method": "www.start", "params": {...}, "detach": true}` makes the
|
|
||||||
call and returns its result to the caller. The new process has no parent, so
|
|
||||||
it does not end with its caller.
|
|
||||||
|
|
||||||
Sent as a notification (without an ID), the same `core.call` starts the
|
|
||||||
process and forgets it.
|
|
||||||
|
|
||||||
### 4.5 Notifications to the caller
|
|
||||||
|
|
||||||
A process can tell its caller about progress. It sends the notification
|
|
||||||
`core.notify {"name": "text", "params": {...}}`. The core gives it to the
|
|
||||||
caller as `core.notify {"call": <ID of the caller's request>, "name": "text",
|
|
||||||
"params": {...}}`. After the call has ended, the core drops such
|
|
||||||
notifications.
|
|
||||||
|
|
||||||
So a process never needs to know who called it. Endpoint calls send their
|
|
||||||
notifications over the endpoint connection (section 6).
|
|
||||||
|
|
||||||
### 4.6 Lifetime
|
|
||||||
|
|
||||||
A process lives while one of these runs:
|
|
||||||
|
|
||||||
- a handler of a call,
|
|
||||||
- a background task,
|
|
||||||
- an endpoint that listens.
|
|
||||||
|
|
||||||
The SDK's own reader of stdin does not count. When nothing of this runs, the
|
|
||||||
SDK returns from `main`, and the process ends.
|
|
||||||
|
|
||||||
A process has three states:
|
|
||||||
|
|
||||||
1. **running**
|
|
||||||
2. **stopping**: the core has closed stdin. EOF on stdin means "stop now".
|
|
||||||
- New calls of the process fail at once, and its pending calls fail.
|
|
||||||
- A stop hook can still write files and send notifications. It cannot
|
|
||||||
make calls, because no response can arrive.
|
|
||||||
- After 5 seconds, the core kills the process.
|
|
||||||
3. **ended**: the core releases the locks of the process (section 7) and
|
|
||||||
removes its folder in `/proc`.
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- `core.stop {pid}` starts the stopping state. `core.kill {pid}` kills the
|
|
||||||
process at once.
|
|
||||||
- When a process ends, the core stops each of its attached children. All
|
|
||||||
children stop in parallel, so a whole tree takes 5 seconds at most.
|
|
||||||
- A detached process has no parent. Only `core.stop`, `core.kill`, an
|
|
||||||
upgrade (section 4.7), and the end of the host stop it.
|
|
||||||
- At host shutdown, the core stops every process in parallel.
|
|
||||||
|
|
||||||
Note: on Unix, children outlive their parent. Here, an attached child ends
|
|
||||||
with its parent. Only `detach` makes a process independent.
|
|
||||||
|
|
||||||
### 4.7 Upgrades
|
|
||||||
|
|
||||||
A process is an instance of one plugin version. When a new version becomes
|
|
||||||
active (section 11.3):
|
|
||||||
|
|
||||||
1. New calls get the new version at once.
|
|
||||||
2. Each process of an older version gets a grace period to finish its work.
|
|
||||||
The default is 60 seconds. A plugin can declare its own value in
|
|
||||||
`rpc.discover` (`x-sausage-grace`). Then the core stops the process.
|
|
||||||
3. When the old process of a service has ended, the core starts the service
|
|
||||||
again with the new version.
|
|
||||||
4. When the last process of an older version has ended, the core starts a
|
|
||||||
process of the new version and calls `rpc.retired {"previous":
|
|
||||||
"<sha256 of the older file>"}`. A plugin can use this hook to migrate its
|
|
||||||
state, or to write a marker that its other new processes wait for. The
|
|
||||||
result goes to the journal.
|
|
||||||
|
|
||||||
During the grace period, an old process can call a method that a new
|
|
||||||
provider no longer has. The time is limited, so this risk is acceptable.
|
|
||||||
|
|
||||||
### 4.8 Environment and network
|
|
||||||
|
|
||||||
- `SAUSAGE_PID`: the process ID.
|
|
||||||
- `SAUSAGE_PLUGIN`: the plugin name.
|
|
||||||
- `SAUSAGE_PARENT`: the parent: `console`, `core`, or a process ID.
|
|
||||||
|
|
||||||
Every process gets the network: outgoing HTTP and HTTPS through `wasi:http`,
|
|
||||||
and TCP, UDP, and DNS through `wasi:sockets`.
|
|
||||||
|
|
||||||
### 4.9 Scheduling
|
|
||||||
|
|
||||||
- Each process is a task on the core's multi-thread tokio runtime, with work
|
|
||||||
stealing.
|
|
||||||
- A scheduler thread advances Wasmtime's epoch every 10 milliseconds. At
|
|
||||||
each tick, a process that computes yields to the runtime. So a busy process
|
|
||||||
cannot block the core or the other processes.
|
|
||||||
- `core.kill` aborts the task. It takes effect at the next tick at the latest.
|
|
||||||
- Most processes wait for IO most of the time. A better scheduler can come
|
|
||||||
later.
|
|
||||||
|
|
||||||
### 4.10 Errors
|
|
||||||
|
|
||||||
The standard JSON-RPC codes apply. The core adds these codes:
|
|
||||||
|
|
||||||
| Code | Meaning |
|
|
||||||
| --- | --- |
|
|
||||||
| -32001 | There is no such plugin. |
|
|
||||||
| -32002 | The process ended without an answer. `data` has the exit status and the last journal lines. |
|
|
||||||
| -32003 | The activation was rejected. `data` has the reasons. |
|
|
||||||
|
|
||||||
## 5. The filesystem
|
|
||||||
|
|
||||||
Each process gets one preopen, `/`. The VFS resolves each path through a
|
|
||||||
mount table:
|
|
||||||
|
|
||||||
| Path | Layer | Access |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `/` | host: the world folder | read-write |
|
|
||||||
| `/proc` | generated by the core | read-only |
|
|
||||||
| `/self` | memory: the folder `/proc/<own pid>/` | read-write |
|
|
||||||
| `/run` | memory, shared, empty at each host start | read-write |
|
|
||||||
|
|
||||||
A listing of `/` shows `proc`, `run`, and `self`. A path from any folder
|
|
||||||
descriptor goes through the same mount table.
|
|
||||||
|
|
||||||
```text
|
|
||||||
/proc/
|
|
||||||
core/
|
|
||||||
PROTOCOL.md the contract
|
|
||||||
graph.json plugins, imports, and the result of each check
|
|
||||||
graph.dot the same graph for Graphviz
|
|
||||||
locks.json held locks and waiting processes
|
|
||||||
plugins/<name>/
|
|
||||||
discover.json the rpc.discover document of the current plugin.wasm
|
|
||||||
client.rs the Rust code that import!("<name>") generates
|
|
||||||
<pid>/
|
|
||||||
info.json plugin, version, method, params, parent, detached, start time, state
|
|
||||||
log the journal lines of this process
|
|
||||||
<files> the files that the process writes to /self
|
|
||||||
```
|
|
||||||
|
|
||||||
Rules:
|
|
||||||
|
|
||||||
- A process writes its own folder only through `/self`. Writes through
|
|
||||||
`/proc` always fail.
|
|
||||||
- In a process folder, the names `info.json`, `log`, and `fs` belong to the
|
|
||||||
core. A write to `/self/info.json` fails.
|
|
||||||
- A read of a generated file returns a snapshot that ends. For example,
|
|
||||||
`/proc/<pid>/log` returns the lines up to the time of the open.
|
|
||||||
- The folder of a process goes away when the process ends. Its files are in
|
|
||||||
memory. Large or persistent data belongs in `/state`.
|
|
||||||
- Do not send a `/self` path to another process. There it means a different
|
|
||||||
folder, like `localhost` on another machine.
|
|
||||||
|
|
||||||
Principle: **inspect through files, act through calls.** A file read is
|
|
||||||
cheaper than a call, and every language and tool can read a file.
|
|
||||||
|
|
||||||
### 5.1 The VFS crate
|
|
||||||
|
|
||||||
The VFS is a crate of its own, `sausage-vfs`, so that it can become public
|
|
||||||
later. It knows nothing about Sausage.
|
|
||||||
|
|
||||||
- One trait with the operations of a filesystem: stat, list, open, read and
|
|
||||||
write at an offset, set size, create a folder, remove, and rename.
|
|
||||||
- The layers:
|
|
||||||
- **host:** a real folder, sandboxed with `cap-std`. Each file operation
|
|
||||||
runs in a blocking task. That task holds a guard of its process until
|
|
||||||
the system call has returned, even if the process was killed.
|
|
||||||
- **memory:** files and folders in memory.
|
|
||||||
- **generated:** files that a function makes at each open.
|
|
||||||
- **mounts:** sends each path to the right layer. Each process has its own
|
|
||||||
mount table, which shares most layers with the other processes.
|
|
||||||
- An adapter implements `wasi:filesystem` for Wasmtime on top of the trait.
|
|
||||||
It replaces the filesystem of `wasmtime-wasi` completely.
|
|
||||||
- Not now: overlays (a union of two layers). Deletes and renames across
|
|
||||||
layers need whiteouts and copy-up. The trait leaves room for overlays.
|
|
||||||
- Tests:
|
|
||||||
1. Unit tests for each layer.
|
|
||||||
2. Differential tests: `proptest` generates sequences of operations. They
|
|
||||||
run on the host layer and on `std::fs` in a temporary folder, and the
|
|
||||||
results must be equal.
|
|
||||||
3. A model test for the memory layer.
|
|
||||||
4. WASI tests: Rust guests use `std::fs` through the adapter. `rusqlite`
|
|
||||||
and `tempfile` must work.
|
|
||||||
|
|
||||||
## 6. Endpoints
|
|
||||||
|
|
||||||
- `listen(path, interface)` binds a free port on 127.0.0.1. When the listener
|
|
||||||
is ready, it writes an endpoint file at `path`, in one atomic step. The
|
|
||||||
file has one line: the address, the owner, and the interface. Example:
|
|
||||||
`tcp:127.0.0.1:41234 www-0197… control`.
|
|
||||||
- Usually the path is in `/self`. A service that others must find uses a
|
|
||||||
fixed path in `/run`, for example `/run/secrets`.
|
|
||||||
- `connect(path)` reads the file and connects. Its first request is the
|
|
||||||
built-in `rpc.endpoint`, which returns the owner, the plugin, and the
|
|
||||||
interface. If they do not match the file, `connect` fails. So a stale file
|
|
||||||
or a reused port cannot connect a client to the wrong process.
|
|
||||||
- If the file does not exist yet, `connect` waits a short time, for example
|
|
||||||
while the services start.
|
|
||||||
- JSON-RPC runs on the connection, one message per line. In an endpoint
|
|
||||||
handler, `ctx.notify` sends the notification over the connection.
|
|
||||||
- `listen` accepts only interfaces that the plugin declares (section 11.1).
|
|
||||||
|
|
||||||
## 7. Locks
|
|
||||||
|
|
||||||
- `core.lock {"path": "/state/notes/notes.json"}` returns when the caller
|
|
||||||
holds the lock.
|
|
||||||
- Each acquisition gets its own place in a FIFO queue. The same process
|
|
||||||
cannot take one lock twice: its second acquisition waits behind its first.
|
|
||||||
- With `"wait": false`, the call returns at once with `{"held": true}` or
|
|
||||||
`{"held": false}`.
|
|
||||||
- `core.unlock {path}` releases the lock. The SDK sends it as a notification
|
|
||||||
when a guard drops. If the future of an acquisition was dropped, the SDK
|
|
||||||
releases the lock when the grant arrives.
|
|
||||||
- When a process ends, the core releases all of its locks, also after a
|
|
||||||
crash or a kill. It waits until the last file operation of the process has
|
|
||||||
returned (section 5.1). So a write of a killed process cannot land after
|
|
||||||
the next process got the lock.
|
|
||||||
- Locks are advisory. The core does not block writes to the file: the path
|
|
||||||
is only a name.
|
|
||||||
- SQLite has no file locks in WASI. Hold the lock while the database is open.
|
|
||||||
|
|
||||||
## 8. The journal
|
|
||||||
|
|
||||||
- An SQLite file at `/state/core/journal.sqlite`. Only the core writes it.
|
|
||||||
- Each entry has a UUIDv7 (it sorts by time), the process ID, the plugin, the
|
|
||||||
stream (`stderr` or `core`), and the line.
|
|
||||||
- The core writes its own events too: start (with method and params), exit
|
|
||||||
status, stop, kill, upgrades, and the warning about discarded stdout.
|
|
||||||
- `core.log {plugin?, pid?, since?, until?, grep?, limit?}` queries the
|
|
||||||
journal. `/proc/<pid>/log` shows the lines of one running process.
|
|
||||||
- Later: structured logging, like `tracing`.
|
|
||||||
|
|
||||||
## 9. Boot and services
|
|
||||||
|
|
||||||
At start, the core starts each service in `/config/boot.json`, in order:
|
|
||||||
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{"method": "secrets.serve", "params": {}},
|
|
||||||
{"method": "www.start", "params": {"port": 8080}}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
- A service is a detached call. The core waits for its result before it
|
|
||||||
starts the next one. The result shows that the service is ready.
|
|
||||||
- If a service fails, the core writes the error to the journal and continues.
|
|
||||||
- After an upgrade of the plugin of a service, the core starts the service
|
|
||||||
again with the new version (section 4.7).
|
|
||||||
- The core does not restart other detached processes.
|
|
||||||
- A singleton is a service with a fixed endpoint in `/run`. Only the core
|
|
||||||
starts it, so two instances cannot start at the same time.
|
|
||||||
|
|
||||||
## 10. The core API
|
|
||||||
|
|
||||||
| Method | Params | Result |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `core.call` | `method`, `params`, `detach`? | the result of the method |
|
|
||||||
| `core.notify` | `name`, `params` | none (a notification) |
|
|
||||||
| `core.stop` | `pid` | `{}` when the process has ended |
|
|
||||||
| `core.kill` | `pid` | `{}` |
|
|
||||||
| `core.lock` | `path`, `wait`? | `{"held": bool}` |
|
|
||||||
| `core.unlock` | `path` | none (a notification) |
|
|
||||||
| `core.activate` | `plugin` and `wasm`, or `plugins: [{plugin, wasm}]`. `force`? | the new `rpc.discover` documents |
|
|
||||||
| `core.revert` | `plugin`, `force`? | like `core.activate` |
|
|
||||||
| `core.log` | filters (section 8) | journal entries |
|
|
||||||
| `core.exit` | none | `{}`. Then the host stops with exit code 75. |
|
|
||||||
|
|
||||||
Everything else is a file.
|
|
||||||
|
|
||||||
## 11. Interfaces
|
|
||||||
|
|
||||||
### 11.1 `rpc.discover`
|
|
||||||
|
|
||||||
Each plugin answers `rpc.discover` with an OpenRPC document. The SDK
|
|
||||||
generates it from the code, with all doc comments. The schemas are JSON
|
|
||||||
Schema Draft 7, as OpenRPC requires. The document has these extension
|
|
||||||
fields:
|
|
||||||
|
|
||||||
- `x-sausage-params` for each method: a `$ref` to the schema of the params
|
|
||||||
struct. So the name and the docs of the struct are not lost.
|
|
||||||
- `x-sausage-endpoints`: the endpoint interfaces by name, each with its
|
|
||||||
methods.
|
|
||||||
- `x-sausage-imports`: the methods that the plugin uses from each provider,
|
|
||||||
with their schemas.
|
|
||||||
- `x-sausage-grace`: the grace period in seconds, if the plugin declares one.
|
|
||||||
|
|
||||||
The schema of params describes what the provider can deserialize. The schema
|
|
||||||
of a result describes what the provider serializes. (Schemars generates both
|
|
||||||
kinds. They differ, for example for fields with `#[serde(default)]`.)
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"openrpc": "1.2.6",
|
|
||||||
"info": {"title": "www", "version": "0.1.0"},
|
|
||||||
"methods": [{
|
|
||||||
"name": "start",
|
|
||||||
"summary": "Start the web server and return the ID of its process.",
|
|
||||||
"paramStructure": "by-name",
|
|
||||||
"params": [{"name": "port", "required": true,
|
|
||||||
"description": "The TCP port to listen on, on all interfaces.",
|
|
||||||
"schema": {"type": "integer", "minimum": 0, "maximum": 65535}}],
|
|
||||||
"result": {"name": "pid", "schema": {"type": "string"}},
|
|
||||||
"x-sausage-params": {"$ref": "#/components/schemas/StartParams"}
|
|
||||||
}],
|
|
||||||
"components": {"schemas": {"StartParams": {"…": "…"}}},
|
|
||||||
"x-sausage-endpoints": {"control": {"methods": [{"name": "stats", "…": "…"}]}},
|
|
||||||
"x-sausage-imports": {"notes": {"methods": [{"name": "add", "…": "…"}]}}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 11.2 Compatibility
|
|
||||||
|
|
||||||
A provider can add methods, optional params, and result fields. For each
|
|
||||||
imported method:
|
|
||||||
|
|
||||||
1. The provider has the method.
|
|
||||||
2. Each params value that the consumer can send is valid for the provider.
|
|
||||||
3. Each result that the provider can return is valid for the consumer.
|
|
||||||
|
|
||||||
- The same rules apply to endpoint interfaces.
|
|
||||||
- There are no version numbers. This check replaces them.
|
|
||||||
- The check is our own. It covers a fixed subset of JSON Schema: objects with
|
|
||||||
`properties` and `required`, the primitive types, arrays, optional and
|
|
||||||
nullable values, string enums, and `$ref`. `#[schema]` gives a compile
|
|
||||||
error for a type outside this subset. So the check can decide every case.
|
|
||||||
- The check covers calls, not state. Old and new processes share the files in
|
|
||||||
`/state`, and `core.revert` restores only code. A plugin must keep its
|
|
||||||
state format compatible, or migrate it in `rpc.retired`.
|
|
||||||
|
|
||||||
### 11.3 Activation
|
|
||||||
|
|
||||||
`core.activate {"plugin": "notes", "wasm": "/state/cargo/builds/7/notes.wasm"}`:
|
|
||||||
|
|
||||||
1. The core copies the file. All further steps use this copy, even if the
|
|
||||||
file changes.
|
|
||||||
2. It starts the copy and calls `rpc.discover`.
|
|
||||||
3. It waits until it has the documents of all plugins (section 11.4). A
|
|
||||||
missing document never counts as "no consumers".
|
|
||||||
4. It checks the imports of the new version against the current providers.
|
|
||||||
5. It checks each current consumer of the plugin against the new version.
|
|
||||||
6. If all checks pass, `plugin.wasm` becomes `plugin.prev.wasm`, and the copy
|
|
||||||
becomes `plugin.wasm`. Then the upgrade starts (section 4.7).
|
|
||||||
|
|
||||||
If a check fails, nothing changes. The error lists each problem, with a
|
|
||||||
counterexample where possible:
|
|
||||||
|
|
||||||
```text
|
|
||||||
notes was not activated: 1 consumer breaks.
|
|
||||||
- agent imports notes.add(text: string) -> Added.
|
|
||||||
The new notes.add has no param "text". It requires "body": string.
|
|
||||||
Counterexample: {"text": "milk"} is not valid for the new params.
|
|
||||||
Update agent first, or activate both together.
|
|
||||||
```
|
|
||||||
|
|
||||||
- The core runs one activation at a time.
|
|
||||||
- `plugins: [...]` activates several plugins together. The checks then use
|
|
||||||
the new set. The core writes all files under temporary names first, and
|
|
||||||
then renames them.
|
|
||||||
- `force: true` skips steps 4 and 5.
|
|
||||||
- `core.revert {plugin}` activates `plugin.prev.wasm` in the same way, and
|
|
||||||
the current version becomes `plugin.prev.wasm`.
|
|
||||||
- A person can also copy a file to `plugin.wasm` by hand. That skips the
|
|
||||||
checks (section 11.4).
|
|
||||||
|
|
||||||
### 11.4 The graph
|
|
||||||
|
|
||||||
- The core keeps the `rpc.discover` documents of all plugins in memory, by
|
|
||||||
file hash. At start, it gets them in the background.
|
|
||||||
- A file watcher on `/plugins/*/plugin.wasm` updates them, for example after
|
|
||||||
a copy by hand. Such a copy skips the checks. `graph.json` then shows each
|
|
||||||
broken import, and the journal records it.
|
|
||||||
- An import of a plugin that does not exist is "missing".
|
|
||||||
- Untyped calls (`call("x.y", ...)`) are not in the graph.
|
|
||||||
|
|
||||||
## 12. Building and tests
|
|
||||||
|
|
||||||
`cargo.build {"plugin": "notes"}` is a native method of the host. A call
|
|
||||||
still gets a process ID, a folder in `/proc`, and journal entries, like other
|
|
||||||
calls. The method does these steps:
|
|
||||||
|
|
||||||
1. It copies `/plugins/notes/source/` once. The tests and the build both use
|
|
||||||
this copy.
|
|
||||||
2. It runs the tests under WASI.
|
|
||||||
3. It builds the release `.wasm` for `wasm32-wasip2`.
|
|
||||||
4. It returns `{"wasm": "/state/cargo/builds/<n>/notes.wasm", "tests":
|
|
||||||
{"passed", "failed", "output"}, "warnings"}`. If a step fails, it returns
|
|
||||||
an error with the compiler or test output.
|
|
||||||
|
|
||||||
- The dependency `sausage-plugin = "0.1"` points to the SDK copy at
|
|
||||||
`/plugins/cargo/sdk/`.
|
|
||||||
- Build scripts can compile C: `CC_wasm32_wasip2` and `AR_wasm32_wasip2`
|
|
||||||
point to a WASI clang.
|
|
||||||
- The warnings include unused imported methods. Remove them from
|
|
||||||
`imports/<plugin>.json`, so that fewer changes of the provider break the
|
|
||||||
plugin.
|
|
||||||
- A kill of the call stops the whole process group of cargo, with rustc, the
|
|
||||||
build scripts, and the tests.
|
|
||||||
|
|
||||||
Tests:
|
|
||||||
|
|
||||||
- `sausage` is the test runner (`CARGO_TARGET_WASM32_WASIP2_RUNNER`). It runs
|
|
||||||
each test in its own instance, in parallel. A panic ends only its own test.
|
|
||||||
Without this, the first panic would end the whole test run, because WASI
|
|
||||||
has `panic=abort`.
|
|
||||||
- The normal Rust test setup works: `#[test]`, `tests/*.rs`, name filters,
|
|
||||||
and `--ignored`. Async tests use `#[sausage_plugin::test]`, like
|
|
||||||
`#[tokio::test]`. `#[should_panic]` cannot work, because WASI cannot catch
|
|
||||||
a panic.
|
|
||||||
- Each test gets its own mount table: an empty temporary world as `/`, the
|
|
||||||
package folder (so that fixture files work), the variables that cargo sets,
|
|
||||||
and its own `/proc`, `/self`, and `/run`.
|
|
||||||
- The runner answers `core.lock`, `core.unlock`, `core.notify`, and
|
|
||||||
`core.log`. A call to a plugin fails with a clear error: a test has no
|
|
||||||
other plugins.
|
|
||||||
|
|
||||||
## 13. The SDK
|
|
||||||
|
|
||||||
Three crates:
|
|
||||||
|
|
||||||
- `sausage-plugin`: the SDK. A plugin depends only on this crate. It
|
|
||||||
re-exports the macros, `serde`, and `schemars`.
|
|
||||||
- `sausage-plugin-macros`: the macros.
|
|
||||||
- `sausage-interface`: the OpenRPC documents, the compatibility check
|
|
||||||
(section 11.2), and the generator of client code. The macros and the host
|
|
||||||
both use it, so `import!` and `/proc/core/plugins/<name>/client.rs` give
|
|
||||||
the same code.
|
|
||||||
|
|
||||||
| Item | Purpose |
|
|
||||||
| --- | --- |
|
|
||||||
| `#[rpc]` | Makes an `impl` an interface. On the main interface: `#[rpc(endpoints(name = Type), grace_seconds = N)]`. |
|
|
||||||
| `#[on_retired]` | Marks the method that answers `rpc.retired`. |
|
|
||||||
| `#[on_stop]` | Marks the stop hook: an optional method that runs at EOF on stdin (section 4.6). |
|
|
||||||
| `#[schema]` | For each type that a method takes or returns: adds the serde and schemars derives, and checks the subset of section 11.2. |
|
|
||||||
| `run(T)` | In `main`: serves the main interface on stdin and stdout. |
|
|
||||||
| `call(method, params)` | An untyped call. Returns a JSON value. |
|
|
||||||
| `.detach()`, `.on_notify(f)` | Options of a call. |
|
|
||||||
| `Ctx` | An optional second argument of a method: `ctx.notify(name, params)`, `ctx.exit_after_response()`. |
|
|
||||||
| `lock(path)` | A lock guard. `.try_now()` does not wait. `.keep()` holds the lock until the process ends. |
|
|
||||||
| `listen(path, value)`, `connect(path)` | Endpoints. |
|
|
||||||
| `background(future)` | Runs a task in this process. The process lives while a task runs. |
|
|
||||||
| `import!("name")` | Generates `mod name` from `imports/name.json`. |
|
|
||||||
| `state_dir()`, `write_atomic(path, bytes)`, `pid()` | Helpers. |
|
|
||||||
| `#[sausage_plugin::test]` | An async test. A plain `#[test]` also works. |
|
|
||||||
|
|
||||||
Rules that the compiler enforces:
|
|
||||||
|
|
||||||
- A method takes `&self`, an optional `ctx: &Ctx`, and at most one params
|
|
||||||
argument.
|
|
||||||
- The params argument is a `#[schema]` struct with named fields. Its fields
|
|
||||||
are the params of the method.
|
|
||||||
- Each method, each `#[schema]` type, and each field has a doc comment.
|
|
||||||
Other agents read these docs to use the interface. Write them like good
|
|
||||||
Rust documentation.
|
|
||||||
|
|
||||||
Other rules:
|
|
||||||
|
|
||||||
- Method docs become `summary` (the first paragraph) and `description` in
|
|
||||||
OpenRPC. Type and field docs become `description` in the JSON Schema. The
|
|
||||||
generated imports contain them again as doc comments.
|
|
||||||
- Each `import!` registers its document in a static list with the
|
|
||||||
`inventory` crate. `run` reads the list for `rpc.discover`. `import!` uses
|
|
||||||
`include_str!`, so cargo rebuilds the plugin when the file changes.
|
|
||||||
- The value that `run` receives must be cheap to create, because the core
|
|
||||||
also starts a plugin only for `rpc.discover`. Open files and databases in
|
|
||||||
the methods.
|
|
||||||
- Log with `eprintln!`. stdout belongs to the protocol.
|
|
||||||
|
|
||||||
### 13.1 A plugin
|
|
||||||
|
|
||||||
```toml
|
|
||||||
[package]
|
|
||||||
name = "notes"
|
|
||||||
version = "0.1.0"
|
|
||||||
edition = "2024"
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
sausage-plugin = "0.1"
|
|
||||||
serde_json = "1"
|
|
||||||
```
|
|
||||||
|
|
||||||
```rust
|
|
||||||
//! Notes: a list of short texts, stored in `/state/notes/notes.json`.
|
|
||||||
//!
|
|
||||||
//! Each call runs in its own process, so two calls can run at the same time.
|
|
||||||
//! `add` takes a lock on the file, so that no note gets lost.
|
|
||||||
|
|
||||||
use std::path::PathBuf;
|
|
||||||
|
|
||||||
use sausage_plugin::{Result, lock, rpc, schema, state_dir, write_atomic};
|
|
||||||
|
|
||||||
/// The notes plugin. It keeps no state in memory: each call reads the file.
|
|
||||||
struct Notes;
|
|
||||||
|
|
||||||
/// The params of `add`.
|
|
||||||
#[schema]
|
|
||||||
struct AddParams {
|
|
||||||
/// The text of the new note. It can contain newlines.
|
|
||||||
text: String,
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The result of `add`.
|
|
||||||
#[schema]
|
|
||||||
struct Added {
|
|
||||||
/// The number of notes after the call, with the new note.
|
|
||||||
count: usize,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[rpc]
|
|
||||||
impl Notes {
|
|
||||||
/// Add a note at the end of the list.
|
|
||||||
///
|
|
||||||
/// Returns the number of notes after the call.
|
|
||||||
async fn add(&self, AddParams { text }: AddParams) -> Result<Added> {
|
|
||||||
let _lock = lock(file()).await?; // released on drop, or when the process ends
|
|
||||||
let mut notes = load()?;
|
|
||||||
notes.push(text);
|
|
||||||
write_atomic(file(), serde_json::to_vec(¬es)?)?;
|
|
||||||
Ok(Added { count: notes.len() })
|
|
||||||
}
|
|
||||||
|
|
||||||
/// List all notes, the oldest first.
|
|
||||||
fn list(&self) -> Result<Vec<String>> {
|
|
||||||
load() // no lock: write_atomic replaces the file in one step
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The file with the notes: a JSON list of strings.
|
|
||||||
fn file() -> PathBuf {
|
|
||||||
state_dir().join("notes.json")
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Read all notes. A missing file means no notes.
|
|
||||||
fn load() -> Result<Vec<String>> {
|
|
||||||
match std::fs::read(file()) {
|
|
||||||
Ok(bytes) => Ok(serde_json::from_slice(&bytes)?),
|
|
||||||
Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(vec![]),
|
|
||||||
Err(e) => Err(e.into()),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn main() {
|
|
||||||
sausage_plugin::run(Notes);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[cfg(test)]
|
|
||||||
mod tests {
|
|
||||||
use super::*;
|
|
||||||
|
|
||||||
#[sausage_plugin::test]
|
|
||||||
async fn add_appends_and_counts() {
|
|
||||||
assert_eq!(Notes.add(AddParams { text: "milk".into() }).await.unwrap().count, 1);
|
|
||||||
assert_eq!(Notes.add(AddParams { text: "eggs".into() }).await.unwrap().count, 2);
|
|
||||||
assert_eq!(Notes.list().unwrap(), ["milk", "eggs"]);
|
|
||||||
}
|
|
||||||
|
|
||||||
#[test]
|
|
||||||
fn list_is_empty_without_a_file() {
|
|
||||||
assert!(Notes.list().unwrap().is_empty());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### 13.2 A server with an endpoint
|
|
||||||
|
|
||||||
```rust
|
|
||||||
//! A web server for the files in `/state/www/`.
|
|
||||||
//!
|
|
||||||
//! `start` starts the server in its own process and returns at once. Run it
|
|
||||||
//! as a service, or call it detached, or the server ends with its caller.
|
|
||||||
//! The running server serves the endpoint `control` at
|
|
||||||
//! `/proc/<pid>/control`.
|
|
||||||
|
|
||||||
use std::{cell::Cell, rc::Rc};
|
|
||||||
|
|
||||||
use sausage_plugin::{Ctx, Result, background, listen, pid, rpc, schema};
|
|
||||||
use wstd::net::TcpListener;
|
|
||||||
|
|
||||||
/// The www plugin.
|
|
||||||
struct Www;
|
|
||||||
|
|
||||||
/// The params of `start`.
|
|
||||||
#[schema]
|
|
||||||
struct StartParams {
|
|
||||||
/// The TCP port to listen on, on all interfaces.
|
|
||||||
port: u16,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[rpc(endpoints(control = Control), grace_seconds = 5)]
|
|
||||||
impl Www {
|
|
||||||
/// Start the web server and return the ID of its process.
|
|
||||||
///
|
|
||||||
/// The server runs until it gets `stop` on its endpoint, until
|
|
||||||
/// `core.stop`, or until its caller ends. Fails if the port is in use.
|
|
||||||
async fn start(&self, StartParams { port }: StartParams) -> Result<String> {
|
|
||||||
let tcp = TcpListener::bind(&format!("0.0.0.0:{port}")).await?;
|
|
||||||
let control = Rc::new(Control { hits: Cell::new(0) });
|
|
||||||
listen("/self/control", control.clone()).await?;
|
|
||||||
background(serve(tcp, control)); // the process lives while this task runs
|
|
||||||
Ok(pid())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// The control endpoint of a running server.
|
|
||||||
struct Control {
|
|
||||||
/// The number of HTTP requests that the server answered.
|
|
||||||
hits: Cell<u64>,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[rpc]
|
|
||||||
impl Control {
|
|
||||||
/// The number of HTTP requests that the server answered since its start.
|
|
||||||
fn stats(&self) -> Result<u64> {
|
|
||||||
Ok(self.hits.get())
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Stop the server. The process ends after this response.
|
|
||||||
fn stop(&self, ctx: &Ctx) -> Result<()> {
|
|
||||||
ctx.exit_after_response();
|
|
||||||
Ok(())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Accept HTTP connections, and answer each request from `/state/www/`.
|
|
||||||
async fn serve(tcp: TcpListener, control: Rc<Control>) {
|
|
||||||
// …
|
|
||||||
}
|
|
||||||
|
|
||||||
fn main() {
|
|
||||||
sausage_plugin::run(Www);
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The same type can serve stdin and an endpoint. The example uses two types
|
|
||||||
for a reason: `www.stats` through the core would start a fresh process, and
|
|
||||||
that process would answer 0.
|
|
||||||
|
|
||||||
### 13.3 A service with a fixed endpoint
|
|
||||||
|
|
||||||
```rust
|
|
||||||
//! Secrets: keeps secrets in memory, for other plugins.
|
|
||||||
//!
|
|
||||||
//! Run it as a service: `{"method": "secrets.serve", "params": {}}` in
|
|
||||||
//! `/config/boot.json`. Clients connect to the endpoint `vault` at
|
|
||||||
//! `/run/secrets`.
|
|
||||||
|
|
||||||
use std::rc::Rc;
|
|
||||||
|
|
||||||
use sausage_plugin::{Result, listen, rpc};
|
|
||||||
|
|
||||||
/// The secrets plugin.
|
|
||||||
struct Secrets;
|
|
||||||
|
|
||||||
#[rpc(endpoints(vault = Vault))]
|
|
||||||
impl Secrets {
|
|
||||||
/// Open the vault and serve it at `/run/secrets`.
|
|
||||||
///
|
|
||||||
/// Returns when the endpoint is ready. The process then lives while the
|
|
||||||
/// endpoint listens.
|
|
||||||
async fn serve(&self) -> Result<()> {
|
|
||||||
listen("/run/secrets", Rc::new(Vault::open()?)).await
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
A client imports `secrets` and calls
|
|
||||||
`secrets::vault::connect("/run/secrets").await?`.
|
|
||||||
|
|
||||||
### 13.4 Imports
|
|
||||||
|
|
||||||
To use another plugin, an agent does these steps:
|
|
||||||
|
|
||||||
1. Copy `/proc/core/plugins/notes/discover.json` to
|
|
||||||
`source/imports/notes.json`.
|
|
||||||
2. Read `/proc/core/plugins/notes/client.rs`: the exact code that the import
|
|
||||||
generates.
|
|
||||||
3. Write `import!("notes")` and the code that uses it.
|
|
||||||
4. Call `cargo.build`, and remove the imported methods that the warnings call
|
|
||||||
unused.
|
|
||||||
5. Call `core.activate`.
|
|
||||||
|
|
||||||
```rust
|
|
||||||
sausage_plugin::import!("notes"); // imports/notes.json
|
|
||||||
sausage_plugin::import!("www"); // imports/www.json, with its endpoint "control"
|
|
||||||
|
|
||||||
/// Add a note, then start a web server and return its number of answered requests.
|
|
||||||
async fn example() -> Result<u64> {
|
|
||||||
let added = notes::add(notes::AddParams { text: "milk".into() }).await?;
|
|
||||||
eprintln!("{} notes", added.count); // stderr goes to the journal
|
|
||||||
let pid = www::start(www::StartParams { port: 8080 }).detach().await?;
|
|
||||||
let control = www::control::connect(&format!("/proc/{pid}/control")).await?;
|
|
||||||
control.stats().await
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- The params of each method are one struct. The fields have the same names as
|
|
||||||
the JSON params.
|
|
||||||
- Types take their names from the schemas. When the provider uses
|
|
||||||
`#[schema]`, those are the names of its Rust types: `AddParams` stays
|
|
||||||
`AddParams`.
|
|
||||||
- The generated items are private to the crate, so rustc warns about each
|
|
||||||
unused method.
|
|
||||||
|
|
||||||
## 14. The agent
|
|
||||||
|
|
||||||
- `agent.chat {"text": "..."}` answers a message from a person. The result is
|
|
||||||
the final text. The interim texts come as `core.notify` with the name
|
|
||||||
`text`.
|
|
||||||
- The agent takes a lock on `/state/agent/history.jsonl`. So a second message
|
|
||||||
waits until the first turn has ended.
|
|
||||||
- The agent declares a long grace period (for example 10 minutes), so that a
|
|
||||||
turn can finish after an upgrade. A new version applies at the next
|
|
||||||
`chat`.
|
|
||||||
- The tools: `call` (method, params, detach), `read_file`, `write_file`,
|
|
||||||
`list_dir`, and `copy_dir`.
|
|
||||||
- The configuration is in `/config/agent.json`.
|
|
||||||
- The prompt stays short. It says:
|
|
||||||
- where the contract is (`/proc/core/PROTOCOL.md`),
|
|
||||||
- where the SDK is (`/plugins/cargo/sdk/`),
|
|
||||||
- how to build and activate a plugin,
|
|
||||||
- where the agent's own source is (`/plugins/agent/source/`),
|
|
||||||
- that logs go to stderr,
|
|
||||||
- that the interface check does not cover state.
|
|
||||||
|
|
||||||
## 15. The console
|
|
||||||
|
|
||||||
- `rustyline` gives line editing and history. Notifications appear while you
|
|
||||||
type.
|
|
||||||
- A short prompt. Results are indented. Agent notifications are dimmed.
|
|
||||||
|
|
||||||
| Input | Effect |
|
|
||||||
| --- | --- |
|
|
||||||
| text | `agent.chat` |
|
|
||||||
| `<plugin>.<method> {json}` | a call. The console prints the result. |
|
|
||||||
| `ls [path]`, `cat <path>` | the filesystem as a process sees it, with `/proc` and `/run` |
|
|
||||||
| `methods <plugin>` | the methods from `rpc.discover`, with params and docs |
|
|
||||||
| `ps` | the running processes |
|
|
||||||
| `log [plugin or pid] [-n N]` | the journal |
|
|
||||||
| `activate <plugin> <wasm>`, `revert <plugin>` | `core.activate`, `core.revert` |
|
|
||||||
| `stop <pid>`, `kill <pid>` | `core.stop`, `core.kill` |
|
|
||||||
| `help`, `quit` | |
|
|
||||||
|
|
||||||
`--headless` stays: no console input, and the host runs until SIGTERM or
|
|
||||||
`core.exit`.
|
|
||||||
|
|
||||||
## 16. Caches
|
|
||||||
|
|
||||||
- Wasmtime's own compile cache stores the compiled code of each `.wasm` file.
|
|
||||||
It supports components.
|
|
||||||
- The folder is `$XDG_CACHE_HOME/sausage/`, or `~/.cache/sausage/` without
|
|
||||||
that variable. `--cache-dir` changes the folder.
|
|
||||||
|
|
||||||
## 17. Decisions
|
|
||||||
|
|
||||||
- **JSON-RPC, not component interfaces.** Every language can read and write
|
|
||||||
lines of JSON, and models know JSON-RPC from LSP and MCP.
|
|
||||||
- **A process per call.** We get parallelism without threads in WASI. A new
|
|
||||||
version applies at the next call, and old processes retire after a grace
|
|
||||||
period.
|
|
||||||
- **Attached by default.** A process does not outlive its caller by
|
|
||||||
accident.
|
|
||||||
- **Services in one file.** Only the core starts and restarts them, so
|
|
||||||
upgrades and singletons need no logic in plugins.
|
|
||||||
- **Files to inspect, calls to act.** This keeps the core API small, and the
|
|
||||||
agent's file tools are enough to explore the world.
|
|
||||||
- **Generated imports and interface checks, without version numbers.** A model
|
|
||||||
that translates a description by hand makes errors. A generator does not.
|
|
||||||
The check catches the most common error in a world that changes itself: a
|
|
||||||
change to one plugin breaks another one.
|
|
||||||
- **A small schema subset.** With Draft 7 and a fixed subset, our own check
|
|
||||||
can decide every case.
|
|
||||||
- **Docs are part of the interface.** Other agents work from them, so the
|
|
||||||
compiler requires them.
|
|
||||||
- **No permissions.** The checks protect against mistakes, not against
|
|
||||||
plugins. Locks are advisory.
|
|
||||||
- **Our own VFS, in its own crate.** It works on Linux and macOS, it makes
|
|
||||||
`/proc` live, and plugin-served files need it later. Mounts now, overlays
|
|
||||||
later.
|
|
||||||
- **TCP endpoints for now.** They need no host code. VFS pipes can replace
|
|
||||||
them later behind the same SDK API.
|
|
||||||
- **Simple scheduling for now.** An epoch tick every 10 milliseconds keeps
|
|
||||||
the core responsive and makes kills reliable.
|
|
||||||
|
|
||||||
## 18. Later
|
|
||||||
|
|
||||||
- Plugin-served files at `/proc/<pid>/fs/`, for example for an FTP plugin. A
|
|
||||||
request gets a timeout of 10 seconds, because a process that reads its own
|
|
||||||
served files would wait for itself forever.
|
|
||||||
- Overlays in the VFS.
|
|
||||||
- VFS pipes for endpoints, instead of TCP ports.
|
|
||||||
- Subscriptions to file changes.
|
|
||||||
- Structured logging.
|
|
||||||
- A secrets service.
|
|
||||||
- Cancellation of calls.
|
|
||||||
- A better scheduler.
|
|
||||||
|
|
||||||
## 19. Plan
|
|
||||||
|
|
||||||
The work happens on the branch `plugins`. Each step includes its tests.
|
|
||||||
|
|
||||||
Already confirmed:
|
|
||||||
|
|
||||||
- `inventory` works on `wasm32-wasip2`.
|
|
||||||
- Wasmtime's compile cache supports components.
|
|
||||||
- C code builds for `wasm32-wasip2`.
|
|
||||||
|
|
||||||
Checks of risks first:
|
|
||||||
|
|
||||||
1. An end-to-end pair: a provider and a consumer with a generated import,
|
|
||||||
through the real transport. This includes `typify` with Draft 7 from
|
|
||||||
schemars, and our compatibility check on real documents.
|
|
||||||
2. Fixture paths in tests under the runner.
|
|
||||||
|
|
||||||
Then the steps:
|
|
||||||
|
|
||||||
1. `PROTOCOL.md` and `DECISIONS.md` from this proposal.
|
|
||||||
2. `sausage-vfs`: the trait, the layers, the Wasmtime adapter, and its tests.
|
|
||||||
3. The core: a process per call, JSON-RPC routing, the process tree, upgrades,
|
|
||||||
services, locks, the journal, scheduling, and the compile cache.
|
|
||||||
4. The SDK and its macros.
|
|
||||||
5. `cargo.build`, the test runner, activation, and the graph.
|
|
||||||
6. The agent and the console.
|
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"openrpc": "1.2.6",
|
||||||
|
"info": {"title": "notes", "version": "0.1.0"},
|
||||||
|
"components": {"schemas": {
|
||||||
|
"AddParams": {
|
||||||
|
"description": "The parameters of add.",
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"text": {"type": "string", "description": "The text to append."}},
|
||||||
|
"required": ["text"]
|
||||||
|
},
|
||||||
|
"Added": {
|
||||||
|
"description": "The result of add.",
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"count": {"type": "integer", "minimum": 0, "description": "The total number of notes."}},
|
||||||
|
"required": ["count"]
|
||||||
|
}
|
||||||
|
}},
|
||||||
|
"methods": [{
|
||||||
|
"name": "add",
|
||||||
|
"summary": "Append a note and return the new count.",
|
||||||
|
"params": [{"name": "text", "required": true, "schema": {"type": "string"}}],
|
||||||
|
"x-sausage-params": {"$ref": "#/components/schemas/AddParams"},
|
||||||
|
"result": {"name": "result", "schema": {"$ref": "#/components/schemas/Added"}}
|
||||||
|
}]
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
{
|
||||||
|
"openrpc": "1.2.6",
|
||||||
|
"info": {"title": "www", "version": "0.1.0"},
|
||||||
|
"components": {"schemas": {
|
||||||
|
"StartParams": {
|
||||||
|
"description": "The server configuration.",
|
||||||
|
"type": "object",
|
||||||
|
"properties": {"port": {"type": "integer", "format": "uint16", "minimum": 0, "maximum": 65535, "description": "The HTTP port."}},
|
||||||
|
"required": ["port"]
|
||||||
|
}
|
||||||
|
}},
|
||||||
|
"methods": [{
|
||||||
|
"name": "start",
|
||||||
|
"summary": "Start the HTTP server and return its process ID.",
|
||||||
|
"params": [{"name": "port", "required": true, "schema": {"type": "integer", "format": "uint16", "minimum": 0, "maximum": 65535}}],
|
||||||
|
"x-sausage-params": {"$ref": "#/components/schemas/StartParams"},
|
||||||
|
"result": {"name": "result", "schema": {"type": "string"}}
|
||||||
|
}],
|
||||||
|
"x-sausage-endpoints": {"control": {"methods": [{
|
||||||
|
"name": "stats",
|
||||||
|
"summary": "Return the number of HTTP requests.",
|
||||||
|
"params": [],
|
||||||
|
"result": {"name": "result", "schema": {"type": "integer", "format": "uint64", "minimum": 0}}
|
||||||
|
}]}}
|
||||||
|
}
|
||||||
+374
-36
@@ -1,52 +1,390 @@
|
|||||||
//! Write single-threaded WASI plugins for Sausage.
|
//! Write single-threaded WASI plugins for Sausage.
|
||||||
//!
|
//!
|
||||||
//! Each core call starts a process. Keep its initial value cheap to construct.
|
//! # Start here
|
||||||
//! Declare documented parameter and result types with [`macro@schema`].
|
//!
|
||||||
//! Declare methods with [`macro@rpc`], then serve them with [`run`].
|
//! A plugin depends on `sausage-plugin = "0.1"` in its `Cargo.toml`.
|
||||||
|
//! Use Rust edition 2024. The cargo plugin supplies the local SDK dependency.
|
||||||
|
//! The SDK re-exports serde, schemars, serde_json, and its macros.
|
||||||
|
//! Build the plugin through `cargo.build`. Activate the returned WASM file through `core.activate`.
|
||||||
|
//! Read `/proc/core/PROTOCOL.md` for the host contract.
|
||||||
|
//!
|
||||||
|
//! Each core call starts a process. Keep the value passed to [`run`] cheap to construct.
|
||||||
|
//! The core also starts a process just for `rpc.discover`.
|
||||||
|
//! Open files, databases, and listeners in methods, not in `main`.
|
||||||
|
//! Log with `eprintln!`. Stdout belongs to JSON-RPC.
|
||||||
|
//!
|
||||||
|
//! # Methods and schemas
|
||||||
|
//!
|
||||||
|
//! Each `fn` in an [`macro@rpc`] impl becomes an RPC method.
|
||||||
|
//! Put helpers in a separate impl block without `#[rpc]`.
|
||||||
|
//! Both synchronous and async methods must follow these rules:
|
||||||
|
//!
|
||||||
|
//! - Take `&self` first.
|
||||||
|
//! - Optionally take `ctx: &Ctx` next.
|
||||||
|
//! - Take at most one parameter argument after that.
|
||||||
|
//! - Use a [`macro@schema`] struct with named fields for that argument.
|
||||||
|
//! - Return [`Result<T>`].
|
||||||
|
//! - Give each method, schema type, and field a doc comment.
|
||||||
|
//!
|
||||||
|
//! The parameter fields become the named JSON parameters.
|
||||||
|
//! A method without parameters accepts an empty object.
|
||||||
|
//! Its discovery entry has no `x-sausage-params` field.
|
||||||
|
//! Raw identifiers such as `r#type` use the plain wire name `type`.
|
||||||
|
//! Method docs become the OpenRPC summary and description.
|
||||||
|
//! Type and field docs become schema descriptions. Generated clients retain these docs.
|
||||||
|
//! Docs are part of the interface. Other agents read them to write callers.
|
||||||
|
//!
|
||||||
|
//! `#[schema]` adds serde and schemars derives. Do not add those derives yourself.
|
||||||
|
//! These types form the supported subset:
|
||||||
|
//!
|
||||||
|
//! - `String`, `bool`, and `()`.
|
||||||
|
//! - `u8`, `u16`, `u32`, `u64`, and `usize`.
|
||||||
|
//! - `i8`, `i16`, `i32`, `i64`, and `isize`.
|
||||||
|
//! - `f32` and `f64`.
|
||||||
|
//! - `Vec<T>` and `Option<T>` with supported element types.
|
||||||
|
//! - Other `#[schema]` structs and enums.
|
||||||
|
//!
|
||||||
|
//! Structs must have named fields. Enums must have unit variants.
|
||||||
|
//! Document the variants to explain their meanings to callers.
|
||||||
|
//! The permitted serde attributes are `rename`, `rename_all`, `default`, and `skip_serializing_if`.
|
||||||
|
//! For example, use `#[serde(default)]` for a field with a default value.
|
||||||
|
//! Use `#[serde(default, skip_serializing_if = "Option::is_none")]` for an omitted optional field.
|
||||||
|
//! `#[schema]` rejects other serde attributes and all `#[schemars(...)]` attributes.
|
||||||
|
//! Tuple structs, unit structs, tuple fields, maps, arbitrary JSON values, and data-carrying variants are outside this subset.
|
||||||
|
//! Such fields cannot appear in a `#[schema]` type.
|
||||||
|
//!
|
||||||
|
//! # A persistent plugin
|
||||||
|
//!
|
||||||
|
//! Calls can run concurrently in separate processes. Protect shared changes with [`lock`].
|
||||||
|
//! Bind the guard to a name: `let _guard = lock(p).await?;`.
|
||||||
|
//! `let _ = lock(p).await?;` releases the lock at once.
|
||||||
|
//! The guard releases the lock when its scope ends. Process exit also releases locks.
|
||||||
|
//! `try_now()` does not wait. Check the returned guard with `held()`.
|
||||||
|
//! `keep()` retains an acquired lock until process exit.
|
||||||
|
//!
|
||||||
|
//! This complete plugin stores notes in [`state_dir`]. [`write_atomic`] replaces the file after a complete temporary write.
|
||||||
|
//! Create the state directory before the first write.
|
||||||
//!
|
//!
|
||||||
//! ```no_run
|
//! ```no_run
|
||||||
//! use sausage_plugin::{rpc, schema, Result};
|
//! use sausage_plugin::{lock, rpc, schema, state_dir, write_atomic, Result};
|
||||||
//! struct Calculator;
|
//! use sausage_plugin::serde_json;
|
||||||
//! /// The numbers to add.
|
//! use std::path::PathBuf;
|
||||||
|
//!
|
||||||
|
//! /// Persistent notes shared by all calls.
|
||||||
|
//! struct Notes;
|
||||||
|
//! /// The parameters of add.
|
||||||
//! #[schema]
|
//! #[schema]
|
||||||
//! struct AddParams {
|
//! struct AddParams {
|
||||||
//! /// The first number.
|
//! /// The text to append. Newlines are permitted.
|
||||||
//! a: i32,
|
//! text: String,
|
||||||
//! /// The second number.
|
//! }
|
||||||
//! b: i32,
|
//! /// The result of add.
|
||||||
|
//! #[schema]
|
||||||
|
//! struct Added {
|
||||||
|
//! /// The total number of notes.
|
||||||
|
//! count: usize,
|
||||||
//! }
|
//! }
|
||||||
//! #[rpc]
|
//! #[rpc]
|
||||||
//! impl Calculator {
|
//! impl Notes {
|
||||||
//! /// Add two numbers.
|
//! /// Append a note and return the new count.
|
||||||
//! fn add(&self, p: AddParams) -> Result<i32> { Ok(p.a + p.b) }
|
//! async fn add(&self, p: AddParams) -> Result<Added> {
|
||||||
|
//! std::fs::create_dir_all(state_dir())?;
|
||||||
|
//! let _guard = lock(Self::file()).await?;
|
||||||
|
//! let mut notes = Self::load()?;
|
||||||
|
//! notes.push(p.text);
|
||||||
|
//! write_atomic(Self::file(), serde_json::to_vec(¬es)?)?;
|
||||||
|
//! Ok(Added { count: notes.len() })
|
||||||
//! }
|
//! }
|
||||||
//! fn main() { sausage_plugin::run(Calculator); }
|
//! /// List the notes, oldest first.
|
||||||
|
//! fn list(&self) -> Result<Vec<String>> { Self::load() }
|
||||||
|
//! }
|
||||||
|
//! // These helpers do not become RPC methods.
|
||||||
|
//! impl Notes {
|
||||||
|
//! fn file() -> PathBuf { state_dir().join("notes.json") }
|
||||||
|
//! fn load() -> Result<Vec<String>> {
|
||||||
|
//! match std::fs::read(Self::file()) {
|
||||||
|
//! Ok(bytes) => Ok(serde_json::from_slice(&bytes)?),
|
||||||
|
//! Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(vec![]),
|
||||||
|
//! Err(e) => Err(e.into()),
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! fn main() { sausage_plugin::run(Notes); }
|
||||||
//! ```
|
//! ```
|
||||||
//!
|
//!
|
||||||
//! Use [`call`] for outgoing calls. Await its builder for the JSON result.
|
//! # Calls and process lifetime
|
||||||
//! Use `detach()` to detach the child and `on_notify(f)` to receive progress.
|
|
||||||
//! Use [`Ctx::notify`] to report progress to your caller.
|
|
||||||
//! Use [`Ctx::exit_after_response`] to end the process after the response.
|
|
||||||
//!
|
//!
|
||||||
//! [`background`] keeps the process alive while a task runs.
|
//! [`call`] returns a lazy builder. Await it to send the request and receive its JSON result.
|
||||||
//! At stdin EOF, pending calls fail and the optional [`macro@on_stop`] hook runs.
|
//! `detach()` lets the child outlive this process. `on_notify(f)` receives its progress notifications.
|
||||||
//! The stop hook can write files and send notifications, but cannot make calls.
|
//! Use [`Ctx::notify`] to send progress to the caller of the current request.
|
||||||
//! [`macro@on_retired`] handles `rpc.retired` after an upgrade.
|
//! Do not call [`block_on`] in a handler. The runtime cannot nest `block_on` calls.
|
||||||
|
//! Write an async handler and use `.await` instead.
|
||||||
//!
|
//!
|
||||||
//! Use [`lock`] before shared state changes. A guard releases its lock on drop.
|
//! A process ends after its request finishes if it has no remaining work.
|
||||||
//! `try_now()` returns a guard whose `held()` method reports success.
|
//! [`background`] keeps it alive while a task runs. An endpoint listener also keeps it alive.
|
||||||
//! `keep()` retains a lock until process exit.
|
//! Neither mechanism detaches the process from its caller. Start long-lived processes detached or as services.
|
||||||
//! [`state_dir`], [`write_atomic`], and [`pid`] provide filesystem and identity helpers.
|
//! At stdin EOF, pending outgoing calls fail with -32000. The SDK stops endpoint listeners and runs the optional stop hook.
|
||||||
//! Use [`macro@test`] for async tests. Native tests fail clearly without a core transport.
|
//! The stop hook can write files and send notifications. It cannot make new calls.
|
||||||
//! [`macro@import`] reads `imports/<provider>.json` and generates a private typed client.
|
//! After the serve loop, the SDK flushes stdout and exits, even if background tasks remain.
|
||||||
//! Its functions return [`TypedCall`] builders with progress and detach options.
|
//! [`Ctx::exit_after_response`] ends the process after its response. This also works in an endpoint handler.
|
||||||
//! Declare endpoint types with `#[rpc(endpoints(control = Control))]`.
|
//!
|
||||||
//! [`listen`] publishes a declared interface at an endpoint file.
|
//! ```no_run
|
||||||
//! [`connect`] verifies its owner before it returns an [`Endpoint`].
|
//! use sausage_plugin::{background, call, rpc, Ctx, Result};
|
||||||
//! Endpoint calls report progress through [`EndpointCall::on_notify`].
|
//! use sausage_plugin::serde_json::json;
|
||||||
//! Local transport errors use code -32000: no transport, stdin EOF, a closed endpoint,
|
//! /// A worker that delegates a job.
|
||||||
//! or an endpoint identity or interface mismatch. Code -32002 belongs to core process failures.
|
//! struct Worker;
|
||||||
//! Log with `eprintln!`. Stdout belongs to JSON-RPC.
|
//! #[rpc]
|
||||||
|
//! impl Worker {
|
||||||
|
//! /// Start an attached background task and return immediately.
|
||||||
|
//! fn start(&self, ctx: &Ctx) -> Result<()> {
|
||||||
|
//! ctx.notify("text", "worker started")?;
|
||||||
|
//! background(async {
|
||||||
|
//! let result = call("notes.list", json!({}))
|
||||||
|
//! .on_notify(|name, params| eprintln!("{name}: {params}"))
|
||||||
|
//! .await;
|
||||||
|
//! if let Err(error) = result { eprintln!("{error}"); }
|
||||||
|
//! });
|
||||||
|
//! Ok(())
|
||||||
|
//! }
|
||||||
|
//! /// Flush local files before this process stops.
|
||||||
|
//! #[sausage_plugin::on_stop]
|
||||||
|
//! fn stop(&self) -> Result<()> {
|
||||||
|
//! eprintln!("worker stopped");
|
||||||
|
//! Ok(())
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! fn main() { sausage_plugin::run(Worker); }
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! # A server with a process endpoint
|
||||||
|
//!
|
||||||
|
//! Declare endpoint types on the main interface. Publish them with [`listen`].
|
||||||
|
//! `listen` returns after it publishes the endpoint file, not after the listener ends.
|
||||||
|
//! `/self/control` belongs to this process. Clients find it at `/proc/<pid>/control`.
|
||||||
|
//! This server counts requests in memory. A new core call cannot inspect the old process's counter.
|
||||||
|
//!
|
||||||
|
//! ```no_run
|
||||||
|
//! use sausage_plugin::{listen, pid, rpc, Ctx, Result};
|
||||||
|
//! use std::{cell::Cell, rc::Rc};
|
||||||
|
//! /// A server with process-local counters.
|
||||||
|
//! struct Server;
|
||||||
|
//! /// The interface of one running server.
|
||||||
|
//! struct Control { hits: Cell<u64> }
|
||||||
|
//! #[rpc(endpoints(control = Control), grace_seconds = 5)]
|
||||||
|
//! impl Server {
|
||||||
|
//! /// Start a counter server and return its process ID.
|
||||||
|
//! async fn start(&self) -> Result<String> {
|
||||||
|
//! listen("/self/control", Rc::new(Control { hits: Cell::new(0) })).await?;
|
||||||
|
//! Ok(pid())
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! #[rpc]
|
||||||
|
//! impl Control {
|
||||||
|
//! /// Record a request and return the new count.
|
||||||
|
//! fn record(&self) -> Result<u64> {
|
||||||
|
//! self.hits.set(self.hits.get() + 1);
|
||||||
|
//! Ok(self.hits.get())
|
||||||
|
//! }
|
||||||
|
//! /// Return the number of recorded requests.
|
||||||
|
//! fn stats(&self) -> Result<u64> { Ok(self.hits.get()) }
|
||||||
|
//! /// Stop this process after the client receives the response.
|
||||||
|
//! fn stop(&self, ctx: &Ctx) -> Result<()> {
|
||||||
|
//! ctx.exit_after_response();
|
||||||
|
//! Ok(())
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! fn main() { sausage_plugin::run(Server); }
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! # A service with a fixed endpoint
|
||||||
|
//!
|
||||||
|
//! A service can publish a fixed path in `/run`. The core clears `/run` when the host starts.
|
||||||
|
//! Configure this service in `/config/boot.json` with `{"method":"secrets.serve","params":{}}`.
|
||||||
|
//! Clients can then connect to `/run/secrets`.
|
||||||
|
//!
|
||||||
|
//! ```no_run
|
||||||
|
//! use sausage_plugin::{listen, rpc, schema, Result};
|
||||||
|
//! use std::rc::Rc;
|
||||||
|
//! /// The service entry point.
|
||||||
|
//! struct Secrets;
|
||||||
|
//! /// The in-memory vault of this service process.
|
||||||
|
//! struct Vault { token: String }
|
||||||
|
//! /// The name of a secret.
|
||||||
|
//! #[schema]
|
||||||
|
//! struct GetParams {
|
||||||
|
//! /// The secret to read.
|
||||||
|
//! name: String,
|
||||||
|
//! }
|
||||||
|
//! #[rpc(endpoints(vault = Vault))]
|
||||||
|
//! impl Secrets {
|
||||||
|
//! /// Load the vault and publish its fixed endpoint.
|
||||||
|
//! async fn serve(&self) -> Result<()> {
|
||||||
|
//! let token = std::fs::read_to_string("/config/secret-token")?;
|
||||||
|
//! listen("/run/secrets", Rc::new(Vault { token })).await
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! #[rpc]
|
||||||
|
//! impl Vault {
|
||||||
|
//! /// Read the token, or return None for an unknown name.
|
||||||
|
//! fn get(&self, p: GetParams) -> Result<Option<String>> {
|
||||||
|
//! Ok((p.name == "token").then(|| self.token.clone()))
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! fn main() { sausage_plugin::run(Secrets); }
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! [`connect`] reads the endpoint file and verifies its owner through `rpc.endpoint`.
|
||||||
|
//! The handshake checks the plugin, owner process ID, and interface name.
|
||||||
|
//! A generated endpoint client also requires its expected interface name.
|
||||||
|
//! A mismatch returns -32000. Endpoint calls bypass a fresh core process.
|
||||||
|
//! `connect` opens and closes a verification connection. Each call opens another connection and repeats the handshake.
|
||||||
|
//! The first call costs three round trips: verification, another handshake, and the request.
|
||||||
|
//! Later calls cost a handshake and a request each. The SDK does not pool connections.
|
||||||
|
//!
|
||||||
|
//! ```no_run
|
||||||
|
//! use sausage_plugin::{connect, Result};
|
||||||
|
//! use sausage_plugin::serde_json::json;
|
||||||
|
//! async fn read_token() -> Result<Option<String>> {
|
||||||
|
//! let vault = connect("/run/secrets").await?;
|
||||||
|
//! vault.require_interface("vault")?;
|
||||||
|
//! vault.call("get", json!({"name":"token"}))
|
||||||
|
//! .on_notify(|name, params| eprintln!("{name}: {params}"))
|
||||||
|
//! .await
|
||||||
|
//! }
|
||||||
|
//! fn main() {}
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! # Import another plugin
|
||||||
|
//!
|
||||||
|
//! 1. Copy `/proc/core/plugins/<name>/discover.json` to `source/imports/<name>.json` in your plugin folder.
|
||||||
|
//! 2. Read `/proc/core/plugins/<name>/client.rs` for the exact generated client API.
|
||||||
|
//! 3. Write `sausage_plugin::import!("<name>")` and the code that calls it.
|
||||||
|
//! 4. Build with `cargo.build`. Remove methods that rustc calls unused from the imported JSON document.
|
||||||
|
//! 5. Build again after that change. Call `core.activate` with the resulting WASM path.
|
||||||
|
//!
|
||||||
|
//! The macro reads `imports/<name>.json` relative to `CARGO_MANIFEST_DIR`, not the current folder.
|
||||||
|
//! It generates a private module with typed [`TypedCall`] builders.
|
||||||
|
//! The module retains schema type names and documentation. Await its functions to get their typed results.
|
||||||
|
//! Builders support `detach()` and `on_notify(f)`. Generated endpoint modules provide typed `connect` functions.
|
||||||
|
//! The macro tracks the JSON file with `include_str!`, so Cargo rebuilds after changes.
|
||||||
|
//! It registers the imported contract for `rpc.discover` through inventory.
|
||||||
|
//! Remove unused methods from the JSON, not from generated Rust. This keeps `x-sausage-imports` small.
|
||||||
|
//! Retain the schemas that the remaining methods reference.
|
||||||
|
//!
|
||||||
|
//! This example imports the notes plugin above and a server with a `control` endpoint.
|
||||||
|
//! The `www` provider exposes `start(StartParams { port })` and `control.stats()`.
|
||||||
|
//!
|
||||||
|
//! ```no_run
|
||||||
|
//! use sausage_plugin::{rpc, Result};
|
||||||
|
//! sausage_plugin::import!("notes");
|
||||||
|
//! sausage_plugin::import!("www");
|
||||||
|
//! /// A consumer of notes and the HTTP server.
|
||||||
|
//! struct Consumer;
|
||||||
|
//! #[rpc]
|
||||||
|
//! impl Consumer {
|
||||||
|
//! /// Add a note, start a detached server, and return its request count.
|
||||||
|
//! async fn example(&self) -> Result<u64> {
|
||||||
|
//! let added = notes::add(notes::AddParams { text: "milk".into() }).await?;
|
||||||
|
//! eprintln!("{} notes", added.count);
|
||||||
|
//! let pid = www::start(www::StartParams { port: 8080 }).detach().await?;
|
||||||
|
//! let control = www::control::connect(&format!("/proc/{pid}/control")).await?;
|
||||||
|
//! control.stats().await
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! fn main() { sausage_plugin::run(Consumer); }
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! # Tests in a world
|
||||||
|
//!
|
||||||
|
//! `cargo.build` runs tests under the WASI test runner.
|
||||||
|
//! Both plain `#[test]` and async [`macro@test`] tests work. Each test gets its own instance and empty temporary world at `/`.
|
||||||
|
//! The current folder is `/`, not the package folder. Read fixtures through `env!("CARGO_MANIFEST_DIR")`.
|
||||||
|
//! `SAUSAGE_PLUGIN` is `test`, so `state_dir()` returns `/state/test`.
|
||||||
|
//! Create directories before writes. `std::env::temp_dir()` does not work under this runner.
|
||||||
|
//! Use an explicit path in the temporary world instead.
|
||||||
|
//! The runner answers core lock, unlock, notify, and log requests. Calls to other plugins fail because no other plugins exist.
|
||||||
|
//! Name filters, `tests/*.rs`, and `--ignored` work. `#[should_panic]` cannot work because WASI cannot catch panics.
|
||||||
|
//!
|
||||||
|
//! ```no_run
|
||||||
|
//! use sausage_plugin::{lock, state_dir, write_atomic};
|
||||||
|
//! #[test]
|
||||||
|
//! fn reads_package_fixture() {
|
||||||
|
//! let path = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("Cargo.toml");
|
||||||
|
//! assert!(std::fs::read_to_string(path).unwrap().contains("[package]"));
|
||||||
|
//! }
|
||||||
|
//! #[sausage_plugin::test]
|
||||||
|
//! async fn writes_local_state() {
|
||||||
|
//! let file = state_dir().join("test.txt");
|
||||||
|
//! std::fs::create_dir_all(state_dir()).unwrap();
|
||||||
|
//! let _guard = lock(&file).await.unwrap();
|
||||||
|
//! write_atomic(&file, b"saved").unwrap();
|
||||||
|
//! assert_eq!(std::fs::read(file).unwrap(), b"saved");
|
||||||
|
//! }
|
||||||
|
//! fn main() {}
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! # Errors
|
||||||
|
//!
|
||||||
|
//! [`Error`] carries `code`, `message`, and optional structured `data`.
|
||||||
|
//! `Error::new` constructs an error without data. Assign `data` when callers need structured details.
|
||||||
|
//! Do not reduce an error to its message when another agent needs its code or data.
|
||||||
|
//!
|
||||||
|
//! | Code | Meaning |
|
||||||
|
//! | --- | --- |
|
||||||
|
//! | -32700 | Invalid JSON. |
|
||||||
|
//! | -32600 | Invalid JSON-RPC request. |
|
||||||
|
//! | -32601 | Unknown method. |
|
||||||
|
//! | -32602 | Invalid method parameters. |
|
||||||
|
//! | -32603 | Internal error, including converted I/O and serde errors. |
|
||||||
|
//! | -32000 | Local SDK error: no transport, stdin EOF, endpoint closure, or identity or interface mismatch. |
|
||||||
|
//! | -32001 | Core: no such plugin. |
|
||||||
|
//! | -32002 | Core: the process ended without an answer. Data includes its exit status and journal tail. |
|
||||||
|
//! | -32003 | Core: activation failed. Data describes the reasons. |
|
||||||
|
//!
|
||||||
|
//! Parameter decoding failures use -32602 instead of the general serde conversion code.
|
||||||
|
//!
|
||||||
|
//! ```
|
||||||
|
//! use sausage_plugin::Error;
|
||||||
|
//! use sausage_plugin::serde_json::json;
|
||||||
|
//! let mut error = Error::new(-32602, "unknown secret");
|
||||||
|
//! error.data = Some(json!({"name":"missing"}));
|
||||||
|
//! assert_eq!(error.code, -32602);
|
||||||
|
//! assert_eq!(error.data.unwrap()["name"], "missing");
|
||||||
|
//! ```
|
||||||
|
//!
|
||||||
|
//! # State and upgrades
|
||||||
|
//!
|
||||||
|
//! The interface check covers calls, not state. Old and new processes share files in `/state`.
|
||||||
|
//! `core.revert` restores code, not state. Preserve compatibility or migrate the state.
|
||||||
|
//! `#[rpc(grace_seconds = N)]` declares the grace period for old processes. The default is 60 seconds.
|
||||||
|
//! New calls use the new version immediately. Old processes can still use old imports during their grace period.
|
||||||
|
//! [`macro@on_retired`] handles `rpc.retired` after the last process of an old version ends.
|
||||||
|
//! This hook is best effort. If the host stops first, the hook does not run.
|
||||||
|
//! New processes must handle unmigrated state. Check a migration marker and migrate if it is absent.
|
||||||
|
//! See section 3.7 of `/proc/core/PROTOCOL.md`.
|
||||||
|
//! Declare at most one stop hook and one retirement hook per interface.
|
||||||
|
//!
|
||||||
|
//! ```no_run
|
||||||
|
//! use sausage_plugin::{rpc, schema, state_dir, write_atomic, Result};
|
||||||
|
//! /// A plugin that records retirement notices.
|
||||||
|
//! struct Migrator;
|
||||||
|
//! /// The version that no longer has running processes.
|
||||||
|
//! #[schema]
|
||||||
|
//! struct RetiredParams {
|
||||||
|
//! /// The SHA-256 of the previous WASM file.
|
||||||
|
//! previous: String,
|
||||||
|
//! }
|
||||||
|
//! #[rpc]
|
||||||
|
//! impl Migrator {
|
||||||
|
//! /// Record a retired version. Normal methods must also handle missing markers.
|
||||||
|
//! #[sausage_plugin::on_retired]
|
||||||
|
//! fn retired(&self, p: RetiredParams) -> Result<()> {
|
||||||
|
//! std::fs::create_dir_all(state_dir())?;
|
||||||
|
//! write_atomic(state_dir().join("retired-version"), p.previous.as_bytes())
|
||||||
|
//! }
|
||||||
|
//! }
|
||||||
|
//! fn main() { sausage_plugin::run(Migrator); }
|
||||||
|
//! ```
|
||||||
#![warn(missing_docs)]
|
#![warn(missing_docs)]
|
||||||
mod discovery;
|
mod discovery;
|
||||||
mod endpoints;
|
mod endpoints;
|
||||||
|
|||||||
Reference in New Issue
Block a user