diff --git a/PROPOSAL.md b/PROPOSAL.md new file mode 100644 index 0000000..0046b06 --- /dev/null +++ b/PROPOSAL.md @@ -0,0 +1,737 @@ +# Proposal: plugins, processes, and JSON-RPC + +Status: a proposal on the branch `plugins`. When we accept it, its content +goes into `PROTOCOL.md` (the contract) and `DECISIONS.md` (the reasons). +Then this file goes away. + +## 1. Why + +Sausage was a world of actors: long-running programs that send each other +fire-and-forget messages through a broker. Agents found this model hard. + +- A reply was a new message. The agent had to match `reply_to` values, wait + with `wait_seconds`, and sort "other messages". +- Mailboxes, subscriptions, taps, watches, versions, and restarts were all + part of the contract. + +This proposal replaces the model with three ideas that models already know: + +1. **JSON-RPC calls.** A call has a result, like an LLM tool call. LSP and + MCP use the same protocol. +2. **Processes.** Each call to a plugin starts a new WASI process. WASI has no + threads, so processes are the unit of parallelism. Calls are asynchronous, + so one process can make many calls at the same time. +3. **Files to inspect, calls to act.** The core shows its state as virtual + files in `/proc`, like Plan 9 and Linux. Each change of state is a call. + +## 2. Terms + +| Term | Meaning | +| --- | --- | +| world | The folder that a `sausage` host runs. It is `/` for every process. | +| core | The host. It runs processes, routes calls, serves the virtual files, and keeps the journal. | +| plugin | A folder `/plugins//` with a `plugin.wasm`. A plugin does not run by itself. | +| process | One run of a plugin. Its ID is `-`, for example `notes-0197f3a2-7c1e-7d2b-9a41-5f0c2e8b1d34`. | +| call | A JSON-RPC request. `notes.add` calls the method `add` of the plugin `notes`. | +| interface | A set of methods. A plugin has one main interface. It can also declare endpoint interfaces. | +| endpoint | An interface that a running process serves on a TCP port. An endpoint file tells others where it is. | +| import | An interface of another plugin that a plugin uses. The SDK generates it from that plugin's description. | +| provider, consumer | The plugin that has an interface, and a plugin that imports it. | +| journal | The central log of the world. | + +## 3. The world folder + +```text +/ + plugins// + plugin.wasm the current version + plugin.prev.wasm the version before, for `core.revert` + source/ the source. `cargo.build` builds it from here. + imports/.json descriptions of the imported plugins + state// persistent state of a plugin, by convention + state/core/journal.sqlite the journal + config/ global configuration + boot.json calls that the core makes at start + agent.json API, base URL, model, and token of the agent +``` + +Every process can read and write all of it. There are no permissions. + +## 4. Calls + +### 4.1 The protocol + +- JSON-RPC 2.0 on stdin and stdout, one message per line, as in MCP's stdio + transport. +- `params` is always an object. Batches are not supported. +- A line on stdout that is not a JSON-RPC message goes to the journal. The + core continues at the next line. The SDK writes a newline before each + message, so a `print!` without a newline cannot damage a message. +- Each line on stderr goes to the journal. + +### 4.2 Method names + +- A method name has the form `.`. The core splits it at the + first `.`. +- Plugin names use `a-z`, `0-9`, `-`, and `_`. The name `core` is reserved. +- `notes.add` starts a new process of `notes` and sends it the request `add`. + `notes.rpc.discover` sends it `rpc.discover`. +- `core.` is a method of the core (section 9). + +### 4.3 What a call does + +1. The core gives the call a UUIDv7 and starts the process `-`. +2. The core writes the request to the stdin of the process. The request ID is + the process ID. +3. The process writes the response to stdout. The core sends the response to + the caller, with the caller's request ID. +4. If the process ends before it answers, the caller gets the error -32002. + Its `data` has the exit status and the last journal lines of the process. + +A process can write calls to stdout at any time, to the core and to plugins. +The responses come on stdin. A process can have many calls in flight. + +### 4.4 Detached calls + +`core.call {"method": "www.start", "params": {...}, "detach": true}` makes the +call and returns its result. The new process does not end with its caller. + +Sent as a notification (without an ID), the same `core.call` starts the +process and forgets it. + +### 4.5 Notifications to the caller + +A process can tell its caller about progress. It sends the notification +`core.notify {"name": "text", "params": {...}}`. The core gives it to the +caller as `core.notify {"call": , "name": "text", +"params": {...}}`. After the call has ended, the core drops such +notifications. + +So a process never needs to know who called it. + +### 4.6 Lifetime + +- A process ends when `main` returns. The core keeps its stdin open until + then. +- EOF on stdin means "stop now". +- `core.stop {pid}` closes stdin, waits 5 seconds, and then kills the + process. `core.kill {pid}` kills it at once. +- When a process ends, the core stops each of its attached children with + `core.stop`. +- A detached process has no parent. Only `core.stop`, `core.kill`, and the + end of the host stop it. +- At host shutdown, the core stops every process. + +Note: on Unix, children outlive their parent. Here, an attached child ends +with its parent. Only `detach` makes a process independent. + +### 4.7 Environment and network + +- `SAUSAGE_PID`: the process ID. +- `SAUSAGE_PLUGIN`: the plugin name. +- `SAUSAGE_PARENT`: the parent: `console`, `core`, or a process ID. + +Every process gets the network: outgoing HTTP and HTTPS through `wasi:http`, +and TCP, UDP, and DNS through `wasi:sockets`. + +### 4.8 Errors + +The standard JSON-RPC codes apply. The core adds these codes: + +| Code | Meaning | +| --- | --- | +| -32001 | There is no such plugin. | +| -32002 | The process ended without an answer. `data` has the exit status and the last journal lines. | +| -32003 | The activation was rejected. `data` has the reasons. | + +## 5. The filesystem + +Each process gets four preopens: + +| Path | Backend | Access | +| --- | --- | --- | +| `/` | the world folder | read-write | +| `/proc` | virtual, served by the core | read-only | +| `/self` | virtual: the folder `/proc//` | read-write | +| `/run` | virtual, shared, in memory, empty at each host start | read-write | + +```text +/proc/ + core/ + PROTOCOL.md the contract + graph.json plugins, imports, and the result of each check + graph.dot the same graph for Graphviz + locks.json held locks and waiting processes + plugins// + discover.json the rpc.discover document of the current plugin.wasm + client.rs the Rust code that import!("") generates + / + info.json plugin, method, params, parent, detached, start time, state + log the journal lines of this process, live + the files that the process writes to /self +``` + +Rules: + +- A process writes its own folder only through `/self`. Writes through + `/proc` always fail. +- In a process folder, the names `info.json`, `log`, and `fs` belong to the + core. A write to `/self/info.json` fails. +- The folder of a process goes away when the process ends. Its files are in + memory. Large or persistent data belongs in `/state`. +- Do not send a `/self` path to another process. There it means a different + folder, like `localhost` on another machine. + +Principle: **inspect through files, act through calls.** A file read is +cheaper than a call, and every language and tool can read a file. + +## 6. Endpoints + +- `listen(path, interface)` binds a free port on 127.0.0.1 and writes an + endpoint file at `path`. The file has one line: the address, the owner, and + the interface. Example: `tcp:127.0.0.1:41234 www-0197… control`. +- Usually the path is in `/self`. A singleton uses a fixed path in `/run`, + for example `/run/secrets`. +- `connect(path)` reads the file. It checks that `/proc/` exists and + that the interface is the expected one. Then it connects. So a stale file + or a reused port cannot connect a client to the wrong process. +- JSON-RPC runs on the connection, one message per line. +- `listen` accepts only interfaces that the plugin declares (section 10.1). + +## 7. Locks + +- `core.lock {"path": "/state/notes/notes.json"}` returns when the process + holds the lock. Processes that wait get the lock in FIFO order. +- With `"wait": false`, it returns at once with `{"held": true}` or + `{"held": false}`. +- `core.unlock {path}` releases the lock. When a process ends, the core + releases all of its locks, also after a crash. +- Locks are advisory. The core does not block writes to the file: the path + is only a name. +- SQLite has no file locks in WASI. Hold the lock while the database is open. + +## 8. The journal + +- An SQLite file at `/state/core/journal.sqlite`. Only the core writes it. +- Each entry has a UUIDv7 (it sorts by time), the process ID, the plugin, the + stream (`stderr`, `stdout`, or `core`), and the line. +- The core writes its own events too: start (with method and params), exit + status, stop, and kill. +- `core.log {plugin?, pid?, since?, until?, grep?, limit?}` queries the + journal. `/proc//log` shows the lines of one running process. +- Later: structured logging, like `tracing`. + +## 9. The core API + +| Method | Params | Result | +| --- | --- | --- | +| `core.call` | `method`, `params`, `detach`? | the result of the method | +| `core.notify` | `name`, `params` | none (a notification) | +| `core.stop` | `pid` | `{}` when the process has ended | +| `core.kill` | `pid` | `{}` | +| `core.lock` | `path`, `wait`? | `{"held": bool}` | +| `core.unlock` | `path` | `{}` | +| `core.activate` | `plugin` and `wasm`, or `plugins: [{plugin, wasm}]`. `force`? | the new `rpc.discover` documents | +| `core.revert` | `plugin`, `force`? | like `core.activate` | +| `core.log` | filters (section 8) | journal entries | +| `core.exit` | none | `{}`. Then the host stops with exit code 75. | + +Everything else is a file. + +## 10. Interfaces + +### 10.1 `rpc.discover` + +Each plugin answers `rpc.discover` with an OpenRPC document. The SDK +generates it from the code, with all doc comments. It has two extension +fields: + +- `x-sausage-endpoints`: the endpoint interfaces by name, each with its + methods. +- `x-sausage-imports`: the methods that the plugin uses from each provider, + with their schemas. + +```json +{ + "openrpc": "1.2.6", + "info": {"title": "www", "version": "0.1.0"}, + "methods": [{ + "name": "start", + "summary": "Start the web server and return the ID of its process.", + "paramStructure": "by-name", + "params": [{"name": "port", "required": true, + "description": "The TCP port to listen on, on all interfaces.", + "schema": {"type": "integer", "minimum": 0, "maximum": 65535}}], + "result": {"name": "pid", "schema": {"type": "string"}} + }], + "x-sausage-endpoints": {"control": {"methods": [{"name": "stats", "…": "…"}]}}, + "x-sausage-imports": {"notes": {"methods": [{"name": "add", "…": "…"}]}} +} +``` + +### 10.2 Compatibility + +A provider can add methods, optional params, and result fields. For each +imported method: + +1. The provider has the method. +2. Each params value that the consumer can send is valid for the provider. +3. Each result that the provider can return is valid for the consumer. + +The same rules apply to endpoint interfaces. There are no version numbers. +This check replaces them. + +The check uses `jsoncompat` if it is good enough. Otherwise we write our own +check for the subset of JSON Schema that `schemars` writes. An answer that +the check cannot prove counts as a failure. + +### 10.3 Activation + +`core.activate {"plugin": "notes", "wasm": "/state/cargo/builds/7/notes.wasm"}`: + +1. The core starts the file and calls `rpc.discover`. +2. It checks the imports of the new version against the current providers. +3. It checks each current consumer of the plugin against the new version. +4. If all checks pass, `plugin.wasm` becomes `plugin.prev.wasm`, and the + file becomes `plugin.wasm`. + +If a check fails, nothing changes. The error lists each problem, with a +counterexample where possible: + +```text +notes was not activated: 1 consumer breaks. +- agent imports notes.add(text: string) -> Added. + The new notes.add has no param "text". It requires "body": string. + Counterexample: {"text": "milk"} is not valid for the new params. +Update agent first, or activate both together. +``` + +- `plugins: [...]` activates several plugins together. The checks then use + the new set. +- `force: true` skips steps 2 and 3. +- `core.revert {plugin}` activates `plugin.prev.wasm` in the same way, and + the current version becomes `plugin.prev.wasm`. +- Running processes keep their version until they end. New calls get the new + version. +- A person can also copy a file to `plugin.wasm` by hand. That skips the + checks (section 10.4). + +### 10.4 The graph + +- The core keeps the `rpc.discover` documents of all plugins in memory, by + file hash. At start, it gets them in the background. +- A file watcher on `/plugins/*/plugin.wasm` updates them, for example after + a copy by hand. Such a copy skips the checks. `graph.json` then shows each + broken import, and the journal records it. +- An import of a plugin that does not exist is "missing". +- Untyped calls (`call("x.y", ...)`) are not in the graph. + +## 11. Building and tests + +`cargo.build {"plugin": "notes"}` is a native method of the host. A call +still gets a process ID, a folder in `/proc`, and journal entries, like other +calls. The method does these steps: + +1. It runs the tests of `/plugins/notes/source/` under WASI. +2. It builds the release `.wasm` for `wasm32-wasip2`. +3. It returns `{"wasm": "/state/cargo/builds//notes.wasm", "tests": + {"passed", "failed", "output"}, "warnings"}`. If a step fails, it returns + an error with the compiler or test output. + +- The dependency `sausage-plugin = "0.1"` points to the SDK copy at + `/plugins/cargo/sdk/`. +- Build scripts can compile C: `CC_wasm32_wasip2` and `AR_wasm32_wasip2` + point to a WASI clang. +- The warnings include unused imported methods. Remove them from + `imports/.json`, so that fewer changes of the provider break the + plugin. + +Tests: + +- `sausage` is the test runner (`CARGO_TARGET_WASM32_WASIP2_RUNNER`). It runs + each test in its own instance, in parallel. A panic ends only its own test. + Without this, the first panic would end the whole test run, because WASI + has `panic=abort`. +- The normal Rust test setup works: `#[test]`, `tests/*.rs`, name filters, + and `--ignored`. Async tests use `#[sausage_plugin::test]`, like + `#[tokio::test]`. `#[should_panic]` cannot work, because WASI cannot catch + a panic. +- Each test gets an empty temporary world as `/`, the package folder as its + current folder, the variables that cargo sets, and its own `/proc`, + `/self`, and `/run`. +- The runner answers `core.lock`, `core.unlock`, `core.notify`, and + `core.log`. A call to a plugin fails with a clear error: a test has no + other plugins. + +## 12. Boot + +At start, the core makes each call in `/config/boot.json`, detached: + +```json +[{"method": "www.start", "params": {"port": 8080}}] +``` + +## 13. The SDK + +Two crates: `sausage-plugin`, and `sausage-plugin-macros`, which +`sausage-plugin` re-exports. The SDK also re-exports `serde` and `schemars` +for the macros. + +| Item | Purpose | +| --- | --- | +| `#[rpc]` | Makes an `impl` an interface. `#[rpc(endpoints(name = Type))]` declares endpoint interfaces. | +| `#[schema]` | For each type that a method takes or returns: adds the serde and schemars derives. | +| `run(T)` | In `main`: serves the main interface on stdin and stdout. | +| `call(method, params)` | An untyped call. Returns a JSON value. | +| `.detach()`, `.on_notify(f)` | Options of a call. | +| `Ctx` | An optional second argument of a method: `ctx.notify(name, params)`. | +| `lock(path)` | A lock guard. `.try_now()` does not wait. `.keep()` holds the lock until the process ends. | +| `listen(path, value)`, `connect(path)` | Endpoints. | +| `background(future)` | Runs a task in this process. The process lives while a task runs. | +| `import!("name")` | Generates `mod name` from `imports/name.json`. | +| `state_dir()`, `write_atomic(path, bytes)`, `pid()`, `endpoint_owner(path)`, `exit_after_response()` | Helpers. | +| `#[sausage_plugin::test]` | An async test. A plain `#[test]` also works. | + +Rules that the compiler enforces: + +- A method takes `&self`, an optional `ctx: &Ctx`, and at most one params + argument. +- The params argument is a `#[schema]` struct with named fields. Its fields + are the params of the method. +- Each method, each `#[schema]` type, and each field has a doc comment. + Other agents read these docs to use the interface. Write them like good + Rust documentation. + +Method docs become `summary` (the first paragraph) and `description` in +OpenRPC. Type and field docs become `description` in the JSON Schema. The +generated imports contain them again as doc comments. + +### 13.1 A plugin + +```toml +[package] +name = "notes" +version = "0.1.0" +edition = "2024" + +[dependencies] +sausage-plugin = "0.1" +serde_json = "1" +``` + +```rust +//! Notes: a list of short texts, stored in `/state/notes/notes.json`. +//! +//! Each call runs in its own process, so two calls can run at the same time. +//! `add` takes a lock on the file, so that no note gets lost. + +use std::path::PathBuf; + +use sausage_plugin::{Result, lock, rpc, schema, state_dir, write_atomic}; + +/// The notes plugin. It keeps no state in memory: each call reads the file. +struct Notes; + +/// The params of `add`. +#[schema] +struct AddParams { + /// The text of the new note. It can contain newlines. + text: String, +} + +/// The result of `add`. +#[schema] +struct Added { + /// The number of notes after the call, with the new note. + count: usize, +} + +#[rpc] +impl Notes { + /// Add a note at the end of the list. + /// + /// Returns the number of notes after the call. + async fn add(&self, AddParams { text }: AddParams) -> Result { + let _lock = lock(file()).await?; // released on drop, or when the process ends + let mut notes = load()?; + notes.push(text); + write_atomic(file(), serde_json::to_vec(¬es)?)?; + Ok(Added { count: notes.len() }) + } + + /// List all notes, the oldest first. + fn list(&self) -> Result> { + load() // no lock: write_atomic replaces the file in one step + } +} + +/// The file with the notes: a JSON list of strings. +fn file() -> PathBuf { + state_dir().join("notes.json") +} + +/// Read all notes. A missing file means no notes. +fn load() -> Result> { + match std::fs::read(file()) { + Ok(bytes) => Ok(serde_json::from_slice(&bytes)?), + Err(e) if e.kind() == std::io::ErrorKind::NotFound => Ok(vec![]), + Err(e) => Err(e.into()), + } +} + +fn main() { + sausage_plugin::run(Notes); +} + +#[cfg(test)] +mod tests { + use super::*; + + #[sausage_plugin::test] + async fn add_appends_and_counts() { + assert_eq!(Notes.add(AddParams { text: "milk".into() }).await.unwrap().count, 1); + assert_eq!(Notes.add(AddParams { text: "eggs".into() }).await.unwrap().count, 2); + assert_eq!(Notes.list().unwrap(), ["milk", "eggs"]); + } + + #[test] + fn list_is_empty_without_a_file() { + assert!(Notes.list().unwrap().is_empty()); + } +} +``` + +### 13.2 A server with an endpoint + +```rust +//! A web server for the files in `/state/www/`. +//! +//! `start` starts the server in its own process and returns at once. Call it +//! detached, or the server ends with its caller. The running server serves +//! the endpoint `control` at `/proc//control`. + +use std::{cell::Cell, rc::Rc}; + +use sausage_plugin::{Result, background, listen, pid, rpc, schema}; +use wstd::net::TcpListener; + +/// The www plugin. +struct Www; + +/// The params of `start`. +#[schema] +struct StartParams { + /// The TCP port to listen on, on all interfaces. + port: u16, +} + +#[rpc(endpoints(control = Control))] +impl Www { + /// Start the web server and return the ID of its process. + /// + /// The server runs until it gets `stop` on its endpoint, until + /// `core.stop`, or until its caller ends. Fails if the port is in use. + async fn start(&self, StartParams { port }: StartParams) -> Result { + let tcp = TcpListener::bind(&format!("0.0.0.0:{port}")).await?; + let control = Rc::new(Control { hits: Cell::new(0) }); + listen("/self/control", control.clone()).await?; + background(serve(tcp, control)); // the process lives while this task runs + Ok(pid()) + } +} + +/// The control endpoint of a running server. +struct Control { + /// The number of HTTP requests that the server answered. + hits: Cell, +} + +#[rpc] +impl Control { + /// The number of HTTP requests that the server answered since its start. + fn stats(&self) -> Result { + Ok(self.hits.get()) + } + + /// Stop the server. The process ends after this response. + fn stop(&self) -> Result<()> { + sausage_plugin::exit_after_response(); + Ok(()) + } +} + +/// Accept HTTP connections, and answer each request from `/state/www/`. +async fn serve(tcp: TcpListener, control: Rc) { + // … +} + +fn main() { + sausage_plugin::run(Www); +} +``` + +The same type can serve stdin and an endpoint. The example uses two types +for a reason: `www.stats` through the core would start a fresh process, and +that process would answer 0. + +### 13.3 A singleton + +```rust +/// Start the secrets service, or return the ID of the one that runs. +async fn start(&self) -> Result { + let Some(lock) = lock("/run/secrets").try_now().await? else { + return sausage_plugin::endpoint_owner("/run/secrets"); // another instance runs + }; + lock.keep(); // hold the lock until this process ends + listen("/run/secrets", Secrets::open()?).await?; + Ok(pid()) +} +``` + +### 13.4 Imports + +To use another plugin, an agent does these steps: + +1. Copy `/proc/core/plugins/notes/discover.json` to + `source/imports/notes.json`. +2. Read `/proc/core/plugins/notes/client.rs`: the exact code that the import + generates. +3. Write `import!("notes")` and the code that uses it. +4. Call `cargo.build`, and remove the imported methods that the warnings call + unused. +5. Call `core.activate`. + +```rust +sausage_plugin::import!("notes"); // imports/notes.json +sausage_plugin::import!("www"); // imports/www.json, with its endpoint "control" + +/// Add a note, then start a web server and return its number of answered requests. +async fn example() -> Result { + let added = notes::add(notes::AddParams { text: "milk".into() }).await?; + eprintln!("{} notes", added.count); // stderr goes to the journal + let pid = www::start(www::StartParams { port: 8080 }).detach().await?; + let control = www::control::connect(&format!("/proc/{pid}/control")).await?; + control.stats().await +} +``` + +- The params of each method are one struct. The fields have the same names as + the JSON params. +- Nested types take their names from `$defs`. When the provider uses + `schemars`, those are the names of its Rust types: `AddParams` stays + `AddParams`. +- The generated items are private to the crate, so rustc warns about each + unused method. + +## 14. The agent + +- `agent.chat {"text": "..."}` answers a message from a person. The result is + the final text. The interim texts come as `core.notify` with the name + `text`. +- The agent takes a lock on `/state/agent/history.jsonl`. So a second message + waits until the first turn has ended. +- The tools: `call` (method, params, detach), `read_file`, `write_file`, + `list_dir`, and `copy_dir`. +- The configuration is in `/config/agent.json`. +- The prompt stays short. It says where the contract is + (`/proc/core/PROTOCOL.md`), where the SDK is (`/plugins/cargo/sdk/`), how + to build and activate, and where the agent's own source is + (`/plugins/agent/source/`). +- A new version of the agent applies at the next `chat`. The current turn + ends with the old code. + +## 15. The console + +- `rustyline` gives line editing and history. Notifications appear while you + type. +- A short prompt. Results are indented. Agent notifications are dimmed. + +| Input | Effect | +| --- | --- | +| text | `agent.chat` | +| `. {json}` | a call. The console prints the result. | +| `ls [path]`, `cat ` | the filesystem as a process sees it, with `/proc` and `/run` | +| `methods ` | the methods from `rpc.discover`, with params and docs | +| `ps` | the running processes | +| `log [plugin or pid] [-n N]` | the journal | +| `activate `, `revert ` | `core.activate`, `core.revert` | +| `stop `, `kill ` | `core.stop`, `core.kill` | +| `help`, `quit` | | + +`--headless` stays: no console input, and the host runs until SIGTERM or +`core.exit`. + +## 16. Caches + +- The compiled code of each `.wasm` file is cached in + `$XDG_CACHE_HOME/sausage/`, or `~/.cache/sausage/` without that variable. + `--cache-dir` changes the folder. +- The key is the SHA-256 of the file and Wasmtime's engine hash. So a + Wasmtime upgrade never loads old machine code. +- Wasmtime has a compile cache of its own. If it supports components, we use + it. If not, our own cache needs about 20 lines. + +## 17. What goes away + +The broker, mailboxes, publish and subscribe, taps, `watch` and +`fs_changed`, actor-served files, `versions/`, the JSON transcript tests, +restarts and the rollback after a crash loop, `/actor`, `/data`, `/public`, +the `sausage-actor` SDK, and the automatic install of a changed +`actor.wasm`. There is no compatibility with old worlds. + +## 18. Decisions + +- **JSON-RPC, not component interfaces.** Every language can read and write + lines of JSON, and models know JSON-RPC from LSP and MCP. +- **A process per call.** We get parallelism without threads. A new version + applies at the next call, so the core needs no restarts and no rollback. +- **Attached by default.** A process does not outlive its caller by + accident. +- **Files to inspect, calls to act.** This keeps the core API small, and the + agent's file tools are enough to explore the world. +- **Generated imports and interface checks instead of versions.** A model + that translates a description by hand makes errors. A generator does not. + The check catches the most common error in a world that changes itself: a + change to one plugin breaks another one. +- **Docs are part of the interface.** Other agents work from them, so the + compiler requires them. +- **No permissions.** The checks protect against mistakes, not against + plugins. +- **Our own VFS.** It works on Linux and macOS, and plugin-served files need + it later. +- **TCP endpoints for now.** They need no host code. VFS pipes can replace + them later behind the same SDK API. + +## 19. Later + +- Plugin-served files at `/proc//fs/`, for example for an FTP plugin. A + request gets a timeout of 10 seconds, because a process that reads its own + served files would wait for itself forever. +- VFS pipes for endpoints, instead of TCP ports. +- Subscriptions to file changes. +- Structured logging. +- A secrets service. +- Cancellation of calls. + +## 20. Plan + +The work happens on the branch `plugins`. Each step includes its tests. + +First, three checks of risks: + +1. The round trip of `typify`, schemars 1.x, and `jsoncompat` on real + documents. The result decides between `jsoncompat` and our own check. +2. Virtual preopens in `wasmtime-wasi`: add `/proc`, `/self`, and `/run` to + the preopen list, and send each descriptor under them to our backend. +3. Whether Wasmtime's compile cache supports components. + +Then the steps: + +1. `PROTOCOL.md` and `DECISIONS.md` from this proposal. +2. The core: a process per call, JSON-RPC routing, the process tree, locks, + the journal, and the compile cache. +3. The VFS: `/proc`, `/self`, `/run`, and `/proc/core/`. +4. The SDK and its macros. +5. `cargo.build`, the test runner, activation, and the graph. +6. The agent, the console, and boot. +7. The removal of the old code.