Compare commits

...
11 Commits
Author SHA1 Message Date
Sausage Agent a72e42a8c9 headless: register signal handlers synchronously, split tests, drop deployment examples and wording
ci / check (pull_request) Successful in 1m36s
ci / check (push) Successful in 11s
2026-10-18 15:31:41 +00:00
Sausage Agent 01a821ab72 Add --headless mode (no stdin read, runs until SIGTERM/exit) + systemd/NixOS examples
ci / check (push) Successful in 2m44s
2026-10-18 15:24:20 +00:00
Sausage Agent c204e1ef0c docs: clarify actor data is world-readable
ci / check (pull_request) Successful in 12s
ci / check (push) Successful in 6s
2026-10-17 17:22:50 +00:00
Sausage Agent 6923ca3e60 preserve the packaged runtime Rust toolchain
ci / check (push) Successful in 6s
2026-10-17 16:36:08 +00:00
Sausage Agent c504532513 drive packaged host runtime check over pipes
ci / check (push) Failing after 1m16s
2026-10-17 16:33:31 +00:00
Sausage Agent eb35a79698 trace the packaged host runtime check
ci / check (push) Failing after 1m47s
2026-10-17 16:29:02 +00:00
Sausage Agent bfcd71e2f4 test packaged host cargo runtime with an empty PATH
ci / check (push) Failing after 1m33s
2026-10-17 16:25:14 +00:00
agent-pool-12 172a776eec cargo: annotate test summary count parsing
ci / check (push) Successful in 5s
2026-10-17 16:00:54 +00:00
agent-pool-12 6a9d35ceb8 cargo: preserve build failures in result wrapper
ci / check (push) Failing after 6m15s
2026-10-17 15:52:37 +00:00
agent-pool-12 22590c608d cargo: run host unit tests before builds
ci / check (push) Failing after 8m51s
2026-10-17 15:50:09 +00:00
Sausage Agent 3bc462ffd2 agent seed: thinking levels low|medium|high|xhigh|max|off on both APIs (card #128)
ci / check (push) Successful in 5s
New config field `thinking` (OpenAI: reasoning_effort still wins): Anthropic adaptive thinking +
output_config.effort (thinking blocks kept and sent back), OpenAI reasoning.effort. A 400 that rejects
the effort is retried with the nearest accepted level (from the error's list, lower on a tie), logged,
never dropped to off.
2026-10-17 15:28:44 +00:00
13 changed files with 512 additions and 87 deletions
+19 -5
View File
@@ -63,10 +63,12 @@ Each instance sees three folders:
| Path | Access | Contents | | Path | Access | Contents |
| --- | --- | --- | | --- | --- | --- |
| `/data` | read-write | the actor's state | | `/data` | read-write by its actor; readable by every actor | the actor's state |
| `/public` | read-write | the files that the actor publishes | | `/public` | read-write | the files that the actor publishes |
| `/actor` | read-only | the `actor/` folder of the world | | `/actor` | read-only | the `actor/` folder of the world |
An actor can write only its own `/data`, but can read every actor's `/data` through `/actor/<id>/data/`. `/data` is not private: any unprotected secret file there is readable by every actor.
`/actor/<id>/` holds everything about an actor: `actor.wasm`, `actor.json` `/actor/<id>/` holds everything about an actor: `actor.wasm`, `actor.json`
(installed version, running, subscriptions, tapped, watches), `source/`, `public/`, `data/`, (installed version, running, subscriptions, tapped, watches), `source/`, `public/`, `data/`,
`stderr.log`, and `versions/<n>/`. `ls /actor` lists all actors. `stderr.log`, and `versions/<n>/`. `ls /actor` lists all actors.
@@ -130,7 +132,9 @@ the request: the request's `type` in past tense (`spawned`, `upgraded`,
| `subscribe`, `unsubscribe` | `actor` | receive (or not) what `actor` publishes | | `subscribe`, `unsubscribe` | `actor` | receive (or not) what `actor` publishes |
| `tap`, `untap` | | receive (or not) a copy of every message the broker routes | | `tap`, `untap` | | receive (or not) a copy of every message the broker routes |
| `watch`, `unwatch` | `path` | receive (or not) `fs_changed` for every change of `path` or anything under it; answer `{"type": "watching" \| "unwatched", "path"}` with the path as an `/actor/...` path | | `watch`, `unwatch` | `path` | receive (or not) `fs_changed` for every change of `path` or anything under it; answer `{"type": "watching" \| "unwatched", "path"}` with the path as an `/actor/...` path |
| `exit` | | answer `exiting`, stop every actor, and end the `sausage` process (a wrapper like `nix run .#loop` starts it again, updated) | | `exit` | | answer `exiting`, stop every actor, and end the `sausage` process with exit code 75 (a wrapper like `nix run .#loop` starts it again, updated) |
**Headless.** `sausage run --headless [dir]` (or `sausage --headless`, or `SAUSAGE_HEADLESS=1`) turns the console input off: the host does not read stdin, so end of input (`/dev/null`) does not stop the world. `nix run .#loop -- --headless <world>` passes it through the loop. It runs until SIGTERM or SIGINT (stops every actor, exit code 0) or a broker `exit` request (exit code 75, as above). Without the flag, EOF on stdin or `/quit` ends the host.
Paths are `/actor/...`, or the sender's `/public/...` or `/data/...`. Paths are `/actor/...`, or the sender's `/public/...` or `/data/...`.
@@ -231,9 +235,19 @@ folder again, as at start.
{"type": "build", "source": "/actor/agent/public/drafts/notes"} {"type": "build", "source": "/actor/agent/public/drafts/notes"}
``` ```
The reply is `{"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", "built_by": "cargo", ...}` The build request may include `"skip_tests": true` to skip Rust unit tests.
or `{"type": "build_failed", "errors": "..."}`. Pass `wasm`, `source`, and By default, `cargo` first runs `cargo test` for the host target (not WASI),
`built_by` on to `spawn` or `upgrade`. so pure-logic unit tests in a `wasm32-wasip2` actor crate can run without a
WASI test runner. If tests fail or cannot compile, the build fails before the
WASI build. The reply carries `tests: {"passed": <count>, "failed": <count>,
"output": "<cargo test stdout and stderr>"}`; counts sum Cargo's test-result
summaries, and a test command/compilation failure without a summary counts as
one failure. With `skip_tests: true`, counts are zero and output says tests
were skipped.
The reply is `{"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", "built_by": "cargo", "tests": {...}, ...}`
or `{"type": "build_failed", "errors": "...", "tests": {...}}`. Pass
`wasm`, `source`, and `built_by` on to `spawn` or `upgrade`.
A Rust actor depends on the SDK with `sausage-actor = "0.1"`. `cargo` points A Rust actor depends on the SDK with `sausage-actor = "0.1"`. `cargo` points
that dependency at the SDK of the host, which it publishes at that dependency at the SDK of the host, which it publishes at
+13 -1
View File
@@ -48,6 +48,18 @@ From a clone (`git clone https://gitea.surma.technology/surma/sausage`), the
same commands are `nix run`, `nix run -- init`, `nix run -- run` and same commands are `nix run`, `nix run -- init`, `nix run -- run` and
`nix run .#loop`. `nix run .#loop`.
### Headless
`--headless` (or `SAUSAGE_HEADLESS=1`) turns the console input off: the host
never reads stdin, so end of input does not stop the world. It runs until
SIGTERM/SIGINT (clean stop, exit code 0) or a broker `exit` request (exit
code 75, so the loop starts the newest version). Create the world first
(`init`), then pass the flag through the loop:
```sh
nix run git+https://gitea.surma.technology/surma/sausage#loop -- --headless my-world
```
In the console: In the console:
```text ```text
@@ -66,7 +78,7 @@ actor.wasm the current version
actor.json installed version, running, subscriptions actor.json installed version, running, subscriptions
source/ the source of the current version source/ the source of the current version
public/ files that the actor publishes public/ files that the actor publishes
data/ the actor's state data/ actor state; owner-writable, world-readable via /actor/<id>/data/ (not private)
stderr.log what the actor and the broker write about it stderr.log what the actor and the broker write about it
versions/<n>/ every installed version: actor.wasm, source/, meta.json versions/<n>/ every installed version: actor.wasm, source/, meta.json
``` ```
+84 -10
View File
@@ -14,11 +14,49 @@ pub struct Config {
pub base_url: String, pub base_url: String,
pub model: String, pub model: String,
pub token: String, pub token: String,
/// Optional reasoning effort for OpenAI models ("low", "medium", ...). /// Optional reasoning effort for OpenAI models only (wins over `thinking` there).
#[serde(default)] #[serde(default)]
pub reasoning_effort: Option<String>, pub reasoning_effort: Option<String>,
#[serde(default)] #[serde(default)]
pub max_tokens: Option<u32>, pub max_tokens: Option<u32>,
/// Thinking effort for both APIs: "low", "medium", "high", "xhigh", "max" (`LEVELS`), or "off"/unset for none.
/// Anthropic: adaptive thinking + output_config.effort; OpenAI: reasoning.effort. A level the model rejects is
/// replaced by the nearest accepted one (logged), never by "off".
#[serde(default)]
pub thinking: Option<String>,
}
/// Thinking levels, lowest first.
pub const LEVELS: [&str; 5] = ["low", "medium", "high", "xhigh", "max"];
/// The level nearest to `want` (lower on a tie) among those a 400 error lists in quotes (else all levels), not
/// yet `tried`. None: nothing left, so the request goes out without an effort field (model default).
fn fallback_effort(want: &str, error: &str, tried: &[String]) -> Option<String> {
let listed: Vec<&str> = error.split('\'').skip(1).step_by(2).filter(|w| LEVELS.contains(w)).collect();
let pool = if listed.iter().any(|l| *l != want) { listed } else { LEVELS.to_vec() };
let at = LEVELS.iter().position(|l| *l == want).unwrap_or(1) as i32;
pool.into_iter()
.filter(|l| *l != want && !tried.iter().any(|t| t == l))
.min_by_key(|l| {
let i = LEVELS.iter().position(|x| x == l).unwrap_or(1) as i32;
((i - at).abs(), i > at)
})
.map(str::to_string)
}
#[cfg(test)]
mod tests {
use super::fallback_effort;
#[test]
fn nearest_accepted_level() {
let anthropic = "HTTP 400: output_config.effort: Input should be 'low', 'medium', 'high' or 'max'";
assert_eq!(fallback_effort("xhigh", anthropic, &["xhigh".into()]).as_deref(), Some("high"));
let openai = "HTTP 400: Invalid value: 'ultra'. Supported values are: 'none', 'minimal', 'low', 'medium', 'high', 'xhigh', and 'max'.";
assert_eq!(fallback_effort("ultra", openai, &["ultra".into()]).as_deref(), Some("medium"));
// No list in the error: step to the neighbour, never to off.
assert_eq!(fallback_effort("max", "HTTP 400: effort max not supported", &["max".into()]).as_deref(), Some("xhigh"));
}
} }
#[derive(Serialize, Deserialize, Clone, Debug)] #[derive(Serialize, Deserialize, Clone, Debug)]
@@ -32,7 +70,7 @@ pub enum Entry {
}, },
ToolCall { id: String, name: String, arguments: String }, ToolCall { id: String, name: String, arguments: String },
ToolResult { id: String, output: String }, ToolResult { id: String, output: String },
/// A provider item that only this API shape understands (OpenAI reasoning). /// A provider item that only this API shape understands (OpenAI reasoning, Anthropic thinking).
Opaque { api: String, item: Value }, Opaque { api: String, item: Value },
} }
@@ -43,10 +81,28 @@ pub struct Tool {
} }
pub async fn complete(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]) -> Result<Vec<Entry>, String> { pub async fn complete(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]) -> Result<Vec<Entry>, String> {
match cfg.api.as_str() { let want = match cfg.api.as_str() {
"openai-responses" => openai(cfg, system, tools, history).await, "openai-responses" => cfg.reasoning_effort.clone().or_else(|| cfg.thinking.clone()),
"anthropic-messages" => anthropic(cfg, system, tools, history).await, _ => cfg.thinking.clone(),
other => Err(format!("unknown API shape {other:?}")), }
.filter(|t| t != "off");
let mut effort = want.clone();
let mut tried = Vec::new();
loop {
let result = match cfg.api.as_str() {
"openai-responses" => openai(cfg, system, tools, history, effort.as_deref()).await,
"anthropic-messages" => anthropic(cfg, system, tools, history, want.is_some(), effort.as_deref()).await,
other => return Err(format!("unknown API shape {other:?}")),
};
match (result, effort.clone()) {
// The model rejected the effort level: retry with the nearest accepted one (thinking stays on).
(Err(e), Some(current)) if e.starts_with("HTTP 400") && e.contains("effort") => {
tried.push(current.clone());
effort = fallback_effort(want.as_deref().unwrap_or(""), &e, &tried);
eprintln!("agent: {} rejected thinking effort {current:?}; retrying with {effort:?}: {e}", cfg.model);
}
(result, _) => return result,
}
} }
} }
@@ -67,7 +123,7 @@ async fn post(url: &str, headers: &[(&str, String)], body: &Value) -> Result<Val
serde_json::from_str(&text).map_err(|e| format!("invalid JSON in response: {e}")) serde_json::from_str(&text).map_err(|e| format!("invalid JSON in response: {e}"))
} }
async fn openai(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]) -> Result<Vec<Entry>, String> { async fn openai(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry], effort: Option<&str>) -> Result<Vec<Entry>, String> {
let mut input = Vec::new(); let mut input = Vec::new();
for e in history { for e in history {
match e { match e {
@@ -104,7 +160,7 @@ async fn openai(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]) -
if let Some(max_tokens) = cfg.max_tokens { if let Some(max_tokens) = cfg.max_tokens {
body["max_output_tokens"] = json!(max_tokens); body["max_output_tokens"] = json!(max_tokens);
} }
if let Some(effort) = &cfg.reasoning_effort { if let Some(effort) = effort {
body["reasoning"] = json!({"effort": effort}); body["reasoning"] = json!({"effort": effort});
} }
let url = format!("{}/v1/responses", cfg.base_url); let url = format!("{}/v1/responses", cfg.base_url);
@@ -143,7 +199,14 @@ async fn openai(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]) -
Ok(out) Ok(out)
} }
async fn anthropic(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]) -> Result<Vec<Entry>, String> { async fn anthropic(
cfg: &Config,
system: &str,
tools: &[Tool],
history: &[Entry],
thinking: bool,
effort: Option<&str>,
) -> Result<Vec<Entry>, String> {
// Merge consecutive blocks of the same role into one message. // Merge consecutive blocks of the same role into one message.
let mut messages: Vec<(String, Vec<Value>)> = Vec::new(); let mut messages: Vec<(String, Vec<Value>)> = Vec::new();
let mut push = |role: &str, block: Value| match messages.last_mut() { let mut push = |role: &str, block: Value| match messages.last_mut() {
@@ -163,6 +226,8 @@ async fn anthropic(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]
Entry::ToolResult { id, output } => { Entry::ToolResult { id, output } => {
push("user", json!({"type": "tool_result", "tool_use_id": id, "content": output})) push("user", json!({"type": "tool_result", "tool_use_id": id, "content": output}))
} }
// Thinking blocks go back unchanged (with their signature): required next to tool use.
Entry::Opaque { api, item } if api == "anthropic-messages" && thinking => push("assistant", item.clone()),
Entry::Opaque { .. } => {} Entry::Opaque { .. } => {}
} }
} }
@@ -171,13 +236,19 @@ async fn anthropic(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]
.iter() .iter()
.map(|t| json!({"name": t.name, "description": t.description, "input_schema": t.parameters})) .map(|t| json!({"name": t.name, "description": t.description, "input_schema": t.parameters}))
.collect(); .collect();
let body = json!({ let mut body = json!({
"model": cfg.model, "model": cfg.model,
"system": system, "system": system,
"messages": messages, "messages": messages,
"tools": tools, "tools": tools,
"max_tokens": cfg.max_tokens.unwrap_or(32000), "max_tokens": cfg.max_tokens.unwrap_or(32000),
}); });
if thinking {
body["thinking"] = json!({"type": "adaptive"});
if let Some(effort) = effort {
body["output_config"] = json!({"effort": effort});
}
}
let url = format!("{}/v1/messages", cfg.base_url); let url = format!("{}/v1/messages", cfg.base_url);
let resp = post( let resp = post(
&url, &url,
@@ -199,6 +270,9 @@ async fn anthropic(cfg: &Config, system: &str, tools: &[Tool], history: &[Entry]
name: block["name"].as_str().unwrap_or_default().to_string(), name: block["name"].as_str().unwrap_or_default().to_string(),
arguments: block["input"].to_string(), arguments: block["input"].to_string(),
}), }),
Some("thinking") | Some("redacted_thinking") => {
out.push(Entry::Opaque { api: "anthropic-messages".into(), item: block })
}
_ => {} _ => {}
} }
} }
+2
View File
@@ -3,3 +3,5 @@ People talk to you through other actors. Your text goes to your subscribers.
Before writing an actor, read `/actor/broker/public/PROTOCOL.md` for messages, the broker, `cargo`, and tests. Before writing an actor, read `/actor/broker/public/PROTOCOL.md` for messages, the broker, `cargo`, and tests.
Read the SDK at `/actor/cargo/public/sdk/` before you write an actor. Read the SDK at `/actor/cargo/public/sdk/` before you write an actor.
You can change your own source at `/actor/agent/source`. You can change your own source at `/actor/agent/source`.
The global filesystem is readable by every actor, including every actor's `/data` at `/actor/<id>/data/`. An actor can write only its own `/data` and `/public`. `/data` is not private: any unprotected secret file there is readable by every actor.
+173 -47
View File
@@ -1,8 +1,9 @@
//! The native `cargo` actor: builds Rust source folders into `.wasm` files. //! The native `cargo` actor: builds Rust source folders into `.wasm` files.
//! //!
//! Request: `{"type": "build", "source": "/actor/<id>/public/<folder>"}` //! Request: `{"type": "build", "source": "/actor/<id>/public/<folder>"}`
//! Replies: `{"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", ...}` //! optional `"skip_tests": true` skips native `cargo test` (tests run by default).
//! `{"type": "build_failed", "errors": "..."}` //! Replies: `{"type": "built", "wasm": "/actor/cargo/public/builds/<n>/<name>.wasm", "tests": {"passed": 1, "failed": 0, "output": "..."}, ...}`
//! `{"type": "build_failed", "errors": "...", "tests": {"passed": 0, "failed": 1, "output": "..."}}`
//! //!
//! The broker knows nothing about Rust. This actor is one way to get a //! The broker knows nothing about Rust. This actor is one way to get a
//! `.wasm` file. Other actors can add other languages. //! `.wasm` file. Other actors can add other languages.
@@ -32,11 +33,49 @@ pub const TARGET: &str = "wasm32-wasip2";
/// The source of the `sausage-actor` SDK. /// The source of the `sausage-actor` SDK.
static SDK: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/../sdk"); static SDK: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/../sdk");
#[derive(Clone, Default, Debug)]
pub struct TestResults {
pub passed: u64,
pub failed: u64,
pub output: String,
}
impl TestResults {
fn value(&self) -> Value {
json!({"passed": self.passed, "failed": self.failed, "output": self.output})
}
}
pub struct BuildOutput { pub struct BuildOutput {
/// Host path of the published `.wasm` file. /// Host path of the published `.wasm` file.
pub wasm: PathBuf, pub wasm: PathBuf,
pub seconds: f64, pub seconds: f64,
pub warnings: String, pub warnings: String,
pub tests: TestResults,
}
#[derive(Debug)]
pub struct BuildFailure {
pub reason: String,
pub tests: TestResults,
}
impl std::fmt::Display for BuildFailure {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
if self.tests.failed > 0 && !self.tests.output.is_empty() {
write!(f, "{}:\n{}", self.reason, self.tests.output)
} else {
f.write_str(&self.reason)
}
}
}
impl std::error::Error for BuildFailure {}
impl From<anyhow::Error> for BuildFailure {
fn from(error: anyhow::Error) -> Self {
Self { reason: format!("{error:#}"), tests: TestResults::default() }
}
} }
/// All builds share one target folder, so they run one at a time. /// All builds share one target folder, so they run one at a time.
@@ -51,51 +90,84 @@ impl Cargo {
Ok(Self { world }) Ok(Self { world })
} }
/// Build the crate in `source` (a host path) and publish the `.wasm` file /// Run native `cargo test` unless opted out, then build and publish the
/// under `/actor/cargo/public/builds/<n>/`. /// crate's `wasm32-wasip2` binary under `/actor/cargo/public/builds/<n>/`.
pub async fn build(&self, source: &Path) -> Result<BuildOutput> { pub async fn build(&self, source: &Path, skip_tests: bool) -> std::result::Result<BuildOutput, BuildFailure> {
if !source.join("Cargo.toml").is_file() { if !source.join("Cargo.toml").is_file() {
bail!("{} has no Cargo.toml", source.display()); return Err(BuildFailure {
reason: format!("{} has no Cargo.toml", source.display()),
tests: TestResults::default(),
});
} }
let started = Instant::now(); let started = Instant::now();
let data = self.world.data_dir(ID); let data = self.world.data_dir(ID);
let build_dir = data.join("work"); let build_dir = data.join("work");
world::replace_source(source, &build_dir)?; world::replace_source(source, &build_dir)?;
let manifest = build_dir.join("Cargo.toml"); let manifest = build_dir.join("Cargo.toml");
std::fs::write(&manifest, prepare_manifest(&std::fs::read_to_string(&manifest)?, Some(&self.world.public_dir(ID).join("sdk")))?)?; let original = std::fs::read_to_string(&manifest).context("read build manifest")?;
let prepared = prepare_manifest(&original, Some(&self.world.public_dir(ID).join("sdk")))?;
std::fs::write(&manifest, prepared).context("write prepared manifest")?;
let mut cmd = Command::new("cargo"); let target_dir = data.join("target");
cmd.arg("build") let tests = if skip_tests {
.arg("--release") TestResults { output: "skipped (skip_tests=true)".into(), ..TestResults::default() }
.arg("--target") } else {
.arg(TARGET) match run_tests(&build_dir, &target_dir).await {
.arg("--message-format=json-render-diagnostics") Ok(tests) => tests,
.arg("--target-dir") Err(error) => {
.arg(data.join("target")) let output = format!("cannot run cargo tests: {error:#}");
.current_dir(&build_dir) return Err(BuildFailure {
.kill_on_drop(true); reason: output.clone(),
let out = cmd.output().await.context("cannot run cargo (is it on PATH, with the wasm32-wasip2 target?)")?; tests: TestResults { failed: 1, output, ..TestResults::default() },
let stderr = String::from_utf8_lossy(&out.stderr).to_string(); });
if !out.status.success() { }
bail!("{}", strip_progress(&stderr));
}
let mut artifact = None;
for line in String::from_utf8_lossy(&out.stdout).lines() {
if let Ok(v) = serde_json::from_str::<Value>(line)
&& v["reason"] == "compiler-artifact"
&& let Some(exe) = v["executable"].as_str()
{
artifact = Some(PathBuf::from(exe));
} }
};
if tests.failed > 0 {
return Err(BuildFailure { reason: "unit tests failed".into(), tests });
} }
let artifact = artifact.ok_or_else(|| anyhow!("cargo built no binary. Does the crate have a src/main.rs?"))?;
let n = next_build_number(&self.world.public_dir(ID).join("builds"))?; let build_result: Result<BuildOutput> = async {
let dest_dir = self.world.public_dir(ID).join("builds").join(n.to_string()); let mut cmd = Command::new("cargo");
let name = artifact.file_name().context("artifact has no file name")?; cmd.arg("build")
let dest = dest_dir.join(name); .arg("--release")
world::write_atomic(&dest, &std::fs::read(&artifact)?)?; .arg("--target")
Ok(BuildOutput { wasm: dest, seconds: started.elapsed().as_secs_f64(), warnings: strip_progress(&stderr) }) .arg(TARGET)
.arg("--message-format=json-render-diagnostics")
.arg("--target-dir")
.arg(&target_dir)
.current_dir(&build_dir)
.kill_on_drop(true);
let out = cmd.output().await.context("cannot run cargo (is it on PATH, with the wasm32-wasip2 target?)")?;
let stderr = String::from_utf8_lossy(&out.stderr).to_string();
if !out.status.success() {
bail!("{}", strip_progress(&stderr));
}
let mut artifact = None;
for line in String::from_utf8_lossy(&out.stdout).lines() {
if let Ok(v) = serde_json::from_str::<Value>(line)
&& v["reason"] == "compiler-artifact"
&& let Some(exe) = v["executable"].as_str()
{
artifact = Some(PathBuf::from(exe));
}
}
let artifact = artifact.ok_or_else(|| anyhow!("cargo built no binary. Does the crate have a src/main.rs?"))?;
let n = next_build_number(&self.world.public_dir(ID).join("builds"))?;
let dest_dir = self.world.public_dir(ID).join("builds").join(n.to_string());
let name = artifact.file_name().context("artifact has no file name")?;
let dest = dest_dir.join(name);
world::write_atomic(&dest, &std::fs::read(&artifact)?)?;
Ok(BuildOutput {
wasm: dest,
seconds: started.elapsed().as_secs_f64(),
warnings: strip_progress(&stderr),
tests: tests.clone(),
})
}
.await;
build_result.map_err(|error| BuildFailure { reason: format!("{error:#}"), tests })
} }
/// The native `cargo` actor. Builds run one at a time, in order. /// The native `cargo` actor. Builds run one at a time, in order.
@@ -106,7 +178,7 @@ impl Cargo {
continue; continue;
} }
let body = self.handle(&broker, &env).await.unwrap_or_else(|e| { let body = self.handle(&broker, &env).await.unwrap_or_else(|e| {
json!({"type": "build_failed", "source": env.body.get("source"), "errors": format!("{e:#}")}) json!({"type": "build_failed", "source": env.body.get("source"), "errors": format!("{e:#}"), "tests": TestResults::default().value()})
}); });
broker.send(ID, Some(&env.from), None, Some(&env.id), body); broker.send(ID, Some(&env.from), None, Some(&env.id), body);
} }
@@ -117,22 +189,76 @@ impl Cargo {
Some("build") => { Some("build") => {
let source_s = env.body.get("source").and_then(Value::as_str).ok_or_else(|| anyhow!("missing \"source\""))?; let source_s = env.body.get("source").and_then(Value::as_str).ok_or_else(|| anyhow!("missing \"source\""))?;
let source = broker.world.resolve(&env.from, source_s)?; let source = broker.world.resolve(&env.from, source_s)?;
let out = self.build(&source).await?; let skip_tests = env.body.get("skip_tests").and_then(Value::as_bool).unwrap_or(false);
let wasm = self.world.global_path(&out.wasm).unwrap_or_else(|| out.wasm.display().to_string()); match self.build(&source, skip_tests).await {
Ok(json!({ Ok(out) => {
"type": "built", let wasm = self.world.global_path(&out.wasm).unwrap_or_else(|| out.wasm.display().to_string());
"wasm": wasm, Ok(json!({
"source": source_s, "type": "built",
"built_by": ID, "wasm": wasm,
"seconds": (out.seconds * 10.0).round() / 10.0, "source": source_s,
"warnings": out.warnings, "built_by": ID,
})) "seconds": (out.seconds * 10.0).round() / 10.0,
"warnings": out.warnings,
"tests": out.tests.value(),
}))
}
Err(failure) => Ok(json!({
"type": "build_failed",
"source": source_s,
"errors": failure.reason,
"tests": failure.tests.value(),
})),
}
} }
other => bail!("unknown request {other:?}. Known: build"), other => bail!("unknown request {other:?}. Known: build"),
} }
} }
} }
/// Run tests for the host triple: WASI crates' pure-logic tests can use the
/// normal Rust test harness without needing a WASI test runner.
async fn run_tests(build_dir: &Path, target_dir: &Path) -> Result<TestResults> {
let mut cmd = Command::new("cargo");
cmd.arg("test")
.arg("--target-dir")
.arg(target_dir)
.current_dir(build_dir)
.kill_on_drop(true);
let out = cmd.output().await.context("cannot run cargo test")?;
let stdout = String::from_utf8_lossy(&out.stdout).to_string();
let stderr = String::from_utf8_lossy(&out.stderr).to_string();
let combined = match (stdout.is_empty(), stderr.is_empty()) {
(true, true) => String::new(),
(false, true) => stdout,
(true, false) => stderr,
(false, false) => format!("{stdout}\n{stderr}"),
};
let output = strip_progress(&combined);
let passed = test_count(&output, "passed");
let mut failed = test_count(&output, "failed");
// Compilation failures and runner errors have no `test result` summary,
// but still make the default test phase fail.
if !out.status.success() && failed == 0 {
failed = 1;
}
Ok(TestResults { passed, failed, output })
}
/// Count the `passed` or `failed` totals in cargo's per-target test summaries.
fn test_count(output: &str, field: &str) -> u64 {
output
.lines()
.filter_map(|line| line.split_once("test result:").map(|(_, rest)| rest))
.flat_map(|summary| summary.split(';'))
.filter_map(|part| {
let words: Vec<_> = part.split_whitespace().collect();
let index = words.iter().position(|word| *word == field)?;
words.get(index.checked_sub(1)?)?.parse::<u64>().ok()
})
.sum()
}
/// Point a `sausage-actor` dependency at `sdk` (keeping its features), and /// Point a `sausage-actor` dependency at `sdk` (keeping its features), and
/// add an empty `[workspace]`, so the crate never joins a parent workspace. /// add an empty `[workspace]`, so the crate never joins a parent workspace.
fn prepare_manifest(text: &str, sdk: Option<&Path>) -> Result<String> { fn prepare_manifest(text: &str, sdk: Option<&Path>) -> Result<String> {
+2 -1
View File
@@ -3,7 +3,8 @@
//! - `@<actor> <json>` sends a JSON body. `@<actor> <text>` sends //! - `@<actor> <json>` sends a JSON body. `@<actor> <text>` sends
//! `{"type": "text", "text": ...}`. //! `{"type": "text", "text": ...}`.
//! - A line without `@` goes to `agent` as text. //! - A line without `@` goes to `agent` as text.
//! - `/help`, `/quit`. EOF on stdin also quits. //! - `/help`, `/quit`. EOF on stdin also quits (not with `--headless`, which
//! never reads stdin).
use std::sync::Arc; use std::sync::Arc;
+44 -8
View File
@@ -5,6 +5,7 @@ use anyhow::{Context, Result, bail};
use clap::{Parser, Subcommand}; use clap::{Parser, Subcommand};
use include_dir::{Dir, include_dir}; use include_dir::{Dir, include_dir};
use serde_json::json; use serde_json::json;
use tokio::signal::unix::Signal;
use sausage::world::{ActorMeta, World}; use sausage::world::{ActorMeta, World};
use sausage::{Host, cargo_actor, console}; use sausage::{Host, cargo_actor, console};
@@ -17,6 +18,11 @@ static AGENT_SOURCE: Dir<'_> = include_dir!("$CARGO_MANIFEST_DIR/../agent");
#[derive(Parser)] #[derive(Parser)]
#[command(version, about)] #[command(version, about)]
struct Cli { struct Cli {
/// Do not read stdin, so end of input does not stop the world. It runs
/// until SIGTERM (or SIGINT), or until an actor sends
/// `@broker {"type":"exit"}`. Also set by SAUSAGE_HEADLESS=1.
#[arg(long, global = true)]
headless: bool,
#[command(subcommand)] #[command(subcommand)]
command: Option<Cmd>, command: Option<Cmd>,
} }
@@ -85,19 +91,47 @@ async fn init(dir: PathBuf) -> Result<()> {
std::fs::create_dir_all(&source)?; std::fs::create_dir_all(&source)?;
AGENT_SOURCE.extract(&source).context("extract agent source")?; AGENT_SOURCE.extract(&source).context("extract agent source")?;
println!("Building the agent actor (the first build takes a few minutes) …"); println!("Building the agent actor (the first build takes a few minutes) …");
let out = cargo_actor::Cargo::new(world.clone())?.build(&source).await.context("build agent")?; let out = cargo_actor::Cargo::new(world.clone())?.build(&source, false).await.context("build agent")?;
std::fs::copy(&out.wasm, world.wasm_path("agent"))?; std::fs::copy(&out.wasm, world.wasm_path("agent"))?;
println!("Built in {:.1} s. Start the world with: sausage run {}", out.seconds, dir.display()); println!("Built in {:.1} s. Start the world with: sausage run {}", out.seconds, dir.display());
Ok(()) Ok(())
} }
async fn run(dir: PathBuf) -> Result<()> { /// Install the SIGTERM and SIGINT handlers now (not when first polled).
fn termination_signals() -> Result<(Signal, Signal)> {
use tokio::signal::unix::{SignalKind, signal};
let term = signal(SignalKind::terminate()).context("install SIGTERM handler")?;
let int = signal(SignalKind::interrupt()).context("install SIGINT handler")?;
Ok((term, int))
}
/// Resolves on SIGTERM or SIGINT.
async fn termination((mut term, mut int): (Signal, Signal)) {
tokio::select! {
_ = term.recv() => {}
_ = int.recv() => {}
}
}
async fn run(dir: PathBuf, headless: bool) -> Result<()> {
// Register the signal handlers before the host starts, so a signal that
// arrives early is never the default (kill) action.
let signals = if headless { Some(termination_signals()?) } else { None };
let host = Host::start(&dir).await?; let host = Host::start(&dir).await?;
tokio::spawn(console::run_mailbox(host.mailbox(console::ID))); tokio::spawn(console::run_mailbox(host.mailbox(console::ID)));
let broker = host.broker.clone(); let broker = host.broker.clone();
let exit = tokio::select! { let exit = match signals {
_ = console::run_input(host.broker.clone()) => false, Some(signals) => {
_ = broker.exit_requested() => true, println!("Sausage headless: running until SIGTERM or an exit request.");
tokio::select! {
_ = termination(signals) => false,
_ = broker.exit_requested() => true,
}
}
None => tokio::select! {
_ = console::run_input(host.broker.clone()) => false,
_ = broker.exit_requested() => true,
},
}; };
println!("Stopping actors …"); println!("Stopping actors …");
host.stop().await; host.stop().await;
@@ -113,16 +147,18 @@ async fn run(dir: PathBuf) -> Result<()> {
#[tokio::main] #[tokio::main]
async fn main() -> Result<()> { async fn main() -> Result<()> {
match Cli::parse().command { let cli = Cli::parse();
let headless = cli.headless || std::env::var("SAUSAGE_HEADLESS").is_ok_and(|v| !v.is_empty() && v != "0");
match cli.command {
Some(Cmd::Init { dir }) => init(dir).await, Some(Cmd::Init { dir }) => init(dir).await,
Some(Cmd::Run { dir }) => run(dir).await, Some(Cmd::Run { dir }) => run(dir, headless).await,
None => { None => {
let dir = PathBuf::from("world"); let dir = PathBuf::from("world");
if !dir.join("actor").is_dir() { if !dir.join("actor").is_dir() {
println!("No world in ./world yet, creating it (same as `sausage init`)."); println!("No world in ./world yet, creating it (same as `sausage init`).");
init(dir.clone()).await?; init(dir.clone()).await?;
} }
run(dir).await run(dir, headless).await
} }
} }
} }
+44
View File
@@ -39,3 +39,47 @@ fn bare_command_without_a_world_starts_init() {
// It asks before it creates anything. // It asks before it creates anything.
assert!(!dir.join("world").exists()); assert!(!dir.join("world").exists());
} }
/// Start a headless host on a fresh world with stdin at EOF; returns once it
/// printed its ready line (printed after the signal handlers are installed).
fn start_headless(label: &str) -> (std::process::Child, std::io::Lines<std::io::BufReader<std::process::ChildStdout>>) {
use std::io::{BufRead, BufReader};
let dir = empty_dir(label);
let world = dir.join("world");
sausage::World::create(&world).unwrap();
let mut child = Command::new(env!("CARGO_BIN_EXE_sausage"))
.args(["run", "--headless"])
.arg(&world)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::null())
.spawn()
.unwrap();
let mut lines = BufReader::new(child.stdout.take().unwrap()).lines();
let line = lines.next().expect("host exited before it started").unwrap();
assert!(line.contains("headless"), "unexpected first line: {line}");
(child, lines)
}
/// With stdin at EOF (as soon as the host starts), a headless host stays up.
#[test]
fn headless_ignores_stdin_eof() {
let (mut child, _lines) = start_headless("headless-eof");
std::thread::sleep(std::time::Duration::from_millis(500));
let alive = child.try_wait().unwrap().is_none();
let _ = child.kill();
let _ = child.wait();
assert!(alive, "headless host exited on stdin EOF");
}
/// SIGTERM right after the ready line stops the actors and exits with 0.
#[test]
fn headless_stops_cleanly_on_sigterm_right_after_ready() {
let (mut child, lines) = start_headless("headless-term");
let pid = child.id().to_string();
assert!(Command::new("kill").args(["-TERM", &pid]).status().unwrap().success());
let rest: Vec<String> = lines.map(|l| l.unwrap()).collect();
let status = child.wait().unwrap();
assert!(status.success(), "exit status {status:?}, output {rest:?}");
assert!(rest.iter().any(|l| l.contains("Stopping actors")), "output {rest:?}");
}
@@ -0,0 +1,6 @@
[package]
name = "cargo-test-fail"
version = "0.1.0"
edition = "2024"
[workspace]
@@ -0,0 +1,9 @@
fn main() {}
#[cfg(test)]
mod tests {
#[test]
fn deliberate_unit_test_failure() {
assert_eq!(2 + 2, 5);
}
}
+8
View File
@@ -1 +1,9 @@
fn main() {} fn main() {}
#[cfg(test)]
mod tests {
#[test]
fn unit_tests_run_on_the_host_target() {
assert_eq!(2 + 2, 4);
}
}
+38 -11
View File
@@ -619,9 +619,10 @@ async fn unknown_broker_request_lists_the_known_ones() {
} }
#[tokio::test] #[tokio::test]
async fn cargo_actor_builds_valid_crates_and_reports_compiler_errors() { async fn cargo_actor_runs_unit_tests_by_default_and_can_skip_them() {
let world = create_world("cargo-actor"); let world = create_world("cargo-actor");
let (host, mut mailbox) = start(&world).await; let (host, mut mailbox) = start(&world).await;
let valid_source = cargo_source("cargo-valid"); let valid_source = cargo_source("cargo-valid");
let valid_id = host.broker.send( let valid_id = host.broker.send(
"console", "console",
@@ -630,12 +631,42 @@ async fn cargo_actor_builds_valid_crates_and_reports_compiler_errors() {
None, None,
json!({"type": "build", "source": valid_source.display().to_string()}), json!({"type": "build", "source": valid_source.display().to_string()}),
); );
let built = receive_reply(&mut mailbox, &valid_id, Duration::from_secs(120)).await; let built = receive_reply(&mut mailbox, &valid_id, Duration::from_secs(180)).await;
assert_eq!(built.body["type"], "built"); assert_eq!(built.body["type"], "built", "{built:?}");
assert_eq!(built.body["tests"]["passed"], 1);
assert_eq!(built.body["tests"]["failed"], 0);
assert!(built.body["tests"]["output"].as_str().unwrap().contains("unit_tests_run_on_the_host_target"));
let wasm = built.body["wasm"].as_str().unwrap(); let wasm = built.body["wasm"].as_str().unwrap();
assert!(wasm.ends_with(".wasm")); assert!(wasm.ends_with(".wasm"));
assert!(host.world.resolve("console", wasm).unwrap().is_file()); assert!(host.world.resolve("console", wasm).unwrap().is_file());
let failing_test_source = cargo_source("cargo-test-fail");
let failed_test_id = host.broker.send(
"console",
Some("cargo"),
None,
None,
json!({"type": "build", "source": failing_test_source.display().to_string()}),
);
let failed_test = receive_reply(&mut mailbox, &failed_test_id, Duration::from_secs(120)).await;
assert_eq!(failed_test.body["type"], "build_failed", "{failed_test:?}");
assert_eq!(failed_test.body["tests"]["passed"], 0);
assert_eq!(failed_test.body["tests"]["failed"], 1);
assert!(failed_test.body["tests"]["output"].as_str().unwrap().contains("deliberate_unit_test_failure"));
let skipped_id = host.broker.send(
"console",
Some("cargo"),
None,
None,
json!({"type": "build", "source": failing_test_source.display().to_string(), "skip_tests": true}),
);
let skipped = receive_reply(&mut mailbox, &skipped_id, Duration::from_secs(120)).await;
assert_eq!(skipped.body["type"], "built", "{skipped:?}");
assert_eq!(skipped.body["tests"]["passed"], 0);
assert_eq!(skipped.body["tests"]["failed"], 0);
assert_eq!(skipped.body["tests"]["output"], "skipped (skip_tests=true)");
let invalid_source = cargo_source("cargo-fail"); let invalid_source = cargo_source("cargo-fail");
let invalid_id = host.broker.send( let invalid_id = host.broker.send(
"console", "console",
@@ -644,14 +675,10 @@ async fn cargo_actor_builds_valid_crates_and_reports_compiler_errors() {
None, None,
json!({"type": "build", "source": invalid_source.display().to_string()}), json!({"type": "build", "source": invalid_source.display().to_string()}),
); );
let failed = receive_reply(&mut mailbox, &invalid_id, Duration::from_secs(120)).await; let invalid = receive_reply(&mut mailbox, &invalid_id, Duration::from_secs(120)).await;
assert_eq!(failed.body["type"], "build_failed"); assert_eq!(invalid.body["type"], "build_failed", "{invalid:?}");
assert!( assert_eq!(invalid.body["tests"]["failed"], 1);
failed.body["errors"] assert!(invalid.body["tests"]["output"].as_str().unwrap().contains("missing_function"));
.as_str()
.unwrap()
.contains("missing_function")
);
host.stop().await; host.stop().await;
} }
+70 -4
View File
@@ -21,9 +21,11 @@
targets = [ "wasm32-wasip2" ]; targets = [ "wasm32-wasip2" ];
extensions = [ "rust-src" ]; extensions = [ "rust-src" ];
}; };
# Crane builds the dependencies once (`cargoArtifacts`, keyed only on # The runtime cargo actor needs Cargo, rustc (with wasm32-wasip2), and
# Cargo.toml/Cargo.lock), so a change to our own crates rebuilds # the native linker, even when the host itself is started with a bare
# just them. # PATH. Keep this same runtime path in the packaged host and exercise
# the installed binary below with PATH deliberately empty.
runtimePath = pkgs.lib.makeBinPath [ rust pkgs.stdenv.cc ];
craneLib = (crane.mkLib pkgs).overrideToolchain rust; craneLib = (crane.mkLib pkgs).overrideToolchain rust;
common = { common = {
pname = "sausage"; pname = "sausage";
@@ -42,12 +44,13 @@
}); });
sausage = craneLib.buildPackage (common // { sausage = craneLib.buildPackage (common // {
inherit cargoArtifacts; inherit cargoArtifacts;
doNotRemoveReferencesToRustToolchain = true;
cargoExtraArgs = "--locked -p sausage"; cargoExtraArgs = "--locked -p sausage";
doCheck = false; doCheck = false;
nativeBuildInputs = [ pkgs.makeWrapper ]; nativeBuildInputs = [ pkgs.makeWrapper ];
# The `cargo` actor runs cargo at runtime. Build scripts need a C linker. # The `cargo` actor runs cargo at runtime. Build scripts need a C linker.
postInstall = '' postInstall = ''
wrapProgram $out/bin/sausage --prefix PATH : ${pkgs.lib.makeBinPath [ rust pkgs.stdenv.cc ]} wrapProgram $out/bin/sausage --prefix PATH : ${runtimePath}
''; '';
meta.mainProgram = "sausage"; meta.mainProgram = "sausage";
}); });
@@ -91,6 +94,69 @@
rm -f cli/tests/actors/toolbox/Cargo.lock rm -f cli/tests/actors/toolbox/Cargo.lock
''; '';
}); });
# Exercise the installed host exactly as the loop launches it: the
# caller has no Cargo on PATH, so only the packaged runtime toolchain
# can run the native tests and the wasm32-wasip2 build.
checks.runtime-toolchain = pkgs.runCommand "sausage-runtime-toolchain" { } ''
set -eu
root="$TMPDIR/runtime-toolchain"
world="$root/world"
emptyPath="$root/empty-bin"
mkdir -p "$emptyPath" "$root/home" "$root/cargo-home"
for actor in broker console cargo fs; do
mkdir -p "$world/actor/$actor/public" "$world/actor/$actor/data"
done
source="$world/actor/console/public/probe"
mkdir -p "$source/src" "$source/tests"
cat > "$source/Cargo.toml" <<'TOML'
[package]
name = "runtime-probe"
version = "0.1.0"
edition = "2024"
TOML
printf 'fn main() {}\n' > "$source/src/main.rs"
cat > "$source/tests/probe.rs" <<'RUST'
#[test]
fn native_test_harness_runs() {
assert_eq!(2 + 2, 4);
}
RUST
echo "runtime-check: run packaged host with an empty PATH"
coproc host {
${pkgs.coreutils}/bin/env -i PATH="$emptyPath" HOME="$root/home" TMPDIR="$root" \
CARGO_HOME="$root/cargo-home" ${sausage}/bin/sausage run "$world"
}
hostInput=''${host[1]}
hostOutput=''${host[0]}
hostPid=$host_PID
trap 'kill "$hostPid" 2>/dev/null || true' EXIT
printf '%s\n' '@cargo {"type":"build","source":"/actor/console/public/probe"}' >&"$hostInput"
built=0
while IFS= read -r -t 300 line <&"$hostOutput"; do
echo "$line"
case "$line" in
*'"type":"build_failed"'*)
echo "packaged cargo actor build failed with an empty caller PATH" >&2
exit 1
;;
*'"type":"built"'*)
case "$line" in
*'"passed":1'*) built=1 ;;
*) echo "native test did not pass" >&2; exit 1 ;;
esac
break
;;
esac
done
if [ "$built" -ne 1 ]; then
echo "packaged host did not build the probe" >&2
exit 1
fi
printf '%s\n' '/quit' >&"$hostInput"
wait "$hostPid"
mkdir -p "$out"
touch "$out/passed"
'';
apps.default = { type = "app"; program = "${sausage}/bin/sausage"; }; apps.default = { type = "app"; program = "${sausage}/bin/sausage"; };
apps.loop = { type = "app"; program = "${loop}/bin/sausage-loop"; }; apps.loop = { type = "app"; program = "${loop}/bin/sausage-loop"; };