sausage/docs/SAU-384-rehearsal.md

Disposable handover rehearsal — SAU-384

Authority: /docs/design/plugin-handover/design.md, Rollout; /docs/symphony/artifacts/SAU-356/plan.md, Approval and operator gates.

Status: blocked, not a live execution approval. Rehearsal evidence and the reviewed target bundle are under /docs/symphony/artifacts/SAU-384. No shipped plugin or host behavior changes on this branch. The temporary migration implementation and native controller were executed only in a throwaway box and deleted after the successful migration verification. Their hashes/deletion record and source-as-log remain, not an executable migration tool.

Required resolution before completing the kit

1. Pin the actual external host launcher/service owner, stop/start interface, host architecture and runtime closure. Those are not projected into this world. The supplied local native artifact is aarch64-linux, dynamically linked to the Nix paths printed in the build evidence; it is NOT a verified live-host package. Do not guess a systemd unit, use core.restart as stop-all, or install a mismatching binary.
2. Resolved rollback choice: unsubscribing works; merely re-keying the registration does not. An actual pre-best-effort session component, reading a registration written by the target component, ignores best_effort and sends history to its new ID. Rehearsal: unsubscribe = one barrier callback and no gateway-shaped replay; re-key = barrier plus historical replay. The test used the actual old manager and a native callback sink, not browser SSE. The runbook now makes unsubscribe mandatory, using the card's authorized option. Do not claim an ID rename is safe.
3. Closed in run 1369: armed/exit-race delay-gate tests passed on real core with actual target WASM; replacement orchestrator readiness/one-owner/all-watch arming passed; actual old-manager/gateway rollback without re-registration reproduced unsubscribe safety and ID-only re-key replay. See resume evidence. The delay/observation hooks were only in a disposable host copy, not the selected host artifact or committed code.

Use the card's explicit unsubscribe rollback option; an ID rename is not a safe alternative. The remaining blocker is the actual external host launcher/native installation manifest. Read-only SSH credential request 055995a51608c57dd25b323bea731fcb is approved, but connection was refused before authentication for an unknown Citadel host. Mediator must add the observed public host key, then resume inspection; workers may not change live config. See run 1372. No new approval request or live stop/restart permission is needed/granted.

What was built and exercised

• Host source e09c4f9f0ef6b5833d6be066771eecd9e0a1f572. Disposable Nix build, native workspace suite passed after correcting the cold SDK-probe dependency cache. Target native binary was built and stripped, not released/deployed. CLI --version passed. Native controller used the real sausage::Host, real VFS, watches, locks, service supervisor and WASM execution; model/skills/memory/ledger/push effects were synthetic or loopback only.
• Reviewed session cb844151 / SAU-409 final; orchestrator c80a5854 / SAU-408; web d883fbf2 / SAU-403; agent/board retained reviewed components. Fresh card-worktree builds: mail 1600 / 53 tests, cred 1601 / 272, cron 1602 / 24, rooms 1603 / 59, all passed. No activation. Component/source provenance and every bundled file hash are in evidence/manifest.json.
• Frontend is exactly approved commit 3d76026749a5d21722033bbfe8743ac368946b0f, retained production build from SAU-375 rework-1318. It remains publish:hold; no publication was attempted.
• World SDK runtime sources already matched the selected host SDK. The only needed source-copy delta is src/lib.rs, documenting effective grace. Preserve world-owned Cargo manifests, relocated interface, and imports. A first broad SDK-copy attempt incorrectly included repository-relative Cargo paths; comparison caught this before any live change, and the final migration used only the one-file patch. Never copy the repository's SDK Cargo manifests over the world SDK.
• Accepted SAU-362 byte snapshot: 1,491 sessions / 580,355,405 bytes, including 70,689,796-byte default history. Pristine snapshot remained unpreopened. Executable clones used synthetic model settings; their historical byte volume was retained. Source snapshot consistency is per-file, not a transactional live snapshot.

Successful checks and limits

Check	Evidence/result
Old ownership → stop-all → offline swap → target boot	Old mail.start, old session send and detached ledger processes inventoried. Native external controller awaited Host.stop, verified zero remaining processes and host exit, then backed up/transformed/swapped. Final shutdown 0.0958 s in this fixture. One target manager, no mail deliverer, cred locked.
Partial swap + rerun	Injected interruption after operation 3; hash-guarded completion and repeated installed-phase apply preserved all 4,374 non-target inventoried files. No live/new work reset.
Exact losses/preservation	Only pending.jsonl, cancel, and unclaimed pending non-Surma delivery rows selected for removal. Original mail messages, done/Surma copies and uncertain unknown deliveries retained without replay; scheduled SQL IDs, calendar/outcome fields and non-session JSON bytes retained. Synthetic grant DB bytes preserved. Loss inventory backs every removed file/row.
Offline failure boundaries	Before-start rollback restored exact bytes; conflicting outside edits refused; ambiguous legacy interrupt:true refused without mutation. After-start apply and whole-world rollback both refused.
Default/advisor HTTP replay	Three identical same-ID POSTs each, before/after consumption; exactly one answered run per session. Default returned null on repeats; advisor returned its same receipt.
Explicit open target run	A run-start/consumed record with a finished-looking assistant tail, but no run-end, recovered under the same agent-run UUID, with one start. Legacy finished-looking history alone produced no target event/run.
Real VFS/runner-boundary work	Actual SDK armed then plugin-write changed; five callback-boundary runs. Run 1369 additionally forced publication before runner armed and after its empty-mailbox exit decision while detached completion was held; no lost/duplicate inputs (disposable host delay gates, actual target WASM).
Real core detached errors	Full nested error data survived a detached core.call.
Real cron research deadline	305.427 s send→stopped in the final gate; real cron invoked warning and Stop. The 305-second test deadline is the existing 300-second warning interval plus design grace 5 seconds, not a new default. Held inference was cut, no follow-up run. Original configured research default remains 1,200 seconds.
Effective grace / crash supervision	Passing native host suite exercises real WASI uniform-grace and before/after-readiness crash-supervision fixtures. Deliberate core.kill is NOT a crash: it disables that boot service; an initial rehearsal expecting its restart failed, correctly, and was replaced by the existing crash tests.
Orchestrator upgrade/idle	Real file-watcher upgrade, FIFO singleton release, old exit before one replacement start event. Paused empty-board/live-size-world host CPU 0.02 s / 20.0002 s ≈ 0.10%. This is whole-host idle CPU, not per-plugin CPU or live active-work load. Run 1369 additionally verified target acknowledgement, one owner and all 13 core watches armed after old exit.

Live-size latency after real records exist

After producing actual run-start/run-end/mailbox-consumed records, subsequent full boot measured 0.544 s from manager service-start to the next boot service-start (includes discovery/ack overhead). Narrow registration catch-up samples ranged 0.0239 s to 12.0419 s in the integrated world: the long sample coincided with other synthetic recovery/surfacing work, and the untagged sink cannot isolate attribution. Do not present this as a bounded manager-only benchmark.

The clean final live-size end-to-end gate measured manager acknowledgement 0.6078 s, registration catch-up 0.0128 s, and mail.send publication to default 0.5376 s. The integrated post-record mail sample was 0.5235 s. Both publication samples are below the existing orchestrator 3 s and cred 10 s caller budgets (SAU-400 report cites the originating SAU-366 review). These warm-cache synthetic-loopback real-core observations are not live latency guarantees. First scanning still delays first new input after the fast boot acknowledgement.

Resume verification

Run 1369 forced recompilation and repeated the complete workspace suite on the restored original host source: pass, including 208 host integration tests. All 144 source files matched the selected source snapshot. Full log /logs/box/2026-10-31/tc-asa3yr-exec-3838117964.stderr (host tests lines 64–434; equivalence line 785). New tests and their limitations are detailed in SAU-384-resume-1369.md; new test controllers and instrumentation were deleted after successful checks. Original bundle and production source are unchanged.

Evidence and failures

All following logs are under /logs/box/2026-10-31/:

• Full workspace pass: tc-asa3yr-exec-2320283969.stderr. First cold-cache/full-load failure: tc-asa3yr-exec-158345583.stderr, missing offline async-task and a queued-build timing test. Populated the separate SDK-probe cache; entire suite then passed unchanged.
• Final stop/swap/start: tc-asa3yr-exec-1922130515.stderr; offline fault/grant preservation: tc-asa3yr-exec-616022371.stderr.
• Final live-size HTTP/deadline gate: tc-asa3yr-exec-1243993190.stderr. Earlier malformed mail to scalar, 30-second startup observation bound and blank JSONL parser mistakes were harness errors; corrected to a sequence, existing 120-second test budget and blank-line skipping. Final gate passed.
• Open-run recovery and boundary runs: tc-asa3yr-exec-903419814.stderr (subsequent console-watch attempt failed). A console is not a running watch caller; the final SDK guest watch passed in tc-asa3yr-exec-3261552495.stderr. Same log retains the real upgrade trace and idle sample. An identical already-acknowledged registration correctly produced no second catch-up; the final probe used a fresh registration identity.
• Actual old-manager unsubscribe/re-key comparison: tc-asa3yr-exec-3307393394.stderr.
• Native artifact architecture/runtime paths: tc-asa3yr-exec-899103518.stderr; bundle build/CLI: tc-asa3yr-exec-1075679130.stderr.
• Executed temporary tool and controller as log only, followed by deletion proof: tc-asa3yr-exec-2902726782.stderr (31,475 bytes). evidence/tool-deletion.json lists removed executables. No transition code was put in a shipped plugin.

The initial broad-copy fixture and earlier failed tests are not the final kit. evidence/migration-plan.json identifies the final verified migration. Provider billing/cancellation, power loss, hung host filesystem behavior, physical browsers and independent per-plugin recovery follow-ups are not claimed complete. No live DB was opened; the accepted session snapshot was reused. No live config/SDK change, stop, activation, release, restart, push, deployment or /www write occurred.

See operator runbook for approval gates and the deliberately blocked live steps.