userland/git: git-compatible CLI on v2's gix code (T10)
Local workflow, smart-HTTP clone/fetch/push and public HTTPS clone/fetch match native git in stdout and exit codes. Only WASI imports (wasi:http, exit-with-code).
This commit is contained in:
Vendored
+137
@@ -0,0 +1,137 @@
|
||||
# WASI patches for Git foundation (SAU-21)
|
||||
|
||||
All crates here are complete crates.io source archives, with their original licenses.
|
||||
`Cargo.toml` uses local `[patch.crates-io]` paths. No upstream submission has been made.
|
||||
Changes apply only to `target_os = "wasi"`; native paths retain upstream behavior.
|
||||
No parallel, signal/interrupt, or worktree-mutation features are enabled.
|
||||
SAU-24 enables blocking-network-client and a custom wasi:http backend (no curl/reqwest).
|
||||
|
||||
## gix-tempfile 26.0.0: no OS PID on WASI
|
||||
|
||||
Exact panic: `src/forksafe.rs`, `ForksafeTempfile::new`, field
|
||||
`owning_process_id: std::process::id()`. Ref transactions acquire gix-lock files,
|
||||
which register these tempfiles. This is registry fork ownership, **not filename generation**.
|
||||
`std::process::id()` panics with `no pids on this platform` on wasm32-wasip2.
|
||||
|
||||
Patch: a private `process_id()` returns 0 on WASI (no fork, registry is per component
|
||||
instance), and the native OS PID elsewhere. Use it in creation and both registry
|
||||
cleanup methods. Random tempfile naming and exclusive file creation are unchanged.
|
||||
Verified the unpatched ref-updating commit panics; patched commits and index writes pass.
|
||||
|
||||
## gix-pack 0.77.0: owned pack/index/multi-index bytes
|
||||
|
||||
`src/lib.rs`: WASI's `MMap` is `Vec<u8>` and the shared `mmap::read_only()` helper
|
||||
uses `std::fs::read`. Other targets retain memmap2. This covers the pack data,
|
||||
pack index, and multi-pack-index constructors that all use this helper.
|
||||
memmap2 0.9.11 selects its unsupported stub on WASI; the original map call returns
|
||||
`ErrorKind::Unsupported` rather than reading a pack.
|
||||
|
||||
Tests generate a tiny pack via `gix-pack`'s `data::output::bytes::FromEntriesIter`;
|
||||
the simple v2 index is encoded in the test (fanout, sorted IDs, CRCs, offsets,
|
||||
pack checksum, index checksum). This avoids enabling the broken streaming-input
|
||||
feature just for the test. A second test packs all commit/tree/blob objects,
|
||||
removes every loose object copy and stores the branch only in `packed-refs`,
|
||||
reopens the repository and reads history, paths, branches,
|
||||
blob contents, and creates/reads its index. No shell or Git executable is used.
|
||||
|
||||
## gix-odb 0.87.0: owned loose-object bytes
|
||||
|
||||
Additional breakage found by the full tests: `src/store_impls/loose/find.rs`
|
||||
uses memmap2 for *every loose object read and header lookup*, not just packs.
|
||||
Without this patch logs and tree/index construction return `unsupported`.
|
||||
Use the same platform-dependent `gix_pack::MMap` backing and std file reads on WASI.
|
||||
Decompression, missing-object handling, and validation remain upstream code.
|
||||
|
||||
## gix-index 0.58.0: read index without mmap
|
||||
|
||||
`src/file/init.rs`: WASI reads into a Vec from the already opened file and rewinds
|
||||
that same handle before checksum streaming. This preserves a single inode snapshot
|
||||
across concurrent atomic index replacements. Native mmap code is unchanged.
|
||||
Nonempty v2 index round trip and checksum-corruption rejection pass under WASI.
|
||||
|
||||
## filetime 0.2.29: metadata timestamp readers
|
||||
|
||||
Additional runtime panic found after the mmap fixes: filetime routes WASI to its
|
||||
generic wasm backend, whose metadata timestamp readers call `unimplemented!()`.
|
||||
`src/lib.rs` now selects `src/wasi.rs` before that backend. It obtains modified,
|
||||
accessed, and optional creation times through std Metadata; inaccessible modified/
|
||||
accessed times fall back to zero, because the upstream API cannot return an error.
|
||||
Symlink time mutation remains an explicit Unsupported error; ordinary file-time
|
||||
setters retain the upstream std File::set_times implementation. Metadata reads are
|
||||
exercised by index tests; timestamp setters and symlink mutation are not exercised.
|
||||
|
||||
## gix-ref 0.69.0: disable packed-refs mmap through existing API
|
||||
|
||||
No additional vendored crate is needed. On WASI, `src/repo.rs::open` calls
|
||||
`repo.refs.set_packed_buffer_mmap_threshold(u64::MAX)` before any reference lookup.
|
||||
Upstream `store/packed/buffer.rs::Buffer::open` uses owned bytes for files at or
|
||||
below that threshold; otherwise it uses unsupported memmap2. The default threshold
|
||||
is 32 KiB. Native repository handles retain the upstream default.
|
||||
|
||||
Regression `large_packed_refs_head_branches_and_log` writes 65,638 bytes of sorted
|
||||
packed refs (1,025 branches), removes the loose HEAD branch and reopens through
|
||||
our helper. It checks symbolic HEAD, all branch tips/current flag, HEAD log order,
|
||||
and log by a non-HEAD packed branch. Before the fix this fails on WASI with
|
||||
`HEAD: Could not open packed refs`; with the fix it passes. Packed refs also consume
|
||||
owned linear memory; this is not a repository size limit.
|
||||
|
||||
## Intentionally disabled / work for later cards
|
||||
|
||||
- `worktree-mutation`: `gix-worktree-state` depends on `io-close 0.3.7`; it lacks
|
||||
a WASI File close implementation (`no method close for File`). Add a WASI backend
|
||||
or write the checkout engine using std file writes. Do not silently ignore writes.
|
||||
- `blocking-network-client` is now enabled by SAU-24: see the streaming-input patch below.
|
||||
- `gix-commitgraph` remains unpatched: `src/file/init.rs` maps commit-graph files
|
||||
with memmap2, unsupported on WASI. Current `log` explicitly uses
|
||||
`use_commit_graph(false)` and does not need this cache. This is NOT disabled
|
||||
centrally by `repo::open`. Later merge-base/ahead-behind/revision operations must
|
||||
disable or avoid commit-graph loading (including direct graph APIs), or patch
|
||||
that crate to read owned bytes before using it. Some high-level gix walks fall
|
||||
back to the object database on graph-load errors; direct graph loading can fail.
|
||||
Existing native Git `objects/info/commit-graph` files are not fixture-tested here.
|
||||
- No signals or threads are provided; keep `interrupt` and `parallel` off.
|
||||
- No large-repo memory benchmark was run. Pack + pack-index bytes are held in
|
||||
32-bit linear memory, in addition to decoded objects; whole-pack reads have a
|
||||
materially higher resident-memory cost than demand-paged native mmap. The tiny
|
||||
fixture is 53 pack bytes for one blob. No peak RSS/linear-memory measurement.
|
||||
- Multi-pack-index reading uses the patched helper but is not fixture-tested here.
|
||||
Delta packs, split index, and external Git-generated fixtures are not tested.
|
||||
|
||||
## SAU-24: gix-pack 0.77.0 streaming-input on WASI
|
||||
|
||||
Existing vendored crate extended (no additional crate patch). In `Cargo.toml` and
|
||||
`Cargo.toml.orig`, allow gix-tempfile under
|
||||
`cfg(any(not(target_arch = "wasm32"), target_os = "wasi"))`. Browser wasm is still
|
||||
excluded; gix-tempfile remains `default-features = false`, so no signals are added.
|
||||
The foundation's PID patch applies unchanged.
|
||||
|
||||
Additional compile error exposed by enabling the feature: bundle writer's
|
||||
`new_pack_file_resolver` and `resolve_entry` explicitly used `memmap2::Mmap`.
|
||||
Change those three type annotations in `src/bundle/write/mod.rs` to `crate::MMap`,
|
||||
reusing the existing owned-byte WASI helper and keeping native mmap behavior.
|
||||
No pack/index format change. No upstream submission made.
|
||||
|
||||
Verified by local fake Git HTTP servers through wasi:http in cargo.build: V1
|
||||
clone writes a received 227-byte / 3-object pack and real index, updates tracking
|
||||
refs and checks out exact bytes; V2 shallow clone writes the shallow boundary.
|
||||
This tests the bundle writer, not merely hand-encoded pack indexes. Pack input
|
||||
is streamed to a tempfile; indexing still reads a whole-pack Vec on WASI and
|
||||
therefore needs 32-bit memory proportional to pack size. Use clone `depth` to
|
||||
reduce history. No large-repo memory measurement.
|
||||
|
||||
## Sources inspected
|
||||
|
||||
- https://raw.githubusercontent.com/GitoxideLabs/gitoxide/gix-v0.90.0/gix-ref/src/store/file/mod.rs
|
||||
- https://raw.githubusercontent.com/GitoxideLabs/gitoxide/gix-v0.90.0/gix-ref/src/store/packed/buffer.rs
|
||||
- https://raw.githubusercontent.com/GitoxideLabs/gitoxide/gix-v0.90.0/gix/src/types.rs
|
||||
- https://raw.githubusercontent.com/GitoxideLabs/gitoxide/gix-v0.90.0/gix-commitgraph/src/file/init.rs
|
||||
- https://raw.githubusercontent.com/GitoxideLabs/gitoxide/gix-v0.90.0/gix/src/revision/walk.rs
|
||||
|
||||
- https://raw.githubusercontent.com/GitoxideLabs/gitoxide/gix-v0.90.0/gix/Cargo.toml
|
||||
- https://static.crates.io/crates/gix-tempfile/gix-tempfile-26.0.0.crate
|
||||
- https://static.crates.io/crates/gix-pack/gix-pack-0.77.0.crate
|
||||
- https://static.crates.io/crates/gix-index/gix-index-0.58.0.crate
|
||||
- https://static.crates.io/crates/gix-odb/gix-odb-0.87.0.crate
|
||||
- https://raw.githubusercontent.com/RazrFalcon/memmap2-rs/v0.9.11/src/stub.rs
|
||||
- https://raw.githubusercontent.com/alexcrichton/filetime/0.2.29/src/wasm.rs
|
||||
- https://static.crates.io/crates/filetime/filetime-0.2.29.crate
|
||||
Reference in New Issue
Block a user