Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/guides/native-candidate.md
Original file line number Diff line number Diff line change
Expand Up @@ -1365,6 +1365,21 @@ the live relay acknowledgement protocol.
When a restored publisher is admitted, older absence-retirement records remain
validated history; the exact current cleanup proof supplies retention authority.

Before the main recovery intent, the operation journals and normalizes owned
bridge reservations that never launched (`reserved` with no relay intent) or whose
exact reservation-v1 helper has exited. The journal binds the original receipt,
boot, owner witnesses and complete same-run bridge inventory. It resumes confirmed
stop/removal interruptions, including canonical pending fence writes, without
removing a replacement socket, helper, allocation or reservation. A helper may
remove its own socket on exit; its exact process identity and retained private
socket receipt remain required. An originally live, unchanged bridge that exits
before main intent enrollment can be added to the journal with fresh exit proof.
Starting reservations, unknown or partial metadata, added assignments and replaced
identities refuse. Live helpers are left for the existing main recovery path;
other graphs are not selected. The normalizer does not run on an existing main
intent retry. Completed normalization journals are generation-bound history after
an independently verified retained restore.

After completion, retained restore can create a fresh owner;
`graph retire-recovered-publisher --run-id RUN --expect-owner OWNER` remains an
idempotent compatibility operation. Interrupted cleanup retains its journal for an
Expand Down
18 changes: 18 additions & 0 deletions packages/runtime-core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,24 @@ idempotent retry, and never restarts the pool. On failure, inspect its
retained graph evidence before another attempt. This fixture does not qualify
application authentication, dependency listeners, browser routing, or publication.

The two ignored tests under
`foreground::native_test::same_boot_absent_relay::bridge_normalization` use the same
300-second external watchdog and a caller-owned capacity-two development pool.
Set `HACK_LOCAL_TEST_NATIVE` to the absolute current `hack-native` binary to run
the final positive recovery through its public CLI. The read-only ignored
`removed_fixture_has_no_engine_resources_or_selected_guest_helpers` child audits
an exact `HACK_LOCAL_GRAPH_RUN` afterward, including engine absence and the
selected guest process identities.
Their pinned image additionally supplies BusyBox `httpd`; the target has two
HTTP-probed services on an internal network. They qualify an exited bridge plus a
never-launched reservation, and separately a second originally live bridge that
exits after normalization selection. Exact pending-fence writes, socket unlink,
owner unlink, registry removal and completion are interrupted and retried. A
replaced reservation refuses. Named markers and a live **unbridged** sibling must
survive; both owned graphs are removed through guarded cleanup on success. These
fixtures do not qualify simultaneous bridged siblings or application/browser
acceptance. Inspect failed-run resources before allocating another fixture.

The ignored foreground-owner fixture
`foreground::native_test::stop14::fourteen_services_retain_named_data_across_cleanup_and_restore`
qualifies normal retaining cleanup of thirteen running services and one completed
Expand Down
25 changes: 24 additions & 1 deletion packages/runtime-core/src/provider/graph/bridges.rs
Original file line number Diff line number Diff line change
Expand Up @@ -628,12 +628,24 @@ fn stop_slot(
store: &mut Store,
slot: u8,
) -> Result<(), CandidateError> {
stop_slot_fenced(candidate, engine, store, slot, &|| Ok(()), &|| Ok(()))
}
fn stop_slot_fenced(
candidate: &Candidate,
engine: &Engine<'_>,
store: &mut Store,
slot: u8,
fence: &dyn Fn() -> Result<(), CandidateError>,
finish_partial: &dyn Fn() -> Result<(), CandidateError>,
) -> Result<(), CandidateError> {
fence()?;
let a = store.slots.get(&slot).expect("selected slot");
super::super::publication::release(
candidate,
engine.guest().incarnation(),
Some((&a.run, &a.reservation)),
)?;
fence()?;
if a.relay.is_none() {
return Ok(());
}
Expand All @@ -645,12 +657,23 @@ fn stop_slot(
if a.phase != "stopped" {
store.slots.get_mut(&slot).expect("selected slot").phase = "stopping".into();
save(candidate, store)?;
fence()?;
relay::operate(engine, slot, &store.slots[&slot], "stop", None)?;
#[cfg(test)]
fault_pause(
&directory(candidate, &store.slots[&slot].run)?,
&store.slots[&slot].run,
"bridge-normalization-after-guest-stop",
)?;
fence()?;
store.slots.get_mut(&slot).expect("selected slot").phase = "stopped".into();
save(candidate, store)?;
}
fence()?;
finish_partial()?;
fence()?;
relay::operate(engine, slot, &store.slots[&slot], "remove", None)?;
Ok(())
fence()
}
pub fn inspect_bridges(candidate: &Candidate, run: &str) -> Result<Value, CandidateError> {
let engine = Engine::connect_cleanup(candidate)?;
Expand Down
5 changes: 3 additions & 2 deletions packages/runtime-core/src/provider/graph/bridges/cleanup.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
//! Retained bridge selection survives registry release and graph archival.
use super::*;
pub(crate) mod normalization;

#[derive(Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
Expand Down Expand Up @@ -964,7 +965,7 @@ mod tests {
selected: BTreeMap::new(),
}
}
fn live_selection() -> (Selection, Store) {
pub(super) fn live_selection() -> (Selection, Store) {
let mut selected = selection(1);
selected.boot = "11111111-1111-1111-1111-111111111111".into();
selected.capacity = 3;
Expand Down Expand Up @@ -1009,7 +1010,7 @@ mod tests {
(selected, store)
}

fn receipt_for_assignment(selected: &Selection, assignment: &Assignment) -> Receipt {
pub(super) fn receipt_for_assignment(selected: &Selection, assignment: &Assignment) -> Receipt {
serde_json::from_value(json!({
"version":1,
"run":selected.run,
Expand Down
Loading
Loading