Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ jobs:
"$RUNNER_TEMP/config-schema/bin/python" scripts/check-config-schema.py
- name: Test transport, projections and compiled bundle without host toolchains
run: |
bunx ultracite check scripts/build-config-compiler.ts scripts/check-config-compiler-cli.ts tests/config-compiler-build.test.ts tests/native-config-compiler.test.ts tests/native-config-command.test.ts tests/native-config-dto.test.ts
bun test tests/config-compiler-build.test.ts tests/native-config-compiler.test.ts tests/native-config-command.test.ts tests/native-config-dto.test.ts
bunx ultracite check scripts/build-config-compiler.ts scripts/check-config-compiler-cli.ts tests/config-compiler-build.test.ts tests/native-config-compiler.test.ts tests/native-config-command.test.ts tests/native-config-dto.test.ts tests/native-project-inputs.test.ts tests/native-project-inputs-acquisition-race.test.ts
bun test tests/config-compiler-build.test.ts tests/native-config-compiler.test.ts tests/native-config-command.test.ts tests/native-config-dto.test.ts tests/native-project-inputs.test.ts tests/native-project-inputs-acquisition-race.test.ts
bun run --cwd packages/cli typecheck
bun run build
bun scripts/check-config-compiler-cli.ts
Expand Down
17 changes: 9 additions & 8 deletions docs/guides/native-candidate.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,8 +16,8 @@ mise exec -- scripts/build-native-candidate.sh /absolute/new/hack-native-bundle

The destination must not exist. The bundle contains `hack-native`, the static Linux
ARM64 `hack-relay-guest`, compiled normal CLI `hack-cli`, the `hack-v5` entrypoint,
the compiled `hack-config-compiler` and generated `hack.project.schema.json`,
provider pins, this guide, `SHA256SUMS`, and one content-addressed shared MCP bundle
the compiled `hack-config-compiler`, generated `hack.project.schema.json` and
`hack.local.schema.json`, provider pins, this guide, `SHA256SUMS`, and one content-addressed shared MCP bundle
under `mcp/BUNDLE_ID/`. Its manifest and native adapter, owner, and compiled backend
are included in the outer checksums. The relay uses
the committed guest Cargo lockfile and Zig linker wrapper, with a separate build
Expand All @@ -39,12 +39,13 @@ selections, set `artifact` to this bundled file's absolute path and
`artifact_sha256` to its entry in `SHA256SUMS`; the runtime verifies it again before
delivery. The `native-stream-relay` feature does not replace this dependency relay.

The compiler and generated schema stay beside `hack-cli` as one checksummed pair.
Packaging and installation refuse a partial, changed, or aliased pair. The compiler
is executable; the schema is private data. Older bundles without either file remain
installable and selectable. An existing channel with an older retained installer
must explicitly run `upgrade-manager` with the reviewed installer before selecting
a bundle with the new pair, or use a fresh channel root. This upgrades the manager;
The compiler and both generated schemas stay beside `hack-cli` as one checksummed
group. Packaging and installation refuse partial, changed, or aliased groups. The
compiler is executable; the schemas are private data. Older bundles with no compiler
or only the complete compiler/project-schema pair remain installable and selectable.
A local schema always requires the compiler and project schema alongside it.
An existing channel with an older retained installer must explicitly run `upgrade-manager` with the reviewed installer before selecting
a bundle with the new group, or use a fresh channel root. This upgrades the manager;
it does not migrate the retained runtime homes.

Shared MCP remains opt-in. Select the verified nested bundle with the existing
Expand Down
12 changes: 7 additions & 5 deletions docs/reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -490,7 +490,7 @@ hack branch open <name> [options]

## `hack config`

Read/write legacy config or validate an explicit native project file
Read/write legacy config or validate native project configuration

### Usage

Expand All @@ -504,7 +504,7 @@ hack config <subcommand> [options]
| --- | --- |
| `hack config get <key>` | Read a value from hack.config.json |
| `hack config set <key> <value>` | Update a value in hack.config.json |
| `hack config validate` | Validate an explicit native project file without starting workloads |
| `hack config validate` | Validate native configuration and selected local overlays without starting workloads |

### Options

Expand Down Expand Up @@ -571,22 +571,24 @@ hack config set <key> <value> [options]

## `hack config validate`

Validate an explicit native project file without starting workloads
Validate native configuration and selected local overlays without starting workloads

### Usage

```bash
hack config validate [options]
```

Uses the matching bundled Rust compiler. This experimental command does not discover a project, resolve secrets, or adopt native configuration for runtime commands.
Uses the matching bundled Rust compiler. Without --file, discovers a native project and resolves permitted worktree-local overlay settings. --file validates only the explicit document. Neither mode reads env values, writes state, or starts workloads.

### Options

| Option | Description |
| --- | --- |
| `--file <path>` | Required native project JSON file |
| `--file <path>` | Validate only this native project JSON file, without discovery or local overrides |
| `--profile <names>` | Comma-separated declared native profiles |
| `--path, -p <dir>` | Run a project command against a repo path (overrides cwd search) |
| `--env <name|base>` | Apply an optional env overlay by name (use 'base' to bypass overlays) |
| `--json` | Output JSON (machine-readable) |
| `--no-interactive` | Never prompt: apply documented defaults or fail with E_INTERACTIVE_REQUIRED (also via HACK_NO_INTERACTIVE=1) |
| `--help, -h` | Show help |
Expand Down
76 changes: 66 additions & 10 deletions docs/reference/native-config-compiler.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
# Native config compiler foundation
# Native configuration validation

This is an experimental, pure compiler for a bounded subset of the planned
`.hack/hack.project.json` format. The compiler does not discover projects, execute workloads,
`.hack/hack.project.json` format. The pure compiler does not discover projects, execute workloads,
import Compose, migrate data, decrypt environment values, perform host admission,
or change how existing projects run. A successful compile is syntax and semantic
validation, not backend capability or application acceptance.
validation, not backend capability or application acceptance. The CLI can acquire
the selected project and permitted local settings for offline resolution.

The CLI recognizes this filename as a project boundary. Native runtime and adoption
are not enabled yet: legacy project commands refuse with
Expand Down Expand Up @@ -76,7 +77,12 @@ versions refuse; omitted fields retain their documented defaults.
- Project `environment.default_overlay` may select a canonical named overlay;
omission selects base. Names must already match `[a-z0-9]+(?:-[a-z0-9]+)*`;
noncanonical spellings refuse rather than selecting a normalized different name.
Local override files are not accepted by this compiler.
Project null refuses; null is supported only in local settings and explicit
command selection.
- Project `worktree.auto_branch` and `worktree.inherit_local` are strict booleans,
both defaulting to true. They appear in the normalized plan. This validation
command uses inheritance policy; it does not create branch instances or execute
`auto_branch` behavior.
- Dependencies are `{service:"db",condition:"started"|"ready"}` or
`{job:"init",condition:"completed"}`. References must match the declared kind.
Ready dependencies require explicit readiness. Services and jobs share one name
Expand All @@ -102,26 +108,38 @@ must be absolute POSIX container paths without `..`.

Routes, shutdown/restart policies, host hooks/processes, endpoint references,
network/security/resources, cache protocols, backend options, arbitrary extensions,
and local/worktree policy are not yet implemented. They refuse rather than being
and local settings other than environment selection are not yet implemented. They refuse rather than being
silently dropped. This foundation does not replace the full native contract or
qualify a migrated advanced project.

## Protocol and diagnostics

The current CLI exposes explicit validation only:
The CLI supports project-aware and explicit-document validation:

```sh
hack config validate --json
hack config validate --path /path/to/native-project --env base --json
hack config validate --file .hack/hack.project.json
hack config validate --file .hack/hack.project.json --profile dev,test --json
```

`--file` is required; this command does not switch project discovery or runtime
execution to the native format. `--json` returns the normalized plan, including
Without `--file`, discovery selects a native project without touching the registry.
Legacy or absent projects refuse; mixed active inputs refuse with
`E_NATIVE_PROJECT_CONFLICT`. `--path` changes the discovery start. `--env base`
explicitly selects base; another canonical name selects that overlay. This selects
a name only: overlay existence, managed metadata, keys and required references are
not inspected, and no env values are read. Runtime/adoption commands remain fenced.

`--file` validates only that document, ignores all local files and performs no
project discovery. It cannot be combined with `--path` or `--env`.
`--json` returns the authored normalized plan, including
authored public literals and commands. Those values are intentionally visible;
diagnostic redaction does not turn the plan into a secret-safe storage format.

- `hack-config-compiler --protocol` emits
`{"transport_version":1,"authored_version":1,"plan_version":1}`.
`{"transport_version":1,"authored_version":1,"plan_version":1,"resolve_version":1,"local_version":1}`.
The CLI requires the two new capabilities for project-aware validation; an older
compiler can still serve explicit-file validation.
- `hack-config-compiler compile [--profile NAME]...` reads one UTF-8 JSON document
from stdin through EOF. Input is limited to 1 MiB and 64 nested containers. These
are parser safety bounds, not container resource or workload-count limits.
Expand All @@ -133,8 +151,46 @@ diagnostic redaction does not turn the plan into a secret-safe storage format.
location; errors for a missing property or CLI-selected profile may point to its
containing object. Input contents and parser excerpts never appear in messages.
- Invalid invocation exits 2 with fixed usage on stderr and no JSON on stdout.
- `hack-config-compiler resolve [--profile NAME]...` reads a versioned request:
`{request_version:1,project:"original JSON text",primary_local?:"original JSON text",checkout_local?:"original JSON text",explicit_overlay?:null|string}`.
JSON texts retain duplicate keys until Rust checks each document. Each document
is limited to 1 MiB, their combined text to 3 MiB, and the encoded request to
20 MiB to allow JSON escaping. Diagnostics add `document` identifying `project`,
`primary_local`, `checkout_local` or `request`.
- `hack-config-compiler generate DIR` writes deterministic
`hack.project.schema.json` (2020-12) and `native-config.ts` projections.
`hack.project.schema.json`, `hack.local.schema.json` (2020-12) and
`native-config.ts` projections.

## Local settings and worktrees

The optional `.hack/hack.local.json` is a separate versioned document:

```json
{"schema_version":1,"environment":{"default_overlay":null}}
```

Only `environment.default_overlay` is supported in this slice. Omission inherits;
null selects base; a canonical name selects that overlay. Other fields, workload
definitions, unversioned documents, duplicate keys and unknown versions refuse.
The effective selection is project default/base, then verified primary local,
current checkout local, then explicit `--env`. Each later present value wins.

Primary inheritance requires a verified linked Git worktree in the same repository
family and a primary checkout with native inputs. Different input families refuse;
redirected/nonregular input files refuse. `inherit_local:false`, CI and slim mode
exclude primary reads. Current checkout local settings remain available. Files are
read in place; no primary files, secrets, keys or generated state are copied.
Native projects nested below a Git checkout root currently use only their own
local settings; primary inheritance requires the project root to be the Git root.

Success adds `local_resolution` with selected `overlay` (null means base), `origin`,
worktree policy and `resolution_hash`. Local settings do not rewrite the authored
plan or its `semantic_hash`. The separate hash binds normalized supplied local
documents and explicit selection, including missing versus null. Neither hash
proves an atomic multi-file snapshot or provides an admission/freshness fence.
Future apply must recheck private input generations. Adding defaulted worktree
policy changes hashes relative to the earlier experimental compiler; do not reuse
historical compiler hashes as resource identities.

Duplicate keys, including escaped-equivalent keys in nested objects and arrays,
are rejected before map insertion. Graph cycle checking is iterative. Serialization
Expand Down
83 changes: 83 additions & 0 deletions packages/config-compiler/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Pure native configuration compiler

This standalone package validates an experimental subset of native Hack project
configuration. It performs no discovery, file acquisition, runtime admission,
decryption, process execution, or host-path resolution. The CLI owns acquisition
of verified document bytes. See the public
[compiler reference](../../docs/reference/native-config-compiler.md) for the
supported project grammar and remaining execution boundaries.

## Local resolution protocol

`hack-config-compiler --protocol` advertises `transport_version`,
`authored_version`, `plan_version`, `resolve_version`, and `local_version`, all `1`.
Existing `compile [--profile NAME]...` remains context-free. The new
`resolve [--profile NAME]...` operation accepts one UTF-8 JSON request on stdin:

```json
{
"request_version": 1,
"project": "{\"schema_version\":1,\"name\":\"example\"}",
"primary_local": "{\"schema_version\":1}",
"checkout_local": "{\"schema_version\":1,\"environment\":{\"default_overlay\":\"qa\"}}",
"explicit_overlay": null
}
```

`project` is required; the remaining document fields and `explicit_overlay` are
optional. Embedded documents are original JSON text, not pre-parsed objects.
Recursive duplicate keys, unknown fields, unsupported versions, and invalid shapes
are rejected. Each document is limited to 1 MiB, their combined decoded size to
3 MiB, and the encoded request to 20 MiB. All JSON parsing has a depth budget of 64.
These are parser resource bounds, not runtime workload limits.

A local document permits only `schema_version: 1` and optional `environment`,
which permits only `default_overlay`. Omission inherits, `null` selects base,
and a string selects a canonical overlay matching `[a-z0-9]+(?:-[a-z0-9]+)*`.
No normalization silently changes names.

The project supports optional `worktree` with strict boolean `auto_branch` and
`inherit_local`, each defaulting to `true`. Resolution applies the project overlay,
then primary local when inheritance is enabled, then checkout local, then an
explicit selection. Every supplied local document is validated, even a primary
ignored by `inherit_local: false`. The compiler reports policy; it does not create
branches or read Git metadata.

Successful output contains `transport_version: 1`, `ok: true`, the authored `plan`
and `semantic_hash`, plus `local_resolution` with `overlay`, `origin`,
`auto_branch`, `inherit_local`, and `resolution_hash`. Origin is `project`,
`primary_local`, `checkout_local`, or `explicit`. Locals never alter the authored
plan or its semantic hash. The resolution hash binds the authored hash, normalized
supplied locals (including shadowed or opted-out primary input), and explicit
selection presence/value. Formatting and key order do not affect either hash;
changing a permitted local selection affects the resolution hash even when another
layer shadows it. Host acquisition paths and decrypted secrets never enter this resolver. Authored
literal environment values remain explicit public plan data, just as in compile.

The normalized plan now materializes defaulted `worktree` policy, so authored
hashes from the earlier experimental compiler change. `plan_version: 1` remains
experimental; this package does not promise compatibility with cached plans from
the earlier prototype.

Failures exit `1` and return `ok: false` with diagnostics containing `document`
(`project`, `primary_local`, `checkout_local`, or `request`) and fixed `code`,
`message`, JSON `pointer`, `line`, and `column`. Values and parser error text are
not echoed. Pointers necessarily include authored key names; consumers must quote
or escape them for terminal display. Unsupported command arguments exit `2` with
fixed usage text on stderr. Success exits `0`.

## Generated contracts and checks

`generate <dir>` emits deterministic `hack.project.schema.json`,
`hack.local.schema.json`, and `native-config.ts` from Rust types. The schemas use
JSON Schema 2020-12. The shared project/local shape corpora are consumed by Rust
acceptance tests and the independent schema validator; graph semantic validation
remains separate from structural schema acceptance.

Use Rust 1.97.1 with the locked dependencies. From the repository root:

```sh
cargo +1.97.1 fmt --manifest-path packages/config-compiler/Cargo.toml --check
cargo +1.97.1 clippy --locked --manifest-path packages/config-compiler/Cargo.toml --all-targets -- -D warnings
cargo +1.97.1 test --locked --manifest-path packages/config-compiler/Cargo.toml
```
36 changes: 36 additions & 0 deletions packages/config-compiler/generated/hack.local.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "LocalConfig",
"type": "object",
"properties": {
"environment": {
"$ref": "#/$defs/LocalEnvironment",
"default": {}
},
"schema_version": {
"type": "integer",
"format": "uint32",
"maximum": 1,
"minimum": 1
}
},
"additionalProperties": false,
"required": [
"schema_version"
],
"$defs": {
"LocalEnvironment": {
"type": "object",
"properties": {
"default_overlay": {
"type": [
"string",
"null"
],
"pattern": "^[a-z0-9]+(?:-[a-z0-9]+)*$"
}
},
"additionalProperties": false
}
}
}
22 changes: 22 additions & 0 deletions packages/config-compiler/generated/hack.project.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,21 @@
}
},
"type": "object"
},
"WorktreePolicy": {
"additionalProperties": false,
"description": "Authored worktree policy; neither flag grants runtime admission or cleanup authority.",
"properties": {
"auto_branch": {
"default": true,
"type": "boolean"
},
"inherit_local": {
"default": true,
"type": "boolean"
}
},
"type": "object"
}
},
"$schema": "https://json-schema.org/draft/2020-12/schema",
Expand Down Expand Up @@ -498,6 +513,13 @@
},
"default": {},
"type": "object"
},
"worktree": {
"$ref": "#/$defs/WorktreePolicy",
"default": {
"auto_branch": true,
"inherit_local": true
}
}
},
"required": [
Expand Down
Loading
Loading