Skip to content

test(projects): measure released extraction contract boundaries - #176

Merged
leemour merged 1 commit into
mainfrom
test/project-extraction-contract
Oct 19, 2026
Merged

leemour merged 1 commit into
mainfrom
test/project-extraction-contract

Conversation

@leemour

@leemour leemour commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Project-context claims need more than valid JSON and a matching quote. Add a bounded, manual synthetic study of released @wirecat/cli-core/documents extractMemories on Core0.358.0/Base0.47.1, using 33 fake-completion cases and the existing external-I/O guard. Record package integrity, executed function/entry/fixture/runner hashes and metadata-only outcomes; link the study from the project index and metric process.

The study preserves observed boundaries: unknown kinds reject, unsupported fields are accepted then stripped, length/count limits and exact substrings are enforced, and one bad quote rejects a batch. It explicitly demonstrates accepted contradictory claims and wrong attribution using a shared exact quote. External message/mail/document/meeting labels remain outside the SDK contract. Fake responses are not an LLM quality, semantic entailment or attribution evaluation.

Validation: 33 cases matched (16 accepted/17 rejected), targeted Biome, full docs check, targeted Markdown and spelling, staged gitleaks, and diff checks pass. All commands used isolated HOME/XDG/store/state/TMPDIR; retained SDK prefix manifests and lock integrity matched exact releases. No store API, model/network call, live account, new runtime dependency, CI suite or frozen benchmark modification. No production extraction/proposal behavior changed. The installation lock hash is run-specific, not a retained portable dependency lock.

@leemour
leemour merged commit 53a3c26 into main Oct 19, 2026
1 check passed
@leemour
leemour deleted the test/project-extraction-contract branch October 19, 2026 11:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant