Every behavior has three sides:
- Intended: what the system should do (the spec)
- Implemented: how it does it (the code and design)
- Verified: what proves it works (the tests and evidence)
OpenSpec gives a lasting home only to the first.
What we ran into
We reviewed the specs from one of our first changes, a desktop app foundation. They passed validation, but many requirements weren't really behavior:
| In the spec |
The actual behavior |
Where the rest went |
| Framework security settings, named one by one |
"Interface code can't access the filesystem or OS processes" |
Design doc |
| "Service code must not import the UI framework" |
Not behavior at all; it's a code-structure rule |
Lint rule |
| "When the service is deliberately blocked… as observed by an external process" |
"When the service stops responding, the window stays usable" |
Test tasks |
Rewriting them made the specs cleaner. But the details we moved out now sit in the design doc, tasks and lint config, and nothing links them back to the requirements they support. Once the change is archived, nothing tells you how a requirement like "the service exits if the app is killed" is verified, or whether it is at all.
The gap shows up in two ways.
Symptom 1: specs fill up with implementation and test details
This doesn't always happen, but we've run into it: agents sometimes put framework details, component responsibilities and test procedures into requirements. Better guidance helps, but it isn't the root cause. Specs are the only artifact that outlives a change: design and tasks are archived with it. If a detail matters later, the spec is the only place it can survive.
Symptom 2: nothing links the three sides
Even when details go in the right place, you can't trace from a requirement to the code that implements it or the tests that verify it. So you can't easily see:
- which requirements are verified, and how
- which have no adequate test, for example because the test infrastructure can't check them end to end
- what a change will verify when it's proposed, and what it actually verified once done
Why it's hard
Everything is markdown. Requirements have no stable identity beyond their heading text, and markdown has no natural way to express links that tools can check. Any solution needs some way to identify requirements and scenarios that survives edits, plus a convention (or custom syntax and linting) for referring to them.
Directions worth exploring
- Stable identifiers for requirements and scenarios, so anything can point at them.
- Links from tests to specs: tests reference the scenario they verify, which keeps specs clean and lets tooling report coverage and gaps.
- A lasting home for implementation and verification context, so it doesn't have to live in the spec or disappear at archive.
- A verification plan at proposal time: which requirements will be verified by which tests, and where verification will be manual or missing, so the user can decide how to handle gaps.
What we're asking for
This needs careful, deep thinking before any implementation. The directions above are starting points from one experience, not a plan to build. The right answer may be several smaller fixes rather than one. Please work out what actually fits OpenSpec and its users, and treat these suggestions as input, not a spec.
Open questions
- What should the unit of linking be: requirement, scenario, or both?
- Should links live in the specs, in the tests, or in a separate index?
- How much structure can we add to markdown before it stops being easy to read and write?
Related: #900 (generating tests from specs), #1652 (archive requiring evidence), #1966 (skill references)
Every behavior has three sides:
OpenSpec gives a lasting home only to the first.
What we ran into
We reviewed the specs from one of our first changes, a desktop app foundation. They passed validation, but many requirements weren't really behavior:
Rewriting them made the specs cleaner. But the details we moved out now sit in the design doc, tasks and lint config, and nothing links them back to the requirements they support. Once the change is archived, nothing tells you how a requirement like "the service exits if the app is killed" is verified, or whether it is at all.
The gap shows up in two ways.
Symptom 1: specs fill up with implementation and test details
This doesn't always happen, but we've run into it: agents sometimes put framework details, component responsibilities and test procedures into requirements. Better guidance helps, but it isn't the root cause. Specs are the only artifact that outlives a change: design and tasks are archived with it. If a detail matters later, the spec is the only place it can survive.
Symptom 2: nothing links the three sides
Even when details go in the right place, you can't trace from a requirement to the code that implements it or the tests that verify it. So you can't easily see:
Why it's hard
Everything is markdown. Requirements have no stable identity beyond their heading text, and markdown has no natural way to express links that tools can check. Any solution needs some way to identify requirements and scenarios that survives edits, plus a convention (or custom syntax and linting) for referring to them.
Directions worth exploring
What we're asking for
This needs careful, deep thinking before any implementation. The directions above are starting points from one experience, not a plan to build. The right answer may be several smaller fixes rather than one. Please work out what actually fits OpenSpec and its users, and treat these suggestions as input, not a spec.
Open questions
Related: #900 (generating tests from specs), #1652 (archive requiring evidence), #1966 (skill references)