Repository navigation
fix: add a daily check that main is branch-protected - #48
Merged
Merged
Conversation
Main had no branch protection and no ruleset. Anyone with write access could push to it directly, force-push over its history, or delete it. That covers the admission ledgers and the workflows that issue and verify admissions, and nothing reported it. What changed: - scripts/check_branch_protection.py reads the ruleset rules for main and, when the token allows, its classic branch protection. It fails unless main requires a pull request and the validate-profile check from GitHub Actions, and blocks force pushes and deletion. When the token can read who may bypass those rules, it also fails if anyone can. - Profile consistency runs the check on its daily schedule and on manual dispatch, and files an issue when it fails. It doesn't run on pull requests, so it can't block unrelated changes. - main-branch-ruleset.json is a ruleset body that satisfies the check. An admin applies it with: gh api -X POST repos/OpenAdaptAI/.github/rulesets --input main-branch-ruleset.json - Tests cover the responses GitHub returns today (the check fails), classic protection, rulesets, rules split across both, and the committed ruleset body. This commit doesn't change any GitHub setting. The check stays red until an admin turns on protection for main. Ledger: B247 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Classic branch protection can list users, teams, or apps in bypass_pull_request_allowances. They push to main without a pull request even when enforce_admins is on, so the branch protection check treated that setup as fully bound. The check now reports the pull request requirement as bypassable when any allowance is listed, or when the allowance has a shape it doesn't recognize. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
abrichr
added a commit
that referenced
this pull request
Oct 14, 2026
* Stop the required validate-profile check from passing by being skipped The code part of this bug merged in #48: main-branch-ruleset.json requires a pull request and the validate-profile check, and a daily job fails while main is unprotected. Main is still unprotected today; the 2026-10-14 scheduled run failed and filed issue #49. The ruleset had a gap that would open once it is applied. validate-profile had a job-level condition and needed reject-lifecycle-app, so on a pull request the lifecycle App opens, the job was skipped. GitHub counts a skipped job as a passing required check, so that pull request would show validate-profile as passed although validation never ran. validate-profile now has no job-level condition or needs. Its first step refuses the lifecycle App and fails, before checkout. The check is red, not skipped, on App-triggered runs. A person closes and reopens the pull request to run validation as themselves. The new test reads the required checks from the committed ruleset and fails if the job behind any of them can be skipped. Ledger: B247 Signed-off-by: abrichr <richard.abrich@mldsai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Fail the skip test when the refusal step can be skipped or ignored The regression test for B247 checked that validate-profile has no job-level condition or needs, and that its first step refuses the lifecycle App. It still passed in three cases that weaken the refusal. A step-level if can skip the refusal. continue-on-error on the refusal step turns its failure into a success, so the job goes on and the check passes. The test now fails on both, and on continue-on-error at the job level as well. A mutation run on the workflow shows the earlier test missed these three cases. The new test catches them and the five cases the earlier test already caught. Ledger: B247 Signed-off-by: abrichr <richard.abrich@mldsai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Signed-off-by: abrichr <richard.abrich@mldsai.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The
mainbranch of this repository has no branch protection and no ruleset. Anyone with write access can push to it directly, force-push over its history, or delete it, with no review and no checks. That branch holds the admission ledgers and the workflows that issue and verify admissions. This change adds a daily check that fails and opens an issue untilmainis protected, plus the ruleset that satisfies the check. The protection itself is a repository setting that an owner applies after merge.Fixes
mainaccepted direct pushes, force pushes, and deletion with no review or required checks, and nothing reported it.scripts/check_branch_protection.pyreads the ruleset rules formain. When the token has Administration read access, it also reads classic branch protection. It fails unlessmainrequires a pull request, requires thevalidate-profilecheck from GitHub Actions (app 15368), and blocks force pushes and deletion. When it can read who may bypass those rules (a ruleset's bypass list,enforce_admins, or the pull request bypass allowances), it also fails if anyone can bypass them. Inprofile-consistency.yml, the newcheck-branch-protectionjob runs the check daily and on manual dispatch, andreport-unprotected-branchopens or updates an issue when it fails. The check doesn't run on pull requests.tests/test_branch_protection.py:TodayTests.test_command_exits_nonzero_on_the_observed_github_responses,TodayTests.test_unprotected_main_without_rulesets_fails_every_requirement,ClassicProtectionTests.test_each_missing_classic_setting_fails,ClassicProtectionTests.test_pull_request_bypass_allowance_fails_only_that_requirement,RulesetTests.test_ruleset_with_a_bypass_actor_fails,RulesetTests.test_status_check_must_come_from_github_actions, and 11 moremainshould have.main-branch-ruleset.jsonis an active ruleset forrefs/heads/mainwith no bypass actors. It blocks deletion and force pushes, requires a pull request with no minimum approvals, and requiresvalidate-profilefrom GitHub Actions. The required settings live in the script, so editing this file can't weaken the check.CommittedRulesetTests.test_committed_ruleset_satisfies_the_checkNot fixed here
mainstays unprotected until an owner applies the ruleset. The scheduled check fails every day until then.MINIMUM_APPROVALSand the ruleset file.verify-production-release-admissionisn't a required check. Other workflows call it, and it never reports on pull requests in this repository, so requiring it would block every merge.BRANCH_PROTECTION_READ_TOKENsecret with read access to Administration lets the check verify the bypass list.validate-profilebecause of its actor condition, and GitHub treats a skipped required check as passed. Those pull requests rely on review, not on the check.How it was tested
main, where the check doesn't exist, and pass on this branch.python3 -m unittest discover -s tests -p 'test_*.py'), up from 370 onmain.scripts/validate_evidence_registry.py,scripts/check_profile.py, andscripts/check_benchmark_claims.py --allow-recorded-driftpass.ruff checkreports nothing in the new files.actionlintreports one SC2016 info finding at line 62 ofprofile-consistency.yml, which also exists onmain.OpenAdaptAI/.githubmainwith an owner token and with no token, and reports all four requirements as not set. Against the classic protection onOpenAdapt,openadapt-flow, andopenadapt-desktop, it reads the real settings and reports that administrators can bypass them, becauseenforce_adminsis off there.Before merging
After merge, an organization owner applies the ruleset from the repository root, then runs the Profile consistency workflow by hand and confirms that
check-branch-protectionpasses:Until then, each daily run fails and comments on the issue "Branch protection on main needs attention".
The ruleset lets nobody bypass it, owners included. Every change to
main, including an owner's, then needs a pull request withvalidate-profilepassing. The lifecycle workflows already push review branches and open pull requests, andproduction-lifecycle-ref.ymlwrites only toproduction-lifecycle-feed, so they keep working.Classic branch protection also satisfies the check, but only with
enforce_adminson and aBRANCH_PROTECTION_READ_TOKENsecret that has read access to Administration. Without that secret, the check can't see classic protection and keeps failing.Decide whether to require one approval. That needs a second reviewer account.
🤖 Generated with Claude Code