Skip to content

ci(workflows): make required gates complete and PR feedback bounded #1633

Description

@jkim323

A green CI summary must prove that every selected required check executed successfully, and release automation must wait for the complete main validation graph. PR feedback also needs bounded execution and machine-readable evidence without conflating superseded runs with failures.

Extend the existing pr-validation-summary contract instead of replacing its required-check name. Keep advisory policies distinct from mandatory validation and preserve noncancelling main and release execution.

Acceptance Criteria

  • A selected required job must succeed and publish execution evidence; skipped is accepted only when its recorded selector says work was not required.
  • Required reusable workflows expose work-status, expected-count, executed-count, and an artifact or report identity where applicable.
  • Matrix and internally filtered workflows publish one aggregate outcome; contradictory counts, missing evidence, and selected planned skips fail closed.
  • Main's aggregate summary includes every mandatory validation, full lock consistency, CPU smoke, fuzz, and runtime-image check including VLA.
  • Release automation depends on the complete main summary and cannot run after a failed, cancelled, missing, or unexpectedly skipped result.
  • The declarative CI contract and parsed workflow graph detect missing mandatory jobs, stale gate entries, invalid outcome contracts, and permission-profile mismatches.
  • PR validation uses workflow-and-PR-scoped cancellation that cannot cancel another PR or main/release work.
  • Ordinary lint, selector, and summary jobs have a 10-minute maximum; unit and Terraform jobs 20 minutes; documentation builds 15 minutes; existing smoke budgets remain until measured evidence supports a change.
  • Selected unit suites publish test counts and uniquely named machine-readable reports while preserving coverage thresholds.
  • Summaries include comparison SHAs, selection reasons, tool versions, executed and skipped counts, required and advisory outcomes, first failing operation, and artifact links without secrets.
  • Contract and behavior tests cover failure, cancellation, unexpected skip, planned skip, empty evidence, full success, cold and warm caches, and controlled transient and persistent failures.
  • Hosted PR and post-merge evidence remain separate from local validation; no test invokes release mutation or shared-main failure injection.

Scope

Use existing artifact uploads and summaries. Do not add merge_group, pull_request_target, new observability infrastructure, blanket retries, or release-concurrency cancellation.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

ci/cdCI/CD pipeline and automationenhancementNew feature or improvement requestinfrastructureInfrastructure and platform changestestingTesting-related issues

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions