Scheduled release reliability

Tracked by pm-e70zh5 and pm-44u3wt. Trigger provenance and installed-package diagnostics are tracked by pm-prrlce and pm-q7c36n.

The Release Reliability workflow evaluates the preceding 30 days after a default-branch Auto Release or Nightly Validation completes, daily at nominal 04:35 UTC, and on manual dispatch. It saves a JSON report and fails its own check when policy is breached. The report is independent of candidate correctness: historical failures must remain visible while a reviewed repair can still pass publication gates.

Policy and clock

The versioned policy declares the nominal release cron as 02:35 UTC, a 60-minute dispatch allowance, a maximum 10% failure rate, and at least seven completed observations. These are explicit operational targets, not thresholds fitted to make the current history green. Missing samples yield an insufficient-data failure, never a healthy zero rate.

Dispatch delay measures created_at against the nearest preceding nominal daily occurrence. Queue delay separately measures run_started_at - created_at. GitHub does not supply the intended scheduled timestamp: the inferred occurrence cannot disambiguate delays greater than 24 hours. Missing the latest occurrence whose 60-minute allowance has elapsed produces missing_scheduled_run. Before 03:35 UTC, absence of today's run is still within the allowance; afterward, inspect the report to distinguish missing dispatch, a pending attempt, and a completed failure. A late run is visible even when it subsequently succeeds.

The release version and one-per-day guard use the UTC execution date captured by release preparation, not the inferred cron occurrence. A delayed run cannot backdate a tag. Reporting never creates, retargets, retries, or publishes a tag. The unchanged release recovery rules apply.

The independent full-history ledger report also reconciles public registry versions, release tags and generated sections. It preserves every historical exception and fails on new or changed drift.

Evidence and denominator

The collector paginates Auto Release runs over an explicit half-open UTC window and checks returned counts and unique IDs. It retrieves attempt 1 when a run was rerun. A later successful attempt, manual dispatch, issue recovery, or closed blocker cannot change the original scheduled failure. Pending runs are listed separately; all completed non-success conclusions count as failures. The native scheduled denominator and policy remain separate from the morning dispatcher series. Each new dispatch declares trigger_origin; the workflow run name and attempt receipt preserve it. If setup fails before the receipt can be uploaded, the collector reads the declared run name. Historical dispatches without either declaration appear in pre_attribution_gap; the failed 2026-09-28 morning dispatch cannot be attributed from its old run record. Dispatcher rows retain creation time, queue delay, outcome, and failure stage. The origins map also retains independent native_schedule, operator, and blocker_retry series, each with original counts, failure stages and outcome evidence completeness. Undeclared historical runs remain explicit attribution gaps; event or actor never supplies an invented origin. These diagnostic series do not alter the native scheduled policy denominator. They omit nominal occurrence and dispatch delay because the native 02:35 UTC cron is not the dispatcher's clock; the scheduled timing policy does not apply to those rows. API errors, truncated pages, ambiguous artifacts, and mismatched run identities fail collection rather than silently reducing the denominator.

Auto Release saves release-observation-<attempt> artifacts for 90 days. Each contains only run/attempt identity, event, trigger origin, outcome and failure-stage name. Gate stdout, stderr, credentials and environment values are not archived. A confirmed publication, verified existing same-day release, unchanged source, tracker-only changes, and an empty changelog are distinct outcomes. Failed gates retain their structured stage; older failures fall back to failed job/step names from GitHub. Receipt writing and upload are non-blocking so an evidence-service outage cannot change publication success into a release blocker. Missing or expired success receipts are unknown_success, with outcome_evidence_complete: false; success alone does not prove publication.

Reports retain the input policy, window, per-run observations, outcome counts, stage counts, denominator, failure rate and violations. Download a report using gh run download <report-run-id> --name <artifact-name> --dir <temporary-directory>. The pure evaluator in scripts/release/release-reliability.mjs permits offline replay with a fixed clock and captured input; the collector is read-only and downloads only data, never executable source. The reporting workflow checks out the default branch rather than the triggering run's code and has read-only token permissions.

Registry acceptance records every exact-package install attempt. A timeout, signal, missing executable, other spawn error, and nonzero exit each keep their own classification, exit status, signal, error code, and a bounded redacted stderr excerpt. Only an explicit registry 404 is retried as possible metadata propagation; successful recovery retains the earlier failure receipts. The control and candidate sessions share a 13-minute deadline under the 15-minute hosted acceptance job limit.

The report is a rolling observation, not a permanent archive of all releases. Retain exported receipts externally if a longer historical window is needed. It does not change the manual same-day version policy or propagate upstream origin through historical lightweight tags. Those remaining outcomes retain their existing PM owners.

Nightly recurrence and public SDK

pm-2zjs0g owns the nightly recurrence meter. The same reporting workflow builds trusted default-branch code and writes a separate nightly-reliability-<run>-<attempt> artifact, including when release reporting is already red. Its versioned policy requires seven completed original attempts per family and at most 10% failures over 30 days. Breaches fail the operational report and appear in the workflow summary; they remain distinct from checks evaluating a candidate repair.

The collector derives current leg identities from nightly.yml and each run's applicable population from the workflow at its original head SHA. Immutable definitions are cached within one census; legacy OS/Node matrices and today's labelled shards both retain their actual generations. Newly introduced families never gain synthetic pending attempts in older runs. It retains historical observed legs, retrieves attempt 1 after reruns, and reads both open and closed alert issues plus their complete comment pages. Original failures, distinct affected UTC days and observed closure durations survive alert repair. Every new alert carries the stable PM recurrence owner and a platform/Node family identity. Counts do not depend on a maintained list of occurrence IDs. An absent job is pending evidence; an empty family has a null rate and fails coverage.

Alert closure is the recorded latency endpoint, not proof of a product fix. When one alert episode links several failed runs, its shared closure is the endpoint for each recorded occurrence's alert-closure latency. This does not assert that an independent per-attempt product repair was verified. Product versus infrastructure cause stays unknown until independent evidence classifies it. Repairs at or beyond the report cutoff remain unresolved in that historical window. API errors, incomplete search results, missing pages, duplicate identities and ambiguous jobs refuse a complete-census claim.

Packages can reuse the harness-independent evaluator through evaluateReliabilityWindow from @unbrained/pm-cli/sdk/governance. Supply ReliabilityAttempt records with unique id, stable family, started_at, original outcome and optional repaired_at/cause, plus an explicit ReliabilityWindowPolicy population, half-open window, maximum failure fraction and minimum completed count. The typed result preserves per-family findings, unknown causes and repair observations. No GitHub dependency, filesystem tracker, clock read or network call is required by this SDK primitive.

For reproducible offline evaluation, retain the report's attempts and policy with its exact window; rerun the SDK evaluator against those records. The GitHub adapter accepts a caller-chosen clock and a 1–90 day policy window, allowing historical family reconstruction while keeping the production 30-day policy fixed.

Provider contracts: workflow run attempts and completion-triggered workflows.