Test strength and documentation content

Tracked by pm-zclzll and pm-dvwm.

SDK mutation testing

pnpm quality:mutation uses Stryker's Vitest runner to mutate the SDK cursor and external-dependency primitives. The declared partition in scripts/release/sdk-mutation.json is deliberately bounded. It does not claim a mutation score for the entire SDK. The existing behavioral tests run without retries; no production functions are mocked in this partition. All-source coverage remains a separate mandatory 100/100/100/100 gate.

The sandboxed runner creates external temporary project/global trackers, disables external telemetry, removes ambient source-workspace references, and copies mutation sources without the real tracker. It holds the build lease until the engine exits and removes the owned temporary directory afterward. The normal source tree is never mutated. Every run executes all mutants; incremental results cannot supply stale evidence. Four workers and per-mutant timeouts bound this partition inside the existing 20-minute static job. The runner fixes sandboxing, fresh execution and the complete mutator set after loading configuration, so configuration overrides cannot disable those controls.

The gate verifies source bytes, exact module scope, nonempty mutant populations, and unique mutation identities. Only assertion-killed mutants contribute to the raw score. Timeouts, compiler errors, uncovered mutants and ignored mutants fail. A survivor requires an exact identity, a behavioral equivalence explanation, and an owning pm item in sdk-mutation-baseline.json. The initial seven waivers cover redundant guards and JavaScript serialization and absent-date equivalences; there are no blanket mutator exclusions or source-ignore directives.

Inspect .cache/mutation/report.json for individual mutations and .cache/mutation/receipt.json for the gate verdict. On a new survivor, first add a behavioral assertion to the existing test that owns the contract. If the mutation is observably equivalent, record its exact identity and justification for review. Remove waivers that disappear or become killed. Improvements require raising minimumScore; CI compares the floor and module scope with the PR base and refuses regression. Do not round the score when updating the baseline.

Both pnpm quality:static and the existing required hosted static job run the partition, as do nightly and release static validation. The runner rejects command-line mutation policy overrides. The static chain uses --prebuilt to reuse its completed build while still acquiring the build lease and rejecting an incomplete generation; standalone mutation runs retain their default build. Extending the partition requires adding the module and its behavioral tests together, measuring runtime, and reviewing every survivor. Gate unit tests inject report/engine boundaries to prove failure paths; the actual Stryker execution is an additional required gate.

Documentation content ratchet

pnpm quality:docstrings parses documentation comments across runtime sources, packages, scripts, plugins and executable examples. It detects four known filler templates, including wrapped comments, without matching quoted strings or ordinary comments. Module, exported-declaration and member documentation coverage remain independently enforced by the existing static gate.

The initial census found 4,192 filler comments. This delivery removes ten from the Commander alias contract and records the remaining per-file counts. The remaining debt is visible; an enforced inventory is not a claim that every existing comment is now informative.

When editing a declaration, replace its filler with the actual behavior, precedence, units or invariants. Do not merely restate the identifier. After reducing a file's count, run pnpm quality:docstrings:update and commit the smaller baseline. The update refuses growth, including in new files. Counts cannot be traded between files, and CI refuses baseline increases against the PR base via PM_QUALITY_BASE_REF. Removed filler cannot silently return by leaving a stale larger ceiling.


Quality_evidence remote
Ein Problem melden