Edit on GitHub

Performance and scale

Tracker: pm-mi2x under the pm-9rxu scale-out initiative. CLI transport overhead is tracked by pm-yse5dt.

pm treats wall time, memory, and agent token cost as one performance contract: project management is context management, so a fast command that emits an unbounded payload is still slow for its caller.

Current targets

Every measured CLI and SDK operation targets:

  • p95 wall time at or below 1,000 ms;
  • default output at or below 5,000 estimated tokens;
  • no feature, flag, output, history, or validation loss.

The committed regression budgets in scripts/bench/scale-budgets.json protect the current baseline while the product targets stay fixed. Short local checks use the best observed latency as a regression floor; statistically meaningful runs of 20 or more use p95. Both receive 25% baseline headroom plus a 25 ms scheduler/filesystem noise margin. Reports always retain min/p50/p95, and product-target status always uses p95. This lets a busy workstation detect deterministic code slowdowns without pretending that three samples produce a meaningful p95 or hiding the real tail-latency target.

Reproducible fixtures

Generate a deterministic isolated workspace without touching the current tracker:

pnpm build
pnpm benchmark:scale:generate --output /tmp/pm-scale --items 10000 --mode direct
PM_PATH=/tmp/pm-scale/.agents/pm PM_GLOBAL_PATH=/tmp/pm-scale-global pm validate --json

Named tiers are smoke (100), ci (10,000), large (100,000), and million (1,000,000). The ci name describes the fixture size for compatibility with existing benchmark data; these performance workloads run locally and are intentionally absent from GitHub Actions. direct writes bounded batches for large fixtures; sdk uses the public SDK atomic-write and history primitives. Both modes serialize the same deterministic item and history bytes.

The fixture models built-in item types, open/in-progress/blocked/closed/canceled states, parents, dependencies, tags, bodies, comments, notes, learnings, and one valid history stream per item. It refuses to generate inside this repository. --force replaces a non-empty target only when that target already contains the scale-fixture manifest, so a mistyped arbitrary directory is never recursively removed.

Benchmark runner

Run CLI cold-process and SDK warm-process measurements together:

pnpm benchmark:scale --items ci --iterations 3 --transport both --check

The JSON report records fixture generation time plus one excluded warmup observation and measured p50/p95/min/max latency, peak RSS on Linux, output bytes, and estimated tokens for list, get, next, context, search, create, and claim. The warmup exposes initial index-build cost while regression percentiles measure the continuously warm derived-index contract across real cold CLI processes and in-process SDK calls. Run the committed local regression check with pnpm benchmark:scale:check. GitHub-hosted runners do not execute the scale suite, which keeps expensive performance work off Actions and avoids consuming hosted-runner capacity.

The scale report also records the CLI-minus-SDK transport delta for each common operation and gates it independently from the absolute CLI/SDK budgets. This separates process/bootstrap cost from storage or domain cost instead of allowing one transport to hide a regression in the other. The synthetic workspace has no installed extensions, while the SDK client explicitly opts out, so both paths measure the same core operation without project package work. CLI mutations do not inject PM_AUTHOR; they exercise the normal harness/default attribution resolver. The 10,000-item delta budgets are measured ratchets. The 100,000-item delta bounds are conservative ceilings derived from the existing absolute CLI budgets until a stable large-fixture measurement can replace them; they do not weaken either transport's existing absolute gate.

For a fast, deterministic one-item cold-start floor covering get, list, context, next, create, and claim, run:

pnpm benchmark:transport:check

The committed CLI transport report contains best/p50/p95 measurements and a stage breakdown. SDK entrypoint import costs are independently gated with pnpm benchmark:sdk-entrypoints:check and documented in the SDK entrypoint report.

The metadata read cache is rebuildable derived state. Workspaces with at least 500 indexed items use its directory-validated fast path to avoid per-item stats; validation and recovery can force a canonical source scan. Every metadata, body, and collection tier carries one source_cursor. A small manifest exposes the base cursor and item count without parsing the full index. A supported create, update, move, or delete acquires the cross-process derived-index writer lock before the authoritative item commit and atomically publishes one collapsed delta containing every compatible tier projection, directory signatures, and the next cursor before releasing the lock. Mutation cost therefore follows changed items rather than total workspace size. A torn, corrupt, or base-mismatched delta is rejected and rebuilt from source.

Large indexes also carry runtime/metadata-query-index.sqlite, a rebuildable row/index projection with the same context fingerprint and effective source_cursor. Source scans replace it atomically; supported mutations update it inside the existing derived-index writer critical section. Bounded default-order light list pages query only their result window plus a count, instead of parsing the full JSON cache. Unsupported filters, heavy body/collection projections, trees, runtime fields, active read hooks, missing files, stale cursors, and corrupt databases fail closed to the established authoritative/JSON path. The SQLite file is optimization state and is safe to delete or rebuild with pm reindex.

The same database carries bounded FTS candidate retrieval for the opt-in create/copy duplicate-governance probe. Minimal/default governance keeps the probe off and performs no query on the create path; advisory/strict modes reuse one dependency-light SDK scorer with the package-owned dedupe audit. Mutation history events use a separate rebuildable ordered SQLite projection so pm events --since <cursor> catch-up cost follows new history rows instead of workspace size. See SDK Context Coordination.

SDK hosts that commit authoritative item documents outside the stock mutation commands use acquireItemMetadataDerivedIndexLock and refreshItemMetadataDerivedIndex from @unbrained/pm-cli/sdk around the same commit boundary. Repeated writes collapse by item path in the delta instead of growing an event log; a later source scan compacts the projection into fresh base tiers. Projection failure removes the rebuildable tiers, delta, and manifest and returns a warning; it never rolls back or outranks the authoritative item/history write. Small workspaces without an active index receive a no-op release function and retain direct external-edit detection.

Context-signal snapshots follow the same rule: they are versioned and cursor-stamped, never authoritative, and rebuild from the metadata index or source-scan fallback when missing, stale, or corrupt. The stock context and next projections use separate snapshot namespaces, persist only caller-independent signals, and recompute author, time, semantic, and usage overlays on every read. SDK hosts can use readWorkspaceContextSignals for the same automatic cursor binding or compose ContextSignalStore directly, while retaining explicit fresh/rebuilt and derived_index/scan_fallback diagnostics. See Context relevance and packing.

At 10,000 items, context and search also build runtime/workspace-memory.json: compact calendar-epoch and epic-lineage summaries over completed history. The snapshot is cursor-bound, token-bounded when attached to context, searchable without expanding closed rows, and skipped entirely below the threshold. Missing, stale, corrupt, or unwritable memory state rebuilds or degrades without outranking item documents or history.

To refresh a baseline after an intentional, measured improvement:

pnpm benchmark:scale --items ci --iterations 5 --transport both --update --headroom 1.25

Review the report and budget diff together. Never update a budget merely to silence a regression.

Startup and observability

Sentry is loaded only when an error-reporting path actually initializes it. Disabled and normal successful commands do not resolve or compile the @sentry/node → OpenTelemetry → Undici graph. The loader uses the package's supported CommonJS export after the opt-out gate, preserving the repo-wide ban on dynamic/inline imports while keeping enabled capture, sanitization, and flush behavior intact.

Performance remote
Report an issue