Compare patches, objective correctness, manual review, solution time, and known provider cost without treating availability or infrastructure failures as code quality.
A focused JavaScript maintenance task executed through the clean-room OpenCode and Codex pipeline, with public tests, a private objective evaluator, immutable patches, time, and provider-reported cost.
The first broad patch comparison: 53 coding-agent candidates solving the same ledger task, with availability kept separate from quality scoring.
Diagnose and repair duplicate IDs, exact money parsing, sales, refunds, invalid rows, and daily ledger totals.
Implement feature overrides and deterministic percentage rollout from a detailed behavioral contract.
Extract one shared event-filtering helper while preserving the public API and all existing behavior.
Explore an unfamiliar module layout, locate the correct formatter, and repair account status labels.
Older multi-task evidence covering bug fixing, feature work, refactoring, repository navigation, and edge-case tests. Audit found shared-history contamination, so this is not presented as a blind benchmark.
The public benchmark runner documents clean-room isolation, separate candidate testing and judging, objective evaluation, timing, cost policy, and immutable result artifacts.