Functional correctness
How fully the patch restores the documented ledger behavior.
Clean-room nomination · Bug fixing
Bug-fixing test: repair ledger reconciliation
This test measures whether a coding model can diagnose an existing accounting defect and repair it without changing the contract or widening the patch.
Nomination prepared · results pendingPublic testsvisible correctness checks
Objective evaluatorprivate deterministic cases
4 criteriaseparate blind review
1 source filestrict change boundary
Test assignment
Fix duplicates, sales, refunds, and invalid ledger rows
The model receives a small ledger reconciler with seeded defects. It must understand the documented behavior, find the incorrect implementation, and make the smallest correct repair while preserving both exported functions and the result format.
What the repaired code must do
- Validate every row and count unusable entries without crashing.
- Process only the first occurrence of a transaction ID and count every later occurrence as a duplicate, including duplicate refunds.
- Parse non-negative dollar amounts exactly, accepting optional
$and correctly grouped commas without rounding malformed values. - Group totals by account, currency, and the row's UTC date.
- Subtract a refund from its previously processed sale's account and currency; count an unmatched refund as an orphan.
- Keep totals sorted and count as processed only rows that actually affect them.
How each reviewed solution will be scored
Each criterion receives 0–10. Public and objective checks remain separate from the manual score.
Reliability / edge cases
How safely it handles malformed rows, exact money, duplicates, dates, and orphan refunds.
Maintainability / clarity
How readable, direct, and easy to reason about the repair is.
Scope discipline
How strictly the patch stays within the allowed file and avoids unrelated work.
Overall is the arithmetic mean of the four scores.
Results are not published yet
This page defines the test before model execution. The future table will show public and objective results, four review scores, time, cost, and runtime route.