Functional correctness
How completely the implementation follows override, flag, and rollout precedence.
Clean-room benchmark · Feature implementation
Feature test: implement deterministic feature rollout
This test measures whether a model can translate a detailed product rule into a complete, deterministic implementation while preserving inputs and the public API.
15 tested runtime routes
Public testsvisible correctness checks
Objective evaluatorprivate deterministic cases
4 criteriaseparate blind review
1 source filestrict change boundary
Test assignment
Resolve feature flags with overrides and percentage rollout
The model must implement
resolveFeature(config, context) from a written
contract. The result must be predictable for the same user and
configuration, and the function must not mutate either argument.
What the implementation must do
-
Reject invalid configuration, context, or feature names by
returning
false. - Give an exact boolean user override first priority, followed by a boolean global flag.
- Support enabled flag objects with optional integer rollout percentages from 0 through 100.
- Require a non-empty user ID for partial rollout.
- Assign users deterministically with the specified unsigned 32-bit hash over Unicode code points.
- Leave all input objects unchanged and preserve the existing public API.
How each reviewed solution will be scored
Each criterion receives 0–10. Public and objective checks remain separate from the manual score.
Reliability / edge cases
How safely it handles invalid inputs, boundary percentages, Unicode IDs, and immutability.
Maintainability / clarity
How clearly the code expresses the rules without unnecessary complexity.
Scope discipline
How strictly the change stays in the allowed source file and preserves the API.
Overall is the arithmetic mean of the four scores.