A verification claim needs an observable result
A patch describes a proposed change. A test run observes its behavior under specified conditions. A deployment receipt identifies the released version. Checking a real user flow observes that version in its environment. Keep these claims separate because each requires different evidence.
Begin with the reported failure and its contract. If free shipping starts at a subtotal of 50, the value 50 is a boundary case. A function using subtotal > 50 handles 60 correctly but fails at 50. Running only 60 gives no evidence about the defect you were asked to fix.
A useful regression test fails on the original implementation and passes after the repair. Run nearby cases as well: 49 should remain ineligible and 60 should remain eligible. These cases check the threshold’s neighboring partitions. They do not establish behavior for taxes, currencies or invalid amounts unless those belong to the contract and the tests.
Review both the changed lines and the affected behavior
Inspect the diff for unrelated edits, altered access checks and changed assumptions. A small patch can affect many callers, while a large test suite can miss the relevant branch. Select checks that distinguish the old and new behavior, then cover the nearby behavior that must remain intact.
The lesson’s fixture changes a strict comparison to an inclusive comparison. Select and run the boundary, below-threshold and above-threshold checks. The table executes both functions on those inputs and compares each result with the contract. This is a real deterministic fixture run, with no model or remote code execution.
In the assessment, the proposed access-control patch allows previews too broadly. The normal owner case still succeeds. A private-item preview by another user exposes the error. Finding that failure is successful verification work, even though it means rejecting the proposed patch.
Report the checked scope and keep the release decision explicit
Report the exact checks and their outcomes. “Three shipping-threshold fixtures passed” identifies the evidence. “The checkout works in production” would require a deployed-version check and an actual checkout flow. An assistant’s confident completion message supplies neither.
When a check fails, preserve the input and observed output, explain the violated contract and return to the patch. After the next edit, rerun the failed check and the relevant nearby cases. Avoid silently removing a failing case to obtain a green result.
When checks pass, state what remains unverified. Visual changes need inspection of the rendered interface; account changes need the applicable account state; external side effects need their own receipts. A release decision can then use the evidence without treating local execution as proof of a broader result.
Try the skill
Contract: free shipping applies to a subtotal of 50 or more. Inspect the JavaScript proposal, select the checks and run them. Choose a conclusion that matches the evidence.
JavaScript
// Beforereturn subtotal > 50; // Proposed changereturn subtotal >= 50;Primary references
- Hypothesis: testing a specified property over generated inputs ↗
- OWASP: grade intended outcomes and report limits ↗