How to Evaluate Agentic Code Before You Ship It
Generated code can be syntactically correct and still be wrong for the product. Evaluation must connect the implementation to the intended outcome and the real operating environment.
1. Scope: did the agent change only what it needed?
Start with the diff or equivalent change record. Unrelated refactors increase the review surface and make rollback harder. A focused change is easier to understand even when it crosses several technical layers.
Check protected assets explicitly. If a preloader, database or neighbouring service was out of scope, prove that it was preserved.
2. Correctness: does the outcome work?
Translate the original goal into observable checks. If an admin setting should hide a product, save the setting, reload the page and inspect the public catalogue. If a purchase should unlock a document, complete a sandbox payment and download the actual file.
Avoid accepting proxy evidence. A 200 response from one endpoint does not prove the entire user journey.
3. Failure behaviour: what happens when inputs are wrong?
Test invalid credentials, expired tokens, malformed URLs, missing files and unavailable external services. A safe system should fail clearly and preserve the last good state.
Error messages should help the operator repair the problem without exposing secrets. “Invalid destination: Product Name” is more useful than “Save failed,” while still revealing nothing sensitive.
4. Security and privacy
Confirm that authorization is enforced on the server, not only hidden in the interface. Check that private files are not part of the static allowlist and that download links cannot be guessed or reused indefinitely.
Collect only the data the workflow needs. Document what is stored, why it is stored and how a visitor can contact the operator.
5. Accessibility and readability
An interface that technically renders can still fail its users. Check font size, contrast, keyboard focus, form labels, error announcements and mobile layout. Motion should respect reduced-motion preferences.
For content pages, use semantic headings, readable paragraphs and clear navigation. These choices help people first and search engines second.
6. Persistence, deployment and rollback
Verify that saved data survives a process restart and that deployment does not overwrite it. Back up the previous code before replacing the running service. Confirm the target path and port.
After release, test the public domain and the most important neighbouring services. A successful local build is the beginning of production verification, not the end.