AI-Generated Laravel Code Can Pass Every Test and Still Be Wrong

A green test suite answers an important question: did the code satisfy the behaviors the tests covered? It does not prove the implementation belongs in your application. Laravel’s team recently reported that AI coding agents can pass every Laravel Boost evaluation and is now focusing on whether their solutions are idiomatic and correct per token. That distinction should change how teams review generated code. An agent can produce a working controller that bypasses policies, duplicates framework functionality, fights the service container, or creates a maintenance trap. Passing tests is the start of review, not the end.

Functional correctness is only one dimension

Production code must be secure, understandable, observable, compatible with supported versions, and consistent with the project’s architecture. Tests often describe outputs but not the preferred mechanism. Two solutions can return the same JSON while one uses established resources and policies and the other assembles arrays inside a controller.

Review generated code against three contracts: desired behavior, framework conventions, and local project patterns. A failure in any one can create future cost even when today’s assertion passes.

Give the agent versioned context

Laravel Boost improves agent output by exposing installed package versions, framework documentation, schema information, routes, logs, and project guidelines. That reduces a common failure mode: code copied from a different Laravel generation or an incompatible Livewire, Inertia, or testing API.

Context must still be bounded. Database access should default to safe read operations, secrets should stay out of prompts, and production tools should not be attached to a coding session without a clear need. More context improves relevance but also increases the impact of a compromised instruction.

Make local conventions executable

A style guide nobody reads will not reliably shape human or agent output. Encode expectations in formatters, static analysis, architecture tests, base test cases, and package boundaries. Tell the agent where new behavior belongs and provide one representative example from the repository.

Laravel’s latest Boost work includes inferring project conventions and journaling decisions. Treat inferred rules as proposals. Review them, correct mistaken patterns, and avoid fossilizing accidental legacy code as the new standard.

Review the diff, not the explanation

Generated summaries are useful navigation, but the diff is the evidence. Inspect authorization, validation, query count, transaction boundaries, error paths, logging, retries, caching, and test quality. Watch for broad changes that were not required by the task and dependencies added to avoid a small amount of native code.

Ask whether the tests would fail if the implementation were subtly wrong. An agent can create assertions that merely restate its output, mock away the important integration, or skip the unhappy path.

Use a Laravel-specific review checklist

• Are routes protected by the intended middleware and policies?
• Does validation use a Form Request or established project pattern?
• Are Eloquent relationships loaded without avoidable N+1 queries?
• Do writes need a transaction, lock, or idempotency key?
• Are queues, events, resources, and notifications used consistently?
• Does the change preserve supported PHP and package versions?
• Do tests cover authorization and failure as well as success?

A short framework-aware checklist catches more than asking whether the code “looks clean.”

Measure rework, not generated lines

Lines produced and tasks completed are easy metrics and poor incentives. Track review time, defect escape rate, reverted changes, security findings, and how often maintainers substantially rewrite the output. Compare similar task classes: scaffolding, bug repair, query optimization, and architectural change have different risk profiles.

If generated code is fast to create but slow to understand, the tool moved work rather than removed it. Correct code per token is an interesting research measure; correct code per unit of team attention is the operational measure.

Use agents to accelerate judgment, not replace it

Begin with bounded changes whose expected architecture is already clear. Require tests, run the complete quality pipeline, and keep an experienced reviewer accountable for the merge. Expand scope when evidence shows the team is saving time without increasing rework.

Laravel’s post “AI coding agents pass tests. Can they write idiomatic Laravel?” frames the next evaluation challenge. Teams should adopt the same standard: working output is necessary, but maintainable Laravel is the product.

Protect the repository from instruction drift

Agent guidance changes over time, and a prompt that worked last month may produce a different architecture after a model or tool update. Version the important repository instructions with the code, review changes to them, and include the agent and tool versions in experimental evaluations. Do not let personal editor rules silently define team architecture.

Run a small set of representative tasks after major upgrades and compare diffs, tests, security findings, and reviewer effort. This is regression testing for the development process itself. It helps the team adopt better models without discovering behavior drift inside a critical production change.

Create a pull-request label for agent-assisted changes during the pilot, not to stigmatize them but to measure them. Sample reviews after a month and look for recurring architectural corrections. Turn those corrections into tests or repository guidance. The feedback loop matters more than any single model score because it converts reviewer judgment into a system the entire team can reuse.

Photo by Jakub Zerdzicki on Pexels.