AI-Generated Code Verification

Pathrule2 Rules • 2 Memories • 2 Skills

AI-generated code is frequently plausible and locally coherent while inventing APIs, missing repository conventions, weakening security, duplicating existing utilities, or changing behavior outside the requested scope. This pattern constrains unverified claims and scope expansion, records evidence and ownership decisions, and supplies separate pre-merge and runtime-verification procedures. It differs from LLM evals by verifying repository changes produced by an AI assistant; model-output scoring remains useful upstream but cannot replace build, test, and runtime evidence in the target system.

Suggested path map

Pathrule places each piece on the matching path, so your assistant only sees it where it belongs. This is the scoping you get on import; you can adjust it in your workspace.

/ workspace root

review-ai-generated-change

verify-generated-change-runtime

src/

Require evidence for every behavior and API claim

Keep generated changes inside the authorized scope

docs/

ai-review/

AI provenance changes review strategy, not accountability

A verification report separates passed, failed, and untested

Rules

Require evidence for every behavior and API claim/srchighstrictVerify symbols against installed code or official sources and report only commands and flows actually executed.

1	Generated code can confidently import a removed symbol, assume a default, or describe a test result that was never produced. Plausibility is not repository evidence.
2
3	- Resolve APIs, configuration keys, defaults, and version-sensitive behavior from the installed dependency, source, or current official documentation.
4	- Run the narrowest relevant build, type, static, unit, integration, and runtime checks and preserve their exact outcome.
5	- Do not claim an endpoint, UI, migration, or recovery flow works unless it was exercised or clearly label the unverified boundary.
6	- Remove or rewrite uncertain behavior that cannot be verified instead of adding comments that repeat the guess.
7
8	See /tests for the adjacent decision or procedure that completes this constraint.

Keep generated changes inside the authorized scope/srchighstrictInspect the final diff for unrelated rewrites, duplicated abstractions, weakened checks, destructive targets, and user-owned work.

1	A coding agent can broaden a small task into cleanup or architecture change that was never requested. Even individually reasonable edits increase review cost and hide regressions.
2
3	- Map each changed file and behavior to the user request or a necessary implementation step and revert no protected user or team work.
4	- Search for existing helpers, services, schemas, components, and conventions before adding a parallel implementation.
5	- Treat permission, authorization, validation, migration, secret, external-effect, and destructive-command changes as high-risk scope that needs direct evidence.
6	- Review generated dependency, lockfile, snapshot, schema, and build output separately from authored source and regenerate only through the repository toolchain.
7
8	See /docs/ai-review for the adjacent decision or procedure that completes this constraint.

Memories

AI provenance changes review strategy, not accountability/docs/ai-reviewUse generated origin to focus on invented assumptions and broad consistency, while holding the same product owner responsible for the merged behavior.

1	AI code does not need a weaker or purely stylistic review. It needs extra attention to repository-wide assumptions the generator may not have seen, but the acceptance bar remains the system's ordinary correctness and safety contract.
2
3	- Review at behavior and ownership boundaries before line polish: input, authorization, state, effects, failure, concurrency, privacy, and recovery.
4	- Ask which repository rule, memory, existing implementation, source, or runtime evidence supports each non-obvious decision.
5	- Use model or tool metadata for audit and improvement where policy permits, but do not treat it as a substitute for code ownership.
6	- Assign a human or team owner who can operate, rollback, and maintain the generated change after the conversation that produced it is gone.
7
8	See /src for the rule or workflow that puts this decision into practice.

A verification report separates passed, failed, and untested/docs/ai-reviewRecord command, scope, environment, result, and remaining risk without converting partial checks into a green narrative.

1	A report that says tests pass without naming the suite, or says production-ready after only compilation, prevents reviewers from judging evidence strength.
2
3	- List each executed check with target, environment, exit outcome, and relevant artifact or observation.
4	- List failures and how they were repaired, because repeated mistakes reveal risk areas that deserve review.
5	- Name untested platforms, browsers, integrations, migrations, loads, credentials, and destructive or external flows explicitly.
6	- Keep inference separate from observation and state what evidence would be needed to close each remaining uncertainty.
7
8	See /tests for the rule or workflow that puts this decision into practice.

Skills

review-ai-generated-change/rootReview an AI-authored diff for intent, reuse, contracts, security, failure, tests, evidence, and operational ownership.

1	---
2	name: review-ai-generated-change
3	description: Review code produced wholly or partly by a coding agent before merge.
4	---
5
6	# Review Ai Generated Change
7
8	Run this procedure when the affected surface changes, before the result is promoted to production. Record evidence for every step instead of accepting a plausible-looking result.
9
10	1. Restate requested behavior and map every changed file, dependency, generated artifact, and external effect to that scope.
11	2. Compare against repository instructions, nearby implementations, schemas, ownership boundaries, and installed dependency APIs before accepting new abstractions.
12	3. Trace untrusted input through validation, authentication, authorization, persistence, output, logging, and cleanup, including error and cancellation paths.
13	4. Run focused and broad checks proportional to risk and inspect whether tests would fail on the pre-change defect instead of merely covering the new code.
14	5. Verify runtime behavior and rollback for the complete user flow, then write a report that separates evidence, inference, and untested boundaries.
15
16	## Exit criteria
17
18	The change is complete only when the expected behavior, failure behavior, and rollback path have all been exercised with representative data. Preserve the evidence with the change so the next operator can repeat the same checks.

verify-generated-change-runtime/rootExercise the full generated behavior from entry point through state and dependencies to response, recovery, and rollback.

1	---
2	name: verify-generated-change-runtime
3	description: Verify an AI-generated feature or fix in the real application environment.
4	---
5
6	# Verify Generated Change Runtime
7
8	Run this procedure when the affected surface changes, before the result is promoted to production. Record evidence for every step instead of accepting a plausible-looking result.
9
10	1. Start from the user or system entry point and capture the baseline failure or expected pre-change limitation.
11	2. Exercise valid, invalid, unauthorized, duplicate, concurrent, timeout, cancellation, dependency-failure, and recovery paths with representative data.
12	3. Inspect API, database, queue, file, log, metric, and external-side-effect evidence so the visible result is tied to authoritative state.
13	4. Restart, reload, navigate, retry, or redeploy according to the feature lifecycle and confirm state remains consistent.
14	5. Execute rollback or disablement and verify migrated data and external effects remain supportable before declaring the change complete.
15
16	## Exit criteria
17
18	The change is complete only when the expected behavior, failure behavior, and rollback path have all been exercised with representative data. Preserve the evidence with the change so the next operator can repeat the same checks.

Why this pattern

AI agents often cite tests they did not run, introduce near-duplicate helpers, use stale APIs, or fix the visible symptom while violating a deeper data or authorization invariant.

Built for Engineering teams using coding agents for implementation, migration, refactoring, or automated pull requests.

Keeps your assistant from:

Merging code that only appears to use a real API
Accepting claimed verification without command evidence
Missing an authorization or data-loss regression outside the happy path
Shipping unrelated generated cleanup with the requested change

View as markdown

License: Apache-2.0
Version: 1.0.0
Updated: 2026-08-25