Engineer reviewing an AI agent’s code change and verification results

AI Agents for Developers: What Changes When Your Coding CLI Can Act

Table of Contents

If you use Codex or Claude Code in a terminal, you’ve seen the shift: the tool doesn’t just answer questions about your code. It reads files, edits them, runs commands, and reacts to the output. That’s the practical meaning of “agent”, and it changes what you supply and what you check.

This post is for developers who use these tools but haven’t studied how agents work. We’ll walk through one small change in an unfamiliar repository and mark where the engineer sets the boundary and verifies the outcome.

What an agent is, in plain terms

An agent is a model working toward a goal by using tools and feedback over multiple steps. It picks an action (read a file, edit, run a test), looks at the result, and decides what to do next, until it thinks the goal is met or it gets stuck.

Compare that with the tools you already know:

  • Autocomplete predicts the next few lines. You drive every step.
  • A chat answer is one response. It never sees what happens when you run it.
  • An agent runs a loop. It can check its work against real output, and it can take many actions before you look.

That loop is the useful part, and it’s why the rest of this post is about boundaries and verification.

The scenario: a small fix in a repo you don’t know

Say you’ve joined a team with a Python service. A helper called parse_duration accepts 90m and 2h but rejects 1h30m. You want combined units supported, with tests.

A reasonable prompt states the goal, the constraints, and how “done” is judged:

parse_duration rejects combined values like "1h30m". Make it accept
combinations of h, m, and s, in that order. Keep existing behavior for
single units. Add tests for the new cases.

Constraints: only change the module that defines parse_duration and its
test file. Don't add dependencies. Before editing, tell me where the
function lives, how it's tested, and your plan.

Here’s how a well-behaved run unfolds.

1. It inspects the repo

With the repo’s instruction files already loaded as context (more below), the agent searches for parse_duration, opens the module, finds the existing tests, and checks how the project runs them.

Your job: check that it found the right places, especially the existing tests.

2. It forms a plan

It replies with something like: “The function is in utils/time.py and uses one regex. Tests are in tests/test_time.py and run with pytest. I’ll allow optional hour, minute, and second groups, reject empty strings, and add four test cases.”

Your job: this is the cheapest point to correct it. If the plan crosses your boundary or changes behavior you didn’t ask about, say so now.

3. It edits files

The agent changes the parsing logic and adds tests. The diff should be small and confined to the two files you named.

Your job: nothing yet. Resist reviewing a half-finished diff.

4. It runs a relevant check

It runs the tests that cover the change, for example pytest tests/test_time.py.

5. It reads the result

Suppose one new test fails: "1h0m" returns 3600 seconds, but the test expected an error. Is the code wrong or the test? That’s a question about intended behavior, and a good agent surfaces it rather than guessing.

Your job: decide the intended behavior. The agent can’t know it.

6. It revises its work

You answer: zero components are valid. It fixes the test, reruns the file, and reports that everything passes.

Reported vs. verified

At this point the agent has told you a story. Keep the parts separate:

ClaimStatus
“I only changed two files.”Reported. Verify with git status and git diff.
“All tests pass.”Reported. Verify by running the tests yourself or by reading the actual command output in the session.
“Existing behavior is unchanged.”Reported. Once you’ve seen the existing tests pass, verified only as far as those tests cover it.
“The regex handles edge cases.”Not verified until you read it and a test covers each case.

Your job: read the diff, rerun the check, and decide. The agent did the typing. You own the merge.

What an agent needs to do this well

That run worked because five things were in place.

  1. Repository context and instructions. A Markdown file that says how to build and test, and what not to touch. AGENTS.md is a shared convention that many tools read, including Codex. Claude Code uses CLAUDE.md. By default it reads AGENTS.md only when no CLAUDE.md exists, so a repo that wants both can import one from the other (this blog’s CLAUDE.md starts with @AGENTS.md). Anthropic’s docs also note that these files are context, not enforced configuration, so treat them as guidance the model usually follows, not a guarantee.
  2. Available tools. File access, search, and a shell to run tests. An agent that can’t run tests can’t check itself.
  3. A clear task. One outcome, stated so that you can tell whether it happened.
  4. Boundaries. Which files it may change, which commands are off limits, whether it may install anything. For hard limits, use your tool’s permission settings rather than prose.
  5. A way to verify. Tests, a linter, a build, a type check. Without one, “done” means “the agent stopped”.

A starter workflow

Try this on your next small task:

  1. Pick a low-risk task. A bug with a clear reproduction, a missing test, a small refactor in well-covered code. Avoid auth, billing, migrations, and infrastructure for now.
  2. State the outcome and constraints. What should be true when it’s done, what’s in scope, what it must not do.
  3. Ask it to inspect and plan before editing. Correct the plan, not the diff.
  4. Let it edit and run the check.
  5. Review the diff and test output yourself.

Where agents help, and where the speed goes

Agents do well on work that is tedious but easy to check: tracing a call path through unfamiliar code, mechanical changes across many files, tests for existing behavior, a failing build with a clear error.

The apparent speed disappears in a few predictable places:

  • Review. A 400-line diff produced in two minutes can take thirty minutes or more to review properly.
  • Rework. An uncaught wrong plan produces a large, wrong diff.
  • Tool failures. Tests that need a database, secrets, or network access the agent doesn’t have. Retries and workarounds cost time.
  • Context and cost. Long sessions that read large files or logs fill the context window and cost more tokens. A fresh session per task is often faster.

Failure modes and proportionate containment

Failure modeWhat it looks likeContainment
Confident but incorrect changeClean diff, plausible explanation, wrong logicRead the diff. Require a test that fails before the change and passes after.
Scope creep“While I was there, I also refactored…”Name the files in scope. Reject unrelated changes.
Unverified tests“Tests pass” with no command output, or tests that assert nothing usefulRerun the check. Read the new tests as carefully as the code.
Stale documentationInstruction files describe a build step that no longer existsUpdate instruction files in the same PR that changes the process.
Sensitive data exposureAgent reads .env files or pastes secrets into outputKeep secrets out of the working tree. Use deny rules for sensitive paths.
Overly broad permissionsAgent can push, deploy, or run cloud CLIs unpromptedGrant the minimum. Require approval for anything that leaves your machine.

Most of these are handled by small tasks, a named scope, and reading the diff.

One agent or several?

For most tasks, one agent with good instructions is enough. More agents add coordination, cost, and output to review. Several agents working in parallel make sense only when the tasks are independent and touch separate files, and each one still produces a diff you have to review.

A specialist pair helps when roles benefit from separate instructions or a fresh context, such as a writer and a reviewer checking a fixed list. This blog’s repo is a small example: it defines post-writer, post-reviewer, and site-ops-reviewer agents in both .claude/agents/ (Claude Code subagents) and .codex/agents/ (Codex subagents), with shared rules in AGENTS.md. The two reviewers are told never to edit files, and the Codex versions also run with sandbox_mode = "read-only", which enforces that rule instead of just stating it. The reviewer doesn’t make the output trustworthy; it adds a second pass before a human reads it.

Checklist before you trust an agent with a repo task

  • The repo has an instruction file with build, test, and “don’t touch” rules, and it’s current.
  • The task has one clear outcome you can check.
  • The files or areas in scope are named.
  • Permissions match the task: no push, deploy, or cloud access unless required.
  • Secrets are not readable from the working tree.
  • There is a test, build, or lint that proves the change.
  • You’ll review the plan before edits, the full diff after, and the real check output before merging.

An agent is useful because it can act and react. That moves the engineer’s role from typing the change to defining the boundary and verifying the result. Keep tasks small, make “done” checkable, and never merge on the agent’s word alone.

Tharaka Ukwatte avatar
Tharaka Ukwatte
Cloud Engineer & AWS Architect · 5× AWS Certified