×
![Modal Image]()
The demo is always the same. Someone types a sentence, the agent writes 300 lines, the tests go green, and the room nods. What the demo never shows is the Thursday afterward.
We saw this firsthand when one of our engineers gave an agent a batch-processing migration and got back code that looked better than most first drafts from a person. Clean, well named, sensible comments. It also never advanced the pagination offset. The job ran the same page over and over, quietly, until someone noticed the queue hadn’t moved in an hour.
What the demo never shows is the Thursday afterward.
We still use agents every day and we’re faster for it. The bug changed where we put the human in the loop, and that is what this series is about: what changed in how we work, and what broke along the way.
The public numbers are real, as far as they go. GitHub and Microsoft measured a 55.8% faster completion time on a controlled coding task. McKinsey reported up to 2x on some developer tasks. Microsoft and Ars Technica have both written about vibe coding as a way for more people to build software from a sentence. None of that is wrong. It was measured on the demo, though, and we live on the Thursday.
At Amplified we’ve been running AI tools on projects since 2023, first as autocomplete, then as something closer to a junior teammate with a terminal. This first article is about what that second thing changes for the person doing the work.
The old model was simple: you ask, the tool answers. It’s still useful for autocomplete, snippets, a quick explanation of a regex, and an isolated function. What it can’t see is the delivery context: the product flow the function sits in, the conventions of the repo, the test standards, the deployment constraints, who owns which module.
The new model is different in kind. You delegate, and the agent executes. It explores the codebase, finds the relevant files, and proposes a plan. Then it edits, runs commands, reads the logs, writes tests, reviews its own diff, and prepares the pull request (PR). Some of those steps do well. Some of them do it confidently and wrong, and from across the room the two look the same.
What changes for the engineer is where the time goes. Less of it on typing the obvious. More of it on defining the problem precisely enough that the agent can’t misread it, steering while it works, reading what it produced, and deciding whether the result is right. That last part is the job now.
Most agent tools now split the work into modes, and we’ve found the split matters more than the model. We call them ask, plan, and agent.
Ask mode is for understanding: how does this flow work, which files touch billing, what did the last person change and why.
Plan mode is for scoping: what would change, which files, what the trade-offs are, how we’d verify it.
Agent mode is for execution against a plan someone has already read.
The mistake we made early, and still catch ourselves making, is reaching for agent mode because it’s the most powerful. On a clear, contained task that’s fine. On an ambiguous one, agent mode produces a large amount of plausible code before anyone has checked the direction. In that situation, ask and plan are the brakes.
The rule of thumb we use now: the mode matches how well the task is understood and how confidently the result can be reviewed. If we can’t say what done looks like, we’re not in agent mode yet.
The tasks where the agent has been a clear win for us share a shape. They’re clear, limited in scope, and easy to verify.
That last one surprised us. One of our engineers pointed an agent at a repo and its infrastructure config and asked for a DevOps guide and an architecture diagram. It came back with a usable first draft of both. Knowledge that had lived in 2 people’s heads became a document in an afternoon.
Every item on that list has the same condition attached: someone who understands the area reads the output. The quality of what the agent produces depends heavily on the engineer’s ability to judge it. When the engineer knows the code, plausible-but-wrong gets caught, and when they don’t, it gets merged.
Across 125 agent sessions on 1 project, we counted about 36 moments where a person had to stop and correct the agent. The breakdown was more useful than the total: 9 were output in the wrong shape that had to be redone, 8 were wrong results or logic, 5 were the wrong thing built entirely, 4 were invented context or actions nobody asked for, 3 were fixes that didn’t fix, and 2 were jobs reported complete with items missing.
36 correction events across 125 sessions
3 of those failures deserve more than a row.
Confidently wrong is the pagination offset. The code reads well, the tests it wrote for itself pass, and the logic is broken in a way a skim won’t catch.
Incomplete end-to-end reasoning is the agent changing one part of a flow without tracing the effect through the rest of the system. In product code, where behavior depends on 3 or 4 layers agreeing, that’s the expensive one.
Autonomy overreach is the one that made us write rules. An agent committed to a branch and opened a pull request nobody had asked for. Nothing shipped. The fact that it could was enough to make “no commits or PRs without approval” a written rule in every repo, where before it had been an assumption.
There’s a quieter failure too, which is drift. The agent misreads the scope, gives a confident answer that contradicts its last one, edits a file you didn’t mention, or makes a choice the plan didn’t agree to. When that happens, correcting the same thread rarely works. We stop, write a clean summary, and start a fresh conversation. It feels wasteful. It is faster.
Have you had the moment where the agent’s output looks productive and you can’t tell whether the codebase just moved in the right direction? That is the risk in one sentence. If the reviewer can’t evaluate the result, the productivity gain is deferred, with interest.
Where we won’t hand the agent the outcome, at least for now:
In those situations the agent can still research, explore, and draft. It doesn’t own the result.
If the reviewer can’t evaluate the result, the productivity gain is deferred, with interest.
The version of the workflow that survived contact with real projects has 4 steps, and the order is the point.
2 things sit around those steps. The first is naming scope out loud: which files and modules the agent may touch, and which it may not. We put explicit walls around authentication, billing, permissions, production data, and legacy code we don’t fully understand yet.
The second is turning corrections into files instead of prompts. Rules that live in SKILL.md, CLAUDE.md, CURSOR.md, AGENT.md, and project rule folders get followed on the next session without anyone remembering to say them. We’ve also started putting short CONTEXT.md files at the root and inside feature directories: what this part of the system is for, how it’s laid out, what it must not do. It’s early and we haven’t measured it properly yet, but the agent finds the right files faster and spends fewer tokens exploring.
Part 2 of this series is about those files: the prompts, review habits, verification steps, and team-level rules that make the loop hold.
Where the agent works, and where a person owns the outcome
Axes: How well the task is understood (across), How confidently the result can be reviewed (up)
Some of this is judgment, and we don’t expect that to change. Ambiguous requirements, architecture, and trade-offs that touch security, privacy, compliance, payments, permissions, or production data get a person, every time.
Some of it is experience rather than review. You can’t read a diff and know whether the motion feels right, whether the copy sounds like the product, whether the interaction is pleasant at 11pm on a phone. Someone has to use the thing.
At Amplified the agent is part of how we deliver now. It speeds up implementation, testing, documentation, debugging, and migration work, and it does that inside a loop with a person at the end of it who answers for whether the result is correct, maintainable, secure, and right for the product. That person is the part of the demo nobody films, and the part we’d hire for first.
If you’re running agents on a real product and want to compare notes on the loop, we’d like to hear how yours is going. Let's build together.
