The first pull request is rarely the last word. The reviewer reads it and says: this is right, but archived invoices should be included too. That is not a bug in the run. It is a change to what the task should do, discovered by looking at the result, which is how most changes are discovered.
For a while CodeLynq handled that badly. Feedback on a built task started a fresh run: a new branch and a second pull request, which we then refused anyway because the task already had one. And the "iterate on feedback" path read only what the git host had, failing checks and review comments, never the feedback written on the task or the spec it was built from.
Feedback becomes a new version of the spec
A change now starts where the task started: in the spec. You write what you want changed on the task, or pick feedback comments that are already there. The AI revises the spec so it describes the work including the change. Every field comes back complete rather than as a diff, so the new version reads on its own, and it carries two extra things:
- The change request: the feedback this version answers, kept with the version.
- The change summary: what the AI says differs from what was built, in a few lines.
The revision is a draft. Earlier versions stay. If an earlier revision was still waiting for approval, the new one replaces it; a draft someone wrote by hand is kept as its own version. Nothing is built until someone approves.
Why go through the spec at all, instead of passing the comment straight to the agent? Because the spec is the record of what was agreed. If the change only lived in a prompt, the pull request would end up doing something no approved document describes, and the next person to touch the task would be working from an outdated spec. It also gives the reviewer a moment to check that "include archived invoices" was understood as meant before anyone pays for a run.
Built on the same branch, in the same pull request
Approving a change version starts a change run. It is its own run, with its own log, cost and status, but it continues the branch of the run whose pull request is open and pushes there, so the existing pull request updates. No second pull request. On a task that spans several repositories, it continues one branch per repository the earlier run changed.
The agent is given the change request, the change summary, the new spec and the previous one, so it builds the difference rather than the whole task again. Once the change is pushed, the pull request can still receive review comments and failing checks like any other, and those are answered by the normal iteration.
There are two refusals. A task with no open pull request has nothing to continue: edit the spec and start a run. And while a run is working on the task, a change waits until it is done. On self-hosted runners, continuing a branch needs runner 1.9.0 or later; if no runner qualifies, CodeLynq says why, including the case where a runner was installed but never connected.
What went wrong: the pull request that was already merged
Our record of a pull request does not follow it on the git host. If someone merged the pull request on GitHub, CodeLynq could still think it was open. A change run then continued the branch as asked, and the pushes landed on the branch of a merged pull request. They never reached the base branch and showed up in no pull request. From the outside, the change had been made and had vanished.
Now, before a change or an iteration builds anything, it asks the git host about every branch it is about to continue. If that pull request has been merged or closed, the work goes on a new branch off the base branch, with a new pull request. The run follows the new pull request, so the next change continues that one. We fixed it for self-hosted runners first; for a few hours, the platform worker refused such a change with an explanation instead, and now it opens the new branch and pull request as well.
The same round fixed a smaller injustice. An iteration where the agent found nothing left to fix used to fail with "made no changes", the message a new run gets, leaving a task marked failed next to a finished pull request. It now completes. A requested change that changes nothing still fails, with a message that says so, because that does mean something went wrong.
And one we found while building it: on self-hosted runs, a manual iteration used to be sent to our worker and also left queued for a runner. The runner could pick it up as a fresh run and open a second pull request. An iteration is now marked as one and only a runner that can continue a branch takes it.
What it does not do
- It does not merge. Every change ends in a pull request a person merges, for the reasons in why we will not let the AI merge.
- It does not skip approval. Each change is a new spec version, and someone approves it before it is built.
- Review comments on the git host do not create a spec version. They go through iteration, which fixes the pull request against the comments. If the comment changes what the task should do, put it on the task as a change.
The question to ask
When you evaluate an AI coding tool, the first pull request tells you less than the second. Ask what happens when you say "almost": does the feedback reach the same pull request, does the spec still describe what was built afterwards, and what happens if the pull request was merged in the meantime. If a correction keeps coming back, it is worth making it a rule, see reviewer feedback should only be given once. Running on your own machines? The runner side is on self-hosted runners.