The Interesting Part of an AI Agent Is the Loop

The moment that interests me in an AI agent demonstration is usually a small interruption. A search returns nothing. A file has a different name. A command fails. The next move tells me much more than the opening answer, because the system has to deal with a world that has declined to follow the script.

A language model can describe a sensible sequence of actions without any of those actions happening. An agent adds a way to carry out some actions, receive observations and decide what to do next. That creates possibilities I find genuinely exciting: a question can become an investigation, and an unexpected result can become useful information rather than the end of the conversation.

The terminology is less tidy than the diagrams. Different products use the word agent for different degrees of independence. I find it easier to understand a concrete loop before arguing about whether a product deserves the label. What can it observe, what can it change, and what makes it stop?

Where the next step comes from

Anthropic's Building effective agents makes a useful architectural distinction. In a workflow, code largely sets the route between steps. In an agent, a model has more control over the route and tool use. The article dates from 2024, so I read it for that distinction rather than as a current catalogue of software.

Consider an invented example: a reader wants to understand why two editions of a public bus timetable disagree. A fixed workflow could retrieve both documents, extract the dates and compare matching routes. That may be entirely sufficient. There is no particular prize for involving an independent planner in a task whose route is already known.

An agent becomes interesting if the evidence changes the route. Perhaps one document contains a notice referring to a third page. Perhaps the route number changed. The model can propose a search using the old name, inspect what comes back and decide whether the new evidence resolves the discrepancy. The investigation acquires branches that would be awkward to list in advance.

This freedom has a practical attraction: people can describe an outcome in the terms they care about. They don't have to specify every intermediate click. Yet the request still needs a boundary. Understanding the discrepancy doesn't imply permission to edit the timetable, contact the transport operator or announce a correction elsewhere. A broad goal can coexist with a narrow set of allowed actions.

In this example, the agent's final explanation should preserve which edition says what. If it finds a dated notice that explains the change, that notice becomes the evidence. If it doesn't, the result may be an unresolved comparison. I would still find that useful. A clear account of what remains unknown can save the next reader from repeating the same search.

The model has company

A model supplies an important part of the reasoning, but its available tools determine which ideas can become actions. A search tool might expose only titles and snippets. A document tool might return the full text and publication date. The difference changes what conclusions the agent can reasonably reach, even with the same underlying model.

I imagine giving two careful readers the same question. One receives a shelf of labelled documents; the other receives several torn pages and a drawer that occasionally jams. Their eventual answers might differ without telling us much about which person is the better reader. The environment belongs in the explanation of the result.

Tools also need useful ways to report failure. An empty list could mean there were no matches, the source could not be reached, or the search terms were rejected. Those are different observations. If they arrive in the same shape, the model has to guess which world it is in. Fluent guessing is already plentiful; I would rather the tool preserve the distinction.

There is a similarly important difference between requesting an action and observing its completion. A tool call can be accepted while the underlying work remains unfinished. For a document export, a returned job identifier may mean only that an export has begun. Treating it as a finished file would make the agent's later summary arrive ahead of reality.

These details make the subject feel less magical and more interesting to me. Progress can come from improving an interface, making a failure legible or returning the right piece of evidence. It needn't always wait for a new model. The intelligence of the whole arrangement depends partly on whether its components can tell one another what actually happened.

Context is a working view

An agent needs some representation of the goal, relevant evidence and recent actions. Anthropic's context engineering article discusses treating that available information as a constrained resource, including retrieval and ways of managing longer tasks. I take the practical question to be: what does the next decision need to see?

For the timetable example, that might include the requested route, two publication dates and a short record of searches already attempted. It probably doesn't need every line from every unrelated route. More material can supply missing evidence, but it can also obscure the few facts that determine the next step. The interesting choice is what to retain and what to reopen when needed.

A short note can help if it preserves the right distinctions. Compare “new timetable found” with “edition dated September found; route number differs; no change notice confirmed yet.” The second version allows another decision without pretending that the investigation is complete. It keeps the uncertainty attached to the evidence rather than leaving it somewhere in the vanished conversation.

I find this similar to taking useful reading notes. A quotation without the page or the question that made it interesting can become surprisingly difficult to use later. For an agent, losing that surrounding information can affect an action immediately. A summary that compresses away a condition may quietly change what the system believes it is allowed to conclude.

A useful memory keeps the reason a detail mattered.

The ambition here is appealing. An agent could resume an investigation without forcing the user to reconstruct it every time. But continuity needs to be earned through records that preserve meaning. A longer conversation and a reliable working memory are related ideas, with enough differences to deserve separate attention.

Continuing across a pause

Anthropic's discussion of long-running agents examines the difficulty of continuing useful work across separate context windows. Its examples involve software tasks and explicit progress records. I find the broader question useful: after a pause, how would a fresh session distinguish completed work from an optimistic description of it?

In the imagined investigation, a record might name the exact documents inspected, the discrepancy found and the next unresolved question. That would be more helpful than a long narrative about being diligent. A new session could reopen the relevant source instead of repeating the entire search or trusting a conclusion with no visible foundation.

The record should also distinguish observation from interpretation. “Both files carry different dates” is an observation that can be checked. “The newer file supersedes the older one” may require another source or an explicit policy. If those sentences are merged into a single confident note, the next session inherits certainty that the previous one never established.

This is where I become interested in the surrounding runtime, sometimes called a harness. It can manage tools, execution, records and limits around the model. The exact features differ between systems. The conceptual benefit is that persistence doesn't have to depend entirely on the model remembering to write a perfect farewell message at the right moment.

There are ordinary engineering questions hiding in the promise of an agent that keeps working. Can a result be saved before interruption? Can a repeated operation create duplicates? Can the process resume from a known point? I like that these questions remain relevant. A system may be new and still benefit from the old habit of leaving its desk in a comprehensible state.

A loop needs an ending

An open question can generate an impressive amount of activity. The agent searches for a notice, finds an ambiguous reference, searches again and discovers another ambiguous reference. Each individual step might look reasonable. The whole sequence may have stopped being useful some time ago, a condition familiar to anyone who has researched a purchase after already choosing it.

A stopping rule gives the task a shape. In the hypothetical timetable search, completion might require a dated authoritative notice connecting the editions. A second legitimate ending would be a report of the unresolved discrepancy after a defined search effort. The user gets an answer appropriate to the evidence instead of an endless performance of trying.

Limits can concern attempts, time, expenditure or permitted actions. Their value depends on the task. A read-only investigation can tolerate uncertainty differently from a process that changes records. I would want the description of an agent to make those distinctions visible, rather than treating all continued activity as evidence of greater independence.

There is also a good reason to ask for help before a nominal limit is reached. The agent may discover that two plausible interpretations would lead to different actions. More searching won't always resolve a question about what the user actually wants. At that point, a concise clarification can be a useful result of the investigation rather than an interruption to it.

The final account should close the loop for the reader too. What was checked? What happened? What remains uncertain? These needn't become a huge log. The useful information is the evidence that lets someone decide whether the task is complete, or whether there is a specific next step worth taking.

What makes me optimistic

The exciting change is that generative AI can participate in a sequence whose next step depends on observation. A useful assistant can go beyond suggesting where to look and actually inspect an allowed source. That reduces the gap between having a reasonable idea and obtaining evidence about whether the idea works in the situation at hand.

I can imagine this being especially helpful with unfamiliar documents, small investigations and tasks that involve several compatible tools. The user describes the question; the system handles some of the navigation; the explanation returns with enough evidence to be checked. The appeal is partly intellectual and partly the very ordinary pleasure of fewer loose ends.

I remain unconvinced by the idea that the most autonomous arrangement is automatically the most useful one. Sometimes a fixed route is easier to understand and entirely adequate. Sometimes a model choosing the next step can handle the exceptions that make a fixed route brittle. The interesting design choice is where that flexibility earns its place.

This also changes how I read demonstrations. Alongside the model name, I look for the observations the agent receives and the operations it can perform. A failure followed by a sensible correction can be more informative than a perfectly smooth run. It shows a little of the machinery that turns a broad request into a grounded next step.

The loop is what I keep coming back to: an action changes the available evidence, and the evidence changes the next action. There is plenty left to learn about making that sequence dependable. It is already a richer idea than an answer that merely sounds as though someone has done the work. I would happily watch a slower demonstration if it left that interesting middle in view.

— John