Agents got cheap. Context makes them useful.

A few weeks ago I wired five agents together to clear a backlog of flaky integration tests. A planner split the backlog into tickets. A researcher pulled logs and stack traces. A coder wrote the fix. A reviewer checked the diff. A reporter wrote up what changed.

The first ticket came back green in about 40 minutes. The coder had wrapped a slow database call in a retry with backoff. The test stopped failing. The reviewer approved it. The reporter wrote a tidy summary.

I almost merged it. Then I remembered that someone had removed that exact retry a few months earlier. It had been hiding a race condition between two writers, and the flaky test was the only signal left that the race still existed. The reasoning lived in a closed pull request and a short decision record. None of the five agents had ever seen either one.

Every agent did its job by its own standard. The system still produced a change that would have shipped a known bug back to production.

I keep coming back to that run, because it shows where the difficulty in agent systems has moved.

Agents are the easy part now

Not long ago, getting one agent to call a tool reliably was a project in itself. Today a framework hands you a planner, a few workers and a supervisor in an afternoon. A sixth agent costs a config file and a system prompt.

So teams add agents. A research agent. A critic. A summarizer for the critic's notes. The architecture slide gets more boxes and more arrows. In what I have built and reviewed, reliability rarely climbs with the box count.

The things that decide whether an agent system holds up sit outside the agents:

• what each agent knows when it starts
• how work moves from one agent to the next without losing meaning
• who checks the output, other than the agent that produced it
• whether actions in the real environment happen the way the plan said

Verification got its own treatment in I built my own agentic harness. Here is what's inside. This post covers context and execution. Most of the agent failures I end up debugging trace back to one of those two.

Context is the missing layer

A model knows a lot about code in general and nothing about your code's past. When an agent fails in a way that looks careless, I check what it could see before I touch the prompt. Almost every time, the information it needed existed somewhere. It never made it into the window.

Missing context comes in different kinds, and each kind needs a different fix.

Project history

Code tells an agent what the system does. It rarely says why. The why lives in decision records, pull request threads, incident write-ups and the commit message someone wrote at 1 AM.

Embedding search alone misses most of it. A decision record about two writers fighting over a row does not look similar to a ticket titled "flaky test in orders suite". What connects them is the file path. So I index decision records and incident notes by the paths and modules they mention. Before an agent touches a file, the harness pulls everything attached to that path.

def history_for(paths: list[str], index) -> list[ContextItem]:
    items = []
    for path in paths:
        for record in index.by_path(path):  # exact lookup, no similarity score
            items.append(ContextItem(
                kind="decision",
                source=record.id,
                content=record.summary,
                fetched_at=record.updated_at,
                reason=f"mentions {path}",
            ))
    return items

That is a lookup, not a search. It is cheap, and it never decides on its own that something is relevant. I wrote about mixing retrieval methods in RAG Isn't a Vector Database. It's a Retrieval Problem.

Domain rules

Some knowledge never appears in code. Refunds above a threshold need a second approver. One region's data cannot leave one cloud account. A customer tier gets a different response window. People carry these rules in their heads.

Writing them into a system prompt as prose works until the prompt runs to a few thousand tokens and the model starts trading rule 14 against rule 31. I keep rules like these as structured data outside the prompt and check them in code after the model proposes an action. The model can still read them while it plans. It does not get the final say. That split is the subject of Neuro-Symbolic AI: When the LLM Doesn't Get the Final Say.

Current state

This one hurts most in operations work. An agent plans against a snapshot: a schema version, a deploy status, a queue depth, a ticket's state. By the time it acts, the snapshot can be an hour old.

Say a migration agent plans against staging on schema v14. Someone deploys v15 at 10 AM. The plan is still internally consistent. It is consistent with a database that no longer exists.

Two rules help. Every state item carries the time it was fetched. And anything an action depends on gets read again right before the action runs, instead of trusted from the planning step.

Context as an artifact

I stopped treating context as whatever ends up in the prompt. Now the system builds it, logs it and tests it like any other artifact. Each task gets a context packet, and every item in it says what it is, where it came from, when it was fetched and why it is there.

from dataclasses import dataclass, field
from datetime import datetime, timezone

@dataclass
class ContextItem:
    kind: str            # "decision", "rule", "state", "code"
    source: str          # file path, record id, API endpoint
    content: str
    fetched_at: datetime
    reason: str          # why this item made it into the packet

@dataclass
class ContextPacket:
    task_id: str
    items: list[ContextItem] = field(default_factory=list)
    token_budget: int = 12_000

    def stale_state(self, max_age_s: int = 300) -> list[ContextItem]:
        now = datetime.now(timezone.utc)
        return [
            i for i in self.items
            if i.kind == "state"
            and (now - i.fetched_at).total_seconds() > max_age_s
        ]

The packet goes into the trace with every run. When an agent does something odd, I can see exactly what it saw, so I debug the packet before the prompt.

It also makes context assembly testable on its own. For a given ticket, did the packet include the decision record it should have? That one eval would have caught the retry fix before any agent wrote a line of code.

More context is not the goal. A packet stuffed with half the repo does worse than a small one with the right few items. The budget is a design input. Stable items like rules and conventions go at the front so the prefix caches, which I covered in Prompt Caching Isn't a Setting. It's a Prompt Architecture Decision.

Execution matters more than orchestration

Orchestration decides who runs when and who passes work to whom. Execution decides whether the thing happened in the real environment, once, and correctly. Frameworks have made the first one easy to draw on a whiteboard. They say far less about the second.

Handoffs leak

In most multi-agent demos, agents pass each other prose. The researcher summarizes for the planner. The planner summarizes for the coder. Every summary drops something.

Run the numbers. If each handoff keeps 95 percent of what matters, five handoffs keep about 77 percent. The last agent is acting on a noticeably different understanding than the first one had. In the opening story, the decision record never reached any agent. If the researcher had found it, there is a fair chance the one line about the race condition would have been summarized away before the coder saw it.

So my agents hand each other typed artifacts instead of chat.

from pydantic import BaseModel

class FixProposal(BaseModel):
    ticket_id: str
    root_cause: str
    files_changed: list[str]
    tests_added: list[str]
    decisions_consulted: list[str]   # record ids the author actually read
    open_questions: list[str]

def review_gate(proposal: FixProposal, index) -> str | None:
    required = {
        r.id for path in proposal.files_changed
        for r in index.by_path(path)
    }
    missing = required - set(proposal.decisions_consulted)
    if missing:
        return f"reject: decision records not consulted {sorted(missing)}"
    return None

The decisions_consulted field is the one I care about most. If the changed files have decision records attached and the author did not read them, the proposal bounces before any model reviews the diff. A schema cannot make an agent think. It can make a skipped step impossible to hide.

Check the world, not the agent's report

An agent saying it closed the ticket is a claim. A ticket whose status reads closed is a fact. I wrap every write action so it checks a precondition before it runs and reads the world back afterwards, independent of whatever the tool returned.

class PreconditionFailed(Exception): ...
class PostconditionFailed(Exception): ...

def execute(action, *, precondition, postcondition, idempotency_key, ledger):
    if ledger.seen(idempotency_key):
        return ledger.result(idempotency_key)

    if not precondition():
        raise PreconditionFailed(idempotency_key)

    result = action()

    # Read the world back. Do not trust the action's own return value.
    if not postcondition(result):
        raise PostconditionFailed(idempotency_key)

    ledger.record(idempotency_key, result)
    return result

execute(
    lambda: tracker.close(ticket_id, note=summary),
    precondition=lambda: tracker.get(ticket_id).status == "resolved_pending",
    postcondition=lambda _: tracker.get(ticket_id).status == "closed",
    idempotency_key=f"close:{ticket_id}:{run_id}",
    ledger=ledger,
)

The precondition handles the stale state problem from earlier. If the ticket moved since the agent planned, the action stops instead of overwriting someone's work. The postcondition catches the quiet no-op, where an API returns 200 and changes nothing. Third-party tools do this more often than their docs admit.

Retries need idempotency

Agents retry. Models time out, tools time out, and an orchestrator that restarts a step has no way to know whether the first attempt landed. Without a key, a retry on "create ticket" gives you two tickets. A retry on "issue refund" gives you two refunds. The ledger in the wrapper turns the second call into a read of the first result.

Not every step needs a model

When I look at a workflow, I ask which steps need judgment. Listing files, running tests, reading git state, and checking a schema version are commands with known output. They do not need an agent. Every step I move from a model to plain code is one less place for drift. The token budgets, loop limits and stop conditions that keep the remaining model calls in check are in Your agent works. That's not the same as production-ready.

Before I add another agent

These are the questions I ask now before a new box goes on the diagram.

1. What does this agent need to know that the system cannot give it today? If the answer is nothing new, the problem is context, and another agent will not fix it.
2. What artifact does it hand to the next step, and what is the schema?
3. Which of its outputs change something outside the system, and what reads that change back?
4. If it runs twice, what happens?
5. Could a script do this step?

If I cannot answer the second or third question, the agent does not go in. More often than I would like to admit, the fifth one ends the conversation.

What it costs

None of this is free. Building a context packet adds lookups before the first token, usually a few hundred milliseconds per task. Postcondition reads double the calls to some APIs, and a few of those APIs will rate-limit you for it.

Reading state back also means the agent needs read scope it might not otherwise have. That widens the access surface I argued for keeping narrow in Your Agent Has More Access Than Your Junior Developer. That's a Problem. I grant it per tool and per resource, never as a blanket role.

And the decision index is only as good as the team's habit of writing decision records. Where nobody writes them, there is nothing to index, and the agents are back to guessing.

I still take the trade. A slower run I can trust beats a fast one I have to reread line by line.

How the retry story ended

I did not add a sixth agent. I added the path-indexed lookup, the decisions_consulted field, and the gate in front of the reviewer.

On the rerun, the coder got the decision record in its packet. It left the retry out and proposed a lock around the two writers instead. The test went green for the right reason this time.

The change took one evening, and none of what I changed involved a model.

Found this useful? I do 1:1 sessions on AI architecture and strategy. → Book a session