Describing the Contexts of Agentic LLM Systems

The structure of an agent's context is one of its most consequential design decisions—and one of the hardest to communicate.

An LLM agent works by making a sequence of calls to a language model. Before each call, the surrounding system assembles a context out of instructions, observations, accumulated history, and tool results, and sends it to the model. The model's response drives the next action, the state updates, and the loop continues. The structure of that context—which pieces appear, in what order, under which roles, and how all of it changes from one step to the next—is one of the most consequential design decisions in an agent. It is also one of the hardest things to communicate.

A description that sounds precise but isn't

Here is a sentence you might find in a paper or a design doc, written the way these things usually are:

In each turn:

  • The agent gets a description of the task, its tools and some examples.
  • For each of its previous turns, it gets the history of the ReAct loop

This description is ambiguous

1
  • No reasoning traces
  • Tool calls, responses:
    • Sent in separate messages
    • With different roles
2
  • Includes reasoning traces
  • Tool calls, responses (& reasoning):
    • Sent in one message
    • Assistant role

Both readings honor the sentence above, yet they produce different contexts—and from the text alone you cannot tell which one the author meant. In ACDL, each reading is written out explicitly:

Option 1

Reasoning dropped · call and response as separate messages

Option1[@T.I]:

System
TASK_DESCRIPTION
env.tool_descriptions
env.in_context_examples
// previous turns
ForEach @t : 1 ... @T
ForEach i : 1 ... @t.substeps
Assistant
sys.tool_used[@t.i]
Tool
sys.tool_used[@t.i].tool_response
// current turn
ForEach i : 1 ... I
Assistant
sys.tool_used[@T.i]
Tool
sys.tool_used[@T.i].tool_response

Option 2

Reasoning kept · call and response bundled into one assistant message

Option2[@T.I]:

System
TASK_DESCRIPTION
env.tool_descriptions
env.in_context_examples
Assistant
// previous turns
ForEach @t : 1 ... @T
ForEach i : 1 ... @t.substeps
resp.tool_reasoning[@t.i]
sys.tool_used[@t.i]
sys.tool_used[@t.i].tool_response
// current turn
ForEach i : 1 ... I
resp.tool_reasoning[@T.i]
sys.tool_used[@T.i]
sys.tool_used[@T.i].tool_response
Two context structures that both satisfy the same prose description, rendered side by side.

Why this matters

Today the options for describing context structure are prose, ad hoc diagrams, or the implementation code itself. Prose and diagrams are quick to produce but ambiguous, as the example above shows. Code is exact, but recovering the structure from it means reading all of it, which is arduous and rarely how anyone wants to communicate a design.

The consequences compound. Published systems are difficult to reproduce, because the logic that assembles the context is left to interpretation. Systems that look similar at a high level are difficult to compare, because the structural choices that actually distinguish them are not visible. And within a team, there is no shared vocabulary to discuss how a context is built, reason about a proposed change, or carry a design from a whiteboard into code without loss.

Ready to learn ACDL?

Start the Tutorial Explore Examples