Demystifying AI loops: The 3 layers of autonomous architecture

From prompting to loop engineering

In my previous post, I wrote about the danger of delegating critical thinking to AI, using the analogy of a climber and a belayer. AI is the fast, strong climber, but we must remain the belayers holding the safety rope and choosing the route.

As I explore this further with my AI pet projects this year, I’ve realised that maintaining this level of control requires significant effort. We are no longer simply typing prompts into a chat window. The industry has moved from prompt engineering to context engineering and is now embracing a new discipline called loop engineering.

I don’t prompt Claude anymore. I have loops running that prompt Claude and figuring out what to do. My job is to write loops.

— Boris Cherny

Loop engineering is the practice of designing the system around the model, not just the conversation with it. The agent perceives its environment, reasons, acts and evaluates feedback in a recursive loop until a goal is met. However, as these systems are scaled up, it becomes clear that this isn’t just a simple while loop. It is a complex, hierarchical architecture.

For more detailed look at whether loop engineering lives up to the hype, The Great Loops Debate is worth watching in full.

The industry already has a useful map for this: Anthropic’s well-known distinction between workflows (predefined paths) and agents (LLM-directed), with patterns such as orchestrator-workers and evaluator-optimizer. Building on this I categorise autonomous systems into three distinct layers of control loops:

Micro-Loop

The micro-loop is the lowest level. This is the basic, iterative interaction between your application and the LLM.

It is based on the ReAct (Reason + Act) pattern, although modern systems implement it using native function calling. The model receives an observation, generates an internal reasoning trace, decides to call a tool and then observes the result. This layer does not understand the overarching business goal, but rather only the immediate tactical step of formatting a prompt, parsing structured output and invoking a tool, whether a native function call or one exposed through a Model Context Protocol (MCP) server.

messages="$PROMPT"
while true; do
  response=$(llm generate opus-4-7 "$messages")
  if wants_tool "$response"; then
    messages="$messages$(execute_tool "$response")"
  else
    printf '%s\n' "$response"
    break
  fi
done

Meso-Loop

While the micro-loop is concerned with executing a single command, the meso-loop is concerned with achieving a goal. This is the second layer and the true domain of the agentic harness.

A raw model is stateless and forgets everything between calls. The harness is an operational software layer that wraps around the model, providing memory, isolation (e.g. Git worktrees for parallel sessions, as in Claude Code and Codex, or Docker sandboxes, as in OpenHands and SWE-agent) and safety guardrails. A well-architected meso-loop requires:

• Automations and triggers: events (like a cron job or webhook) to autonomously wake the agent up.
• Memory and state: Storing progress permanently on disk so the agent can survive long time execution without context rot.
• Stop rules and verification: Explicit, machine-checkable termination conditions, such as a CI pipeline passing or a linter returning zero errors. However, passing tests only proves that the code runs; whether it is the right architecture remains a matter of human judgement.

The harness acts as a governor or kill switch for the LLM. Just as a microservices circuit breaker protects a downstream service from cascading failures, the harness limits the agent with iteration limits and budget caps. It prevents the agent from falling into infinite loops, repeating failed actions or hallucinating success.

The harness can bound and steer the model, but it cannot fix a limitation rooted in the model itself. Coding models are reinforced on whether tests pass, not on whether the architecture stays clean, so they erode maintainability over time, and no loop engineering can patch that gap. It is also why the lab that owns both the weights and the harness keeps pulling ahead.

This argument is set out in detail in Harness Engineering is Not Enough.

attempts=0
while (( attempts < MAX_ATTEMPTS )); do
  opencode run opus-4-7 "$PROMPT"
  if run_tests && lint; then
    break
  fi
  attempts=$((attempts + 1))
done

Macro-Loop

The top layer is the macro-loop, which coordinates ecosystems comprising multiple agents. When a single harnessed agent is overwhelmed by the scale of an enterprise, its performance deteriorates due to ambiguous decision points and excessive context information.

At this level, we stop creating harnesses and start creating orchestrators. The macro-loop manages the distributed workforce using patterns borrowed from how human teams operate:

• Hierarchical delegation: a manager agent decomposes a large task and delegates sub-tasks to highly specialised domain agents, bypassing context limits at the cost of an unsolved shared-memory and access-control problem in production.
• Evaluator-generator (maker-checker): borrowing the separation-of-duties idea from security and finance, one agent writes the code and a separate process verifies it. The checker works best when it is deterministic, e.g. tests, linters and type systems. However, an LLM grading another LLM only raises the floor slightly, as if it could reliably recognise good design, it would have written it in the first place.
• Governance and routing: applying conditional branching and enforcing budget limits or human-in-the-loop approval gates before high-impact actions are taken.

opencode run manager "decompose $TASK" | while read -r subtask; do
  opencode run specialist "$subtask" &
done
wait
run_tests && lint

Holding the rope

To build reliable autonomous systems, it is crucial to understand these three layers: the micro-loop’s execution, the meso-loop’s goal-oriented harness and the macro-loop’s multi-agent orchestration.

However, no matter how sophisticated our loop engineering becomes, these patterns are merely the safety gear. They execute the vision, but they don’t create it. We must still define the architecture, set verifiable goals and understand the trade-offs. AI can run the loops, but we are the ones who must hold the rope.