Two years ago we shipped our first multi-agent pipeline. Since then the architecture has been rebuilt twice, and most of what we believed at the start has been revised. These are the notes we wish someone had handed us.
What survived
Specialists beat generalists, but only with a planner. Narrow agents with one job each, draft, research, format, check, consistently outperform one large agent doing everything, provided something upstream decides who works on what. The planner does not need to be sophisticated. It needs to be legible: when the pipeline misbehaves, the first question is always “what was the plan?”, and there has to be a readable answer.
The critic earned its keep (we wrote about it separately). So did checkpoints: fixed points in the pipeline where state is saved and a human can inspect, correct, or roll back. Every production incident we have had was resolved at a checkpoint. None were resolved by reading agent-to-agent chatter.
What did not survive
Free-form agent conversation went first. Letting agents negotiate with each other in natural language demos beautifully and debugs terribly, we replaced it with structured handoffs, where each agent receives a typed brief and returns a typed result. Self-organising teams went next: agents choosing their own collaborators sounds adaptive and behaves unpredictably under load. The planner assigns; the agents work; that is the whole social structure.
We also stopped letting the planner and the critic argue indefinitely. When they disagree, the plan says ship, the critic says no, the dispute goes to a human, with both positions summarised. Disagreement between agents turns out to be a feature precisely because it is a good signal for when a person should look.
“The pipeline works because it is boring: typed handoffs, fixed roles, and a human at every fork that matters.
If we rebuilt again tomorrow, we would keep the shape and spend the savings on better checkpoints. The glamour in multi-agent systems is in the agents. The reliability is in what sits between them.