The Real Work of Building Reliable AI Agents
For the last several months, I have been building and operating a growing network of AI automation, agents, multi-agent workflows, scheduled processes, memory systems, model-routing configurations, and human approval gates across my business.
At one point I counted 34 active automation and scheduled workflows operating across OpenClaw, Claude, ChatGPT, local scripts, content systems, CRM and meeting workflows, reporting processes, podcast production, backups, and other parts of the business.
I want to make an important distinction before going any further: I do not call all 34 of them agents. Some are automation. They run on a schedule, perform a defined operation and produce a predictable result. Others have become genuinely agentic. They inspect current state, retrieve context, choose between tools, make decisions about what happens next, work across multiple steps, preserve information between runs, evaluate results and escalate situations they should not handle autonomously.
That distinction has become increasingly important to me because I think we are spending too much time talking about AI agents as if the intelligence of the model is the most interesting part of the system.
After operating these systems, troubleshooting them and watching them fail in ways that had very little to do with model intelligence, I no longer think that is where the difficult work sits.
The model is not the system.
The system around the model is becoming the work.
We are moving from model engineering into harness engineering
I recently read S Sankar's article on what is increasingly being called the "agent harness." He defines it as the infrastructure surrounding an AI agent that controls, tests, monitors and evaluates what the agent is doing. His framework includes state, tools, planning, memory, orchestration, context management, safety and governance, observability and evaluation.
That vocabulary immediately connected with what I have been building because I had arrived at a similar architecture from the opposite direction.
I was not trying to build an "agent harness ", I was trying to make agents dependable enough to use inside an actual business.
Those two paths eventually lead to the same place.
My own operating model has evolved around context, rules, memory, tools, action, evaluation and feedback. Sankar's framework describes the infrastructure components. Mine describes how intelligence moves through them. Together they expose something I think we need to understand as agents become capable of performing longer and more consequential work.
Giving an AI more capability creates a larger control problem.
The more tools an agent can access, the more important permissions become. The more memory it retains, the more important context management becomes. The more tasks it performs autonomously, the more important state becomes. The more agents you connect together, the more important orchestration becomes. The longer those systems operate, the more important observability and evaluation become.
Anthropic is now explicitly describing its own work in these terms. Its engineering team has written about harnesses for long-running agents and about the need to continually reconsider the assumptions encoded in those harnesses as models improve. Anthropic has also documented the difficulty of maintaining coherent progress across context windows, including the need for agents to leave artifacts and state for the next session rather than expecting the model to somehow retain an uninterrupted understanding of the work.
OpenAI's current agent architecture reflects many of the same concerns. Its tooling separates agents, tools, state, orchestration, handoffs, guardrails, human approvals, tracing and evaluation rather than treating an agent as a model with an elaborate prompt.
This is not terminology for terminology's sake. I have repeatedly watched these layers fail independently.
Building agents is getting easier. Building reliable systems around them is where the work begins. I work with leadership teams and organizations that are trying to move from individual AI use into governed, repeatable workflows and agentic systems. If that is where your organization is heading, let’s talk.
A healthy system can still have a dead agent
One of the more useful lessons OpenClaw taught me came from a deceptively simple failure.
- The Gateway was healthy.
- The configuration looked healthy.
- RPC connectivity was healthy.
- OpenClaw itself was running.
- The actual main-agent model turn failed.
From an infrastructure perspective, several green lights were showing. From the user's perspective, the system did not work.
That changed one of my health-check requirements permanently. A system check can no longer stop at "Is the gateway running?" I require an actual agent turn.
That sounds painfully obvious after the fact. Plenty of operational lessons do.
It also illustrates a much larger issue with agentic systems.
We tend to evaluate them at the wrong layer.
- A server responds, so we call the system healthy.
- An API call succeeds, so we call the integration healthy.
- A tool invocation completes, so we call the task successful.
- An agent says it finished, so we call the work finished.
None of those conclusions necessarily follow.
In another OpenClaw failure, the configuration, OAuth, Gateway and RPC were functioning, but the actual model could not execute because a provider usage limit had been reached. The architecture contained fallback models, yet the behavior of pinned model sessions meant fallback did not happen in the way the configuration appeared to promise.
Again, the model was not the interesting failure.
- The routing architecture was.
- The session state was.
- The authentication layer was.
- The assumptions encoded around fallback behavior were.
That is harness engineering.
OpenClaw itself describes its architecture around a Gateway capable of connecting models, devices and channels. In my environment, it has become one part of a larger control layer around agents, tools, memory, scheduled work and model routing.
Once you operate systems like this long enough, the abstraction of "the AI did it" becomes almost meaningless.
- Which AI?
- Using which model?
- With what context?
- Through which tools?
- Under which permissions?
- From what state?
- Using which memory?
- With what fallback?
- Who evaluated the result?
- Who was allowed to release it?
Those questions are becoming far more consequential than the original prompt.
Multi-agent systems do not automatically make work more trustworthy
I have another system built around six Claude Code agents.
Three perform work. They find code, run tests and review implementations.
Three exist primarily because I do not trust completion to verify itself.
One is a Verifier. It takes claims of completion and traces them back to the artifact. If an agent says something was changed, the Verifier looks for the change. If it says something passed, the Verifier looks for evidence that it passed.
One is a Voice-check agent that evaluates content against a 36-section writing and voice system.
The final Gatekeeper reviews the work and the reports and returns a RELEASE or HOLD decision, with a deliberately limited number of issues so the review itself does not become another endless AI process.
During early runs, those checks caught incorrect Git tracking claims, unsafe shell permissions, factual errors in documentation and numerous writing-rule violations.
The interesting part was not that agents made mistakes. Humans make mistakes too.
What interested me was how confidently a system could report completion when completion had not actually been proven.
That produced a rule I now use across my agentic work:
A completion report is not evidence of completion.
The artifact is the evidence.
OpenAI's current evaluation guidance reaches a similar engineering conclusion from a different direction. It recommends inspecting end-to-end traces that include model calls, tool use, handoffs and guardrails, then evaluating those traces against structured criteria.
That matters because an agent can generate a perfectly reasonable final response after taking an entirely unreasonable route to get there.
When we only judge the final output, we lose the execution history.
Agentic systems require us to evaluate both.
Long-running work changes the reliability problem
There is another claim circulating in conversations about agents that deserves more precision.
We frequently hear that agents can now complete tasks lasting many hours or days. METR's research is often cited to support that conclusion, but its "time horizon" measurement does not mean the agent itself literally ran autonomously for that amount of wall-clock time.
METR measures task difficulty using the amount of time a human expert would require to complete the same task. A 50 percent time horizon means that the agent is predicted to succeed half the time on tasks of approximately that human-equivalent duration. METR explicitly warns against interpreting the measurement as the amount of time an agent can simply operate autonomously.
The real story is still significant. METR has documented a substantial increase in the complexity and duration of software tasks frontier agents can complete, and its 2026 reporting describes systems successfully performing work that would take humans hours, days and, in some benchmark settings, considerably longer.
But longer capability creates more opportunities for state to drift.
That is the part I think business leaders should be paying much closer attention to.
A five-second answer primarily needs to be correct.
A five-hour workflow needs to remain correct.
Those are different engineering problems.
As an agent operates, it may retrieve new information, modify files, call external services, hand work to another agent, encounter a permission boundary, compact its context, resume from saved state, switch models, retry a failed operation and incorporate human feedback.
Every transition introduces another place where the representation of reality can diverge from reality itself.
I think state reconciliation may become one of the hardest enterprise AI problems
This is where my thinking has moved beyond the question of whether an individual agent is reliable.
Consider a relatively ordinary business environment.
An email says something has been approved. A calendar invitation confirms the meeting. The project database still says "waiting." An agent's memory says follow up next Tuesday. A CRM record was never updated. Another workflow sees the old CRM state and sends a reminder.
Every individual system may be functioning correctly.
Every API may have returned 200 OK.
Every agent may have followed its instructions.
The business state is still wrong.
Nothing had to hallucinate.
Nothing had to crash.
There was simply no reliable mechanism ensuring that all of those systems represented the same current reality.
That is a very different AI reliability problem than the one most organizations are preparing for.
The conversation around AI risk still spends enormous energy on whether a model might produce an incorrect sentence. That remains relevant, but agents introduce an operational category of error where the sentence can be correct and the system can still be wrong.
I increasingly think the difficult question will be:
Can a network of models, agents, tools, memories, databases, automation and humans maintain a coherent version of reality while work is continuously changing that reality?
That is not a prompting problem.
It is an architecture problem.
Building agents is getting easier. Building reliable systems around them is where the work begins. I work with leadership teams and organizations that are trying to move from individual AI use into governed, repeatable workflows and agentic systems. If that is where your organization is heading, let’s talk.
Human-in-the-loop cannot mean "the human checks everything"
The obvious response is to put a human in the loop.
I agree with the principle. I disagree with how casually the phrase is used.
If a person has to inspect every action taken by an autonomous system, we have built an extremely expensive notification machine.
Human governance needs architecture too.
In my systems, I increasingly think in terms of escalation boundaries. First-contact communication, apologies, money, sensitive personal references, messages involving client frustration and decisions that could materially affect a relationship remain areas where I want direct human judgment.
Other actions can be automated under defined constraints.
OpenAI makes a similar distinction in its agent tooling by separating automated guardrails from human approval gates. Automated checks can evaluate inputs, outputs and tool behavior, while actions with meaningful side effects can pause for human review.
That is much closer to how I think mature agent systems need to work.
Human attention is finite.
The system therefore needs to know not merely what it is capable of doing, but where its authority ends.
Governance is not a policy document sitting beside the agent.
Governance has to exist inside the execution path.
The next AI skill may be knowing how to engineer the environment around intelligence
We spent the first phase of generative AI learning how to communicate with models. Then we learned how to connect those models to tools. Now we are giving them memory, planning capabilities, computer access, APIs, scheduled work, and the ability to work with other agents.Every addition increases what AI can do, but every addition also introduces another dependency that can fail and change the outcome.That is why I think the professional conversation around AI needs to move beyond prompt engineering and even beyond agent building. Increasingly, the challenge is about harness engineering: building the systems around the model that make its actions reliable.Once an agent is operating inside a business, we need to know what it believes to be true, what state survives between runs, which memories are retrieved, and what happens when a provider fails, authentication expires, or two systems disagree. We need to know what proves a task is complete, which actions require approval, where the execution trace lives, and how failures are investigated.Perhaps most importantly, we need to ask what happens when every technical component reports success while the business outcome is still wrong.Those are the questions I find myself working on now.I still care deeply about better models. Model capability determines what becomes possible. But once AI begins acting inside a business, capability is only the beginning of the engineering problem.Reliability has to be designed around it.Increasingly, the intelligence of an AI system may be judged as much by the quality of the architecture surrounding the model as by the model itself.About The Author
Anca Platon Trifan, CMP, WMEP helps event professionals build the physical energy, mental resilience, and cognitive capacity needed to perform at their best in demanding environments. Her work connects high-performance principles with the realities of event production, focusing on sustainable energy, wellbeing, and leadership under pressure.With more than 20 years of experience in event production and event technology, Anca brings a practical understanding of what it takes to stay effective through demanding event cycles. She is the CEO of Tree-Fan Events Productions LLC, host of Events: Demystified, and founder of the #FIT4EVENTS™ framework, which helps event professionals strengthen their energy, focus, resilience, and overall capacity for high-performance work.An Ironman triathlete and competitive pro bodybuilder, Anca brings firsthand experience in training, recovery, discipline, and sustainable performance to her work with event leaders and professionals.Work With Me
If your team is exploring AI but feeling the friction between tools, workflows, and real execution, this is exactly where I come in.
I deliver:
- AI Keynotes for conferences and leadership teams
- Hands-on Workshops focused on real workflows, not theory
- Leadership Training on decision-making, cognitive load, and AI integration under pressure
This is not about adding more tools. It’s about building systems that hold when the work accelerates. Book a conversation here.
