By: Kumar Dattatreyan

In McKinsey's 2026 State of AI survey, 40% of respondents at organizations with more than $1 billion in revenue said they're scaling AI agents, up from 27% a year earlier. Across all 1,719 respondents in 97 countries, 37% attributed at least some EBIT impact to AI, and that share barely moved from the year before. About 6% qualified as high performers, the group that credits AI with at least 5% of EBIT and calls the impact significant. That share held flat too.

Adoption climbed while results stayed put, and the usual explanations are reasonable, from weak data to tools bought before anyone defined the problem. I think a large part of the gap traces to a question that most deployments never ask. Who's assigned to notice when the agent is wrong?

A coworker, in any workplace that functions, is a person whose mistakes somebody else is responsible for catching. A new analyst has a manager who reads the first few reports closely, teammates who expect certain things and a role that tells everyone what good work looks like. An agent that joins the same workflow often arrives with a license and an owner on the procurement side. Nobody's job includes reading what it produced last Tuesday.

That matters more for agents than it ever did for people, because agents fail in a different way.

The agent won't tell you

Suzel Wyvill-Jones ran a $120 million portfolio before she started telling executives the truth about AI, and she put the difference between software and AI failures in two sentences. "Right now, when things break with software, you know there is a problem because nothing is working. With AI, you're not going to find out until someone notices, well, there's a problem with this data."

She described loading data that was already 60 days old and watching a report populate on schedule with nothing to signal that it was wrong. She also said something that should change how leaders staff these projects: "AI projects pretty much start when you deploy the product." The launch is where the work of watching begins, which means the budget for watching has to exist before the launch does.

A person who made that mistake might at least say so. Carnegie Mellon researchers built a simulated software company called TheAgentCompany, with 175 tasks and simulated colleagues, to see how agents handle the work of an office. The best agent in the study completed about 30% of the tasks on its own, and the authors reported that most models struggled most on the chat platform where the simulated colleagues talked. Their conclusion was that the models still lack communication skills. Those were 2025 models and newer ones score higher, but the finding points at something scores don't fix. An agent that can't tell a colleague it's stuck won't flag its own failures, so the organization has to build the flagging into a person's job.

A role is what makes an error visible

You can only notice an error in work that someone defined. If nobody wrote down what the agent is for and what a good output looks like, then every output is plausible and no output is wrong.

Diana Larsen, who has written about launching teams through chartering, sees where this is heading. She asked what teams will look like with "a couple of humans and a team of supporting agents all collaborating together," and her answer starts with the charter. "Each functioning part of the team, whether AI or human, needs to know what their role is. That will determine how quickly they're able to do the work."

I'd push her point one step further. The charter is also the place where noticing gets assigned. A charter that says what the agent does, what a correct result looks like and which human reviews a sample of its output each week turns a vague hope about oversight into a job with a name on it. Diana also noted that the agents of today are "very capable but very narrow," and a narrow agent is the easiest kind to charter, because the role fits on a page.

Consider a hypothetical finance team that deploys an agent to match vendor invoices against purchase orders. The charter says the agent escalates any mismatch above a set amount to the analyst who owns exceptions, and that it never approves a payment itself. Every Friday the analyst pulls twenty matched invoices and checks them by hand against the source documents. Three months in, that sample is how the team learns the agent started mishandling one vendor's new invoice format, and nobody had to read all four thousand invoices to find out.

Ashwini Kumar builds with narrow agents. His proposal generator splits the work so that one agent focuses on formatting and another on the context of the request for proposal, and he described how the pieces connect: "And then you have an orchestration agent that tells each agent what to do." It worked well enough that the system produces a full proposal in one run. It also creates a management layer made of software, and the layer raises the question this whole article circles. When an orchestration agent assigns the work, a human still has to be assigned to notice when the orchestrator is wrong. A chain of agents with no person above it makes a black box that nobody can see into.

James Taylor, who helped build the decision management category, describes the shape that human role should take. In a well-run automated decision system the person isn't in each case, because that doesn't scale. "You're not part of each individual decision. You're controlling the set."

The supervisor reviews the decisions the system made and tests changes before they go into production. That's noticing at the level of the whole set rather than the single output, and Taylor adds that it never finishes, because "decisions are almost never static." Competitors move and regulations change, so a review that was right in March needs another look in June.

What the high performers did differently

McKinsey's survey found one difference that stands out. Nearly three quarters of the high performers said they had fundamentally redesigned their workflows, up from 55% a year earlier, and about a quarter of everyone else said the same.

The survey doesn't say what those redesigns contained, and I don't want to claim it does. What I can say is that a redesigned workflow is the only place a reviewer and a standard can be written in. Bolt an agent onto the old workflow and none of them has a home. The agent runs, the output flows to the next step and the person downstream assumes somebody upstream already checked it.

Software doesn't have a manager

The people who build and govern these systems will object, and they have a fair point. An agent is software. Calling it a coworker invites people to treat it as an accountable party, and it can't be one. Nalini Garg of Deloitte makes the governance version of the argument: accountability has to be settled before an agent acts, because once it has acted, organizations often can't tell whether a person or a program is responsible. Garg cites a 2026 Cloud Security Alliance survey in which 68% of organizations said they can't reliably tell AI-agent activity from human activity. You can't notice what you can't tell apart.

I agree with the objection, and the argument above depends on it. The coworker comparison earns its place as a test and nowhere else. If you'd assign someone to review a new hire's work for the first ninety days, assign someone to review the agent's. An agent can't be held accountable, which is exactly why a person has to be.

A second objection says the models are improving so quickly that the review burden will fade. Gartner expects over 40% of agentic AI projects to be canceled by the end of 2027, on escalating costs and unclear business value, which suggests the models aren't the only variable. Better models also fail less often, and that creates its own trap. A model that fails one task in three trains its users to check everything. A model that fails one in a hundred trains them to stop checking, and the hundredth failure is the one that reaches a customer.

What to change first

List every agent that touches real work, and next to each one write the name of the person who reviews a sample of its output on a fixed schedule against a standard written beforehand. If you can't fill in the name, that agent has no manager yet and shouldn't run unattended.

Then put the agent into the team's charter or working agreement, in the same document that tells the humans what they own. The people reading the output need to know the agent's role as well as the agent does.

How the team responds when the reviewer finds an error matters as much as the catch. Diana Larsen's argument that blame is the enemy of learning applies here, because a team that punishes whoever reports a bad agent output will soon stop receiving reports. Treat each caught error as information about the charter, since the standard may have been unclear or the agent may have been pointed at work it can't do, and revise the document before you revise the agent.

Related Podcast Episodes

Episode 169: Co-Intelligence, Why Learning Together Beats Knowing Together, with Diana Larsen Diana Larsen argues that teams should treat learning as the work AI can't take over, and she describes what chartering a team of humans and agents could look like. Her case that blame is the enemy of learning applies directly to how you respond when an agent gets something wrong.

Episode 172: She Ran a $120M Portfolio, Now She's Telling CEOs the Truth About AI, with Suzel Wyvill-Jones Suzel Wyvill-Jones explains why AI demos mislead executives and why so many corporate AI initiatives fail for reasons that have nothing to do with the technology. Her point about data that quietly goes stale shows why agents need a human watching.

Episode 173: The AI Decisions Are Wrong, and Your Data Isn't the Problem, with James Taylor James Taylor treats the decision as a thing you design, apart from the data and the process. He's blunt that a single large language model can't reliably run a regulated decision, and he shows what the human role looks like once the system is in production.

Episode 174: AI Is Not a Magic Bullet, What Enterprise AI Gets Wrong, with Ashwini Kumar Ashwini Kumar has built agentic systems himself, and he talks about what worked: narrow agents with a limited scope and a person who still makes the final call. The proposal tool he describes cut drafting from about two weeks to a day or two.

Choose one agent already running in your organization and find out who read its output last week. If the answer is nobody, book a 30-minute call and we'll work out who should.