Why Running AI Agents Is Harder Than Adopting Them
Interest in adopting "AI agents" is growing fast across industries, and IT operations is no exception. Beyond simple internal chatbots, agents that classify alerts, trace incident causes, and even propose recovery steps are now arriving in earnest. With this shift, a new question is coming up inside operations teams: who is accountable when automation goes wrong? In many cases, adoption comes first, before there is a clear answer to that question.
Major monitoring and observability solutions and ITSM tools are adding AI agent capabilities one after another, from alert classification and incident root cause analysis to automated recovery loops. Reference cases showing where agents fit into operations work are piling up quickly. The hard part comes next. How you split the work between what people must own and what you can hand to agents decides how the adoption turns out.
Gartner predicts that over 40% of agentic AI projects will be canceled by the end of 2027 without making it past the pilot stage. The DORA 2025 report also shows that projects do not fail for technical reasons alone. Its conclusion: teams with well-established operating practices get more out of AI, while teams without them see their weaknesses exposed faster because of AI. In other words, what happens after adoption matters more than the adoption itself.
This post maps the boundary between the parts of IT operations that AI agents can automate and the parts people must still own. If your team is evaluating AI agents or already running them, drawing this boundary first is a good way to cut down on trial and error. Let's walk through it based on recently published official sources.
1. Where AI agents have taken hold in automation
Looking across what IT operations solution vendors published this year, the work AI agents actually handle in production narrows down to 3 areas.
1. Alert classification and prioritization - grouping noisy alerts and ranking them by priority
2. Incident root cause analysis and hypothesis generation - reviewing telemetry, change history, and topology together to narrow down candidate causes
3. Automated recovery loops with human review - running runbooks, proposing fix code, and applying it after approval
The 3 areas differ in how far they are automated and in how much the operations team still has to handle directly.
1-1. Alert classification and prioritization
Alert classification is where operations teams spend the most time every day, but alerts themselves follow many repetitive, regular patterns. The same type of event recurs, and a single incident often splits into several alerts. Because of this, alert classification and prioritization is the first area where AI agents have taken hold in IT operations automation.
AI agents in major ITSM solutions summarize alerts in plain language so operators can grasp the context right away. According to the vendors' own announcements, some environments have reported cutting alert noise by up to 99% and reducing mean time to resolution (MTTR) by about 40%. These figures assume an optimized operating environment, though. Early in adoption, or where alert policies have not been cleaned up, the actual effect may differ.
In observability, features that automate alert correlation and group multiple alerts together, showing the impact scope and likely cause side by side, are also spreading quickly. In the end, the point of automating alert classification is not just to reduce the number of alerts. It is to help operators decide faster which alerts to look at first, which alerts come from the same cause, and which situations could turn into real service impact.
1-2. Incident root cause analysis and hypothesis generation
Root cause analysis is a step harder than alert classification. A meaningful hypothesis only emerges when you look at multiple kinds of telemetry such as metrics, logs, and traces together with change history and service topology.
Major observability vendors say AI agents developed and evaluated on thousands of real incidents can sharply cut the time it takes to identify a root cause. Some vendors cite their own verification figures claiming root causes are found 90% faster, and incident management tools are also advancing toward less time spent on manual investigation through automated diagnosis and context collection. APM and observability cloud vendors are adding AI troubleshooting agents in the same direction, which also means agents are quickly taking on a larger supporting role in root cause analysis.
In root cause analysis, though, the agent's role is not to confirm the answer. It is closer to quickly presenting possible causes and the evidence behind them. Which hypothesis to adopt, what action to take, and how that action will affect the service are still for the operator to decide. The point of this area is not to replace people but to narrow the scope of investigation and the hypotheses so people can decide faster and more consistently.
1-3. Automated recovery loops with human review
Recovery is where automation is handled most carefully. The SRE agent workflow published by a major cloud provider shows today's standard structure well. When the agent analyzes an incident, the result is automatically filed in the issue tracker, and a code assistant proposes candidate fix code on that issue. Only after a person reviews and approves the code does it reach production.
SRE agents released by incident management tool vendors follow the same flow. Diagnosis, context collection, and action proposals run automatically, but actual execution is set by default to happen only after human approval.
What has been automated is hypotheses and procedures. Decisions and accountability still sit with people. How far to open up the scope of automation is the first criterion to set early in adoption.
Principles for designing automation
If you go fully automated from day one with no human review, it becomes hard to trace accountability when something goes wrong. The approach proven in real operations is to automate only up to hypothesis generation early on, keep execution behind human approval, and then expand the scope of automation step by step once the agent's judgment accuracy has stabilized.
2. Areas the operations team must handle directly
The areas AI agents cannot automate are also becoming clear. Gartner estimates that while thousands of vendors claim agentic AI capabilities, only about 130 actually offer real agent capabilities. The term Gartner used to describe this is "agent washing": a market trend of rebranding existing chatbots, RPA, and AI assistants as agents, which makes it harder for adopting companies to judge what they are buying.
The gap between adoption speed and trust also shows up in the numbers. According to the DORA 2025 report, 90% of technology professionals use AI at work, and more than 80% say they feel more productive. Yet 30% said they have little or no trust in AI-generated code. Adoption is growing fast, but the responsibility for verifying results still rests with people.
The work the operations team must handle directly generally falls into the following 4 areas.
Areas the operations team handles directly | Why |
Judging business context | The same incident can have a different priority depending on the affected customer segment, revenue timing, and contractual SLA.
Agents read telemetry well, but the organization has to define the business context itself. |
Permissions and accountability | Changes to operational permissions, security policy enforcement, and changes affecting external systems require accountability tracking.
If an agent performs them directly, the approval process and accountability structure must be clear. |
Never-before-seen incident patterns | Agents are strong on patterns found in their training data. In areas with no incident history, such as after adopting
a new architecture or adding a new integration, people have to form hypotheses from scratch. |
Reliability of the agent itself | The rate at which agents misjudge or hallucinate is not zero.
The operating framework needs a process for re-verifying the agent's decisions. |
Fast growth in automation does not make autonomous operations possible right away. If anything, the boundary between what agents handle and what people must own is becoming more important, and setting that boundary is emerging as a new role for the operations team.
3. New responsibilities for IT operations teams
So does the operations team end up with less work? In short, the work is changing in kind more than shrinking. As agents automatically handle alert classification and first-pass hypotheses, the center of gravity for people shifts toward verifying hypotheses and designing the safe scope of automation. Google's published SRE operating guidance makes the same point: the SRE role is shifting from cleaning up after incidents to preparing in advance so the service recovers quickly even when incidents happen.
In operating environments where AI agents are part of everyday work, the operations team has 3 new responsibilities: verifying what agents produce, preparing the material agents draw on, and setting safety criteria for automation.
3-1. Verifying what agents produce
Verifying whether the hypotheses and action proposals an agent generates automatically are actually correct remains a human responsibility. The SRE agent verification flow we looked at earlier is a typical example. When candidate fix code produced by the agent lands in the issue tracker, it is applied to the actual code only after a person reviews and approves it. Without verification, an agent's wrong judgment can go straight into production, so human confirmation acts as the safety mechanism for automation.
At the verification stage, the operations team mainly checks the following: whether the telemetry the agent used is sufficient, whether the hypothesis has ruled out other possibilities, whether the impact scope of the action is clear, and what results similar changes produced in the past.
3-2. Preparing the material agents draw on
Once verification of agent output is in place, the next step is preparing the material that determines the quality of that output. For an agent to produce good hypotheses, the material it can reference has to be well organized. Deciding which operational data to show the agent, which manuals to organize in searchable form, and which past incident history to provide as reference is a new area the operations team has to look after.
This work overlaps with what operations teams already do. Writing runbooks, organizing postmortem reports, maintaining asset management data (CMDB), and documenting service architecture diagrams are already established as basic IT operations work. What has changed is that this material can no longer be shaped only for people to read; it has to be refined into a structure agents can read and use. You also need to decide where to store documents, what tags to classify them with, and who gets access. Defining how far agents can access sensitive operational data is especially important from a security perspective.
The key point is that agents do not get smarter on their own. The domain knowledge and incident history the operations team has accumulated directly determine the quality of the agent's answers. In an environment where this material is not organized, any tool you bring in will hit the same limits.
3-3. Setting safety criteria for automation
Automation comes with risks: misconfigurations, uncontrolled automated recovery, and unverified changes can go straight into production. From an SRE perspective, automation is an important way to reduce repetitive work, but it cannot fully replace every operating procedure. Interpreting the context of an incident, judging the impact scope of an action, and deciding whether to execute it in the end are still up to people.
So the operations team has to define safety criteria in advance: under what conditions, and how far, an agent can execute on its own. The criteria fall into 3 broad types.
- Time windows for automated recovery: During peak traffic hours or right after major deployments, restrict automated actions from running without human approval.
- Scope of resources that can be changed automatically: Clearly define which servers, services, settings, and policies can be changed automatically.
- Limits on change types that can proceed without human approval: Set criteria that separate idempotent changes, meaning changes guaranteed to produce the same result no matter how many times they run, from changes that are not.
Defining these criteria in advance has become a new role for the operations team. In the end, the operations team's work has not shrunk. It has moved up a level, to verifying hypotheses and designing safety criteria.
4. Changes in Korea's IT operations landscape
If everything so far reflects the trend seen in global vendor material, IT operations in Korea are moving in the same direction. For Korean IT operations teams evaluating AI agents, keeping track of regulatory changes matters as much as following the technology. Changes in regulation and policy stand out in particular.
- Korea's AI Basic Act (Framework Act on the Development of Artificial Intelligence and Establishment of a Foundation for Trust) takes effect: In force since January 2026, the AI Basic Act sets out obligations for high-impact AI providers, impact assessments, and transparency requirements. It is worth checking in advance whether agents used in IT operations fall into the high-impact AI category, and what accountability standards should apply to operational data and decision-making processes.
- AI on government administrative networks: AI use, which had stayed mostly on internet-facing networks because of security concerns, is expanding to internal government networks. Public-sector IT operations teams now need to separately define the scope of AI use on internal networks, standards for handling operational data, and how access permissions are managed.
- AI guidelines for the financial sector: Among the principles set out by Korea's Financial Services Commission, the most relevant for IT operations teams is that "at the current stage, AI is a tool that supports work, and final decisions and accountability must rest with employees." IT operations teams at financial institutions should set up audit trails for AI-driven results and clarify where accountability lies in advance.
All 3 changes point the same way. Sorting out operating procedures, data integration, access permissions, and accountability tracking first, rather than focusing on adopting AI agents itself, is what ultimately leads to results.
5. Before you start evaluating adoption
The alert classification, root cause analysis, and recovery loop automation covered so far are the first ways AI agents are entering IT operations. At the same time, the areas the operations team must handle directly and the new responsibilities it has to look after are growing too. If you are evaluating AI agents, deciding up front which areas to automate first and which areas people will verify will cut down on trial and error.
One thing that is easy to miss at the evaluation stage is integrating operational data. For an AI agent to produce good hypotheses, DB, APM, infrastructure, and log data need to be in one place. In environments where monitoring tools are scattered by domain, you need to sort out an integrated monitoring structure before bringing in agents. Collecting data in one place is not enough on its own, though. As we saw earlier, integration only pays off when material such as runbooks, incident history, and service architecture diagrams is organized in a form agents can read and use.
Checking the following items before you evaluate adoption will help reduce trial and error.
- Is alert noise not going down and eating up the operations team's time?
- Does incident root cause analysis depend on a specific person, so analysis stops when that person is away?
- Are DB, APM, infrastructure, and log monitoring scattered, making integrated monitoring difficult?
- Are incident history and runbooks disorganized, leaving agents without enough material to reference?
If any of these apply, put automation scope design and an integrated operational data structure on your evaluation schedule before comparing automation tools. That running AI agents is harder than adopting them is already the direction that global cases and regulatory changes in Korea point to together.
Sources
"Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027" - Gartner (2025.06.25) 2025 DORA State of AI-assisted Software Development - Google Cloud / DORA (2025) Now Assist for IT Operations Management data sheet - ServiceNow Datadog Launches Bits AI SRE Agent to Resolve Incidents Faster - Datadog (2025.12.02) Agentic DevOps: Evolving software development with GitHub Copilot and Microsoft Azure - Microsoft Azure Introducing Event iQ: Smarter Event Correlation in Splunk IT Service Intelligence (ITSI) - Splunk Framework Act on the Development of Artificial Intelligence and Establishment of a Foundation for Trust - Korea Law Information Center "AI chat services" become available on government administrative networks - Korea Policy Briefing
Related articles
