{"id":19280,"date":"2026-05-26T17:16:01","date_gmt":"2026-05-26T17:16:01","guid":{"rendered":"https:\/\/globalnewstoday.uk\/index.php\/2026\/05\/26\/why-ai-agents-must-be-proven-before-they-are-deployed-cx-today\/"},"modified":"2026-05-26T17:16:01","modified_gmt":"2026-05-26T17:16:01","slug":"why-ai-agents-must-be-proven-before-they-are-deployed-cx-today","status":"publish","type":"post","link":"https:\/\/globalnewstoday.uk\/index.php\/2026\/05\/26\/why-ai-agents-must-be-proven-before-they-are-deployed-cx-today\/","title":{"rendered":"Why AI Agents Must Be Proven Before They Are Deployed – CX Today"},"content":{"rendered":"

Editorial Type
Tech Categories
Hot Topics
Business Priorities
Industry Verticals
Connect
More from CX Today
Home<\/a><\/span> \u2192<\/span> AI & Automation in CX<\/span><\/span>
AI agents can transform customer experience \u2014 but without rigorous testing, simulation, and observability, they risk undermining the trust enterprises depend on
Published: May 26, 2026
Rob Wilkinson
The push to move AI agents from pilot to production is creating a new kind of tension inside enterprises. The technology may look ready, but leaders still\u00a0have to\u00a0answer harder questions: What happens when the\u00a0<\/span>AI\u00a0<\/span>agent makes the wrong decision? Who sees it, and how fast? What does the human <\/span>inherit when\u00a0the handoff\u00a0happens? And what damage is invisible until customers start leaving?<\/span>\u00a0<\/span>
In a recent conversation,\u00a0<\/span>Tony Shen, Senior\u00a0Product Manager\u00a0at Amazon Connect<\/span><\/b>\u00a0Customer<\/span><\/b>, and\u00a0<\/span>Jeremy Puent, Principal Solution Architect at Amazon Connect<\/span><\/b>\u00a0Customer<\/span><\/b>, laid out a pragmatic view of what responsible deployment needs to look like. Their\u00a0central point\u00a0was simple: the fastest way to scale agentic AI is to build confidence first\u00a0using testing, simulation, and observability that match real customer behavior.<\/span>\u00a0<\/span>
Puent\u00a0put it plainly: \u201cPoor customer experience equates to losing trust.\u201d\u00a0<\/span>And the business cost of lost trust is quantifiable:\u00a0<\/span>According to Harvard Business\u00a0<\/span>Review\u2019s<\/span>\u00a02025 Customer Retention Technology report,\u00a0<\/span>acquiring\u00a0a new customer costs 5\u00a0to\u00a025\u00a0times as much as\u00a0retaining\u00a0an existing one. Every customer interaction that erodes confidence has a downstream price tag that rarely shows up on the deployment dashboard.<\/span>\u00a0<\/span>
He\u00a0went on to\u00a0argue that many organizations are still\u00a0validating\u00a0AI agents in ways that do not reflect\u00a0production\u00a0reality.\u00a0<\/span>\u00a0<\/span>
Manual testing can help early on, but it\u00a0doesn\u2019t\u00a0scale to the number of scenarios an AI agent will face once\u00a0it\u2019s\u00a0handling live customer interactions. \u201cPeople testing AI agents today is largely a non-automated function,\u201d Puent said. \u201cThey\u2019ve got a human being sitting down, interacting with it, subjectively saying\u2026 it was good or bad.\u201d<\/span>\u00a0<\/span>
The bigger gap is realism. Desk-based roleplay\u00a0can\u2019t\u00a0recreate the messiness of real calls\u00a0\u2013\u00a0noise, interruptions, stress, mumbling, changing context mid-sentence. AI-powered simulation can replay and vary thousands of real-world\u00a0interactions\u00a0so teams can test broader coverage and tougher conditions without exposing live customers to failure.<\/span>\u00a0<\/span>
\u201cYou\u2019ve got to find ways to minimize the risk and test more scenarios, complete with confidence,\u201d Puent said.<\/span>\u00a0<\/span>
In other words, the rush is not just about speed. It is about the mismatch between how AI agents behave, and\u00a0how many organizations still evaluate readiness.<\/span>\u00a0<\/span>
Testing that\u00a0doesn\u2019t\u00a0reflect the full range of real customer scenarios\u00a0isn\u2019t\u00a0testing,\u00a0it\u2019s\u00a0wishful thinking. And\u00a0wishful thinking\u00a0is how you discover your AI agent\u2019s worst moments\u00a0live, in front of customers who will remember.<\/span>\u00a0<\/span>
Puent warned that when organizations treat deployment as a purely technical exercise, they may check the engineering boxes while missing the customer experience outcomes that\u00a0actually matter.<\/span>\u00a0<\/span>
\u201cYou might be missing the feedback loop\u2026 the things that lead to the customer experience,\u201d he said. \u201cEven if it passed all of the tests, but it\u2019s a miserable experience for the customer, you\u2019re doing the wrong thing.\u201d<\/span>\u00a0<\/span>
Shen focused on what changes inside the customer experience\u00a0operation,\u00a0when\u00a0AI agents become decision-makers between IVR and a human advisor. In traditional flows, the customer chooses to reach a person. With an AI agent, the system makes decisions that shape the conversation before a human ever joins. \u201cThere\u2019s an AI agent in\u00a0the middle,\u00a0<\/span>making<\/span> individual decisions,\u201d Shen said.<\/span>\u00a0<\/span>
\u201cOn the human side, they need to know what decisions that AI agent made and have\u2026 observability\u2026 or else\u2026 they\u00a0might be doing the duplicate things or\u00a0might be confusing the customer.\u201d
This is where many deployments stumble. Without visibility into what the AI decided and why, escalations become slower, more confusing, and\u00a0<\/span>significan<\/span>tly\u00a0<\/span>more costly.<\/span>\u00a0\u00a0When\u00a0<\/span>your team <\/span>has\u00a0<\/span>to piece together a broken AI interaction, handle times spike<\/span>, an<\/span>d as Forrester\u2019s January 2025 report,\u00a0<\/span>I<\/span>mproving CX Can Drive More Than $1B In Revenue<\/span>, highlights<\/span>, these operational friction p<\/span>o<\/span>ints actively drain millions from the bottom line.<\/span>\u00a0<\/span>
Systems designed to avoid human escalation will\u00a0optimize\u00a0for containment. Systems designed\u00a0for intelligent\u00a0and flexible AI (agentic and\u00a0deterministic with a human handoff when needed) will\u00a0optimize for\u00a0resolution. The architectural choice\u00a0determines\u00a0which outcome your customers experience \u2014 and which one they remember.<\/span>\u00a0<\/span>
Shen tied the risk to Amazon\u2019s \u2018earn trust\u2019 leadership principle and connected it to a business reality CX leaders know well.<\/span>\u00a0<\/span>
\u201cTrust is hard to earn and easy to lose. Once the trust is lost, it is more expensive to earn back.\u201d
He also pointed out that\u00a0retaining\u00a0existing customers is cheaper than winning new ones, which means failures that erode loyalty can become a compounding cost.<\/span>\u00a0<\/span>
That compounding effect is more severe than most deployment teams expect: one bad AI interaction can drive churn from a customer who\u00a0wasn\u2019t\u00a0planning to leave.<\/span>\u00a0<\/span>
Lost trust is costly, and often impossible, to win back with the same customer. Retention is always cheaper than acquisition, so deployment\u00a0isn\u2019t\u00a0just a product decision;\u00a0it\u2019s\u00a0a retention decision.\u00a0<\/span>\u00a0<\/span>
For Shen, the answer is not fear. It is discipline. Enterprises need confidence that AI agents can handle real scenarios before customers are exposed to them.<\/span>\u00a0<\/span>
That requires testing beyond the happy path.<\/span>\u00a0<\/span>
\u201cYou got to let AI agent\u2026 behave in those edge cases,\u201d Shen said. \u201cMy team and I, we\u2019re building those tools at Amazon Connect<\/span>\u00a0<\/span>Customer<\/span>\u00a0to let you validate at scale before you deploy.\u201d<\/span>\u00a0<\/span>
Shen argued that skipping structured pre-production stages is a dangerous shortcut because it shifts uncertainty into the environment where mistakes are most visible and most expensive.<\/span>\u00a0<\/span>Forrester\u2019s 2025\u00a0<\/span>Total Experience Score\u00a0reinforces this, showing that brands suffering public CX failures face steep financial penalties in immediate customer\u00a0churn\u00a0and long-term\u00a0<\/span>recovery<\/span>\u00a0costs.<\/span>\u00a0<\/span>
Pre-production testing\u00a0isn\u2019t\u00a0just technical\u00a0validation,\u00a0it\u2019s\u00a0the basis of customer trust. Every scenario you\u00a0don\u2019t\u00a0test before\u00a0launch\u00a0is one your customers will test for you, with their\u00a0real experience\u00a0at stake.<\/span>\u00a0<\/span>
\u201cWhen you\u2019re\u00a0testing in\u00a0production, it\u2019s just too risky,\u201d he said. \u201cYou\u2019re exposing all the issues you haven\u2019t found out before\u2026 you really don\u2019t want the real customers to test your product.\u201d<\/span>\u00a0<\/span>
He described a phased approach, moving through environments such as dev, beta, gamma, and pre-production. Each stage exists to catch issues early, automate repeatable validation, and make rollback possible if changes introduce problems.\u00a0<\/span>\u00a0<\/span>
This\u00a0approach allows you to slow down\u00a0and then\u00a0speed up.\u00a0Resolving\u00a0any\u00a0issues\u00a0means you\u00a0go live quickly and smoothly,\u00a0then adding\u00a0more use cases\u00a0also goes quickly and smoothly.\u00a0<\/span>\u00a0<\/span>
For AI agents, the requirements are fundamentally different than with deterministic automation \u2014 because agent behavior changes with context and language.<\/span>\u00a0<\/span>
\u201cAI agents, they\u2019re not static,\u201d Shen said. \u201cThey respond in context\u2026 and they\u2019re non-deterministic.\u201d<\/span>\u00a0<\/span>
In other words,\u00a0they\u2019re\u00a0probabilistic: they adapt to tone, phrasing, and context in real time, which is what makes them feel conversational. But it also means you\u00a0won\u2019t\u00a0always get the exact same answer twice\u00a0-introducing a class of risk traditional automation\u00a0doesn\u2019t.<\/span>\u00a0<\/span>
That\u2019s\u00a0why \u201cslowing down to speed up\u201d is a strategy, not caution.<\/span>\u00a0<\/span>
Shen also challenged the idea that skipping stages \u201csaves time.\u201d \u201cIt just moves the issue and the problems from testing to production,\u201d he said.<\/span>\u00a0<\/span>
One step Shen believes is skipped too often is simulation, particularly at scale. For AI agents, simulation is not only about testing responses. It is also about testing the quality of the knowledge base the system depends on.<\/span>\u00a0<\/span>
\u201cIf you put garbage in, you get garbage out.\u201d
He explained that customers ask the same question in many different ways.\u00a0They bring different moods, different assumptions, and different language patterns.\u00a0That\u2019s\u00a0why effective simulation needs to be grounded in real interactions\u00a0\u2013\u00a0not desk-written scripts.<\/span>\u00a0<\/span>
With Amazon Connect Customer, teams can replay thousands of historical customer conversations against an AI agent before it goes live, testing the full range of how people\u00a0actually speak, escalate, get confused, and change their minds. Done at that\u00a0scale,\u00a0simulation can surface ambiguity in documentation and expose scenarios where the agent may fail or hallucinate,\u00a0before any real customer is affected.<\/span>\u00a0<\/span>
Each gap discovered in simulation is a piece of customer trust you just protected. Each gap discovered in production is a piece of customer trust you just lost.<\/span>\u00a0<\/span>
\u201cWith large-scale simulation, you\u2019re able to test your documentation, your knowledge base thoroughly,\u201d Shen said. \u201cCustomers\u2026 ask the same question in many different ways.\u201d<\/span>\u00a0<\/span>
They also ask it from everywhere\u00a0\u2013\u00a0at home, in a noisy coffee shop, in the car with kids in the backseat\u00a0\u2013\u00a0often distracted, stressed, or mid-task. That real-world variability is hard to recreate in desk-based testing, and\u00a0it\u2019s\u00a0where large-scale simulation earns its value.<\/span>\u00a0<\/span>
Puent connected that to a broader shift: expectations change continuously, so the system needs continuous feedback and refinement.\u00a0\u201cWhat customers will tolerate today\u2026 is different than it was\u2026 even six months ago.\u201d<\/span>\u00a0<\/span>
That framing matters because it positions testing and simulation as ongoing operational capability, not pre-launch housekeeping.\u00a0As customer expectations outpace deployment cycles, organizations that build continuous simulation into AI operations will outperform those that treat testing as a one-time gate.\u00a0It\u2019s\u00a0the operational habit that separates trusted AI programs from fragile ones.<\/span>\u00a0<\/span>
So how do teams know when it is \u201csafe\u201d to progress? Puent said it will always be a business decision, but it should be made with evidence.<\/span>\u00a0<\/span>
The key questions are not abstract. They are operational: Have you\u00a0identified\u00a0edge cases? Have you measured outcomes? Are\u00a0customers getting\u00a0stuck? Is customer effort going down, or are you just shifting work elsewhere?<\/span>\u00a0<\/span>
He then highlighted a practical technique inside Amazon Connect\u00a0<\/span>Customer\u00a0<\/span>that supports controlled experimentation: gradual rollout using traffic percentages.<\/span>\u00a0<\/span>
\u201cYou can take one percent of your traffic and point it to a different flow,\u201d Puent said. Start with 1% of live traffic. Measure. If it works, increase to 5%, then 10%, then 50%. If it\u00a0doesn\u2019t? Roll it back instantly. The next inbound interaction never touches the failed experience.
This\u00a0isn\u2019t\u00a0A\/B testing a webpage.\u00a0It\u2019s\u00a0controlled experimentation on live customer conversations with zero-risk rollback\u00a0\u2013\u00a0built natively into the platform.
\u201cEverybody hates it.\u00a0They\u2019re\u00a0getting stuck in loops\u2026 Great. Roll it back,\u201d Puent said. \u201cNext inbound call is not at risk of going down there.\u201d<\/span>\u00a0<\/span>
For enterprise teams, that combination of controlled exposure, measurement, and rollback is what turns AI agent rollout into a repeatable operating model, rather than a\u00a0high-stakes\u00a0bet.<\/span>\u00a0It also transforms each deployment stage from a binary launch into a series of evidence-based trust decisions, building organizational confidence alongside customer confidence, incrementally and measurably.<\/span>\u00a0<\/span>
Shen and Puent also pointed to a broader trend: enterprises are rethinking testing as AI agents become more autonomous and cover more use cases.<\/span>\u00a0<\/span>
\u201cWhile human-in-the-loop is key to building confidence and earning the trust of the business\u2026 it has long-term limitations to scalability as AI agents evolve to handle an increasing number of use cases and scenarios that require frequent updates.\u201d<\/span>\u00a0<\/span>
They argued that moving toward autonomous testing is no longer optional. As AI agents expand, the volume of scenarios requiring validation will easily outpace a human team\u2019s capacity.\u00a0Scaling successfully requires a platform where simulation and continuous validation are native capabilities.\u00a0Otherwise, human-in-the-loop testing becomes a bottleneck that slows every new use case.<\/span>\u00a0<\/span>
\u201cIntegrating testing into both the build and deployment processes is key to successful deployment.\u201d
Once systems are\u00a0live, observability becomes the engine for continuous improvement. Puent emphasized that backend data is what allows teams to make decisions based on reality instead of assumptions.<\/span>\u00a0<\/span>
\u201cHaving data on the backend allows for data-driven decisions, enabling current interactions to inform future experiences,\u201d he said. \u201cCustomer expectations, language, and needs will continue to evolve. You need data to keep your finger on the pulse\u2026\u201d<\/span>\u00a0<\/span>
This is where AI agents become either a strategic advantage or a reputational risk. Without observability, leaders can only react after damage is done. With it, teams can detect failure patterns, refine knowledge, tune flows, and prove readiness for the next stage of rollout.<\/span>\u00a0<\/span>
Observability is not a\u00a0feature;\u00a0it is the operating requirement for deploying AI agents at scale with confidence. Without it, trust in your AI program is a guess. With it, trust becomes a metric you can manage, improve, and prove to every stakeholder who needs to sign off on the next expansion.<\/span>\u00a0<\/span>
When asked what leaders should do before approving an AI agent for production, both guests focused on momentum paired with rigor.<\/span>\u00a0<\/span>
Puent\u2019s advice was about starting and iterating:<\/span>\u00a0<\/span>
\u201cStart\u00a0now, action wins every time, start small, iterate, learn, improve, and keep driving forward.\u201d
Shen\u2019s advice was about grounding validation in real customer data and treating testing as continuous:<\/span>\u00a0<\/span>
\u201cTest thoroughly using real data, not assumptions.\u00a0Identify\u00a0possible scenarios\u00a0from your actual data and create tests based on that. Testing\u00a0isn\u2019t\u00a0a one-time process; continuously update your tests as you scale and expand your AI agent experiences\u2026\u201d<\/span>\u00a0<\/span>
Together, the message is clear: move fast, but only within a framework designed to protect trust.<\/span>\u00a0<\/span>
AI agents are quickly becoming the front door to the enterprise, and that changes what \u201cready\u201d means. In customer experience, a failure is not just a bug. It is a moment the customer remembers.\u00a043<\/span>% of customers are now willing to switch providers after a single poor service experience, a number that has climbed every year for three consecutive years<\/span>\u00a0<\/span>\u00a0<\/span>When tied to Forrester\u2019s data showing CX improvements can drive over $1B in revenue, it becomes clear that a single bad AI handoff isn\u2019t just a technical glitch; it\u2019s a direct hit to\u00a0<\/span>profitability<\/span>.<\/span>\u00a0<\/span>
Shen and Puent are not arguing for hesitation. They are arguing for readiness: phased rollouts, large-scale simulation, testing that keeps up with change, and observability that enables improvement.<\/span>\u00a0<\/span>
\u00a0<\/span>If\u00a0you\u2019re\u00a0a CX leader reading this and nodding \u2014 but wondering how to get your C-Suite to fund the discipline \u2014\u00a0here\u2019s\u00a0the framing that works: the cost of not proving AI agents before deployment\u00a0isn\u2019t\u00a0a technical debt.\u00a0It\u2019s\u00a0a trust debt that compounds quarterly. Every unvalidated interaction is a retention risk. Every retention risk has a dollar value. And the cost of winning back a customer who left because your AI failed them is 5-25x what it would have cost to keep them. This\u00a0isn\u2019t\u00a0a technology investment.\u00a0It\u2019s\u00a0a retention insurance policy with measurable ROI.<\/span>\u00a0<\/span>
AI & Automation in CX
Anthropic\u2019s \u201cMythos-Class\u201d Release Plans Should Put CX Leaders on Alert as Security Risks Accelerate
AI & Automation in CX
Kustomer Pushes Outcome-Driven AI as CX Leaders Demand Proof
AI & Automation in CX
Keep Your Contact Center AI Stack Flexible Without Vendor Lock In
AI & Automation in CX
Zoom Is Turning CX Into A Revenue Engine, Should Legacy CCaaS Be Worried
AI & Automation in CX
Supporting Human CX Agents In An AI Era
AI & Automation in CX
Salesforce Panel Highlights Cautious Public Sector AI Adoption in Frontline Use Cases
Share This Post
AI & Automation in CX
Anthropic\u2019s \u201cMythos-Class\u201d Release Plans Should Put CX Leaders on Alert as Security Risks Accelerate
AI & Automation in CX
Kustomer Pushes Outcome-Driven AI as CX Leaders Demand Proof
AI & Automation in CX
Keep Your Contact Center AI Stack Flexible Without Vendor Lock In
Get our Free Weekly Newsletter<\/span>, straight to your inbox!
Handpicked News, Reviews and Insights delivered to you every week.
Tech
Industries
Trending Topics
Featured Brands
About
More
All content © Today Digital<\/a> 2026<\/p>\n

source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Editorial TypeTech CategoriesHot TopicsBusiness PrioritiesIndustry VerticalsConnectMore from CX TodayHome \u2192 AI & Automation in CXAI agents can transform customer experience \u2014 but without rigorous testing, simulation, and observability, they risk undermining the trust enterprises depend onPublished: May 26, 2026Rob WilkinsonThe push to move AI agents from pilot to production is creating a new kind of […]<\/p>\n","protected":false},"author":1,"featured_media":19281,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11],"tags":[],"class_list":["post-19280","post","type-post","status-publish","format-standard","has-post-thumbnail","category-technology"],"_links":{"self":[{"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/posts\/19280","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/comments?post=19280"}],"version-history":[{"count":0,"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/posts\/19280\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/media\/19281"}],"wp:attachment":[{"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/media?parent=19280"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/categories?post=19280"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/globalnewstoday.uk\/index.php\/wp-json\/wp\/v2\/tags?post=19280"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}