Is your AI agent actually ready? 11 questions to ask first

A checklist for readiness

In 2019, we built our first chatbot.

It was manual, rigid, and could only demo a small Income & Expenditure use case. But it taught us more than any polished deployment since.

Six years and a lot of hard lessons later, I can tell you this with confidence: the pace of change in Customer-facing AI right now is unlike anything I've seen in this industry. We're not talking about smarter chatbots. We're talking about Agents that reason, negotiate, act and escalate - across voice and digital, in real time, inside some of the most consequential conversations in financial services.

Collections and Recoveries. Vulnerability. Consumer Duty. Fair outcomes.

This is not the space to move fast and fix it later.

The technology is no longer the barrier. The design is.

At arum, we've been working across the full AI landscape - from Automation and Decision AI through to Conversational AI and GenAI - since before it was fashionable. And the question we ask every client is the same:

• Is your AI Agent actually ready?
• 11 questions worth asking before you go live
• 1.Does it stream? Voice AI that waits to process feels broken to a customer. Real-time streaming isn't a nice-to-have.
• 2.Where does context live - and who controls it? Is it hardcoded? Manageable within your platform? Or sitting at your LLM provider - OpenAI, Anthropic, Google? How it's designed determines your flexibility, your risk exposure, and your data sovereignty. Most teams haven't thought this through.
• 3.What data can it access - and what can't it? Scope and boundaries matter as much as capability.
• 4.Can it explain every decision? In regulated environments, "the model decided" is not a compliant answer. Explainability is a regulatory expectation, not an engineering nice-to-have.
• 5.Are your guardrails built in or bolted on? There's a significant difference. One protects you. The other gives you false confidence.
• 6.Is it performant under real conditions? Under load, across channels, with customers who don't behave like your test cases.
• 7.Does it recognise vulnerability signals? In the UK, Consumer Duty makes this non-negotiable. But this isn't a UK-only issue. Across every market, regulators and consumers alike are raising the bar on how organisations identify and respond to vulnerability. If your agent can't detect distress, confusion or financial difficulty in real time - and adapt accordingly - you're exposed regardless of where you operate.
• 8.When - and how - does it hand to a colleague? The handoff moment defines the customer experience more than almost anything else. AI Agents are an extension of your workforce, not a separate channel. Any transition should feel colleague-to-colleague - warm, informed and unbroken. The moment your customer senses a hard boundary between "the bot" and "a real person" is the moment you lose their trust. Design for continuity, not handoff.
• 9.Does it surface what's going wrong? Exceptions, frustrations and failures should be feeding your improvement loop. If they're disappearing, so is your insight.
• 10.Does your customer know what's happening and why? Transparency builds trust. In this space, trust is everything.
• 11.What tools does your agent actually have, and how were they built? Can it search a customer record in real time? Complete an Income & Expenditure assessment? Offer, negotiate and confirm a repayment arrangement? Apply forbearance, and does it know when it should? How those tools were developed matters as much as what they do. Are they robust API integrations or fragile workarounds? Who governs what the agent can and can't action? Is there a full audit trail of every tool called and every decision made? In Collections and Recoveries, an agent that can hold a great conversation but can't actually do anything in your system is just a very expensive IVR.
• I'm curious - for those of you currently building or evaluating AI Agents in financial services: which of these is causing the most friction internally? I'd genuinely like to know.
• We've helped clients across Collections, Recoveries and Customer Operations get this right.
• It's not quick, but it's also not as complicated as some vendors will have you believe.
• Book a free 30-minute AI Readiness call with one of our specialists. No pitch. No agenda. Just an honest conversation about where you are and what good looks like from here.