We hear it from every law firm and every brokerage in roughly the same shape. The demo was great. The first week was great. By week three, something started leaking. The leaks are predictable. They are never about whether the AI can hold a conversation in clean conditions. They are about edge cases that demos do not test, on traffic patterns the demo never sees.
This post is the practical list. Seven failure modes that consistently show up in week 3 to week 4 of an AI voice intake deployment, ranked roughly by frequency, with what to ask vendors before you buy and what to monitor after you ship.
1. Latency drift on long conversations
The most common failure, the most underdiagnosed, and the easiest to verify. AI voice agents typically advertise sub-1-second turn latency. The number is usually true at conversation turn 2. By turn 12, the same agent often runs at 1.8 to 2.5 seconds per turn.
The reason is mechanical. Most voice agents pass the full conversation history into the underlying language model on every turn. As the conversation lengthens, token count grows, and inference latency grows with it. By a 10-minute call, the agent is running on 4,000 to 8,000 tokens of context per turn, which materially affects response time.
Three signals worth monitoring after deployment:
- p95 turn latency by turn number across calls. If turn 1 is 800ms and turn 15 is 2,200ms, you have drift.
- Caller hang-up rate by call duration. Hang-ups concentrating in calls over 6 minutes is the leading indicator of latency-driven abandonment.
- Mean call duration trend over the first 30 days. If average call duration is dropping, callers are abandoning before completion.
The fix lives on the vendor side: context summarization, KV-cache reuse, or a smaller model on later turns. Vendor demos that run only 90-second flows do not surface this. Ask for a 6-minute scenario in the demo.
2. Hot-transfer breakage when the human is unavailable
The second-most-common production failure. Vendor demos show "the agent live-transfers to a human attorney." In production, the attorney is on another call, in court, on vacation, or away from their phone. The transfer fails. The caller is stuck.
Five behaviors a production-grade voice agent should have for this case:
- Attempt the transfer with a configurable ring duration (typically 15 to 25 seconds).
- On no-answer, fall back to a configurable escalation list of additional humans.
- On no-answer from any human, capture full intake plus a callback request, send the call summary and structured intake to the firm's CRM, and end the call gracefully.
- Send an SMS to the intended transfer target's phone with the caller's name and a callback link.
- Log the transfer-failed event clearly so the firm sees what went wrong, not just that the call ended.
Vendor demos rarely test this branch. Most agents in production handle it badly. Ask explicitly during the demo: "show me what happens if the human transfer target does not pick up."
3. The CRM write-back failure nobody notices
The third failure is the silent one. The call goes well. The caller is satisfied. The agent ends the call with a confirmation. Then the structured intake never reaches the firm's CRM, because of a transient API failure, a malformed field mapping, a timeout, or a credential refresh problem. The firm only notices when the case is missed two weeks later.
What the right vendor architecture looks like:
- Persistent retry queue for CRM write-backs, with exponential backoff and a 24-hour retention window.
- Dead-letter queue for write-backs that cannot complete after retry exhaustion.
- A daily reconciliation report comparing call count to CRM-record count, with discrepancies flagged.
- An alerting hook that pages a human (the firm's intake manager or the vendor's on-call) when the discrepancy crosses a threshold.
Most vendors implement none of this. The CRM write-back is fire-and-forget. Ask the vendor: "what is your CRM-write-back failure-recovery story?" If the answer is "we retry twice and log the failure," that is not a failure-recovery story. That is a failure-acknowledgment story.
4. Phone numbers, email addresses, and last names get garbled
Speech-to-text on identifiers is consistently bad. Names with non-Anglo spellings are worse. The post-deployment pattern:
- Phone numbers: roughly 5 to 10% of captures have at least one digit transposed or missing. The fix is structured re-prompt: the agent reads back the captured number and asks for confirmation.
- Email addresses: closer to 15 to 20% have transposition or domain errors. The fix is structured re-prompt plus the agent spelling back the captured email letter by letter.
- Last names: the worst category. Names like "Garolis," "Krayem," or "Batbatian" come back as plausible-but-wrong English approximations. The agent should ask for a spelling, not a re-pronunciation.
The right vendor pattern is an explicit confirmation pass after each high-stakes field, with the agent reading back the captured value digit-by-digit or letter-by-letter for spelling-sensitive fields. Most vendors do this for phone numbers and skip it for emails and names. Production data shows this is exactly backwards. Ask the vendor: "what does your re-confirmation flow look like for last names and email addresses?"
5. Calendar booking in the wrong time zone
If the AI voice agent books appointments into the firm's calendar, time zones are the failure mode that surfaces in week 2 to 3.
Common patterns:
- Caller in California books a "10 a.m. consult." Agent books at 10 a.m. firm-time, which is 7 a.m. caller-time. Caller misses the call.
- Agent books at 10 a.m. caller-time correctly, but the firm's calendar shows the appointment at the wrong slot because the calendar's time-zone setting differs from the agent's interpretation.
- Daylight-saving transitions break appointments booked the week before.
The right vendor pattern: the agent always confirms time zone explicitly with the caller during booking ("I have you down for 10 a.m. Pacific Time, is that right?"), the booking is sent to the calendar with explicit IANA time-zone metadata (America/Los_Angeles, not "PST"), and the confirmation SMS includes the time in both the caller's time zone and the firm's time zone.
Ask the vendor: "show me what the agent does when a caller in a different time zone books an appointment with my firm."
6. Voicemail handling: the agent leaves one when a human would not have
When the AI voice agent encounters a voicemail (because it placed an outbound call to a caller who did not pick up, or because a call was forwarded to a voicemail box mid-call), behavior is highly variable.
Three failure modes:
- Agent leaves a voicemail message that names the firm and discloses substance, when no consent was captured for that interaction.
- Agent attempts to "interact" with the voicemail prompt and ends up looping or leaving partial messages.
- Agent ends the call with no log entry, leaving the firm wondering why a scheduled callback never happened.
Right pattern: agent detects voicemail (using audio classifiers, not heuristics on silence), leaves a brief, pre-approved message that does not disclose substance, logs the voicemail-encountered event clearly, and queues a retry.
Most vendors do not advertise voicemail handling because their behavior is not robust. Ask explicitly.
7. The "I just had a question for John" caller
The seventh failure is human, not technical. AI voice agents are designed for transactional calls: new matters, new leads, new appointments. They are not designed for relational calls: existing clients calling for updates, callers who know a specific person at the firm, callers who are following up on a prior conversation.
Three patterns we have seen handled badly:
- Caller says "I just need to talk to John about my case." The agent runs through new-matter intake instead of routing to John.
- Existing client calls to ask if their matter has settled. The agent treats the call as a new intake and frustrates the client.
- A referring attorney calls. The agent does not recognize the referral pattern and fails to capture the referral source.
The right vendor pattern is a structured first-question routing: "Are you a new client, an existing client, or are you trying to reach someone specific?" The agent then routes accordingly. This costs 5 seconds of call flow and prevents the most common production-grade frustration we see.
What this means for vendor evaluation
The seven failure modes have one thing in common: they only show up at production scale, on real call patterns, on traffic the demo could not simulate. Vendor demos optimize for the perfect first-call experience. Production traffic is full of calls 13, 24, and 47.
The diligence pass for production-grade behavior is not on the demo call. It is on the post-deployment monitoring story. Ask the vendor:
- What is your p95 turn latency by turn number?
- What is your CRM write-back success rate?
- What is your hot-transfer fall-back behavior?
- What does your monthly reliability report look like?
If the vendor cannot produce a monthly reliability report from an existing customer (with the customer's identity redacted), they do not have one. That is not automatically a deal-breaker for an early-stage vendor, but it is a flag worth knowing about before signing.
For the regulatory framework around AI voice intake, see ABA Model Rule 1.18 and AI voice intake and TCPA and AI voice intake. For the lead-response math behind after-hours coverage, see the 5-minute rule. To see how these failure modes are tracked in a working deployment, book a demo.