Chapter Three

Conversational AI:
Chat, Copilots,
and the Trust Trap

Conversational AI feels like magic because it meets people where they already live — natural language. That same familiarity is the trap. Users project human-level competence onto really good pattern matching.

📖 ~12 min readPages 21–26
scroll

This chapter is for the PM who's been asked to "add a chatbot" or "build us a copilot" and now has to decide: is chat the right interface, or just the easiest one to ship? We'll cover where it works, where it dies quietly, and when chat is a genuine product feature versus a lazy wrapper around search.

Where Chat Works

Conversational interfaces shine on triage and scale — simple, repetitive, high-volume stuff.

Vodafone TOBi
70%
first-contact resolution. 1M+ interactions/month.

Vodafone UK's TOBi handles over a million interactions monthly with 70% first-contact resolution. Cost per query drops from £10+ to pennies. Eye-oo resolved 82% of support issues automatically with €177K extra revenue from cart recovery. Monday.com embedded Kapa.ai in docs — 10x engagement, 30 minutes saved per query, zero extra headcount.

Where Chat Fails Silently

The scary part: confident wrongness that users don't catch until it hurts.

Air Canada ruling"The chatbot is a tool of the airline — it is responsible for its representations." Your bot speaks for the company.

Air Canada, 2022. A grieving customer asked the chatbot about bereavement fares. The bot invented a policy that didn't exist. He booked, got denied, sued, won. Tribunal ruled: "The chatbot is a tool of the airline — it is responsible for its representations."

McDonald's Olivia hiring bot (2025): applicants reported loops, ghosting, bizarre questions. A Reddit thread went viral. Classic case of shipping without human-in-the-loop monitoring.

Brendan Falk's data
74%
of agent deployments remain heavily constrained.
OBVIOUS SUBTLE SILENT Error messageUser flags it Wrong butplausible Confident &wrong Authoritative& never flagged DANGER →
Figure 3.1 — Failure Modes Spectrum. The worst failures are on the right — ones users don't flag because the answer sounded authoritative.
Chapter 3

Unlock the full chapter

The first two chapters are free. Chapters 3 through 18 unlock with a one-time purchase on the same account.

$18.99one-time purchase via PayPal

Already purchased? Sign in with the same account you used at checkout.