Derek sat in the MidStage war room at 9 PM, staring at the Slack thread exploding with "the AI just told a customer our refund policy allows 90 days when it's clearly 30." As the PM owning their new Knowledge Assistant, he'd pushed hard for RAG. Their docs were scattered across Notion, Google Drive, old Jira tickets, and Slack archives.
Launch day was promising. Queries about "how do we set up SSO?" pulled the exact Confluence page and spat out accurate steps. Users loved the citations. Then reality hit. The system confidently explained a pricing tier that changed last quarter because retrieval grabbed an old announcement PDF.
"Grounded" doesn't mean reliable. RAG sits smack in the middle of the Trinity — Data that's always messier than you admit, Models that still hallucinate on top of "perfect" retrieval, and UX that can either build trust or create the most dangerous kind of false confidence.