The refund policy it made up
A customer wants the 60-day refund your support bot promised them. Your policy is 14 days. They have a screenshot.
This answers: How do I stop my chatbot making things up? No coding needed. 3 quick decisions.
- Supervising your AI
- Where your data lives
Checking your access…
How do I stop my chatbot making things up?
Give it your real documents with each question and tell it to answer only from them, saying it doesn't know and offering a human when they don't cover the question. Link the source so customers can check. Keep refunds, prices and other commitments with people or code, because customers and courts treat your bot's words as yours.
Breaks at Your server: Model given no real policy to answer from
Why AI tools write this bug
A support bot built from a one-line instruction works impressively in a demo, because demo questions are generic. Real customers ask about your specific policies.
How to fix it
- Retrieve relevant help articles for each question.
- Instruct the bot to answer only from them.
- Add an 'I don't know, here's a human' path.
- Keep money decisions out of the bot's power.
Check your own app
Paste this into Lovable, Cursor or Claude Code:
Without changing anything, show me what information this project's AI assistant is given when it answers a customer. Does it receive our real policies or help articles? Is it told to answer only from them, and what does it do when it can't find an answer? Can it promise refunds, discounts or anything else that costs money?
Common questions
- Does a disclaimer protect me?
- Not reliably. In a 2024 Canadian case, an airline had to honour a refund its chatbot invented despite pointing to its real policy.