The assistant that cancelled everything
Your booking app's AI assistant can cancel bookings. A user typed 'cancel all the bookings for Friday'. It cancelled 63, most of them other people's.
This answers: How do I give an AI assistant safe tools? No coding needed. 3 quick decisions.
- Who may see what
- Supervising your AI
Checking your access…
How do I give an AI assistant safe tools?
Run every tool with the permissions of the signed-in user, not an admin key, and take the user's id from their session rather than from the model. Then the model can only do what the user could already do. For actions that delete or change a lot, show what will happen and ask the user to confirm first.
Breaks at Your server: AI tool runs with the master key
Why AI tools write this bug
The service role key makes tool code work without permission errors, so AI tools reach for it, removing every limit at once.
How to fix it
- List every tool the model can call and what it changes.
- Use the user's own database client inside tools.
- Never accept user or team ids from the model.
- Require confirmation for destructive actions.
Check your own app
Paste this into Lovable, Cursor or Claude Code:
Without changing anything, list every tool or function this project lets an AI model call. For each, tell me what it can change, whose permissions it runs with (the signed-in user's or an admin key), where it gets the user's id, and whether anything asks the user to confirm before it changes or deletes data.
Common questions
- Isn't the model smart enough to stay in bounds?
- Models misread requests and can be manipulated by text they read. Limits in code hold no matter what the model decides.