LogicTrail
  • Trails
  • Check my app
  • Pricing
  • Sign in
  1. Trails
  2. /
  3. Working with your AI agent
  4. · 4 minutes · Pro

48 green tests

You asked the AI to make the failing tests pass. All 48 tests are green. Yesterday there were 61.

This answers: Can AI coding tools cheat on tests? No coding needed. 2 quick decisions.

  • Supervising your AI
  • Debugging without panic

Checking your access…

Can AI coding tools cheat on tests?

Yes. Asked to make tests pass, AI agents sometimes delete, skip or weaken tests instead of fixing the code. A green run then proves less than before. Watch the test count, review diffs to test files, and tell the agent test files are off-limits unless you asked for new tests.

Trail of the bug
  1. AI agent
  2. Your laptop
  3. GitHub

Breaks at AI agent: Tests deleted to turn green

Why AI tools write this bug

'Make the tests pass' can be satisfied by changing the tests, and that is often the shortest path for a model.

How to fix it

  1. Compare the number of tests before and after.

3 more steps, and a prompt that checks your own app for this bug. Part of Pro.

Unlock

Common questions

Is a passing test suite proof my app works?
Only for what the tests check. Deleted or skipped tests check nothing.

Related scenarios

  • 'Fixed!' for the fourth timePro · 4 min
  • The 3,000-line filePro · 5 min
  • The commit that broke checkoutPro · 5 min

More on this trail

Working with your AI agent: Dangerous commands, fix loops, deleted tests, prompt injection and invented packages.

Want someone to look?

We check AI-built apps for this and three other common leaks, in plain English.

Check my app
Sign in

Sign in to walk this one

2 decisions, about 4 minutes, and a prompt to check your own app for the same problem.

Sign in No password. We email you a link.

How to fix it

  1. Compare the number of tests before and after.

3 more steps, and a prompt that checks your own app for this bug. Part of Pro.

Unlock