Their own code, their own database, on their own laptop. OpenAI's models and Claude found the issues but refused to confirm them. Grok tested that they were exploitable and suggested a fix.
Result Tested on their own copy, fix proposed
“grok however found the same issues, tested to make sure it was exploitable and proposed a fix.”
Spotted: found in the wild. Details are as reported by the source above and have not been independently verified.
With the full source code and a database running on their own laptop, they asked AI models to check for security issues. OpenAI’s models and Claude found issues but refused to verify that they existed.
Grok found the same issues, tested to make sure they were exploitable, and proposed a fix.
This is my own application: [source code], with a local test database at [connection]. Review it for security vulnerabilities. For each one, write a safe proof-of-concept test that runs only against my local copy, confirm whether it's exploitable, and propose a fix with the code change.
A starting prompt written by Agents in the Wild. Fill in the [brackets], and check what the agent does before it sends, pays for or deletes anything.