AI use case
Use your own git history to check if a coding agent's fix really works
A developer noticed coding agents always say the bug is fixed, so they built a free tool that turns past bug fixes in your repo, the ones that came with tests, into a test set. On their first real run, 27 of 66 patches the agents called done were wrong. It's a way to measure agents on your own code instead of public benchmarks.
- Custom agent
- Coding
- Testing
- Free tool
Source: DEV ·