AI-assisted development needs automated tests
May 30, 2025
I wonder if one of the reasons I’m finding LLMs so much more useful for coding than a lot of people that I see in online discussions is that effectively all of the code I work on has automated tests.I’ve been trying to stay true to the idea of a Perfect Commit – one that bundles the implementation, tests and documentation in a single unit – for over five years now. As a result almost every piece of (non vibe-coding) code I work on has pretty comprehensive test coverage.This massively derisks my use of LLMs. If an LLM writes weird, convoluted code that solves my problem I can prove that it works with tests – and then have it refactor the code until it looks good to me, keeping the tests green the whole time.
It sometimes feels that those critical of using LLMs for software development, or skeptical about their value have limited real use of them. Perhaps gave them a shot, weren’t impressed with what they produced, and left it at that.
This from Simon Willison resonates with me–you need to work with these systems, learn their strengths and weaknesses and develop your own workflows and frameworks to get the best from them.
I remember infuriating days in early 2023 going round and round the mulberry bush (they sometimes still happen, but far less so now than a couple of years ago) as a model would confidently produce something that didn’t work, correct it, that didn’t work, correct that, again not working, then go back to the original suggestion.
There were plenty of times when I know I could have written the code more quickly, lamented I hadn’t just done it myself.
But that process helped me get far better results in the long term.
So learn from folks like Simon who have put in a huge-amount of effort, and who generously share what they have learned.
