Testing has come a long way from the days of mechanical computers, which has evolved side by side with our coding practices and culture. Before we can learn where we are and where we should be going, let’s glance backwards to where it all began.
Testing is synonymous with software, but we don’t spend a lot of time actually thinking about.
The term bug comes from bogey, a source of fear, perplexity or harassment, of unknown origin. Although it was popularised in software when Grace Hopper found a literal bug (moth at least) in a relay.
37.7% of devs aren’t using tests. That’s insane! Why aren’t they doing it? Carelessness? Apathy? We don’t know! But we also know the who do use tests are happier!
Testing is another name for Quality Assurance, which is a very old term that dates back at least to the 1000s. At medieval markets, you had to trust the people who were selling things that the wares were good. As markets grew, you could no longer rely on knowing the person.
Many bakers were cutting flour with sawdust to make bread more cheaply, because it was precious – it was expensive. This kept the volume of the bread, but lowered the weight. Those who didn’t cut the flour wanted a way to show they created a better quality product. Bakers created a guild, standards, and a standard mark to advertise the fact their bread was good. People who faked the badge were Dealt With…
- manual – this is only really good because there’s no barrier to entry, but it doesn’t scale. You can really throw this one out.
- unit tests – (classic joke about ordering 1 beer, 0 beers, -1 beers, 9999 beers, 1 lizard…) trying different inputs to see if a unit of code produces the right output
- integration tests – do units work together
- system/e2e tests – everything from user input, through app, data and network layers etc
- behavioural – this is testing state, if a certain sequence of events occurs, do you see the expected result in the state
What comes next? Well we don’t really know… but maybe…
Machine learning tests – can you fake data? can you predict what is “correct” for an ML system by faking ten years of activity? Or do we just fall back to manual? We actually do that a lot with CAPTCHAs – Google is learning how to detect cars at the moment.
Some people hate tests, or they hate a particular kind of test, or they just want features and don’t want to spend time on tests.
Testing = Freedom & Creativity
You can try a lot more interesting things when you know what you’re breaking.
Testing shows the presence, not the absence of bugs. – Edsger W Dijkstra, 1969
Dijkstra released The Humble Programmer in 1971, and it holds up today as it set down so many ideas about testing theory.