(happy keyboard music) (audience applauding) (orchestra music) - Hmm.
(audience giggling) (lips sucking) Welcome class, my name is Michel-- (orchestra music) Okay, let's try that again.
My name is-- (orchestra music) Ugh, (inhales deeply) God damn it.
What, ugh, damn it! Ah, my grandfather's monocle (inhales deeply), oh well, all right.
Ugh, stop it with the pleasantries.
My name is Michel Boudreau.
I'm the co-founder of the Pact Foundation.
We're a contract testing framework.
Come talk to me afterwards if you wanna know a little bit more about that. I'm here to talk about the anthropology of testing. Now, testing is synonymous with software.
So much so, that you don't really think about it. There's like, who here knows of a testing theorist? Exactly, there are dozens of us, literally dozens, but we don't spend a lot of time actually thinking about it, but, I'll ask you the question, what is testing for? Anyone, anyone? - [Audience Member] Positives.
- All right, that's one.
The other one is finding bugs, like this one.
This is actually a card that was written by Admiral Grace Hopper, in the navy.
I think it was like 1959 or something of that, or something along those lines, maybe there's a date on there.
No need to be a date.
Anyways, she found a literal bug, a moth in the tape. And, she actually said first actual case of a bug being found.
Which, I find so ironic, that first of all, she found a literal bug and two, her name is Grace Hopper, grasshopper, the irony? (audience laughing) So, (chuckles) where does the name bug come from? You think it was a software term, but no, it's actually not.
It actually comes from bogey.
So, in ancient, archaic ways of saying bogey, a source of fear, perplexity or harassment of unknown origin, sounds familiar? It became bug and then there was also bugbear which is like a monster and Shakespeare was actually using the word bug in his time, in his writing.
So, why am I here? Well, there was a stack overflow survey that came out not too long ago.
63,000 people responded to this question of, do you employ unit tests? And, to my surprise, wha'?! A third, more than a third is not using test. Okay, look to your left, look to your right, find that who and find out who that is and send 'em to me for reeducation.
(audience laughing) That to me, that's insane.
That should be 100%.
It's like code coverage, right? We wanna get closer to the 100%.
Why are they not doing it? Carelessness, apathy, they just don't care? They just, I dunno...
But, one thing that the survey did bring up is that yeah, 62.3% are actually happier, much happier.
So, maybe those 37.7% is just miserable folks, you know? Like, people that you see on Twitter.
But, anyways, point is you should be testing 'cause it just makes you happier, hooray! All right, so, where did it come from? What is the genesis? What's the thing that created testing? Now, testing essentially synonymous or another word for quality assurance and that was a thing that was invented a very long time ago. How long, well, it's hard to find texts by then, but a lot of people are thinking around the second millennia that's the 1000s, right.
And, of course, like a lot of stuff with human culture, it revolved a lot around food.
Did I just hear a collective stomach growling like, right there? Everybody's hungry (gurgles).
So, at mediaeval markets, you didn't know exactly what you were buying, right. You had to trust the person behind the stall. So, how would you know, I mean most of the time you come from a small village. You know the person behind the stall.
But, as places got bigger, you actually don't know the person behind the stall. You don't know what they're selling you, if it's of good quality, if they use human excrement to fertilise their crop and that goes into your food and then, you know, whatever.
That went a weird way.
But, actually, what I wanna talk to you about, specifically is bread.
Probably the most important thing back then. Now, at the time, to raise profit, they would cut their flour with sawdust or lime, like powdered lime, not like the fruit, you know, different.
Sawdust, like, it doesn't hurt people.
But, super cheap because everybody was cutting down trees. They're makin' logs and stuff like that.
Nobody had really use for sawdust, so much as their kindling starting up fire and stuff like that.
So, they would actually put it in their flour 'cause you have to understand, flour back then was precise.
You had to get the wheat.
You have to bring it to a miller, an actual windmill.
You had to stand in queue with all the other bakers. You actually put your wheat in.
It'd take hours, sometimes, days to get your flour at the end.
So, they put sawdust in it to enhance the volume, but then there's less weight in your bread. Doesn't hurt the populace, but does increase profits.
So, what was the fix for that? Well, a lot of these craftsmen got together and like, well, this is kinda hurting us.
For people that wanna have a better quality product, they need to know where to go and since we don't know everybody and know where to go, we need to create a standard.
So, how would you create that standard? They got together.
They informed a guild, sound familiar? They formed a guild and this is not just around bakers, this was all other crafts, as well.
We're talkin' blacksmiths and anything, leather workers, I don't know how that's called, cobblers.
But, this one specifically around bread, they had to have a mark to say that you pass the standard.
You were minimum trying to do this, right? So, there's always a minimum.
So, if you had that badge, your product was good, was okay. If you held the highest of standards, you got a special little badge, you know, it's gold! And, it's so funny too, like, this thing is called a laurel wreath.
You see that pretty much anywhere.
Like, you'll see it on movies and stuff like that. You can see it on bottled wine, you're like, oh, this has gotta be good.
But, really it's just, you don't know what it actually says. You don't care.
You just see the laurel wreath, ah, that's gotta be good. I'm gonna buy it.
Anyways, so, that is actually kind of where the badge kinda started and you could go to a market and if you saw that badge, you knew it's gonna be a good quality.
Of course. people tried to replicate the badge and blah, blah, blah.
They would get a nice beating from the guild. That's another thing altogether.
But, anyways, moving on to testing itself.
Testing levels, so, how many levels, right? Currently, I'd say there's about five, right. So, we'll start with the simplest one.
Sorry, before I start into this.
Levels are almost like looking into a microscope. Right, you start off looking really, really close to like the atom level and then you go up, you go to molecule and then you go up and up and up because you go further away from the actual code. So, the first level, I say, is manual.
Now, I don't like manual.
Manual is good for one thing.
There's very little barrier of entry.
You can just start doing it.
You can do it right now.
But, it doesn't scale.
It doesn't work.
And, if you wanna scale, you need to hire more people and people suck.
Like, they're meat bags.
They need water and food.
They cost money.
From time to time, they need to see light.
You can't just keep 'em in their dungeon.
(audience chuckling) Like, it's annoying.
So, I'm just gonna throw that one the hell away. We can't automate that, not reliably at least. I mean, maybe, in the future, we'll just have a bunch of clones or something of that, or, whatever, we'll see. Enough about that, but anyways, first one, the first real one is unit test.
Of course, I would imagine everybody knows unit testing, but then again, a third of you don't use tests so, I don't know.
But, I have a bit of a joke here for trying to explain what unit test is, right. So, you have a software engineer.
He builds a bar.
He walks to the bar.
Goes to the bartender and orders one beer.
He orders zero beers.
He orders minus one beer.
He orders 99,999 beers.
He orders one lizard.
He orders a asdkjsdofi.
(audience laughing) What's that developer doing, or engineer doing? - [Audience Member] Fuzz testing? - There's fuzzy testing, anybody else? - [Audience Member] Exhaust testing? - Yeah, essentially, he's just trying inputs. Trying inputs and see what the output is.
If I order one beer, I then expect one beer to come back. If I order zero beers, no beers coming back. Minus one beer, I don't know, bartender throws you an error or something or throws you out of the bar (exhales deeply). Which is kinda funny because the end of that joke goes and then the first real customer goes into the bar, asks for the bathroom and the bar suddenly explodes.
(Michel and audience laughing) Ah, so much fun.
Okay, so, we just did our unit test.
What's the next step after that? Integration, now, most people think of integration like, end-to-end test.
That's actually not at this level.
Integration is essentially, oh, you've created a button or something else and it needs to work with another component within your system, right.
Another button or a page or that you click that button and it's supposed to submit that form that then does something else.
That's actually the integration level, right. But, we get confused with that when we're talking about our integration test. So, you can do unit test across modules and stuff like that. There's also a bunch of different ways of doing test. But, anyways, so this integration level.
Next one is the system level.
This is where end-to-end test normally is, right, because you're doing like a full stack.
You're doing, oh, if I click this button, I submit this form and sending it to the servers, servers suppose to respond this particular way and I'm suppose to see this thing in front of me. So, that's at that level, right.
And then, right at the top is behavioural.
So, at this point, you're not really testing in input and output. You're testing behaviour.
You're testing state.
If I click this form, I'm expecting this and this when I come back to the main page the thing that I just submitted that I just saved, is now showing up on my main page which wasn't there before.
That's a behaviour, all right.
Now, there's a thing, an ongoing debate with testing theory of if you're doing a test at this level, do you need to do the test at this level? Because in a way, it is testing it, right? Or, at least, you should be testing it.
What do you guys think? (snickers) So, some people aren't sure, like, on the fence, you're just like, I don't know, and the thing is, unit test is a lot faster normally than behavioural test.
To me, I find it really fascinating when you start talking about APIs, right.
APIs is literally input in, output out.
You don't know how many functions get called underwards, it's a black box.
I mean, you can do the unit testing and integration testing and stuff like that within an API, but if you're testing at the high level of the HDP response and stuff like that, do you really need to test everything else? And, APIs are pretty darn quick and at the API level they're actually pretty darn quick, you know. So, this is actually where Pact is actually useful, shameless plug (snickers).
So, Pact actually does API level testing from a literal input and output.
We don't really care about anything else.
It's a black box.
If you do that, you really need to do unit testing, okay.
I'll leave you that to your team to debate. Anyways, so, what does testing the future mean? What's the, I don't know, have you noticed something with this particular circle, right? The further out you go from the middle, the more complexity is involved.
The more state is involved.
The more actors, the more variables, the more, everything can go wrong.
Which is why, often times, behavioural test or accepting test or, whatever you want to call it, are extremely flakey 'cause sometimes the behaviour just changes because somebody changed something at the unit level but then affects the higher level, right.
But, more complexity is not particularly bad depending on what you're trying to do.
But, I have a question for you.
What's the next step after behavioural? Right, nobody knows.
(audience member murmurs) - What's that? - [Audience Member] You can do manual testing. - Oh, that's the next slide, shut up.
(audience laughing) (chuckles) So, it's kinda funny because when you're doing a behaviour, you're only doing a behaviour on a single user, I'm clicking here, I'm doing this, I'm doing that, what if we were suppose to do collective behaviour? What if it was cultural behaviour? What if it was a predictive behaviour? If I'm searching through Google, I'm looking for, I don't know, Michel is speaker at Web Directions, does it bring the much more attractive female Michel or does it bring my mug to the screen, right. Which one is the correct one? And, this is where stuff like machine learning kinda comes involved, right. So, machine learning test it's interesting. 'Cause machine learning is essentially a way of having a black box, kinda like how I was saying before with the API. Your unit testing integration test or just you throw it away.
You don't need it.
That's not at that level.
'Cause a unit test would be like say, all the talks we had yesterday with the AI and everything else on the activation function, but that's not how it works.
You train it.
So, that's essentially the unit test and integration test. It's the training, right.
Afterwards, this is where you are around the system level 'cause you're expecting an input and an output comes out. Is it right or wrong, whatever you train it properly, you get close enough.
But, with the behavioural, how do you do the behavioural for machine learning? For predictive, right, because if you've trained it, do you just train it and that's your model, that's it, you're never gonna touch it every again? Because if that was the case, Google wouldn't be as good a search as it is now. It's being trained by people currently doing searches and clicking on a particular result.
Doesn't mean that's first result you're clicking on for you sir, madam, whoever.
Specifically, they're gonna know your model, but over the course of time.
If I've been with Google for 10 years and me keep searching for 10 years, my results are very different from the person next to me. But, how do you test that? Can you input 10 years, can you fake 10 years of data? Anybody ever tried to fake real data, like data looks actually real? It's a massive pain in the ass.
It hurts, anyways.
So, all this to say, for behavioural, does it really come down that we're going back to manual? Thanks search, you've ruined my slide.
Because that is what's happening.
You're going back to manual.
You're going back to humans.
But you're doing it at a scale in production. There's no testing.
Every time you release something, it's like, err, hope this works.
I would like to get back to the point where I'm confident in doing something around machine learning, predictive algorithms and stuff like that.
To be able to say, I know this is not gonna break, but you have no idea.
And, you know, for manual system, perfect example, Google reCAPTCHA, you know, like you're filling in a form and this says, I'm not a robot and then little grid of images come back and say, point out all the cars.
That's Google learning to recognise cars, right. So, you spread out your input test to actual humans that need to validate your data. But, really, I don't know what's the next thing for machine learning.
I wanna be able to test that out locally.
I don't wanna just release something and just say, let's see if it works.
Anyways, that's just me.
I want that confidence.
Which comes back to my last point here, why test? Everybody has a different reason.
Some people hate test, hate it.
And speaking of which, you can ask me over lunch, which I can still hear some gurgles, you can ask me at lunch what my opinion of TDD is or BDD. But, anyways, so some people just hate test. They think it's a time sinker.
Your business probably thinks it's time sinker. What you're not adding features, and I was like, yeah you're right, we're not. We're just testing to make sure that they don't screw up in production.
So, why do I test personally? For me, it's freedom and creativity.
Like, most people don't associate testing with freedom and creativity.
But allows me to be free and creative.
It allows me to experiment with knowing that I have almost a 100% test coverage and if I'm changing stuff, I'm not gonna break the world for the people using my software and that's the reason why I do it, right.
So, I'll leave you with this because I don't think I have a lot of time left. I'll leave you with this, testing shows the presence, not the absence of bugs. Whoa, dude (sucking air) (exhales deeply) Now, here's a guess.
Who can guess what year somebody said this? - [Audience Member] 1962.
- Anybody else? - [Audience Member] '72.
- [Audience Member] 1981.
- Ugh, God damn you sir, why do you have to be the first one? It's actually not '67, it's '69.
♪ The summer of '69 ♪ I'm probably gonna butcher this, if anybody's from the Netherlands and stuff like that, Edsger W Dijkstra, anyways, he said that in October of 1969 at a NATO conference.
This guy was freakin' generations above the rest. He was thinking about testing theory.
I actually consider him the grandfather of testing theory. He didn't invent test.
He just kinda thought about it correctly and I definitely recommend that you read this book, "The Humble Programmer". It actually came out in 1971 and it's still relevant today. The guy was so around developer experiences which I'm a big passionate about.
So, it hits all of those points, barriers of entries and stuff like that, definitely recommend it. (sighs) So, I hope this talk has at least inspired you to add more test or to just add test, you know, for in case I come over to reeducate you, you know.
So, please do so and thank you for listening to me. My name is Michel Boudreau. (audience applauding)
(happy keyboard music)
