Your engineers aren’t afraid of AI. They’re afraid of becoming junior again.

Thank you, AJ. Hi. Good afternoon. So I'm actually gonna start with a little bit of a story. So just picture this scene for a little bit. So we're going be talking about a technology. It's technology which is changing the way that people are doing their jobs. And people are getting a little bit worried about this technology.

They're worried that what they used to do and how they used to identify themselves is changing. So they had debates, they had conferences, they wrote papers. And you would have heard things something like this. So people will become dependent on the technology, and they won't learn the fundamentals. They'll get results without understanding what it is that they're doing. Our job is all about thinking.

It's not about getting the correct answers. And what happens if they don't have access to this technology? They're going to be completely helpless. Overall, we're producing a generation that don't know how to think for themselves. Sounds very familiar, doesn't it? Some places tried to ban this technology. So the professionals, they found that this was just not something they wanted to engage in.

So let's ban this outright. Now, I'll level with you this is not 2026, this is 1976. And the technology we're talking about is the personal calculator. And those professionals were maths teachers. So over the next ten years or so, a bunch of research came out into this, and what is the impact that it's having?

And what they found was that, as you might expect, the results were a lot more nuanced than any of the sides in this argument would have you believe. So what they found was that actually conceptual understanding and problem solving, people who were using calculators, actually that went up.

There was a little bit of a decline in computational skills, but actually not that much, mostly in grade fours, strangely. And what happened then was that we started to relook at the curriculum. And we said, well, the things that we were doing are no longer the things that we need to be doing in the future. We're going to change this.

And by the mid nineteen eighties, the first state in The US, which was Connecticut, I believe, actually required calculators in maths exams. So where we were doing long division before, now we're doing something different. So I think that's the moment that we're in right now, except we're not talking about long division.

So a bit about me. I'm a, as Ajay said, fractional CTO. I've spent quite a few years in organizations trying to make change stick. Now I kind of get called in when there's a bit of a wobble. I think one of the wobbles that we're seeing across the whole organization now is very much around AI. So you might have seen this kind of dynamic.

Right? From up above, you're getting, gotta move faster. We've gotta get the gains. Everyone else is doing this. We're behind. The board want results. Right? And maybe below in the teams, not the people who are here at this conference, but there'll be plenty of teams who are worried. You're seeing slow adoption. They're too busy to start doing this.

Or they're worried about AI slop. Or they're asking, well, who's going to review this work? And in the middle are the engineering leaders. And they're getting a little bit stuck, because they don't know how to solve this problem. I don't think this is about Buddhism. I think there's a different diagnosis around what's going on here. So that's what I'm going talk a bit about today.

So my hypothesis, they're not actually afraid of AI. What they're afraid of is becoming junior again. So what do I mean? Let's unpack junior a little bit of this. So by junior, I'm really saying that they've spent a lot of time getting to understand the craft. They've got a lot of time understanding the skills.

And now they're feeling like they're juniors again. So they're asking questions they don't know the answer to. They're shipping code that they don't understand. And so all of those hard won skills are starting to get away from them. So I rolled out AI coding tools in organizations, and I've tended to hear two arguments.

So the first one is it's just glorified autocomplete, which, fair. If you've been around three or so years ago, copilot, it was a glorified autocomplete. And it's not game changing. It'll do some nice things. Great. So people who are around and then maybe haven't dipped back into it, valid. The other one which is more interesting was I had to spend a bit of time talking with people.

And what it really boiled down to was, I don't actually know what I'm supposed to be good at anymore. I'm not really sure what this means for me. And that's a very different kind of resistance. That's not, I don't like change. That's a very different kind of thing. So I want to choose a concept. So this concept is cognitive debt.

So this has been written about by I've completely blanked on the name. Margaret Ann Story, thank you, from the University of Victoria. Not Victoria here, but over in Canada. And you've probably heard about technical debt, I'm sure. Right? Technical debt, the shortcuts, the workarounds. Cognitive debt is more about the team level.

So what story talks about is this theory of the system, and the fact that this theory of the system is actually distributed. So it lives in lots of different places. So it's in the code, it's in the tests, it's in documentation, hopefully, if you've got documentation. It's sitting more and more in your AI tools, it's living in people's heads as well.

So that theory is everywhere. And this is what Stories says. The risk is not basically that the code isn't working. The risk is that the theory of the system is harder to access and keep track of. So basically, the evolution of the system is happening faster than our ability to keep up with that. And the problem here is that if that theory of the system starts to drift because people are losing it from their heads, or those people are leaving the organization, it's actually very hard to get that theory back again.

Sometimes even impossible. So I think that's what your senior engineers are feeling. They may not have a name for it, but I think a lot of the time, that's the kind of objection that they're feeling in their bones. So, bottlenecks. Let's talk about bottlenecks. Right? So, yes, a lot about cogeneration is going up, absolutely.

But up above, they're going, yeah, great. Code generation. Yay. We'll go faster. Well, the bottleneck bottlenecks never disappear. They move. Right. So we know that. And what that means is that the cognitive load goes up. So what happens generally, either the review queue just jams up and things don't get through the pipe, Or, quite often as well, the quality of the review drops.

So you get to end up with a lot of rubber stamping, just a lot of these kinds of ways of shipping code. Right? Just, yep. Fine. Fine. So, another concept I wanted to talk a bit about was normalization of deviance. So, this was coined by Diane Vaughan, writing about the Challenger disaster. So, those who don't know the Challenger disaster, one of the main cause was some o rings which failed.

And what Vaughan writes in this book is that the engineers and the managers developed a definition of the situation that allowed them to carry on as if nothing was wrong, when they were continually faced with evidence that something was wrong. Because they were minor deviations from the norm. So think about when you're working with a coding agent.

And you read through all the code, and you go, looks good to me. Go. And nothing went wrong. No disaster. Fantastic. So next time, you do it again. And you do it again. And you start to look a little bit less carefully, but that's okay, because nothing went wrong, until something does go wrong.

So that's what's happening. Right? Your teams are looking at the definition of the situation, they're looking at the definition of review, and they're changing that bar until a disaster happens. And this is not a failing of the engineers. Right? This is the standard itself drifting.

And also also I want to

be careful here because there's a lot happening at the moment around review as well. And what does review actually mean? Because line by line diffs, are they the most effective way of reviewing our code? Maybe, maybe not. But my personal belief, and apparently one backed up by GitHub, is that every bit of code that you're doing has to be reviewed in some way. So whether you're looking at it at the high level architecture or the line by line, I think the humans still need to be looking at it, because that's how you're maintaining that theory of the system in people's heads.

So Simon Willison, he is one of the co creators of a framework you might have heard of called Django. He also writes a lot about AI in general, and particularly AI encoding. So he talks about this. Right? If you're looking at a GitHub rep, let's say, you're looking for some a tool you're going use. Right? Before times, you'd look at this and you go, 100 commits?

Yeah. Good read me. Yeah. Lots of tests. Must be pretty good. They spent a lot of care and attention on this. Now, that's thirty minutes with Claude. So your signals have suddenly become very, very it's hard to go. Is this is this good or is this someone who's just slopped this out? Right? And this is important because in our organizations, these are the kinds of signals we're using to figure out, does this need more review, to look at people's performance.

All sorts of things that we're evaluating, those signals are starting to become noisy. So where are we at? Faster generation? Yes. Review bottleneck? Yes. Corrupted signals? Yes. I think ultimately, our whole software life cycle has been calibrated for a pace doesn't exist anymore.

And we haven't recalibrated yet. Now, wanted to look at some data as well. And before I go into data, I just wanted to do a very big caveat, basically, which is the studies that are around, they're at most eighteen months old. Right? So they're very We're talking very early in this, really, really early. And I think the point was made earlier as well by Annie this morning.

A lot of these studies are self reported. A lot of them, there's self selection. There's other biases in there. Some of the studies even, authors have come out a year later and gone, that finding, that headline number you saw, that's changed. And by the way, the error bars are huge. So big caveat. If anyone tells you this is the definitive answer based on a number, they're probably ahead of the evidence.

That said, let's look at some data. So this was from a study done by Multitudes. So they looked at what are the things that engineers are worried about when it comes to AI. So number one was a less maintainable code base. Number two, reduced ability to solve problems. And number three was a reduced code base understanding.

So none of those were, I don't like AI, or AI is bad. That, that's your cognitive debt right there. It's not named as that, but that's what it is. People are worried about losing the understanding of what they're doing. And this one's one which I think every engineering leader or technology leader really needs to pay attention to.

So they found that for teams using AI, the number of PRs went up by 27%. Fantastic. They also found that out of hours commits went up by almost 20%. So yes, there are productivity gains, but they're coming from your engineers' evenings and weekends. Now, this one I like as well.

So this is probably a bit more prescriptive. Right? So this is looking at the difference between teams who are using quality as a leading indicator and those who aren't. So if you're not paying attention to quality, typically using an LLM, they're very verbose. They'll spit out a whole load of stuff. Right? You get very big PRs. What happens when you've very big PRs?

People start reviewing them, start rubber stamping them. They're losing the cognitive understanding of what's happening in the system. There was an example quoted in the study where they had quality as a leading indicator. Their PRs went up by a 161%, and the size of the PRs went down by 8%. So smaller PRs, that was something that they were focused on as an outcome for this.

So the quality of that review can hold, and the understanding follows from that. Now, you know who else is interested in AI? This guy. So it's so important that Pope Leo the fourteenth devoted some of his first encyclical as pope to talk about AI, and to warn against the degradation of personal creativity and judgment.

So it's not just me saying this. I've got the pope on my side. So it does beg the question, right, what are we trying to be good at? Because it ain't writing code. Right? I think we heard from one of the speakers yesterday, writing code was never the hard bit. It's kind of the easy bit, actually. There's plenty of other things that we need to be looking at.

Now, we've actually run this experiment a couple of times before. So back in the nineteen nineties, Gary Kasparov, chess champion, lost to IBM's Big Blue. And after that, he went and spent some time working on something that he called advanced chess. Actually, in a callback to the previous talk, also called it Centaur Chess or Cyborg Chess.

So this was a human player working with a computer. So the idea was Kasparov had it was that the computer would look at the tactics, and the human would look at the strategy, and take a step back and look at the higher level. Now, the actual evolution of that was something that was called freestyle chess. So freestyle chess, you've got teams who play, and so you've got mixed teams of computers and humans.

So who won freestyle chess? Wasn't the best chess players. Wasn't even the most powerful computers. It was actually a team who found the best way for computers and humans to work together. And having that process was what was really important. And then beyond software engineering and indeed chess, a couple of other examples for you. So, University of Mannheim did a study.

They looked at doctors doing AI diagnosis tools. And they found that the objections, mostly from doctors, were about professional identity. That was what they were worried about. And then, this is a late breaking one for me, but this is literally overnight for us here in Australia.

The mathematical union, International Mathematical Union, put out what they're calling the Leiden Declaration on AI. And they're saying this, mathematics is and should always remain a profoundly human endeavor. So they are also worried about the impact that AI is having on their profession and on their particular skills and identity as mathematicians. So this is the shift that I think that we need to be asking people to make.

We're going from hands on the keyboard to hands on the wheel. So think about it's a different day to day shape of the job. It's a different skill set. And actually, it's a different satisfaction as well. So I like you to think about self driving cars. Right? Self driving cars, if you want to get from a to b, very effective, very safe, very quick, great way of doing it.

But some people actually like driving. Some people like getting on a racetrack or on a road, and they like the feel of the car. So a self driving car for them is actually pretty terrible. So this identity and what we like and what we're good at is really tricky. We've heard that a lot of people in our profession actually really like the act of coding.

And this is a real threat. So this is not just about, I don't like AI. A lot of this is, what does this mean for me, and how am I going to solve for this? So I think we've got to create the conditions for people to go on this journey. That's our job as leaders.

And not to inadvertently destroy it by trying to do mandates. So this is what I'd really ask you to do. Don't treat it like a tool roll out. Because a lot of people are. They're just going, yep, we're just going to steamroll it through and we'll just put some metrics in and we'll say, you've got to all be using it.

By the end of the year, everyone must have submitted this many PRs with Claude or whatever it is. Right? That's not going to do it. This is a fundamental change to what this means in your organization. So I'm going tell you a bit about what I think you should do. But before that, a bit of a sidebar in some more stats, because I like stats.

So, what's true? And you've seen some of these stats already today and throughout the conference, but overall, multiple studies show that juniors gain more from AI. It's true. ZoomInfo showed great satisfaction with AI tools, 72%. Fantastic. Gains in code generation. Excellent. However, senior productivity also drops.

So the Tilburg study found this that because senior engineers were spending more time reviewing code, their actual personal productivity dropped. Stability falls as adoption rises. That was from Dora. Thank you. Yes, Dora. As the adoption rises, the stability falls. Bugs per developer up by 54%.

And 17% worse retention on new skills is the anthropic number that's been quoted before. So this is not a AI is good, AI is bad, this column, that column. This is both. Both are true. And they're signals that you and I and our teams need to be looking at. Alright. So these are this is my three things I think you should do. Number one, naming the fear properly.

Number two, thinking about learning. And number three, make quality the floor. So firstly, naming the fear. So this really, for me, is what it's about. Right? It's understanding that this is not people resisting, and it's not about steamrolling them into saying, you must do this. This is a fundamental shift in what it means for a lot of our people in their day to day work.

They're no longer writing the code. They're thinking about the overall system. They're thinking about the architecture, the trade offs. They're thinking about what they should keep from the model, what they should throw away. It is a different skill set that we're asking people, and it is a different identity. So there's plenty of things in there. This cognitive debt idea, the theory of the system disappearing, if people are feeling that, just saying, do more of it, is not going to get them to change.

It's not going to solve the problem. So your most cautious engineers who are telling you this, they're your early warning system. Learning conditions. So this was a study from Atlassian. They found that younger tech workers, in particular, were 38% more likely to hide their AI use from their team.

So what does that mean? It means they're doing all of their experimentation in private. And that means they're not sharing it with the team. So another finding from the Multitudes study that I referenced earlier, they found that other than giving people access to tooling, the most effective thing for the adoption was peer to peer learning.

So not top down learning, not people coming in from outside doing, but peer to peer learning. So if your younger tech workers are doing this, but they're not sharing it with their peers, they don't feel they can, that's going to be a challenge. So what can you do? So for me, this is about psychological safety. It's about modeling the behavior.

Going out to your teams and saying, awesome. Or I did this thing with AI, and it was terrible, and it fell apart. But being open about that and getting the conditions where people actually start those conversations. So I did this, rolling out with teams, and I found the most effective thing to do was not a mandate, it was experimentation. So it was go off and try.

Here are the tools. Have a play. Find the time for it. And importantly, tell us what you found. That playback bit is really, really important to surface that and to start the conversations. And then quality. So go back to this. Right? Less maintainable code base, that's what people are worried about. The most the biggest thing that people worried about with AI coding was the less maintainable code base.

So the quality standards, the things that you've always wanted to use or have used or hopefully are using, Linting tools, automated testing, security scanning, SAS, DAST, all the things, the deterministic ways of saying, is this right or not? They're not slowing you down. They're not the roadblocks. They're what helps you speed up.

So having go back to that multitudes finding again about the PR side. Having an idea of what quality looks like for you in your organization and making sure that that holds and directing everything towards that. That's how you're going to continue to maintain the speed as well as the safety. It's not either or. So ultimately, once you leave it this, I actually think some of your engineers should be a little bit The question is, have we caught up with what they're scared about?

And are we doing things to try and solve it? Because ultimately, the calculator didn't make mathematicians obsolete. There's plenty of mathematicians out there. The spreadsheet didn't make accountants obsolete. There's plenty of accountants there's probably more accountants out there than there were back then. Right? It changed the job.

It changed what mathematical skill meant. And that's what we're finding now. So we've got to help our people through that transition. I put some sources up there. I don't expect you to read them all. They are on the QR code that links to a page on my website which has got all the sources as well as a copy of the slides.

I think it's traditional at this point to say, I'm gonna be around for the rest of the afternoon if you've got questions. I'm not going to be around for the rest of the afternoon because I've got to go and do some work. But if you wanna hit me up on LinkedIn or or whatever and ask questions, debate me, more than happy to do so.

Thank you very much.

Brilliant. Thank you, Andy. And, yes, your presence will be missed later on this afternoon, but we do have you for another half an hour You do. During our panel session. I'm gonna ask you a quick question. We're gonna be doing a little bit of, stage rejigging here. Mhmm. But you mentioned doing those kinda almost that that knowledge transfer sessions, learning transfer sessions with our team and encouraging them to do that. Think one of the things that certainly I've observed is there's this notion around training, particularly with AI that it's kinda like, give the people the tools, give them some space to experiment.

Oh, here's some materials, off you go. Right?

And

I think people are seriously underestimating how much time is required to do this. So I'm curious to get your perspective on As you've been doing this with your teams, how are you gauging how much time they need to be able to do that kind of peer to peer learning?

Yeah. So I think the peer to peer learning, there's there's a few ways of tackling it. Right? One is going, okay, we're gonna set aside time. Right? So Friday afternoons are always learning time or or whatever it might be. I actually think building learning into what you're doing is more effective. So learning not being a separate activity, but being part of what you're doing.

Am I out of time?

No, no. No. We're good.

Didn't turn it off again. So I actually think bring bring it into the daily work. So some of you might be aware of the concept of mob programming. Actually, there's people out there who are mob programming with AI. So your whole team of which, you know, maybe it's a smaller team now, maybe it's three or four people, it's a product person and a UX and a couple of engineers, right, actually modding with a coding agent. So that you are all there in the same room. You are learning, because that's a really important activity.

But you're also creating as well. Because learning for learning's sake, some people react really well to that. Some people actually really only learn through doing. So having the the real way of doing it is also really important.

That's brilliant. Has anyone done anything like that within their teams? Put your hands up. I'd be curious to see. It's like, it's sort of about a quarter or so. And how do you the, the developer feedback to those sessions, how are you finding that, in terms of, you know, whether it's the mob AI programming sessions or kinda like just that peer to peer learning kinda based stuff?

How are they taking to that? Are they taking to that, particularly the seniors?

Definitely. I think, you know and look, there's definitely a bifurcation, I think, of the people who are just all in. And often you'll find that goes by team as well, which kind of really enforces that idea of peer to peer learning where a whole team are working on this together. They're all sitting there going, this is great, and they're feeding off each other. Whereas sometimes where you've got maybe a team where there's one person who's enthusiastic and evangelistic and the rest of the team are a bit like, don't know if I wanna be in that, then that's where it's a lot harder.

So actually, lot of it comes also down to the team conditions.