What We Learned Taking a Culture-First Approach to AI Adoption at scale

Thank you so much. Good to go. So I want to start by asking everyone a question. This is just like a sort of a poll. So I'd like you to all take yourself back in your mind's eye to when you started your AI coding journey. Maybe it started with you using a tool yourself, or maybe you had to lead a rollout, or you do some change management at your company. So when you think back to that moment and then think about the journey that you've been on since then, you know.

We've had new tools come out. We've had the whole agentic revolution this year. The game has really changed. So when you think back to that starting moment, is there anything you wish you knew when you started? Is there any advice you'd have yourself? Just put it put your put your hand up if there is something that you wish you could tell yourself.

So this talk is about what we would tell ourselves.

Yes. Lucky. My hands up. Thanks, Eric. So when we we started this journey from the outset, we had this conviction that the success of our AI adoption was gonna come fundamentally down to people and some of the research that Chris shared just before. And so the way we thought of it was like AI is probably the most powerful amplifier of human potential that we've ever created as a tech industry, what would it look like to try and approach things in such a way where we could get like the the reap the benefits of that and have people at the center and reap that amplification?

So that's kind of like what guided us. And we knew we needed to have like a really good framework, really good tools, process metrics for this rollout. But we also knew if that was the we over focused on that and we didn't have our people enough at the center, we could probably have a negative impact on all of our 200 plus engineers and we could leave a lot of the really special results just still sitting there on the table.

So we started asking ourselves all the way through, what does a culture first approach to our AI adoption look like? Just check with my clicker There we go. So before I talk about what we actually did, I want to say a few things about culture first. Because when we use that term, it sometimes can be interpreted in different ways. What we really think of when we say culture first, at its essence is that people and culture are the prime like our primary focus. We have a primary focus on those things.

We think of those things first. It doesn't mean trying to like overly please people at the expense of the mission. So like focusing on people and pleasing people are very different things. What it means is that in all the decisions we make, we genuinely bring the value and the well-being of our people into those decisions.

And not like as a checkbox exercise, but as an actual non negotiable input into how we think and act. And the reason I care about this so much personally, it comes down to something I believe about human beings. And I think every single person has extraordinary potential. Like, you, me, every person back in our teams, we have these incredible golden things inside of us, like our creativity, our ideas, our ability to learn, our intellectual capability, our heart for helping other people, our courage, our character.

All these things are amazing. And if they have the chance to shine out and really manifest, then amazing things happen. Amazing, magic things happen. But that doesn't tend to happen by default. That only happens they come out when we feel safe, when we genuinely feel safe. They come out when we feel trusted. Chris was talking about trust being that foundation before.

They come out when we feel inspired. So it's like, how do we create conditions such that it really is safe and it really is there's trust there for these things to shine, because then we'll get one good thing building on another and it will just keep upward spiraling. But when there's fear, when we're operating from fear and we're operating from a sense of pressure in the form of, if this doesn't go well, I'm going to be judged or if this doesn't go well, I'm going to be penalized.

That causes us to clam up. It causes those things to be kind of like held inside. And we may still comply and we may still like get the minimum requirement on things, but it's so different to what's possible when those things shine out and it impacts us and the business in those things. And so in practice, this culture first idea, I think, looks like a lot of little things added up over and over.

It it looks like the the recognition of someone and the belief in them in a moment that encourages them to do something. It it manifests in the joy and the fun of the things that we're doing. It's also in the situations where maybe someone's resisting the thing we're doing. How do we respond? And it's like responding in curiosity and genuine listening rather than than judgment of the person for raising that.

And it looks like bringing people into discussions and having open conversations on hard topics and being transparent about the things we're doing, even if it makes it look like we're not quite sure of where we're going or if we're like flip flopping over time, which I'll talk about a little bit in a second. But I'll set it up, Eric.

You should know. Keep going.

I love how passionate you get when you talk about this. He always gets so fired up when we get to this stage of the talk. It's awesome. But it is really important to highlight something, which is that this this whole culture first approach, and you could You can very clearly see it just now. It needs to come from a genuine authentic place.

It can't be performative. It's not a way of doing. It is a way of being. And we really try to embody that with everything that we do in our workplace, and I think that's why it works.

Yeah. And so jumping into now what we actually did with this AI rollout. So we started this journey with a simple idea at the at the center of it. It was called the AI learning loop, is what we described it as. And it had three parts. It was use, share, inspire. And the the concept was pretty straightforward.

The more people that used AI and tried things and then shared what they they did and what they learned and what they experienced and then inspired others by that, the faster we would learn as an overall group and that would lead to good things. So we we rolled out a whole bunch of tools for people. We we started with, on the coding side, GitHub Copilot, and then moved on to like ClaudeCode and Cursor in our journey.

We had a tool called Glean for enterprise search. We had Gemini and then now we're on Claude Enterprise. But we had all these these tools out there, and we really encouraged experimentation at all levels. And the goal was to make trying things feel normal and not feel like a risk. We started a monthly meeting with all of our product group, literally called the AI Learning Loop, And that's where people would come along with things they were experimenting with.

So they'd be like And and there was no pressure to have the right answer or to have done something really profound. And there was also no expectation of polish on what you did. It was just, here's what I did. Here's what I found. That was interesting. This went well. This didn't go well. And we kept it fun and we kept it safe.

A lot of sharing was spreading through the group. So in our teams, in our camps, which are our team of teams, there was sharing, bringing brought into all those different rituals. We had Slack channels that we spun up, some going deep, some going broad, and they all kind of take on a life of their own as well.

So there's heaps of sharing going on. And then underneath all of this, Eric and I were talking a lot about, like, the overall transition of our journey and how we thought about our maturity in that and where we were. And we came up with this thing, which we eventually ended up calling the BAP model, which was base, advanced and pioneer.

And anytime I wanted to talk about it, I said, Eric, I want to go into the BAP cave. But we would the recognition of this was that different people were gonna be at different stages of this journey and then And for very valid reasons, and that's like totally totally fine. And let's not try and do a one size fits all approach that just like treats everyone in exactly the same way with it.

Let's like set things up so that everyone can find their own unique journey and their own moments for where they are with it. One of the cohorts that we really like work closely with was our pioneers or our AI champions. So we we put a lot of stuff around them to help them pull all of us forward.

And that was like a lot of pairing with other people to transform what they were doing. Some of them recorded videos and things like that. Other things that were like Some things that were really transformative for us were our events. So we had some hackathons and we had some accelerators. And the idea of the hackathon was to get the entire product group. So 350 people all together having an experience where every single person is doing stuff with our tools regardless of their role or the experience they had to see what we came out with.

And again, it wasn't about what did the teams create. It was about what did we learn while we were trying to create those things. We similarly did some accelerators where a team or two would get together to try and accomplish an actual real goal in a short period of time using only agentic AI techniques to see how far they could get.

And again, I was talking to Kevin earlier about this from Culture. He was saying like, in one of those accelerators, the team didn't achieve a lot of the goals they set out to do in that week, but the learning that they had has grown. And in the months ahead, that team has now really started accelerating from the thing. So again, it was all about the learning in those events. One thing we also did try and do is make the events that we did fun and engaging and warm and just make it a really good experience for people.

And part of that was getting volunteers. So we went out and said, hey, we want everyone to help us break come this to life, this hackathon that we did. And I think 40 people put up their hand. And we said to every single person, absolutely, we'd love your help. Like, we engaged a team of 40 people to help us do this and let people run with what they were passionate about.

So someone was into three d printing and at home printed these amazing three d robot trophies that corresponded to the event and everyone loved them. Someone else was into video stuff and they got the chance to do a full video live interviewing and video production of the event. Eric's into music and he he spun some like cool custom tracks for us.

Myself, like, as we were leading up to that first hackathon, I joined a meeting to sort of give an update to everyone and thought, I'll try and keep it fun. So I came on with this avatar, this Google Meet avatar that was like a robot and introduced myself differently. I'm like, hey, I'm pale. Paul couldn't be here today, but I'm p a I l, and I'm probably better to talk to you about this event.

And it got a bit of a laugh, and then but then some people were saying some things, and so I did a little bit more pale. And then eventually, like, people were taking pale videos home to their kids at home and going, daddy, we want to see more pale. So I was like coming in with more stuff.

And then over time, pale kind of took on a life of his own in a literal way. And I want to show you this clip to to describe what I mean by that. Wow. So so Pao got his Pao Nokia idea, why it coming through, become a real robot.

I can't tell you what I've done to my job in the last few years. Occasionally, I have to keep coming out dressed in a silver yoga suit. But the thing with all of that is nothing was planned and nothing was like pre prepared. It all emerged pale and so many other things emerged organically from the conditions we've created.

And that's the one thing I want to leave about with you, the things we did. It's not there's lots of good specific things there, but underneath it all, it was those conditions of making a really safe and trusted environment that let a lot of things shine out. And like in my own personal terminology, if you've heard me speak before, you've heard me describe it like this, Like, I would say for me, it was like the foundation of what we did was love. And that's the best word I've got to kind of like holistic to describe that.

Over to you, Eric.

Yeah. So on the subject of what we built from a tooling perspective, we didn't make any pale robots or anything like that. We did a lot of the standard stuff. We rolled out MCP servers. We're been building agentic workflows, all that kind of thing. But I think what's really interesting and relevant to this talk, I think, is the approach that we took with the tooling.

So I've been in the developer experience space since before the sort of AI revolution sort of came up. And one thing that sort of I always held onto as a core focus, and I know that even the triangle came up in one of the keynotes this morning, is the whole idea that developers are knowledge workers. Right? I mean, the the value that developers and engineers create is ultimately in their heads and then is expressed through these different tools.

And so when the work happens inside someone's head, the only person who can really tell you how to make them more productive or what's slowing them down is that person. Right? So we've always really focused our developer experience and our productivity efforts on understanding what is slowing developers down, what is causing them friction. And so we gathered a whole bunch of data.

So that's what I wanna talk about, if you could roll me on to the next slide or two. So about twelve months ago, Culture Amp partnered with engineering telemetry company out of New Zealand called Multitudes. I think the Multitudes study did actually get brought up a little bit earlier in this in this room. So we were actually I think we contributed something like 200 engineers to that study. So our data was about half of the whole dataset that was in that study.

And, you know, at the time, when we commissioned this study, it was about halfway through last year. And people were starting to work with a lot of coding tools and sort of using that copilot style of like auto complete and having chats, with LLMs in the process of coding. And there was this There was so much hype.

There was so much hype around everything. And I'll admit that I was a little bit skeptical, and I I really wanted to see some data to say, is this actually making people faster? So we ran this study. We we shared about six months of engineering telemetry data with multitudes. They surveyed about a 100, so about 50% of our engineers.

And they also ran these in-depth interview conversations with about 12 of them. And the finding was that, surprise surprise, AI makes you more productive. Yes. Everybody knows that. But we didn't know that at the time. And so, you know, it was really great and really gratifying to see the data actually showing us that AI did indeed improve merge frequency, which was really fantastic.

But what was much more interesting to us is that the data also revealed some points of friction that were emerging.

Yeah. So one of those, and it was talked about earlier as well, was out of hours commits went up after that study. And we also saw mean time to restore had also gone in a negative direction. So there were two signals we were paying close attention to from that quality and well-being front. We also had a bunch of people feeling that even though we were taking a very people focused approach to this, we were still moving very fast and going, hey, this feels like a lot is happening and it's moving fast.

And we'd been doing a lot of peer sharing, but people craved even more of that and saying, I want to do more sharing. I want to do more pairing with people. Kind of what Chris said earlier as well is like, want to pair with people who are very similar to where I'm at in the journey because sometimes someone that's far ahead, it feels like there's there's such a gap to bridge.

Yeah. And that was a really interesting finding. And it wasn't just unique to us. It was sort of uniform across the whole cohort of companies that were part of the study. But there was also an interesting finding where us and one other company actually differed to the cohort, and that was, with PR sizes. So what we saw is that, when companies did these big AI coding rollouts in their engineering organizations, the average size of pull requests, grew. This is because AI is quite verbose.

It created a lot of slop. I'm sure people remember when they first started working with AI. They're like, oh my god. This is amazing. I'm just gonna shove everything through. And then someone else reviewing it's like, what? I don't understand any of this. But what was interesting is if you can see the the red boxes, those were the non culture M companies that I would say sort of had emerging engineering cultures.

And this is, by the way, this is where our interpretation of the stats is a little bit different to what's in the published report. And I'll explain that in a second. And then on the right, in the green, you have companies like us who had an established engineering culture. And what we saw is that PR sizes for our engineers and this other company, they actually stayed steady or in some cases actually went down slightly. And, you know, the researchers we were working with were fascinated by this.

And when they looked into why and they asked these engineers, like, what are you doing differently? Is there some sort of maximum PR size that's being enforced or something like that? Is there explicit quality standards? It wasn't anything like that. It was actually just this sort of intrinsic intuitive behavior that our engineers were undertaking. They just said, you know, well, we know that huge PRs are really bad, and we know that it's a better principle to slice things thinly.

So we're just adjusting our team processes. No one told them to do that. We didn't, as a central enablement group, say, hey, you have to do this. They just sort of did it. And for us, you know, what we realized is that this whole investment in all the stuff that Paul was talking about, these learning loops, the sharing loops, but also sort of like our architectural standards.

We got Kevin, one of our architects in the room. All of these sort of cultural elements that came together to form a strong engineering culture, then kind of influence the mindsets of these people. So we basically found that a strong engineering culture was a structural mitigant for some of the friction that AI creates, which wasn't intended necessarily, but was a really interesting finding.

Yeah. So one thing that since we submitted this talk proposal and since we're talking about it today, we have one significant thing changed in the way we've been approaching things. And that is that, like, when we started this journey, we didn't have any intention of mandating the use of like agentic or AI coding tools in the way we're working.

And we're now at a point where we are actually asking all of our engineers to embrace agentic engineering techniques in what we're doing. And that's not a decision I was expecting we were going to make, but when we look at our own data and we look at our lived journey and we look at the system we're now building for how we do our work and where we're going, we're realizing that it's only really going to come together if we're all contributing to this and all leveraging it.

And we're going ahead with that. And everything I've been describing today earlier still applies, but we're getting clearer, I suppose, about what our engineering looks like as Culture Amp as we're moving forward.

Yeah. And I mean, if we if we look at the data and we compare our sort of pioneers to the people at the the b or the a of the the back. We basically have seen that in the last few months, there's begun to be this sort of categorical shift in the productivity of people who are really adopting the, you know, agentic way of doing things and they're running sub agents and they're really pushing the capability of the technology versus people who are still sort of lagging behind and working in the old ways, still using that auto complete, that chat kind of function. And that's that's, you know, that's that's fine that people are taking their their own time with the journey.

But what we are noticing is that we're starting to have instances where we have these pioneers in teams. And maybe there's one pioneer in a team of 10 people. And then you've got this one engineer who's literally outputting 35 PRs a day, and then the other nine folks are doing one or two PRs a day. And the whole team's collaboration system just completely falls apart.

It's like a categorically different way of working. And so what we've realized is that a lot of our efforts have been about raising the ceiling of what's possible for engineering cohort, making these cool tools available, developing new processes, that sort of thing. But it's just widening the gap between the pioneers and everybody else. So one thing we're starting to do is try and raise that floor. And so we're looking at, you know, this mandate as part of a sort of a categorical shift in, you know, embracing a new way of working with a bunch of support, but just sort of acknowledging like the way we do things has changed.

It's not just a new tool that's out there anymore. It's a fundamental shift.

Yeah. And and you when I look back to that culture first idea in this context as well, I think it illuminates a really interesting part of that because like being culture first as well doesn't mean you're not going to make hard decisions or it doesn't mean you're not going to set clear direction when it's needed and it doesn't mean you're going to just always seek consensus, But it does mean that when you are going to make a significant difference, it's very very a decision like that is very considered and that you've built enough trust that people are ready to go on that journey with us. And that's kind of when we were about to talk about this change, I wasn't sure how this was going to be received. And our camp leaders like Prakriti is one of our camp leaders and have done an amazing job working with all the teams across this. And what we're actually seeing is everyone embracing this direction.

And there's still a lot of questions. There's still a lot of the things that we talked about this morning and talked about with like Andy talked about earlier. Like, they're really valid things, like the engineering role is changing and what does this mean for me and what things am I losing that I used to love doing? And we're still wrestling with those questions, but we're kind of in this place where we're wrestling with it all together and I think that's that's the the next horizon for us is to is to go through that next phase.

Yeah. So I think, you know, if I if I was to take myself back to that that point twelve or so months ago and, you know, if I had something to share with myself, there's there's probably two things I'd wanna tell myself. So number one, like I spoke about a little bit earlier, but that first principle of, you know, speaking to people and understanding what's causing them friction and then sort of working on that.

I thought I I kinda threw away that principle when all these tools started coming out and I thought, oh, the game has changed. There's some new problem that I have to solve. And over time, what I've realized is that AI is an intensifier. And yes, it's making people faster, but it's also putting more pressure on the existing bottlenecks that always existed that that were always out there. Right? I mean, in all the time I've been gathering data on, engineering productivity and I look at change lead times, I'm always noticing that PR review times and PR wait times are sort of the consistent part that slow.

And that still exists. It's more of a problem now because there's more throughput, but it is the same problem. So I think what I would tell myself for one is to kind of trust my gut and keep following those first principles. And number two, I was really focused on measuring the ROI of, AI and making sure that I could prove that it is indeed increasing productivity and so we can spend, you know, more on these tools and roll it out.

And I I believe in this stuff. It's so great. But what I've realized is that the true value is measuring where AI creates friction and what it's making difficult because those are the interesting problems to solve and that's where the next level of unlocks lie.

Yeah. They're great ones. And I think I think I would tell myself if I go back to the start of it is like, it's okay to not have everything figured out and that things are probably going to change along the journey and that's okay as well. Like through this journey, there have definitely been some great high points like some of those, like when we did our first hackathon and the experience of that altogether and the vibe we had going was amazing and some points of clarity.

But there's also, if I'm being honest, times when I've been like, wow, this is overwhelming and like, am I keeping up with this change and like, am I adding value and what should I be focused on and like going through that whole thing. And we've had a few points where we've pivoted on some things, like the one I described to you with just earlier.

And so I think I would like say to myself, that's okay. And like those things, putting people first and all the stuff we described today, really trust in that and especially in those hard moments when you're not quite sure, keep doubling down on that and hold it with both hands because it matters and it matters more than you even probably realize.

And like when I look at all the ripple effects that have flowed through over the last year or so on this journey, I knew there would be positive impacts of focusing on people, but the extent of it is beyond what I would have imagined and it's been really, really special. So I'd be like, hold on to that and it's okay to to not have it all figured out.

And as long as you're open, you'll just keep adapting to things and we'll we'll all figure it out together. And if I don't know, which is so often the time, someone else will come up with the idea or the thing that will actually like help us help us get there. So that's my one.

Yeah. So just to wrap it up, I I really just wanna highlight that point of taking a culture first approach to these kind of things. It's it's not like it doesn't have to be a fork in the road. It doesn't have to be a choice of I can do it, you know, I can focus on the tool, I can focus on the tech, I can focus on the culture.

But culture is the the choice that can sort of imbue everything. And it's like I said before, it's a way of being operating a culture first way. So we hope that, you know, this talk inspires you to maybe make some of those choices of your own. But that's it. These are our details on the screen. Feel free to connect with us.

We're gonna be around a little while as well. Just look for the curly hair. And thank you so much. Thank you

both. While we do the tech changeover, I've got a quick question for you. The stuff you were doing in the beginning with the pail and the person three d printing the trophies, it feels like there's this silliness and fun and love. You used the word love. That's not something we often hear a lot in these types of conferences. It seems like it's really important to you.

Like, why why is it important to you?

I just think there's there's something like us as humans, there's something that's really real about connecting with one another in in relational ways. And I think like that when I use that word love, like there's it's a very overloaded word. Like I think the ancient if you go back to the ancient Greeks, they had seven different words that all meant love.

And the one that I am talking about is like unconditional love of people. And it's like when we see each other and value each other, and that's not just like a logical left brain thing. It's like a beholding of someone and being with them and really being in their presence and doing something that connects. It's just really meaningful, I think.

So yeah. And so It shows.