Dispatches from the frontline: building AI with AI at Atlassian
Introduction: AI Transformation at Atlassian
The speaker introduces their role as design manager on Atlassian's Studio product within the Rovo AI offering, and frames the talk around five key lessons learned during Atlassian's AI transformation. They outline the five topics to be covered: garbage in garbage out, quality as a team sport, avoiding forced AI integration, using the right tool for the job, and letting AI handle tedious tasks.
Garbage In, Garbage Out: The Power of Context
The speaker explains how Atlassian's Rovo AI leverages the 'teamwork graph'—a connected dataset from Confluence, Jira, Slack and other tools—to produce highly bespoke, relevant answers compared to generic AI models. They discuss the added value of personal context via Loom meeting transcriptions and private notes, and demonstrate using voice prompting in Robodev to quickly generate rich, structured output from unstructured verbal input.
Quality Is a Team Sport: Slowing Down to Learn Together
The speaker discusses how rapid AI advancement has created competition between roles like PM, design, and engineering, and how Atlassian countered this by running structured 'AI Builders Weeks' and training 150 designers to open pull requests via vibe coding. They cover lessons learned about enterprise trust, the need for deterministic versus probabilistic experiences, and how chat interfaces aren't always the right fit, especially in B2B contexts.
Dogfooding: Learning by Using Your Own Tools
The speaker explains Atlassian's cultural commitment to 'dogfooding'—using their own AI tools internally before shipping to customers—and shares a personal example of an agent/automation built in Studio to manage design review scheduling and feedback. They emphasize that dogfooding helps catch UI bugs and build genuine product feel, while cautioning that it should complement, not replace, customer testing.
Don't Put a Bird on It: Avoiding Forced AI Integration
Using a Portlandia reference, the speaker warns against wedging AI into every experience, describing how multiple Atlassian teams converged on nearly identical chat-style UI widgets that confused customers. They discuss the risk of premature convergence on AI-generated solutions, the tendency of models to produce 'average' ideas, and the importance of applying AI to genuinely hard problems (like research synthesis) rather than just making easy tasks easier.
Right Tool for the Job: Balancing AI and Human Craft
The speaker describes how AI fits into different stages of the design process—from rapid prototyping to fixing production code—while stressing the continued importance of human critique and iteration. They share a personal anecdote about writing this very talk with a pen after an AI-generated draft fell short, illustrating that knowing when to use AI versus manual thinking is itself a skill Atlassian is trying to cultivate through its 'three commandments' of prototyping, pushing boundaries, and shipping to production.
Getting AI to Do Your Dishes: Automating the Tedious
Referencing a popular meme about wanting AI to do chores so humans can focus on creativity, the speaker details how Atlassian built MCP integrations with their ADS design system, introduced a 'design technologist' role, and created a Slack-based content assistant agent trained on content design principles. They close by sharing their personal 'second brain' Markdown-based assistant system built in Obsidian, which helps manage goals, projects, and meeting notes, and summarize the talk's five key takeaways before concluding.
Thank you so much, Steve, and thank you for having me. I've decided to really frame this talk around some things that we've learned over the last year or so at Atlassian. We've gone through a pretty big transformation, to use the buzzword. We're really trying to not only bring AI into the technology that we provide to you through JIRA and Confluence and all of those other tools, but also really bring AI into our toolkit.
And in design, we've really been very much at the forefront of this. I, in fact, would say design is ahead of engineering at Atlassian at the moment. And just a little bit about where I am in the organization. It's massive. We're 15,000 people. I am the design manager on a product called Studio, which is inside Rovo, which is our AI offering.
And Studio is where you come to build automations and agents and little bits of custom UI that you can use to sort of customize your Atlassian product suite. So today, I like things that match, and I like structures. And so I thought twenty five minutes, I'll do five things and see if I can do five minutes on each thing.
And this is what we're going to cover. So garbage in, garbage out. Quality is a team sport. Don't put a bird on it. I will explain these references. Right tool for the job and getting AI to do your dishes. So let's start with garbage in, garbage out. One of the things that we have learned over the last year is that the difference between sort of slop and something that's really amazing that comes out of the end of the AI machine is context. And for us, organizational context has been pretty incredible to really supercharge what comes out.
So our offering of ROVO allows you to use different AI models. We've got the Claude's. We've got the Codex's. We've got the Gemini's and all of those different models. But what it really is is more than that because it is trained on the incredible amount of data that is inside of your Atlassian instances. So all of your Confluence documents, all of your Jira work items, all of and even your third party tools like Slack, those things can become what we call the teamwork graph.
And that graph is a connected set of data points about what essentially is the collective knowledge of your organization. And so when you talk to Rovo, which is like our chat interface, you are getting really bespoke customized answers based on that knowledge graph in your company. And so there is a ton of opportunities from that.
But one thing that is really obvious is that the relevance is really dialed in. So when you ask for an answer from Claude, you're going to get the generic answer for everyone. But if you ask for an answer from Rover, you're going to get something that's really bespoke to not just your company, but your team and even you as an individual and what's been relevant for you in the last hour or so.
It's very, very highly bespoke. And so we have this sort of visualization of the graph as this sort of neuron kind of connection between things. But this is really what we are trying to get across with the teamwork graph as being that really layer of relevance. But beyond that, I have found that personal context is also really critical. And I'll talk about this throughout the talk, but basically, we have a really huge advantage at Atlassian being a remote first company.
So tons of our work, basically everything is in Confluence documents. It's in Jira. And even now, we are having this real explosion of available data to feed into this personal context layer through Loom. So we use Loom in our meetings. And Loom is on our Zoom calls. That's me. We have essentially every Zoom meeting we have.
The Loom notetaker is in there. And those transcriptions of the meetings are then something that I can feed back into the AI to really extract all of that stuff that happened verbally. So not just the stuff that we put into PRDs, but the argument that we got into about that PRD is now also added into that knowledge graph that we use to really take the work forward.
And so for me, I use this in a way where I also document stuff personally that maybe I wouldn't put onto Confluence, stuff like my secrets, you know, my hot takes, maybe my, like, notes about an interaction, maybe new things that I'm having that I'm is not necessarily public, but I'm having almost this private graph being developed that I'm increasingly using to get even more highly bespoke answers out of that AI layer.
And so the final piece of this layer of increasing specificity is this idea of iteration and specificity. And so pretty much everyone here will have done a one shot prompt where you ask for the thing that you want and what came out at the end was just average.
And you sort of have your trust broken. You're like, maybe this thing can't do the thing that I want. But what we have found is that the more that you put in to that prompt and those series of prompts and those iterations on the prompts, the more that you can really dial in the output. It's kind of obvious.
Right? But for me, the big thing is voice. So this is an example of me using a tool that we have called Robodev, which I use in the command line. And you can see that at the bottom there, there's that animation. That's me using voice to prompt into it. And the reason why I use voice is, as you will see in this demonstration today, I can write fast, but I can yap a lot faster. And I can really get a large amount of information into that prompt in a way that is very natural to me. And even if it's unstructured, as we've just heard, AI is very good at taking unstructured data and turning it into something structured.
So I can maybe I'm starting off on a project, I'm just like, Okay, I'm just going to do a big brain dump. And I'll sort of just rant into the microphone for a bit, and then what will come out is like a beautifully structured document that is really rich with that context through those three layers my specific prompting, my personal graph, and the underlying knowledge graph as well.
So this takes me to my second point, which is that quality is a team sport. And we've heard a little bit about this today, but one of the things that I have talked about to a couple of people already is that noticing that there's this kind of musical chairs going on where all of the roles, like PM and design and engineering, are sort of like the music's playing, we're all doing everyone else's job, and then the music is stopping, the chairs are being removed.
And I think it's been I think it's been kind of forcing us to compete against each other. But what we are seeing now after, like, a kind of a year of heavy competition is that we're really needing to stop and get people to play together again. And one of our values at Atlassian is play as a team.
So as you will know, the industry is really moving incredibly fast right now. You can literally go away for a week or two and come back and like some new it's like Opus five has dropped. And you're like, oh god. Now I have to learn this. And there's a new design tool out. And there's you know, something else has come out.
And I think it's very tempting and we will hear more about this today to just rush at the moment. But like if you've ever learned to play a musical instrument, you will know that you sort of have to learn to play the tune slowly before you can play it fast. And we are really using that as a framework at Atlassian.
We're taking the time to sort of slow work down intentionally to force people to take the time to really understand this and not just rush into it with more prompts, but actually learn how the technology works and really get that sensory sort of muscle feel for it. So we have already run three AI builders weeks. A lot of companies are doing this.
But we stop work for a whole week and we say, everyone, we're doing the structured learning together. We're going to go through this content. We're going to build some stuff. And even a couple of weeks ago, I flew up to Sydney to deliver some training for the entire design org where we got 150 design well, actually, just this was only one shard of the design org. We did a lot of it remotely.
About 150 designers at a room. And we taught them how to open pull requests. So using vibe coding, getting into a prototyping code base, and then actually developing something and being able to open that into a pull request that your team can then review, which for many designers, this is new. I mean, not everyone, but for a lot of people, this is a new skill.
And to expect that people are going to pick this up on their own, I think, is unreasonable. So I'm really glad that we are doing that work to really bring everyone together. The other thing about bringing people to do this at the same time is it really sort of defangs that competitive nature of it. We're actually all in this together, and that's been really important for us as well.
We'll hear more about this from Riley's talk, but one of the big learnings for us around, you know, quality is that this rush to sort of put AI in everything has, to some extent, damaged some trust with some of our enterprise customers. Admins don't want to just sort of go rush into the brave new world. They really want the ability to roll this out gradually or understand the specifics of how the data is stored or to be able to turn it on over here but not over there.
And we have done some research into this already and noticed these patterns emerging. We had a quote from Rachel which said, until you understand someone's emotional context, you're not really designing for them. You're designing at them. And this was in the context of how some of these enterprise admins were feeling when we were just sort of we actually turned Rovo on by default and asked them to opt out.
And that was actually the wrong move, and we learned from that. We've also noticed that some experiences actually need to be deterministic, that probabilistic way of working is not appropriate for everything. For example, in some large orchestration flows, there are some steps that you just want to make sure always happen. You don't want anything to be up to chance.
You don't want the model to be deciding. You want to make that decision. But also, some interactions are much better suited to just direct control. So for example, we had an interface which is kind of like a mini site builder. And we started with that being a chat interaction but quickly learned that that was incredibly frustrating, and people just wanted to click that thing and then change the color of it directly.
And so we're not putting it in everything. We're finding the right moments to put it in. And as I'm sure you'll hear more from Riley, b to b is also a very different context to b to c. We are selling our tools to other businesses who are then rolling it out. And so that is very different paradigm from a design perspective than it is for rolling a chatbot out to customers.
And even though the chat is the really popular paradigm at the moment, it doesn't always suit everything. And it's really important to spend that time and do those reps and figure out those different interaction design patterns for the appropriate context. Finally, we are really passionate about this horrible term, but it is the term, dogfooding, which is the idea that if you're making dog food, you should eat your own dog food and see how it tastes. I have heard people call this catfooding.
I don't think that's better. At Atlassian, like, it's for better or worse. We are culturally very hardwired to using our own tools. So I kind of think sometimes we have a confluence brain. We sort of we almost use that tool too much in a way that's unnatural. But I actually think the good thing about dogfooding is it forces you to really get that visceral feeling for using the tool before you roll it out to customers.
And we have a sort of stage pipeline in which we do that. Of course, you have to spend time with customers. I'm not saying you shouldn't. And you should never just dog food it. And you absolutely don't want your customers to be doing that quality assurance for you and catching the things that you didn't catch yourselves. So for example, this is I hope you like my silly design style here. I got a bit carried away.
But this is an automation and an agent that I have set up. And basically, this is something that you can make in Studio. So I have an agent that I've set up that monitors a Confluence page where people can sign up for regional design reviews with leadership. And then it does the paperwork of setting up the review template, setting up the calendar, setting up the Slack message to tell us that the review has been scheduled.
And then it ingests the transcript from the meeting, and it fills that in with the feedback. And then it closes the loop. And so this is not rocket science, but it's an annoying piece of paperwork that I don't like doing. And I am really glad to have an agent in automation supporting me on this, and this is something that is now fully democratized that other people can set up for themselves as well.
When I was doing this, and you can see it too if you have keen eyes, I found a number of UI bugs. And that's why you should dog food things because these are complicated pieces of technology that have a ton of surface area, and it's hard to catch it all just in one process. Okay. So this is a Portlandia reference.
I don't know if you've seen this skit where they're in a little shop and they decide that they're going to put a bird on it because that's just the thing that you do when you're running a knickknack shop in Portland. But I think it's really important that we think about not putting a bird on it in terms of AI, by which I mean, let's not wedge AI into every experience.
And for me, I have noticed this happen in a few ways. So one thing that has happened because, again, this all happened at light speed we noticed a ton of convergent thinking across the business. So a whole bunch of UIs just started to emerge from different teams looking basically identical. And I'll show you that in a second.
But it's interesting because not only did that was that just kind of boring and not very fun, but also customers were getting confused. They were like, oh, this this box looks like that box. Are they the same box? And they weren't. They did different things, but we had converged on the same UI. So this is a a little carousel of all of the, what will you do today?
How can I help boxes that were all popping up around different parts of the company? And as you can imagine, users were like, oh, this one made a page for me. Will this one make a page for me? No. This one is different. And so this is really interesting to see how popular that this became and how quickly that we converged on this UI across the business. And as we heard earlier with Sam's talk, sometimes that early prototype is not actually the best way to move work forward because the prototype is not the same as the PRD.
And I often notice this as a type of premature convergence where people have settled on the solution idea too fast, and this is part of the democratization of the tools. So just because you can doesn't mean you should. And I think there is a real muscle to actually being able to hold back from jumping into that solution space with that prototype spending a bit more time with the problem.
And of course, the final point here is that, just to reinforce, the models themselves, they give you an average idea. Right? They will take all of the ideas, and they will give you the average of those ideas. So if you want to do something that's exceptional or standout or different or unique, you're probably not going to get that from a prompt. And I think this convergence is a real thing to try to avoid when you're using these tools.
And it's something that we have really noticed through the process as well. To my earlier point, some of these experiences really are quite a lot harder to do agentically. So even though the chat paradigm has been super dominant, we did notice that there were a lot of moments where it wasn't appropriate and that customers were starting to actually get annoyed when we were getting in their way.
So this is a quote, hindering the default actions of basic things like column deletion, that is too frustrating. If we were trying to be clever, trying to be chat, trying to be sort of fancy with it, if you are actually stopping people from doing the simple thing that they came to do, they will get mad at you.
And I think this is a hard way to learn this, but I think it's a really important lesson to learn. This is something I've been saying. I don't know if it makes sense, but, I really think that there's a real temptation to try to make easy things easier with AI because it's a great it's it's almost like low hanging fruit.
So for example, like, let me AI, like, let me make a document for you. I know how to make a document. But I would really like it if AI could help me do, like, a complex database query and a data visualization on top of that because that is actually really hard. And we have started to see there's so many opportunity spaces in Atlassian across our many surface areas for AI natural language automation, format transfer so turn a Confluence document into a slide deck, blah blah blah, processing really large data sets.
There's a lot of difficult things that we can help people do with AI. And I think in that rapid rush towards put it on everything, there was probably a little bit too much of making the easy things easier. So for example, research very much a complex space with a lot of data to wrangle. Our researcher and our team has developed a three agent system I didn't know a good way to show this because it's basically just code but where we have a research synthesizer, an integrity auditor, a human based quality gate, and then an insights critic that actually does those reviews on the insights.
And so you provide the topic and keywords optional sources. And it outputs a verified literature review. So these are things that are manual, annoying tasks that we are leveraging this technology to help make a lot easier for us. By the way, these slides are all going to be available. They're already on the internet. You don't have to take all the photos.
It's all good. But I appreciate it. Right tool for the job. So again, this is really going back to let's not just if everything if all you have is a hammer, everything looks like a nail, this thing. We really see AI being involved in the design process at different stages.
So AI is definitely in that prototyping stage. You can bring something to life really quickly. But you can also use AI, for example, to give designers the ability to actually fix code a lot easier than they ever could before. They can be opening pull requests onto the production code base now, which is really exciting. So we are seeing both of these ends of the spectrum as well as through research, coding, product management.
But for me, there's also this really interesting human in the loop moment where humans are still involved in that critique, doing those revs, doing that back and forth, iterating on the thing. And so even though we might be using Robodev for the prototyping, we might be refining with Figma Make, we're still doing that critique as human beings. Then we might even be going back into the AI later on when we're collaborating with engineering to actually open pull requests on the front end code base.
So there's a lot of really interesting opportunities, but it is about making sure that you have the right tool when you're in the right part of the process. So sometimes, this is an interesting point for me, is you actually just need a pen. And I'm not an AI maximalist in that even making this talk, I actually started with a big prompt.
And I started collating all of my ideas and my resources. And I went back and forth on it a number of times. And then I ended up just writing it down. And I fed that back into the prompt once I had figured out the structure, and I got to where I got to. And I was actually this was the the most important part of my process, was reading what I had gotten, being really unhappy with it, having a shower, thinking it through, and realizing that I had a better structure that came out of my brain.
And we really have been careful to say this. Like, this whole deck, by the way, I've vibe coded so that it's not using slides at all. This is a mini website. And I wanted to do that to demonstrate how this has pushed the boundaries of what's possible in design. But, yeah, ultimately, I needed to use a pen to get to where I wanted to get to.
And knowing when to leverage AI is in itself a skill. I think for me, one of the biggest things is that there's almost like an instinct that you develop about seeing a process and understanding that that could be a good candidate of something to use AI with, something to automate, maybe something that you can speed up. And it's something that I think you only build over time when you get more and more familiar with the capabilities of the technology.
So these are our designs three commandments for AI transformation, which is and this is sort of our foundation. We build prototypes, we push what's possible, and we ship to production. But on top of that is really building that instinct for where those things are appropriate. And if you do this in a scattershot way, you're really not going to get anywhere with it.
So finally, getting AI to do your dishes, which if you remember this meme, I think about this all the time. This lady who is quoted in a newspaper saying, I want AI to do my laundry and dishes so that I can do art and writing, not the other way around. And that is me. That is how I feel about this.
So one of the things that's really amazing about Atlassian is we have this powerful design system called ADS, and we have won awards for it. But all of this is built on top of that. So being able to leverage our design system as an MCP to use inside FigmaMake or inside Replit or inside other coding tools like RoboDev has really taken these tools to the next level to allow us to leverage these existing resources that we've built into ADS and really supercharge them and give them this new creative surface.
So we've also added a new role into our team called design technologist. And these people are essentially almost front end and almost design. They're somewhere in between. I think there's probably a few of you here. And this has really helped us push the envelope in our foundational structures that support the design team at large. So the MCPs, the skills.
We have a prototyping repo that has things like tokenize and polish and componentize and things like that that are skills that we have written to allow other designers to get, from idea to value a lot faster. We have also transformed our content design practice. We wrote a blog about this recently.
This was actually one of the first things we did. Unsurprisingly, large language models are really good at parsing really well documented language principles. Who'd have thunk it? And so we have really strong guidance for content design. And that was a really great surface for us to train an agent who has become our content assistant agent, which you can now talk to in Slack.
And so in Slack, you can see designers coming into this channel and saying, hey, I need some content for this screen. I'm trying to achieve this. Like, what would you suggest? And they can get a really dialed in recommendation based on our content design principles that our team has developed, which is a really smart use of the technology and is already really paying dividends for the team. We've also been using it for documentation.
We almost at this stage, we have self healing documentation where if someone opens up a feedback or a request about an error in the documentation, can almost be fixed on its own, which is also really exciting and a great way for us to really scale that craft. But finally, and maybe the thing that I get the most excited about, is that I personally created a kind of second brain assistant for myself, which is basically a bunch of markdown files.
But this is how my day works now. So I have a repo on my computer. It's in Git, where I have every goal, every person, every project, every team, everything that I do, and every side project and stupid thing that I'm interested that I shouldn't have taken on because I don't have enough time. And I use this to keep me on track.
It briefs me for my day. I can take transcripts of meetings and feed it into it. I can put small ideas in and then allow it to collaborate with me. And it is entirely bespoke to me. And this is allowing me to really add that personal context layer and that specificity layer into my workflow. So what I can generate now with this is super dialed into me. It's almost like the millibot.
And this has gone sort of semi viral at the company, and and I've made a skill for other people to set up their own ones. So it interviews you about your work and your projects and goals, and it sets up a folder structure for you that you can then work in. And this is really because I am a manager.
My job is managing tons of different I've got a team of eight plus two or three that semi report to me, 180 people in the org that I'm across. There's a lot of projects. And so we have a very, very open culture with tons of information. And this is a great way to pass through that information and bring the signal out from the noise.
This is what it looks like as a graph in Obsidian. I don't know if you use Obsidian, but I've got a premise of making sure that everything is explicitly linked so the AI doesn't hallucinate anything, which kind of works. But this is sort of my second brain at work over the last few months, and it's been pretty transformational for me.
It actually changes the way that my job feels at a very visceral level. So we did it. Garbage in, garbage out. Quality being a team sport. Don't put a bird on it. Use the right tool for the job and get AI to do your dishes. I've added some further reading because these slides are online, and you're more than welcome to peruse with the link.
And that is all from me. Thank you.
People
- Rachel
- Riley
- Sam
Technologies & Tools
- Claude
- Claude Opus 5
- Codex
- Figma Make
- Gemini
- Git
- Loom
- MCP
- Obsidian
- Replit
- RoboDev
- Slack
- Zoom
Concepts & Methods
- AI Builders Week
- Design Technologist
- Dogfooding
- Garbage In, Garbage Out
- Human In The Loop
- Premature Convergence
- Second Brain
- Self-Healing Documentation
- Teamwork Graph
- Vibe Coding
Organisations & Products
- Atlassian
- Atlassian Design System
- Confluence
- JIRA
- Rovo
- Studio
Works
- Portlandia
Atlassian is rapidly pivoting into an AI-forward strategy, both in our features and our tools. I’m not going to get on stage and tell you AI is a magic technology that can do anything and everything; instead, I’ll share some of the lessons already learned as we have gone down this road —not just about the technology itself, but also about how our customers are thinking about it, how you can upskill designers at scale and how to reconcile the problematic parts with the potential for real transformative value.















