Most AI Products Aren’t Very Good (Yet)

Hi, everyone. Thanks so much for having me. So as Steve said, I'm Amanda. I use theythem pronouns, and I work at MaineCode. We're an AI research lab in Melbourne building Matilda, our flagship AI assistant. Everything in this talk, the visuals, the ideas, were generated by our most state of the art model, Matilda.

Just kidding. But I am curious. I saw a lot of polite eye contact. How did that actually make you feel about what you were about to hear? Did you mentally start to check out? Did you internally groan? I promise that the ideas in this talk are my own and the visuals are courtesy of my Canva Pro subscription, which presumably is human made as well.

If I haven't totally lost you already, I invite you to hear a little bit about why most AI products aren't very good yet and what we can do about it. So I wanted to share a little bit about the very first time that I used Claude to help me write an email. I've used Claude, especially Claude code quite a lot.

It's really great for vibe coding and prototyping and doing side projects. I use it both for work and for my own passion projects. However, this was my very first time using Claude to help me write a professional email. I usually had used ChatGPT in the past and I decided, hey, why not give Claude a try? I was writing an email to someone who I had recently met in a professional context And my tone was slightly apologetic as I was slow to respond because I had been sick.

I wrote, thanks for bearing with me. And Claude suggested I rephrase that as, thanks for putting up with me. Which to me is a pretty insane thing to say. I would never say that. But it's interesting because you can see how the model got there. It's technically fine. It's grammatically correct and it's semantically close enough, but it's so inappropriate to the context.

It makes me sound a bit chaotic and overly emotional within the context of that email. And I think that small interaction captures something important about where AI products are right now. They're increasingly capable and increasingly useful but often missing something that's harder to measure. It was technically correct but it misunderstood the tone I was trying to strike, the relationship that was implied by the message itself, and how I wanted to present myself.

And I think that's important because those are the exact things humans use to judge whether something feels thoughtful or trustworthy or genuinely good. And the issue isn't that the models are bad. In many ways, they're extraordinary. I think we've spent the last few years assuming that if capability kept improving, product quality would naturally improve alongside of it.

And when I say this, I'm employing the royal we a bit because I also know plenty of designers and engineers and researchers and ethicists who have raised the alarm about this assumption since the beginning. But I think overall the consensus is becoming that it's increasingly not the case that increase in model capability leads to better product quality.

Which leads me to the question of what does it mean to be good? AI systems are usually evaluated through benchmarks demonstrating various capabilities such as accuracy and speed and output. A lot of the AI hype cycle revolves around the biggest and baddest, most parameters, most data products being brought to market, but humans experience products differently. A product can be intelligent and still feel slightly off. It can technically work without actually landing.

And to be clear, I'm not saying that AI products are doomed. I'm not anti AI. I also build this stuff. I'm saying that there's a gap between what AI can generate and what humans experience as thoughtful and trust worthy and good. One of the best experiences I've had with AI recently was actually outlining a course syllabus. So when I'm not doing product management at MainCode, I'm actually affiliate faculty at the University of Washington at their information school and I teach a design ethics class.

It's a really fun class. Basically every week me and a bunch of undergraduates go through a design problem where we have an ethical framework and then we apply it to the design of a different technology. So we think about like the gig economy, we think about social media, we think about generative AI. And when it comes to that class, I already know what I want to teach. And the piece of feedback that I always get from my students is that I'm a good professor except for my organization could use some work.

So it's great to use AI when you kind of already know what you want to teach. The challenge isn't expertise, but the challenge is turning this messy set of ideas that make sense in my head into something coherent enough to communicate with other people. AI was genuinely useful there because it helped me externalize and iterate on my thinking faster.

I still made important decisions about what mattered and the tone that the course should have and what kind of learning experience I wanted people to have. Similarly, I also use Figma Make all the time as a product manager. Here I have a small example from the Matilda build process where we were discussing prompt scaffolding. If you've ever used Claude you've probably encountered a situation where you ask it to do something and it says hey I need a little bit more information and you get into a Q and A with it. So we were thinking how could we have a similar disambiguation process within Matilda.

So this was super helpful for aligning on the goal of the product and the general look and feel, but I also want to be clear that we would never actually implement a pop up as this is shown right now into the product. We wouldn't ship it as is. I think experienced engineers know this too. Vibe coding can generate tons of code, but it isn't necessarily elegant or useful. This tweet reads, Cloud four just refactored my entire code base in one call.

25 tool invocations, 3,000 plus new lines, 12 brand new files. It modulized everything, broke up monoliths, cleaned up spaghetti. None of it worked, but boy was it beautiful. And I'm actually curious. When it comes to using AI to generate interfaces, can you spot the difference between which one of these is AI generated and which one of these was curated by a human?

So raise your hand if you think the one on the left was AI generated. Raise your hand if you think the one on the right was AI generated. Okay, pretty split audience. So AI was used in the development of both of these. But the one on the left that says ship code 10x faster, that was just a one shot Claude prompt saying make me a landing page. The one on the right was more thoughtfully curated by our design engineer and is also what you would see if you went to maincode.com.

So AI can generate options. It can generate screens. It can generate code. It can generate entire products. But it can't tell you which option is worth choosing. And that's where taste enters the picture because when making becomes cheap, deciding becomes expensive. AI has massively scaled production, but taste hasn't scaled alongside it. And I think that changes the role of product teams entirely. Because when making is easy, deciding is everything.

So this brings us to the topic of taste. And I know taste is like a big buzzword right now in the tech world. It kind of feels like everyone is talking about it. I think that one of the misunderstandings about taste is that people think it's additive. What do you create? What do you add to the product?

What are the features you're building? But often taste is actually subtractive, especially in the age of AI. Taste is deciding what not to automate, what not to say, and what not to build. One product that comes to mind here is that TikTok recently added an AI summary of videos if you click on the caption. So the way TikTok works, you're scrolling, you see videos, you click on the caption, then there's an AI generated summary of what you are presumably watching. This AI summary is often incorrect and it's generally not particularly useful. People actually screenshot the summaries and complain about them in the comments of these videos pretty frequently.

And in this case, it's just not really adding to the user experience. And I'm going to expose myself a little bit and show you what I get on my TikTok algorithm. Okay. So here we see a woman debuting her turkey vulture costume at a house party. I I did like this video.

But what we see is that with the AI overview it says this video captures a turkey vulture with a distinctive bald head showcasing its unique presence. So the costume fooled the AI. And I guess you could argue that making fun of a bad AI feature actually does add to the community experience of the platform, but it's definitely not in the way that was intended.

So I think the issue with this feature is that it lacks taste or discernment. Below the AI overview, if we were to keep scrolling down, would see related videos that are presumably related to the video you're currently watching. Overall, it feels like another shortcut and another entryway to keep your attention and keep you engaged on the platform.

And also, why would people want a summary of a video they're presumably coming to the platform to watch in the first place? I think there are endless examples of what a lack of taste looks like when it comes to products and putting AI into products, which kind of begs the question of how do you develop taste? One place I think about taste a lot is fashion.

And I've become increasingly interested in how something can work because it's actually a little bit off. It's a little bit wrong. There's a slight asymmetry. There's an unexpected proportion. There's tension. There's contrast. Taste isn't about perfection. It's about intentionality. And I know that this seems a little almost counterintuitive because I just said that when AI is a bit off, it's uncanny. It doesn't lead to good UX.

But now I'm saying that when humans do it, it's tasteful. It's done well. And I don't think that people are actually looking to AI to create these tasteful human like experiences. I think this is where a lot of AI products are leading themselves astray. I think people look to AI to automate some of the tedium of their decision making so that they have more time to be creative and create those human like experiences.

So for example, let's say I'm a fashion stylist coming to an AI tool to generate an outfit using wrong shoe theory. Which just to go back briefly, essentially wrong shoe theory is the idea that you could have a cohesive outfit and take the shoe and make it not quite work with the rest of the outfit. And then it just elevates and adds a little bit more interest to what you're wearing.

So let's say a stylist is coming to an AI tool to do that. They probably don't want AI to generate the outfit for them. And with current models, I actually don't think that it would necessarily do a great job at this. I think that people want AI to be smart enough to give different variations of outfits within the same colorway, with the same silhouette, maybe varying whether it's a chunky platform or a more minimalistic sleek kind of shoe. But they don't want AI to be doing the choosing on their behalf.

And I think that one of the key aspects of taste in UX is always anchoring on leaving creative work to the humans and using AI to automate the tedium. As long as we keep using AI to create a human like experience, I think we'll live in this uncanny valley. Instead, humans should decide what tasteful experience looks like because I don't think you can automate human touch out of that process.

There's a Japanese concept called wabi sabi, which is the appreciation of imperfection. A cracked ceramic bowl repaired with gold isn't beautiful despite the crack. Beauty. And I don't think humans just tolerate imperfection. We often interpret it as evidence of humanity, which is why a slightly awkward outfit can feel stylish or a handwritten note can feel more personal or a rough edge can feel a little bit more authentic. The difference isn't perfection, it's intention.

So the scenarios I've spoken about so far are basically assuming that someone is coming in with their own predefined idea of what taste is or their own sense of taste, which leads to a more fundamental question of how does one develop that sense of taste in the first place. Taste and accompanying inspiration are often described as mysterious and serendipitous and almost like a gravitational pull.

However, I don't think taste is inherent or automatically emerges. It's developed through exposure, comparison, critique, and most importantly, noticing. One of the places I think about this the most is social media. You can see from this presentation that I'm a little bit chronically online. But here I've pulled together a very small lookbook that is from, like, my TikTok, my Pinterest, my Instagram of things that keep me inspired. I think it's through exposure to other people's opinions and tastes and experiences that we start to develop our own.

And now I'd like to do a group exercise where we can all flex our taste muscles a little bit because taste requires contrast and you can't develop taste in an echo chamber. Okay. So between these two chairs, which one do you like better? It doesn't have to necessarily be which one you want in your house or which one you want to sit on, but which one do you prefer? So hands for option a?

Hands for option b? Alright. We're going with the comfy chair. A lot of folks are. Alright. So those are actually sourced through Cosmos and Pinterest, those two chairs. So now I'm asking about the platforms themselves. How many people would prefer, just based on the landing page alone, option a, Pinterest?

How many people prefer Cosmos? All right. Next we have two magazines that are about reused and reloved clothing and other house goods. We have reloved and we have mildew. So how many people would prefer option a? Option b? I agree with option b in this case.

That's my own taste coming out. All right, and then which one do you prefer better here? We have very similar lists of make a lot of money, help a lot of people, and have a lot of fun. Would you prefer option A or option B? Interesting. So option B is actually an AI generated version of option A that I ran through.

I think it was chat GBT. Yeah. So So something interesting just happened. Everyone tended to have an opinion. You could tell which one you preferred, which one felt more human. And you may not have agreed, but that's actually the point because I don't think taste is about consensus. Taste is about having a point of view.

And developing taste requires reps. Exposure gives you range. Comparison gives you contrast. Critique gives you language. And iteration gives you judgment. And as I mentioned earlier, when I'm not doing UXR or PM work at MaineCode, I'm also a professor at the University of Washington in Seattle. So one of the ways that the Wait, can I make this play?

Oh, okay. Never mind. There is a video, but that's okay. So in our class, every week we have design studios. And in the design studios, essentially students have a challenge of how can you take this ethical framework we're working through like utilitarianism and apply it to the design of a technology. What would a utilitarian social media platform look like?

And increasingly, students are using Figma Make and other AI generated tools to create their prototypes. And while this vastly increases the speed of their output, I really worry about the discernment and the eye for detail that these students are developing or not developing. So I have a few things that I'm cooking up for the next time I teach this class in order to combat that.

And I wanted to share with you in case it's something that you could also take away from this talk today. The first thing that I'm working on is doing paired critiques. I think that the inherent power structure of a student and a professor and someone being there to learn or get a grade versus someone being there to teach, it's just not like a perfect motivational structure. So I can understand students wanting to take the easiest path to get the A.

I think that becomes a little bit more there becomes a little bit more accountability when you ask students to then critique each other. So what I'm going to be doing is asking students to pair up and critique each other's work and give them a framework for how to do so. The next thing I'm working on is what I'm calling digital diaries.

So I want students to essentially notice 50 instances of dark patterns in design to understand where are they being manipulated as they're using different platforms. Platforms. What is trying to undermine their sense of agency as they're using different tech? I think another way that I'm considering implementing this is how do you make sense of the AI features that you come across on a day to day basis?

Where does this work? Where does it not work? Where does it feel forced or uncanny? What makes aspects of the work that you're trying to do more easy? What makes aspects of the work that you're trying to do actually too easy where there's too much friction removed. I think that curiosity and exposure meeting discernment is what will build judgment and taste. And I think another much more simple way to say this is that taste is a muscle built through time and effort. I just don't think there's any way around the fact that you have to put in, you could call it reps, you could call it curiosity, you could call it exposure, but you have to put in time and effort.

AI can compress production time a lot but it can't compress all the reps required to develop discernment. And that's one of the things I worry about when it comes to being a professor. It can be frustrating to develop taste because I think that usually you can learn to recognize quality before you learn to produce it yourself. And there exists this really uncomfortable moment where you have the eye before you have the skill.

And that can be really discouraging because everything you make falls short of the standard that you've set for yourself and you know what good looks like. But I think that's where the growth really happens. The gap isn't evidence that you lack taste. The gap is evidence that your taste is developing. And one of the things I worry about with AI is it makes that gap less visible because if a system can generate something that looks polished immediately, we may never really experience the discomfort of understanding why our own work just isn't quite there yet.

And while that discomfort isn't a fun time, it's often where judgment is formed. So that's to say I don't think that AI itself is the risk. It's the temptation to outsource judgment through the use of AI, which AI products currently make very seamless. We now have a tool that can become a crutch so much so that our taste muscles can atrophy if we're not careful.

So that's why I believe AI should augment human creativity, not replace it. Tasteful experiences will never be fully AI generated, and remaining curious and applying judgment is the key to maintaining taste. The danger isn't that AI makes bad decisions, but the danger is that we stop making decisions ourselves. So if the risk isn't AI itself but our willingness to outsource judgment to it, then I think the question becomes, what is the responsibility of the product?

Because every AI product is making a choice. A choice about how much thinking it does on behalf of the user, a choice about how much agency the user retains, And my best AI experiences haven't replaced my judgment. They've supported it. And they've reduced friction enough so that I could focus on the decisions that truly matter. And to me, that's what good AI UX looks like.

Not removing humans from the process, but helping them stay meaningfully involved in it. And as I think we all know, AI really is fundamentally changing where design happens. Whereas traditional software asks users to navigate systems through buttons and menus and flows and predictable outcomes, where the interface teaches you how the system works, AI systems are different.

Increasingly, the interaction challenge is not navigating the interface. It's expressing intent clearly enough for the system to interpret. Whereas traditional UX tries to reduce ambiguity, AI UX often introduces it. Users are not just learning an interface anymore. They're learning how a model interprets language and context and goals. They're building relationships with the product. Jacob Nielsen recently described the shift as moving toward intent UX, and I think that framing is really useful because we're not just designing pathways through software. We're designing systems that help people articulate what they actually want. And this changes the user experience very fundamentally.

Traditional UX asks users to navigate systems. You learn where things are and what buttons do and what pathways exist. But AI systems are different. Users are increasingly refining intent, recalibrating expectations, experimenting with phrasing, and trying to understand what the system is actually capable of, which is a very different kind of cognitive load.

And honestly, I think we underestimate how difficult this actually is. One thing that we've thought about as we built Matilda is that it's materially harder to express intent through text than through selection. Traditional interfaces are reducing ambiguity. AI systems are introducing it. So when we use something even like Google Search or AI Airbnb or Spotify, the interface helps to narrow possibility.

It helps to provide you a pathway of where to go next. It guides people towards decisions. But an empty prompt box asks people to articulate goals, communicate context, and anticipate outcomes clearly enough for a probabilistic system to interpret, which is a pretty difficult cognitive task. And I think sometimes we mistake open endedness for empowerment.

But infinite possibility can also become a cognitive burden, and most people actually don't want infinite possibility. They want help getting to what they mean. So there's a huge amount of psychology research that shows that humans are not especially good at navigating endless choice. There's decision fatigue, cognitive overload, choice paralysis.

One of the most famous examples is the jam study by Younger and Lepper. When people were presented with more jam options, they were initially more attracted, but they were significantly less likely to make a decision and purchase any particular jam. And I think we've all had this experience, spending longer choosing something on Netflix and actually watching it.

There's an influencer who I follow who actually said that her favorite date night is just scrolling the Netflix homepage without making any decisions and she described it as what a rush. More possibility does not automatically create a better experience. In fact, too much possibility often increases anxiety and hesitation and cognitive effort, which I think is one of the central tensions in AI UX. These systems are incredibly open ended, but we need structure in order to think clearly, which means good AI UX isn't just about maximizing freedom. It's about creating meaningful constraints.

So I want to play another game. It's called would you rather. So with your hands, would you rather a, a purse perfectly personalized infinite Pinterest feed or a trusted friend give you three specific recommendations? Hands for a. No one.

Hands for b. Oh my goodness. Okay. Would you rather a restaurant with a 40 page menu or a tiny restaurant with six things done extremely well? Hands for a? A few people. Very brave. Hands for b. Okay. Would you rather an infinite AI generated learning content platform or b, a structured syllabus from someone who deeply understands the topic?

I am biased on this one. Would you rather a or b? Okay. And last one, would you rather an AI automatically reply to all your messages or more time and energy to thoughtfully reply yourself? Hands for A? I love the honesty.

Hands for B? Great. So what's interesting about these examples is that most of us didn't choose the option with the most possibilities. We chose the option with the more curated judgment. The trusted friend, the chef with the six excellent dishes, the teacher who thoughtfully structured a syllabus, the person who still gets to decide what they want to say.

Because humans often want or because what humans want often is an infinite possibility, but it's guidance, it's curation, and it's someone or something helping us navigate complexity. And I think that's one of the central tensions in AI UX. The goal isn't to maximize choice. It's helping people arrive at better choices, Which leads to the question of how do you guide users without over constraining them? Because users need support, but they also need agency.

And if infinite possibility creates a cognitive burden, then good AI UX has to create structure. But there's a difference between guiding someone and doing the thinking for them. And honestly, I don't think people want AI to replace their judgment. I think they want help exercising it. The best AI experiences I've had didn't lead me to surrender my decision making. They reduced the cognitive friction and overhead so I had more energy left for the things that I truly cared about.

And I think this is an important distinction because automation is not always the same thing as empowerment, and removing too much friction can remove intentionality too. Good AI systems shouldn't just be optimizing for speed or efficiency or endless generation. They should help people remain meaningfully involved in decisions Because once users stop exercising judgment, their taste erodes too.

And I think that's why so many AI products currently feel a little bit hollow. Another thing I've started noticing is that a lot of AI products are beginning to sound the same. Not just in what they can do but in how they behave. They're very affirming, very agreeable, very eager, everything is a great question, fantastic, exactly right. And individually none of these phrases are on their own a big deal.

Sometimes they're useful. But when AI has the same endlessly supportive personality it starts to create this really strange flattening effect. The product may be capable, but it just doesn't feel like it understands your situation. It doesn't feel like it has judgment. It doesn't feel like it understands the moment that needs encouragement versus the moment that needs precision, the moment that needs pushback.

And this matters because personality is not just decoration in an AI product. It's part of the interface. It shapes how users trust the system, how much they defer to it, and how much they feel like they're collaborating with it versus being managed by it. And if every AI assistant defaults to the same relentlessly agreeable mode, then we're not really designing personality, we're inheriting one.

Which leads to a bigger product question of should AI always try to do more? A lot of AI product design right now is still organized around maximizing capability. Generate more, continue the interaction, offer infinite flexibility, automate wherever possible, always assist. And again, it's not inherently bad, but the capability is the reason these systems are powerful.

But I think the next stage of AI product design requires a different instinct as well, which is restraint. Sometimes the better product decision is not to generate more but to curate more. Not to keep the interaction going forever but to create closure. And not to offer infinite possibility but to provide guided clarity.

I think the thing that's easy to miss is that restraint is not a lack of ambition when it comes to product. It's a product decision. It's the difference between asking what can the system do and what should the system do. And that should question is where taste and trust and judgment come in. Because if an AI product has no restraint, it can quickly become overwhelming, overconfident, or just kind of exhausting.

So the question becomes what would it look like to design not just for engagement but for the right kind of disengagement? This sounds counterintuitive because so much product design has historically been about engagement. More time in the product, more interaction, more prompts, more loops, more things to click and generate and automate and refine. But I think that good AI products sometimes need to design for disengagement. By that, I don't mean making the product less useful.

I mean helping the user get to the point where they can leave with more clarity than when they arrived with. Sometimes the best AI experience is not the one that keeps you chatting forever. It's the one that helps you make a decision, finish the thought, send the email, understand the trade off, or move forward. It knows when to stop.

And that's a really different design posture because the goal is not just to keep the user in conversation with the system. The goal is to help the user return to their own work and their own life with less cognitive overload. That is the part of what we mean when we talk about agency, Not just giving people infinite options, but helping them arrive somewhere.

So my point here is not that AI products should do less. It's that doing more is not automatically the same as helping more. AI changes the shape of the user experience. Instead of just navigating a fixed interface, users are negotiating with an interpretive system. They have to express intent, clarify ambiguity, make choices, and figure out even what they want while they're using the product.

And that means good AI UX has a different responsibility. It should reduce ambiguity, not just add possibility. It should guide decisions, not replace judgment. And it should use personality intentionally rather than defaulting to the same endlessly agreeable assistant voice. And it should know when the job is done. Because sometimes the best AI experience is not the one that keeps the conversation going forever.

It's the one that helps someone arrive at clarity and leave with enough confidence to act. That's what I mean when I say good AI UX supports human agency. It's not just about giving people more options, automation. It's about helping them think clearly, make better decisions, and return to their work with less cognitive load.

Up until now, I've been mostly talking about AI products at the interaction level. But increasingly, I don't think these are just UX decisions. They're cultural ones because taste is not neutral. One thing I think we often assume is that AI is reflecting our taste back to us. But increasingly, I think it's also doing something else.

It's producing taste. And I don't mean that in some sort of mystical way. I mean that we I mean that the systems we use repeatedly start to shape what feels normal and polished and professional, beautiful, or desirable. I noticed this with TikTok. When I started using it, my feed already reflected things that I liked, like fashion and design and architecture.

But after a while, I noticed something strange. My feed wasn't just showing me what I already liked. It was teaching me what to like. It was shaping my preferences. It was influencing what I paid attention to, what I found beautiful, and what I found interesting. And AI systems work in a similar way. When enough people are writing and designing and making decisions through the same systems, those systems don't just reflect culture.

They start to shape it. Because taste isn't only formed by what we choose, it's formed by what we're repeatedly exposed to. But increasingly, AI is becoming one of the most powerful engines of exposure we've ever built. And you can already see this happening in a very visible way online. If you've spent enough time on LinkedIn recently this is corporate Memphis illustration, which some of you may recognize.

But if you've spent enough time on LinkedIn recently, you've probably developed an instinct for AI generated writing. There's the enthusiastic opener, something catchy. There's the three line paragraph spacing, the inspirational takeaway, the rocket ship emoji. I make fun of LinkedIn voice with my friends all the time, and you know the format. Here's what x y z taught me about b to b sales.

It was already pretty insufferable before AI, but now it's become pretty laughable. And I say this as someone who is absolutely guilty of using LinkedIn voice on LinkedIn. I am not above it. I'm forever tempted honestly to see how absurd I can be without it becoming too obvious. It's my guilty pleasure, and I'm I'm not above it.

But I actually think this is a useful example because the issue isn't that AI generated writing is necessarily bad. Sometimes it's fine. But the issue is that AI often produces average taste. It pulls everyone towards the center. It smooths out the weirdness, the specificity, the friction, the personality. And when people use the same systems to write, pitch, design, and decide, everyone starts converging towards the same version of good, which is not terrible. It's just the same.

And that's where this becomes bigger than style because what counts as good or professional or helpful isn't neutral. What feels intuitive or trustworthy or professional or polite is not universal, and that means AI systems are not neutral. They encode assumptions about communication and behavior, and some of those assumptions are obvious and others are subtle.

How direct should an answer be? How enthusiastic should an assistant sound? What counts as concise? What counts as rude? What counts as confident? What counts as helpful? There are cultural questions as much as there are product questions. And increasingly, these systems are becoming interfaces to work, learning, communication, and decision making. So whose assumptions get embedded into those systems really matters.

Because if AI is producing taste and taste is cultural, then AI products are not just shaping individual outputs. They're shaping defaults. They're shaping norms. They're starting to shape what feels natural, which also raises the question of who gets to define what's helpful. Whose communication norms become defaults? Whose values become embedded into systems?

When interaction models become globally standardized, culture can flatten too, and I think product teams have a responsibility to think critically about that. And increasingly, trust in AI systems is not a UX problem. It is becoming geopolitical. Honestly, I think a lot of us use these AI products without fully trusting them. Most AI mainstream or most mainstream AI systems today are deeply tied to US infrastructure, US companies, and increasingly US political systems. And as political conditions shift, people are starting to think differently about sovereignty, governance, and where their data actually lives.

One of the reasons that local AI systems matter is not nationalism, it's agency. The ability to make different choices about what gets optimized, what gets protected, what gets localized, and what kind of assumptions are built into the product. And that's the context for Matilda. Matilda is MainCode's flagship AI assistant.

At the simplest level, it's an assistant for everyday work, writing, research, thinking through ideas, and making decisions. We've recently opened access to a small group of users for testing and feedback, So we're still early, but the product is being shaped by a pretty specific point of view here in Australia. For us, Matilda comes from the belief that AI products are not just model wrappers. They're full of product decisions.

Again, thinking about what does helpful mean? How much should AI disagree? How much friction should remain? What should it remember about you? When should it challenge you, and when should it just get out of the way? These are product questions, and increasingly, I think these product questions matter just as much, if not more, than whether our benchmark has moved by a few percentage points.

Because if taste is cultural and trust is cultural, then building an AI product is not just about maximizing capability. It's about making intentional choices. One of the biggest choices we've made with Matilda is data sovereignty. Matilda is made in Australia, built on Australian infrastructure, and designed to run on Australian shores. We're not just putting an Australian interface on top of someone else's system. We own the stack end to end.

The cluster, the infrastructure, the deployment, the model environment, and the product experience, metal to model. That matters because sovereignty isn't just a nice story. It changes what you can actually control, where data lives, how the system is governed, what assumptions are built in, and how much ownership you really have over the thing you're asking people to trust.

So when we talk about Matilda as an Australian assistant, we mean an AI product built with full stack ownership and a particular point of view about trust and governance and control. So all that's to say, I don't think the future of AI design is just better models or faster generation or more automation.

I think the real challenge is judgment and discernment and intentionality and restraint. Deciding what matters, deciding what feels human, what earns trust, and what should exist at all. Because in a world where almost anything can be generated, the real differentiator becomes judgment. So what do we do? If most AI products aren't very good yet, what can we take away from this? I don't think the answer is better benchmarks or more parameters or faster generation.

But I think the good news is that the problems we've talked about today are design problems and product problems and judgment problems, which means as hard as they may be, they're solvable. The first thing that we can do is design for taste, not just capability. One of the questions I think product teams need to ask more often is not what can AI do, but what should AI do? Because AI makes it incredibly easy to add features.

There are so many ways to add summaries or recommendations or generated content, automation, assistance, just everywhere. But good products are defined by what they leave out just as much by what they include, And taste is deciding what deserves to exist in the first place. The second thing we can do is design for agency, not just automation. The best AI experiences I've had don't replace my judgment.

They support it. They reduce friction so I can focus on decisions that actually matter because automation isn't automatically empowerment. Sometimes removing all friction removes intentionality too. And I think good AI products should help people exercise judgment rather than surrender it. And finally, I think we need to design for culture, not just scale. Because what feels helpful and trustworthy and professional or polite is not universal. Every AI product encodes assumptions about communication and behavior.

And increasingly those assumptions are shaping how people learn and work and communicate with each other. So I don't think the goal is to create one universal version of helpfulness. I think the goal is to build systems that remain aware, culturally specific, and intentionally authored. So, ultimately, I hope there are three things that people can take away from this. The first is to design for taste, not just capability. AI systems are getting dramatically more powerful, and that matters.

But capability alone does not create good product. Good experiences still require judgment, discernment, and intentionality. They require someone deciding not just what the system can do, but what the system should do, and what to leave out and when to stop. Second, design for agency, not just automation. I don't think the best AI products are the ones that automate human effort away. The best ones help people think more clearly.

They reduce friction so that people have more time for the decisions that they want to be making. They guide without over constraining, and they support judgment rather than quietly replacing it. And finally, design for culture, not just scale. Because helpfulness is not universal, trust is not universal. What feels professional, polite, intuitive, or useful depends on context.

So as AI systems become interfaces to work, learning, communication, and decision making, the assumptions we embed into them really matter. We're not just designing interfaces anymore. We're designing guidance, interpretation, agency, and systems of judgment. And I think that's much more interesting than simply asking how capable the model can become.

So when I say most AI products aren't very good yet, I don't mean the technology isn't impressive. It is. The pace of capability growth over the last few years has been extraordinary. But I think we're still early in understanding how to design these systems well, how to create meaningful guidance, and how to preserve human intentionality inside a highly generative system.

Because the challenge is no longer just to make things functional. It's deciding what deserves attention, what earns trust, what feels human, and what's actually worth making. With that, I'm Amanda at MaineCode. Thanks so much for having me, or as Claude would say, thanks for putting up with me.

Thank you. We have time for some questions before we move on to our next speaker. Stephanie, does anyone have a question for Amanda? Yes. Hang on. I'll bring the mic and then people at home can hear it.

Thanks, Amanda. Beautiful presentation. Amazing storytelling. I loved it.

Thank you.

My name is Asma and

I wanted to ask a question. You know, you showed some of the phrases which, you know, any of the AI, Gen AI platform would use constantly agreeing with you. I've kind of trained my platform to not agree with me every time. And I'll give you an example. So I'm exploring to consider more relocating to European countries

and I

wanted to actually see which visa categories will work best for me and how do I present my portfolio so it gets accepted mainly for The UK. So I was pushing it that I think my portfolio works for a specific visa category and it continued to tell me no and it continued to give me reasons for why my profile will not be accepted and my application will be rejected.

And I really really enjoyed that process. Any thoughts on that you know when sort of AI platforms are disagreeing or presenting you a different perspective?

Yeah, I think that's such a cool example of training your AI to really work for you. I will say one of the things we're thinking about a lot is how like a chat GPT or a Claude, like the way that they kind of optimize for engagement is to get you to answer their follow-up questions and continue to have the conversation.

One of the things that we're thinking about is how can you decide when something is done and how can you just give the user what they want and allow them to leave the platform. I think that this even what you're talking about has this, like, engagement model where you have to keep asking them like, is this right, is this right, and they'll just continue to say no.

So it's almost a similar engagement model but just with this negative twist to it which is interesting. I would love to see AI actually give you all of the information you need up front and then let you exit platform. Yeah. Thank you. Thank you.