Thanks Andrew, and welcome everyone. So my talk today is titled From AI Survey to Reduction, What the Readiness Gap Actually Looks Like. I'm gonna be taking you on a journey which started last August.

Started with a request to talk to a private group of CEOs about AI. And they were all sort of scratching their heads wondering how to use AI. They're worried that they're going to get left behind. So, there's the workshop for which I do a pre workshop survey and I'm going to take you through that.

And then we'll go on to how one of those CEOs engages me to conduct employee survey, identify use cases, we prioritize those, take one of those use cases into pilot, and then through production. First, some statistics from various research reports.

You've probably all heard of the MIT one, that 95% of AI pilots fail to make it into production. There's a bit of controversy around that, a bit of bias in that report, but it's actually backed up by a lot of other reports. The one that I find interesting is Prosky, the Change Management Institute, where they found that sixty three percent of AI implementation changes stem from human factors, not technology.

And just to be clear on that, user proficiency accounts for 38% of AI failures. And we touch on user proficiency in the employee survey later on in this talk. I'm going to take you through the workshop that I did for these CEOs. They represent companies with annual turnover from, say, 10 mil all the way up to a billion dollars.

The largest one, an Australian company, global now, in the adhesives industry. The chap, the executive representing that company heads up AI globally. And literally, he was about to jump on a plane to go to The US and have a one week strategy session with McKinsey.

And here I come in with my three hour workshop. Hopefully, he learned something. I asked 10 questions before the survey just to get a gauge of what the CEOs were looking for and to gain a bit of background of what their priorities were. The first question was strategic objectives: revenue growth through market expansion, customer experience differentiation, market share capture from competitors, profitability improvement through operational efficiency.

So quite a range of priorities there, but generally a lot of them chasing the same goals and lots of them. Market pressures, increased price competition, customer demands for faster delivery response times, economic uncertainty affecting consumer customer spending.

Strategic focus, 50% saying sales and marketing effectiveness is their priority, also in their product innovation development, operational efficiency, customer experience and digital capabilities.

Some of them are in regulated environments, some aren't. Five weren't, but five of them are governed by some level of regulation.

In finance, APRA sent a note out to all industry participants to up their game when it comes to AI governance. It's not just enough to have an AI usage policy. You need to have guardrails, you need to be auditing for breaches and so forth, much more rigorous governance process. In terms of their AI usage, 70% basic tools, ChatGPT, Copilot, mainly by individuals. In another survey I conducted, I found that even though they had this is a financial planning business, even though they had an AI usage policy, not everyone in the organization knew about it or was adhering to it.

And when you're providing a statement of advice to clients, that's a big red flag. The technology environment. Most of them are using Microsoft, so three sixty five Azure, some Google Workspace Cloud and so forth.

In terms of their data maturity, so again self assessments, a mixed bag, but know, data scattered across multiple systems. Some centralized data with quality issues. Good data governance in most areas, and one respondent comprehensive data strategy and governance. So a bit of a mix.

I know for sure that the respondent who in the green there, comprehensive data strategy, that was not actually the case because I went on-site with that client after this workshop and that would not be my assessment of the data resident, data maturity at all. In terms of executive AI understanding, 70% have some awareness but limited strategic understanding. Leadership has limited AI awareness whatsoever.

Some of the operational challenges. This is quite interesting. So reporting people on the production floor. So is that a scheduling issue or is that, okay, here come the robots? But one respondent here, the second one, actually has a very specific thought out response to the question.

We have a significant number of staff who are creating unique codes for products to be quoted. Our conversion rate is 30%. Can we use AI to automate this process? So he's actually given this a lot of thought. And finally, for 10 executives answering this survey, what is the primary objective of the workshop?

40% saying to identify specific AI opportunities for our organization. Well, can't do that in three hours. 30 40% saying develop organizational AI strategy. So I can't do that for them either, but what I can do is deliver them some tools and a process that they could take away and apply within their organization.

So where were they? Most of them agreed they had similar goals, customer experience, market capture, increasing revenue. Where did they differ? Where did they diverge? Some had no idea, some had a comprehensive idea, infrastructure readiness, scattered data maturity, various levels.

The outlier was Anthony Collins, the MD of StarCraft. StarCraft is a reseller of high end furniture, but that's 10% of their business. 90% of their business is office fit outs. Okay? They're headquartered in Sydney. They have showrooms in every major capital city around Australia, and they have a new showroom on Bourke Street around the corner here.

During the workshop, I asked the CEOs to use the Cisco six pillar framework to evaluate their organizational readiness. So this encompassed six pillars: strategy, infrastructure, data, governance, talent and culture. Anthony Collins, the one that had the, you know, the very specific problem from that survey, where 30% of the staff 30% of the problems needed to be automated, he had conducted an employee survey.

The employee survey comprised what is your self assessed AI proficiency, how could AI improve your day to day work, and how do you think AI could improve the organization as a whole. This is across 50 employees, across five different departments. From that survey, from those qualitative responses, I was able

to

infer what the organizational readiness was from the workforce's point of view. I noticed a couple of gaps there straight away. Anthony's response is he believed that the organization was he overestimated the level of governance. I mean, there's no AI governance at this point in time. It's all data governance.

But the workforce underestimated that. They had a gap there. In terms of talent for AI adoption, he underestimated the talent, and I've seen this time and time again as well. A lot of organizations, they have people adopting AI, they're using it, they're incorporating it into their workflows to get ahead.

They're embracing it. But some people also just can't get going with AI. They don't understand it. And there's another readiness gap there. You can't just put together an AI literacy training program for everyone, because the person that is more advanced, they're going to go, oh yeah, I know this already. The person that's starting from the beginning, they're going to feel embarrassed and left behind. They're going to disengage.

So you've got to really be careful to not widen that gap. From the survey, the employee survey, 50 employees, 70 use cases were surfaced. From 50 respondents, there were two types of thinking.

There were about 20 unique use cases that were identified. But there are two lines of thinking. One was tactical thinking and the strategic thinking. The people with the strategic thinking were thinking end to end solutions. They were describing agentic solutions.

Some of them didn't have the terminology, but they were describing end to end solutions. Others were just thinking about point solutions, those tactical solutions that help them with their day to day work. The most popular use case that was surfaced was rendering of images.

So digital imaging, rendering of various furniture and so forth in a digital setting for a prospective client. The one that had the most business value though was a care and maintenance manual. Seven respondents all responded with the same unprompted response.

The care and maintenance manual So Starcraft recently fitted out the Jakarta Apple Store. And with that, it's up to, you know, 60 different products that they have to have in a technical manual. It's a care and maintenance manual, 60 different products, a technical data sheet on each product has to have all the information, dimensions and so forth, but also what materials, fabrics and so forth these products are made of. That's a requirement.

In the workshop, Anthony Collins, the MD, he had identified out of all those 70 use cases, six to go ahead with, to evaluate for the purposes of the workshop. I used the Microsoft BXT framework, which is used to evaluate Gen AI projects, but it also lacks the governance risk factor. I incorporated that into the assessment and ranked each of these six use cases. So the care and maintenance manual automation came out on top. There's an internal chat GPT capability.

This is a solution where users wanted to be able to integrate Outlook with Teams, with the ERP system. For example, have we done any business with defense in Adelaide and pull up any historical information about that. So integrating different systems, a more complicated type of solution. Product search, finance, automation, website content management system, and the visual rendering.

So the process of ranking these use cases was I applied the Microsoft BXT plus the G. And we came out with the care and maintenance manual on top without really understanding a whole lot about their infrastructure at this point in time. After the workshop, Anthony engaged me to go on-site for two weeks. Sitting next to Anthony, he was showing me all the various issues that they encounter. It became reasonably apparent to me that the care and maintenance manual was going to be the easiest one to implement.

But as we were going through the other use cases, I didn't really know much about the infrastructure. And so Antti doesn't have an on-site IT team. He outsources all the managed services to two providers. One looks after the ERP system, another one looks after all the subscriptions, the SharePoint, the Microsoft licenses, even the hardware.

While I was on-site, and I was I said, you know, let's just do a deep dive on each of these six use cases and we'll do a proper evaluation. He said, but while you're here, I want you to build something. So I said, okay, I'll see what I can do. So I ended up building the care and maintenance manual at the automation. It's Python application, lang chain workflow automation, agentic workflow.

No AI, LLM calls, because really all this is doing is read only. It's from very specific fields. I could have used an LLM API call, but I didn't want this to be a token consuming monster either. And it didn't need it, to be honest.

We've heard a lot about token maxing at this conference, so if I had adopted that, this would have been quite a costly solution. Generating a care and maintenance manual can take days to weeks. For 60 products, repeating that process time and time again, that becomes an expensive solution. So very conscious not to consume the tokens.

So I built the solution. It's doing 80% of the automation. It's working and it's great, but then we come to deploying it. Anthony, if you go to the h the headquarters in Sydney, he's got this beautiful Top Floor view over Sydney Harbour Opera House, the Harbour Bridge, Garden Island below him.

He's got music playing, super stylish environment, all this luxury furniture everywhere. He wants a solution that's shmicko, super stylish. So I'm thinking, okay, well, let's integrate this into the the SharePoint, nice little icon that everyone can click. It's easily deployed. Then comes the problem because I'm building an Azure function all of a sudden, but then I'm relying on the managed service provider to provide me with the admin access to implement that, becomes messy.

They want to charge another fee to do this. And so that's where that breaks down. Anthony at this point is becoming a bit dejected because he had this vision of what the solution would look like, And what he's got now is, you know, an executable. You double click it, you install it on every C drive. It's pretty basic. Not ideal, but he said, look, show me it in operation. So we ran it and he looked at the code producing the output lines, and he just said, this is the best thing I've ever seen.

His eyes live up. Even though it wasn't the ideal deployment, he loved the functionality. So we deployed it. That was September 2025. That's when the rollout begins in the Sydney office. About four staff members using that.

November rolls around. I get a message from Anthony. Can you come in? We need to change some of the fields in the output of the care and maintenance manual. It's a PowerPoint file. Somebody looks at that, they go over it, they produce a PDF that ships to the client. So I had to make a few minor tweaks to that one.

That's, you know, in there, in and out within a day, no problem. February rolls around, there's another enhancement request. This time, it's Cornelia, who's the senior sales support manager. She's saying, the care and maintenance manual's great, but it's not processing archive sales orders. So for active sales orders, it's fine, but for some clients, a care maintenance manual wasn't developed at all.

So we have to actually build a care and maintenance manual now from archived data. Have to work with Eagle three sixty to do some enhancements to the ERP, but that's fine. That's done in a day. Get home at half past five and Canadian sent me an email. Sydney support just completed a trial of the archived sales order to generate the CNM document and it worked perfectly.

This is great. It will save us so much time. Fantastic. Next, April 1. Another email from Cornelia. Please can you come in now and instead of just sales orders, want you to do sales quotes. So sales quotes are now part of the bidding process, tendering process.

And so another day on-site, in we go. But she's saying, we wanted to express how much of a game changer the CNN generator has been for our support efforts. Based upon our discussions yesterday, we have further requests for you. So to do those sales quotes, which is fantastic.

So what we have here is a company, they don't have all the infrastructure. It's not an ideal deployment, but given how basic that deployment is, it's something that the sales team were able to use, even though it was, you know, go to the c drive, double click on the XE, up comes the application, enter in the sales order number, hit return, it processes, it spits out a power PowerPoint file, and it's done.

Pretty simple, cuts time from days, weeks to seconds. Deployed across Sydney, Melbourne, multiple showrooms, sales staff generating manuals daily. Three enhancement cycles driven by user feedback. No IT mandate, no formal training required, no change management.

So this is not a usability story in its own right, it's a value story. The research says, if you go through McKinsey reports, BCG, so forth, MIT, KPMG, Deloitte, McKinsey, I've extracted seven of the factors that are required for a successful that they say are required for a successful pilot to production implementation.

Executive ownership is the first one. Proved from the moment that Anthony made the decision to ship that pilot. Even though he didn't like the way it was deployed, not ideal, he shipped it anyway. It's a narrow, well defined use case. There's business impact over novelty.

It wasn't the the visual rendering that was the the use case that was selected. It was something that actually had impact. It was saving enormous amount of time, freeing up the sales team to pursue sales. There's an internal product owner along the way. Cornelia, even though she wasn't discovered as an AI champion in the employee survey, she actually became the product champion.

She took ownership of that product during those various enhancement cycles. So she drove, she managed the adoption, and she came back for the requests, the enhancement requests. There's a low governance risk. This is a read only API human review. Real users and real data from day one.

So we've got data coming from the ERP system. Part of the reason it's only 80% is because not all the fields in the ERP system are actually populated. But 80% of the process is automated. It still requires that human to validate the results. But also, the other reason why this is successful is because it involved the workforce. Anthony values his people and it's evident.

He's on the floor with them every day, and I think that's one of the biggest successful contributors to this whole story is that he values his employees. And for any other leader,

if

you want AI to be successful, you really do need to engage your employees. Especially now with AI, the sphere of AI replacing jobs and so forth. Engaging your employees upfront tells a couple of things. It says you value their input, they're part of the story, and it empowers them. It reassures them that, you know, AI is not here to replace you.

So to close, three things, three takeaways for you. Number one, ask employees before the executives. Number two, find your product owner. And number three, ship something. Ship something within the constraints of your infrastructure. Don't wait for the infrastructure migration to happen.

Ship something that can work. Instead of waiting for the next enhancement to come along, I received this message a month later. So seven months after deployment that disappointed him, three enhancement cycles driven entirely by the people using it, no mandate, no training program, no change management campaign. That message is not a product success story, it's a trust story. And trust is the infrastructure that everything else runs on. Thank you.

Thank you, Chris. I've got one question as we as we kind of move on to the next speaker. You started this whole case study off as somebody engaging you to talk about AI and build something to do with AI. And then your thing you built was nothing to do with AI. We we

Well, I used Clorico to accelerate the software development process.

You did?

Secretly, wanted to use, you know, API calls and, you know, I built MCP applications with agentic workflow solutions in the past. And I was thinking about, oh yeah, love to be building that now. But honestly, I couldn't see the value in the AI LLM calls.

And