Replatform projects are hard. Certainly for Product Managers, but also for customers. Even if your development process is design-led, your products can fall flat if you fail to support your customers through change. So, what does a great onboarding experience look like?
(8-bit music) – [Lauren] So, hi out there.
All right so we know change can be hard.
Earlier in October, there was bit of an outcry on social media when Apple moved their delete button and their reply button.
They actually switched the order of them.
It’s amazing how such as small change can have such a great impact to our users.
But we know as product professionals, as designers that change is inevitable.
We have new features to deliver.
We have to adjust our layouts, we have to make things better.
But change doesn’t need to be hard for the users and it doesn’t need to be hard for the business so today Ali and I are gonna talk you through some tips that we have around how you can help your users through those types of changes and how you can help your business through that change as well.
So if we think about types of change in a product world we can put them into roughly three categories. So the first one is small incremental change. So when Google released material design we didn’t see them automatically roll out across all of their products.
We slowly started to see a drip feed down so the Roboto font, we started to see that in Gmail very slowly.
That type of change doesn’t have a lot of impact to the user it’s quite material and it’s normally aesthetic. They don’t fairly impact of it too much.
And think about it like a bedroom, it’s got my bed there, might change the doona cover so the aesthetics are changing but the overall functionality isn’t changing too much. The next type of change is a new product or feature that we introduced so we’re actually changing the user’s workflow. Think about our bedroom again we might add some blinds to the room, so now there’s a new function in the room the user can go over and they can put the blinds down, they can take them up, keep the sun in or out. When Gmail released, we had a talk about yesterday in the keynote, their auto reply feature so know when you hit reply on Gmail you get some preselected texts for you.
It’s a new feature in their product does it change to the user’s workflow? So instead of hitting reply and me just typing what I normally would say there’s some auto suggestions in there so the user might need a little bit of context as to why that’s happening or what the benefit is for them. And then the last type of change is the one that we all kind of want to avoid the one that we don’t wanna be doing and that’s a complete overhaul of a site.
That big bang so we think of our room again and we’ve know converted it into an office space. So know when I walk into the room, I’m used to putting my keys on my bedside table but now my bedside table isn’t there anymore. I’ve got my office desk there.
So it’s disorientating, don’t know where I am anymore.
When Gmail did their redesign of their product in about 2018, they had the revert back to classic button don’t remember if you guys remember that.
I distinctly remember turning that on and let leaving it on until they sunsetted that feature. So we’ve had a load of experience at Sportsbet of replatforming.
We’ve actually gone through three major replatforming projects where we replatformed our Android, Web and IOS platforms over the last four and a half years and so we’re gonna talk you through what we’ve learnt, some of the mistakes that we made, and how with our last replatform with IOS, some of the successes we had and the techniques that we used there.
– So you might be able to tell that our desktop website was just a little bit outdated. (laughs) It had been customised so many times over the years by different website authors that it had become just completely inconsistent and totally unmanageable from a design system point of view. From a technical perspective, we were maintaining three separate code bases across our website and our native apps.
And this massively impacted our speed to market. Ultimately this meant that we had kind of stopped developing for desktop and so our desktop hunters were missing out on new features.
And so we start about our replatform so dreaded big bang change and we followed the design thinking process so empathise, define, ideate, prototype and test.
And we brought in quite a lot of customers and we were getting really positive feedback and we weren’t really seeing any red flags come up. And I guess when it came to launch we had always intended to create an onboarding flow to educate users through that journey but as kind of as crunch time came and dads lines for looming, this became seen as a bit of a nice to have and it got caught from scope.
And I guess we thought you know we’re starting from a pretty, pretty low base here maybe we’ll just give it a crack and see how it goes. And so in 2018 we were very excited to launch our new desktop website.
Pretty quickly however, we realised that we had massively underestimated seven years of deeply ingrained customer behaviour.
So the feedback started floating in.
And it ranged from the fair, to the nostalgic, (audience laughs) to the ones that was actually really hard to not take personally. (laughs) (audience laughs) Basically in summer we got a pretty brutal reminder that people hate change.
– [Lauren] So as I mentioned we got another chance to try this again with our IOS replatform as most organisations was for has a really strong culture of reflect learning prove, so we sat down and we talked a lot about where did we go wrong? What do we need to look out to improve and we wanted to put some of those into action. So take two, what did we do differently this time? So we continued with usability testing however we up the frequency of how often we got customers through the door and so with our Web project we were having them roughly in a monthly.
We actually up the anti on that and had customers through every couple of weeks but we also paired that with having some standard metrics that we would measure against.
So we did a benchmark of our existing IOS app and we just used the standard SUS course and we’ve had several core journeys identified which we had task completion rates.
So as we were redesigning the new app we were able to constantly measure against that benchmark to see how we were tracking which is a really good indication to see any early warning signs and then the other things that we did which we’re gonna go through in a little bit more detail we did some priming with our highly engaged customers we went and run a pilot or a beta programme. We had a plan around how to actually onboard the massive customers and then we also had a post release plan as well and we’re gonna talk through each one of these. So priming your engaged customers.
So engaged customers for us are those customers that are on platform most frequently they can be something different in every business that might be who buys your product the most frequently, who spends the most with you but they’re the customers that if you change something you’re gonna hear from them first and they’re the ones leading and impact you the most and have impact to your business.
So what we did we got in about 30 of these customers that we had identified as our engaged customers when our app was about 85 percent complete they came in they were able to log in using their real accounts go through and actually place bets and we just got them to go through like they would organically even though it’s still a simulated environment of actually using the app and collecting their feedback. This was actually one of the text messages that were sent from a customer to one of our relationship managers that just goes to show some of the benefits of doing this with those customers so they were primed for that experience they knew what was coming and they became advocate to the product now obviously we didn’t get it quite right ’cause it was still like only eight out of 10 but you know still pretty posh.
(laughs) So the next thing we did, we ran a pilot or a beta programme and we ran this in several stages so the first thing that we did was with our staff and we ran the staff pilot and when you’re going through a replatforming project it involves many different areas of the business and we are a technology company so this affects everyone and IOS was and it is our flagship platform with over 60 percent of actives on IOS so that’s where the majority of a turnover actually comes from.
So it was important for us to take the business on the journey with us and for them to be across what were their design changes and the product changes that we were making throughout the app as we went through it so this was a great way to include staff in that. So all staff were included and we used a Facebook workbook page to be able to post up any of the bugs they found or any things that they found quite strange about the experience.
We then rolled it out to customers so we did this in three waves and we gradually increased the amount of customers that we’d actually invite.
And we had to find criteria that would allow us to move through to the next phase so for example to move from pilot one to pilot two we needed to have more features enabled so when we went live with pilot one we had about 85 percent of the features in there but pilot two we wanted to have 90 percent of the features. So the pilot gave us a lot of confidence.
This were 10 of these or experience metrics that we looked at measuring.
We did a benchmark against our classic app or our legacy app beforehand and then we had some overall satisfaction rates coming in so we had a really high benchmark for our legacy app our customers loved our old app, 96 percent which is crazy, crazy high.
Our benchmark after the last pilot sorry our satisfaction after the last pilot was 91 percent we had 92 percent of customers as part of the pilot said that they would continue to use our new app and 74 percent considered the new app a step forward so this gave us some real, real confidence. And the pilot allowed us to do a couple of other things as well.
So we were getting real feedback from customers in their real environment.
It also allowed the customer service teams to do a test run to see the type of questions they’d see coming in from customers it showed us as well when customers were deleting the pilot app and then they were switching back to our old app. We were able to contact those customers and find out why they were doing that and it also helped us as I mentioned before we only went out with about 85 percent of our features in our app for the first pilot so we were able to follow up on feedback if there were specific features that customers were looking for and that helped us prioritise what we would build next into the next pilot and have to go live.
– So the pilot I guess gave us quite a bit of confidence that we were on the right track but this time we really wanted to make sure that we nailed that on platform, onboarding as well. So we spent quite a bit of time researching, checking at best practise, looking at what other companies were doing checking at some of those companies actually that John just showed as well.
We came up with four steps that we really wanted to build in to this IOS launch.
The first one was on platform priming.
So lesson learn to the last time do not take your customers by surprise.
So you want to create an awareness that change is coming.
And we paired on platform messaging like this with email campaigns in the run up to launch. We started those comms about two weeks probably prior to the go live date and we kinda did a bit more frequency in that for more highly engaged customers.
The second one, as John mentioned was welcome. So it’s kind of obvious what you want to welcome your customers into the new experience.
This is also a really great opportunity to try and explain the why behind the change. So what has changed but more importantly, what’s in it for them.
Understands really straight forward.
It’s actually incredibly challenging to land on a solid valued proposition for projects like replatform.
We couldn’t go ahead for customers and say, Hey you know, we needed to build some tech efficiencies, we’ve recoded everything and react here you go. (laughs) So in the absence of having a whole new products at which we could go right to our customers width, we landed on the one central message of speed. We have built you a faster app and we used that in our on platform messaging and it was echoed in anything we did above the line and below the line.
The central message was speed.
Third bucket was educate and we’ve actually split this into two.
The first one ties in quite closely with my previous point about wanting to kind of explain to your customers why you’ve changed.
And I’m sure everyone can recognise this as what’s new screens.
So what’s new screens can be a really great way to tell your customers about the new products that are available to them.
However we would caveat that when we created these this time around we wrote some pretty strict guidelines around what should make it into a what’s new screen. We really tried to reserve them for product launches or like very prominent new features.
And as a general rule of thumb we kind of say You know if it wouldn’t get a marketing campaign it probably shouldn’t get into a what’s new screen. But then that leaves everything else and how do we educate about that.
So for that we chose to educate contextually. We used a tool tip approach and this really focused on educating in journey without interrupting those journeys.
So we used that for kind of smaller features that we released and also any functionality changes that had changed because of the replatform.
And then we wanted to round it all out with our final book of which was feedback.
So we wanted to build a really easy avenue for people to provide feedback into the onboarding experience.
That being said we were also quite conscious that this is a pretty intrusive pattern, a slide of model. So this was actually the only part of the onboarding flow which we chose to turn on for pilot and we actually turned it off for the full launch so we didn’t remove that functionality we still wanted feedback but we moved that into our more menu where it became more of discoverability and often piece.
– [Lauren] Cool, so post release.
So it’s really important when you are going to market you have a plan around how you actually gonna track in market so this were some pre defined metrics that we came up with as a business and this was the score card that were sent around initially this was sent around daily and then as we kind of got through those initial few weeks it was weekly then monthly and now this are done every quarter but this were some of our key business metrics and gave us some indication of how we are actually tracking in market.
The other thing we had to think about is how what are we gonna do with all of the feedback that comes in, how are we going collate it.
So we had feedback coming in through the channel that Ali just showed from the feedback model but there were also feedback coming in from customer service and from social media channels as well so that all got funnelled down through a central team the project team that was left doing kind of the swat pieces on IOS and prioritise and looked at every morning. But the one thing that was really important with that was just making sure that we didn’t have any majoric reactions and that we will looking through trends.
So when you are introducing changes especially with a replatforming project, there is a normalisation period that happens. So you significantly change the way that your app is structured and functioned and you need to let it have a little bit of time in market before you react. We actually learnt that from when we did the Web project and after the first couple of days when we released it we got a lot of feedback around the navigation is to hard, I don’t know how to find anything and there was a lot of frantic executives running around which made the designers very frantic of coming out with new solutions, we have to completely redo our IA, we got it completely wrong.
But then we notice we didn’t actually take any action we just had internal panics.
After a few weeks, we actually saw more feedback coming in and again reaching out to those engaged customers that oh I’ve got news to the navigation, it’s actually a lot more seamless that what it was before not having three separate navigation systems has made it a lot easier.
So it can be really hard but as much as you can you wanna make sure that you wait for those trends to come in before you have majoric reaction. So how did we go? So as I mentioned before our legacy IOS benchmark was crazy high the satisfaction rate at 96 percent when we released with our desktop app one month post release we were tracking at about 69 percent which wasn’t great for overall satisfaction. Our mobile web which was part of the desktop code base because it was a responsive site that was tracking slightly higher at about 80 percent but our new IOS app when we went to market with that after all they will likely done with priming customers putting onboarding in there it was actually 92 percent which was pretty amazing and since then, last month we were up at 95 percent so we’ve almost hit that benchmark that we’re originally at.
So a lot more positive feedback coming in from customers which was awesome but change is hard.
And there are still things that we need to improve on. So I’m not sure if Apple were taking queues from us or what but yes we did actually move the position of our clear button with our back confirm button and for anyone that is a designer out there that is like oh my gosh you guys are doing dark UX there the position (laughs) the position of our confirm button was actually on the left and the delete on the right which is not a standard pattern you normally have your positive action on the right and the negative one on the left so it wasn’t following convention we did to a lot of A, B testing around this one.
So some key takeaways I hope you guys got from today like change is inevitable but there are a lot of things that you can do not only for the users but for your business as well and it’s really important to take them on the journey with you.
As Ali mentioned earlier change activities they’re really easy to fall victim to scope cutting so getting buy-in from the business early is super important.
In our case we were lucky because we’ve gone through some replatforming projects beforehand so getting that buy-in was easy a bit because we had those lessons there.
But referencing other case study are best practises you know some of the ways that you can do that. And then finally just really beware of moving the delete button.
(laughs) Thank you.
(audience clapping) (8-bit music)