The 15-Minute Model for Optimising Device Experiences

[Music] you thank you everyone thank you very much and good morning Bristol this is important slides are up there already I'll repeat the link at the end but this is important let's talk really important for two reasons number one this will be the most insightful and interesting 15 minutes that you spend with a excel this year and nobody believes to me everyone's really skeptical at its start but wait to get to the end I'll explain why and number two I'm really pissed off because this has been going on for twenty years there's no getting any better let me explain let me take you back in time he said looking for their clicker no clicker right I like the guy sorry the clicker I need the USB key for this one by the way so what are these what are these it's just a little hint I'll explain exactly what these pound figures are when we get to the end but from some named shapes these are the three people that are involved and putting together all the stuff that you've seen their stake we've been optimized and device experience for clients know for five years and we've developed some pretty good methods that will actually work very well with qualitative research you know it's not about numbers core vs quan here it's very core make one fell in love and ran away together and live forever happily ever after right that's what this is about but let me take you back to the year 2000 cd-roms DVDs being sent in the post right and the star of mobile experiences basically on the web right back then Google was actually in beta and it had been for two or three years yes he works thank you very much Google had been beta right but I was also working for these guys right when they launched the first e-commerce website and that's the interesting thing yeah isolated range of products it joined Lewis No who the is out of joint loose oh it doesn't exist because it was a product that in the belt and spent a million pounds off and it didn't work on any but he's browsers right they actually built in using dhtml if you're interested and it didn't work on anybody's Broder's apart from the developers office right you know customers you know so this is the clap I'm talking about right let's flip forward 20 years people think oh you have a really glamorous life you get to travel to sunny places like Bristol you know you get to do all these amazing things right but actually my life if it's full of this kind of crap and you know it's almost like I'm shouldering kind of all of this misery so I've collected some stuff to share with you this is a great one right this is Norwegians website where they see to you they have a dialog box comes I see would you like us to fill in your passport details yes I would please fill in my passport details which it pulls in with some stars in it and then it tells me to off already filled the end it says you have passport numbers and valid I said but I didn't give it to you you put it in and then you tell me is up how terrible is that it gets better I was sitting at freezing freezing whooshing train station porn marine amazing right I saw this I'd vote for food child in Madeira right great I went to go in here it looks warm that looks amazing let's do it let's go to the landing page at British Airways is put on this billboard ad bear and again that right there's no holidays Normandy red no visceral imagery about the holiday I'm about to get and if a nine million Scrolls down the page I eventually find something that vaguely looks like it and then I click on that and I'm like holy oh right this is a mobile rate know hey this is not a 10 de P monitor on its side this is an eye for a look-see right see how small this stuff is right I mean I've got it in a monitor here and you they all can't read it right how you're gonna do that in the floor inch one it's just impossible when you look at the science right the touch targets are too close together and they're also too small the error rate rises exponentially when you have these kind of tiny little touch targets and people are thumb dominant these these should you're trying to get that onto door smashing things it requires precision it's impossible right pretty sure we spend a million and a half quit earnest oh wait with whom can it be me next time can I waste our money I'll do it quick up where I know pretty less therefore enter I promise right but just imagine what fingers like on that let's show it real skill the giant meat puppet slaw let's try and use the pagination control shall we slap you know but it gets worse than this when you eventually get tired to this and leave the st. you get one of these survey pop-ups read for your feedback we'd like to take your feedback up the bay rookie yeah no there isn't a yes please because it's off canvas I can't even tell them how angry I am when I'm in the middle of rage because I can't click the button oh such a shame they'll be totally unaware of that lies happened and that's my feeling about British Airways here's another one sheelane I was stumped by next thing you know you get an error message and you think whoa holy what's that this field can contain up to 12 characters what did you mean this filter so pick my password and again this field you can seat up to 12 characters and I'm like oh wait a minute so account my password I've got 13 in my passport so I have to like truncate my password because they want to see from space in the database are you kidding do you know how cheaper is these days B is a really important point about the mass this idea this fallacy this myth that password complexity makes it harder to crack it doesn't so if you te characters two numbers one special character like y0 you are mo am i saying right really hard to craft for people right but if we just take nine characters of letters no no special characters I mean the fact that no one knows what a special character is anyway you know whether thinking something in a Cell you know but nine characters actually it has three times more variants and passwords right to get to crime but this isn't how people steal passwords anyway so it's all layers of wrapped around each other why are we doing this to people it's pure evil and I have a friend who who has cognitive impairments that means he can't use the internet so I have to buy all this stuff one in today it's like I'm is internet proxy bitch right he's my friend right you know I'm happy to do it for him but because I'm buying him bus tickets everywhere I used the national experience today all the time and they have an intermittent but this is what happens you get this mean free to terms and conditions translate mean future conditions of carriage translate that's really really helpful Annina look at a landing page experience I think oh oh come on wind on the pager the most-used with a laptop holy it's just vomit you know I call this viewport stacking use real people's viewports and then you see how all the crap that they stack in there actually weeks like that's what it looks like impossible to use not landing page experiences terrible and then that waste for two months I had running conversations with not waste about their cookie handling that they keep resetting and for a two-month period I keep not using that waste on Chrome this would happen every time and I wish did hours with future I'll conversations in that waste of a chat transcript if I went through it now you'd be in tears by the end of it who wouldn't listen they don't care these do not care about me they do not care about this problem they meet me use internet explorer for two months it's not fair it's almost enough to meet me what she's change Bank you know and then you get ridiculous things like this you know auto-filled will put my phone number and like that's the travel loads of places it's in that format you mean programmers can't work out how to parse strange way come on come on plus four four I wonder what that could be maybe it could be that country could but this this is my favorite one when you're texting you're going through doing the heuristic evaluation you occasionally stop right right things dainty little noise but other times you go wait a minute were they see invalid for number format only digits only digits are allowed only only digits what would you mean only did I put all the digits in there know what they meant was I 0 is not a digit and I'm like wait a minute this is a.m. and you're having a philosophical conversation me about whether zero is a denture or not your validation rule suck man they totally did but this is a big problem so I've got a new definition for you it's called our UX bug this is where the developers and everybody swear that the cord is correct but it's still look at this one right this is a site where you sign up but when you're not signed up you get advertising would you think this blue button those behind yes the sign up for people who haven't given you any money so you're advertising this just shafted your entire business model it's great it's wonderful but this happens a lot more doors overlays chance of a pop-up smack jail where you can scroll up product image jail where you can't move their mobile page is the image seems there's no scroll bars and chrome you can't dismiss the models viewport biggest determinants of conversion rate on mobile is how your product works and small of you force trust me right and then touch target size and spacing and even simple stuff like color contrast you don't have the right contrast set up nobody can read any words on your website right so they're not going to buy anything if they can't do that this is a great one these guys have revenue of 812 million pounds a month they should know better they asked me that's really stupid question I need to frame this hey Creek on our form analytics the most edited field is a customer name can you explain why people are having to edit their names all the time and I'm like no people normally get the names by the age of three or four can I work out how to spell them and then they don't forget right what's actually happening is your developers have been really stupid right auto spelling right one of the biggest problems around on mobile one single form attribute autocorrect there's lots of places you shouldn't have it names email addresses passwords URL field search fields captures inputs we are autocomplete can compete with and shaft the other elements of your UI right this is ridiculous stuff but 80% of the ecommerce sites that beam are tested recently had the wrong keyboard type say that is the real world right I mean you get this like oh we want you to put your telephone number and but no we're not going to give you a numeric keypad we're going to ask you to spell it phonetically using less else for 9r0 as my phone number and this makes me really angry because it's the same that we were doing back 20 years ago and we're still doing it we're still introducing it into products but nobody calls up and complains about it nobody tells you so Mazal a hierarchy of needs my hierarchy of needs from a product here just starts with us the walks on customer devices that's your starting point it Lords runs quickly customers can actually read the text right and then they can use it right and the end of some other stuff up they are marketing and a whole load of other but that stuff is taken care of if you have a foundation that isn't weak right so device compatibility performance accessibility usability and then some easy on the top and if you're not doing this if you're trying attempting to deliver an ad personalization project when you have all these problems of performance and device compatibility lurking in there what you're doing is you're burning rubber in the UX car part right what happens is everybody turns up having a great time there's light there's sound of smoke everyone thinks that having an amazing time thousands of miles are clocked up on there are on the speedo but at the end of the night you're still in the car park right feels great right everyone's really happy but you haven't actually gone anywhere so we all have product defects our companies have them our clients I have them their end products everywhere right you know it's just until you have actually tested to see whether you have these problems or not you don't know it's a Schrodinger's cat situation it's just a assumption people say to me our site works on all the popular devices sure be the problem here is nobody ever calls up right or if they do call up something happens to that call and what if only one person calls what if nobody calls that doesn't mean that your site isn't broken right so you cannot rely upon customers calling you as the canary in the coalmine of the user experience right when I see problems on these sites I think oh it's just bad cord filed rendering issue right but what do customers see their lens is I see in the retail experience you have no idea how hard that is to paint a picture of you know I show up on Google at least see that's the thing it takes time these presentations searching for images of dog shake two hours away all right but it's true if you had this in a store and someone just picked this up and said nevermind just what Ranger it's okay it's normal service right but that's what it's late they don't see a cording problem or a little slip they think your site is broken defective and that you don't care about them which is generally true the other problem is is that the call center acts as the largest nappy in the world and this is the large snappy in the world and it basically absorbs all of customer problems that go there to die it's the elephant's graveyard of customer problems because they're often designed in ways where the only thing that escapes the the terminal velocity of the black caller is a call center our KPIs so you just see all these KPIs that come out of them but people don't have a proper bug hunting approach you don't have a way of recording their stuff and they're often passed on to support teams will say oh we can't reproduce your problem so it doesn't exist right the costs are huge the effort to find it is low and it's very very rare to see this people offer bounties for security by which companies would start offering bounties for bugs that are costing them money because it's the same thing manufacturing is solved this years ago you know back in these days when you're building cars if you didn't build them properly people are gonna die from it right you know it's not just product recalls it's about the management of risk to a company of quality and defects in product so lean and Six Sigma and all these things have been around for a long time but you want these kind of things in your web product right you know and you want to avoid people kind of dying when using your product which is some of the stuff we've been showing you already but you know we're not using these methodologies in terms of trying to get quality code out right there's got to be a better way but what one thing would make a difference for these companies what one thing would change all of this and stop these problems happening it would just be if they prove that stuff actually worked on devices and this is a way of doing that so what I'm gonna show you here is just a very quick tailee or all the workings for this are available on the day and there's a medium article online there's some great work that you have to do here before you start someone should have audited your Google Analytics configuration because if you're pulling data from it and it's made up of then you have a problem but 95% of all the companies that I look at have something wrong with theirs so there's a complete guideline for how to check that stuff and make sure it's actually working so you pull your numbers but what I what I'm then interested in doing is going into Google Analytics can I use Google Analytics to tell me stuff about device experience and how well it's working and what the audiences are what kind of they have in their hands right because that'll be really useful when it comes to qualitative research so I'm looking what kind of data sure to pull out here you know I want to kind of find out what the major of devices groups are but also and see which ones are performing well and poorly where do they suck do they suck on the landing page do they suck further end do they suck at the checkout and these are the things that I'm trying to pull out but this one little diagram is a really important one because this is the conceptual model of how you split the data and Google Analytics and if you do it this way it's all explained in the article you will actually get the right audience segment so if Google Analytics to work out what the performance is so when you look at Google Analytics just cement race and stuff Mobile is predominantly an Apple and Android sure right windows phones are there but it's a tiny percent for most companies like one or two percent Google Analytics will not tell you the iPhone model reliably don't look at the iPhone model and Google Analytics because it's wrong right and the article explains this but what you can do is look at screen resolution and on the Apple developer say they have a scaling of the screen geometry to the uik size that then gets passed into Google Analytics so you can see if it has that resolution then it must be an eye for an X that's really handy for splitting out the models is one of the things that we find with iPhones is that everybody belts for this right but nobody tests on these two and this phone this size the for the 325 sixty and three seven five five six six seven are almost all of your audience these are tiny right so you must test on the smaller phones because they are most representative of your audience and then when it comes to desktop it's you know Chrome and auto updating browser edge and auto updating browser Firefox and not what dating brazo and eggs well I really and in fact this has almost disappeared right ed today Explorer 11 it's very small Microsoft edge is peeking over from this and the number of browsers that are now auto updating make some waves are an awful lot easier and then or desktop max there's a couple of interests and things because you know we could just taste the latest version of Safari and Chrome some people have may have large older Safari audiences but generally everyone's getting onto here but one of the interests and things that you often see is the you know you look at a Mac laptop and just think oh that converts it six percent of them by right but on Windows it's like two percent does that mean that our Mac experience is better no because the difference in performance figures and Google Analytics should be down to the device experience they're getting but it's also influenced by the person behind the keyboard and a li really can afford the Mac these days they must have lots of money right okay so it's because of the socio demographic skew the people behind the keyboard that they normally look like they're a better experience than the Windows ones interesting and true tablets have been falling in my figures is dominated by Apple some countries have a small amount of Android and almost nothing on Windows but here's I kind of finished example if you go through the model I'm just about to mention then you can fill in all these things so that's our traffic figures we can see relatively how much sheer mobile and desktop ads and then we can see how that splits in terms of operating system underneath but these figures also can I tell us about conversion then the first leg that I have here for this client is that Apple is converting lower than Android and that almost never happens right on mobile so that's the flag I need to test these Apple devices okay and over here I have the same thing Apple iPads are converting we worse than Android tablets and that's another low flag there's all these signals to look out for but this is what you can get if you take the model of 15 minutes you can have one of these felled out for any website and you'll find the whole thing on their complete step-by-step nor the instructions which is download that custom report from Google Analytics get that excel sheet with blanks in it yeah fill in the blanks to share that and plug it in there under auto calculates so you just kind of want to meet the earlier this is just that event interesting so why is it that these Android phones almost convert at half ray Android tablets and Android phones almost convert at half or even less than than iOS and we found 13 bugs that affected almost every Android device we fixed them and it caused most of that gap right it's it's not telling you why it's just giving you a clue as to how big and where it's your job then to figure out why there is that difference small screen if-- once we see these patterns all the time you know you go great banker I four-inch to convert better but it is to do with the viewport there are critical elements of navigation overlays the first kind of view that you have and viewport with smaller screen devices so you need to taste with lower common denominator when it comes to mobile phones so a Samsung essay or an iPhone sexiest he's a really good bear I it's very representative and here's another word tech example pulled all this kind of stuff out see you're getting a map you're getting a a mental map of what customers are using and what's working well and not so well for them now strictly speaking to find out where these people are dropping out you then need to finalize the experience and there are various tools and ways of doing that they're explained in that document too but the let's then talk about me of it I've done the analysis have looked at some stuff I want to run some tests now once you run a heuristic evaluation or I want to actually recruit people who actually match this information that I've found but this is a minimum viable testing list if you buy this crap that's on here that will do almost all of it and this is true for almost every site that I work on get a small iPhone get an essay or Lane get two laptops right one set say to 1440 by 1801 1366 by 768 and some browsers on there and two tablets right you have six devices there that will cover almost all the stuff so let's imagine you you break something on Android mobile right that will break reliably across every Android or almost every Android mobile that you have if you fix it it will resolve itself across every Android mobile so there's a great deal of consistency there right you can figure this stuff out but that should be your testing rig if you if you run the site or you have clients you need to look at this stuff Luis six things will cover almost everything so here's a good example I showed to a client they said what should be in our testing license I said well you should test half of your customers with these two or if you add this then we're up to 84% and then if you add these on then we're up to 92% and I said well what will test all these four small releases and for a bigger releases will test all three of these brilliant so they know what to test now nobody's making the shell or tell the developers it's their problem to work out what to test it's not their problem they may have to do the testing but somebody else needs to figure out what this should be casting here's another interesting thing when you actually analyze the screen Heights no one's notices but screen height relative to width has been getting smaller over the last few years why is that widescreen formats it's a big a ratio of Wed today so the dates is smaller because the laptops haven't been getting bigger but what we find is that a huge amount of people have less than 900 or less than 800 pixels available right so this is incredibly important people have design in shape with these size monitors and then given it to customers with these size viewports and you need to stop this because that's why that landing page experienced knockdown later on so this client has four lanes onto the device experience and these represent you know a huge percentage of their visitors and you don't need to buy this stuff you can use cross browser testing other online services Amazon and device anywhere do rentable devices in the clothes so if I want to rent a particular Android device then I can do that if I want to rent an iPhone in Rio de Janiero and visit the Brazilian App Store and install something I can rent that device sitting in real okay so there's no excuse you can get hold to these from friends and family from your colleagues there's even open device labs if you go to open device lab comm they will actually show you where all their local device labs are and they are free right there's a lot of them all around the world so there's no excuse you need to get the lenses that customers have onto your private experience but it gets more interesting because it's a sort of my methodology for sort of baselining a site but this is interesting for you because how does all this quantitative stuff help out qualitative research well it means that you know which groups the customer devices get a raw deal right you can make sure that they come in for the testing whoa I'm sneaky right you can set up heuristic walk walkthroughs to be representative do you're not doing your heuristic walk through in a 1920 by 1080 monitor you're do not own a particular browser on a laptop because that's what your customers have you can work out the split of mobile and desktop participants and a test you can say I'm your lab kit to be representative there are all sorts of benefits in terms of recruitment and testing right and if you finalize the data you can even work out we'll maybe we should spend 40 minutes looking at the product page because that's really where everybody drops out and we would then expect to find a massive amount of problems at that step so what this stuff can help you do is it cuts down your problem - sighs halves it and how's that again you cannot usability test the whole site in an hour people say we're usability testing outside today yeah how long is that gonna take or we have people in for an hour right so you're gonna test our whole site in an hour right are you kidding you know you might get like three or four pages done in that time but which three or four pages the ones that you pick that you quite fancy or the ones that the data tells you that you should be looking at so how do we then make this a lot simpler and the answer is you don't even need to use the Excel model you can actually automate all of this stuff so what I do for a lot of clients sites now is to create a thing called a grid and I basically pull out some traffic some behavioral data from e-commerce if it's our ecommerce or engagement stuff and then sales figures and then I'll plug in some wee points like did the gate to a list page to get your product page did they get to the start of the checkout and why then do is run this through a tool that massively dimensionalize is all this and gives me a funnel for iPhones a funnel for people coming from India a funnel for people from Spain a funnel for Mac browsers and looking across that grid there's a loads of stuff that doesn't interest me at all but occasionally my I will go that's wrong that's weird we need to check that out and then we'll go and look at those three to five things rather than trying to test the worlds right it's impossible to have the kind of coverage that you want the data helps cut the problem down to size but also it means that you're actually focusing on areas that won't cover a massive problems through your client which is a good thing and here is a tool that we've built that will actually do that dimensionalization for you this is a massive travel site and we ran this great and we thought this is really weird that tablets are low converting in the middle of the booking process and it turns out that if you try and book a train ticket on a tab tablet well if you go London to Paris Paris to London that's cool but if you do Edinburgh or Bristol to London to Paris you can't boot the return journey because the JavaScript is broken on the button right it's just like and what people are doing is we thought they were losing money well they were but what they were doing is they were rebooting on desktop so you've got tens of thousands of people every month really pissed off because they've had to book twice you know what's that doing to your your satisfaction scores never mind the numbers feel the anger so it's worth the 100 keeper Montana additional sales for those guys and there's a whole load of stuff here if you can use Excel then you can automate pulling data of Google Analytics and all this stuff is free free Google sheets iodine our guide to getting started and then how to pull stuff from the Google API which is actually really easy to use I don't code and I can pull this stuff so you can too here's another example of automation and this is a this is this was the product jail one we look at yield a bit like a farmer what yields do I get from these different types of fields you know and so we take clusters of pages product pages home page people that land on brand pages category and blog and then we look how many of those then go on to actually convert you know and this tells us if there are any massive problems that particular landing routes and we were like why is that so long we don't know so we checked and we found that it was just happening on touch devices really we have this Apple touch devices went in it was all touch devices we went and you can't move the product the product page up because when you touch the image it zooms in and you know it's like when you're trying to move the map and there's this tiny a little sliver and say you'd be good like that to get it to move and the deer average basket is four thousand four hundred pounds right how much money over the losing well within one week that went up to not point nine six percent seven times the conversion is added thirty percent on to the revenue this client gets every week one bug one point and they said but nobody called us up and told us well why no they won't you have to test it right I said and the guy said me trade that one bug is actually gonna earn us more money this year than all a be tasting maybe there last year all that I be testing that was basically putting a sparkler in a dog tugged it's still stellar dog tongue oh it's highly personalized we've got a really shy experience but we've personalized that Shakespeare's so it's personally shitty for you just for you it won't do the dog and sparkle thing but this is what all of this is like to me quantitative and qualitative although it feels like a quant diagram people say to me I help my products not working help my watch isn't showing the right time on the front right and in order to be able to fix that problem I have to understand the gearing and how they how these things work inside the site which device is doing torn suck is part of that kind of gearing and traffic and every site is unique right the problem here is really difficult to solve this problem of device experience if you don't understand that and quantify and cut it down to size but there is a there is an impact from this and it's a very important point because you know maybe people aren't doing this intentionally but unintentional discrimination is still discrimination so if you're not checking whether people can actually read your text it's sufficient contrast you have people who we are corrective lenses like me then you've got a problem because what you're seeing is you know what you we don't want your money you glass wearing glasses rearing bastards we only want people with 20/20 vision on this website that's our brand value and then what a baby if you've got a Mac right got a Mac and you've got a nice a nice our Wayne white table to sit it on as you you've got loads of money we don't want your money you know high socio-demographic Mac user then okay you should be wealthy and old and using Internet Explorer 7 and win several classic cars right and we'll have loads of money but we don't want your money either because we don't like the cut of your browser right screw you and the other problem is is that a lot of people don't pay attention to Android did I enjoy various hugely by Markus the difference in from Spain to UK is quite big and if you go to the US it's even warm are two difference but what I'm serious because it gets kinda left behind you discriminate against poor people it's like say you have less money you all right the world you the world that's what it's like since we don't care for our works on your and this is the problem because that diagram that I showed you earlier on was the amount of money in millions of panes that these people were losing every month right one issue five issues all of these bugs in total on all of these websites took less than five days of developer time to fix after we tested them so what's the ROI on that huge you don't need to work it out so really the problem here is you need to understand that accessibility people being able to access your product it just massively expands your audience reach and if you don't know where your audience reaches then you're doomed because you can't solve it and there's all this device testing that you get to do especially if it's incisive it drives customer happiness and profit all that friction or what pain all that misery counts up to hundreds of thousands of hours of crap in people's lives every day and we're actually doing a social service for the world that's where our productivity is gone it's dealing with all this every day and all these applications that should work but really the most important point from stairs use these tools to end device discrimination on your site today thank you very much for your time later [Applause] you