Solve for one, extend to many: Inclusive Devices, Accessories, and Augmentations

I've got a slightly, I'm lucky.

I've got a slightly different talk to give you today where I get to talk a bit more broadly about inclusive design, a beyond just the purely technical code sort of thing.

So to start off, I just want to, I tried to look it up.

I couldn't say this term anywhere on the internet.

So I'm going to try and coin it today.

The absolute front end.

So when we talk about front end design, we always talk about things like, to the screen, or maybe it's inside the browser or it's the code that we're using or the CSS and the front end development and back front end development.

But I want to extend that further now and say.

I want to talk today about how people interact with the technology itself, how people actually interact with the computers, the devices they use to interact with that, and how we should be thinking about these things and the way people want to interact with computers while we're building our front end.

A great example is like we build most things around keyboard and mouse.

And not everyone can actually use the keyboard and mouse.

And in some cases, maybe we put keyboard shortcuts in there, but what if people can't manipulate a keyboard?

So there has to be times where we consider that people might want other options.

There's things like screenwriters, narrators, narration, auto captions.

But what about things like.

Joysticks, for instance, like these are things that have been done with gaming for a long time, but we often don't consider these things when we're actually building, building the front end.

So I work for a company called Clipchamp, we're part of Microsoft, we're a video editing software inside Microsoft, and I thought I'd just drive this car completely off a cliff right at the start here and do a live demo on slide two and see how we go, just to give you an example of what I'm talking about here, because.

What happened is that I came to Web Directions last year.

It was actually the first time.

And I spoke at Web Directions there on gaming.

But I had the chance to meet some people from Intopia, who are here.

So you should chat to them.

And they introduced me to a lady who had cerebral palsy and I'd been thinking a little while about like video editing.

It's always done with a keyboard and mouse.

It's it's just what we do, especially on desktop, unless you're on your phone and I love video editing.

It was like.

This isn't right.

Like what about other people?

What if they want to edit videos?

So I was talking to this lady and I just wanted to understand some of the barriers she had using a computer.

And she said, I said, Oh, what do you like doing in your spare time?

She's I like making YouTube videos.

And I was like, haha, you're the person I actually need to talk to.

And she explained to me the biggest, some of the biggest challenges for her were like things like putting, just putting a finger on a key or pushing two keys at the same time, or even holding a mouse steady just to, drag something around.

So she needed some options.

Now I can't necessarily solve the problems that she exactly has, but it made me go back and go, okay, what can I do as engineers?

That's what you do best is you go and go, what can I do to solve this problem with what's around?

So I went home, looked around, turns out the only thing I really had was an arcade fighting stick.

So I took my arcade fighting stick, which I actually lugged here from Melbourne today.

My daughter thought I brought it to play games for this talk.

And I was like, okay, I want to try and map this somehow to Clip Champ.

So that other people can edit videos.

So I'm just going to hop out of here for a sec.

So it needs a Clip Champ.

If you're looking for Clip Champ, by the way, it's built into windows and it's also at clipchamp dot com.

So I went into here and I was like, no, just for the record, I'm actually not an engineer, so I understand front end to some degree.

Cause I do what's called technical marketing.

So I'm the guy that.

It probably gives you all your pop ups.

So sorry about that, but thanks.

Thanks for the talks that showed me how to do that more efficiently now.

So I got in, I was like, okay, I'm in Clipchamp.

What could I possibly do to help solve this problem?

And I thought, okay, we have keyboard shortcuts.

We already have keyboard shortcuts built in.

I don't have time.

This was during a hackathon week.

I don't have time to connect this to an API or figure out how to do that.

I don't have access to the backend, but I realized I could look for a program.

This is called anti micro X to try and map the control of the device that I had to the keyboard shortcuts we had to figure out how to use, Clip Champ.

And this was all just to prove that this concept of maybe using other things to edit videos was possible.

Now, so I took my trusty control and you can see here, if I move my joystick, this actually registers what's happening on the screen.

All my buttons are mapped to things, but they're all just mapped to keyboard shortcuts that are already inside the video editor.

Now if I go back into Clip Champ, fingers crossed this works.

Now tabbing is the biggest challenge I must say.

I'm actually now doing all this completely with my controller.

So I'm actually using a completely different device.

I've started editing a video at this point.

There we go.

And I can actually start going through and actually importing media into here.

So I'm just going to put this down for a second to show you that.

But what I did is I basically tried to take a proof of concept to figure out if there's other ways that people could access this technology.

And this is just something I think people need to be considering, when you're creating technology that other people may need other ways to access it.

Now just go down here.

I'm going to just bring a video in here for you just to show that this.

It does exactly what I said it would do.

I've lost my videos.

That one.

Alright.

And maybe we'll go, alright, I'm just going to grab this because I've lost my cursor.

So say I put this video in here now.

And for someone that couldn't normally use a joystick at this point, if they had like a large selection of soft touch buttons, things like, hardware devices that suit them, which I'll touch a lot more on that in a minute.

They would actually now be able to edit this video.

I can move the cursor around, I can split things, and this is all happening completely.

I can trim the ends of my video, I can select the clip that I want, I can take all that, and I can do it all inside, inside here.

Just using this joystick.

And this is just what I had at home.

And things like this are actually really, useful.

We often think we need these sort of high tech custom solutions to solve these problems, but sometimes it's really just what people have on hand is exactly what they need because it's what they're used to using.

So that was my initial foray into inclusive hardware design.

Cause that's where I find I have the most interest in that field.

But today we're really talking about like why you should be thinking about inclusive design more broadly when you're building things.

So what happened after that is I did that I made that video.

I made a video about that for a hackathon that we had, showed it at Clipchamp, come next hackathon, some actual engineers that we have working for us, that's pretty cool.

Why don't we take that?

And then we'll actually try and put native Xbox controller support into Clipchamp.

And this is exactly what they did.

They're actually doing the same thing now.

With an X Box controller connected to a computer.

It could be this one.

It could be the adaptive controller.

But the important thing here is that when we are creating solutions, not only should we be creating them and thinking about these things when we're doing them, we should be making aware of what we're doing and what we're building so that people can iterate on this and build on this, because this is not a solution that will be solved by one person.

These are solutions that will be solved by lots of people, in lots of different ways.

And they actually did this project with what's called the X Box adaptive controller, which is down here at the bottom, the white thing.

It's, a device that was built by Microsoft for the X Box to help people play games.

And what you can do is you can actually get it and connect a multitude of devices to that.

So that was them trying to make gaming inclusive for everybody.

And what was great is we could now use that to start making video editing inclusive for everybody.

But the only reason this actually worked for me is because we had keyboard shortcuts in the first place.

But maybe somewhere along the line we should have actually stopped and thought, oh, maybe we should actually make controller support, part of Clipchamp at the beginning so that there was no barrier in the first place and we didn't have to go and solve this problem.

So why should you always be thinking about inclusive design?

Now, some of these stats are pretty impressive.

So 75 percent of disabilities, actually what we call non apparent and these often get missed.

So the things like hard to access public spaces, if someone can't get into the space, you've actually created a disability for that person.

Noisy workplaces where people can't actually hear what's going on because of all the noise that's there or difficult to access emails.

If people can't open the email or figure out how to use the software, then they, don't have the same sort of access that everyone else has.

1.3 billion people worldwide actually have some form of disability.

We often stop and think about, maybe someone's missing a hand for instance, but this includes things like anxiety, autism, things like blindness and deafness that most of you would be aware of, but it also includes chronic pain.

Cerebral palsy, and things like even diabetes is a form of disability.

And I've written Scott Hanselman there.

Some of you might know Scott Hanselman.

He's the VP of developer relations at Microsoft.

He's got a podcast called Hansel Minutes if you haven't heard it, but he talks very openly about how diabetes is a disability for him in the workplace.

Cause he has to manage that and it creates barriers for him while he's at work.

And the World Health Organization believes by 2050, 3.5 billion people will require assistive technology.

This could be screen readers, speech recognition.

It could be, magnifiers for example.

But in some cases people may not currently have a disability, but as we age and become frail, we start to form disability.

So this could actually apply to many people in this room just from aging alone, which we don't often think of.

Age itself is not a disability, but it can create disabilities over time.

And I should mention too, this 1.3 billion people, the maths on that's actually 16 percent of the world's population.

So we're not talking small numbers here.

We're actually talking about a large group of people who deserve access to things the same way that you or I would.

Now one of the principles of inclusive design, this is how Microsoft defines this so first of all, it's just to recognize that exclusion exists, because we often use our own biases to measure things by.

So we have to let our biases go and just recognize that there is barriers for some people.

We have to recognize there's also a lot of diversity.

We have to learn from the diversity of the way that people use things.

And we also, solving for one and extending to many is a phrase we use a lot.

And that's talking about we often solve for one person, normally with the person who might have a permanent disability, but in the process we're actually solving for more people.

And I'm going to speak on that in a sec, which means, makes this cause so much greater.

Now before I just go too far though, I just wanted to really clarify for anyone who hadn't seen the actual definition of what a disability is.

A disability is not a personal health condition.

A disability is actually a mismatched human interaction.

And just to put that into context, say, say I had challenges with my legs where I couldn't climb stairs and I get to the building and you've built stairs and you haven't built a ramp.

The disability is not that my legs struggle to go upstairs.

The disability is the fact that you didn't put a ramp in.

It's a mismatched human condition.

So I can't get around in the environment the way that I should be able to.

The barrier is the stairs, not my legs.

Now, as humans, as I touched on before, we often evaluate ideas based on what we know.

So we'll evaluate things based on, what our friends tell us or what's happening in the environment, or, maybe the way we interact with technology in the world around us.

The problem is though, as engineers and developers, If we do that and we use our biases, we tend to end up building products where that's the baseline, which means the product's probably great for us and people like us, but in the process, sometimes we're making it more difficult for other people to use, or we're just not including certain groups of people.

And this is why we need to be really careful about our own biases when we're actually creating products in the first place.

And I'll just let this statement that I read sink in for a little minute.

So every decision we make can raise or lower barriers to participation in society.

Just take that in for a minute.

And that could apply to anything.

It could apply to engineering.

It could apply to just anything in life.

Every decision we make can raise or lower barriers to participation in society.

And when you think of things through that lens, it really makes you think, okay, I really need to think this through carefully because I, don't want to be the person putting barriers in place for other people.

I mentioned before about solving for one and extending to many, and this is, this is a great outline of why that's important and I'll get another slide about it in a minute, but there's actually different forms of disabilities.

So we often talk about permanent disabilities, which would be things like, maybe having one arm for instance, but in the same category of people, there's temporary disabilities where someone might have an arm injury.

So therefore they can't use their arm at the time.

But then we often don't take into account what's called situational, which might be say a new parent who's carrying a newborn child, for instance.

So we might develop something where someone with one arm could use it.

Maybe my joystick controller, they could manage that with one arm, but that also helps the person with the arm injury in the time that they need it.

And it also supports maybe a new parent who needs to do whatever that task is with one hand at the same time.

Another example of that might be say, hearing, you might have someone who's deaf, so they need something like auto captioning, but someone might have an ear infection, it's a temporary disability.

So the, auto captioning for them is really useful.

There's an example of this one, a bartender, he's in a bar and he can't hear what's going on.

But maybe there's captions on the TV screen.

He could still interact and understand what's going on the TV screen.

And these things apply across sight.

We might have, distracted drivers, and even things like non verbal, where people might not be able to speak, but there's laryngitis, that even included heavy accents.

Sometimes people can't understand each other because of the accents and the tools that we create for people with permanent disabilities can actually solve some problems for those groups as well.

And this is why we talk about solving for one and extending to many.

But businesses love numbers.

So here's some numbers for you.

So if we be mindful of this.

Permanent disability.

I'm pretty sure this data was measured in the US and it was a population study.

I'm not saying this is everyone.

People with a permanent, physical disability, like losing an arm, is around 26,000 people.

But, we designed a solution for these people and we solved that problem.

That's what we should be doing.

But then, in the process, there's people with temporary arm injuries, ends up being about 13 million people.

So we designed a solution for 26.

It also applied to 13 million.

Oh, and then we've got the new parents, another 8 million people, and in total, just by taking this group of 26,000 and designing a good, useful solution for them to solve a problem, we actually ended up solving a problem for 21 million people in the process.

And this would apply across all the different types of, different types of devices we might create.

And we've even seen this at Microsoft, like even the Xbox Adaptive Controller.

It was originally built so people with, severe physical disabilities could play games But there's even some people who have them on their desks that have sore wrists sometimes.

They like the big buttons, they use that to interact.

We've seen people put them under the table and use them with their feet.

And these are people who don't require them.

They just found it just took some cognitive load and pressure off them during the day and helped them out with other things.

And that's just one small example of many times we've seen this used.

We've seen it with magnifiers, we've seen it with captioning.

We know plenty of people who watch Netflix with captions now because they just think the sound's too much.

I have a video now, now this is a, this is actually a really interesting use case.

This is really what we call this solving for yourself, but somehow it extended to a lot of people in the process.

And this was just a really interesting story about a league of legends player from China about he, how he tried to build a foot pedal for himself to play league of legends on his stream for a bit of fun, and didn't realize the knock on effect that was going to have.

But the important thing here is that.

He was on his stream.

He let people know he was doing this.

So people knew this solution existed.

How cool is that?

I like, he just went out to make that for himself.

But the really key thing with that story, apart from the fact it turned into something phenomenal is about the awareness.

Like we can't build these things in silos.

We can't keep this kind of technology and ideas to ourselves.

We have to make people aware that we're trying to do these things.

An example being, even today, speaking here.

Cause people with disabilities, they're actually the true experts on, what they need and the barriers they might face, but they may not quite be able to understand the solutions or designs they might need to overcome that.

And that's where people in this room are experts in those sorts of things about designing solutions and engineering things to solve problems and overcome barriers.

Another, great example of this is physical augmentations.

So disabled people have a rich history of developing things for themselves, and especially even more so now it's 3D printers.

So if you have a 3D printer and you want to help people out, just find people who need something 3D printed that can't do it for themselves and print them devices that would actually be useful for them.

Now, a great example of that, this is the Xbox Adaptive Joystick.

This just came out.

This is so people can use the Xbox controller with one hand.

But the most important bit about this is this.

They built this entire library of toppers that can be 3D printed for free and you just jump online and do this.

So sometimes people don't need a new bit of technology, they don't need a new device, they don't need this joystick for example.

I don't know if I'm meant to not sell that, but whatever.

What they actually need is just something to augment whatever they already have.

But sometimes they don't have the tools to do that.

They may not have the software.

They may not have the 3D printer, for example, to do that.

And then at the other end of the spectrum, sometimes physical augmentation is just not required at all.

Like maybe there's already a tool that already exists that's perfect for the job, but they don't know how to connect that to the technology or software that they're using.

And this is where experts in this room would be.

And a great example of that is a company called Brailliant.

They actually make braille keyboards to read and write braille.

But they've made it, so, in this example, it's connected to her phone and she's SMSing the same as you or I would, but people may not know how to make that technology work.

And it's the job of people like us to connect these things.

So the technology existed, but putting them together was the barrier.

And I've seen people program with these.

I've seen all sorts of amazing things happen as a result of this.

And just cause it's, it's been a long day for you guys.

I've got some bonus videos just around this out as well.

So this is, I was talking before about cerebral palsy and, having things like tremors, people with Parkinson's, this software is phenomenal.

This is called SteadyMouse.

And what this is you can see that's, what's happening with the tremor.

It actually removes the tremor from the cursor and makes an estimate of what they're actually trying to do.

So you can see the actual cursor moving underneath.

And then it will take that away.

So the person is still tremoring, but this software actually removes the tremor for that.

So if you know anyone that has Parkinson's, for instance, steady mouse is a great program to look into.

And at the end of the day, that's actually a pretty simple solution to solve a very complicated problem.

And this last video I've got for you is called smile mouse.

So this uses facial recognition.

Google talked about that yesterday, which is amazing, but it actually uses smiling to click on things, which I think is just a great way to click on things anyway, I would actually probably enjoy using this myself, but this was software created, especially for people like quadriplegics, who may not be able to use any motor skills at all, below the neck.

But they can actually now highlight and use texts right on the keyboard.

And surely the action of smiling just makes you feel better during the day as well.

So these are just great solutions that have been made that you and I could probably, could be looking at doing for people to reduce barriers.

Now just to round this out, how did I get here?

Like, how did I get to this point?

I talked about Web Directions and meeting people last year.

But just, to be clear, because I'm a marketer, like this isn't, my field, this is just something I started doing on the side.

I talked about the hackathon I did, somewhere in the middle there my wife actually built a toy called my buddy boo, which was for seriously ill children that had cancer.

And it ended up solving for one and extending to many and kids with autism and diabetes and schools and else only use this toy.

So been through this process personally.

But then, did another hackathon this year and started working on a project that I can't actually talk about it because it actually, won a challenge, but it was designed specifically for children with autism, won an award for best use of AI.

But all this, is not by any means some sort of brag book.

This is just to let you know that anybody in this room is capable of inputting to this because the way I see it.

I'm not actually special.

I'm just really curious.

And I know as, engineers, as developers, as product people, we're, genuinely curious and we're great at solving problems.

So what if I, a lot of people are like, okay, how do I get started?

Especially if it's hardware, for instance, like how do I, help people that have a barrier?

The question I would always pose back to that is what if I asked you to create an app that had never been built before, what would you do?

And you'd be like, Oh, I'd go investigate this.

And I'd look for documents.

I'd, find things that are already built.

I'd go check if there's an API.

I'd go, find solutions that are existed.

I'd paste them all together and I'd make an MVP and I'd ship that out to the world on Tuesday.

It's exactly the same.

You go talk to the people that have these barriers, you find out what they need, and then we just go and find out what solutions exist.

And then we work backwards from there.

So I just want to encourage you, if anybody in this room, has any interest in this field, especially inclusive design.

Just encourage you next week or tomorrow or whenever you get home, just to, go out and find someone that has a barrier and speak to them about what they need and just see if you can start solving solutions for them as well.

And thank you everybody.