SpeakingNaturally on Dragons and other alternative navigations

Dragon switches, not witches, switches, and controlling a computer with your eyes.

Doesn't that all sound like magic?

Well, any sufficiently advanced technology does appear to be like magic, but these are actually everyday tools that people use to access the internet, to do their banking, their shopping to learn, to share things on social, everything else that people do on the internet.

What I'm talking about is assistive technology.

It's an umbrella term for any type of assistive or adaptive device, or even a rehabilitated device that people with disabilities use.

Assistive technology can be thought of as extensions or adaptations of a computer's typical inputs and outputs.

I want to zero in on a specific type of assistive technology called alternative navigation.

Lots of people have heard about screen readers, which are commonly used by people who are blind, but other people as well.

And then there's also screen magnification, whether it's just using your normal zoom on your browser or using your operating system magnification tool, but there's another category of assistive technology known as alternative navigation.

And it's used by people who can't use a mouse or keyboard with their computer or they can't tap on the screen of a smartphone.

Instead they're using alternative navigation technology to access their devices.

One of those types of technology is called Dragon Naturally Speaking, hence the title of my talk.

So we just call it a Dragon for short.

And it is a speech recognition software, works on Windows or Mac.

Although I typically see people using it on Windows machine and what it does is it takes your speech what you're saying, and it can convert that into text.

So for instance, if you're writing a document, you can dictate and have the writing done by the software for you.

And it also recognizes commands to operate a computer.

So you can talk to your computer and tell it what to do instead of using the keyboard to do that.

And it can also be used to read the content of a document.

So it does to tech to speech the same way a screen reader would as well.

So it goes two ways with Dragon and there's another type of assistive technologies specifically use for iPhones iPad and iPod touch.

And that's called voice control.

Now it sounds like a really generic name, voice control, because that's, what it does.

It control things with your voice.

And it works very similarly to Dragon, except you're just using it on your apple mobile device only.

And so it's a replacement for your normal gestures to like swipe or tap on the screen.

It uses the Siri speech recognition engine and that's available in English only.

So that's one limitation of this assistive technology.

But other than that, it is something that allows people to navigate and use their phone, use it on the go.

Of course there are considerations when you're using voice technology and you're outdoors, it might be noisy or something like that.

So always something to keep in mind there.

And we're going to see a demo of how this tool is used by someone named Tim.

Tim works for Fable, and he uses Dragon Naturally Speaking.

He used to at first to write essays when he was in school.

And then he recently started using it to navigate the web.

And by recently, I mean, I think it's like one or two years ago.

So, not that recently.

And I want you to watch this demo of him using his assistive technology.

[Tin] So Dragon is a dictation program and pretty much it's used for, I guess, people with very little hand mobility because they wouldn't be able to use the mouse or keyboard as efficiently as a person without a disabilty.

So I use that before I used to use it for only for example, writing essays or writing something really, really long.

I never used it to really navigate before.

I only started using it to navigate recently.

Like the slightly, maybe like about a year before hearing about Fable was when I just started using an aggregate not, because I didn't want to, I just didn't know.

I honestly had no idea.

I thought it was just for, you know writing long essays or something like that.

I guess for someone without a disability, why is it helpful when writing something long?

Like an essay?

Obviously using a keyboard You have 10 fingers and you can pretty much go through, you know, writing a lot of words fairly quickly someone for myself.

For example, my hands, I don't have much mobility in it, as you can see, I have like one finger that doesn't really so I type with one finger normally.

So if I am writing a very, let's say a thousand word essay or something like that, it would take me some time.

So Dragon is helpful because I can just speak.

And not only that I can you can just kind of talk through your essay and then you can kind of fix it afterwards.

You don't have to worry about the keyboard whatsoever.

And then when it comes to navigating I personally don't use it for navigating too much because I do have a mouse pad that is pretty, like I said, I use one finger, right?

Which is pretty good for controlling my mouse.

So that's not too much of an issue, but you know, sometimes I'll be.

Maybe my fingers are sore or my arm is tired.

I won't be able to do it as long as I want to.

So I would use Dragon just to navigate around the website.

The first thing I like to do when I get on a website is just to see if these labels, these links here.

So there's only links on the left and I think there is one right there.

It's a little hard to see there.

But the first thing I like to do with Dragon is to see if these are proper links that I can use my voice for.

And if I need to do that, then I need to just say, show links.

Okay.

So actually that worked.

There's five links as I can see there.

Now, the question is, are they labeled correctly?

So I'm going to assume welcome is labeled welcome.

And our studio is labeled our studio and so on and so forth.

So let's try our studio, click our studio.

Hmm.

Okay.

So down here does say our studio too.

So for things that are kind of labeled the same it doesn't really matter.

It doesn't affect Dragon whatsoever.

It just lets you pick a number.

So there's one and two there.

Let's do a one.

So choose one.

And yeah, it worked great.

It allowed me to go down to our studio.

Would you like me to play this video at all?

Or just continue with the links on the side?

[Kate] Let's see if you can play the video, give it a go.

[Tim] All right.

That's actually good that we can try that as well, because sometimes the the media buttons aren't labeled that well on most websites actually.

Show links.

Yeah.

So as I expected it, there's still four here.

The play button is not, so I'm going to try saying play anyways and then see if it works, click play.

It actually worked.

Actually, Very surprising.

I did not expect that.

That's actually a very good let's see if the pause works, click pause.

Oh, wow.

That's amazing.

That's actually really good.

So very surprising there.

Actually I want to press play again and I want to try something because when we move our mouse out of the way the the style here disappears.

So I'm wondering if we can still pause it.

If it disappears, click play, and I'm going to move my mouse to the side, and now you no longer see that pause button, right?

So I'm just going to see if it's still positive.

Click pause.

No, see it opened.

Oh, wait.

Did it work?

I think I might've worked.

I'm going to try that again.

Click pause.

I guess it doesn't work.

I think we got lucky there somehow, but yeah, I'm when this disappears, you can't pause it.

So you're going to have to move your mouse into the screen again, to get that click our classes.

Beautiful.

Click.

Get in touch.

Go to sleep.

Yeah, that was actually really good Kate, I did not expect for all the links to work so well, but that was perfect.

[Kate] Yeah.

Issue with like our studio.

They had two links that had the word, our studio.

So there they're a bit slow there to pick that first link, but yeah.

[Tim] Yeah.

But like I said, a Dragon and realizes when it's like a duplicate and they'll just give us kind of like a listing, then you can just pick a number.

So it works well too.

Okay, so let's fill in this name here.

Click name, press down, press enter, go to sleep.

Perfect.

All right, so I got the name there.

Is there a message?

You want me to write?

[Kate] Let's just do a test message?

You can just write test message.

[Tim] Okay.

Wake up, Click Message.

Choose one, this is a test message.

Period.

Go to sleep.

Now, there you go.

These were labeled actually really good.

I really liked that.

It's kind of weird that message and send message is kind of labeled the same though.

I'm just wondering if I hit send message.

Will it show both again?

[Kate] Let's find out.

[Tim] All right, let's go ahead and try to send this and see that top message.

One is also labeled the same.

Wake up, click send message.

Okay.

So go to sleep.

It looks like we need an email before we can send a message.

[Kate] Yeah, which is fine.

We're just testing out the form just to see how Dragon would fill out a form.

How do you feel about the errors here?

How clear is it what the errors are?

Okay.

[Tim] I really liked this, those errors.

Now, although I like it, I kind of this big one.

I'm just gonna assume, because I didn't write enough.

Right.

But [Kate] It's hard to see if there's an error message in there.

[Tim] Yeah.

I'm trying to see if there is....

oh there it is, you must include at least 50 characters in your message.

Yeah.

Honestly, that's not terrible.

It's pretty clear, but I feel like it should maybe be down here somewhere.

Yeah.

Only because my focus is here.

Yeah.

It's weird that I have to look up there, but I definitely like the red border around it.

This is new, definitely new to me.

I don't think I've seen one like that before.

But yeah, that's pretty good.

[Kate] we generally recommend putting the error message close to the field and also putting the word error in case people can't see red people who are color bind, or just have low vision, which doesn't affect you as a Dragon user, but it could have.

[Tim] Yeah, exactly.

I, yeah, but I do really like this form filling out the form was actually really, good.

One of the big issues with Dragon is for example, if I have this name selected and.

I started talking, for example, dictating another command, for example click message.

Instead of clicking the message, it starts writing out click message.

You know what I mean?

So I, guess it, this form works out really good because it doesn't affect my dictation whatsoever.

Wake up, wake up, click message.

Choose to.

And go to sleep.

Yeah.

See, like I can have this selected and I can just dictate perfectly fine without dragging, typing out what I'm saying.

So that's actually really good too.

[Kate] So as we saw in that demo, Tim said click on our studio and Dragon highlighted, choose several of the links on the website.

So.

It's important that links are clearly labeled and think about how many links with the same name you might have.

Oftentimes I see pages where there's tons of learn more links.

And if Tim were to say, click learn more he's going to have to go through all of those links and pick which particular learn more he wants to actually click on.

So it's always better to link the name of the article instead of having a learn more link.

And also in that demo, we saw that Dragon wasn't able to pause the video when the video controls were hidden.

So that's another thing to keep in mind with interactions are, all the controls available at all times.

And we do that really commonly with videos where we hide the controls or other things where we're hiding things, but just realize that when things are hidden, they're not always accessible in that hidden state and think carefully about how you're designing for that.

Grid 3 is another type of assistive technology, and that allows people to control a computer with eye gaze.

So it's really cool how technology has advanced so that instead of using a mouse, you can actually use your eyes to control by looking one way or another.

You don't even have to turn your head ...trying to just move my eyes here.

There we go.

And you can use that also with a head mouse, which we'll see a demo of as well.

Carolyn wears a head mouse, which she wears glasses with the little dot in the middle, so that it's looking at where that dot is so she can just move her head the way she would with the mouse.

And she also uses that with a switch system.

Grid 3 is primarily used by people who have physical mobility challenges, and they might not be able to move, you know, below their shoulders or they just might have poor motor control of their hand.

Or they may not have hands and just any kind of difficulty using a keyboard and mouse.

So Grid 3 is an alternative to that.

And it's often used with a utility, which is known as onscreen keyboard.

Let's say you need to type something and maybe voice control doesn't work for you.

Some people are non-verbal or sometimes their voice is really hard to understand, or sometimes it's just very tiring for them to speak.

So that's where an onscreen keyboard can come in handy.

[Carolyn] Now there is a field of text that needs to be entered here.

So what I could do is use my onscreen keyboard, which I call up from the menu so I could type in that way.

And it sort of prompts me.

That's usually what I would use if it's not a lot of writing, but if I wanted to use Dragon, if I had to do longer strings of text, I can show you what that looks like as well.

So with Dragon there are different commands.

Like how I wake the microphone up.

But if there's noise in the room, even like a loud fan or other people talking even quietly, it's a little hard to use.

So a quiet environment is key.

So that's again, why I would use the head mouse for doing things if I'm, you know, with other people or I want privacy and I don't want to be, you know, shouting my passwords and visa number at the computer where everyone could hear.

So I would do them using head mouse, but let's look at Dragon.

Standby, go to sleep.

I am going to make a fake name so that it doesn't prompt me because this I guess my pre selects are saved in Firefox Martha Smith, [email protected].

And if I wanted to edit this I could use Dragon as well and switch it to editing mode where I would say something like scratch that type M.

But if it's just something little like this, I would then do my onscreen keyboard and delete and change things that way.

So then I can not have to rely on voice as much and.

Scratch that Toronto Ontario.

And I cheated a little bit because I had jumped to the second bar by using my head mouse, but then it entered with Dragon.

So what I found with technology in general is it's great to have choices and it's great to have a selection and I'm lucky enough to be able to have these different switch options and voice dictation so that I can use them all together because you know, something is always going to fail.

Something's always going to freeze or the environment will be too noisy.

So it's really great to have a lot of choice at my back.

Be it, the physical sip and puff switch, the onscreen keyboard, microphone.

So all those working together makes a more enjoyable and an easier experience navigating.

Now if everything were to freeze.

So both of my onscreen keyboards both of my mouse selector tools, I could use Dragon and it's grid mode, which makes it a bit more challenging.

But if you think of it as like, okay, I'm playing a video game now that's how, you know, you can use it without getting too frustrated, so I'll show you that.

Switch to mouse grid.

Select mouse grid, scratch that, scratch that.

Mouse grid Nine four, two, two Canada.

Select that Canada.

So what mouse grid does is you would see them grid with numbers and you keep selecting the number.

Until it's almost on your target and then it will click that sort of target.

So you could continue dictating and the new updates to the iPhone also with voice dictation has that sort of numbered grid type thing, which makes smartphone navigating at least through an iPhone.

It's not easier, but another selection.

So you don't just have to rely on switches.

So again, it's great to have many options when interacting with a different technology, be it laptop or cell phone even driving a wheelchair, it's good to have, you know, backup sources for steering and those types of things.

So thanks for listening.

And I hope you learn something new.

Thanks.

Bye-bye.

[Kate] We're going to see a demo from Yvette and she uses eye-gaze technology.

So it tracks where she's looking and she's using her eyes, just like a mouse.

She has on-screen buttons that give her access to the same controls as a mouse.

There's a left click, a right.

DoubleClick and more, and she also uses an onscreen keyboard to type, but let's watch that.

[Demonstration of Yvette using eye-gaze.

There is text that appears during the demonstration.

This text is synchronised to the video in our slides tab”] The last type of assistive technology that I want to talk about is the switch system and a particular product called the tecla-e and this allows people with disabilities to control their smart devices and their entire home environment through a switch attached to their wheelchair.

And so you can send and receive emails, you know, anything on your computer or your phone browse, the web.

You can turn on your TV.

If it's a smart TV, adjust the temperature.

As long as you're a thermostat is one of the newer digital thermostat.

You could turn on or off your home alarm system.

You can control Alexa or Google home.

You can turn on lights, even if you have smart integrated lighting.So, pretty much anything that you need to do, you can do with this one switch system device called the tecla-e.

And we're going to see a demo of Carolyn using the tecla-e.

And she can talk about how it helps her in her life and gives her the freedom and independence to do all the things that she wants to do.

[Carolyn] My name is Carolyn and I'm a TECLA user.

10 years ago, I was working as a circus performer in Toronto, and there was an accident on the flying trapeze.

And unfortunately, I was paralyzed from the shoulders down.

I'm now I'm someone who lives with quadriplegia, so I have no movement or senses below my shoulders.

So in the beginning, I had to spend money on some very specific devices that would help me activate things like a cell phone or a home phone.

And it would work in your basic sense.

But again, there were some frustrations with the rise of the smartphone and tablets, and I was left with my basic flip phone and I was sort of cut off from a part of communication culture.

Just knowing that they could do this and I couldn't, and it was another sort of frustration with being injured.

My name is Mauricio Mason.

I'm a co-founder of Komado OpenLab Kamado OpenLab develops technology to improve the lives of people with disabilities.

So we have developed a hardware interface that we call the tecla shield . And the tecla shield allows someone with very limited mobility, so someone with a spinal cord injury, multiple sclerosis or cerebal palsey to control a smartphone or tablet using very simple interfaces.

This can be a big button that someone can just press with their head.

It can be a switch where they can blow into, some switches where they can blink, or it can even integrate into a wheelchair and the user can use the same control they use to drive their wheelchair to control the smartphone.

While I was at Lyndhurst spinal cord rehab.

After my injury, I met Mauricio who was the assistive technology consultant and there, he taught me how to use my desktop computer when Maricio started developing tecla.

He asked me if I wanted it to be a beta tester, which I jumped at the chance to do.

I was finally able to text I was able to check emails on the go and use different apps, like book readers and magazine readers, because I'm not able to turn pages independently.

And with the phone, I was, there's a new safety factor that it's been a sort of a positive result of being able to use tecla through any smartphone.

I feel more supported.

It's almost like tecla has my back because.

I'm able to use multiple methods of communication through just the smartphone and, tecla.

I use the phone now for work.

I've had to obviously change careers from circus.

I now work in magazine publishing and editing, and it's just this being connected to peers, to friends, to, you know, a work world that was impossible with my little flip phone in the beginning.

Yeah.

I'm happy to have this and sort of be part of the world again.

[Kate] In this next demo.

Carolyn is going to show how she uses it, the control on her wheelchair, head, rest behind her, how she uses a shoulder switch, that you can just kind of shrug and tap.

And she uses her head mouse, which is a dot attached to the middle the glasses that she wears and it detects the position of her head so that she can use her head just like a mouse.

And she's going to demonstrate how to navigate an accessible website, using all that technology she has on her wheelchair and on her glasses, but also Dragon Naturally Speaking and an onscreen keyboard.

So Carolyn is a great example of what many assistive technology users are doing, which is using multiple types of assistive technology in combination.

She is going to show how easy it is to navigate a menu on a website where the menu has been designed for that is the hover open menu, but the hover.

Doesn't require you to remain focused on the menu item, it doesn't just automatically close.

And so many menus do nowadays make it really hard for alternative navigation users.

[Carolyn] Hi, it's Carolyn again with the demo, I'm going to show you guys a bit of how I interact with the computer and the different platforms that I can navigate to and see.

So I've driven in front of my computer using my motorized wheelchair.

I'm not sure if you can see, but I do have a headrest behind me and that's the wheelchair.

I don't have any motor movement or sensation below my shoulders due to my paralysis.

So I use my wheelchair, the head controls and the shoulder switch.

So that's how I would drive around.

And that's how I would get situated in front of my computer.

I'm surrounded by many, many thermoses of tea with straws so that I can add drink independently.

If no one's around, I'm just driving up and having a sip.

That's super important for voice dictation.

When I use Dragon and I have to keep hydrated, then Dragon would be used for writing emails, reports, essays and papers, anything that has sort of long amounts of texts.

I rely on Dragon.

And then I also use a head mouse, which is situated on top of my monitor.

And I use the little glasses on my head, which are just old frames with this little reflective target in the middle, and that's how the head mouse tracks my mouse movement side to side and using, you know, the different buttons for clicking and dragging which you'll see when I share my screen.

So I will go ahead and do that.

Stand by.

All right.

So here we are.

And I'll just open Firefox, and there is a site loaded for you, as you can hopefully see, these are my mouse this is my sort of mouse grid of different icons and they give the mouse different functions.

So when I'm moving my head, you can see that my cursor moves back and forth.

And I can choose things like left, click, right, click, double click by hovering over the different buttons.

And I have two of those because as you probably know, with software things freeze, things, glitch.

So if one Row of mouse target dots, doesn't load or freezes.

Then I can use the other one.

Especially if I'm alone, then I don't have to, you know, run out into the lobby and, you know, ask concierge to help me with my computer.

So what I'll do is a little demonstration, like to set everything up.

So its out of the way, because these mouse icons do Sort of block some of the screens, so I'm constantly moving them around and you can see that this is the click and drag, so I'm moving my head back and forth and it's moving a little item back and forth as well.

So here we go, using the head mouse with the on-screen switches, you just have to hover.

You do have to keep your head very still.

So if a table has a bit of vibration to it, if it's got other technology on it, I have to make arrangements that those aren't there, because then it's really, really shaky and difficult to hover.

So I have to like, keep my head still and hover above things.

And then I'm able to click them like how I am sending pictures of this carousel forward and backwards.

So I've already seen with this, it's my nemesis, the dropdown menu.

So as you can maybe tell it's a little tricky to hover and then move down, but it's great.

Like this website is where I can move by cursor a bit away.

And the dropdown stays in place because there are so many websites where you have to hover and then scroll down without really changing your location of your head.

So I find those really, really challenging.

So this one I'm happy to see.

You can take your time to hover and then be able to make selections that way.

[Kate] Now that we seem demonstrations of alternative navigation in use.

Let's talk about what are some of the common barriers that are created when we don't design with accessibility in mind.

One of those would be having very small targets.

So those might be links that are too crowded, too close together.

It could be icons that are really small and this affects both websites, web apps that you would access on your laptop, but also anything on your mobile device, whether it's a responsive website or a native app.

People think of clickable regions as being important for tap you know, on your, phone, trying to tap a button and making sure that it's big enough that you can, you know, activate it with your fingers.

But there's also people using alternative navigation.

And one of the things that Yvette does when she's on her phone is she got a scanner and there's a line that come down the top of the screen across her device.

And she can tell it when to stop.

And then another line comes across the side and at the intersection of those two lines is the point and she can click on that point.

So now imagine if that target is small and she's got like a moving target, she got two of them and they got to line up perfectly to hit that target.

So if that target is too small, she's not going to be able to do that.

So it's not just finger sizes that we're worried about.

But even on a responsive site on a mobile device, or even on a desktop or a laptop as well.

Another consideration would be accessible names that don't match the visible label or icon that, you know, might not be clear.

People might not know how to activate that icon.

If you're controlling your computer with voice, you need to know what to say.

Imagine a gear icon.

There's so many things that could be for, it could be setting, it could be options.

Could be even called menu.

I've seen some of them called that or those random hamburger menus.

Hopefully they're labeled menu.

I think that's the most common thing people would try to call it, but keep in mind that if people don't see a label and they're trying to activate it by voice, how do they know what to say and try to make those really obvious.

Some of them are like, I've seen home icons printer icons for printing.

There might be a back forward type button, but it's always better to have the visible label with your icons or at least make sure the underlying name of the thing the aria label or whatever underlying code you're using to expose the label is something that's very obvious and it'll match visually.

The other thing people do sometimes if they try to put some hidden text.

Or a screen reader user.

And so the link that you see, the texts that you see have the link on the screen, and then the accessible name in the code are not the same.

And that can throw off a voice technology user, because they're trying to say the link, they can see what it says, and they're trying to dictate they click and they, you know, say the name of the link, but it actually has a different programmatic name, and so that command does not work.

Another consideration, carousel.

I think, nobody really wants to use carousel, but sometimes there's a lot of pressure from stakeholders or client to do so.

Another consideration though, as to why not to use carousel, if they're notoriously hard to use with alternative navigation technology.

So we've got all of these little icons, there's usually a left arrow or right arrow, three dots, you know, they're small hit target to start with.

They don't have a visible name.

So there's no easy way to figure out how to control a carousel by voice.

Some of them are auto rotating, which makes it a moving target and even harder to interact with.

So it's very challenging to make a carousel actually accessible to all different types of assistive technology.

And it's just better to avoid them entirely if you can.

Another consideration is timeouts.

Timeouts can be hard to use with assistive technology as well.

And I don't mean the timeouts where, you know, after 20 minutes on bank website, you get logged out and there's a notification that you're going to be logged out.

Or you can ask for more time.

Those are fine, but there are some smaller timeouts that happen all the time.

Like interactions that require you to move your mouse really quickly and select something like fly out menus or something like that.

There can be custom made widgets that require complex interaction and really common one toast.

Now imagine you're using a toast notification and there's an actual link in that.

That maybe it's a link to undo an action that you just made.

And you've only got three to five seconds.

How are you going to get to that link fast enough when you're using alternative navigation?

You know, your notification need to be persistent if they're important and especially if they're Interactive, they can't just disappear on their own.

Challenges with password requirements.

If you have a password on your site, it's got complex requirements for security.

You need to have a show password option.

It can be really hard using alternative navigation to Input all of the characters needed.

And if you can't see what you've put in already, just the complexity of using your onscreen keyboard and making sure you hit all the right numbers and characters and symbol without any confirmation and you might put in a password and it might work and you might have typed something wrong and not even realize.

And then all of a sudden you're going to have to reset your password.

So having a show password option is always a good idea.

Any type of action that only available through a gesture on mobile is not going to be accessible.

Not everyone can do a tap on the screen or a swipe or a pinch and a zoom.

So you need to have a button equivalent.

It's fine to provide gestures on mobile apps so that people can quickly interact, delete things, move thing, drag and drop.

But make sure you also have an accessible alternative for each of those gestures.

You're going to need to have a button that somebody using assistive technology can access as well as having the gesture.

So we all know that there's no problem with having a content below the fold, right?

Like that's generally accepted as true, except.

Not if you're an alternative navigation user, it actually is harder to scroll.

It can be very time-consuming and it's still important to put the most critical information higher up on the screen to stop people from having to do extensive amounts of scrolling, or if you have a really long screen or really long form it is worthwhile while breaking it up into separate pages for alternative navigation users.

I'd like to end with a quote from Kat Holmes who wrote the book Mismatch, where she talks about how barriers are created when we cause a mismatch between people's needs and the things that we're designing.

And she says that we need to seek out the perspective of people who are, or risk being the most excluded by a solution.

So these are what you might call the outliers.

And it's often the people who carry the greatest burden of exclusion, who also had the greatest insight into how to shift design towards inclusion.

So those people, those outliers, those alternative navigation users, they can be the key to innovation and problem solving and coming up with brilliant accessible websites and apps.