Responsible Intelligence: Ethics, AI, and Equitable Product Design

Thank you so much, John. So it was earlier this year that I was sitting on a panel about accessibility, inclusion, and AI. And someone who was on the panel uttered the words, AI is ableist. And I came away and I sort of started to to think about that. And it made me reflect on what ableism actually is. We often think about ableism as discrimination or prejudice towards people with disabilities. And that is definitely a part of it.

But it is actually deeper than that. Ableism is that fundamental assumption that being nondisabled is the default, and I'll use air quotes on that one, setting for humanity. But what it begs the question is, how does a machine become ableist or racist or sexist or ageist?

For me, I believe that artificial intelligence in and of itself as a technology is neutral. Its job is to find patterns. It doesn't necessarily have morals or ethics. It lacks consciousness or empathy or agency, And it doesn't really feel guilt if it makes a mistake, nor does it necessarily care about the outcome of its actions.

It is simply following data and following a framework provided by its creators, us. And you know who isn't neutral? We aren't as humans. Ableist and ableism is a term because humans made it one. And it is the same for sexism and racism and ageism and so many descriptions of things where people are discriminated against.

It is the way that we act, the way that we think, and how we treat others. That is what breathes life into the concept of discrimination. And so for me, we do have a problem, and I'm probably taking the negative view here to start. AI is data driven. And because that data has come from us, AI is inheriting our historical biases in a lot of these communities.

As I was preparing for this presentation, I actually came across a LinkedIn post by Erin Eisinger. I don't know Erin, but I thank her for this wonderful example. Erin in her post says she asked ChatGPT, make me an image of an Americana cowboy on horseback smoking a cigarette. And Chatchipiti replied, I can't create or depict someone smoking, but I can make an image of a cowboy on horseback in a cinematic Americana style. Dusty landscape, warm sunset lighting, rugged denim, and a hat, but without the cigarette.

And it did that. But what it produced in Erin's words was a pretty sad dust bowl worn man, but no cigarette. But that wasn't what Erin wanted. And so Erin tried again. She typed, he should look more powerful, youthful, and have sex appeal. Now Chad Chad GPT updated the image.

Suddenly, the image was of an attractive man, that might be subjective, in rugged jeans, shirt open at the neck, riding a brown horse. And guess what? He had he had a cigarette hanging out of his mouth. So to me, this was a perfect illustration.

AI refused the explicit command to show smoking, but it had been trained on decades of media that cool or sexy, potentially within a cowboy setting, was equivalent or at least equated with smoking. So the bias overrode the safeguards that were in place. But, of course, it goes far beyond cowboys.

If the data we feed AI is biased, the output is not just necessarily just wrong. It can be harmful. Here are just a few examples. So first, a disability related example. So rich researchers at Penn State tested sentiment analysis models. So that's the kind of model where it extracts subjective information, like what are the thoughts or the attitudes, what are the emotions or sentiments that might be coming from, say, social media posts or product reviews, maybe a political analysis or market survey research.

Their research concluded, and I quote, all of the public models we studied exhibited significant bias against disability. There was a problematic tendency to classify sentences as negative and toxic based solely on the present presence of disability related terms, such as blind without regard for contextual meaning, showcasing explicit bias against terms associated with disability.

And, of course, it's not just disability. Our second example looks at gender. So Bloomberg analyzed over 500 images generated by stable diffusion, and they were generated related to job titles. The analysis revealed clear demographic bias in AI generated images.

High paying jobs were overwhelmingly depicting people who had lighter skin tones, while darker skin toned individuals appeared more often in prompts for lower paid roles, like fast food workers or social workers. Gender bias also appeared. Stable diffusion produced nearly three times more images of men versus women, with most professions skewing males except for low paid jobs like a housekeeper or cashiers. And when both of these factors were combined, lighter skinned men dominated the images for high paying jobs like politicians, lawyers, judges, and CEOs.

And our final example also pertains to language. A study by the University of Washington looked at h speech speech detectors. So that's AI that me is meant to keep us safe. They found that tweets written in African American English were flagged as flagged as offensive or toxic at a rate of two times higher than tweets that were written in standard American English.

The AI really wasn't detecting hate. It was just penalizing a dialect. In my own history, the history of AI powered overlays has made people really hesitant to use or endorse AI for technical accessibility. You've probably seen these overlays or these widgets.

They're the little things that promise to fix accessibility on your website with one line of code. And many companies, particularly those in The US that are at where digital accessibility is much more litigious, use these tools as insurance policies against lawsuits. But the data actually shows the opposite.

According to AbilityNet in 2024, roughly about twenty five percent of all digital accessibility lawsuits were filed against companies that were actively using accessibility overlays. Having those widgets didn't really protect them. In many cases, it was a bit like a red flag. It was a way to signal, hey.

Didn't really build with accessibility in mind, but no kind of I've gotta do something about it, but looking for the quickest fix possible. Here you go. I'll slap this on our website. Now why AI, I do believe, will get better at detecting technical accessibility issues beyond what we currently can do? They just aren't yet there.

Sorry. They just aren't there yet, and certainly not to the extent that overlays claim. But, of course, this is an ethical consideration. These overlay companies have fantastic marketing. They're preying on your fear. They're preying on the fear of of a lawsuit. They're preying on the fact that you might not have actually realized you should be doing accessibility, but maybe don't have the time or the budget to to fix things.

Finn Serf, one of the fathers of the Internet said, we want people to have a conscious sense of both the power and the peril of artificial intelligence. And so far, I have focused very much on the peril. I've told you about bias, exclusion, and the potential harm of these systems. But we also can't ignore the power.

Because when AI is guided by ethical frameworks and principles, it can lead to great things. So my colleague who is blind shared how he used AI to reminisce on his wedding to his late wife. He simply used AI to describe his wedding images.

And it reminded him of moments that he had forgotten, of people, of events that were captured within those images. And it was important because it was independent and a really moving moment for him. He wasn't relying on someone else to tell him what was in that photo. He was reclaiming those memories for himself.

This type of visual independence is also available in apps. For years, blind users relied on volunteers to describe images via video calls through apps like Be My Eyes. But Be My Eyes has partnered with OpenAI and uses chat GPT vision. So instead of waiting for human volunteer, a person can snap a photo of, say, maybe what's in their fridge and AI that can then tell them what is in their fridge and even maybe tell them what they can make with those ingredients.

But here's the important point, and this comes directly from Be My Eyes or Be My AI website. It says, if Be My AI can't answer your question, if you want to check it its results, or if you just need a little bit more assistance than Be My AI can provide, you can still easily reach out to one of our wonderful volunteers just like before. What's important here is choice and agency.

And that's not just in AI. That is in any system that we design. Consider the power of cognitive scaffolding. Neurodivergent people may have difficulty with tasks that require executive function, the thing that helps us set and also carry out our goals. A vague instruction like organize a workshop without clear direction may make a task feel overwhelming.

This is why I this is where AI tools like Goblin tools can actually help. They can support this executive function. So you could ask how you organize a workshop, and it would break it down into individual steps for you. Email the venue, draft the agenda, order the post it notes, Maybe, maybe not these days.

It even helps with communication. It can actually check what tone your emails or perhaps your documents have. It could answer questions like, do I sound rude here? Or am I being passive aggressive? And I know personally that some of my neurodivergent colleagues use Goblin tools regularly in their day to day work.

Our last example is Apple's personal voice. So for people diagnosed with a ALS or motor neuron disease, losing their voice is often a devastating reality. Apple launched personal voice. It uses an on device machine learning to actually take voice clip clips. And when that person eventually needs text to speech function, it can actually make them sound like themselves.

So no more of these robotic voices that do not portray who you are. In many cases, it could come down to risk versus impact. What's the risk if AI gets it wrong? For my colleague with his wedding photos, the risk is fairly low, and he also had his own memories to draw on.

But there are lots of more higher stake environments. Imagine a parent who is blind using AI to read the dosage on the label, and if AI reads five mil instead of point five mil, and the implications of that. We must stop asking ourselves, can AI do this? And start asking, should AI be trusted with this, particularly without a human in the loop?

When I submitted this presentation, it wasn't because I felt like an expert in this area. I'm far actually from it. But I submitted this because I feel like I have a responsibility to talk about this. I have spent fifteen years advocating for and helping organizations create more inclusive experiences. And the ethical use of AI is just the latest in new technologies and new challenges that we are facing.

And if you work in this industry, you have that responsibility too. So be the data ethicist. Question the input. Ask those uncomfortable questions. Who is not represented in this data set? Whose reality is being marginalised? Are we measuring actual fairness, or we're just measuring accuracy for the majority? Be the human in the loop.

Our role is to champion the potential of AI, but only when it supports everyone fairly. This is not just about oh, sorry. Be the advocate for equality. The human in the loop comes next. Our role, though, is to champion the potential of AI, but also make sure it doesn't reinforce systemic discrimination that we have spent decades fighting. And be that human in the loop.

So this is not just a buzzword. It is a safety valve. Humans in the loop mean embedding diverse human oversight throughout the life cycle during training, during validation, during the real time operation of these systems. And it's not just a moral obligation. It's also becoming a legal reality with the EU AI act mandating oversight for high risk tools.

But for me, it's not just about having a human in the loop. It's about having diversity within your teams. It's about talking to people before you even decide that you're going to start to use AI, and it's also about testing those outputs to make sure that they aren't biased. We can't outsource our ethics to an algorithm, but what we can do is be the ethicist, be the advocate, be that human, and also be the change.

Thank you.