Re-architecting PeerChat for on-demand mental health support

Yeah. It's interesting because context and custodians of context is exactly what we're gonna talk about with Em being AP work team lead. Yes. The ultimate custodian of context.

Yeah. Let's start. Hello, everyone. So to start us off, I'll be open about I do have a stutter. I had a whole life. I'm just open about it, So I'm less, like, nervous. So if you hear me pause or hear my words break out, hence why. Also, heads up, like, we will be talking about, like, young people and mental health, so heavier topics may come up.

You need a moment, it's okay to step away, and we will be here afterwards like, if you wanna have a chat.

Awesome. Thanks, Sam. We'll try and do it this way. I know an acknowledgment of country was already given this morning, but thanks for having us. We're happy to be here with you on Gadigal land. This land was never ceded, always was, always will be aboriginal land. We also recognize our connection to country as integral to health and well-being.

Yeah. I guess you already heard, like, a little bit about us, but context over the last year, we supported over 2.2 and, like, million like, young people who, like, needed that, like, anonymous and confidential support.

So the model works from broad reach to scale depth. Over on the left pillar, you see the expert mental health content. We really are positioned as a digital first step model within the mental youth health landscape. The service supports young people to take small steps to feel better by using human connection and technology that's co designed with young people and grounded in evidence.

Within that broader landscape, this really looks like earlier interventions and nonclinical support. That broad read self guided content then moves into the middle pillar, which is AI. It's it's a chatbot called AskReachOut, and it allows young people to have short focused conversations in their own words. It draws from our extensive library of evidence informed content and provides service recommendations where speaking to a human is the next best step.

And that's where we really move into that third pillar, which is the deepest level of impact and relational care, and that's peer chat. While fewer young people engage with peer chat, those that do are seeking human connection to navigate everyday issues with varying degrees of complexity. For some young people, the very first need that they have when starting a conversation is to verify that they're actually speaking to a human and not AI.

And it demonstrates that that kind of authentic human connection is the core user need that sits above all others.

So, yeah, what is, like, peer chat? This is where I live and breathe, I guess. We are in, like, one to one text and like base and like chat. Like over web chat when like young people are able to just hop on like then and there and just have a chat and like in the moment and like around like, whatever's, like, on their mind. And then on, like, the other end of the chat is one, like, of our peer workers who have their own experiences and like journey of I guess of life and mental health.

And like they really like use that in a way that helps the young person like feel heard, under and less alone. We are, like, one of like, the only, like, on demand, like, chats operating under, like, a peer work, like, model in the country.

And we are co designed, like, by young people, like for young people. And I was actually like one of those young people. I started helping on the project when I was 20 years old. And like when it was like a tiny idea.

So it's been really cool to watch it grow and I like I guess and like be all around like young people. I've talked about, like, we operate under a, like, P work model. So what that means is, like, we're really using our lived and living experience to use this in a way where it has purpose.

We're able to talk about it openly, like with a young person. We're, like recovery orientated, like young person led. And like we aren't here as like a friend or like a counselor, aren't here to give like advice or to diagnose.

However, we are here to use those, like, active listening and, like, validation skills to really help like, empower young people in their journey.

So it's really important that we continue to evolve the service in a way that remains person centered and focused on that centrality of the one to one human connection. So a young person can be, for example, smoothly handed over from AskReachout, the increasingly conversational AI chatbot. The core need is designed around a single service session model, so it's up to forty five minutes.

Young people, it's very low friction and simple and safe for onboarding, yes, but they do have to verify their phone number. That's a more recent introduction of friction. Young people then, if they do return, can opt in to have an AI chat summary safe that's only visible to a peer worker, but, again, it's their choice.

Then they go through a single question deterministic flow, and then the core experience is, very human. It's all contained in the messenger. And if this plays, you might get to see, that in action. So this is really how a young person experiences the service. It's a simulated conversation, so not real.

You'll save them into the flow. The peer worker manages the entire conversation in the messenger.

Just forward a bit so you get a bit of

a sense of how the conversation happens. And shout out to Karomi. He's one of our team leads.

Cool. So, yeah, we've talked about that we're operating on an on demand model. However, we weren't always on an on demand model. We started as like a booked model, but like you can imagine like young people and online like free service that didn't work as well because like not as many young people would actually turn up to their booking. We were having, like, young people, like SMS hour, like, booking and like a number of all hours of the night.

Like they would like try and hop on right away. So and with AI, we kind of also like noticed that it was having an impact on how, like, young people wanted to, like, engage and how they really, like, wanted it, like, then and there.

So to meet, like, the needs of our young people and also, like, utilize our team, we then were, like, able to pivot to the on demand model. Like, we will let's talk about, like, the changes in more, like, detail, but, you know, it did, like, mean like, we had to increase, like, the sizes of our team.

We had to add layers of leadership roles. We had to then, like, also change our, like, digital and data infrastructure as well. And then we as like a result of all that, we had more, like young people hopping on in like active high stress and it led to having more external escalations like to ambulance services and like police.

And then we also didn't like notice that young people were hopping on to talk about was also changing as well.

But it's really young people's experience that has experience gone through the or undergone the most radical change, really. We now know that young people engage with the on demand modality at twice the rate that they were with the previous booked model, so that means we're supporting more young people than ever before. In some ways, that kind of simplicity of the entire proposition experience really portrays the complexity of embedding a lived experience workforce.

In a product largely product led organization within a mental health sector. So how do we co create with our lived experience workforce?

Yeah. So I guess, like, a core, like, part of how we work is really working together, having all, like, voices heard and, like, taken into consideration. So what this meant is we had, like, peer workers who are actually, like, on the ground running chats, actually involved in every stage, you know, like in discovery, exploring tech options, testing and implementation around, like, recruitment, like, onboarding, like, training, all of that.

And known and, like, decisions and, like, changes and, like, frameworks happen, like, without us, like, involving, like, all of the teams listed above. We're all, I guess, like, mutually, like, informing each other.

I guess, like, as an example, like, we worked with the social impact team to change what impact looks like on demand and how we able to better measure that.

Like with the increase of the expected active high stress. We worked with the clinical governance team around how we're able to really, like, implement our lack of care, like frameworks and escalation and like pathways.

And then we and like, of course, worked really closely in like with the product team around, I guess, like almost everything, you know, our new, like, techs, creating, like, workshops, opportunities, like,

where the peer workers are able to learn in a way where it's quite hands on and, like, they feel confident and equipped. So switching modalities was initially piloted. We were able to prove the model actually quite quickly within three months. During that pilot, we ran the booked sessions alongside the on demand modality, but the 40% utilization of the booked model really didn't shift while the on demand rate quickly surpassed it.

So we sunset the booked model and focused entirely on delivering instant care. The service rapidly matured. It was through a series of really rapid iterations. And in parallel, which we've noted in the background, the organization was also developing an AI chatbot for young people to use.

So cocreation acting is the bridge between Ems team and the technical kind of strategic design from product. We applied all the same codesign principles that we'd use working with young people to develop the service as a whole, but this time to our internal peer, lived experience workforce. It's where it became very clear that it wasn't just a pivot.

The new service had its own distinct platform, logics, rhythms, and a scale that presented more challenges, but also a lot more opportunities for structured, automated, and assistive tools for the team, but they really needed to come alive as a representation of what I discovered was actually a lot of internalized thinking in the team.

It's where two really different disciplines at their core can complement each other and work incredibly well if care is taken to understand why, but also how some processes came to be, even if that means wrangling multiple off platform logics with a lot of embedded thinking in them. So three problems to kind of solve quickly emerged in this new modality, and we wanted to solve them with the lived experience team embedded.

One of the first problems was that we had these data models that, weren't adaptive from the booked model to the on demand model. So the way we classified conversations, was operational and business logic. It was very partial or completed. It really told us nothing about the way young people are engaging with the service or their underlying needs, and it wasn't an effective translation of happy workers conceptualize the conversations or their attributes.

Similarly, with, like, the topic taxonomy and service theme reporting taking hours, and we wanted to center that diversity of the peer workers' professional and personal judgments and map these very human concepts onto how the service and young people's engagement with it were evolving and to cocreate what ended up looking like a kind of harmonious classifier, but also one that guided peer workers to make decisions at the end of the chat that then flowed through all the data model logics.

We started in a pretty familiar place for a lot of people with analyzing a lot of data, including a thousand de identified randomized transcripts. And one of our first findings was that time wasn't really a determinative factor. What that means is that a young person can experience real impact and engagement at, like, a fifteen minute mark or a forty five minute mark.

So that operational metric flattened all meaning, and if session duration isn't the major factor, then what really defines impact? To find out, we co created with our lived experience peer workforce and really tried to ground that in daily peer practice, not abstract categories. It allowed us to look at where team judgments aligned and where they diverged. We overlaid time onto the transcripts, and then we spotted these really large zones of inactivity.

In the data, it actually just looks like disengagement from a young person. Without peer worker annotation, we didn't really understand what was happening. But what can be happening for a young person when they pause during a conversation with a peer worker is that they can be building courage to articulate something really difficult. They can be re responding to prompt prompts from a peer worker.

They can be reflecting on underlying issues, or they can be switching physical settings like leaving school, stepping off a bus, going to work. Even sudden drop offs in a conversation became a lot clearer, so often it was simply things like dinner's ready. I've gotta go or life getting in the way. We also embedded strength based language into our data models, moving away from, like, a tendency to use very online vernacular around certain behaviors, especially when young people engage in unhealthy ways.

So things like trolling really hides underlying needs, and it's not inclusive. It's not strength based. So we moved away from that altogether. All of this led us to really flip our classifier logics and anchor it in peer worker principles of mutuality and reciprocity. So an impactful session, as we've established, isn't about time, but did the young person gain perspective from speaking to a peer worker, and was the peer worker able to walk alongside them?

The in app classifiers now use this profess progressive logic to evaluate really meaningful attributes, like the disclosure of core concerns from a young person. Was it effective see peer support? Were any duty of care actions or interventions raised during that conversation? And how did the conversation end? So implementing the classifiers allowed us to also further reduce data retention.

So by design, we try and maintain a really minimal dataset. We now routinely redact and delete conversations. We only leave metadata traces, and it supports the privacy of a young person. So the classifiers are also really dynamic. The feedback loop is directly informed by peer practice every day, and a recent analysis shows us that pregnancy fears and porn or porn addiction will form part of the next iteration because they're coming up in our service. This also helps the team strengthen support for peer workers to deal with these conversations, but also to design trigger pathways and escalations to other services that are more appropriate for a lot of those things. We also better understand where there are a few drop offs so that we can design better waiting experiences and offer alternate types of support for this otherwise hidden demand.

It's also sorry. Well, I flip. It's also helped us understand a lot about what do young people wanna talk about and when. Taking a broader view from national data, we know that young people are facing the highest rates of mental ill health of any group and the least access to help.

There's so many compounding factors that face young people, including high psychological distress, cost of living, study and or work pressures, and cost of living can actually be a barrier to both study and work, but also to seeking professional support, So very compounded. We also see increasing diagnosis and shamefully intimate partner violence is a top risk factor for young women.

I say this because you might be surprised to see abuse, trauma, and harassment rate so highly in our daily demand. So, yeah, what you see there is really all the macro pressures translate directly into everyday human issues for which young people want to speak to another human. You'll see that relationships, and in particular, we find romantic relationships and breakups present at the highest volume every single day.

It's completely unsurprising that young people wanna speak to a human to understand interpersonal relationships and how to navigate relational conflict. It's also completely unsurprising that we see that young people who are handed over from our AI chatbot overrepresent on gender and sexuality because these are also really complex things that they wanna talk to a human about.

Yeah. So I guess one of our, like, focus areas was in ensuring that even, like, when we scale, like, we were able to uphold the safety and, like, for both our young people and, like, but also our staff. So I guess, like, one of the ways that we do this is around, like, the language, like, we're using.

Is it, trauma informed, young person led, we're career orientated and very much strengths based. Also I talked about earlier we noticed we were having more, like, young people hop on an active high, like, distress, and that often meant, like, we were calling other external agents to help us out and having, I guess, limited, like, identifying, like, information often, like, made it hard to get them that imminent help that they needed.

So what we did is we have, like, recently added at one time, like, password, like which helps us, like, verify the numbers and I guess, like, know that they're real so that we can get them that help as urgently as possible.

Young people can also opt in to have an AI summary saved, as I mentioned, a bit earlier, but we only released that kind of return service user feature around a month ago, and we already know about 18% of our young people are opting in to have that summary saved even though it's only visible to a peer worker.

They're automated. They're AI generated. But really importantly, for this piece of work, we worked with a young developer in our team who's also a peer worker team lead in ENSTEAM. So we really had those safeguards embedded from the ground up. We chose to architect a very lightweight solution here, but the prompt adherence to the peer worker framework and principles is bound by strict constraints so that we can do our best to contain what's returned so that it doesn't infer things like motivation and emotion.

It doesn't speculate on the contents of the conversation. It deidentifies the peer worker and the young person, and it's free of judgment. So it's designed to emulate the most ideal conversation with a peer worker, really.

So, yeah, we understand that with everything happening in their lives, like, we're not gonna be able to meet all of a young person's needs and, you know, I guess the scope and boundaries of our role. So we also acknowledge that it can be really tricky and overwhelming on a young person to know, okay, where do I go when and how do I get that other help?

So we are able to recommend other options in a really warm, gentle and informative way, which allows the young person to have the opportunity to ask more questions around like how does that service work or kind of what does it look like.

So we're able to help them, I guess, like, feel more confident and, like, empowered to take that, like, next step.

So, yeah, designing an offboarding experience as a connection to another service is a bit counter to a lot of the product thinking that we're taught. It's definitely counter to commercial product thinking, but two things had to happen here. A framework for human assessment and in platform affordances to seamlessly recommend the young person to a a formal partners.

These partners have different types of care, different stages along the spectrum of need. Peer workers identify the young person's level of readiness to take their next step. They then build the trust and confidence and set expectations for what onboarding at a more clinical service will look like, and that's really important. We know that from codesigning with young people that when we meet all these core needs that they have, they can and will fluidly move between services, and that's a really great thing.

Our proof of concept shows that it was about 8% of conversations that resulted in a pathway to external care and that care provider was specifically for ongoing complex and clinical needs. Really importantly, once we look at the data from that partner, we understand that young people who are referred from P CHAP present with actual moderate, high or acute needs already in their service.

So that means we're referring a young person to the right service at exactly the right time, and that's that's a really positive move to reduce the care gap. And, yeah, unlike a lot of user experiences that are designed to kind of reduce exit, boost retention, we prioritize those, cross sector partnerships to reduce the care gap and increase what many in the sector called connected care. I'm gonna hand over to Em just to finish our thoughts, finish us off on impact, which actually looks kind of made up. It's so good.

Yeah, it is real, promise you. So at the end of each of the chats, young people are able to answer some off boarding questions. And you can see here it's really evident young people are feeling heard, understood, better in that moment after the chat rating us quite highly. We also have a box where you can type in if you have any extra feedback.

And it's really lovely actually watching them write about the peer worker or write really nice notes about how it's helped them. I guess overall, we hear young people really like how it's available then and there when they're right in those emotions.

You don't have to pre book, you don't have to enter a heap of information or answer a heap of questions. They also like that the peer worker is someone who can really relate, has their own journey, and is on their level, and overall they're able to talk about whatever is on their mind and, like, without, like, any judgment.

I'm Nadine.

I'm Emily.

We're around all day, and come up and say hello. Yep.

Thank you.