Collaboration and Scale

(upbeat instrumental music) - Good day, good day.

I've been practising my Australian accent.

It's terrible.

My main inspiration is Kath and Kim.

Look at moi, audience, look at moi.

Also terrible, I'm so sorry.

Okay, so today we're here to talk about collaboration and scale.

Collaboration teams working together.

Scale is working at scale, not just nationally, but internationally.

And so if I were to build a new product, we often start with what we know, and who we know. Let's say I was building a new messaging app. I would start with probably my own experiences and say okay, these are all the messaging apps that I've used previously.

These are the problems that they haven't solved. I will build a new messaging app that solves problems that are unsolved today in the market.

I'd also think about people that are familiar to me, my friends, relatives, coworkers, think about the experiences that I've had with them, and problems I could solve for them as well. The problem with that is that is inherently a biassed perspective.

We start from our own perspectives individually, and then we think about those people that we know in our lives, but that's a very local perspective. And so usually the first step in kind of testing a new app would be maybe an alpha or a beta test in our local area, maybe New York City, maybe Melbourne, and then maybe going broader to like all of the United States or all of Australia.

And in this process, we lean heavily on our design partners. Research, content, strategy, product design. I think the big, huge step for us is how do you take a well functioning app domestically, or even in a city, and take it international? How do you know if your app is working well in Australia that it will work well in Thailand or Mexico, or Indonesia or Brazil? So that's what we're here to talk about today is taking a product that might work well regionally or nationally and scaling it up internationally, and how the foundation of that is really in collaboration. - So thank you, Jamie.

I am the Australian here, which is really good. My Australian accent is also pretty awful if I spend too long outside the country, so again, apologies to my countrymen.

But as Jamie said, there's a lot of things that need to happen to make products that work at scale. A lot of things have been discussed today that are important and that have been dropped on teams like ours, but one of the ways that we ensure that these things don't happen is by making sure that there are good, strong collaborative teams. Facebook does place a huge amount of emphasis on collaboration.

It's key to building for billions of people, it's key to making our products better and helping them grow in the right sorts of ways. It's really important for making calls when products shouldn't grow as well, when you maybe need to take a step back and things like that.

This directly impacts how easy it is to localise our products, and we're going to share some stories with you today about how collaboration feeds this process of localization, scaling beyond our borders.

I'm going to spend most of the time talking about collaboration where I think works in design teams, cross functionally as well. Have you all heard this term, cross functionally? It's a good buzz term.

Like another word I say, and I may say that, just so you know.

Things work in lots of different disciplines. And Jamie will focus more on stories about how localization actually takes place and how you build these insights and how collaboration feeds directly into that. - Yeah, so localization is the idea that you, like what I was saying before, you've built a product that works well in some culture, some context, some country, or city, and then you localise it for a different market. So this works well in New York City.

Will it work well in Bangkok? I'm going to change the way the app functions, or not, so that it does well in Bangkok, but it's still like the same platform, right. And so I think this breaks down into two things that I'll be talking about today.

They both have to do with understanding cultural nuances. The first is understanding the cultural nuance of language for an interface.

The second is understanding the cultural nuance of the actual interface itself, of features. And those together create a strong localised product. But first, introductions.

My name is Jamie Kimmel, not Jimmy Kimmel, the talk show host.

I have a big branding problem.

I've been a researcher at Facebook for about two and a half years.

Before that I was in the field of applied behavioural economics, and I focused on the concept of the difference between intentions and actions.

An intention is like I have the good intention to go to the gym four days a week or save money, but do I follow through on those actions? And so I focused mainly on helping businesses follow through on good intentions on Facebook. - And as I mentioned, I'm Ben Barone-Nugent. I am literally the only person in the world with this ridiculous name.

My parents being hippies of the 80s decided that I would get both of their surnames, which is a good thing.

I don't have a branding problem.

I mean I have a branding problem, but for other reasons. Part of that branding problem is that I chose the old man emjoi for myself here and I gave Jamie the young man emoji, but it had glasses, right, whatever.

But I've been a content strategist for about 10 years now. I was one of the first people, I think, in Melbourne, to have that UX writer title.

I was working on the agency side then.

I've worked for big companies like Nav, and I know there are a lot of people here who work for those companies, which is awesome.

Moved over to the United States and ultimately started working with Facebook, and have helped build the practise of content strategy there, and we'll get the different ways that we can collaborate better with product designers and researchers and things like that.

Before we dive into some of the stories and things we've got to share, I think it helps to get some context around the work that we've been, that we do, that Jamie and I work on.

We both work on the same product team at Facebook. So I'd like Jamie to talk a little bit more about that because it will help add colour to the problems that we face and the way we think about solving them.

- So we work on, in an org in Facebook that is a little bit counterintuitive.

We work in an org called local discovery, and the goal is to help people connect with businesses. And the consumer facing goal there is that we want people to go to Facebook to discover things to do around them, things to do when you're travelling, and ultimately connect with businesses to give businesses value.

And so I'll give you some examples of the things that we work on.

So one is this asking for recommendations product. Some of you may have seen this where you might be travelling to New York City, it's a great place, but you don't know what to do, so you ask all of your friends on Facebook, via a post, hey I'm going to New York City, looking for some like food and drink recommendations.

Do you have suggestions? As people make comments on the post, they'll get structured on a map and in a list and you can take action from there.

Another is simply searching on Facebook.

Most people think about search on Facebook via people search, searching for a friend that I'm not friends with yet or someone I'm already friends with, but this is an example of searching for things to do, burgers, like nearby, and you can actually get results for businesses around you.

A third example is like this local bookmark. There's a tab in Facebook.

You can explore things around you that works really well in Australia, actually. Ben was using it when he was here just a few weeks ago. The whole idea behind these concepts is this phrase called social at scale.

It's the idea that pairing social information with search results could be, in some contexts, more compelling than search results without social context.

So I might say, you know, burgers nearby.

I get a list of restaurants and I see oh, this friend has been here, this friend has checked in here, this friend has reviewed the business.

I know that friend's taste, therefore it's easier to make an informed decision than if I were to just search for burgers, get a list of reviews.

I don't know any of the reviewers, I don't know any of their tastes, so it's harder to make that decision, alright. So we are trying to take advantage of the social scale on Facebook to make these decisions easier and stronger. - So now that we've got some of the exciting stuff out of the way, which is we work on, or maybe not so exciting stuff, depending on how you look at it, I want to talk a little bit about how collaboration manifests in team structures and org structures and things like that.

As I said, I came from agency background, and there's a lot of people with job titles like UX designer, UX instructor, really valuable, important roles in the process of design. But Facebook doesn't have any of those, and other big tech companies also have those. I'm not saying it's better or worse, but it does put this interesting responsibility on every individual on the design team.

If none of you have UX in your title, then you're all responsible for the experience, and I know that's like a truth, but it's very, very obvious in the way we set up our teams.

Teams are typically made up of three core design practises. You have design, these, they get the title product designer. They're responsible for visual systems and visual things like the colours we choose or the way buttons look.

Prototyping as well, things like that.

We have writers as well, who take on titles like content strategists.

We're focused on informational systems, the language that appears in our interfaces, all those types of things, and naming, which I'll be talking a lot about today.

And then we have UX researchers, like Jamie, and this was actually one of the most profoundly interesting things that I learned when I came to Facebook was that there is this discipline that's a core part of design that is focused on only getting things in front of users all day every day, and I know we heard some really wonderful talks earlier about research and the value of that.

It's just so core and it's really exciting to work with people like that.

It means that I can always come up with some new language and get that in front of people really easily. So when you look at it like this, product design is really a process.

It's a process of how all these different things interact, and it really focuses on the tension between these different things, and I don't mean tension in like a negative thing, like I'm going to fight with Jamie.

I mean, we may, but for non design related things. And that puts, that's where I think the responsibility lives.

The responsibility lives in these collaborative practises. The responsibility lives in our ability to have conversations with each other, and that's really cool and it's really, really interesting. Some things that make this work is this focus on constantly having conversations with other people. We're always expected to meet with people, we're all expected to be completely open and honest with them, and I'm going to talk about some of these tactics later, things that I think work really well, and we also co locate with our design partners. We don't necessarily have a lot of meetings. I mean we do, but usually for broader team things. When we're solving design problems, we just swivel around in our chair and we have a conversation, and it works beautifully.

And I think that's something that some of the agencies that I worked in lacked, and that is because we're working on so many clients, we're not always able to sit with the people we're working on and solving those problems with, so it's really nice to be constantly working on these big, long term problems and just being co located with our design partners.

The other thing that's really cool, and this is actually more of my own reading of some of the stuff, is that we try to blur boundaries between what we do.

I work with a lot of product designers who are really marvellous writers and information thinkers, and I often rely on them to do sort of what I am on paper supposed to do.

There are some researchers who are really wonderful designers and they'll prototype things, and there's an encouragement for these boundaries to be blurred.

Think back to what I was saying about the tension between disciplines.

That's where it lives, and this is a way that it comes out, and it's really, really cool, and I think it results in better work.

I can talk a little bit in a few minutes about some of the tactics that I think foster this and help other people, help you as an individual get other people to open up their processes and collaborate with you more effectively.

- So now we'll go on to an example of what this actually looks like, an example when I was humbled and had lots more impact as a researcher because of collaboration.

So in the beginning I talked about two aspects of localization.

One is understanding cultural nuances of language for the design of international products.

The other is how cultural nuances might effect features and use of features and the design of them. I'm going to start with language in this example. And so what Ben was talking about, about like blurring the lines between disciplines is really important.

I, as a researcher, might be super confident about my ability to conduct research studies, but there are plenty of examples when I put my research plan, the specifics of the research plan, with questions I was asking, in front of other people, and it significantly changed my impact as a researcher. And so I think the idea here is that we can all become super over confident in our own abilities to solve problems if we silo ourselves, and so that's why collaboration is super important, especially when empathising with people who are not similar to ourselves, when you're building for cultures you don't understand initially.

And so an example of this is we'll go back to the asking for recommendations feature.

It's one of the things that I work on directly. I wanted to, we launched this nationally first, and tested it and tested it and iterated and iterated, and we wanted to go international.

One of our first test markets was Thailand. So we did specific testing in Bangkok, Thailand, and the language piece was super important because of how this thing pops up.

There's two ways that you can ask for recommendations on Facebook from the normal news feed.

The first is that you can go to news feed, compose a post in like the normal post creation flow, go down to this option that's kind of far down called ask for recommendations, and it creates the post and you have this interface and you can go from there.

The other that's a bit more interesting is maybe one of my favourite examples of natural language processing, and it's that you might just create a normal text post and not know that this feature exists, but you'll say something like hey, I'm heading, or yo, I'm heading to Melbourne, and need some good food and beer recommendations. And then what happens is that we see that you're saying hey, I need like food and beer recommendations, I'm going somewhere, and we'll allow you to opt into this new format that you might not know about.

And in that case, about 80% of our recommendations posts come from our natural language processing model, because the awareness of this feature is quite low in Facebook.

And so why is language important in this context? Because I know the phrases people use, or my team knows the phrases people use when asking for recommendations in English, but when you translate English to Thai, not only do you lose like some nuances, but they might have totally different phrases they use, slang or not, in Thailand, that we have no understanding of. And so if we want this thing to be successful and people to know about it and actually get value out of finding things to do when they travel, or when they stay local, we need to figure out the language piece.

It's pretty essential.

And so this was a three step process from the research perspective.

I was planning to run three studies in Bangkok. Specifically we hired a company to run studies locally for us with people that live in Bangkok, and the first piece was foundational research. You understand the cultural context of how people ask for recommendations, how people give recommendations, some broader things.

We showed them like mocks of the feature that wasn't launched yet and get their feedback and impressions, and some other like general workflow questions. And so we would talk about workflow, cultural nuances, and show concepts.

The really interesting thing here was I have done this a number of times before and the studies were impactful and they were great, whatever, but this time I decided to put my research plan in front of two designers, a content strategist, and another researcher, and the content strategist came back and said yeah, this is like fine, whatever, we'll get impact, but what about like the specifics of language? What about the exact translations of English into Thai and how that might affect significantly the way this thing is used or interpreted in Thailand? And she said what about the most important word in the entire interface, and it's the word recommend. If I'm tapping into this interface or my post converts to Jamie is looking for recommendations, but that word doesn't translate right, then the whole experience is destroyed.

And so we added in a few questions, like what's the word recommendation, (foreign language) in Thai mean to you? What are similar words to recommend, or.

(foreign language) What do those mean? What are the nuances of that? Can you use them in sentences? Can you use them in examples? Can you show us specific examples on your phone of when you've asked for recommendations or provided recommendations? And we found out some really crazy things from that. We found that the word (foreign language) has a dual association.

One is that it's the same as what we hear in English, I'm looking for suggestions from friends, family, relatives, whatever.

The other is I'm looking for help, like personal help, and that was a bit jarring to people in Thailand. I'm making a post to maybe a thousand Facebook friends saying hey, I'm looking for help.

It made them a bit uncomfortable.

And so that uncovered a really important cultural nuance from just understanding the difference that one word makes when you're translating it, right? Translation wasn't direct.

And that fundamentally changed how we approached the entire project and changed the way our natural language processing model worked, and also forced us to understand how important empathy is with cultures that we don't yet understand. - Before we go into some more detail about some of the way we solve these problems and solve them in scale and thought about language and these types of things, I want to talk about some of the principles of collaboration because they add some really rich context and I think we've learned a lot of cool stuff about how collaboration can work a lot better and how it can feed into these things.

When I started, when I joined this team, I was the first content strategist with a big team of product designers.

They'd been together for a long time, just working with each other.

Maybe they'd had some research folk, but really it was just a bunch of designers aggressively and quickly solving, you know, lots of just like maintenance problems with these products. And I was brought on and they didn't really want me there, to be honest.

They were just like ah, it's another word nerd. He just like wants to work his way into meetings and make my life harder.

I'm an okay writer, so ugh, I just don't want him to be there.

So my job was really primarily about building credibility and building that empathy, and building the collaborative foundations, and I wanted to be intentional about it.

I worked in a lot of places where my solution was to go and give a presentation, like what is content strategy, why am I important, and that's important, and that is needed a lot of time, and in a lot of teams, but it was different, and I needed to think about it and be much more delicate about how I approached it. What I realised is that it starts with you. In this case it starts with me, but the point stands is that good collaboration starts with you as an individual.

I had realised that I needed to change my practises and think about how I was choosing to open up to other people before I could expect them to do the same for me.

As frustrating as that can be, it's really, really important.

But it comes from a simple idea that you are to do unto others as you would like to have done to yourself as well, which is really, I think just like a good mantra to have in design teams.

There are a couple of broader principles that I sort of started seeing would have effect. Always remembering to give and take, whether that's like give Facebook, give feedback, sorry, and expect honesty in return. But also taking people's help and making them feel like they're part of your process.

Opening up your own process is also critically important. You can't just tell people that you need to be in their process and not expect them to be in yours. You need to take them on the journey of your thought patterns and your problem solving. And this is true with me and other content strategists as well.

It's not just me and other disciplines, it's just between any two design practitioners, or even cross functional partners as well, with engineers or research, or whoever.

And get to know your partners as people as well is really important.

What I realised is if I didn't know the people I was working with on some human level, then I wouldn't always have the context to be able to interpret their feedback, interpret the way they were solving problems. One partner might have been going through a bad break up, and if I didn't know that, I wouldn't understand why he was short tempered with me.

It's little things like that, and it helps you have better and more constructive conversations.

There are a couple of that I'd like to say, and this comes back to my, what I was saying about being really, really intentional.

I would say things like hey, I need help solving this problem, even if I didn't really need help solving it, I'd want to bring it into my process.

It's a good way of bringing them along, you know, on a journey with you as well, and verbalising that is a really good way of opening that up.

Letting people win some arguments, or letting people get their content up when they're a product designer makes them feel like they have ownership over the work that you're doing, and over time they'll start to realise that we're all working to the collective goal and they need to open up their process to you as well. Really pushing them on feedback.

Think about what I was saying about giving and taking. Making it known that you want people's feedback. And it's not just about saying it once.

Make them know that you really, really want that thing, you've got to give and take at every turn.

Also just empathising with what they want to achieve and what they enjoy is just like a good thing to make teams function well, so I'd say things like hey, look, I know you're loving this thing.

Why don't I do this other thing? Or how about you sit this meeting out and focus on that thing you like and I'll go to the meeting and represent us both? These things just work really, really well. Simple things as well, just like asking how their kids are, just seeing how their life is outside of your product team is really good.

It actually works better with product partners you might not see quite as often.

There might be designers that I'm only working with a little bit, but I'll always try to ask them how their life is outside.

I don't push them on it, obviously, but I think it just shows that empathy, that understanding that they are people, beyond just like a good designer.

There was one designer who I worked with for a long time and I had this graph that I would show other people on teams when I'd give similar presentations that had the number of stories I'd heard about his kids to the quality of our product work.

It was very unscientific, but I think it shows something in kind of an interesting way.

We'd just really build like this really good tradition around understanding and being good friends first. Again, like you don't have to be like that all the time, but you can get little bits, or a lot of it, depending on who you're working with, and what's needed on the team.

- So the first part of localization was talking about how an understanding of the cultural nuances of language has an affect on interface design and product design.

The second part that I want to get into is how the cultural nuances, understanding cultural nuances effect actual interface design features.

And I want to talk to you about an interesting research concept.

I think about it all the time, called the extreme response bias.

The idea here is that, so we'll focus on Latin America and also Asia.

If you give the same survey to a number of people across the world, you'll see, on average, quite different response rates.

So in Latin America you'll see what's called extreme response bias, where in a Likert Scale, like one to five, they're more likely to answer towards the extremes, the ones and the fives, than they are in the middle.

We see this behaviour in an opposite way in Asia, and this is generalising, but we see middle response bias. Likert Scale, one to five, two, three, and four is much more likely.

And so how might this have an effect on the design of systems of things, of whatever it is that you're building? A good example would be let's say you have a ratings and review scale, platform, that you're trying to launch.

This is a pretty important piece of the digital product landscape today, Uber, Airbnb, Google, Yelp.

Facebook has a ratings and reviews platform. If you were trying to internationalise a ratings and reviews platform, you might see a trend of more one and five star reviews in Latin America because of the extreme response bias. You might see a trend of more two, three, and four star reviews in Asia because of middle response bias.

And does that have an effect on the richness of the ratings and reviews platform? The big question that I think that we should have there is that are you taking a mental model that you have. I grew up with a five star review system, I know it on Yelp, on Foursquare, on Uber, on whatever else, and trying to cram it into cultures that might not make sense.

- So I think a lot of this stuff.

Does everyone know the film Call Me By Your Name? It was a stupid joke that we made up, but whatever. I think that a lot of the stuff that we've been talking about is going to start coming together, and I want to share some interesting things that I've gone through about, thinking about product naming.

So hands up whoever's had to name a product or a feature or anything like that.

Okay, wow, okay, so there's a lot of you.

That's awesome.

Actually, just out of interest, how many people here are content strategists or product writers or something like that? Oh, that's awesome.

This is great, I love this.

But I want to start off with this question about like what is in a name? It's often either the first thing or the last thing that people think about.

It's either the first thing because people just have a name and that's where an idea starts or a name comes along with the idea, and that's okay. Or it's the last thing.

It's like we've got this wonderful product and now we need to think about what we call it. How do we brand this thing, how do we refer to it out in the market and stuff like that.

I actually worked on a product years ago for Qantas. Does anyone know who, does anyone remember that brand? Okay, great, so it's still out there, but I remember it was like a really complicated naming process that went into this, and that was one of my first really, like large scale exposures to naming, and I've done a lot more of that since, but there is this question about like what is in a name, what value does a name bring to a product or to a feature or something like that? The fundamental truth is that naming is really, really, really important.

It says a lot about your product, it says a lot about how you think about your audience. It says a lot about how you, as a platform, or a brand, or a company, wants to engage with your audience and how you think about them.

And I'll tell you some stories about how like we've thought about that and tried to implement that into our workflow, but I want to stress that naming is really, really important.

It's not something you just like do randomly or you just like come up with a snazzy word or give it the name of the internal project. You need to think about these things and you need to be really careful about it. And I'm going to show you a story about how I learned that. But before I tell you the specifics of that story, let's talk about some of the things that a name does and that a good name does.

First of all it sets really clear expectations for what your product is.

This is a little bit of a fuzzy area.

I think if you're thinking about naming through the brand prism, you can give them like very unique names and things like that, but when you're thinking about products and you're thinking about features, you've got to always consider how a name sets expectations for the product and how people will engage that, what they expect to be able to do there.

It does have a huge identity role as well, so it shouldn't obviously clash with other products in the market, the features shouldn't confuse themselves with other parts of the system that you're designing within. You can imagine how that's a big concern for a bank or a company like Google or Facebook or anything like that. But it gives products better identity, and that obviously has effects usability as well. And I chose empathy.

Remember what I was saying about how this says you want to interact with your audience and your users? Well a name shows that in a profound way.

If it's overly technical but you're trying to build a consumer product, that says one thing.

If it's overly cutesy and you're trying to build like a high end industrial platform for I don't know, CNC-ing metal.

That says something very different as well. It shows like a level of out of touchness and just the way that you think about the audience and the way you empathise with them.

Thinking back to the things that we work on, I actually want to talk a little bit about this product here, which is the local bookmark in Facebook.

It's a place that people can go to to find bars and restaurants, things to eat and drink.

Ultimately it should people find services.

You can already find them in there, but it's this one landing platform where you can come and browse things.

The system will recommend things to you and help you do those.

The word we give it is either one of two things. It's either local, which you see here.

That's what it's called in North America, or it's nearby places, which it can be called in other markets as well.

But it's one of those two things, typically, at least it was when I started working on it. And that actually has a lot of problems.

It seems right on the surface, like it makes a lot of sense, but it's this one size fits all approach that I felt like was losing us some markets. And this product was much more successful, actually, outside of the United States than it was within the United States, so there's this odd question about like well why are we naming it for an American market and then just translating this to other makers as well? And that's the way we were thinking about it, and that's really problematic, because assuming that you've got a name right in EN US doesn't mean that that name's going to make a lot of sense in Portuguese in Brazil, or Thai, or even British English.

You need to think about these things in a more nuanced way.

So we started thinking about regionally specific names and what value this might bring to our product and how it might make our products stronger. Now what I mean by this is not necessarily translation for the sake of translation, but making sure that we're coming up with new names that understand the markets, that have that empathy to the people who are going to be using it. The process I went through was actually pretty simple. Has anyone heard the term namestorm before? Has anyone? Okay great, this is good.

It's the idea that you get a lot of people in a room, you ask and everyone just like thinks of names. We throw them at a board.

You know we have different words that trigger discussion and shape the discussion, things like that. So we spent a lot of time doing namestorming, just coming up with different ideas for different names. This involved a heap of collaboration and a heap of that tension between views that I was talking about.

We had researchers and engineers and designers and content strategists and product managers arguing about whether a name makes sense, arguing about whether it's going to be easy to translate or whether it's too cutesy or too serious or all these different types of things.

The next step is that I, as the content strategist, took that out of the room, and I just realised these arrows here are in a funny direction. I don't know why.

It's Keynote's problem, not mine.

But I would take these names and I would go and just put structure around them.

So I'd think about like, what are some tweaks that we can make with them, make to these names to make them clearer or better or something like that. What are some that should just be ruled out? What are some that we should, you know, iterate on, something like that.

The next step was to like have, I had all these names, right, in English, because it was in New York obviously, to have a lot of discussion about this, and then I went to people that we, that work with Facebook in different parts of the world. I texted people in Brazil, people in Mexico, some people in Argentina, Thai collaborators, all these different people within the company who had a nuanced understanding of the market and they'd never been engaged from product teams like I'd engaged them.

And I worked with them to take this big list of names in English, to not only translate them, but make sure that those names made since in the different places where we were going to roll them out.

And we uncovered some really interesting stuff, like the word local in English translated directly into Thai, can have a pejorative sense.

It can mean that somebody is more of like a local, which means they may be out from the sticks or something like that, and if you want to get people in cities on board, that's another thing.

The word local translated directly into Portuguese, and used in isolation, just doesn't mean a whole lot. There was an Argentinian product that had a very similar name.

We started to see these problems arise that happen when you just do a one size fits all thing. So ultimately we could whittle down this huge list of English names into some that made sense for different markets, and then once we had those, we were able to go and run experiments with these names and see which ones actually improved the product. So I want to come back to this question, I want to come to this question of like what else, that begs the question what else should be hyper localised? Should it just be a name? I don't know.

Maybe other parts of products should also be thought about in a similar way.

Do button paradigms or the way forms work or the way products flow or information architectures work, should they be a one size fits all thing? It's an open question.

I think that it's something that a lot of companies and a lot of tech companies and a lot of agencies and things like that are thinking about a lot. But let's come back to this idea of what's in a localised name? Well I mean, a lot of just good things.

It makes more sense to people, doesn't get them off side, it shows their empathy. It can make everything just work a little bit better. In our case, we saw our uptake in core metrics, but honestly like metrics just don't matter that much. I think I'm not measured on metrics at the end of the day. We saw this like very straight product success stuff, so the most important thing is we just had a stronger product.

The product set better expectations about what you're expected to do with it.

The product showed more empathy for users, and we saw all this reflected in metrics, which is like a good way to have conversations inside of a company, but at the end of the day, I just read that as we made a product that was better. It's easy to navigate to, it's easy to have conversations about, and that's why it's really important.

- So some final thoughts from Ben and I.

What Ben's talking about is something that we both really care about personally, and I think maybe when we came in, before we came into Facebook, we had the intuitive perspective on localization. The intuitive perspective is you create an app, or some digital platform.

It does well in, let's say the United States. If you want to launch it to Mexico, you just translate the interface, and then you're done. That makes sense.

That doesn't sound crazy, right? If your translations are good, then you're good. The actual process is what we've been talking about this whole time.

You understand the cultural nuances of language, you understand the cultural nuances of how people interpret features, and then you might change the product in that region and then launch.

You have to empathise with people that you're building for. We've empathised with people in the US, in New York, in Australia, in Melbourne, because we know them.

What about people in Bangkok, Thailand? The last example I want to give on this was, so we talked about how the language really mattered a lot in Thai.

The word recommend translated into this weird, kind of dual meaning word of suggestions, but also help, and we saw that in our second and third studies, when we asked people to actually use these features in like small tests, they were really hesitant to, because what I said, they didn't want to ask for help from their 500, 1,000, 1,5000 friends on Facebook. So even actually correcting the language didn't really solve that problem because the cultural nuances are so different in Thailand than they are in here that we shouldn't think about jamming that feature down their throats.

It's a different culture, it's a different feature, it's a different product.

But it doesn't mean it wouldn't be successful, and so what we learned by talking to more people and empathising with them was that the asking for, the market for asking for recommendations is just as large as it is there in Thailand as it is in the US.

It's just in the US people are more comfortable asking large groups of people.

In Thailand they're more comfortable asking very small groups.

And so what they, how they tend to do it is through messaging apps, through LINE, through Messenger, through WhatsApp, maybe through SMS. They'll message one friend or small groups that have recommendations, base conversations, but we hadn't solved for that problem from a product perspective.

And that's what I'm talking about from the language and features piece is you can translate interfaces and then you're done and you wont' solve the problem, but if you understand cultural nuances of how words translate, you begin to figure out solutions like the localised name thing Ben was talking about, but you still might have problems with product launches. But it doesn't mean there isn't an opportunity for you, and that's what we ended up building was a recommendations based system actually in Messenger, in a messaging app for the people of Thailand. And sorry, and the last part of this is, of course, that was all because of collaborating with designers and other researchers, and especially my content strategist, Katelyn, on my team, who really pushed me to initially think about the cultural nuances of language, which honestly fundamentally transformed how I thought about other cultures and how I thought about empathy.

- If I wanted you to take something away from this whole experience, in addition to this really interesting insight about how you've got to push through biases and that requires collaboration, I just want you to take away some simple tactics that can help your teams collaborate a little bit better, help you work together a little more closely. So I want to just like go over a couple of those things that I was talking about earlier in the hopes that they help you just be wonderful designers, that I'm sure you already am.

I don't want to make it sound like I don't already think you're amazing.

First of all, is like get to know your partners as people. Again, this is one of the most valuable, profoundly important things that helps me in every day of my working life is to remember that they are humans making these decisions, and humans are emotional, and humans don't have brick walls and hard lines between their work and home life all the time, and that's okay. You need to be able to contextualise things and that requires knowing people.

Meet with people a lot.

This is one thing that I really value about my working teams is that I'm always having conversations with my collaborators.

I'm always having conversations with Jamie and other people, not always just about solving language problems and how do we make our recommendations product better? I'm having conversations about sports games and video games and movies we've seen.

It was out and in between those conversations come interesting ideas and things often refer back, lead back to discussion about work.

At the end of the day, we like our jobs, so that's also something we want to talk about. Co locating is also really important.

This is not always something we have control over. In fact I've been on teams at Facebook and elsewhere where there's an insistence that I sit with other content strategists who aren't necessarily working on the same products as me. That does have its benefits.

Don't get me wrong.

But the most important thing for me to be able to create good products that show empathy for the people who are using them, and are successful, and are doing what we intend them to do, requires me to work with people across functional lines. Sorry, that was a hard sentence to say.

It requires me to be able to turn around and have conversations with people, be they about video games or about the work that we're doing, and that requires being near them. Be open and honest and human.

Again, I just want to stress this.

It's so important.

You don't, you can never forget that the human beings you're working with, if they're product designers or writers or project mangers or engineers, are human beings.

That helps you build empathy and understand the decisions they're making.

It should also drive you to be more open and honest with them.

If I want to give an engineer who I'm very close with hard feedback, if we know each other, we walk out of the room and that person knows me as a good human being and can contextualise that hard feedback, and that helps us do our jobs. But also names are really important.

There's clearly a lot of people in this room are thinking about names and they're thinking about the impact they have.

Always remember that you need to think about it very carefully and you can't think about it through your own prism.

Think about all the things that Jamie was saying about breaking through those biases.

To be able to name things correctly and successfully in a way that shows empathy and sets good product expectations, you need to spend a lot of time thinking about it's not the first stop that you make on a product's journey and that's it, it's not the last stop.

It's something you need to really think about. It's its own product in many ways.

But thank you very much.

You've been wonderful.

(applause) (upbeat instrumental music)