Fugu and the Future of the Web Platform

(upbeat music) – Hi, and welcome to Code 20.

My name is Kenneth Christiansen and I work at a company called Intel.

At Intel, I work on the Chrome browser and I also work on Web standardisation in the World Wide Web Consortium, the W3C.

In the W3C, I’m also an elected member of the TAG. The Technical Architecture Group, overseeing the development, all the specs and standards going on on the Web and making sure that everything is nicely aligned, secure, and safe.

So, why does Intel care about the Web? Well, if you ask our friends at Microsoft, they say that more than 60% of the time people spend on the PCs that do so within the Web browser. We also know that more apps, more and more apps are built with electronic is also based on the Web app technology.

So, it’s probably more than 60%.

I have loved and in love with the Web for since many, many years and it has really formed and shaped my whole career.

So, why do I love the Web so much? Well, I love the Web, because it has really great reach and very low friction.

It’s also safe.

Like you just go to different Websites and you can feel pretty safe.

It’s indexable, composable, it’s also ephemeral. So, it’s not like these apps that you instal that will stay on the phone, whether you use them or not. If you go to a site and haven’t used it for a long time, those resources will automatically be removed from your device.

On the Web everything starts just from URL. Type in the URL and you get to experience.

That experience could be a progressive Web app. If you’ve been to any conference within the last couple of years, you’ve most certainly heard about progressive Web apps. These experiences that can work offline, be fast and responsive and be really well integrated. So, they look just like any native app.

The Web Platform is an amazing place to be. It has worldwide reach, it works across different operating system, architectures and form factors.

It also has a thriving open source ecosystem around it. It’s also flexible for all kinds of experiences as a games, news books and apps.

And you can make Web experiences that are fast. Apart from this, there are a lot of different API. So, you can basically do a lot of things on the Web. Let’s have a small recap of what progressive Web apps are. So, here, I’m looking at a site on my computer, it’s a desktop site.

I go and look at it and you see in my address bar, there’s this bottom core installed.

This is on Chrome by the way.

If I click on instal, it gives me the ability to instal this app to my system. When I do that, it turns into this like desktop app and it looks just like any other app and it can work offline by using technology called service workers.

The Web is really great, But sometimes it might not be feasible for what you’re trying to accomplish.

Maybe it’s missing that API that is core to your experience. There’s also the lack of an API roadmap.

All of this together brings distrust in the platform. So, for instance, let’s look at this made-up example. Let’s say the next year someone is launching the magic native API.

It drives a lot of traffic to our competitors’ apps. And you don’t know when it’s gonna be available on the Web if ever, so maybe you need to do something else. Maybe you can, polyfill, emulate some of it, but it might not have the performance that you’re looking for.

So, you might wonder, should I just go native then? But sometimes native is not always better than the Web. There are a lot of advantages to the Web itself. So, for instance, Web is kind of frictionless. Every site loads quickly and requires no instal. Everything starts at the URL.

You don’t need to download 100, 220 megabytes before. It’s also composable, meaning that you can embed foreign content sorts of, for instance, the tweet inside your news article, and you have these links to everything.

So, you can share a link to a specific tweet. I could do that with my mother.

She can click on it and she will see exactly that tweet. She doesn’t have to instal in that first and then figure out like, where was that tweet in that app.

And out of the box, it actually does accessibility quite well.

also native there’s a different skill set and requires is different per platform.

There’s like native windows, native Android and native iOS. It also doesn’t scale so well with different form factors and it can get quite costly.

So, you might be wondering, well, there are, I’ve heard, there are some kind of work around, like you just mentioned an Ektron, didn’t you? Well, yes, those exist, but it’s not always the best of both worlds. That’s not really a thing.

For instance, with this abilities, you often lose like deep-linkability.

You often gain friction by having to instal upfront. And often UDs are extended by native capabilities and thus require have like native dependencies per platform. And though you can embed Web things into this experience, you might not be able to do the opposite like you getting embedded into that in other experience. So, you might wonder, Can we fix this? How can we regain trust in the Web platform? So, you might be wondering like, What is blocking people from adopting Web technology? And this is my set of, this is what I believe, at least like when I’m looking at something I wanna do. I think sometimes you might have existing potentially, very very performing code that might not be as easy to bring to the Web.

You might also have bad performance such as scrolling, animation, on like one of your target devices, which might not be the fastest phones in the world. Sometimes there’s a lack of like a good toolkit or to have like platform tie-in.

And they’re like, just capabilities.

Like there are all these things you can do on native that you just cannot do on the Web.

But it’s not really that bad.

If you look at it for existing like code, you can bring that to the Web.

Now with newest standards as Web Assembly, SIMD+ Threads and in the future we even getting Web GPU. For bad Scrolling performance and alike, we often have things to make it able to run things in different threads, such as com link, or you can import from a worker like these libraries that build support on top of Web workers and works quite well.

We’re also getting CSS Houdini, like CSS Paint, Animation Worklet, and the like and we’re getting more and more APIs from the Houdini project.

It’s taking a while, but eventually it will just be there and be part of the platform. There’s also something called sub-tree visibility which is invented so you can do something like virtual scrolling list, or at least getting the same performance characteristics. If you look at the toolkits, this is partly solved by Web components.

They’re quite popular at the moment.

You have ionic, material web component and newer design system such as FAST by Microsoft others by SAP, Salesforce, Adobe are all based on Web components as well.

But we’re here to talk about capabilities.

And this is where a puffer fish comes in.

You might have something like Autodesk, Auto cut that runs really well in the Web browser using Web Assembly. But a lot of people working with audit, destroying stuff and have local files, and you might not necessarily want to upload these to a specific or the disc store solution on the online. And it might even be legal reasons why you cannot do that. So, this is where product Fugu comes in.

This is a project that is started by Google, but has been joined by my employer Intel and by Microsoft.

Fugu is the Japanese word for the puffer fish. This is very nice cute fish you see on the picture. This fish can actually be eaten and it’s apparently very good.

I have tasted myself and it was actually quite delicious, but here because it can actually be very deadly. So, this is kind of what we want with product Fugu. We wanna enable Web apps to do anything.

Basically anything that native apps can do by exposing those capabilities to the Web platform while maintaining user security, privacy, and trust, which are an other core tenets of the Web platform. If you go to this URL, you can read a bit about like, how the product started and like, what is the plan of it. You can also look at the chromium buck tracker to see everything, all the issues being worked on, which has relation to product Fugu.

But wha t I really recommend you to do is to go to the Fugu API Tracker.

It shows you what everyone is working on.

There is even like a small icon say it was a G for Google, I for Intel M for Microsoft, like who’s working on it. What is for, applies to phones and laptops, you can see if it’s shipping, when you’re supposed to ship, was it when you see the Fugu fish.

If it’s green is something called an origin trial and all of that is available from that site. You might be wondering about like, what is the process for enabling all those Web APIs and bringing them to the Web.

Well, the way it works is that you start by identifying needs and use cases.

So, you basically write this call it the explainer. On the screen, you’ll see a link that will bring you to the TAG explainer explainer. Is an explainer that explains you how to write an explainer.

And when you write this explainer, it explains like why, what the use case or what you want to do.

Maybe some solution you’ve tried something you’ve abandoned, you solicit feedback and iterate.

Often you would even send it to the TAG for an early review to see if it’s a good idea. And then you start working on it and formalising a spec and looking for public support.

Eventually you get to the point where you can do like an original trial and then eventually even ship it. To learn more about original trial, I’m not going into that because Phil Nash will explain all the details about origin trials.

So, definitely you should watch that.

I haven’t got a small list here on the screen. This is like the way it works, to do like short summary is that you can go to this specific Website.

There’s another site for Edge, and you can find a feature that is an origin trial, and you can get the token and this token on your Web app, inside the CML or it that a header.

You can actually enable that feature just for your Website until it’s actually shipping.

So, you might be wondering like, what are those APIs? You said that you’re going to like expose all native capabilities to the Web, But what are you working on currently? Well, some of the first thing we started working on was actually hardware connectivity.

And this was actually before a product Fugu. So, there’s like support for a Bluetooth.

This is not all of Bluetooth.

This is not Bluetooth classic, it is actually what’s called a gap profile or Bluetooth Low Energy and even like USB the support for USB.

And this has been really, really important for a lot of businesses.

I know of a few, like one company did like CPR dolls and they’re connected to a computer where they can do trainings for resuscitation dolls, like for medical use and the like, and they can do that and build a Web app.

And it doesn’t matter whether they’re using different versions of windows and a Chromebook and whatnot, as long as they have a process to support Web USB which is Chrome and Edge at this point.

Really great.

Of course, also a lot of people that are playing around with Bluetooth to connect to their own devices. For instance, here, you see like a Batmobile squid with Lego and I’ve been playing with this myself. I’ve even been working on it on some IOT products where we could connect like IOT device tool, like an IDE that’s running in the browser and just code directly.

And it was like really, really nice.

This is something that’s really good for education, easy to deploy, you don’t need to store anything. If you wanna learn a lot about Web USB and Web Bluetooth, I really encourage you to watch this, all the talk that I did a couple of years ago, it goes into a lot of details.

So, this is shipping today in Chrome and Edge. So, there’s another API.

A lot of people say that, when we’re having like a Web USB API, that’s really cool. And a lot of like devices today use something called serial connection that is today, often over USB. But sometimes you don’t have the ability to go and change these devices and sometimes do host OS like windows would pick up that device because you said, “Oh, it’s a serial device, I know how to deal with it” which meant that it was really difficult to make work with Web USB.

So, there’s also coming a specific API for just for dealing with serial.

This is currently in an original trial gathering feedback. There’s also Web HID.

HID was invested for USB, it’s called Human Interface Definition, I believe. And a lot of like keyboards and mouses uses that mices, but there’s also a lot of other things like maybe it could be like a barcode scanner that you could just connect via USB and the like an impasse game pass.

So, this HID can actually also work over Bluetooth. So, it’s a more broad than standard, but we’re also adding support for that in Chrome and is currently behind a flag.

Something I’ve been involved with myself, is called NFC or Near Field Communication.

Near Field Communications is a lots of different standards is really big, really on top of like low level standards, not a standard on top.

One of them like more generic standards for NFC is called in-depth.

It’s a more simple standard basically for data exchange. It stands for NFC data exchange format.

And this is what we have been working on supporting on the Web as well.

So, you can read and write and have records and is really complete and that’s support, and you can learn more on web.dev/nfc.

I’ve also written a few examples, So, you can play around with those.

I even showed it to the inventor of the Web searching Berners, and he seemed pretty excited about it.

So, I really liked that photo.

We also have a small example of like, you can do a game where you can really like do a game or something with NFC, and it’s all really, really great.

You can do a lot of things with us.

So, here’s another example I did.

Let’s just have a look, you see I created like a shopping list, So, you can add and remove items or check them. So, now you’ve bought everything you need for the week, this week.

See something I never needed again, I can remove it. But the cool thing is that I can, instead of just like adding things to my shopping list, I can actually click on this thing, like write to NFC tech instead, let me just do that now. So, right to NSC tech.

And now if I can write to this in a NFC Tag every time in the future, when a tap it’ll just automatically add that or re-add that item to my shopping list.

So, if it’s about having the kitchen where I’m needing some, I don’t know, new apples, and I’ll just like tap this every time I’m out of apples. So, very nice example, I find it pretty nice and useful. So, you can use this for all kinds of other things, even like treasure games and even integrating with existing solutions based on NFC.

So, we’re here, just have a short example.

It just shows you like how simple it is.

Like you just create a new NDEF-reader and then you subscribe to reading event.

And if you wanna cancel you just essentially use the Abort-control and cancel that.

Of course, like all the data, every NDEF, NFC tech based on NDEF would have different records. So, you can like iterate over these.

For instance, one might be an opaque record was like having an image, like a main type. And then of course you’ll have to deal with that and here in this example, I’m just like getting the data out and putting it into an image element and putting that into the pitch. So, this is clearly also in origin trial, you can play around with it today.

Something else we’ve worked on at Intel, which is a bit of a while ago is Generic Sensors. So, we looked at the different sensors on the Web as a sorrt of (mumble).

They were like different.

They’re not configurable.

Like if you didn’t need like 50 faces per second, like there’s no way that you could say that, “Hey, I wanna, say, battery life,” and just giving them like 10 face per second. They’re also very high level.

So, there’s a lot of interoperability issues between different implementation of browsers. And like, if you wanted to do your whole like sensor fusion, like you are lacking timestamps.

Sort of basically a lot of issues with these. So, we really started from scratch.

I did a much more modern API, and this is like, we have different sensors.

Like it’s a relative gyroscope, absolute orientation centre and the like.

And it’s actually really interesting.

It’s really interesting about how sensors work and it’s really interesting how’s your sense of fusion? So, I have a very nice presentation.

I think it’s very interesting.

So, if your fingers interest as well, you can check it out at bit.ly/generic-sensors. So, here’s one example because the API is via generic. I made this example, this compass thing, and which we liaise in my phone, but I just like create this IOT device and I hooked it up via Bluetooth.

So, I just emulated a generic sensor API using Bluetooth, and it just worked out at the box.

So, I think that’s a really nice way to show you how flexible this API is.

Not all browser supported though, as this currently only in Chrome based browsers. So, for instance, if you have iOS, you might still wanna use this API.

For this reason, I created a set of polyfills that allows you to use this on processes that doesn’t support this standard yet.

Something else we’ve been working on is very popular and very important in this day of sharing and caring. Like it’s about sharing content and receiving shares like on social media.

You might see this very nice Wikipedia article about Fugu, and you really want to share that to your friends. So, it would be nice to be able to bring up this share sheet, like maybe go through messenger or messages and sending it to a friend.

You can do that using WebShare.

It’s a very, very basic.

Like you basically just call navigate.share with the share options, and that’s it.

The other side of this is receiving shares. And this you can also do with the WebShare target API. Is a part of, you added to your Web manifests for your progressive Web app, because it needs to be installed.

Otherwise you cannot share to it.

And then you just fill out these things and you have like a, a place where you basically use the form submission thing for receiving these shares.

And this is very, very flexible as well.

This is shipping today, but we’re currently working on supporting, sharing and receiving files as well.

Talking about files, what people normally care about, especially on the desktop is reading and writing to files. And for this, we have the native file system API which also is work in progress that allows you to get access even like permanently, like for your session and when you restart to that directory as well. This would allow people and application sort of visual studio code to work on the Web.

At least with those features needed for file access. Here I have a small example of how this API looks with a proposed API.

It might not be final biggest part of the original trial process is getting feedback and improving upon the API design.

So, this is going for an original trial.

I think it’s a second origin trial because you’ve already gathered API feedback and even moved the new version to be based on the stream standard.

We also care a lot about like media and playback on the Web. And for this, we have the media session API that allows you to get access to this like permanent notification.

That’s like permanent while you’re playing something where you can see like, what are the cover art you can skip track and the like.

And this also works with the buttons that you might have on your keyboard, like even on a laptop.

So, you can hook into these very simple API. You just sit and action handler for each of these like previous, track, play, pause, and then you do whatever you need to do.

This is shipping already.

Another thing we worked on at Intel is the media stream image capture.

So, really wanting to make it possible to do like all you see all these fancy apps like Instagram and Snapchat, and now TikTok. Why can’t we do all these things in the Web? So, we’ve been trying to expose more and more of these visual camera capabilities to the Web Platform. So, even like depth stream extensions, pan, tilt, zoom, and the like so, for instance, for a media conference, you might be able to have like soft or mega zoom in on who’s talking, if you’re in a room and the like, and you can even do fancy things, like there’s like funny hats assembled that my coworkers worked on.

So, here’s another example.

You see the pen children zoom to light and it’s like really, really exciting what you can do with this.

This is shipping most of it, except the pan, tilt, zoom is coming very soon. We’ve also been working on depth extensions, because there are a lot of new cameras that have a depth sensors.

So, you can do something like the similar, like what you see my coworker (mumbles) doing here. Like he’s not just like trying to figure out what machine learning, where the background is, is actually measuring the distance.

So, you can see like, just whatever that is further away to like, say a metre, like you just like put in Something else there, just replace it.

And like, you can use this for very advanced thing. Like, see, like you can even like interacting with things in the scene, just using these step sensing cameras.

Very, very exciting.

Like I find like this example, like really exciting like picking up something with your fingers, a small box and putting it on top of another box, like all using Web technology.

So, this is currently behind a flag and you of course would need a depth sensing camera for it to work. also what is very common use case today is like, it’s kind of similar to NFC.

It’s like the barcodes it’s because you might have a business, you might be tracking your packages and the like, and it would be nice to be able to use a Web app just to like scan a barcode and then get that into your app without having to create like a native app.

It allows you to find these different barcodes and actually scan it directly.

And it’s very simple.

They even like other API as well as for face detection. And if a text detection.

You basically give it an image, or if it’s extreme, then you’ll check per face and then it just called the detector on it.

This has behind a flag and also intended to be shipping sooner.

That might actually already be shipping, I will need to check.

So, something else that is really cool about like reservoir is the whole system integration. So, progressive Web apps.

It would be nice if you have it.

Like you can see in, for instance, the tray here that listen to message.

So, for this, we have API that allows you to set like, a dot or just like a number.

It might be the same platform don’t allow having like a number, like for instance, Android will only just insert the dot.

So, this is also already shipping.

So, very cool.

And our API I’ve been working on myself is called Wake Locks API.

This is about keeping your device alive.

So, imagine that, you got this nice recipe Website and you’re cooking. You have really dirty fingers.

And then the screen turns off.

You continue and it turns off again, wouldn’t it be nice that you can just like ask for the screen to not be turned off just like when watching videos. So, you don’t get into this experience where you get the screensaver or just a black screen.

So, this is where the Wake Lock API comes in. So, this could also be used for navigation and the likes. Directly to different kinds of wake locks.

There’s a system one that will keep the CPU awake as well as just the CPU awake, and there’s a screen one, they’ll keep the CPU awake as well as the screen. Currently in Chrome, we only implementing the screen Wake Lock.

So, it’s a very simple API.

You request a Wake Lock and that’s it.

And then you can cancel it at any time.

So, this is just shipping since yesterday in Chrome 84. So, it wasn’t ordered to until then.

In a lot of third world countries, people have been asking to be able to get access to like your contacts, because it’s nice to like, to really boot start and you like social network experience. Maybe you just like need to get their phone number or something, or the email address for just sending an email at one point.

So, this is like, it’s really scary about sharing all your contact to the Web.

So, we’ve been working on an API that really puts the user in control.

So, first of all, the developer will ask exactly like, What do you really need? Like email, phone number, name, and when you ask for this, then that will be showed up.

But then the user is in control.

They can select one.

If there is multiple, they select the consent, like maybe multiple contact, not all of them. And then those will be shared as an eerie, like you see, like I’m sharing my own name and my email to this Website. So, this is shipped already, but people are working on a few improvements. We’ve also been working on, this is a very tough one. There’s a lot of abilities of doing fingerprinting with different stalled funds, but we’re really, really working hard and figuring out the best way that we can bring like actual local fund access to the Web, Because this is really important, not for like everyday use cases, but for people who are doing like, like really like working on design tools and stuff like that.

You really want to be able to access your local funds, even like doing a presentation.

I might want to use the Intel fund.

And I might only have that available online, like locally on my machine.

At Intel, we’ve worked together with Microsoft on supporting an upcoming kind of devices called like Daul screens or Foldable screens. So, foldable screens is the screen that can fold itself. Dual screen is when there’s like two connected screens, but it’s like kind of the same idea.

But you might have like a hinge in between. So, this is what you see in the screen now is a foldable screen display.

And this one is a hinged, a dual screen.

So, they’re all kind of like nice experiences you can do with this.

But you can imagine like all of this will be seen as one view port.

So, that might mean that your context like caught in the middle.

So, if you have like a send it model dialogue, that will just be shown in the middle, and that is not really what you want.

So, we wanna be able to do something called spanning where you can like put something on one screen and something on the other screen.

So, this example we did at Intel, So, she has like very nice experience when you’re on one screen.

If you put it in the middle, suddenly everything is cut.

And if you click on an image, that image will show up in the middle.

This is not really the experience that you’re looking for. So, using spanning, we could do better than that. So, let’s see if it does that again.

Yeah, this is the same thing.

So, spanning it’s this idea like that, you know exactly what will be on what screen. So, there’s, this is implemented by a CSS feature called Spanning Media Query and some CSS environmental variables.

So, you really know where things are put.

So, this is really, really interesting.

And this is how the experience look when you use those new features.

So, you can show like all the images on one screen and the big image of clicked on, on the other screen. So, this is behind the flag in Microsoft Edge. And we also worked on a few, like polyfills that allows you to play around and work with this and already create these experiences today.

Microsoft is also working on really customising the title bar.

So, we talked about this integration of like how native apps look and what they’ve done is that they’re looking at a way that you could really customise a status bar and the title bar without actually like, without like fishing.

Because if you could take over everything, you could make it look at for instance, like Chrome, and maybe you could fish some information from people. So, it’s kind of a complicated situation, but they’re working on mitigation that will make this safe because they have noticed that a lot of experience today actually use all that real estate.

Like you see teams on the screen, it has a search bar and that the same does a Spotify. So, this means that today we have like all, we already have all kinds of APIs on the Web, but very soon we are gonna get all these new Fugu APIs as well.

And a lot of these have already been shipping already this year.

If you have something that you need for your experiences, something that you need to build and embed on the Web, please go to bit.ly/new-fugu-request and file a request for this feature.

To learn more about Fugu, please consult the API tracker, and the capabilities product on developers.google.com. And with that said, I would like to thank you very much for your attention, and I hope you have an awesome conference.

Bye.

(upbeat music)