Performance Upgrades with WebAssembly and the Edge

Hello, welcome to my talk: "Performance Upgrades with WebAssembly and the Edge".

So, what will we be talking about today?

First, we'll be talking about WebAssembly, also known as WASM.

(or Wasm}}.

Wasm is a relatively new portable binary format that runs on the web.

We'll be talking about how it runs on both the browser and the server and Wasm's relationship to JavaScript.

Then, once we talk about what WebAssembly is and all of its benefits and cool stuff, we'll be talking about some of the awesome upgrades you can get from implementing WebAssembly in your front end today.

Then, we'll be talking about 'the Edge', specifically in terms of Edge computing and how it differs from traditional distributed computing that, you know, we see oftentimes being used.

Then we'll be combining the two.

Why is WebAssembly such a great fit for Edge computing?

You know once we discover these two things?

And then we'll close things out with: "What benefits can you get from using WebAssembly on the Edge?

And specifically how a front end application or a front end developer can use WebAssembly at the Edge for their own applications.

So first, what is WebAssembly?

This image is from WebAssembly Summit.

It was the first WebAssembly conference, if I am not mistaken, ever!

So that's really exciting.

And this was back before coronavirus.

We just had the second one and then hopefully a third will be in person too.

We'll see, you know, we'll see how things go.

Back to what WebAssembly is.

WebAssembly is a universal low-level bytecode on the Web.

Because of this, WebAssembly's really, really good at running logic-heavy tasks compared to JavaScript.

So just things like image manipulation, physics for games, or any type of heavy math computation.

But why is it great for this?

This is because WebAssembly offers predictable performance.

You see, JavaScript needs to be parsed, evaluated and then compiled many times, and this can get extremely fast or fall off this fast path as we keep compiling over and over with a Just-In-Time compiler that a lot of browsers ship with.

Bytecode offers a more consistently predictable performance that doesn't fall off any fast path.

It's just already fast and just stays that way.

So, in Wasm's current state, a lot of runtimes don't run the bytecode with any interpretation, it just runs straight through because it's already like ready and compiled and ready to go.

Doesn't need to be evaluated or parsed or anything like that.

There are some WebAssembly interpreters, but that's more of a small detail that, you know, it's kind of used on IoT and things.

We'll just keep it pushing.

You'll have questions you want to ask later.

So, one more thing about WebAssembly is that it's extremely portable.

And by portability, you know, let's talk about where it runs.

Currently it runs on all major browsers.

According to 'Can I use', (caniuse.com lets you show and see like all the different web specs and how many browsers support them and things) it's 92.9% of users globally.

WebAssembly also runs in Node, so you know, you can have some image transformation that you want to run on your server or you want to bring it to the client.

You can use the exact same code in both places.

There's a lot of standalone WebAssembly runtimes, such as Wasmtime and Lucet.

So if you did just want to run WebAssembly, there's a runtime for that.

And there's also this thing called WASI, which is the WebAssembly System Interface.

So if you want to do things like file manipulation or other, you know, other like, POSIX-ey type system calls and things.

What's another good one?

Like time, for example, on the system - WASI offers those two WebAssembly line areas if you have the right runtime and environment.

And it's also very 'bundle-able'.

So what I mean by this is that if you want to put WebAssembly in your JavaScript library and throw it on npm, you totally could.

Like, it would just fit in there and then you can bundle it and then ship it out.

So, it'sreally exciting!

And there are a lot of languages outputting WebAssembly.

The most mature three are: AssemblyScript, which is a TypeScript-lite language that compiles to WebAssembly - as a disclaimer, I'm on the team there - and what we mean by that is that you can't compile just some Node TS or React TS application to WebAssembly and get it for free, but it is very close.

Like if you know TypeScript already, it is like: you can read it and you can write it, it's, you know, feels very familiar.

You use npm for all the tooling, and so on and so forth.

It fits into a JavaScript workflow for the web.

And speaking of tooling, we have things like as-pect, which is a testing library for really good unit testing and things of that sort.

So check it out if you're interested in watching this talking and you like what's going on here.

Another language is Rust, which probably has, you know, some of the best tooling I would say.

It's very mature and it's a great language if you want to check that out as well.

And there's also Emscripten, which is a toolchain for C.

It's super mature - it evolved from kind of the precursors of WebAssembly from asm.js.

Essentially it takes C and C++, and turns it into WebAssembly.

Another thing about WebAssembly is that it uses this concept of linear memory, and kind of the way I can describe it to someone who's only written JavaScript before is it's kind of like imagining you only have like a huge single heap or array that you can write to.

And so it's just one single heap that gets shared that we can write to between like the hosts being like JavaScript and WebAssembly or guest modules, let's say.

It makes it really easy to sandbox.

So, you know, if you have multiple WebAssembly modules, they're not going to like write over each other's memory, because they all have their own sandboxed memory.

And it also leads to all types of different, great security benefits as well.

Another thing is that WebAssembly relies on a capability-based kind of permission system.

So for example, WebAssembly modules - like the guests if we want to call it - the modules that we load to JavaScript - the host (so in JavaScript) can only provide access to WebAssembly what it wants to.

So for example, WebAssembly can't go and decide to just manipulate DOM or something like that.

Like, JavaScript will just say, "Hey, here's like a handle and call this function and I will do that on your behalf." So WebAssembly itself can only really do math and move memory around.

But in terms of like - where even when it's moving memory - only that sandbox memory that it has access to.

But in terms of doing, you know, more dangerous operations - that has to be provided by the host.

And again, another really cool security benefit of WebAsembly.

So cool.

We've talked about what WebAssembly is and all these cool features, let's talk about how we can apply it to our browser applications.

So, probably my favorite most recent case studies of like using WebAssembly instead of JavaScript is in an article by Marcel Duin.

The article documents (I hope I said the name right?

Marcel Duin?

I think so]]], but the article documents their experience of moving their application, which is a - it's called Micrio - it's a storytelling platform for kind of like Storyboarding and things like that.

And it used to be a JavaScript only application, and they decided to introduce AssemblyScript or WebAssembly into their JavaScript application and kind of like took some benchmarks and things and see how it improved their app.

So the first thing I'd like to highlight is they got a drastic code size difference.

They were able to shave off about 73 kilobytes off of their entire application which their java uses, which is a huge win.

And if you're looking at the numbers, their new application is about 0.7x the size of their old application.

So like, you know, I guess you can say they shaved off like 30% of their bundle size, which is like really exciting and it'd be great for performance, especially over slow networks and things.

And they also highlighted some of the CPU usage that they're going through.

So for example, now that they're using WebAssembly, they're using 65% less CPU for those performance-intensive tasks.

And this is awesome because, you know, from this, we can assume that their application's a lot more snappy.

And even things like battery life, like their application isn't as demanding on the device in terms of resources, especially things like battery as you're going ahead and using it, which is really exciting and kind of like a cool benefit you get from using WebAssembly.

I also have my own article highlighting the improvements WebAssembly can have on JavaScript applications.

I have a project 'WasmBoy', which was an emulator that I had written for WebAssembly for all these - for compiling to WebAssembly - and try to try it out for myself.

But the thing that's exciting about this benchmark is I was able to take my AssemblyScript code - and again it's so similar to TypeScript - I was able to take the same code base, compile it to WebAssembly with AssemblyScript and TypeScript with a TypeScript compiler, and then able to run them side by side and see like, you know, how this runtime performance - kind of trying the answer to the question of like, "If I had just written in TypeScript normally, would I have gotten the same benefits?" Or like, "Was it really worth it?" kind of thing.

So yeah - let's take a look.

So what I saw in my experience, and again, your experience may differ depending on the application and things, but for my emulator, which is very compute intensive and things, what I saw was about a 30% increase in performance for WebAssembly on like desktop Chrome.

But the most interesting thing that I saw was that when you start to look at more of the less powerful devices, that performance delta starts to increase, and they start to, you know, you get better performance.

So for example, I noticed a 60% increase in performance on these budget Android devices that I was testing on and things.

So that's really exciting.

Shifting gears, I want to talk about what was some of this code portability.

So Esri is a map, they're like a mapping API and they have their own kind of like mapping system they use and things, but they have a mapping projection engine called 'ArcGIS'.

ArcGIS?

I don't know how to pronounce it properly, but so what they would do in this mapping projection engine is that for their more powerful devices, what they would do is ship down the WebAssembly module and run it on the client.

But, and, you know, from this, that's great because if you have a fast device that can handle running with WebAssembly, then you don't have to make any more slow network calls.

You know, kind of like waiting for the API to get back to you with all of the mapping stuff, you could just do it on the device.

But, if your device is less powerful and the network calls are going to be faster when you're trying to compute all this mapping stuff, you can compile the same Wasm code and you run it on the server.

So something we had written for the browser can now run on the server for our less powerful devices, so we're not hogging up all the CPU and draining its battery, trying to render whatever we're trying to do in our application.

So like, what's really cool to show here is that, you know, you can kind of have like the same code base and have it benefit both your fast and your slow devices and get some better performance there.

And this code portability can also happen across multiple platforms.

It doesn't just have to be a browser/server thing.

For example, Google Earth has a talk they gave at WebAssembly SF, which is a great resource for learning more about WebAssembly - they have a whole YouTube channel as you'll see in this slide here.

And what they do is they compile their whole code base with Emscripten - it's a C++ code base- to WebAssembly.

And what's great about this is, now they have one code base and what they can do is compile to a WebAssembly module that gets run inside of JavaScript on the web app, but also that same C++ code base is what they use in you know, their different platforms.

So like Android and iOS and things that they ship for Google Earth.

So it's really cool to have one code base that you can share across so many different platforms.

So code portability again.

That's awesome.

Another awesome ability we get from WebAssembly is feature polyfilling.

And I think this is one that maybe not mentioned as much, but I'm really excited about it.

And this is another amazing talk.

This one was given at Wasm Summit, which is a conference I help run with Surma, a friend of mine, and it's given by Brion Vibber.

So Brian gave a talk from their work at Wikimedia.

Due to the various patent licenses on different media formats, Wikimedia had to use more open media formats.

So for example, they would use Ogg Theora for like video and things like that.

However, not all browsers support various if media formats.

In particular, like, you know, you couldn't find a media format that was both open and supported by all major browsers.

So, Wikimedia turned to WebAssembly to see, "Hey, can we use WebAssembly to code these videos rather than trying to get around these funny licensing things?" And what they ended up doing was building their own video player to polyfill these unsupported formats in the browsers that didn't support the formats they wanted to use for their videos.

So that's really exciting because now they get that, you know, kind of the feature they wanted, but polyfilling it with WebAssembly.

Because WebAssembly is fast enough to do things like decoding video and things of that sort.

And there's plenty more.

These are just some of my personal favorite use cases.

I maintain a website called "madewithwebassembly.com".

If you're looking for WebAssembly projects and kind of seeing different use cases there, I'd definitely say check out this website.

Next, let's talk about the Edge, and in particular Edge Computing.

The Edge represents putting distributed computing closer to your users.

So traditionally for example, you would have a handful of servers or data centers.

But one thing about light is that it can only travel so fast.

So like what if they're across the country or across the globe?

You know trying to go all the way across the world to get some data and coming back isn't going to be the fastest thing for your user.

So what the idea of the Edge is, is putting smaller servers closer to your users, and these smaller servers can do smaller tasks like caching and things like that.

And your larger servers are going to be the ones that do all those heavy database queries and things like that.

And this is great because now all of your users across no matter where they are, get a really fast experience all over the globe, really.

And Fastly, which is where I currently work, is specialized in the idea of the Edge.

Current - you know - they have these Edge servers, we call them 'PoPs' and they're all across the globe.

So Fastly has about 76 PoPs as of March 31st, 2021 that are strategically placed, and these locations are particularly chosen to give users the best possible speed, no matter where they are.

Fastly is probably most well-known for its CDN or its Content Delivery Network, primarily used for things like static assets and images and all that caching things of that sort.

And the CDM allows programming in VCL, which is, you know, kind of like DSL that we use that allows you to hyper tune and fine tune things on how things are cached and optimized, but Fastly is much more than the CDN.

And we have multiple areas of products that we have, and we are growing, And again, you know, to this idea of the Edge, one of the places that we're thinking of growing is compute - where we are growing is compute.

You often see distributed networks offering storage at scale, but it's also possible to do things like compute with the same like infrastructure.

It's often known as serverless, which is kind of like these one-off functions that get run, and traditionally serverless has been done on someone else's origin servers, but that space has just recently started moving to the Edge and Wasm's here to kind of do it.

So: here comes the punchline.

Why dive into, you know, let's dive into why WebAssembly is great for the Edge.

So one thing again is that language portability.

Many different languages compile to Wasm, meaning any language that compiles to Wasm could technically be supported by any Edge WebAssembly serverless platform.

So all we really need to do is provide a Slim SDK, and then you're kind of off to the races and just can use the language as you normally would.

Another thing is security.

So again, WebAssembly has that linear memory sandbox to continuous heap and you can't access outside that sandbox, which is perfect.

It also has that capability-based security model.

WebAssembly can only do what we give it access to, which is great because we don't want them doing things we don't want it to do.

So, you know, we can fine-tune and make sure it's - the system has guardrails for whoever's running WebAssembly on our platform.

And again, this is all just kind of like a good mix for like running untrusted code in the platform.

So WebAssembly is offering these things for us, which is awesome.

And then also the runtime performance is really good for WebAssembly.

Again, every use case - wherever you see WebAssembly benchmarks are gonna be different - but in general, with a big grain of salt WebAssembly is about 37%, not 30, 30% slower than native.

This is very much depending on the benchmark and the use case.

So relative to running just straight C code, for example, there is some performance loss, but very little and much better when you compare like other alternatives ways of running untrusted code, for example.

Memory usage is another good thing.

It's excellent.

WebAsembly modules are extremely small, for example, in AssemblyScript, 'hello world' can fit into about 55 bytes, which is, you know, bytes - so not kilobytes, just bytes.

And also, using one of those standalone runtimes we don't need to use a large OS primitive, like a container wheel, so, you know, we can skim all that off and just use straight WebAssembly runtimes.

And, the elephant in the room is cold start times.

It's kind of always been like, "Ooh, serverless".

But what's good is that cold start times can be drastically improved with WebAssembly and there's a lot less bootstrapping than a container, and you only have to make an instance of the standalone Wasm runtime once really, you know, I mean one instance per WebAssembly module you're running and just immediately start executing it so we can just create that instance and just boom, we're off to the races again.

So it's lightweight and things, which is great.

So now that we understand why WebAsembly is such a great fit for the serverless compute on the Edge, let's explore how this is also great for our front end applications.

So this is kind of like some interesting idea that I have, but I'd like to share an idea that I have about where Edge computing's going to fit into Web Performance.

This is kind of like my crystal ball, and so, you know, maybe we'll see it go somewhere else, but this is what I'm really excited for personally.

I think WebAssembly on the Edge has a huge potential to expand on the benefits that WebAssembly already offers when you're running WebAssembly just on the client in a front end application.

Similar to like, you know, how you add a little bit of lemon to your water - it just makes it a little bit better.

So I'm hoping WebAssembly on the Edge can offer the same for front end.

Yeah, and I think all in all, again, jumping back to that lemon analogy, I think it offers its own refreshing solution to a, you know, already existing problem that we're already solving on the client, which is like exciting.

So let's revisit some of those use cases I mentioned earlier and see about, how WebAssembly at the Edge could take you just a little extra step forward.

Not to...

you know...

so...

yeah.

So, cool!

Let's take a look at our feature polyfilling use case.

So the gotcha at the end of Brion's talk is that their library requires Wasm threads and SIMD, and these are both only currently supported in Chrome as WebAssembly features.

These features will come to all browsers eventually, but you know, kind of like who knows when?

It's going to take a while as people work through it, you know - people prioritize their roadmaps and things.

And then how well they'll be supported is probably going to differ between platforms.

The point being is that platform features can sometimes only be available in certain browsers.

So what can we possibly do, is move that poyfilling code to another platform like WebAssembly on the Edge.

So even looking outside the browser, there are features that are unique to platforms in general, that can't be replicated.

For example, in the mobile ecosystem, we can see media fragmentation and support across all the different platforms and things.

So moving some of that logic that, you know, maybe we're polyfilling in that platform.

If you move to the Edge, then it doesn't really matter what the platform supports, because then we can just throw it over there and we get that same similar performance because again, it's so close to the user that like that network call kind of gets to that point of like, "Oh, you know, it's like fast enough" and things.

So, yeah.

You know, we can use WebAssembly at the Edge for feature polyfilling again, but just maybe just a little bit better.

Next I think it's good to touch on the general benefit you get from Edge Computing, closer servers to your users.

Again, looking at that previous example I mentioned with from Esri: the projection engine will offload work to a server.

And when it's on the client, when the client device can't run in time, moving that server-side logic closer to the user on an Edge server makes that tipping point of 'faster on the network' versus 'faster on the client' much shorter, which means that devices that have - will have a more fast and snappy experience on this application design.

And I think it's probably one of the main ideas I find the most interesting.

It's kind of like where that tipping point is between 'faster on the client' versus 'faster on an Edge server'.

Unless we visit my benchmark.

I mentioned I was able to improve performance for mid-range mobile devices or budget mobile devices using WebAssembly and my JavaScript hot pass and my emulator.

But what about even lower end devices?

Like lower than budget Android devices?

Well, if you run on like really low-end devices, you'll see, it's still not fast enough.

So imagine I wanted to run this emulator on like IoT or like feature phones or recycled hardware.

And what if we want to support all users, like really low income users or like NBU next billion users, for example.

And what just, what about ones who just can't afford high-end devices?

Well, I have a personal story on these types of users.

And that story is kind of about Obama.

I want to talk about 'Obama Phones'.

Obama phones is kind of the slang term for a free government cell phone and phone plan.

The program wasn't started by Obama, but it rapidly expanded during their term.

And the program was formerly known as Lifeline Assistance.

And it's quite popular.

I've seen lots of these tents around cities like San Francisco and the Tenderloin, as well as my own city: Long Beach, California.

But I think it's also worth taking a look at the numbers, and you will see there are a lot of people with these devices.

1.8 million people in California - the home state of Silicon Valley - has, you know, are using this program.

And California has a population of about 39.51 million people.

So if our math is correct, that's one in every 25 people that are on this program using these like feature phones or not feature, but 'Obama' phones.

And in the US total population of using this program, it's about 8.7 million people, so that's a lot of users that if your applications are running fast enough, they just can't use your app really.

So there are a lot of these Obama phone devices.

Let's take a look at, well, I guess take a step back.

Let's find a device that these users are using and doing a quick Google search, we can see a general free device that seems to be given by the program is the Alcatel Dawn.

So let's compare this Obama phone to a budget device that I was targeting.

For example, I'm looking at wired.com.

They recommend a Google Pixel 4A - it's about $350.

Let's compare our free device against the best budget device coming out around $350: and looking at device ranks AnTuTu Android benchmarks, the best budget Android device according to wired.com is about 29x faster than the Obama phone.

That is a huge performance difference between the two devices.

So even though WebAssembly was able to enable a new class of devices to run my emulator or my application, it can't magically make it run on all devices.

So how do we produce the CP - amount of load on the CPU, even more for these devices or devices with general limited CPU like in general?

Like ones that just don't have a lot of processing power?

Well, here's an interesting idea.

I was of able to port some of my emulator application, like some of its logic to the Edge.

And again, this is possible because of that amazing Wasm portability.

And you will notice that when I'm running the application on my work laptop, it's using about half of my CPU power, but offloading that heavy emulation task to a server greatly reduces the work that my device is doing.

But when it feels super slow, since all of this is coming over the network, I mean, it's, it's an emulator, it's running 60 frames per second and it just has to look slow, right?

Well, this is where things get really interesting.

Edge servers are really cool.

They greatly reduce that roundtrip time over the network.

Paired with a secure, flexible, performant technology, we can get something that feels like it's running on the device.

Like it feels like it's running on the client.

So please see my example that I'm showing now, this example is from me porting again JavaScript to AssemblyScript on my faster - well I guess taking a step back.

This is an example from my blog post about porting, JavaScript to AssemblyScript on the Fastly blog.

And here I am writing markdown on the left side of the screen.

And then you will see on my network tab, it's just pinging the server, and as I'm making every key down request it does a round trip and re-renders the HTML and you can see, it looks like it's running on the client, but it's actually happening over the network.

Because Edge servers are able to do that roundtrip so fast, it looks like it's happening there.

And that's really cool.

And I hinted at how fast it's going to be, but I'm also very happy to show you, like again, in like a, I guess a gaming context: Justin Lu, a colleague of mine, was able to get Doom running on computed Edge, which is like, Doom the game.

This is an awesome demonstration of the platform.

Again, the average game runs about at 60 frames per second, which means everything needs to be rendered within about 60 milliseconds.

But things could still be slightly playable at that, so if we aim at something for like 24 to 30 frames per second, which is about 32 to 40 milliseconds per frame, we can get something that's achievable, playable and like, you know, running over the network on an Edge server by just pinging the server.

And that's really cool!

Now let me be clear.

I'm not saying you should be building games on top of WebAssembly on the Edge, but what I am trying to show you and show what's possible, is that this Edge server stuff is really applicable to, you know, sometimes we need to run things on the client, but just kind of un-breaking that barrier of, "Oh no, there is some things in the client that could be run on the server that would feel like it's running on the client", offers some great performance and can really help our users out in terms of like the device, whether it be slow networks or, you know, their device isn't powerful enough so that we can throw it on the network and that way.

So again, this technology helps remove limits and improve experiences on our client-side applications.

And there's even more use cases.

There's a ton of use cases for Wasm on the Edge.

Here I focused on the idea of having client-side performance righ in the client, but Wasm on the Edge shines really well for traditional serverless applications as well.

I gave a talk at our previous Web Directions event that highlight this very well, so please feel free to check out the computed Edge developer hub for demos and things, but also feel free to explore your organization and ask, you know, ask around like, "Hey, is there any like server-side code that we could really speed up over the network, you know, by maybe checking out this whole Edge WebAssembly thing?" And check it out.

So with that, let's wrap things up.

WebAssembly is super exciting.

It's a super cool tech that opens a lot of doors.

Again, just kind of like removing those limits, you know, "let's unlock new cool things!" The Edge is an exciting space for compute that can solve many different problems and drastically improve user experience.

And WebAssembly I feel like can be a key part of that implementation and WebAssembly and he Edge with their powers combined can remove even more limits and improve experiences, just even more on our client-side applications.

And I hope you agree, and I hope this all makes sense to you.

And again, thank you very much for listening to me.

I usually name someone from our team, but our team has been growing really well, the whole computed Edge thing, we doing pretty all right.

So I'm really happy I get to work with these people every day.

Thank you conference organizers for having me.

Thank you for watching and please reach out over Twitter or, you know, let's continue here at the conference.

Let's continue the discussion.

I'm excited to hear what you all have to think.

Thank you very much and enjoy the conference.