What the heck is Edge Computing anyway?

Hi, my name is Austin Gill.

I am a developer advocate for Akamai and lately I've been thinking about knitting and selling hat for dogs.

And why am I even telling you that?

Because they're freaking awesome.

I mean, look at this thing, it's adorable and I think everyone's gonna want one.

And it's also a great segue into the topic that we're gonna be discussing today, which is what the heck is edge compute anyway.

And you may have also heard of it being referenced to as 'edge functions' or 'edge workers'.

Before we get into answering that question, there's a couple things I want to cover.

So we're talking about compute and when I'm talking about compute, I'm generally referring to generating HTML for a user's browser.

And there's also we need to discuss some places that are related to where that compute can happen.

So we're gonna cover servers, content delivery networks, client side rendering, and cloud functions.

Now servers are going to be machines that are running some software that maybe you chose that knows how to talk to the internet, knows how to receive a request.

And it's going to execute some code to generate your HTML.

servers can be running locally on machines that you own, maybe in a building that you're in, or they can be in the cloud, which are machines that someone else owns in, in a building that they're in and they are renting to you.

They're going to be running 24/7, ideally.

And you pay for servers, even if you don't have any traffic.

And in the case that you have a lot of traffic, you could run out resources and your server could crash.

So scaling up or down requires some planning because on one hand you wanna have enough power to handle the performance and reliability needs.

On the other hand, you don't want to overpay for servers that you don't need.

And it's important to keep in mind that servers can be very far from users.

So when a user makes a request, they could be halfway around the world from your server.

And there's a lot of time that that request has to travel to get to your server and back to the user.

It's also important to be aware that when you are running a server, you're responsible for that server software and keeping it up to, up to date as well as the application logic for running your actual application.

So we can think of servers, sort of like commercial workspace, where if I'm going to knit hats for people, they may have to come all the way across town.

Come to me, say "hello.

I would like a dog hat please".

And I would say, "what size and what color", and I would knit it up and I'd give it to you and say, "have a nice day.

Here's a treat for your dog".

And they would go back home.

And I am paying for, I may own the building myself, or I may be renting space in that building.

Next we'll look at content delivery networks.

Content delivery networks are a network of thousands of globally distributed and connected servers that are designed to deliver your static assets.

Things like CSS, JavaScript, images, fonts, things that don't change very frequently.

So each server, the way they work, is you take your assets and you distribute one copy of your asset on each of these servers or the CDN provider would actually probably do the copying.

And then when a request comes in, the server that is nearest to the user where the request is made is going to deliver a copy of that asset that they're asking for.

This effect can greatly reduce latency or the distance that, that data has to travel to go from the server to the user because after all the server is right next to the user.

This can also work for, work really well for static site generators or pre rendered HTML, where you have some code or some software that's running on your machine and you press the build command and it generates the HTML file.

And then you distribute that HTML file to the CDN, which gets copied all over the world.

But there's no changing that file per request.

So content delivery networks are a lot like convenience stores that are kind of distributed all over town.

And I can knit up a bunch of the most popular hats in the most popular colors and sizes.

And I can go and put some of them at each of these convenience stores so that when you and your friend want to get dog hats for your dogs you don't have to come all the way across town to my workspace.

You can just go to the couple blocks away to the nearest convenient store and get the dog hats that you want.

Moving on to client side rendering client side rendering is when we deliver some JavaScript to a user's browser and ask that user's browser to, you run that JavaScript in order to generate the HTML on their end.

Now this can work great with CDNs because after all it's a JavaScript file.

We can store that in a CDN and have the user download that that would reduce latency.

But because we're asking for all that logic to happen in JavaScript, we're asking users to download larger JavaScript files.

Another downside to client side rendering is that we can't keep secrets on the client side, because any asset that's used to generate the HTML or any, anything that's running in the browser, you can view source on and see the HTML or the assets for the CSS or the JavaScript.

Another downside is that performance is greatly impacted by the user's device.

So if they have an old device or they have a device that's running a lot of applications at once it may not have a lot of performance available to generate that HTML.

So it might be slower.

And because of this and other reasons, we're, I'm starting to see a movement away from client side rendering, there's SEO and performance and accessibility, and several reasons.

So we can think of client side rendering, somewhat like a do it yourself dog hat kit, where you might come to my workspace, or you might come to one of the convenience stores and pick up this kit.

And instead of having the hat already ready for you, it has the instructions and the tools and the resources and everything you need to make it yourself, but you take it home and you might make it yourself, which may take you longer or not.

Next, we look at cloud functions.

These are also sometimes referred to as Lambda functions or serverless functions.

And these work a little bit different.

They're, they're sort of like servers, but they're slightly different because the servers are running on a service provider and all you have to do is provide the function that is going to handle a request.

So service provider manages the machines and the server.

And when a request comes in, it's passed to the function that you wrote.

So in order for that little dance to work, you have to follow certain conventions around how you name your function, maybe how you name your files or the parameters that your function accepts, and maybe what your function returns.

The reason for this is because, or as a result of this the, the service providers can take your function and run it on whatever machine that they want.

It doesn't matter because they're, you followed the conventions.

So that means when a request comes in, that function can be handled maybe the first request on machine A and the second request on machine B.

Now this introduces the concept of 'stateless functionality' where your functions don't have a shared memory or a shared file system, because it can run on different machines that you have no control over.

You don't know if something that you shared in memory or saved in memory is going to be there the next time that function runs.

However, a plus side of all this is that because those functions can just run on whatever machines the service provider has, they can actually bring on more machines, if you have a lot of requests to your function.

So this is this concept of infinitely scalable, and automatically scalable functions.

As you have more traffic coming you'll have more machines handling that request with your function and as traffic subsides, your function will be called less and there will be fewer machines needed to call that function.

Cloud functions are also very easy to provision, after all all you have to do is give a credit card and then give your code.

And there you go, you get a, a URL that your cloud, that your function can run on.

So this is very handy for teams that are working with maybe different parts of an application, or if you wanna migrate from one function to a new version of that function, you just have a different URL to run it at.

And a nice feature of cloud functions is that you only pay for what you use.

So if you don't have a lot of traffic, you don't execute those functions, you don't have to pay for them.

Something to be aware of with cloud functions is that you will be limited in the amount of resources and the languages available to run those functions because you are not running the server- someone else is building machines and building the servers.

So they get to choose how many resources to make available and what languages to make available.

And in addition, similar to traditional servers, cloud functions can be very far from users because you can dictate where they get distributed around the world, maybe to specific regions, but if someone lives opposite on the opposite end of the world to that region, they may, the, the request still has to travel a long distance.

Big plus side though.

My favorite thing about cloud functions is that you only have to think about managing the code that you write.

You don't have to think about maintaining hardware or maintaining server software.

So, when we think about cloud functions in this dog hat world it's sort of like we trained robots to knit dog hats for us.

So maybe we don't need to have employees that we have on salary that we're covering.

We can just have robots that when someone comes to our workspace and says, I would like a hat, the robot turns on.

It asks for whatever customizations they want, like favorite colors or football teams or whatever, and it builds a hat for them.

And then it gives them the hat they walk away and the robot shuts down.

So we no longer have to even pay for the electricity to run the robot.

It's pretty great.

The biggest downside to this is that there wasn't a, a good emoji to really capture robots trained to knit dog hats.

So I had to make, do with what I.

Now there's a horse for every course.

And what I mean by this is that all of these sort of tools and resources available to us have their own pros and cons.

In some cases, they're going to be better.

In some cases they're going to be worse.

And they're all still tools in our tool belt.

And yeah, it, it's going to be dependent on your use case, which one is best.

So.

With that of the way let's have a quick talk about performance.

In my experience, users experience performance in 3d, which is something that I just made up for this presentation.

And I don't even know what it means.

No, I do kind of know what it means.

We'll get into that.

The first D is distance.

So this depends on how far a request has to travel-in other words, latency-before anything comes back.

The second D is how much that request has to download before the code can execute.

So a browser generally needs most of the resources before it starts showing the user something.

And lastly is that device.

So we talked about a little bit about the hardware capabilities.

If a user has an outdated device or a device that's running a lot of things, it might not have the processing power available to, to do a lot with rendering things or with compute.

Hot tip and a hot tip that I have for anyone writing sort of pitches is that alliteration goes a long way and is sometimes better than coherence.

So in this ideal sort of 3d world that we are talking about, we would do things like move things closer to the user, sort of similar to a CDN.

But we would also do more of the work on servers because we don't wanna rely on potentially old or busy devices.

So we do things like cloud functions or cloud servers.

And lastly, we would send smaller assets to, in a request.

But this is highly subjective based on your application.

So there's not a lot of advice that I can actually give here.

In reality, however, we see that client side rendering is maybe low latency because nothing has to travel very far after all you're doing the rendering on the client, but it can be very slow because user devices are doing other things.

And server side rendering is very fast, but it can suffer from high latency because those requests have to travel a long way before they can even be shown to the user.

Okay.

Now we can talk about the thing, which is edge compute.

So edge compute is really awesome.

If, if I was to make an analogy, it would be like those robot trained, the, the robots that are trained to knit dog hats, but we've also placed them all over the, all over town at those different convenience stores.

So, what we get is edge compute are programmable run times sort of like cloud functions that are globally distributed like CDNs.

And the result of that is we get this dynamic server side functionality with all of its benefits that executes as close to the user as possible.

Less of that latency.

In addition to things like cloud functions or servers edge compute has access to well, many many platforms have access to the location information where that ex...

where that request was handled, which is handy.

This has a lot of benefits for users.

It means a lot less latency because they're not going way around the world for, to wait for requests to come back.

If we're comparing to servers or cloud functions there's less for users to download because they don't have to run compute on their device.

And that's of course, if we compare to client side rendering.

It also keeps a lot of the work off the device, which might improve battery performance or improve the speed of other applications that users have running.

There's also benefits for developers these cloud functions, because again, we're only writing these functions, they might have a low barrier for a proof of concept or to push things to production if we need to, to test things out.

We also get those consistent execution environments, similar to servers, because we don't have to worry about cross browser issues.

And for teams that are working on different things at different times they can each own their own respective part of their responsibilities and have their own set of cloud functions without having to worry about this giant code base or having to be cross functional and know many different languages.

We get a nice benefit of that location based logic without having to worry about things like GDPR compliance, because we are getting location information from a server and not from the user.

And we also don't have to worry about servers or infrastructure to manage or software to update or things like that.

We just run our function or deploy our function and it gets deployed, very similar to a cloud function.

Also similar to the server benefits is that the secrets stay secret.

We can use API tokens and things like that without having to worry about them leaking to the client.

And as someone who does both frontend and backend development there is a common programming language that most platform, most edge compute platform providers support JavaScript, which is the most ubiquitous programming language in the world, which is great for hiring people or, having a, a cross functional team.

In addition to those benefits, there's benefits that are more specific, more specific to website owners particularly by using, by moving compute to the edge, you can take some of that workload off of your origin servers.

Now, this has a direct impact to the performance of those servers how reliable they are, because by having less load, they will execute-they'll, they'll have a less probability of running out of resources and crashing, and this can directly impact cost as well, because you can either choose to downgrade some of your servers because you don't need as much compute power on those servers because some of that compute has moved to the edge.

Now we also get this benefit of automatic scaling, which also impacts that performance and reliability, and there's no sort of hassle or song and dance that we have to go through for that scaling.

As traffic increases, that performance will stay the same.

That reliability will stay the same because that distributed compute will happen sort of globally and scale up automatically.

And, last nice feature, very similar to edge functions or cloud functions is you only pay for what you use, which is great.

Now no technology is perfect.

And of course, I'd be remiss if I didn't mention some of the limitations edge compute platforms are these run times, let's say JavaScript that are distributed globally.

So you're going to be limited on certain platform specific features, because if you think about it it's very, it's very easy to have a node JS server that's running on a single server instance and you can have access to a larger runtime, but if you want to distribute your runtime to 250,000 servers around the world, you kind of want that runtime to be as small as possible.

So most of these platform providers run V8 isolates, which essentially is the JavaScript programming language, plus maybe a dozen APIs, platform specific APIs.

Whereas you compare to nodeJS runtime, which has maybe hundreds of platform specific APIs.

Other limitations are going to be around compute because once again, when you're running so many servers you really wanna be careful on how much compute resources are available to each function running at those servers.

So compute resources like CPU or memory, and a lot of platform providers put more strict limitations on the time that functions can execute.

So the timeouts.

So given all of that, the benefits and the costs it's important to consider that edge compute is not an addition, it's, or not a replacement.

It's an additional option for compute and deciding where you're going to run compute or how you're going to build things is difficult because you have to consider latency and the bundle size and device capabilities and all those things we discussed before.

And before the world looked like this, you had to choose between rendering something or running compute on the client side in main thread JavaScript, or maybe on the client side in a service worker, or as a cloud function or maybe on a traditional server.

And now what we have with edge compute is it doesn't replace any of those.

It kind of sits somewhere in between doing things on the client side or doing things more on the server side.

So to give some advice on when it might make sense, is anytime you're doing these stateless operations or when you're doing things that aren't going to take too long of a time, so just generating HTML is totally fine.

If you're doing things that are latency sensitive, so it depends on how long those requests have to travel or anything that's very, very local, edge compute is a really good possibility for those.

Cases where you might see red flags where edge compute may not be the right idea is things that require memory access to a shared memory or file system.

This is not referring to external services like managed databases, edge compute is fine with that, but sort of similar to cloud functions because requests can be handled by different machines you don't want that shared memory or file system.

Anything that requires a lot of computational resources.

So if you're building data models or doing artificial intelligence training probably not gonna be great for edge compute.

Anything that has long running operations, so things that maybe have to take hours or days or weeks, aren't going to be great for edge compute.

At least not today.

And anything that has sort of sequential or waterfall requests.

So if you have to talk to a database, let's say, and get a, get a response from a database and do that two or three times based on previous responses that can actually add latency because that's very similar to running those requests from the user and going maybe halfway around the world to the database and back, and then forward and back and forward and back.

Whereas if you could move that, do the request from the user to a function or server that lives right next to the database that might make those back and forth trips shorter.

So with all that, the common use cases that we are seeing today for edge compute are things like-anything that has to do with geolocation.

If you have some fast auto-complete, or type ahead feature where as a user types, they get suggestions.

That's great for edge compute.

If you need to do some modifications to a request or response between the user's client, the client, and an origin server.

If you need to do some sort of redirect management where our user makes a request, maybe they go to the edge compute location right next to them, and they're going to a page that no longer exists and needs to be redirected somewhere else.

That's great for edge compute.

You can also do things like token based personalization things like A/B testing or turning on or off feature flags for users, maybe based on something like a cookie.

Stateless authentication authorization.

So if you have something like JSON Web Tokens you can see whether a request is authenticated as it comes in at the edge.

And if it's not, you can maybe send that user to an unauthenticated page.

And if it is authenticated, you can send that along to the origin server.

Doing things like API proxies or orchestration can also be a great use case for edge compute.

So in my opinion edge compute sort of looks like the next phase of Web development where it sort of sits in a perfect space in between the client and traditional servers.

So what does this mean?

Dog hats for everyone.

It's going to be awesome.

And I'm, I'm very excited.

I think it's, I think it's gonna be the next wave of, of change.

And I'm not the only one that thinks this because we're seeing a lot of major framework builders starting to either adapt for edge compute or even build directly, specifically to live in an edge compute world.

Astro, Remix, Next, Nuxt, and more.

It's it's going be great.

It's super exciting.

But also, this whole conversation gives me a bit of existential crisis because after all, when I think about this thing, yes, it's exciting.

And, and the opportunities are endless.

But at the end of the day, we're mostly concerned with latency here, reducing the amount of latency and how much latency is actually how much can we actually reduce by moving things to the edge?

Maybe hundreds of milliseconds.

So is 300 milliseconds even a big deal?

And anytime we have these conversations about performance and latency and speed we see some compulsory slide with lots of statistics.

So.

Of course, I need to have one.

In 2017, Akamai did a, some research on the retail performance for online retailers and the correlation with performance and sales and saw that, yes, in fact, there's a lot of reasons.

There's a lot of correlation between speed and sales.

So speed is good for sales.

And I saw another study that suggested that, Walmart for every second of speed that they increased, they saw sales increase by 2%.

Now in 2021, Walmart made 500 billion.

And so if you take that and take 2% of that, that's about $10 billion.

Which means that if Walmart hired a 133,000 developers to take one second off of their website, based on the average salary of developers, they would still make a profit.

So they could hire 133,000 developers and still make a profit as long as they, as long as those developers made the second, made the site load one second faster.

Which is crazy.

So.

Why is it always about money?

That's another question that kind of bothers me because I don't know if that's a major motivator for me.

Those aren't the sorts of sites that I build.

And the answer is kind of that money is very easy to quantify.

It's something that is very understandable by a lot of reason, by a lot of people, but I kind of feel like there's other reasons beyond just money to care about performance.

And yeah, speed and user experience is a good thing and reliability is a great thing for users.

But I think as I've been, as I've been thinking about this topic, I came onto something else, is that the importance of access to information matters a lot.

Reliability corresponds to access of information.

And in times of unprecedented crisis, like global disease or war going on, a lot of people are dealing with fear.

And when you're living in a world of fear, access to information is vitally critical, vitally important.

So for some people, I'm sure that it's not as important about money, but it is still the speed.

And the 300 milliseconds actually still matters a lot in certain scenarios.

So is it worth it?

For some people undoubtedly, it's going to be a yes.

For other people, maybe not so much.

So it depends, but I'm still very excited about the prospect of edge compute and what it means for the shift in the Web development space.

And I think it's going to be a major shift and I hope that you are interested enough and you wanna give it a try.

So I just have one more slide here somewhere around here.

One minute.

That's not it.

Hold on one second.

Okay.

Here it is.

Yeah.

So I just wanna say, thank you.

Thank you so much for giving me your time and attention and being here.

I also want to ask one thing of you is that if you could reach out to the other presenters and the organizers of this conference and give them your thanks and let them know what this conference means to you, because I know how much time and energy goes into putting something like this together.

It would really mean a lot to me, and I'm sure it would mean a lot to them as well.

If you have any interest in checking out more information on Akamai's Edge specifically you can head over to akamai.com or these other links.

And if you have, if you wanna reach out to me and you want my information you can find me austingil.com or any of those other links except that last one, that last one is for my dog nugget.

So thank you again.

I really appreciate it and I hope to hear from you sometime soon and I'm excited about the edge.

It's gonna be great.

See ya.