Hello, everyone.
I hope you are enjoying global scope.
I am James Snell.
I'm a core contributor to Node JS, a member of the Node technical steering committee and I'm systems engineer at Cloudflare on the worker's runtime.
Today, I'm here to talk about the work to establish a standard API library of sorts for JavaScript run times.
Start with an example.
What is the correct way of generating a SHA256 digest over a stream of data in JavaScript?
Is there just one way of doing it?
Unfortunately, the answer is no.
If you are using Node.
The way you'd generate that digest is different than if you are running Deno or Cloudflare Workers or Web browsers or any other JavsScrip run times that are out there available for you to use.
Now a lot of the run times like Deno and does have a Node compatibility layer that's being developed.
I mean there's ways you can make it work from one the same way across platforms, but outta the box it's not easy and they work differently.
When I first started contributing to Node seven years ago, I was repeatedly told by several of the core contributors at that time, that Node was not a Web browser and it should never behave like one, the reasoning this reasoning has been used for years as the justification for Node not to implement the same kinds of APIs that developers have available in Web browsers.
Readable streams for instance.
Right?
Or Web crypto or abort signal, event target, blob, file.
Whatever.
Node has a streams, API.
Why would it need to implement this browser streams API?
Node has a crypto module.
Why does it need Web crypto?
Node has event emitter.
Why does it need event target?
There's been this ongoing conversation for years.
And, this reluctance on the part of the Node project to look at these things that are that are happening out in, browsers and say, "Hey, you know what, maybe those APIs would work here.
Maybe we should provide them here".
And the justification has always been well we're not a browser we don't need it.
And while it is true, that Node is not a Web browser.
The fact remains that there is still significant overlap and the kinds of things developers need these platforms to do.
It doesn't matter if your JavaScript is running on a browser or on the edge or on a server or on IOT device, you'll still need to access access into the same fundamental capabilities.
You're still gonna need to be able to stream data.
You need to be able to parse the URL.
You need to be able to schedule and cancel asynchronous tasks.
You access crypto these things are ubiquitous, so it doesn't make any sense for them to be different in every single JavaScript runtime you're running.
Why is there a separate API for for streaming in Node versus everything else?
Why is there a separate would be, there would be a separate API for the file system access?
These things should work the same way, no matter what JavaScript enabled run time you're using you are after all, all using the same programming language.
And if you look at other languages, know C++ has a standard library Go Rust all these other environments have these standard libraries and you can write to any standard APIs that you can write to so that things will just work.
But in JavaScript, we haven't had that.
All right.
For the past year I've been working directly on Cloudflare Workers.
Workers is a JavaScript runtime, serves HTTP requests and more from a globally distributed edge network.
It sits between your browser and an origin server.
Runs JavaScript there in the middle.
One of the top requests we get from users, is the request to support any arbitrary JavaScript module published out on NPM just like XYZ module, just install this and, make it work in Workers.
There are a number of modules that will just work and unfortunately reality is that there are many more, most of them, they just simply don't, simply because Workers is not Node.
And most of those modules that are in are NPM have been written going back for the past 10 years, 11 years of Node JS APIs they're using things like the buffer API, which is specific to Node or Node streams or Nodes event emitter.
All right.
These are things that only exist in Node or are under Node's control and Node can make changes whenever they want.
And we just don't have those, the implementations of those APIs in the Workers environment.
know, Rightfully so we don't like for instance, we don't have a file system in Workers.
Having a file system API doesn't make sense.
But it's there, there's all these things that just out on NPM that are just written for a very specific platform, but need to be used everywhere.
Now, Worker's, users have found their way around such limitations, by using polyfill implementations of these APIs and they take those polyfills, which are just like pure JavaScript implementations of, these Node core things.
Sometimes they're, just handicapped.
They don't actually do what the full features of, a Node API, but they, get close.
And and they'll take these JavaScript polyfills and bundle 'em with their worker scripts and deploy those.
And often these are the same polyfills that developers use in the browsers.
They're out there.
They, work to an extent and while we absolutely can make things work using polyfills the fact that we have to rely on them is, pretty unfortunate.
Now these are after all just JavsScript runtime environments, why can't things work the same way across multiple, these multiple environments without relying on polyfills of platform specific APIs?
Certainly where those areas of overlap exist.
Modules that are out there doing crypto why do they have to depend just on Node core APIs?
Why can't there be a standard?
Fortunately there are standard APIs out there that we can use to start helping drive more alignment between JavaScript runtimes.
All these are APIs that developers are already familiar with.
We're using in the Web browsers-readable stream for instance, or URL or a abort controller, event target things I've already mentioned here developers are already using these millions of developers around the world in, millions, upon millions of applications are already using these things.
They are, they've already gone through standards process.
WHATWG or W3.
And, they've already been tested and they've already been they definitely work, so why can't we just use these APIs rather than creating something new or creating something specific to any one platform?
So the standard API for JavaScript are all the APIs, the developers are already using in Web browsers today.
And it's, important to recognize where these are, but also recognize some of the limitations for these things.
It, it was probably about five years ago that we introduced URL to Node.
There's always been a, an API in Node already URL dot parse for parsing URLs.
But about five years ago we, started receiving some reports of some bugs in that implementation.
And I started looking into those and found that I could not actually fix those bugs without completely killing the performance of URL parse.
That code has been very highly optimized to be fast.
And every time I started poking around in the code, it felt times if I just looked at the code wrong, it would slow down.
That code is so complicated, so complex that there was really no way to get the proper handling the proper parsing of, all of URLs in all edge cases, without just completely having to almost rewrite the thing in order to maintain performance.
So rewriting is exactly what I did.
I took it as an opportunity to say, Hey, you know what, there's a standard URL class out there.
There's a, this URL object that exists in browsers.
There's a whole spec that says how this thing is supposed to work.
That defines the exact parsing steps.
Exactly what you're supposed to do.
And it's been battle tested.
It's been developed for it was developed for years.
And browsers had, worked on these, this parsing algorithm for a long time.
And it was like it's about time.
Let's just get this in Node.
And at that time it was a very controversial thing.
Introducing this second URL parser implementation.
But we got it in there and found that yeah, there were still developers that wanted to keep using URL Parse and still do today despite its bugs and, issues.
But having that URL class available in Node, the same API that was available in browsers.
Helped a lot of developers not have to write special case code.
They wanna write JavaScript once that it runs in multiple environments.
It's nice that they don't have to am I running in Node?
Okay.
I gotta use URL parse I'm running in a browser.
Okay.
Then I gotta use new URL.
Not, they can just use new URL.
It's available everywhere.
That same URL class is available in Workers it's available in Deno it's available in Bun.
Having this, API there just made things easier for developers and it took several years, but we, we started making progress on other APIs as well.
I think it was 2019.
I introduced the Web crypto implementation in, Node Web crypto API.
It's not the best.
It's, nowhere near as as feature packed as Node's crypto API-doesn't need to be it's just it's a standard API.
That's there.
If you wanna do some basic crypto operations-signing and verifying encrypting and decrypting using certain algorithms,.
You can do that in a consistent way, whether it's in Node or Deno or, Workers or anywhere.
It's the same API.
Last year right about this time last a year ago I introduced the Web streams.
It's a readable stream writeable stream a transform stream even though Node has had a streams API for going on almost 10 years now that was very specific to Node itself.
There is now this implementation of the same streams API that you find in the browsers and that you find in Workers, in Deno.
So now it is possible to create stream code that calculates the streaming digest of data, I going back to our original example in a way that works, whether you're running in Node, whether you're running in Deno, whether you're running Workers regardless you can write the code once and it just works in all of those environments.
So working towards going back to this original idea, the standard API for Node and that being that, the APIs that developers are already familiar with.
The challenge, however, has been a lot of these APIs have been written specifically with browsers in mind.
The fetch API is a good example of this.
Fetch is a standard, right?
It works and we've received lots of requests in Node to, provide an implementation of fetch and there is one now the challenge with it, however, is that there is a lot in fetch that is only relevant to browsers.
CORS support, for instance there's a lot of requirements specifically relating to the CORS standard, but only is relevant to Web browsers.
There's nothing about the CORS requirements that make any sense whatsoever when fetch is running on a server or on the edge, or really in any environment, that's not a browser.
Web streams.
There's a lot about Web streams that are are suboptimal for running in server environments, where you have where your JavaScript's not just running in a single tab in your browser on one person's computer where but instead running in a server, a shared server environment that's serving hundreds, if not thousands if not millions of requests.
There's a number of performance issues and things that, that just are are unfortunately part of the specification itself.
And the reason those things exist the reason for those limitations is that the, WHATWG and the, working groups at the W3C that have developed these standards, over the years have been specifically chartered only to look at the requirements of Web browsers and it [still falls through the work???].
It's just, it's the result of the charter.
The working group is just focusing on the, problem that's been laid out before them, which is specifically the needs of Web browsers.
When Node goes off and implements one of these APIs and says this it's unfortunate that it works this way because we needed this requirement for, servers the, feedback from WHATWG and others have these other working groups has been well, great.
Go do that if you want.
We're only concerned about browsers, so if you're not a browser it's not something we're going to, look at.
And, that's changed recently.
There's, been a lot more consideration and cooperation and, stuff with these groups.
Looking at these requirements for Node and, Deno and others, but that's just been historically where things have gone.
So there really hasn't been a good venue for folks you know from Node and Deno and Workers and stuff to get together to say, "okay, great, we all wanna implement these APIs, but how do we make sure that we're doing so in a way that's interoperable and compatible with each other." So a few months ago, I got together with a few others in the JavaScript community, including fellow Node core contributors and folks from Deno, the company, Vercel, Shopify, and others to launch a new W3C community group called the Web interoperable run times community group, or WinterCG for short.
The goal of WinterCG is simple: to provide a space for JavaScript run times to collaborate on API interoperability with a specific focus towards building and maintaining compatibility with Web platform APIs.
Among the initial work items of this community group is the establishment of a new minimal common Web platform API.
What this is, essentially a minimum list of APIs that all the different run times can implement.
It's things like abortController, abortSignal, readableStream, compressionStream, right event, target all these APIs that exist in browsers today.
What this minimal list that the WinterCG is putting together is basically just saying that, Hey, if you are a JavaScript runtime, and you wanna be compatible with browsers, implement these set of APIs, implement abortSignal, and implement it in a way that is spec compliant com compatible with the browsers.
Provide an implementation of eventTarget, even if Node already has eventEmitter that works a particular way.
Node will also now has eventTarget, right?
That works the exact same way as the browsers Node has the implementation of crypto subtleCrypto, even though it has a crypto API, it also has a subtleCrypto.
The development of this list of this minimal list is the, way we put this list together was that we took we took a look at Node, we took a look at Deno and we took a look at Workers the three in a most popular non Web browser, JavaScript run times that are out there today.
And we looked at what web APIs, what Web standard APIs that also exist in browser browsers, did these platforms already implement?
And we just compiled that into a list as long as it was implemented by at least two of the run times that ended up in this list.
And what we find is a, list that covers a, broad majority of the overlapping requirements between these platforms they all need to do asynchronous activity cancellation or scheduling.
They all need to do streams.
They all need to do crypto.
They all need to to, to parse URLs, that kind of thing.
So we started to see this this, common collection of APIs that, that, are being put together.
Now some of the other work that the community group is, working on is out in fetch for instance, like I was talking about, there's a lot of requirements there in fetch that are only relevant to Web browsers.
There's a lot of API choices there that really only makes sense in Web browsers.
So when, when we've gone about implementing fetch in Node and Deno and Workers and others there's compromises that have been made about we're gonna get as close as possible to what's in this spec.
But we're only gonna implement what actually makes sense on, in server environments.
Unfortunately Opera implementing these things in a vacuum independently of each other.
There's a number of incompatibilities that have arisen between the, implementations across these different platforms.
So, WinterCG is providing a place for, these implementers to get together and start talking through these issues.
What are we gonna do with cookie headers?
What do we do with, CORS?
How do we deal with those things?
How do we deal with global states that may be required by fetch and those kind of questions.
And how do we make sure that when you implement, when you're calling fetch in Deno, that it works the same way as fetch in Node, that it works the same way as fetch in in, in, Workers and in really driving the consistency and interoperability between those platforms.
And the goal of this new community group is not to go out and invent a bunch of new standards.
But we can, if we if, we want to and if we need to if the, implementations get together and start working on some set of APIs or some set common set of requirements.
And we take that to the WHATWG and, they say, well, browsers don't care about this at all.
Or we take it to the W3C, and they're like browsers don't really care about this.
We don't care.
We still have this venue where within WinterCG where we these other implementations can talk about it and works kinda work up a spec for the to make sure that things are still consistent.
And still done in an interoperable way.
The work of this community group is just starting.
It we're, still really just at the beginning of this work and there is so much more to do.
If this is an area that interests you come and get involved all of the community group's work is being done out in the open out of GitHub.
The Github address on the screen below right here we do have biweekly calls that we are that we have right now to work through the, work streams, the different items and you things like fetch and and those kind of things that we're working through.
But yeah, the can I just going back to, like I said the, standard API that we're developing, it's just the stuff that you already use in, browsers today and the interoperability and the compatibility of these things in these platforms is just gonna continue to get better.
And hopefully we'll get to a point where regardless of where your JavaScript runs you will have the common APIs common ways of of accomplishing the, common tasks.
And just make a life easier for everybody.
Well, with that that, that is all, I hope you all enjoy the rest of your Global Scope and talk to you all later.
