(energetic upbeat music) - Hi, WebDirections, my name is Mejin Leechor and I'm a developer in Los Angeles, where I currently work for Pivotal Labs.
We just became part of VMWare this year.
Today, I'm going to talk about modules in JavaScript, which is one of my favourite topics to talk about. I have no special qualification for talking about modules, except my own curiosity.
I started learning JavaScript about three years ago and what caught my eye was the import and export statements I was seeing all over code bases, and what those import and export statements told me was that my perception of JavaScript was very outdated. I had never gotten over my first impression of JavaScript, which happened when I was nine, and building my first website on GeoCities. And back then a lot of the web looks like this. JavaScript was not the star of the web at that time, static websites like these were.
And as a nine year old, the only JavaScript that I cared about was inline, copy pasted and responsible for effects like sparkly trails on cursors.
Needless to say, modules were not a part of JavaScript in those years.
The web is a pretty different place in 2020 than it was back in the early years.
JavaScript and web technologies have come a long way, but even with all our fancy libraries and frameworks, support for native modules is actually a fairly recent development.
Before we go further, I want to clarify this term native modules.
When I say native modules, I'm not talking about some of the more popular module formats out there that you might be using.
So I'm not talking about CommonJS modules and node. I'm also not talking about AMD modules which you might know, from having used RequireJS. When I say native modules, I'm talking about modules that are actually part of the language and written into the ECMAScript specification.
These ECMAScript modules or ES modules as they're popularly called are the first and only native module system that JavaScript has ever had. You might recognise ES modules by this import and exports syntax, and you might even be using that syntax in a modern framework like react. Now, these modules are new.
They might even be newer than you think.
The ES6 pack officially introduced modules in 2015, but it didn't offer any guidance on how to load modules in different JavaScript environments.
So we didn't actually have a working module system at that time, it took some additional work to bring ES module support to the browser and to node. So by mid 2018, we had native module support in all major browsers and no chip to support for modules initially behind an experimental modules flag, so you could play with it, but that flag was actually removed in November, 2019. Please note, it's actually still not considered a stable feature.
Now, if you've been working in JavaScript for awhile, chances are good you've worked with one of the module systems I just mentioned. And that's because for most of JavaScript's history, we have had modules they've just been hand-rolled or developed in user land.
And this is the story that I'm excited to tell to you today, how modules became part of a language that didn't use to have them or any apparent need for them. The fact that we have native modules in JavaScript is not an accident, it's actually the hard one result of a decades long, ground up effort on the part of the JavaScript community to evolve the language. So it could do more and be more than anyone had initially imagined.
Now, modules along with other advances helped transform JavaScript into the powerhouse of a programming language that we use to power the modern web.
So, what's all this fuss about modules? What's so important about them that we couldn't live without them for years? Well, let's get into it.
And we'll start by talking about modularity. I like to think of modularity as providing structure through boundaries, if you don't have boundaries, you don't have structure, you have a morphous systems and you have source code files that are in the tens of thousands of lines long.
And maybe you have that many global variables too. Another way that I like to think about modularity is through the analogy of building blocks.
So you compose programmes of smaller units of data and functionality that we can mix, match and recombine. And these smaller units, which we call modules, hang together through a couple well-defined relationships. So these modules can expose data in functionality via their interfaces, and they can also have dependencies on other modules so that they can use their data and functionality.
To facilitate these module relationships, modules need to have their own private scopes. Now name spacing can get you part of the way to modularity, but to truly have independent modules, you need independent scopes.
And a couple of the benefits of modularity. Well, there are so many benefits to modularity, but two in particular stand out to me.
The first is our ability to reason independently about the component parts of a system, which lightens our cognitive load, we don't have to think about the whole system we can think about one piece at a time.
That also encourages loose coupling and helps us write more maintainable code.
The other big benefit of modularity that I see is the ability to reuse code, whether that's code that you wrote or someone else wrote. Think whenever you use a package from NPM, The bottom line is that modularity is for us, it's for human beings, we're the ones who are gonna suffer when our systems grow beyond our ability to manage them. We're also the ones who have to grok and maintain our own code.
And on the other hand, we're the ones who benefit when our systems are clear and organised, and when we have a community to share our solutions with.
Modularity is what facilitates all of this. One final note, I wanna make a distinction between modules as a programming language construct, and modularity, the more abstract concept.
So modules and JavaScript and other languages are a means of getting at modularity, but it's certainly possible to use modules and not have any of the benefits of modularity because you've tightly coupled your modules. And for more on that, I would look into things like the solid principles of object oriented design, single responsibility principle, et cetera. So, how'd we get here? Let's take a quick trip down memory lane.
I personally find that it's helpful to understand where we came from to understand where we are with modules in 2020.
We'll hop through history in roughly five to 10 year increments, based on this little timeline I put together while playing module historian. Quick disclaimer, all the dates are pretty rough, very approximate, but this is just meant to be a teaching tool.
I've divided JavaScript history into four phases, or eras. We'll start in the early years of JavaScript in the time before modules.
And when we look at the history books, it turns out there's a really good reason.
We didn't have modules back then, which goes back to the original plans that Netscape had for the language.
And if you don't believe you me, listen to the creator of JavaScript himself, Brendan Eich, who said, "HTML needed a scripting language, a programming language that was easy to use by amateurs and novices, where the code could be written directly in source form as part of the Web page markup." So Java script was supposed to be baked into web pages. It was for lightweight use and there was not actually an obvious need for a module system at that point. I also love this quote.
"To write large code, you don't just want this little snippet language that I made easy for beginners to start buying by the yard.
You want strong APIs ways of saying, this is my module, and this is your module, and you can throw your code over to me, and I can use it safely." I really appreciate that, Brendan Eich recognised that the absence of modules was an inhibitor in maturing the language, and that we would actually need them if we wanted to write large code. Post 2000 we move into the do it yourself module era. And to give you a little bit of context, this was a time of transformation for the web. Gmail came out in 2004, and it was really widely regarded as the quintessential web 2.0 app.
And it really raised the bar about what was possible. It could do so much, and with so few page reloads, the web was becoming a more interactive place, thanks in large part to technologies like Ajax, web apps, not websites were the new Vogue, it's against this backdrop, that we started to see the shortcomings of living in a world without modules. And one of the biggest problems was global variables, which made scripts more prone to variable naming collisions and unexpected state changes.
In fact, global variables were such a problem that the Yahoo user interface library team, published a blog post in 2006 with the ominous name, "Global Variables Are Evil." And they said, "An objective measure of the quality of a JavaScript programme is how many global variables and global functions does it have.
A large number is bad because the chances of bad interactions with other programmes goes up." Now, I would argue that the chance of bad interactions within a programme goes up too.
And we're starting to see strategies emerge in this period to combat the global variables problem.
The most classic of these patterns is the module pattern, which you might actually be familiar with.
And the essence of the module pattern is to pair a closure within immediately invoked function expression with the effect of shielding private variables, but still allowing indirect access to them through publicly exposed methods.
So we can take a look at an example here.
Now we've got a do it yourself module, and in it, we have defined inside this IIFE function scope, we have defined a private variable, a counter which we initially set to the value of zero. And we've got a couple exposed functions that we're gonna return out of this IIFE, including an increment function that increments counter and a print function that's gonna let you see what the current value of that counter is.
Now, if we want to actually use that module, we can, we can call the print method, and we can see that the counter is initially zero, initially has a value of zero, and then after we increment it, it has a value of one, but if we actually tried to access the counter itself, and this is the most important part, we can't access it. We've achieved data privacy because we've exited out of the IIFE scope, and the variables that were scoped to that function are now gone.
And that's the ingenuity of the module pattern, the use of function scope to create these makeshift module boundaries.
And that's a trick we'll see throughout module history. CommonJS uses a similar mechanism and it's required method by creating a wrapper function that contains the module scope.
Now, there were other problems that started to rear their heads during this period, especially with dependency management.
If you want it to pull on multiple scripts, you had to worry about having those script tags in the right order, and you had to think about the interactions between those scripts. We were also seeing performance problems like blocking the thread of execution while waiting for those external scripts to load, and code wasn't easily reusable or shareable. And we all know that copying and pasting is no fun. And that brings us to the next phase of module history. What I call the era of specification, and what's getting specified are module formats, which make code sharing a real possibility. Now the promise of code sharing and reuse is what spurred a lot of the innovation in this period, Much of the energy came from the server side JavaScript community, which had a lot of excitement about JavaScript's potential as a language, but also some frustration about its limitations. Around 2009, things really started to heat up. Kevin Danger wrote a blog post called "What Server-Side JavaScript Needs" back in January of 2009. And I think it does a great job of capturing the zeitgeist at the time.
It's basically a wishlist of features in JavaScript and we'll look at three that pertain to modules. The first is a module system and Danger noted that namespaces were easy to achieve in the language, but there was no standard way to load a module. The second and third are a package management system and a package repository, and Danger noted that if there was an easy way to instal packages and those packages dependencies, people would be more likely to use more libraries in their apps.
Now, Danger also had the insight that the absence of these things wasn't actually a technical problem or strictly a technical problem. It was a community organising for a problem, and he understood that if people just came together and worked on these problems, it would have a huge impact on the future of JavaScript. He wrote a blog post that ultimately, or this blog post rather ultimately led to the formation of CommonJS, which was a grassroots group of developers who set out to work on the goals that Danger outlined in this post.
And a couple others that I didn't include here. Now, just doesn't note this CommonJS group. isn't the same as the CommonJS module specification in node, but when node came out in May, 2009, it delivered on one of those hopes for the CommonJS effort, which was a module specification.
And we now call that module spec, CommonJS, not confusing at all.
Now this was all good and well for the server side JavaScript community, but other JavaScript developers recognised the CommonJS wasn't ideal for the browser environment.
And that's what spawned asynchronous module definition or AMD unlike CommonJS and it's synchronous thread blocking require method, AMD provided for asynchronous loading of modules and their dependencies.
And this ultimately performed better in the browser loading, smaller JavaScript files, and only as they were needed. There was also an attempt to bridge the gap between CommonJS and AMD called universal module definition.
But the syntax turned out to be pretty tedious and ugly, didn't catch on, turns out when you try to make everyone happy, you can't always win. But the proliferation of specifications in JavaScript of module specifications, this was a legitimate problem. A member of Ember's core team put it this way, "Unless all dependencies use the same module format dependency tree of depths greater than one is so painful, nobody does it.
Nobody agrees on a solution to this problem." And that's how these became the years of specification hell. It was a dumpster fire.
There's one other major development from the specification era that I'd like to highlight, and that's bundling.
And bundling is when we stitch together modules, and their dependencies into one or more files for browser consumption.
A couple of players of node include Browserify, which came out in 2011, bringing CommonJS modules to the browser.
So you could actually use NPM packages in your browser code, pretty cool.
But I would be remiss if I didn't mention Webpack, which brought static assets into the mix.
And there are a couple other libraries out there, actually, a lot of other libraries out there, but I'll just highlight a couple more, Rollup and Parcel, which is a newer one.
Please make a quick mental bookmark here because bundling ain't over yet.
So we finally arrived in the era of standardisation and it couldn't have come soon enough.
We finally converged on a single module system. Let's see what history has brought us.
Now, since the ES6 pack came out in 2015, we've had this lovely, concise module syntax. We have default exports, you get one of those profile. We have named exports, and you can have as many as your heart desires. On the import side, we can import all the exports from a module in one go, and assign them to a variable. And then we can access the exports by name with dot notation.
We can also import one or more named exports, from a module by using what looks like destructuring syntax, it's not destructuring syntax, it's just named imports.
So those curly braces are common to both destructuring and named imports.
And if we're importing a default export, we can just assign it to a variable name of our choosing. Fun fact, the default export of a module is actually just a named export with the name default. And you can import it by name, if you like, just like in that last line where we're importing default and then aliasing it to suite export.
One important note about ES modules.
What you're exporting or importing is a read-only live binding.
So let's say you import a variable called 'fu', and it has the value two, the exporting module could at some point update fu's value to be seven, and your fu value would also you're importing modules, from fu value would also update to seven.
But the importing module, because this is a read-only binding, can't actually change fu's value directly.
Not unless it had a mutater or something that was exposed. This is not at all like CommonJS.
So please pay attention to that if you aren't familiar with ES modules and are coming from CommonJS, this is one reason why it was actually really difficult to implement ES modules in node.
ES modules also are loaded and connected to each other prior to execution.
And again, this is different from CommonJS modules where the required method is synchronous and loading and evaluation basically happen together in sequence. And ES modules are better adapted to the browser for this reason, because you're not actually blocking the thread of execution on import.
However, if you did want a dynamic import, JavaScript has you covered down.
Dynamic imports are supported in all browsers, and that support followed really quickly on the heels of having imports in the static form. And we can invoke the import keyword as a function, give a module specifier, which is basically a path to a module and you get this nice promise-based API to work with, by the way, this is real code from when I was playing around with dynamic imports. And if you're wondering what it does, it does this. You could also use async await given that you're just working with promises here.
And this can be useful if you want to do lazy loading or conditional module loading, or if you need to compute the module specifier on the fly. I mentioned earlier that there's now native support for modules in the browser, and this is how you can actually use that yourself if you'd like.
So you can specify a type attribute on a script tag as a module, and as module, and that tells the browser to parse this file as a module and not as a regular script.
And if you provide a script tag, the nomodule attribute that lets you fall back...
that basically lets you have a fallback script for older browsers that don't yet support ES modules. So that's pretty nifty and all, but are we there yet? Have we arrived? After all these years are we tasting the promise of a native module system with code sharing, and loose coupling and clean perfect modularity? Well, unfortunately not quite, it's still playing out. We do have native modules, but right now they coexist alongside CommonJS primarily and other prior module specs. And we're still working through some of the interoperability issues between module formats. And this in a nutshell is how I describe where we are right now.
You're probably still bundling.
You're using module syntax when you develop, but Webpack or Parcel or some other bundler's still in the picture.
In production you just hand the browser that bundled and transpile JavaScript, even though browsers can handle native module code the newer ones anyway.
So why is that? Why haven't we all jumped ship already if ES modules are the new hotness? Well, it all comes down to one thing, and that's performance.
A year or two ago, most of the conventional wisdom was to stick with bundling for your production apps, even in a world where native modules and native module support in the browser is real. Now it didn't seem like there was a way for native modules to outperform bundling when the browser has to make a round trip for every module that it needs in a script.
And the Chrome team released a study back in 2018 that did a bottleneck analysis, comparing ES module loading performance to bundled scripts. Their recommendation was that you keep on bundling even with HTTP/2 server push isn't quite there yet. But the jury is still out and there might be optimizations that we haven't fully considered or explored.
I'd like to leave you with a takeaway that can help you think about your options in this present moment in module history, where we're sort of, but not all the way there. And this is what I'm calling the ES Module Maturity Model, inspired by the Richardson Maturity Model.
And if you've seen that or if you haven't, it describes how closely a rest service follows restful principles.
And this ES module maturity module is following a similar sort of look and format, where it's giving you a graded scale.
And the idea is that you have different levels and you can opt in to using ES modules, and reaping their benefits to the extent that you would like.
So at level zero, we don't use native modules at all, but at level one, that's where we actually start to get it, get our hands dirty in this new ES module syntax. We use the new module syntax, but we're still bundling. And that's probably where most of us are right now. At level two, now this is where we're getting into our imagination more, and this is much more aspirational, but I got an idea from Snowpack, which is a library that can serve your application unbundled during development, from the demos I've seen it's actually pretty snappy.
So you actually have unbundled development and ES modules there, but you still bundle for production. And then finally, level three, that's where we're fully bought in using native optimised modules and production. Aspirational? Yeah.
Possible? Well, if not today, eventually.
Well, even though we're not there yet, we've certainly come a long way, and I hope we get there someday.
Judging from our past, I think we will.
That's all I've got for you today.
Thank you so much for listening.
(high energy upbeat music)
