The Realms (API) of JavaScript

Leo Balter at Global Scope '21
Sign up for Conffab Pro to watch this and hundreds of other videos for 19.95 a month

Hello everyone.

I am Leo.

I am Lead Standards Engineer on the Salesforce UI platform team.

And I'm glad to be here.

Thank you.

Tday, I'm here to talk about the Realms of JavaScript and let's start.

The first question of this is: What is a Realm?

So what it means to have a Realm - in ECMAScript we have a definition of Realms as being a new set of intrinsic objects with an ECMAScript global environment, and all the ECMAScript code that is loaded within the scope of that global environment and other associated state and resources.

That means, basically a website page runs in the Realm.

So when you have the console log, this code will run inside of Realm record.

But it also means when you have multiple scripts, that web page will run that those scripts under the same Realm records.

So here I'm set in a global foo and I can observe the global foo within another script tag.

And we also have a set of Intrinsics.

For those Intrinsics is also included the well-known Intrinsics from ECMAScript.

You can have 'Array.Object'.

Most of them that, you know, as global objects and global values, global names, but you can also get values such as 'AsyncFunction.prototype'.

Or the async function, there is an Intrinsic that is not available as a global, but it's available through syntax.

If you get - if you create AsyncFunction through syntax, you can still fetch AsyncFunction.prototype from it.

The set of Intrinsics are preserved for Realm.

So that means if you change the Intrinsics, you also have - you see the effects of this.

So here I'm adding the method '.last' and I can reuse last within other script tags too.

And it also means the modules will also share a Realm.

So the modules, they run in the same Realm of that page.

So when you import a module, that module is imported using that Realm it belongs to - that it's been imported from.

So here I have a module 'array-last' and inside the code of this - this module file I do the extension of 'Array.prototype' and the 'last' method, and when I use an array, I can have it available now.

That's because the module is still re-used in the same Realm.

It's reusing the same global, it's reusing the same set of Intrinsics.

Here comes the part where we actually work with multiple Realms.

When we actually work with them.

Multiple Realms, they already exist today in the web platform.

You have the iframes, you have Workers, you have Node's vm.

You have APIs in Android and iOS - JavaScript APIs that provides you a way to create and operate through the Realms as well.

Talking about the iframes here - let's see some examples using an iframe to create - to have its own new Realm.

So when you create an iframe, you append it to the page - you add it to the page, you can see that even if I add a method to the 'Array.Prototype', 'Array.prototype' belongs to the page - the main page Realm, and if I get 'Array.prototype' from the iframe, I can see the method is not added there and that's because the array prototype from the main page is different from the array prototype from the iframe.

They are using a different Realm.

That's why they have a different global, they have a different set of Intrinsics.

And the Realms - they can also be connected.

That means the Blue Realm - and I'm connecting to the 'Array.prototype' from the Red Realm.

When I talk about the Blue Realm, I talk about the Main Page Realm.

And when I talk about the Red Realm, it talks about the Realms that are associated.

In this case here it has the iframe -the Red Realm from the iframe, and I have two values running under the same code.

And these two values - if I compare to them, they are both the 'Array.prototype' - they're just copies of each other.

They're not the same, so they will fail if you try to compare them.

One of the things that you can also have with iframes, you can also connect the code from the iframe to its parent Realm.

So here, in this case, I set a global value in my Blue Realm - or the Main Page Realm - I set the value 'foo' I set - for the name 'foo' I set the value '42' and then I can observe in the iframe, 'foo' is not available - there is no definition of this global value there.

But still the iframe can use the global or 'window.top' to get access to the Main Page - to the parent Realm.

So I can still get access to it.

Now we know that we have multiple Realms, where they can also be applied to?

There is a simple use case here - one use case among many others there - is Web IDEs.

For the Web IDEs today, some of them are familiar with many IDEs.

One of them that is a popular is Visual Studio Code from Microsoft.

It's an IDE that is web-based and it's coming to the web as like a full web application.

And when we have this, we transform not only the web platform, but the web application to become a platform.

That's because you not only have your base application, you have all the components of your application: running a base code, loading sensitive information and everything loading, but you also load - you have a marketplace of extensions.

You have extensions that is not only like your extensions - you might have a curated list of extensions that are community-based, they're submitted by other developers.

Who are going to use those extensions?

The users!

The users are gonna decide what they want as extensions and how many extensions they want to use.

They can set like 5 or 10 extensions.

So they might be like from different third parties, from other developers as well, or from other companies that might develop different kinds of extensions.

It's according to their usage and their needs.

These codes - they are not code that are supposed to just run in an isolated case.

You actually need code that run according to a shared state.

It's the state of your base application, its the state of - the base of that data - and you still need to have some control of what state, what we should provide to reach extension, like what kind of data the extension would need?

Maybe an extension needs just access to a file tree.

Maybe an extension would just need to - a code that has been added at the moment for code highlighting, for linked in tools, et cetera.

And for that you need real time access to it.

You need immediate information because you can imagine the developer writing down their code and trying to set code there and you need to provide it in real time, linked in, you'll need to provide code highlighting - highlighting the current line, or actually highlighting the brackets where they open and close for the current code that you're running, or you're working in a toolbox.

This is all like, based on a real example that is coming in from GitHub CodeSpaces which is an instant dev environment.

And there is extensible with plugins and color schemes, et cetera, et cetera.

Here comes another question.

Can we use the iframes for this?

Quick answer is no.

The iframes, they have a huge payload and they have the unforgeables.

That means when you have an iframe, you load a whole DOM API.

You create a new window proxy.

You create a new DOM tree.

This is already have enough just by itself, but the iframe is also bundled with unforgeable values.

Those are values that you cannot delete, you cannot remove, you cannot modify them.

You have some of these values being a window.top', 'window.location'.

And 'window.top' providing access to the parent Realm that will give you, give the user - or the extension - an opportunity to leak code, leak access - for code to modify what is in the base application.

Remember the platform.

Maybe you can talk - we can talk about workers?

And with workers you also have the problem where communication is not synchronous.

It's not in real time.

Workers, they operate in a separate thread.

You cannot have control or just wait when the thread is going to be ready to provide you information in real time, you don't have that.

And that's why today we are working on a Realms API - we are calling it the Callable Boundary API - that has been proposed at TC39 and this is the current shape of the proposal.

So you have the Realm constructor, you have an 'ImportValue' to inject and import modules, and you have the evaluate method where can evaluate strings.

It's a little bit powerful where you can not only get simple values such as strings, but you can get more values such as strings, anything that is primitive.

But beyond primitive values, you can also get functions.

You don't really have the function from the other Realm, but when you have an evaluation that results into a callable object, or a general function, you create a wrapped function.

There is a new function that, when it's called, it chains the call to the function from the other Realm.

You create a channel from your Realm to the new associated Realm, and you send the arguments there and you get the results back.

So that means you get the results and you operate and you create synchronous communication.

This chain, this channel is synchronous from one Realm to the other.

That also means you can get any callable object.

It doesn't need to be a simple ordinary function.

You can also use arrow functions, bound functions, proxy wrapped functions, and you can wrap functions in both sides.

It's not only during the first evaluation.

Here I have a function from the other Realm - from the Red Realm that I am evaluating and creating 'doSomething'.

When I call 'doSomething', I also send a function here that is going to be wrapped in the other realm.

And it doesn't matter like how many times you send functions back and forth, they're always going to be wrapped.

And so you have a wrap.

When this wrap is called, it's going to call - it's going to send the arguments to the other function on the other side in the other realm, and will try to fetch - it will fetch the return value.

You don't have object access.

If you try to evaluate something that returns - that results in an object, such as 'globalThis' on array 'Object.prototype', it will throw a TypeError, it will fail.

That's because you still would have problems with the identity.

Remember when we took the 'array.prototype' when you get from the parent realm and the iframe realm - you still could have an array, but it would belong to the other realm, you would still have problems with identity and that's been avoided.

But here you have a way to inject code not only during the evaluation through strings, you have a way to inject code by using 'importValue' that is analogous to dynamic import.

So you run something that is like a dynamic import, and you fetch a name - a binding name from this module needs space.

So in my example, here, I'm fetching 'runTests' to start to trigger task execution of 'my-tests' framework and I can run tests in a separate realm without all the pollution from my base 'test-framework' or anything.

And you can even expand that in the future.

For the things that we've been talking about at TC39, you have Module Blocks coming in.

So that means you can, you don't need exactly to load a file, but you can have all your modules set in your code already.

And you can just tell the realm to execute that module code within the realm in a separate set of global and intrinsics.

So you have way more clean space to, to do that execution.

What we want to enable with this Realms API - something that we call a 'proper Realms API' - we have a new global object and a new set of intrinsics of course, but you still have a separate module graph remember when its' modules are executed within its' realm.

So when you inject a module, that module is going to be evaluated within that realm when it's loaded.

You also still have the synchronous communication.

When we have the wrap functions, you can call a function to trigger to an immediate execution in the other realm.

You have a proper mechanism to control the execution of a program that enables virtualization, code sandbox.

That helps with a lot of frameworks that will do virtualization today.

And you have a consistent API across any JavaScript ecosystem, including Browsers and Node.js.

So you can have a single API for both of them.

You don't need to rely on iframes or node.vn.

You don't need to switch from one to another, you still use the same API.

And the proposal today is presented to TC39 - it's currently on Stage 2 and we believe we are gonna be advancing to Stage 3 by the end of July, right about when this conference is also happening.

What do you think about it?

Are you excited?

I am very excited and I hope to see the news soon.

So yeah, let me know what you think and let's chat.

Thanks for having me today.

Thank you so much.

Bye.

The Realms (API) of JavaScript–Leo Balter

The Realms of JavaScript

  • Web Directions
  • Global Scope 2021
  • Lead Standards Engineer @ Salesforce
  • Leo Balter

What is a Realm?

ECMAScript Realms

  • A set of intrinsic objects.
  • An ECMAScript global environment.
  • All the ECMAScript code that is loaded within the scope of that global environment.
  • Other associated state and resources.

A website page runs in a Realm!




   
   Realms


   <script>
       console.log("hi! This code is running in the page's main realm!");
   </script>

The global environment!

<script> globalThis.foo = 42; </script> <script> console.log(globalThis.foo === 42); // true </script>

ECMAScript set of Well-Known Intrinsics

  • Array, Object, Map, WeakMap…
  • Array.prototype, Object.prototype, …
  • %AsyncFunction.prototype%

The set of intrinsics are preserved for Realm

<script>
   console.log(Array.prototype.last); // undefined
   Array.prototype.last = function() {
       return this[this.length - 1];
   };
</script>
<script>
   console.log(['a', 'b', 'c'].last()); // 'c'
</script>

Modules run in the same Realm

<script type="module">
   import './array-last.js';
  
   // ./array-last.js will patch Array.prototype:
   // Array.prototype.last = function() {
   //     return this[this.length - 1];
   // };

   console.log(['a', 'b', 'c'].last()); // 'c'
</script>

Multiple Realms

  • Multiple Realms already exist today
  • Web Platform: iframes, Web Workers, …
  • Node's vm
  • Android and iOS APIs

Each new iframe will have it's own Realm

// creates and attaches a new iframe to the page
const iframe = document.createElement('iframe');
document.body.appendChild(iframe);

// Adds Array#last to the main page's realm
Array.prototype.last = function() {
   return this[this.length - 1];
};

// Fetches Array.prototype from the iframe's Realm
const arrProto = iframe.contentWindow.Array.prototype;

// The iframe doesn't have the last method
console.log(arrProto.last); // undefined

With a new Realm, a new set of intrinsics

// creates and attaches a new iframe to the page
const iframe = document.createElement('iframe');
document.body.appendChild(iframe);

// Fetches Array.prototype from the iframe's Realm
const arrProto = iframe.contentWindow.Array.prototype;

// In fact, it's a different object!
console.log(Array.prototype === arrProto); // false

The Realms can be connected


// creates and attaches a new iframe to the page
const iframe = document.createElement('iframe');
document.body.appendChild(iframe);

// Fetches Array.prototype from the iframe's Realm
const arrProto = iframe.contentWindow.Array.prototype;

// In fact, it's a different object!
console.log(Array.prototype === arrProto); // false
  • Main Page => Blue Realm
  • iframe => Red Realm
  • Array.prototype belongs to the Blue Realm
  • arrProto belongs to the Red Realm

iframe Realms can connect to their parent Realm

const iframe = document.createElement('iframe');
document.body.appendChild(iframe);

globalThis.foo = 42; // Adds a global named `foo` to the main realm

// The iframe realm doesn't have this global value
console.log(iframe.contentWindow.globalThis.foo); // undefined

// The iframe can access its parent Realm!
console.log(iframe.contentWindow.top.foo); // 42

Why multiple realms are used?

Use Case Sample: Web IDEs

screenshot of the developer tools view in Microsoft's Visual Studio Code

The Web App is a Platform

  • The IDE application is a platform running a base code, loading user sensitive information and also running extensions, including 3rd party code plugins.
  • The plugins are picked by each user
  • The plugins will need synchronous access to different parts of the IDE, such as text sandbox, file tree, console execution, toolings, etc.
  • Each plugin might also need to load and manipulate dependencies such as libraries through modules.

Realm time means Real time access!

screenshot of IDE demonstrating type ahead for JavaScript

GitHub CodeSpaces

  • An instant dev environment
  • Extensible with plugins, color schemes, etc

screenshot of developer tools in a browser

Can we use just iframes for this?

Nope!

iframes have a huge payload and unforgeables!

  • Each iframe will load not only ECMAScript intrinsics, but also create a new DOM tree, including a new copy of the Window proxy object, the document API, etc etc.
  • The iframe is also bundled with unforgeable values. They can't be removed and some lead to cross-realms leak! E.g.: window.top, window.location
  • This is just the tip of the iceberg of problems from iframes!

Maybe Workers?

  • Communication with Workers are not synchronous, they operate in a separate thread.
  • Status of the base page could easily change during communication.

The Callable Boundary API

The new Realms API proposed at TC39

declare class Realm {
   constructor();
   importValue(specifier: string, bindingName: string): Promise<PrimitiveValueOrCallable>;
   evaluate(sourceText: string): PrimitiveValueOrCallable;
}

Cross-realm code evaluation!

  • Not limited to strings & numbers
const realm = new Realm();

Symbol.for('x') === realm.evaluate('Symbol.for("x")'); // true

Auto wrapped functions

  • When one Realm sends a callable object, a new Wrapped Function Exotic Object is created in the other realm connected to it.
const realm = new Realm();
const wrapped = realm.evaluate('x => x * 2');
  • When the Wrapped Function Exotic Object is called, it chains the call to its connected function with the same arguments and returns its return.
wrapped(21); // returns 42

Wraps any Callable Objects

Any object with a [[Call]] internal

  • Not limited to ordinary functions
  • Function
  • arrow functions
  • bound functions
  • Proxy wrapped functions

Wrapped in Both Directions

The API allows sending and receiving callable objects

const realm = new Realm();
const doSomething = realm.evaluate('(x, wrappedCallback) => wrappedCallback(x * 2)');

doSomething( 2, (done => console.log(done)) );
// Logs 4

No cross-realm object access

const realm = new Realm();

realm.evaluate('globalThis'); // Throws a TypeError

// or
realm.evaluate('[]'); // Throws a TypeError

// or
realm.evaluate('Object.prototype'); // Throws a TypeError

importValue

// ./inside-code.js
export { runTests } from 'test-framework';
import './my-tests.js';

// from the incubator Realm
const r = new Realm();
const runTests = await r.importValue('./inside-code.js', 'runTests');

Coming next: Module Blocks!

(with the module blocks proposal)

module insideCode {
 export { runTests } from 'test-framework';
 import './my-tests.js';
}

const r = new Realm();
const runTests = await r.importValue(insideCode, 'runTests');

Enablement of a proper Realms API

  • a new global object and a new set of intrinsics
  • a separate module graph
  • synchronous communication between realms
  • proper mechanism to control the execution of a program
  • enables a callable boundary cross-realms without enabling cross-realm object access.
  • consistent API across any JS ecosystem, including Browsers and Node.js!

Current status

  • Proposed to TC39, currently at Stage 2.
  • Possibly Stage 3 by the end of July!

Thanks You!