ES modules in Node today

Introduction

“Before software can be reusable, it first has to be usable.” – Ralph Johnson

Modules are independent building blocks of a software program. They are basically a design pattern that implements features of modular design in programming languages. The module system is supported in many languages and is quite popular since the way dependencies are handled, packaged, and managed determines how easy it is to work with a large and growing source code.

In modular design, business logic pertaining to a particular feature or functionality is packaged (modularized) in a standardized format for reusability, flexibility, and for the sake of reducing complexity. This setup affords a loosely coupled system due to a smooth interface of communication, as there are no global variables or shared state.

Although the concept of modules is quite different depending on the language, they are akin to the idea of namespaces in languages like Java. Modules enable code organization by splitting a codebase into reusable components such that each performs individual functions and can be combined or composed to form larger functionalities or an entire application.

In Node.js, the module system has come a long way from its earlier adoption of CommonJS. Today, ECMAScript modules (ES modules), though still experimental at the time of writing, are the official standard for packaging code for reuse in both client- and server-side JavaScript.

Table of contents

In this article, we are going to learn about ES modules in Node. However, we will briefly explore other ways of handling and organizing server-side code with CommonJS.

Why? So that we have a point of reference to recognize the benefits of ES modules. In essence, we will learn about the challenges it tries to solve that earlier module systems were not adapted to solve.

We will be looking at:

• An introduction to ES modules — here, we introduce ES modules in an exciting way
• A brief history of ES modules — here, we learn about the transition from the earlier module system to ES modules. We will also briefly examine how interoperable these modules systems are with one another
• Adding support for ES modules in Node — here, we learn about how we can incrementally add support for ES modules in Node. We also learn how to migrate an old codebase to start using ES modules
• Comparing and contrasting features — here, we will learn about the features of both these module systems and how they compare
• ES modules moving forward

Prerequisites

To easily follow along with this tutorial, it is advisable to have the latest version of Node.js installed. Instructions on how to do so are available in the Node documentation.

Also, for better context, readers may need to be fairly knowledgeable with the CommonJS module system in Node. It is equally welcoming for newcomers learning the Node.js module system or applying ES modules in their Node projects today.

Introducing ES modules

With the release of Node version 13.9.0, ES modules can now be used without an experimental flag since they are enabled by default. With ES modules, modules are defined with the use of the import and export keywords instead of the require() function in CommonJS. Here is how they are used:

We made a custom demo for .
No really. Click here to check it out.

Click here to see the full demo with network requests

export function sayLanguage(language) {
    console.log(`I love ${language}!`);
  }

//f.js


import {sayLanguage} from './f.js';

console.log(sayLanguage('JavaScript'));

//g.js


retina@alex es-modules in Node % node -v
v13.7.0
retina@alex es-modules in Node % node g.js 
(node:77133) ExperimentalWarning: The ESM module loader is experimental.
I love JavaScript!
undefined
retina@alex es-modules in Node %

Details about these keyword bindings can be found in the spec here. Also, we can peek at the Mozilla development network doc for more information.

Note: The code snippet above shows the ExperimentalWarning for the module loader due to the Node version on my local machine. However, it is likely this would change in later Node versions.

Adding support for ES modules in Node today

This support was previously behind the --experimental-module flag. Though this is no longer required, the implementation remains experimental and subject to change.

Files ending with .mjs or .js extensions (with the nearest package.json file with a field type) are treated as ES modules, as shown on line 9 below:

{
  "name": "es_modules_in_node",
  "version": "1.0.0",
  "description": "A sample for enabling ES Modules in Node.js",
  "main": "g.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "type": "module",
  "keywords": [
    "ES",
    "MODULES",
    "NODE",
    "MODULES",
    "JS"
  ],
  "author": "Alexander Nnakwue",
  "license": "MIT"
}

So, in essence, when we run node g.js in the same folder as the above package.json, the file is treated as an ESM. Additionally, it is an ESM if we are passing string arguments to the Node.js standard input with flag --input-type=module.

Note: Without the type field in the parent package.json file, Node throws the error shown below:

(node:2844) Warning: To load an ES module, set "type": "module" in the package.json or use the .mjs extension.

Package scope

A package scope, defined by the type flag in a parent package.json file and all folders below it, is present in the current scope of that package, as explained earlier. Furthermore, files ending with the .mjs extension are always loaded as ES modules regardless of the scope of that package.

In the same light, all other forms of files without extensions and without the type flag in the parent package.json file are treated as CommonJS. Additionally, files ending with .cjs extensions are treated as CJS modules regardless of package scope.

Import and export syntax

In ES modules, specifiers are like string-based file paths used after the from keyword. There are both algorithms to load an ES module specifier and to determine the module format of a resolved URL. An example is shown below:

import {cat} from 'animals';

The animal specifier in this case is an npm package, for example. Other ways specifiers can be referenced include from both absolute and relative file paths or URLs, and paths within other packages. Examples are shown in this section of the documentation.

Although import statements are permitted only in ES modules, they can reference both an ESM or CommonJS modules. For example:

import packageMain from 'commonjs-package'; // Works

import { method } from 'commonjs-package'; // Errors

Note: Import statements that reference other types of files like JSON or Wasm are still experimental and require the --experimental-json-modules and --experimental-wasm-modules flags, respectively, to be passed. Details can be found here.

For exports in ES modules, we can make use of the following:

• Named exports

  module.exports.name = "Alex"

• Default exports

  export default function sayName() {console.log('My name is Mat')}

Note: All built-in Node packages support all types of exports (named, default) described above.

Package entry points

There are now two fields that can define entry points for a package: main and exports. More details can be found in the documentation.

Note: Despite the current support for ES modules in Node, there are still some known concerns with interoperability with CommonJS. We will explore this further in the section on interoperability.

CommonJS module system

Prior to the introduction of ES modules, the community relied heavily on CommonJS for packaging server-side JavaScript code. In the CommonJS module system, each file is treated as a module, which exposes a set of APIs (via a well-defined interface) with the use of the exports object. To understand this better, here is an example using the object created by the module system:

function sayName(name) {
    console.log(`My name is ${name}.`)
  };

function sayAge(age){
  console.log(`I'm ${age} years old.`)
  };


module.exports = {sayName, sayAge};
//a.js

To use these functions (imported as modules in a different file), we can use the require function. This accepts a module identifier (ID) specified by either a relative or an absolute path or by name, based on the module type of the exposed APIs, like so:

const {sayName, sayAge} = require('./a') 
// assuming a.js is in the same folder path

console.log(sayName('Alex')) // My name is Alex.

console.log(sayAge(25)) // I'm 25 years old.

//b.js
//TO RUN THE CODE SAMPLE TYPE: $ node b.js on your terminal

As we can see above, the require object returns the module content exported from the a.js file. To learn more about the implementation of the module, export, and require keywords, we can peek at the module wrapper here.

The CommonJS specification is also available here. The spec highlights the minimum features that a module system must have in order to support and be interoperable with other module systems.

The CommonJS implementation allows for a defined structure in how files are loaded. In this approach, code required from other files are loaded or parsed synchronously. For this reason, catching and detecting failure points or debugging code are easier and less tedious.

Why? Because variables present in the modules or exported files are within the scope of that module or private to it and not in the global scope, as such errors are properly propagated. Also, because of the huge separation of concern, modules are loaded from parent to child, traversing down the dependency graph.

Note: When the wrapper function returns, the exports object is cached, then returned as the return value for the require() method. This is like a cycle. How? On the next call to the module.exports method, the same process of wrapping the module with the wrapper function is repeated. But not without checking if that module is already stored in the cache.

The signature of the wrapper function is shown below:

(function(exports, require, module, __filename, __dirname) {
// Module code actually lives in here
});

The Module object, which takes in an ID and a parent module as parameters, contains the export object:

function Module(id = '', parent) {
  this.id = id;
  this.path = path.dirname(id);
  this.exports = {};
  this.parent = parent;
  updateChildren(parent, this, false);
  this.filename = null;
  this.loaded = false;
  this.children = [];
};

The updateChildren method scans through the file path until the root of the file system is reached. Its job is to update the children property of the Module object with the new parent, as the case may be. Here is the signature below:

function updateChildren(parent, child, scan) {
  const children = parent && parent.children;
  if (children && !(scan && children.includes(child)))
   children.push(child);
}

Let’s see an example to understand this better. In the b.js file above, add this line of code to print the module and the argument object:

console.log(module, arguments);

After running node b.js, we get the following output:

retina@alex es-modules in Node % node b.js
My name is Alex.
undefined
I'm 25 years old.
undefined
<ref *1> Module {
  id: '.',
  path: '/Users/retina/Desktop/es-modules in Node',
  exports: {},
  parent: null,
  filename: '/Users/retina/Desktop/es-modules in Node/b.js',
  loaded: false,
  children: [
    Module {
      id: '/Users/retina/Desktop/es-modules in Node/a.js',
      path: '/Users/retina/Desktop/es-modules in Node',
      exports: [Object],
      parent: [Circular *1],
      filename: '/Users/retina/Desktop/es-modules in Node/a.js',
      loaded: true,
      children: [],
      paths: [Array]
    }
  ],
  paths: [
    '/Users/retina/Desktop/es-modules in Node/node_modules',
    '/Users/retina/Desktop/node_modules',
    '/Users/retina/node_modules',
    '/Users/node_modules',
    '/node_modules'
  ]
} [Arguments] {
  '0': {},
  '1': [Function: require] {
    resolve: [Function: resolve] { paths: [Function: paths] },
    main: Module {
      id: '.',
      path: '/Users/retina/Desktop/es-modules in Node',
      exports: {},
      parent: null,
      filename: '/Users/retina/Desktop/es-modules in Node/b.js',
      loaded: false,
      children: [Array],
      paths: [Array]
    },
    extensions: [Object: null prototype] {
      '.js': [Function (anonymous)],
      '.json': [Function (anonymous)],
      '.node': [Function (anonymous)]
    },
    cache: [Object: null prototype] {
      '/Users/retina/Desktop/es-modules in Node/b.js': [Module],
      '/Users/retina/Desktop/es-modules in Node/a.js': [Module]
    }
  },
  '2': Module {
    id: '.',
    path: '/Users/retina/Desktop/es-modules in Node',
    exports: {},
    parent: null,
    filename: '/Users/retina/Desktop/es-modules in Node/b.js',
    loaded: false,
    children: [ [Module] ],
    paths: [
      '/Users/retina/Desktop/es-modules in Node/node_modules',
      '/Users/retina/Desktop/node_modules',
      '/Users/retina/node_modules',
      '/Users/node_modules',
      '/node_modules'
    ]
  },
  '3': '/Users/retina/Desktop/es-modules in Node/b.js',
  '4': '/Users/retina/Desktop/es-modules in Node'
}

As shown above, we can see the module object on line 6 with all the properties, including the filename, id, children, path depth, etc. Also, we can see the argument object, which consists of the export object, require function, file and folder path, and the Module (which is essentially what the wrapper function does, but it executes the code contained in a file/module).

Finally, as an exercise, we can go ahead and print the require function in the b.js file. To learn more about the output of the require function, we can check the implementation in this section of the Node source code.

Note: Special emphasis should be placed on the load and validateString methods. The validateString method, as its name implies, checks whether the passed-in module ID is a valid string. To understand the require function on a much higher level, you can check the Knowledge section of the Node documentation.

Interoperability for both module systems

In CommonJS, modules are wrapped as functions before they are evaluated at runtime. For ES modules, code reuse provided via import and export binding are already created or loaded asynchronously before they are evaluated. To understand how ESM works under the hood, you can check here. Now let’s explore further 🙂

For a quick comparison, a CommonJS module goes through this phase in its lifecycle:

Resolution –> Loading –> Wrapping –> Evaluation –> Caching

This validates the fact that for CommonJS, there is no way to determine what gets exported as a module until the module is wrapped and evaluated. This is quite different for ES modules, as the imported symbols are already parsed and understood by the language before the code gets evaluated.

When the code is parsed, just before it is evaluated, an internal Module Record is created, and only after this data structure is well-formed are the files parsed and the code evaluated.

For example:

//d.mjs
const check = () => {
  console.log('Just checking`);
};
export.check = check;


//e.mjs assuming they are on the same folder path
import {check} from './d'

In the e.mjs file above, Node.js parses and validates the imports before going further to execute or evaluate the piece of code. This is not the case for a CommonJS module: the exported symbols are only made known after the module is wrapped and evaluated.

This incompatibility is one of the many reasons the standard body in charge of ECMAScript intended to implement interoperability for both ESM and Node’s existing CommonJS module system.

Furthermore, the current specifier resolution does not support all default behavior of the CommonJS loader. One of the major differences is automatic resolution of file extensions and the ability to import directories that have an index file.

For example, if we do an import './directory' from, say, a directory that has an index.js, ES modules does not look for an index.js file in the specified folder, as was the case in CommonJS. Instead, it throws an error. This can be fixed by passing the experimental flag --experimental-specifier-resolution=[mode].

Note: To customize the default module resolution, loader hooks can optionally be provided via the --experimental-loader ./loader-name.mjs argument to Node.js. The loaders APIs are currently being redesigned, which means they would change in the future.

More details about interoperability with CommonJS can be found in this section of the documentation.

Features of both module systems

• Dynamic import() is supported in both CommonJS and ES modules. It can be used to include ES module files from CommonJS code.
• ECMAScript 6 also provides that modules can be loaded from a URL, while CommonJS is limited to relative and absolute file paths. This new improvement not only makes loading more complicated, but also slow.
• Sources that are in formats Node.js doesn’t understand can be converted into JavaScript. More details can be found here.
• Support for extensionless main entry points in ESM has been dropped.
• In the current release of the standard, loading internal modules from disk has been enabled, as specified in the changelog.
• proposal-import-meta provides the absolute URL of the current ES module file. It is currently a stage 3 proposal in the TC39 spec.
• The dynamic imports proposal, currently in stage 4 of the TC39 draft, can be used to import both ES and CommonJS modules. Note that this statement returns a promise.
• A file extension must be provided when using the import keyword. Directory indexes (e.g., './database/index.js') must be fully specified.
• Dual CommonJS and ESM are now possible with the use of conditional exports. Now, Node.js can run ES module entry points, and a package can contain both CommonJS and ESM entry points.

Note: Function require shouldn’t be used in ES modules.

ES modules moving forward

ES modules is still tagged experimental since the feature is not fully ready for production environments. This is because there is a need to remove current changes that lack support for backwards compatibility.

The challenge, therefore, is upon package authors, maintainers, and developers to be explicit with package naming and other useful conventions. More details about this can be found here.

“The main goal is to preserve backward compatibility moving forward, now that Node.js supports both CommonJS and ES modules.”

Nowadays, it is possible to use both CommonJS and ESM in one application, but there are still some issues associated with it. CommonJS modules need to know if the module being loaded is a CommonJS or an ES module since the latter is loaded only asynchronously.

Also, in accordance with the ESM spec, using the import keyword does not complete the file path by default with the file name extension, as for CommonJS modules. Therefore, this must be explicitly stated.

Conclusion and next steps

Prior to the introduction of the ES6 standard, there wasn’t any native implementation for organizing source code in server-side JavaScript. The community relied heavily on CommonJS module format.

Nowadays, with the introduction of ES modules, developers can enjoy the many benefits associated with the release specification. This article has highlighted the transition between both module systems and their interoperability.

Finally, due to the existing incompatibility issues between both module formats, the current transition from CommonJS to ESM would be quite a challenge due to the issues we have highlighted in this article. Tools like Babel and esm, which translate the newer syntax into code compatible with older environments, can make the transition easier.

In the long run, this entire draft process is an important step and paves the way for further future improvements. Let me know if you have any questions in the comment section below, or message me on my Twitter handle. Thanks for reading 🙂

200’s only Monitor failed and slow network requests in production

Deploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third party services are successful, try LogRocket. https://logrocket.com/signup/

LogRocket is like a DVR for web apps, recording literally everything that happens on your site. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.

LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.