Multithreading in Node.js with worker threads

Editor’s note: This post was last updated by Elijah Agbonze on 8 January 2024 to reflect any updates and changes in the most recent Node.js version, add more detail about worker threads, and compare other multithreading methods. It was previously updated on 18 January 2022 to include some new information about the Web Workers API and web workers in general, improve and add definitions of key terms, and reflect stable support for the worker_threads module.

Node.js v10.5.0 introduced the worker_threads module, which has been stable since Node.js v12 LTS. In this post, we will discuss what this worker thread module is and why we need it, touching on concepts like the historical reasons for concurrency in JavaScript, common problems and their solutions, and more.

The history of single-threaded JavaScript

JavaScript was conceived as a single-threaded programming language that ran in a browser. Being single-threaded means that only one set of instructions is executed at any time in the same process — in this case, the browser, or just the current tab in modern browsers.

This made things easier for developers because JavaScript was initially a language that was only useful for adding interaction to webpages, form validations, and so on. None of this required the complexity of multithreading.

Ryan Dahl saw this limitation as an opportunity when he created Node.js. He wanted to implement a server-side platform based on asynchronous I/O to avoid a need for threads and make things a lot easier.

However, concurrency can be a very hard problem to solve. Having many threads accessing the same memory can produce race conditions that are very hard to reproduce and fix.

Is Node.js single-threaded or multithreaded?

Our Node.js applications are only sort of single-threaded, in reality. We can run things in parallel, but we don’t create threads or sync them.

The virtual machine and the operating system run the I/O in parallel for us. When it’s time to send data back to our JavaScript code, it’s the JavaScript that runs in a single thread.

In other words, everything runs in parallel except for our JavaScript code. Synchronous blocks of JavaScript code are always run one at a time:

let flag = false
function doSomething() {
  flag = true
  // More code (that doesn't change `flag`)...

  // We can be sure that `flag` here is true.
  // There's no way another code block could have changed
  // `flag` since this block is synchronous.
}

This is great if all we do is asynchronous I/O. Our code consists of small portions of synchronous blocks that run fast and pass data to files and streams. As a result, our JavaScript code is so fast that it doesn’t block the execution of other pieces of JavaScript.

A lot more time is spent waiting for I/O events to happen than JavaScript code being executed. Let’s see this with a quick example:

db.findOne('SELECT ... LIMIT 1', function(err, result) {
  if (err) return console.error(err)
  console.log(result)
})
console.log('Running query')
setTimeout(function() {
  console.log('Hey there')
}, 1000)

Maybe this database query takes a minute, but the Running query message will be shown immediately after invoking the query. Additionally, we will see the Hey there message a second after invoking the query regardless of whether the query is still running or not.

Our Node.js application just invokes the function and doesn’t block the execution of other pieces of code. It will get notified through the callback when the query is done, and we will receive the result.

The need for threads to perform CPU-intensive tasks

Given all of the above, what happens if we need to do synchronous-intense stuff, such as making complex calculations in memory in a large dataset? Then we might have a synchronous block of code that takes a lot of time and will block the rest of the code.

Imagine that a calculation takes 10 seconds. If we are running a web server, that means that all of the other requests get blocked for at least 10s because of that calculation. That’s a disaster — anything more than 100ms could be too much.

JavaScript and Node.js were not meant to be used for CPU-bound tasks. Since JavaScript is single-threaded, this will freeze the UI in the browser and queue any I/O events in Node.js.

Going back to our previous example, imagine we now have a query that returns a few thousand results and we need to decrypt the values in our JavaScript code:

db.findAll('SELECT ...', function(err, results) {
  if (err) return console.error(err)


  // Heavy computation and many results
  for (const encrypted of results) {
    const plainText = decrypt(encrypted)
    console.log(plainText)
  }
})

We’ll get the results in the callback once they are available. Then, no other JavaScript code is executed until our callback finishes its execution.

Usually, as we said before, the code is minimal and fast enough. However, in this case, we have many results and we need to perform heavy computations on them.

This process might take a few seconds, and any other JavaScript execution will be queued during that time. As a result, we might be blocking all our users during that time if we are running a server in the same application.

Why we will never have multithreading in JavaScript

So, at this point, many people might think our solution should be to add a new module in the Node.js core and allow us to create and sync threads. But that isn’t possible. If we add threads to JavaScript, then we are changing the nature of the language.

We can’t just add threads as a new set of classes or functions available — we’d probably need to change the language to support multithreading. Languages that currently support it have keywords such as synchronized to make threads cooperate.

It’s a shame we don’t have a nice way of solving this use case in a mature server-side platform such as Node.js. In Java, for example, even some numeric types are not atomic — if you don’t synchronize their access, you could end up having two threads change the value of a variable.

The result would be that, after both threads have accessed the variable, it has a few bytes changed by one thread and a few bytes changed by the other thread. Thus, it will not result in any valid value.

The naive solution: Synchronous code-splitting

Node.js won’t evaluate the next code block in the event queue until the previous one has finished executing.

So, one simple thing we can do is split our code into smaller synchronous code blocks and call setImmediate(callback) to tell Node.js we are done. This way, it can continue executing things that are pending in the queue. In other words, it can move on to the next iteration or “tick” of the event loop.

Let’s see how we can refactor some code from our previous example to take advantage of this. Imagine we have a large array that we want to process, and every item in the array requires CPU-intensive processing:

const arr = [/*large array*/]
for (const item of arr) {
  // do heavy stuff for each item on the array
}
// code that runs after the whole array is executed

As we said before, if we do this, it will take too much time to process the whole array and the rest of the JavaScript execution will be blocked. Let’s split this into smaller chunks and use setImmediate(callback):

const crypto = require('crypto')

const arr = new Array(200).fill('something')
function processChunk() {
  if (arr.length === 0) {
    // code that runs after the whole array is executed
  } else {
    console.log('processing chunk');
    // pick 10 items and remove them from the array
    const subarr = arr.splice(0, 10)
    for (const item of subarr) {
      // do heavy stuff for each item on the array
      doHeavyStuff(item)
    }
    // Put the function back in the queue
    setImmediate(processChunk)
  }
}

processChunk()

function doHeavyStuff(item) {
  crypto.createHmac('sha256', 'secret').update(new Array(10000).fill(item).join('.')).digest('hex')
}

// This is just for confirming that we can continue
// doing things
let interval = setInterval(() => {
  console.log('tick!')
  if (arr.length === 0) clearInterval(interval)
}, 0)

Now, we can process 10 items each time the event loop runs and call setImmediate(callback) so that if there’s something else the program needs to do, it will do it in between those chunks of 10 items. I’ve added a setInterval() to demonstrate exactly that.

Although this works, as you can see, the code gets more complicated. In addition, in real-world scenarios, the algorithm is often a lot more complex than this, so it’s hard to know where to put the setImmediate() to find a good balance.

Besides, the code now is asynchronous, and if we depend on third-party libraries, we might not be able to split the execution into smaller chunks.

Running parallel processes in the background, without threads

So, setImmediate() is sufficient for some simple use cases, but it’s far from an ideal solution. Can we do parallel processing without threads? Yes!

What we need is some kind of background processing, a way of running a task with input that could use whatever amount of CPU and time it needs to return a result back to the main application. Something like this:

// Runs `script.js` in a new environment without sharing memory.
const service = createService('script.js')
// We send an input and receive an output
service.compute(data, function(err, result) {
  // result available here
})

The reality is that we can already do background processing in Node.js. We can fork the process and do exactly that using message passing, which you can imagine as simply as passing a message from one process to another. This achieves the following goals:

• The main process can communicate with the child process by sending and receiving events
• No memory is shared
• All the data exchanged is “cloned,” meaning that changing it on one side doesn’t change it on the other side
• If we don’t share memory, we don’t have race conditions, and we don’t need threads!

Well, hold on. This is a solution, but it’s not the ideal solution. Forking a process is expensive and slow — it means running a new virtual machine from scratch and using a lot of memory, since processes don’t share memory.

Can we reuse the same forked process? Sure, but sending different heavy, synchronously-executed workloads inside the forked process creates two problems:

1. You may not be blocking the main app, but the forked process will only be able to process one task at a time
2. If one task crashes the process, it will leave all tasks sent to the same process unfinished

If you have two tasks — one that will take 10s and one that will take 1s, in that order — it’s not ideal to have to wait 10s to execute the second task. It’s even less ideal if that task never reaches execution because another process got in its way.

Since we are forking processes, we want to take advantage of our OS’s scheduling and all the cores of our machine. Just as you can listen to music and browse the internet at the same time, you can fork two processes and execute all the tasks in parallel.

To fix these problems, we need multiple forks, not only one. But we need to limit the number of forked processes because each one will have all the virtual machine code duplicated in memory, meaning a few MBs per process and a non-trivial boot time.

Using worker-farm to pool threads

Like with database connections, we need a pool of processes that are ready to be used, run a task at a time in each one, and reuse the process once the task has finished. This looks complex to implement, and it would be if we were building it from scratch!

Let’s use the Worker Farm library to help us out instead. This tool helps efficiently distribute processing tasks across multiple child processes, like so:

// main app
const workerFarm = require('worker-farm')
const service = workerFarm(require.resolve('./script'))


service('hello', function (err, output) {
  console.log(output)
})

// script.js
// This will run in forked processes
module.exports = (input, callback) => {
  callback(null, input + ' ' + 'world')
}

So, is the problem solved? Well, technically yes — but while we have solved the problem, we’re still using a lot more memory than a multithreaded solution.

Threads are still very lightweight in terms of resources compared to forked processes. This is why worker threads were born.

What are worker threads?

Worker threads operate in isolated contexts, exchanging information with the main process using message passing. This approach helps us avoid the race conditions problem regular threads have. At the same time, worker threads live in the same parent process as the main thread, so they use a lot less memory.

You can share memory with worker threads by passing ArrayBuffer or SharedArrayBuffer objects, which are specifically meant for this process. These objects allow you to avoid the need for data serialization, but you should only use them if you need to do CPU-intensive tasks with large amounts of data.

Using worker threads for multiple tasks

You can start using the worker_threads module today if you run Node.js v10.5.0 or newer. However, between v10.5.0–11.7.0, you need to enable the module by using the --experimental-worker flag when invoking Node.js.

Keep in mind that even though creating a worker is a lot cheaper than forking a process, it can also use too many resources depending on your needs. In that case, the docs recommend you create a pool of workers.

You can probably look for a https://itnext.io/a-workerpool-from-scratch-in-typescript-and-node-c4352106ffdegeneric pool implementation or a specific one such as workerpool in npm instead of creating your own pool implementation. Node.js provides the AsyncResource class to provide proper async tracking of a worker pool.

Let’s see a simple example. First, we’ll implement the main file, create a worker thread, and give it some data. The API is event-driven, but I’m going to wrap it into a promise that resolves in the first message received from the worker:

// index.js
// run with node --experimental-worker index.js on Node.js 10.x
const { Worker } = require('worker_threads')

function runService(workerData) {
  return new Promise((resolve, reject) => {
    const worker = new Worker('./service.js', { workerData });
    worker.on('message', resolve);
    worker.on('error', reject);
    worker.on('exit', (code) => {
      if (code !== 0)
        reject(new Error(`Worker stopped with exit code ${code}`));
    })
  })
}

async function run() {
  const result = await runService('world')
  console.log(result);
}

run().catch(err => console.error(err))

As you can see, this is as easy as passing the filename as an argument and the data we want the worker to process. Remember that this data is cloned and is not in any shared memory.

When we’re finished, we wait for the worker thread to send us a message by listening to the message event. Implement the service:

const { workerData, parentPort } = require('worker_threads')

// You can do any heavy stuff here, in a synchronous way
// without blocking the "main thread"
parentPort.postMessage({ hello: workerData })

Here, we need two things: the workerData that the main app sent to us and a way to return information to the main app. This is done with the parentPort, which has a postMessage method where we will pass the result of our processing.

That’s it! This is the simplest example, but we can yet build more complex things. For example, we could send multiple messages from the worker thread indicating the execution status if we need to provide feedback.

We can also send partial results. Imagine that you are processing thousands of images. Maybe you want to send a message per image processed, but you don’t want to wait until all of them are processed.

To run the example, remember to use the --experimental-worker flag if you are using any version prior to Node.js 11.7:

node --experimental-worker index.js

For additional information, check out the worker_threads documentation.

From the example above, we had to separate the worker thread from the main thread, making it possible to use both in the same file while still avoiding shared memory. You want to make sure you do so in different blocks — you can use the isMainThread variable to do that:

const { Worker, isMainThread, workerData, parentPort } = require('worker_threads')

if (isMainThread) {
  function runService(workerData) {
    return new Promise((resolve, reject) => {
      const worker = new Worker(__filename, { workerData });
      worker.on('message', resolve);
      worker.on('error', reject);
      worker.on('exit', (code) => {
        if (code !== 0)
          reject(new Error(`Worker stopped with exit code ${code}`));
      })
    })
  }

  async function run() {
    const result = await runService('world')
    console.log(result);
  }

  run().catch(err => console.error(err))
} else {
  // You can do any heavy stuff here, in a synchronous way
  // without blocking the "main thread"
  parentPort.postMessage({ hello: workerData })
}

Best practices while using worker threads for CPU-intensive tasks

Worker threads are designed for parallelizing CPU-bound tasks. As mentioned earlier, worker threads work in isolated contexts and therefore have no effect on the main thread.

Let’s put together some of the examples we’ve seen so far to perform a CPU-intensive task and see how the worker thread would not block the event loop:

const {
  Worker,
  isMainThread,
  workerData,
  parentPort,
} = require("worker_threads");
const crypto = require("crypto");

if (isMainThread) {
  function runService() {
    return new Promise((resolve, reject) => {
      const worker = new Worker(__filename, { workerData: "world" });
      worker.on("message", resolve);
      worker.on("error", reject);
      worker.on("exit", (code) => {
        if (code !== 0)
          reject(new Error(`Worker stopped with exit code ${code}`));
      });
    });
  }

  async function run() {
    const result = await runService("world");
    console.log(result);
  }

  run().catch((err) => console.error(err));
  console.log("I should run immediately");
} else {
  const codes = [];
  for (let i = 0; i < 5000; i++) {
    const val = doHeavyStuff(`${workerData}-${i + 1}`);
    codes.push(val);
  }

  function doHeavyStuff(item) {
    return crypto
      .createHmac("sha256", "secret")
      .update(new Array(10000).fill(item).join("."))
      .digest("hex");
  }
  parentPort.postMessage(codes);
}

In the example above, we used the doHeavyStuff function we used earlier to perform a heavy task. It takes some milliseconds to run depending on your system, but no matter how long it takes, the console message I should run immediately would always run immediately.

A common example of CPU-intensive task is image resizing. Let’s take a look at that with the sharp library, which provides a high-speed module for image processing in Node:

const {
  Worker,
  isMainThread,
  workerData,
  parentPort,
} = require("worker_threads");
const sharp = require("sharp");

if (isMainThread) {
  function runService() {
    return new Promise((resolve, reject) => {
      const worker = new Worker(__filename, {
        workerData: {
          imagePath: "input.jpeg",
          outputPath: "output.jpeg",
          width: 24000,
          height: 16000,
        },
      });
      worker.on("message", resolve);
      worker.on("error", reject);
      worker.on("exit", (code) => {
        if (code !== 0)
          reject(new Error(`Worker stopped with exit code ${code}`));
      });
    });
  }

  async function run() {
    const result = await runService("world");
    console.log(result);
  }

  run().catch((err) => console.error(err));
  console.log("Resizing image");
} else {
  const { imagePath, outputPath, width, height } = workerData;

  sharp(imagePath)
    .resize(width, height)
    .toFile(outputPath, (err) => {
      if (err) {
        throw err;
      }
      // Send a message back to the main thread
      parentPort.postMessage("Image resized successfully!");
    });
}

The Resizing image message may not be the best way to relay such information, but its purpose is to always run while the image processes — that is, while the worker thread is running.

Think of any CPU-intensive task, like video compression, complex calculations, encryption and decryption, etc. You can perform all of these on the worker thread while simultaneously running your main app on the main thread.

While performing these operations, you want to ensure you apply as much care as possible to avoid some of the side effects of using worker threads in Node.js. For example, one common side effect is having race conditions. Another is the overhead of creating new workers.

To avoid these side effects, let’s discuss some of the best practices regarding how to manage using workers in your app:

• Creating a pool of workers: This mechanism can help you efficiently manage and reuse a pool of worker threads to perform concurrent and parallel tasks. Rather than creating and terminating worker threads for each task, you can reuse threads with the help of a pool, which can in turn improve performance. You can use a library like workerpool to create and manage a pool of workers for your app
• Handling shutdown: Ensure proper shutdown and cleanup of worker threads when they are no longer in use. In the case of threads in a worker pool, you don’t have to worry about shutting them down, as each thread would be released back to the pool to make it available for a new task. Other than that, you can use the exit event to shut down a worker thread that is no longer in use
• Passing data: Ensure that your worker and main thread do not share the same state, as this may cause race conditions. Passing data is recommended over sharing state because each worker thread has its own context and runs independently of the main thread
• Performance evaluation: Evaluate the performance implications of using worker threads for your specific use case. For example, in cases where you intend to perform I/O-bound tasks, using workers may not be the best fit for you

After evaluating the potential performance implications of worker threads in your Node.js project, you may decide against using workers for your specific needs. In such cases, you can use one of the other options we’ll explore in the next section.

Multithreading alternatives to worker threads

There are multithreading methods other than worker threads that may be better for specific use cases. These include thread pools for I/O-bound tasks, child processes, and clustering. Let’s see how each one works and what their pros and cons are:

Method	Description	Pros	Cons
Thread pools	A mechanism in Node.js designed to efficiently manage I/O-bound operations. Despite the single-threaded nature of the primary event loop in Node.js, you can delegate tasks like file system operations, network requests, and other asynchronous I/O operations to a separate thread pool. You can use a library such as libuv to achieve this.	Streamline the management of asynchronous I/O operations in user code Particularly effective for applications with a high volume of concurrent I/O operations, such as managing file system operations, network requests, and database queries Enables the efficient handling of I/O operations without causing blockages in the main event loop	Not suitable for tasks that are CPU-bound
Child processes	Facilitated by the child_process module. They allow us to initiate separate instances of the Node.js runtime or other executables. We can facilitate communication between these parent and child processes through inter-process communication (IPC).	Facilitates the parallel execution of tasks in distinct processes, making optimal use of multi-core systems Supports communication between the parent and child processes through IPC mechanisms such as standard input/output, events, and messages Suitable for both CPU-bound and I/O-bound tasks	Introduces additional processes, leading to increased resource overhead Communication between processes involves serialization and deserialization, potentially impacting performance
Clustering	This module streamlines the generation of child processes, each operating on a distinct core, and offers a shared server port to facilitate load balancing. It enables us to create a cluster of processes, optimizing the way we utilize multi-core systems.	Efficiently utilizes all available CPU cores Enhances load balancing by distributing incoming connections among multiple worker processes Simplifies the handling of incoming requests by providing a shared server port	Debugging and profiling can be more complex compared to a single-threaded application due to the involvement of multiple processes Primarily suitable for CPU-intensive tasks; less effective for I/O-bound tasks

What is the Web Workers API?

You may have heard of the Web Workers API. The API is different from worker_threads because the needs and technical conditions are different, but they can solve similar problems in the browser runtime.

The Web Workers API is more mature and is well-supported by modern browsers. It can be useful if you are doing crypto mining, compressing/decompressing, image manipulation, computer vision tasks such as face recognition, and other such processes in your web application.

Web workers vs. worker threads

Worker threads and web workers serve similar purposes but have some differences due to the distinct environments in which they operate:

• Worker threads in Node.js are part of the Node.js runtime and are specifically designed for server-side JavaScript
• Web workers are a feature of web browsers and are designed for client-side JavaScript running in the browser

It’s also worth noting that both worker threads and web workers communicate with the main thread in almost the same way. They both use the postMessage method, although the implementation in web workers is slightly different.

In summary, worker threads are well-suited for parallelizing CPU-bound tasks, data processing, and other computationally intensive operations on the server side. Meanwhile, web workers are commonly used in web applications to offload intensive computations from the main thread, keeping the user interface responsive.

Introducing: Partytown

Web workers can also be used to run third-party scripts.

Running heavy scripts from the main thread can cause UX issues on your site, which isn’t ideal. However, running a script apart from the main thread can create issues, as we directly don’t have access to the main thread APIs like window, document, or localStorage.

Here’s where Partytown comes in. Partytown is a lazy-loaded, 6kB package that helps you solve the issue mentioned. Your third-party packages will run as expected without affecting the main thread.

If you’re interested in trying this out, see our other post exploring the Partytown library. You can also check out their documentation for a more in-depth discussion.

Conclusion

worker_threads is an exciting and useful module if you need to do CPU-intensive tasks in your Node.js application. They provide the same behavior as threads without sharing memory and, thus, avoid the potential race conditions threads introduce.

If you plan to use worker threads in your Node project, you should keep in mind all the other best practices we mentioned above to get the best out of performing CPU-intensive tasks in your app. In the case of I/O-bound tasks, you should consider making use of Node.js built-in asynchronous I/O operations.