node-gyp, short for generate your projects, is a tool that lets us build native add-ons for Node.js. Due to historical reasons, which we’ll outline in this article, using node-gyp can easily result in problems that are hard to resolve.
In this article, we’ve grouped all the major issues with node-gyp into three categories, including dependencies of node-gyp, add-on compilation errors, and binding contract violation. We’ll review each category one-by-one, offering solutions along the way.
If you aren’t already familiar with node-gyp or haven’t run into these issues yourself, I recommend reviewing the project’s README file first. Let’s get started!
Before we start, it’s important to note that if you have Node.js installed, you probably already have node-gyp installed but not exposed globally. If you only need to compile add-ons during the project setup, Node.js should cover it for you. However, if you are an add-on developer, you probably need to install node-gyp globally.
To use node-gyp, first, we’ll need to install a Python runtime, the make utility, and a C or C++ compiler. Already, we run into an issue with the first requirement. node-gyp expects Python ≥v3.6, not Python v2.x. If you’re not a Python developer, you might not realize that Python v.3 isn’t backward-compatible with its previous major version.
Additionally, many modern operating systems provide default runtimes from the first boot-up, however, these runtimes may not comply with node-gyp’s requirements. Be sure to properly install the correct dependencies for your system before starting with node-gyp.
Lastly, many software developers tend to miss the CPU architecture, frequently assuming that the 64-bit version of the x86 instruction set will work. But as new CPUs constantly appear on the market, this could not be further from the truth.
Usually, developers write add-ons for Node.js in C with N-API bindings or C++ with NAN bindings. We’ll cover bindings in depth later on, but for now, keep in mind that there are at least two ways to bridge the add-on code to the Node.js runtime.
Usually, when you install an external dependency that uses node-gyp, node-gyp will download pre-built binaries that match your architecture and the Node.js version on your system. node-gyp will compile the add-on code only when it doesn’t find a match, which can cause problems.
Most programming languages are backward-compatible, or at least have a minor release version that is. Imagine that the author of an add-on wrote the code but used a higher version of a language than the one supported on your system. You simply would not be able to compile the add-on.
To solve compilation errors, you can either fork its codebase or upgrade or downgrade your compiler toolchain.
Previously, we mentioned two types of bindings, NAN, Native Abstraction for Node.js, and Node-API, formerly known as N-API. The third type of bindings include V8, libuv, and Node.js header files.
V8 is a JavaScript and WebAssembly engine used by Node.js. Libuv is a C library used by Node.js to get support for asynchronous I/O.
The first binding method, NAN, provides header files that enable add-ons to interact directly with the V8 runtime. It is similar to using plain V8 header files, just with extra precaution. Since every Node.js release might use a different V8 version, add-ons that use NAN indirectly depend on the V8 version utilized by their users.
If an add-on code expects more or different functionality from V8 than what it provides, the add-on can fail during the linking phase. Or worse, linking may succeed, only for the add-on to crash during runtime due to the presence of incorrect header files.
To remedy these pitfalls, Node.js now exposes Node-API, a stable API independent from the JavaScript runtime. However, this method is not perfect either. The target Node version needs to implement Node-API in the version used by the add-on. To know which version of Node-API you can use against a particular Node.js installation, you should check the version matrix.
Sadly, there is no simple solution for binding contract violations. If you’re the code’s author, you could adjust it, otherwise, you may need to upgrade or downgrade your Node.js version. Later in the article, we’ll cover a few more generic ways to deal with similar issues.
You may have already realized that the problems reported as node-gyp issues are, in fact, issues with the toolchain used for building add-on code. Now, let’s cover how to categorize the issue to apply the correct solution. When you run into a problem, you can ask yourself the following question:
Simply put, the add-on publisher has more tools at their disposal when fixing potential problems than a consumer. The architecture and the toolchain influence the whole build process. The target Node.js versions might impair the way NAN cooperates, and some low versions do not support Node-API. The programming language of the add-on influences the contact point between Node and the NAN/Node-API bindings.
It’s worth noting that one installation of an add-on that uses node-gyp for building could trigger many issues, each in different categories.
Checking the architecture and toolchain requires knowledge about the software and hardware of your machine. You should acquire information about your operating system and the CPU/GPU. You likely work on an x86-64-compatible instruction set, but it’s better to verify this information.
To check the version of Python 3 currently used on your terminal, run the following command:
python3 --version
In your terminal, run the command below to check what version of the C/C++ compiler toolchain you’re using:
gcc --version
To check the Node.js version on your machine, use the following code:
node --version
You can dynamically change the executed Node.js version with a Node.js version manager, like either n, available on Linux and macOS, or nvm. However, to avoid using each tool in an automated way, which could cause some of the issues we’ve previously listed, we’ll check the add-on bindings.
We need to have access to the codebase of the said add-on. The most important file we need to look into is the binding.gyp file, usually located in the root directory of the add-on. We should search for all require statements in it; require("nan") indicates that the add-on uses NAN, and require("node-addon-api") indicates that the add-on uses Node-API.
For further clarification, you can see examples on GitHub of binding.gyp files in NAN and Node-API.
We can also search the entire project for either NAN or Node-API macros, #include <nan.h> and #include <node_api.h>. Some projects may also use V8 header files, which use #include <v8.h> macros.
You should see the programming language by the filename extensions. C uses .h for header files and .c for source files. On the other hand, C++ uses .h, .hpp, and .hxx for header files and .cpp and .cxx for source files.
Add-ons that use the node-addon-api module are C++ projects. Otherwise, it would be pointless to get declarations of C++ wrapper classes with no intention of using them.
At this point, if you’ve implemented the above solutions and still haven’t solved the problem you’re facing, we’ve prepared a few generic ideas to try out.
Setting up the Node.js environment and node-gyp dependencies might not be straightforward on your operating system. As an alternative, let’s try building the add-on in a Docker image instead of your local machine, allowing you to control all the dependencies, including the operating system. If you can build the Docker image but still have trouble with your local setup, you’ll know where the problem lies.
As an example, we’ve prepared the following Dockerfile snippet, which builds the SQLite3 add-on on an Alpine Linux instance:
FROM alpine:3.14.2 RUN apk update RUN apk add --no-cache \ build-base \ curl \ git \ linux-headers \ nodejs \ npm \ py3-pip \ python3 \ python3-dev \ sqlite RUN pip3 install --upgrade pip WORKDIR /opt/showcase RUN npm init -y RUN apk add sqlite RUN npm install sqlite3 --verbose --build-from-source
You can save it as Dockerfile in the directory of your choice and run with docker build .
We used Alpine Linux because of its lightweight nature. You may have noticed that we had to add our dependencies by hand. As an exercise, you can remove some dependencies and see how each influences the installation of SQLite3. Additionally, you might check the proposed Dockerfile for node-rdkafka, although it is somewhat dated.
You might also try changing the architecture within the Docker image, for instance, you can choose a 32-bit version of Ubuntu. We trust you can use this hint for old projects, but please note that it is rather unlikely for you to run a 64-bit virtualized operating system on a 32-bit host.
Upgrading or downgrading a Node.js version is trivial with Docker. To speed up the process, the Node.js Docker Team provides Docker images for different versions.
If you have access to the codebase of a NAN-based add-on, you might try to switch to Node-API. It requires you to have prior experience with writing add-ons. You should as well have enough resources to undertake such a task.
Hopefully, by the end of this tutorial, you’ve been able to resolve your issue with node-gyp. We learned how to categorize issues into three common categories by asking a series of questions, then offered unique solutions tailored to each category. We then provided a more generic, universal solution to node-gyp issues that uses a Dockerfile.
In rare instances where the steps proposed above don’t work for you, you can browse the following resources and support channels. I hope you enjoyed this tutorial and have a stronger understanding of node-gyp.
Resources:
Support:
Monitor failed and slow network requests in productionDeploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third-party services are successful, try LogRocket.
LogRocket is like a DVR for web and mobile apps, recording literally everything that happens while a user interacts with your app. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.
LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.
Would you be interested in joining LogRocket's developer community?
Join LogRocket’s Content Advisory Board. You’ll help inform the type of content we create and get access to exclusive meetups, social accreditation, and swag.
Sign up now
Compare Prisma and Drizzle ORMs to learn their differences, strengths, and weaknesses for data access and migrations.
It’s easy for devs to default to JavaScript to fix every problem. Let’s use the RoLP to find simpler alternatives with HTML and CSS.
Learn how to manage memory leaks in Rust, avoid unsafe behavior, and use tools like weak references to ensure efficient programs.
Bypass anti-bot measures in Node.js with curl-impersonate. Learn how it mimics browsers to overcome bot detection for web scraping.
Learn more →
