In this article, we’ll demonstrate how to create a Node.js server to access IPFS content via the clearnet. Before we explain how to do this, let’s define the key terms and create some context for why this matters.
The clearnet is essentially the publicly accessible internet. It refers to all web content consumed over public networks using standard protocols supported by all browsers, like HTTP, WebSocket (used for real-time communications), and WebRTC (used for most audio and video streaming platforms).
These protocols often require central servers and service providers to work, which can potentially become a vector for gatekeeping, particularly in countries with restrictions on free speech and privacy.
There are a number of other non-standard network protocols, although they may not be supported by all browsers. Examples include:
Similar to BitTorrent, IPFS is a protocol for storing and sharing data in a distributed file system. The key difference, however, is that IPFS aims to create a single global network. In IPFS, an individual file that is hosted is assigned a content ID (CID), which is used to access it.
To illustrate the difference, in HTTP, we use a URI like https://www.google.com
, which is then sent to a DNS server. The server identifies the resource’s IP address and serves the resource. If that host is turned off or otherwise inaccessible, however, that resource is just… gone.
In IPFS, we use a CID like ipfs://xjd9809bnuiue900sdfnuiwerwwer
, which is identified by searching through the distributed hash table (DHT) hosted by all the participants in the network. Instead of identifying one single provider of the resource, all hosts who have a copy of the resource are identified, and we download portions of the resource from all of them.
As with any internet protocol, there are pros and cons to IPFS’ design.
Let’s install the IPFS CLI and add some content to IPFS. You can reference the IPFS documentation as we progress.
First, install IPFS. For Linux and Mac, you can install it with Homebrew using brew install ipfs
. When it completes, make sure to read the output for important information; for Linux, it should give you the exact path for running the daemon.
Once installed, run ipfs init
and, again, make sure to read the output for some important details — you may want to save it somewhere safe. The output should share a cat command to see the documentation; run that command to confirm everything has been set up.
Start the daemon with ipfs init
. To add files to IPFS, just use the command ipfs add FILENAME
and you’ll receive the CID. Easy as that!
To consume IPFS content, simply run ipfs cat /ipfs/CONTENTID
in the command line to retrieve the content.
To consume IPFS content via browser, you’ll need to use Brave, which allows you to view content from IPFS. In the browser, use this pattern ipfs://CONTENTID
to access content. For example, here’s a link to a “Hello, World!” text file I posted to IPFS: ipfs://QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u
.
IPFS is pretty cool, but most everyday internet users don’t want to download an alternative browser or use a CLI tool to browse resources on the web. So maybe you want to deliver your IPFS content to clearnet users by creating a proxy application to fetch the content in a mainstream browser. Let’s do that with Node.js.
N.B., this demo application must be running on a machine that is also running an IPFS server. This means you’ll want to use a virtual private server (VPS) for deployment, where you can run an IPFS server parallel to your Node.js application.
Start by creating a new folder — you could name it ipfs-proxy
or similar — and change directories into that folder. Create a server.js
file by running touch server.js
, then initiate a new Node.js project with npm init -y
. Lastly, install Fastify (although any web framework will do) and ipfs-http-client by running npm install fastify ipfs-http-client
.
Add the following code to your server.js
file:
// import fastify const fastify = require("fastify"); // create the server object const server = fastify({ logger: true }); // example route server.get("/", async (request, reply) => { return { message: "The Server is Working" }; }); // start server function const start = async () => { await server.listen(3000).catch((err) => { fastify.log.error(err); process.exit(1); }); console.log("Listening on port 3000") }; // turn server on start();
Run the file node server.js
, go to localhost:3000
in your browser, and make sure you see the message. Turn the server off with Control+C
in the terminal.
We need to update our server.js
to the code below, but first let’s walk through what we’re doing.
First, we’ll import the ipfs-http-client library as ipfs
. We then create a client that connects to localhost:5001
, which is the default port our local IPFS server should be running on.
We use the client’s get
method to get the data CID. The function returns an Uint8Array
of bytes wrapped in an AsyncGenerator
(fun!).
We use a for await
loop to loop over the generator, then use a text encoder to decode the array of bytes into a string. After the loop, the resulting text has several null characters in it, so we remove them using string.replace
. Finally, we send back the resulting string as JSON.
// import fastify const fastify = require("fastify"); // import ipfs-http-client const ipfs = require("ipfs-http-client") // connect to local ipfs node const client = ipfs.create() // create the server object const server = fastify({ logger: true }); // example route server.get("/", async (request, reply) => { // fetch the content from ipfs const result = await client.get("QmWATWQ7fVPP2EFGu71UkfnqhYXDYH566qy47CnJDgvs8u") // create a string to append contents to let contents = "" // loop over incoming data for await(const item of result){ // turn string buffer to string and append to contents contents += new TextDecoder().decode(item) } // remove null characters contents = contents.replace(/\0/g, "") // return results as a json return { message: contents }; }); // start server function const start = async () => { await server.listen(3000).catch((err) => { fastify.log.error(err); process.exit(1); }); console.log("Listening on port 3000") }; // turn server on start();
Notice the results aren’t the most friendly in the world. The resulting string includes the CID and a timestamp; in other words, additional info along with the contents of my “Hello, World!” file. If this were an HTML file, I’d probably have to do some additional cleanup to remove all the non-HTML, but at least we’re getting the content.
So, theoretically, we could host this app on a VPS that also has IPFS installed, and users on the clearnet/HTTP can see our content without having to download a special browser or command line tool.
IPFS presents some really interesting ideas on how the internet can be more decentralized and free, but it does come at the cost of some of convenience. With clever use of Node.js, however, we can start to bridge that gap for the masses.
Deploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third-party services are successful, try LogRocket.
LogRocket is like a DVR for web and mobile apps, recording literally everything that happens while a user interacts with your app. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.
LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.
Would you be interested in joining LogRocket's developer community?
Join LogRocket’s Content Advisory Board. You’ll help inform the type of content we create and get access to exclusive meetups, social accreditation, and swag.
Sign up nowSOLID principles help us keep code flexible. In this article, we’ll examine all of those principles and their implementation using JavaScript.
JavaScript’s Date API has many limitations. Explore alternative libraries like Moment.js, date-fns, and the new Temporal API.
Explore use cases for using npm vs. npx such as long-term dependency management or temporary tasks and running packages on the fly.
Validating and auditing AI-generated code reduces code errors and ensures that code is compliant.