How to build a URL shortener with Node.js

Editor’s note: This article was updated on 29 September 2022 to add information on why one should use a URL shortener, how a URL shortener works, using a program vs. making your own URL shortener, and more. Changes were also made to the code, including converting the app to an ES6 module, migrating from shortId to nanoid, and using the $inc operator for incrementing the number of clicks. If you wish to check the older version of the code, you can check this commit of the application.

URL shorteners like Bitly and Cuttly are incredibly popular. But what if we could build one ourselves?

In this article, we are going to create a similar tool to that of Bitly and Cuttly by building an API service that shortens the URLs provided to it. You can see an example of what our program will look like in the GIF below:

For the project, we are going to use MongoDB and Node.js, so you should have basic knowledge of them for this tutorial.

• Why use a URL shortener?
• Using Cuttly and Bitly vs. building your own URL shortener
• How does a URL shortener differ from a short link generator?
• How does a URL shortener work?
• Planning the URL shortener build process in Node.js
• Initializing the app and installing dependencies with MongoDB
• Connecting the database to the app
• Creating the Mongoose schema in MongoDB
• Building the URL and index routes

Why use a URL shortener?

A URL shortener decreases the length of a URL for you. Large URLs can be complicated to remember or share with others. A shortened URL version can help you share your favorite link more easily.

A URL shortener can help you in many other ways as well. For example, links shortened with the same tool look nearly identical, at least for the domain section. This overall increases the authenticity of the URL.

Also, adding a large and complicated URL in your social media bio or where the link is clearly visible looks cluttered. Instead, a shortened URL looks professional and minimal.

A shortened URL further lets you easily track the number of clicks to the page. This is not possible using third-party websites without a URL shortener. These metrics can help you measure the performance of the link.

Using Cuttly and Bitly vs. building your own URL shortener

Cuttly and Bitly are two very popular URL shortener services. But there are a few drawbacks to them that strengthen the need to build your own URL shortener. Let’s discuss them briefly.

First off, we have pricing. Services like Cuttly or Bitly easily charge up to $100 per year for their basic paid plan. They limit the number of shortened URLs you can generate. Using your own URL shortener service will cost you a lot less if you set it up properly in a cloud hosting service, like AWS. And if you are eligible for the AWS free tier, you can easily run your basic URL shortening service for free for a complete year (excluding the domain name charges).

Next, most URL shortening services don’t allow you to use custom domains for your shortened URLs. The option for using a custom domain charges a lot extra. But without using a custom domain, your brand will not stand out.

Finally, most of the free or basic plans of URL shortening services don’t allow you to view analytics. But by doing some simple tweaks in your custom application, you can easily track the analytics, like how many clicks were made to the link.

How does a URL shortener differ from a short link generator?

As already discussed, your custom short link generator gives you a few benefits, such as using your custom domain, pricing, and more. But other than that, your data is also secure in your own space.

If you are thinking of building an internal tool for your organization, having your data locked inside your institution is very important in terms of privacy. Using a third-party service doesn’t guarantee you data privacy.

How does a URL shortener work?

The basic workings of a URL shortener is straightforward. In the short URL generation part, the API takes in a POST request with the original URL as a payload. A unique ID is generated that corresponds to the original URL. This ID is added to the end of the base URL, i.e., the URL of your application.

The generated URL and the original URL are stored in the database.

When a user visits the shortened URL, the shortened URL is searched in the database. The user is redirected to the original URL if the URL is found. Also, the number of clicks on the URL increases by 1 in the database. Otherwise, it returns an error:

The above flowchart explains how the application works in a very straightforward manner. The first flowchart explains the shortened URL generation part, and the second flowchart describes what happens when a user visits the shortened URL.

Planning the URL shortener build process in Node.js

Let’s first plan out the building process. As aforementioned, for each URL passed into our API, we will generate a unique ID and create a short URL with it. Then, the long URL, short URL, a variable click with the value of 0, and unique ID will be stored in the database.

When a user sends a GET request to the short URL, the URL will be searched within the database, and the user will be redirected to the corresponding original URL. Sound complex? Don’t worry, we’ll cover everything you need to know.

Initializing the app and installing dependencies with MongoDB

First, we are going to need a database. Because we’ll use MongoDB, we’ll need a MongoDB SRV URI. You can create a database from this link. Our next step is to initialize the project folder with NPM.

Let’s use the command npm init in the project directory to initialize. After the initialization, we need to add one more line in the generated package.json file. Open the package.json file and add the line "type": "module" at the end of the file. Adding this line in the package.json file will allow us to import dependencies using the import statement. Once the project is initialized, we are going to install the required dependencies.

The dependencies that we need are:

• dotenv: this package loads the environment variables from a file called .env to process.env
• Express.js: this is a minimal and flexible web application framework for Node.js
• Mongoose: this is a MongoDB object modeling tool for Node.js
• NanoID: This package enables us to generate the short IDs for our URLs

The only developer dependency that we need is nodemon. The nodemon package is a simple tool that automatically restarts the Node.js server when a file change occurs.

Now, let’s install the dependencies. To install the dependencies that we are going to need in our app, we will use the following command:

npm i dotenv express mongoose nanoid

After the dependencies are installed, we’ll install the developer dependency:

npm i -D nodemon

Let’s create our server in our app.js file using Express. To set up an Express server, we need to import the Express package into the app.js file. Once the package is imported, initialize and store it into a variable called app.

Now, use the available listen function to create the server. Here’s an example:

import express from 'express';
const app = Express();

// Server Setup
const PORT = 3333;
app.listen(PORT, () => {
  console.log(`Server is running at PORT ${PORT}`);
});

I’ve used port 3333 to run the server. The listen method in Express starts a UNIX socket and listens for a connection in a given port.

Now, create a .env file inside the config folder to store the MongoDB SRV URI and the base URL. The base URL will be your local host server location for now. Here’s my .env file code:

PORT=3333
MONGO_URI=mongodb+srv://nemo:[email protected]/myFirstDatabase?retryWrites=true&w=majority
BASE=http://localhost:3333

Remember to change the <password> field in the MongoDB URI with your database password.

Connecting the database to the app

Now, we’ll connect the database to the app. To do so, import the Mongoose and dotenv dependencies into your db.js file, which is inside the config folder:

import mongoose from 'mongoose';
import dotenv from 'dotenv';
dotenv.config({ path: './.env' });

The path object key is passed inside the dotenv config because the .env file is not located in the root directory. We are passing the location of the .env file through this.

Now create an asynchronous function called connectDB within a file called db.js, inside the config folder. I’ll use async/await for this article:

const connectDB = async () => {
  try {
    await mongoose.connect(process.env.MONGO_URI, {
      useNewUrlParser: true,
      useUnifiedTopology: true,
    });
    console.log('Database Connected');
  } catch (err) {
    console.error(err.message);
    process.exit(1);
  }
};

export default connectDB;

In the try block, we wait for Mongoose to connect with the given MongoDB URI. The first parameter in the mongoose.connect method is the MongoDB SRV URI. Notice that the two key-value pairs are passed in the second parameter to remove the console warnings. Let’s understand what the two key-value parameters mean:

• useNewUrlParser: true: the underlying MongoDB driver has deprecated the current connection string parser. This is why it has added a new flag. If the connection encounters any issue with the new string parser, it can fall back to the old one
• useUnifiedTopology: true: this is set to false by default. Here, it is set to true so that the MongoDB driver’s new connection management engine can be used

If any error occurs within the catch statement, we will console log the error and exit with process.exit(1). Finally, we export the function with module.exports.

Now, import the db.js file into the app.js file with import connectDB from './config/db.js'; and call the connectDB function with connectDB().

Creating the Mongoose schema in MongoDB

We’ll use a Mongoose schema to determine how data is stored in MongoDB. Essentially, the Mongoose schema is a model for the data. Let’s create a file called Url.js inside a models folder. Import Mongoose here, then use the mongoose.Schema constructor to create the schema.

import mongoose from 'mongoose';

const UrlSchema = new mongoose.Schema({
  urlId: {
    type: String,
    required: true,
  },
  origUrl: {
    type: String,
    required: true,
  },
  shortUrl: {
    type: String,
    required: true,
  },
  clicks: {
    type: Number,
    required: true,
    default: 0,
  },
  date: {
    type: String,
    default: Date.now,
  },
});

export default mongoose.model('Url', UrlSchema);

The parent object keys are the keys that are going to be stored inside the database. We define each data key. Note that there is a required field for some and a default value for other keys.

Finally, we export the schema using export default mongoose.model('Url', UrlSchema);. The first parameter inside mongoose.model is the singular form of the data that is to be stored, and the second parameter is the schema itself.

Building the URL and index routes

The URL route will create a short URL from the original URL and store it inside the database. Create a folder called routes in the root directory and a file named urls.js inside of it. We are going to use the Express router here. First, import all of the necessary packages, like so:

import express from 'express';
import { nanoid } from 'nanoid';
import Url from '../models/Url.js';
import { validateUrl } from '../utils/utils.js';
import dotenv from 'dotenv';
dotenv.config({ path: '../config/.env' });

The utils.js file inside the utils folder consists of a function that checks if a passed URL is valid or not. Here’s the code for the utils.js file:

export function validateUrl(value) {
  return /^(?:(?:(?:https?|ftp):)?\\/\\/)(?:\\S+(?::\\S*)?@)?(?:(?!(?:10|127)(?:\\.\\d{1,3}){3})(?!(?:169\\.254|192\\.168)(?:\\.\\d{1,3}){2})(?!172\\.(?:1[6-9]|2\\d|3[0-1])(?:\\.\\d{1,3}){2})(?:[1-9]\\d?|1\\d\\d|2[01]\\d|22[0-3])(?:\\.(?:1?\\d{1,2}|2[0-4]\\d|25[0-5])){2}(?:\\.(?:[1-9]\\d?|1\\d\\d|2[0-4]\\d|25[0-4]))|(?:(?:[a-z\\u00a1-\\uffff0-9]-*)*[a-z\\u00a1-\\uffff0-9]+)(?:\\.(?:[a-z\\u00a1-\\uffff0-9]-*)*[a-z\\u00a1-\\uffff0-9]+)*(?:\\.(?:[a-z\\u00a1-\\uffff]{2,})))(?::\\d{2,5})?(?:[/?#]\\S*)?$/i.test(
    value
  );
}

We will use the HTTP post request in the urls.js file to generate and post the details to the database:

import express from 'express';
import { nanoid } from 'nanoid';
import Url from '../models/Url.js';
import { validateUrl } from '../utils/utils.js';
import dotenv from 'dotenv';
dotenv.config({ path: '../config/.env' });

const router = express.Router();

// Short URL Generator
router.post('/short', async (req, res) => {
  const { origUrl } = req.body;
  const base = process.env.BASE;

  const urlId = nanoid();
  if (utils.validateUrl(origUrl)) {
    try {
      let url = await Url.findOne({ origUrl });
      if (url) {
        res.json(url);
      } else {
        const shortUrl = `${base}/${urlId}`;

        url = new Url({
          origUrl,
          shortUrl,
          urlId,
          date: new Date(),
        });

        await url.save();
        res.json(url);
      }
    } catch (err) {
      console.log(err);
      res.status(500).json('Server Error');
    }
  } else {
    res.status(400).json('Invalid Original Url');
  }
});

module.exports = router;

The const { origUrl } = req.body; will extract the origUrl value from the HTTP request body. Then we store the base URL into a variable. const urlId = nanoid() is generating and storing a short ID to a variable. You can also specify the size of the urlId by passing the size inside the nanoid function. For example, writing nanoid(8) will generate a unique ID of length 8.

Once it is generated, we check if the original URL is valid using our function from the utils directory. For valid URLs, we move into the try block.

Here, we first search if the original URL already exists in our database with the Url.findOne({ origUrl }); Mongoose method. If found, we return the data in JSON format; otherwise, we create a short URL combining the base URL and the short ID.

Then, using our Mongoose model, we pass in the fields to the model constructor and save it to the database with the url.save(); method. Once saved, we return the response in JSON format.

Unexpected errors for the try block are handled in the catch block, and invalid URLs that return false in our validateUrl function send back a message that the URL is invalid. Finally, we export the router.

Previously, we needed to install the body-parser package, but now it is integrated into Express, so head back to the app.js file and add these two lines to use body-parser:

// Body Parser
app.use(Express.urlencoded({ extended: true }));
app.use(Express.json());

These two lines help us read incoming requests. After these two lines of code, import the URL route:

import urlsRouter from './routes/urls.js';
app.use('/api', urlsRouter);

Because we are using the /api endpoint, our complete endpoint becomes http://localhost:3333/api/short. Here’s an example:

Now create another file called index.js inside the routes folder to handle the redirection process. In this file, import the necessary dependencies.

Here, we are first going to search our database for the short URL ID that is passed. If the URL is found, we’ll redirect to the original URL:

import express from 'express';
import Url from '../models/Url.js';
const router = express.Router();

router.get('/:urlId', async (req, res) => {
  try {
    const url = await Url.findOne({ urlId: req.params.urlId });
    if (url) {
      await Url.updateOne(
        {
          urlId: req.params.urlId,
        },
        { $inc: { clicks: 1 } }
      );
      return res.redirect(url.origUrl);
    } else res.status(404).json('Not found');
  } catch (err) {
    console.log(err);
    res.status(500).json('Server Error');
  }
});

export default router;

The HTTP GET request is getting the URL ID with the help of :urlId. Then, inside the try block, we find the URL using the Url.findOne method, similar to what we did in the urls.js route.

If the URL is found, we increase the number of clicks to the URL and save the click amount. As Claude mentioned in the comments, to avoid the race condition, we are using the MongoDB $inc method here for incrementing the clicks.

The updateOne method is called on the Mongoose model. In the updateOne function, the first parameter is passed as the condition. Here, the condition is to match the urlId in the model with the urlId found in the URL parameter. If the condition matches, the clicks value is increased by 1. Finally, we redirect the user to the original URL using return res.redirect(url.origUrl);.

If the URL is not found, we send a JSON message that the URL is not found. Any uncaught exception is handled in the catch block. We console log the error and send a JSON message of “Server Error.” Finally, we export the router.

Import the route to the app.js file, and our URL shortener is ready to use. After importing it, our final app.js file will look like this:

import express from 'express';
import connectDB from './config/db.js';
import dotenv from 'dotenv';
dotenv.config({ path: './config/.env' });
const app = express();

connectDB();

import indexRouter from './routes/index.js';
import urlsRouter from './routes/urls.js';

// Body Parser
app.use(express.urlencoded({ extended: true }));
app.use(express.json());

app.use('/', indexRouter);
app.use('/api', urlsRouter);

// Server Setup
const PORT = process.env.PORT || 3333;
app.listen(PORT, () => {
  console.log(`Server is running at PORT ${PORT}`);
});

Conclusion

In this article, we learned how to build a URL shortening service API from scratch. You can integrate it with any frontend you want, and even build a full-stack URL shortener service. I hope you liked reading this article and learned something new along the way. You can find the complete source code on my GitHub repo.