Build a sitemap generator in Next.js

Next.js is one of the most popular React production frameworks. With features like route prefetching and hybrid static and server-side rendering, Next.js is an efficient tool for optimizing React applications in production.

Additionally, Next.js is known for deploying technologies that address some of the challenges of building SPAs using React. For example, Next.js is shipped with features like static site generation, image optimization, fast refresh, automatic content caching, and filesystem routing, which are not available in React.

Despite being a production-ready framework, handling SEO in Next.js isn’t so straightforward. To implement effective SEO in our webpage, we first need to understand how Next.js handles page rendering. A popular way to handle SEO is with a sitemap, which is a collection of all the visible URLs in your web application. Every website needs a sitemap to help search engines like Google crawl and index your website faster, improve SEO, and give you a better chance at ranking near the top of the search engine results page.

In this tutorial, we’ll set up a dynamic sitemap in Next.js. To follow along with this tutorial, you’ll need:

• Familiarity with CSS, HTML, and JavaScript ES6
• Node.js and Watchman installed on your development machine
• iOS Simulator or Android Emulator for testing
• A code editor installed in your development machine
• A basic understanding of React and React Native

Let’s get started!

Installation and setup

First, we’ll create a new Next.js project, then configure it to automatically generate a sitemap for each page present in our Next.js application upon build.

To initialize a new project, navigate into your work directory. Then, run the respective code for Mac or Windows in either your terminal or the command prompt:

npx create-next-app@latest next-sitemap
# or
yarn create next-app next-sitemap

The command above sets up a new Next.js project in the next-sitemap folder. To confirm that the installation was successful, run the following code in next-sitemap:

npm run dev

The app preview should look similar to the screenshot below:

With our new Next.js app up and running, let’s build our sitemap generator, which we’ll use to generate the sitemap for our webpage.

Building the sitemap generator

To build our sitemap generator, we’ll need to traverse the Next.js file system, specifically the Next.js pages folder. We’ll retrieve all the file paths associated with files in the pages folder, excluding the _api and _document files.

To do so, we’ll install a package called globby, which provides methods for scanning your file system and returns path names that match a specified set of rules:

Run the code below to install the globby package:

npm i globby

Setting up the next-config.js file

Once globby is installed, update the next-config.js file with the code below:

module.exports = {
...
  webpack: (config, { isServer }) => {
    if (isServer) {
      require("./scripts/sitemap-generator");
    }
    return config;
  },
};

The webpack block extends webpack’s configuration, then checks if the Next.js config is executed for client-side or server-side rendering using the isServer, which in our case, will always return true.

Each time the webpack config is executed for the server-side, the next-config file will execute the sitemap-generator script.

Setting up the sitemap generator

Next, create a new folder called scripts at the root level of the next-sitemap folder. Navigate into the newly created folder and create a file called sitemap-generator.

A typical sitemap schema looks like the one below:

 <url>
    <loc>http://website.com/page</loc>
    <lastmod>date created</lastmod>
    <changefreq>monthly</changefreq>
    <priority>1.0</priority>
</url>

To curate a list of all the pages in our project, we’ll need to dynamically fetch the pages and place them in the <loc></loc> block. Add the code below to the sitemap-generator file:

const fs = require('fs')
const globby = require('globby')
function addPage(page) {
  const path = page.replace('pages', '').replace('.js', '').replace('.mdx', '')
  const route = path === '/index' ? '' : path
  return `  <url>
    <loc>${`${process.env.WEBSITE_URL}${route}`}</loc>
    <lastmod>${new Date().toISOString()}</lastmod>
    <changefreq>monthly</changefreq>
    <priority>1.0</priority>
  </url>`
}

In the code above, the addPage function searches the pages folder for all .js and .mdx files, replacing their extensions with an empty string. Now, let’s create a new function to generate the actual sitemap:

async function generateSitemap() {
  // excludes Nextjs files and API routes.
  const pages = await globby([
    'pages/**/*{.js,.mdx}',
    '!pages/_*.js',
    '!pages/api',
  ])
  const sitemap = `<?xml version="1.0" encoding="UTF-8"?>
  <urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
${pages.map(addPage).join('\n')}
</urlset>`
  fs.writeFileSync('public/sitemap.xml', sitemap)
}
generateSitemap()

We created a generateSitemap function, as well as a pages constant to hold all the pages generated by globby. Finally, we rendered all the pages as sitemaps by mapping through each of them with ${pages.map(addPage).join('\n')}.

Next, to test the app, create a .env file and add the code below:

WEBSITE_URL=http://localhost:3000

Testing the sitemap generator

Now that we’re done creating our sitemap generator, let’s see our sitemap in action! Run the code below to start the Next.js development sever for our project:

npm run dev

You may run into the error message below:

Error [ERR_REQUIRE_ESM]: require() of ES Module

The error arises from the globby package we installed earlier. globby ≥v12.x dropped support for ES module import and export. While we could consider changing the import to a dynamic import(), as suggested by the error report, doing so would cause even more complex errors from webpack.

---

Over 200k developers use LogRocket to create better digital experiences

Learn more →

---

Our best bet is to use globby v11.0.1, which has been tested for the tutorial. Update the globby package version in package.json file:

"globby": "^11.0.1"

Next, run the following command:

npm i

Run npm run dev once again, and you shouldn’t have any errors. Open the public folder, and you should see a newly created sitemap.xml file.

In your web browser, visit http://localhost:3000/sitemap.xml, and you should see a result similar to the screenshot below:

Notice that we only have a page URL returned on the sitemap. When we create new pages with our sitemap generator, they’ll be added to the sitemap automatically on build.

Let’s demonstrate this with an example. Create a new file called about.js in the pages folder and add the code below:

import React from 'react'
const About = () => {
    return (
        <>
        <Head>
        <title>About page</title>
        <meta name="description" content="This is the about page" />
        <link rel="icon" href="/favicon.ico" />
      </Head>
      <main>
        <h1>About</h1>
      </main>  
        </>
    )
}
export default About

Now, visit http://localhost:3000/sitemap.xml once more, and your sitemap should look like the screenshot below:

Feel free to add as many pages as you need in your app. Each will automatically be added to the sitemap.

Conclusion

SEO is an important factor for every developer to consider. One way to easily make your webpage more SEO-friendly is with a sitemap. In this tutorial, we built a sitemap generator in Next.js that automatically generates a sitemap for each page in your web application. Adding the sitemap generator can help boost your rankings on search engine results pages (SERP) and draw more users to view your website. I hope you enjoyed this tutorial!

LogRocket: Full visibility into production Next.js apps

Debugging Next applications can be difficult, especially when users experience issues that are difficult to reproduce. If you’re interested in monitoring and tracking state, automatically surfacing JavaScript errors, and tracking slow network requests and component load time, try LogRocket.

LogRocket is like a DVR for web and mobile apps, recording literally everything that happens on your Next.js app. Instead of guessing why problems happen, you can aggregate and report on what state your application was in when an issue occurred. LogRocket also monitors your app's performance, reporting with metrics like client CPU load, client memory usage, and more.

The LogRocket Redux middleware package adds an extra layer of visibility into your user sessions. LogRocket logs all actions and state from your Redux stores.

Modernize how you debug your Next.js apps — start monitoring for free.