Job scheduling in Node.js using Agenda.js

Often when developing an application, there is be need to perform a recurring task or to remind a user of a particular event. This can range from sending the user billing information for a service once a month to performing database backups. It can even be as simple as sending the user an email to remind them of new offers and promotions.

A simple approach would be to use the built-in methods in JavaScript, which are setTimeout and setInterval. However, this doesn’t scale horizontally because there is no way to keep track of jobs that have been completed or aborted, hence the need for a job scheduler.

In this tutorial, we’ll show you how to do job scheduling in Node.js using Agenda.js.

• Why Agenda.js?
• Setting up Agenda.js
• Initialize a singleton instance of Agenda.js
• Definitions
• Scheduler
• Controllers

Why Agenda.js?

There are many options to consider when picking a scheduler in Node.js, ranging from the established node-cron to more recent and modern solutions such as Bull, Bee-Queue, and Agenda.

What sets Agenda.js apart? Agenda uses MongoDB for persistence, which offers the advantage of less configuration when compared to the Redis option used by most schedulers.

The Redis option doesn’t behave as expected in the case of a server crash or restart and requires some special configuration on the database.

Agenda.js is both lightweight and robust in terms of its feature set. Below is a diagram from the official documentation showing that Agenda.js comes with some mainstream modern schedulers.

For an exhaustive list of how Agenda.js compares with other notable schedulers, check out this guide.

Setting up Agenda.js

The example below shows how to set up Agenda.js:

---

Over 200k developers use LogRocket to create better digital experiences

Learn more →

---

npm install agenda

// then we can go ahead to require and use it 
const Agenda = require("agenda");
const agenda = new Agenda({ db: { address: "mongodb://127.0.0.1/agenda" } });

agenda.define("send monthly billing report", async (job) => {
  // some code that aggregates the database and send monthly billing report.
});

(async function () {
  // IIFE to give access to async/await
  await agenda.start();

  await agenda.every("1 month", "send monthly billing report");
})();

When using Agenda.js, you’re likely to use the following methods regularly:

• . agenda.every repeats a task at a specified interval — e.g., monthly, weekly, daily, etc.
• . agenda.schedule schedules a task to run once at a specific time
• . agenda.now schedules a task to run immediately

For a full list of all possible ways to engage this library, take a look at this official documentation.

For this tutorial, I’ll show you how I set up Agenda.js in an Express application.

In the following sections, I’ll demonstrate how to structure an existing Express codebase to use Agenda.js.

Initialize a singleton instance of Agenda.js

First, initialize Agenda.js and create a singleton that will be used across the application.

jobs/index.js:

const Agenda = require(‘agenda’);
const env = process.env.NODE_ENV || "development";
const config = require(__dirname + "/../config/config.js")[env];
const { allDefinitions } = require("./definitions");

// establised a connection to our mongoDB database.
const agenda = new Agenda({
db: { 
    address: config.database, 
    collection: ‘agendaJobs’, 
    options: { useUnifiedTopology: true }, 
    },
    processEvery: "1 minute",
    maxConcurrency: 20,
});

// listen for the ready or error event.
agenda
 .on(‘ready’, () => console.log("Agenda started!"))
 .on(‘error’, () => console.log("Agenda connection error!"));

// define all agenda jobs
allDefinitions(agenda);

// logs all registered jobs 
console.log({ jobs: agenda._definitions });

module.exports = agenda;

Definitions

All definitions are supplied with the initialized instance of Agenda.js via closures.

jobs/definitions/index.js:

const { mailDefinitions } =  require("./mails");
const { payoutDefinitions } = require("./payout");

const definitions = [mailDefinitions, payoutDefinitions];

 const allDefinitions = (agenda) => {
  definitions.forEach((definition) => definition(agenda));
};

module.exports = { allDefinitions }

Then, each individual definition is defined as such.

Mail definition

Below, we are grouping all of our mail definitions and their associated handlers.

jobs/definitions/mails.js:

const { JobHandlers } =  require("../handlers");

 const mailDefinitions = (agenda) => {
   agenda.define("send-welcome-mail",JobHandlers.sendWelcomeEmail);

   agenda.define("reset-otp-mail",JobHandlers.sendOTPEmail);

   agenda.define(
      "billings-info",
      {
        priority: "high",
        concurrency: 20,
      },
      JobHandlers.monthlyBillingInformation
    );
};

module.exports = { mailDefinitions }

Here, JobHandlers is a function that performs the specified tasks.

The jobs/definitions/payout.js file has similar contents.

JobHandlers

The JobHandlers file contains the actual function definitions that perform the required tasks.

jobs/handlers.js:

//NB this imports are relative to where you have this funtions defined in your own projects
const PaymentService = require("../relative-to-your-project");
const mailService = require("../relative-to-your-project");

 const JobHandlers = {
  completePayout: async (job, done) => {
    const { data } = job.attrs;
    await PaymentService.completePayout(data);
    done();
  },
 sendWelcomeEmail: async (job, done) => {
      const { data } = job.attrs;
      await mailService.welcome(data);
      done();
    },

// .... more methods that perform diffrent tasks
};

module.exports = { JobHandlers }

Now that we have most of the boilerplate out of the way, we can use the singleton instance of Agenda.js in our controllers.

We can also wrap in another module — I prefer this latter approach because it helps keep things more organized and separate.

Scheduler

jobs/scheduler.js:

import { agenda } from "./index";

const schedule = {
  completePayout: async (data) => {
    await agenda.schedule("in 1 minute", "complete-payout", data);
  },
  sendWelcomeMail: async (data) => {
    await agenda.now("send-welcome-mail", data);
  },

// .... more methods that shedule tasks at the different intervals.
}

module.exports = { schedule }

Then, finally, we can make use of this in our controllers or routes, depending on how we structure our application.

Controllers

user/controller.js:

//import the schdule file 
const { schedule } = require("../schedule"); 

app.post("/signup", (req, res, next) => {
  const user = new User(req.body);
  user.save((err) => {
    if (err) {
      return next(err);
    }
     //use the schedule module
     await schedule.sendWelcomeMail(user);

     return res.status(201).send({
      success: true,
      message: "user successfully created",
      token,
    });
  });
});

// use other schedule methods for other routes 

While the above example shows a neutral way to structure Agenda.js in our application, it does conflict with the single responsibility principle (SRP).

Because we often need to do more than just send a welcome email when a user signs up, we might want to run analytics or some third-party integration. A simple controller that signs up a user might end up requiring 1,000 lines of code.

To prevent this, we’ll use an event emitter:

npm i eventemitter3

Then, in our Express application, we’ll make a singleton available across our entire application.

var EventEmitter = require('eventemitter3');

const myEmitter = new EventEmitter();

// where app is an instance of our express application.
//this should be done before all routes confuguration.

app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.set("myEmitter", myEmitter); // this makes my emitter available across all our controllers.

.....

module.exports = { myEmitter }

Having set this in place, we can edit our signup routes from above to dispatch a registration event that we can react to.

//import the schedule file 
const { schedule } = require("../schedule"); 

app.post("/signup", (req, res, next) => {
 const myEmitter = req.app.get("myEmitter");

  const user = new User(req.body);
  user.save((err) => {
    if (err) {
      return next(err);
    }

    //an emit event to handle registration.
     myEmitter.emit('register', user );

     return res.status(201).send({
      success: true,
      message: "user successfully created",
      token,
    });
  });
});

Then, we can react to our register event by importing the same instance of myEmitter set in across our Express application.

const myEmitter = require("../relative-from-our-express-application");
const { schedule } = require("../schedule"); 

// then we can do perfom different actions like this
myEmitter.on('register', data => {
 await schedule.sendWelcomeMail(data);
 // run third party analytics 
 // send a sequence of mails
// ..do more stuff
})

Conclusion

In this tutorial, we demonstrated how Agenda.js can help us handle cron jobs in a structured manner. However, there are some use cases that are outside the scope of this article. You can refer to the documentation for Agenda.js and the eventemitter library to see an extensive guide.

200’s only Monitor failed and slow network requests in production

Deploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third party services are successful, try LogRocket. https://logrocket.com/signup/

LogRocket is like a DVR for web and mobile apps, recording literally everything that happens while a user interacts with your app. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.

LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.