Often when developing an application, there is be need to perform a recurring task or to remind a user of a particular event. This can range from sending the user billing information for a service once a month to performing database backups. It can even be as simple as sending the user an email to remind them of new offers and promotions.
A simple approach would be to use the built-in methods in JavaScript, which are setTimeout and setInterval. However, this doesn’t scale horizontally because there is no way to keep track of jobs that have been completed or aborted, hence the need for a job scheduler.
In this tutorial, we’ll show you how to do job scheduling in Node.js using Agenda.js.
There are many options to consider when picking a scheduler in Node.js, ranging from the established node-cron to more recent and modern solutions such as Bull, Bee-Queue, and Agenda.
What sets Agenda.js apart? Agenda uses MongoDB for persistence, which offers the advantage of less configuration when compared to the Redis option used by most schedulers.
The Redis option doesn’t behave as expected in the case of a server crash or restart and requires some special configuration on the database.
Agenda.js is both lightweight and robust in terms of its feature set. Below is a diagram from the official documentation showing that Agenda.js comes with some mainstream modern schedulers.
For an exhaustive list of how Agenda.js compares with other notable schedulers, check out this guide.
The example below shows how to set up Agenda.js:
npm install agenda
// then we can go ahead to require and use it
const Agenda = require("agenda");
const agenda = new Agenda({ db: { address: "mongodb://127.0.0.1/agenda" } });
agenda.define("send monthly billing report", async (job) => {
// some code that aggregates the database and send monthly billing report.
});
(async function () {
// IIFE to give access to async/await
await agenda.start();
await agenda.every("1 month", "send monthly billing report");
})();
When using Agenda.js, you’re likely to use the following methods regularly:
. agenda.every repeats a task at a specified interval — e.g., monthly, weekly, daily, etc.. agenda.schedule schedules a task to run once at a specific time. agenda.now schedules a task to run immediatelyFor a full list of all possible ways to engage this library, take a look at this official documentation.
For this tutorial, I’ll show you how I set up Agenda.js in an Express application.
In the following sections, I’ll demonstrate how to structure an existing Express codebase to use Agenda.js.
First, initialize Agenda.js and create a singleton that will be used across the application.
jobs/index.js:
const Agenda = require(‘agenda’);
const env = process.env.NODE_ENV || "development";
const config = require(__dirname + "/../config/config.js")[env];
const { allDefinitions } = require("./definitions");
// establised a connection to our mongoDB database.
const agenda = new Agenda({
db: {
address: config.database,
collection: ‘agendaJobs’,
options: { useUnifiedTopology: true },
},
processEvery: "1 minute",
maxConcurrency: 20,
});
// listen for the ready or error event.
agenda
.on(‘ready’, () => console.log("Agenda started!"))
.on(‘error’, () => console.log("Agenda connection error!"));
// define all agenda jobs
allDefinitions(agenda);
// logs all registered jobs
console.log({ jobs: agenda._definitions });
module.exports = agenda;
All definitions are supplied with the initialized instance of Agenda.js via closures.
jobs/definitions/index.js:
const { mailDefinitions } = require("./mails");
const { payoutDefinitions } = require("./payout");
const definitions = [mailDefinitions, payoutDefinitions];
const allDefinitions = (agenda) => {
definitions.forEach((definition) => definition(agenda));
};
module.exports = { allDefinitions }
Then, each individual definition is defined as such.
Below, we are grouping all of our mail definitions and their associated handlers.
jobs/definitions/mails.js:
Learn more →const { JobHandlers } = require("../handlers");
const mailDefinitions = (agenda) => {
agenda.define("send-welcome-mail",JobHandlers.sendWelcomeEmail);
agenda.define("reset-otp-mail",JobHandlers.sendOTPEmail);
agenda.define(
"billings-info",
{
priority: "high",
concurrency: 20,
},
JobHandlers.monthlyBillingInformation
);
};
module.exports = { mailDefinitions }
Here, JobHandlers is a function that performs the specified tasks.
The jobs/definitions/payout.js file has similar contents.
JobHandlersThe JobHandlers file contains the actual function definitions that perform the required tasks.
jobs/handlers.js:
//NB this imports are relative to where you have this funtions defined in your own projects
const PaymentService = require("../relative-to-your-project");
const mailService = require("../relative-to-your-project");
const JobHandlers = {
completePayout: async (job, done) => {
const { data } = job.attrs;
await PaymentService.completePayout(data);
done();
},
sendWelcomeEmail: async (job, done) => {
const { data } = job.attrs;
await mailService.welcome(data);
done();
},
// .... more methods that perform diffrent tasks
};
module.exports = { JobHandlers }
Now that we have most of the boilerplate out of the way, we can use the singleton instance of Agenda.js in our controllers.
We can also wrap in another module — I prefer this latter approach because it helps keep things more organized and separate.
jobs/scheduler.js:
import { agenda } from "./index";
const schedule = {
completePayout: async (data) => {
await agenda.schedule("in 1 minute", "complete-payout", data);
},
sendWelcomeMail: async (data) => {
await agenda.now("send-welcome-mail", data);
},
// .... more methods that shedule tasks at the different intervals.
}
module.exports = { schedule }
Then, finally, we can make use of this in our controllers or routes, depending on how we structure our application.
user/controller.js:
//import the schdule file
const { schedule } = require("../schedule");
app.post("/signup", (req, res, next) => {
const user = new User(req.body);
user.save((err) => {
if (err) {
return next(err);
}
//use the schedule module
await schedule.sendWelcomeMail(user);
return res.status(201).send({
success: true,
message: "user successfully created",
token,
});
});
});
// use other schedule methods for other routes
While the above example shows a neutral way to structure Agenda.js in our application, it does conflict with the single responsibility principle (SRP).
Because we often need to do more than just send a welcome email when a user signs up, we might want to run analytics or some third-party integration. A simple controller that signs up a user might end up requiring 1,000 lines of code.
To prevent this, we’ll use an event emitter:
npm i eventemitter3
Then, in our Express application, we’ll make a singleton available across our entire application.
var EventEmitter = require('eventemitter3');
const myEmitter = new EventEmitter();
// where app is an instance of our express application.
//this should be done before all routes confuguration.
app.use(express.json());
app.use(express.urlencoded({ extended: true }));
app.set("myEmitter", myEmitter); // this makes my emitter available across all our controllers.
.....
module.exports = { myEmitter }
Having set this in place, we can edit our signup routes from above to dispatch a registration event that we can react to.
//import the schedule file
const { schedule } = require("../schedule");
app.post("/signup", (req, res, next) => {
const myEmitter = req.app.get("myEmitter");
const user = new User(req.body);
user.save((err) => {
if (err) {
return next(err);
}
//an emit event to handle registration.
myEmitter.emit('register', user );
return res.status(201).send({
success: true,
message: "user successfully created",
token,
});
});
});
Then, we can react to our register event by importing the same instance of myEmitter set in across our Express application.
const myEmitter = require("../relative-from-our-express-application");
const { schedule } = require("../schedule");
// then we can do perfom different actions like this
myEmitter.on('register', data => {
await schedule.sendWelcomeMail(data);
// run third party analytics
// send a sequence of mails
// ..do more stuff
})
In this tutorial, we demonstrated how Agenda.js can help us handle cron jobs in a structured manner. However, there are some use cases that are outside the scope of this article. You can refer to the documentation for Agenda.js and the eventemitter library to see an extensive guide.
Monitor failed and slow network requests in productionDeploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third-party services are successful, try LogRocket.
LogRocket is like a DVR for web and mobile apps, recording literally everything that happens while a user interacts with your app. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.
LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.
Mutative processes data with better performance than both Immer and native reducers. Let’s compare these data handling options in React.
Radix UI is quickly rising in popularity and has become an excellent go-to solution for building modern design systems and websites.
In this article, we’ll explore CSS cascade layers — and, specifically, the revert-layer keyword — to help you refine your styling strategy.
Nushell is a modern, performant, extensible shell built with Rust. Explore its pros, cons, and how to install and get started with it.
One Reply to "Job scheduling in Node.js using Agenda.js"
Great write up, it helped a lot.
I had issues with getting the job to run. Inside the definition functions, one needs to call await agenda.start(); before agenda.define().
The code in the definition might have worked for previous versions of the package, but [email protected] needs the start function.
Cheers.