Ukpai Ugochi I am a female Nigerian with a Bachelor's degree in Marine engineering and Bootcamp certificates in Software development. I'm a full stack JavaScript developer (MEVN) stack. I love to share knowledge about my transition from marine engineering to software development in the form of writing, to encourage people who love software development and don't know where to begin. I also contribute to FOSS in my free time.

Node.js buffer: A complete guide

6 min read 1844

Node.js Buffer: A Complete Guide

Think back to when you were taught that computers only understand binary language. You may recall that buffer is a space in computer memory (usually RAM) that stores binaries.

In this tutorial, we’ll walk through the basics of the Node.js buffer module and how to use Node.js buffer methods in your application. We’ll cover the following in detail:

What is binary code?

To understand what buffers are and the relationship between computers and binaries, let’s review the general the concept of binary code.

For computers to understand, process, and store data, data has to be converted to binary code. This is mainly because the computer processor is made of transistors, electronic machines that are activated by on (1) and off (0) signals.

Every piece of data sent to the computer is first converted to binary by a microprocessor before processing and outputting the result. Therefore, it’s crucial to be able to distinguish between different data types. By a process called encoding, the computer encodes dissimilar data types differently to tell one type from another.

What is Node.js buffer?

A binary stream is a collection of large amounts of binary data. Due to their massive size, binary streams are not sent together; they are broken into smaller piece before sending.

When the data processing unit can accept no more data streams, excess data is stored in a buffer until the data processing unit is ready to receive more data.

The buffer class in Node.js

Node.js servers most often need to read and write to the filesystem and, of course, files are stored in binaries. Also, Node.js deals with TCP streams, which secure connections to receivers before sending binary data in small chunks.

Streams of data being sent to the receiver need to be stored somewhere (buffer) until the receiver is ready to take in more chunks of data for processing. This is where the Node.js buffer class comes into play — it handles and stores binary data outside V8 engine.

Let’s move into understanding the different buffer methods, and how we can use them in our Node.js application.

We made a custom demo for .
No really. Click here to check it out.

Node.js buffer methods

One cool thing about the Node.js buffer module is that you don’t need to import it into your application before using its methods.

Below are some important Node.js buffer methods you should know.

Buffer.alloc()

The Buffer.alloc() method creates a new buffer of any size. When you use this method, you assign the size of the buffer in bytes.

The expression below creates a buffer with a byte size of 6:

const buf = Buffer.alloc(6);

console.log(buf);
// This will print <Buffer 00 00 00 00 00 00>

Buffer.byteLength()

You can check the length of a buffer object with the Buffer.byteLength() method.

Here’s how to create a buffer, attach a size to it, and check the size of the buffer you just created:

var buf = Buffer.alloc(6);

//check the length of buffer created
var buffLen = Buffer.byteLength(buf);

//print buffer length
console.log(buffLen);
// This will print <6>

Buffer.compare()

The Buffer.compare() method enables you to compare two buffer objects to check whether they are equal with the Buffer.compare() method. This method returns -1, 0, or 1, depending on the result of the comparison.

Here’s how to compare buffer objects with the B``uffer.compare() method:

var buf1 = Buffer.from('xyz');
var buf2 = Buffer.from('xyz');
var a = Buffer.compare(buf1, buf2);

//This will return 0
console.log(a);

var buf1 = Buffer.from('x');
var buf2 = Buffer.from('y');
var a = Buffer.compare(buf1, buf2);

//This will return -1
console.log(a);

var buf1 = Buffer.from('y');
var buf2 = Buffer.from('x');
var a = Buffer.compare(buf1, buf2);

//This will return 1
console.log(a);

Buffer.concat

Just like string concatenation, you can join two or more buffer objects into one object. You can also get the length of the new object gotten from concatenation.

var buffer1 = Buffer.from('x');
var buffer2 = Buffer.from('y');
var buffer3 = Buffer.from('z');
var arr = [buffer1, buffer2, buffer3];

/*This will print buffer, !concat [ <Buffer 78>, <Buffer 79>, <Buffer 7a> ]*/
console.log(arr);

//concatenate buffer with Buffer.concat method
var buf = Buffer.concat(arr);

//This will print <Buffer 78 79 7a> concat successful
console.log(buf);

buf.entries()

With buf.entries(), you can return a loop of indexes and bytes from the content of a buffer object. This is used to know the position and size of buffer contents.

var buf = Buffer.from('xyz');

for (a of buf.entries()) {
/*This will print arrays of indexes and byte of buffer content \[ 0, 120 \][ 1, 121 ][ 2, 122 ]*/
  console.log(a);
} 

Buffer.fill()

You can fill buffer with a specified value using the Buffer.fill() method. Buffer.fill() enables you to create a buffer, allocate a size, and fill with a value. The expression below shows how to use this method.

const b = Buffer.alloc(10).fill('a');

console.log(b.toString());
// This will print aaaaaaaaaa

Buffer.from()

The buffer.from() method enables you to create a new buffer from any object (string, buffer, array or ArrayBuffer()). All you have to do is specify the object you want to create a buffer from.

The syntax for using this method is Buffer.from(object[, offsetOrEncoding[, length]]). The object parameter can hold strings, buffer, arrays, or ArrayBuffer(), which means you can create new buffers from strings, buffer, arrays, and ArrayBuffer() with the buffer.from method.

The offset or encoding parameter is optional; it’s used for specifying string encoding while converting to buffer. If you don’t specify the encoding when creating buffers from string, the buffer will be created with a default encoding of utf8. You can also specify the number of bytes to expose with the length parameter, especially when creating buffer from ArrayBuffer.

While creating a buffer from an array, the array bytes should fall between the range of 0 and 255. Otherwise, the array entries will be shortened to fit in.

The example below shows how to create a buffer from strings, arrays and ArrayBuffer() using the buffer.from() method.

// Create a buffer from a string
var mybuff = Buffer.from("Nigeria");
//Print Buffer Created
console.log(mybuff);

// Create a buffer from a buffer
// Create buffer from string
var mybuff = Buffer.from("Nigeria");
// Create buffer from the first buffer created
var buff = Buffer.from(mybuff);
// Print out final buffer created.
console.log(buff);

// create a buffer from an array
const buf = Buffer.from([0x62, 0x75, 0x66, 0x66, 0x65, 0x72]);

// Create a buffer from an arraybuffer
const ab = new ArrayBuffer(10);
// Specify offset and length
const buf = Buffer.from(ab, 0, 2);
console.log(buff);

buff.includes()

If you want to determine whether a buffer object contains any values, you can use the buff.includes() method. With this method, you can search buffers to ascertain whether they contain expressions you wish to search for. The method returns a boolean true or false depending on whether a value is found.

const buf = Buffer.from('this is a buffer');
console.log(buf.includes('this'));
// This will print true

console.log(buf.includes(Buffer.from('a buffer example')));
// This will print false

Buffer.isEncoding()

Since binaries have to be encoded to tell them apart, how do you know whether a particular data type supports the character encoding allocated to it? You can use the Buffer.isEncoding() method to confirm whether a particular character encoding type is supported.

console.log(Buffer.isEncoding('hex'));
// This will print true

console.log(Buffer.isEncoding('utf-8'));
// This will print true

console.log(Buffer.isEncoding('utf/8'));
// This will print false

console.log(Buffer.isEncoding('hey'));
// This will print false

buf.slice()

Just like the slice method in JavaScript, buf.slice() is used to create a new array from selected elements in an array. When you slice an array, you create a new array with a list of elements you choose in the slice.

var a = Buffer.from('uvwxyz');
var b = a.slice(2,5);

console.log(b.toString());
//This will print wxy

Buffer swap

Buffer swap is used to swap the byte order of a buffer object. This method can also be used for fast endianness conversion.

You can use buf.swap16(), buf.swap32(), and buf.swap64() to swap the byte order of a 16-bit, 32-bit, and 64-bit buffer object, respectively.

const buf1 = Buffer.from([0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8]);
console.log(buf1);
// This will print <Buffer 01 02 03 04 05 06 07 08>

//swap byte order to 16 bit
buf1.swap16();
console.log(buf1);
// This will print <Buffer 02 01 04 03 06 05 08 07>

//swap byte order to 32 bit
buf1.swap32();
console.log(buf1);
// This will print <Buffer 03 04 01 02 07 08 05 06>

//swap byte order to 64 bit
buf1.swap64();
console.log(buf1);
// This will print <Buffer 06 05 08 07 02 01 04 03>

buf.json()

You can return a JSON object from a buffer object with this method. The buf.json() method returns a JSON version of the buffer object.

const buf = Buffer.from([0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8]);

console.log(buf.toJSON());
// This will print {"type":"Buffer", data:[1,2,3,4,5,6,7,8]}

Buffer offset Read

You can determine the number of bytes to skip before starting to read an unassigned integer from a buffer object. The number of bytes to skip is determined by the offset you attach to this method.

You can read 64-bit double, 32-bit float, 16-bit integer, and 32-bit integer. According to the Node.js method you use, the result could be in little endian or big endian.

You can also read 8-bit integer with buf.readInt8().

Buffer offset write

This method is used for writing a specified byte to a buffer object according to the offset attached to the method.

This method can write 64-bit double, 32-bit float, 8-bit integer, 16-bit integer, and 32-bit integer in little endian or big endian — again, depending on the method used. The value in each method should correspond to the integer specified by the method. For instance, buf.writeFloatBE() should have a value of 32-bit float.

Conclusion

It is essential that you understand what’s going on under the hood when you execute a program. You should now have a solid, foundational understanding of the relationship between binaries, streams, and buffer and how Node.js buffer works. You should also understand why you need to use the Node.js buffer class and the various Node.js buffer methods.

We didn’t get a chance to cover every single method available in the Node.js buffer class. You can also create a buffer object with allocUnsafe() method, but the buffer created will be nonzero-filled. If you want to compare buffer objects, Buffer.compare() method is a good fit for you, but buf.equals() returns true or false after comparison instead of 1, -1, and 0 returned by Buffer.compare().

You can read more about the Node.js buffer module in the Node.js docs.

200’s only Monitor failed and slow network requests in production

Deploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third party services are successful, try LogRocket. https://logrocket.com/signup/

LogRocket is like a DVR for web apps, recording literally everything that happens on your site. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.

LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. .
Ukpai Ugochi I am a female Nigerian with a Bachelor's degree in Marine engineering and Bootcamp certificates in Software development. I'm a full stack JavaScript developer (MEVN) stack. I love to share knowledge about my transition from marine engineering to software development in the form of writing, to encourage people who love software development and don't know where to begin. I also contribute to FOSS in my free time.

Leave a Reply