Editor’s note: This article was last updated 14 July 2022 to include additional buffer methods.
If you think back to when you were taught that computers only understand binary language, you may recall that buffer is a space in computer memory, usually RAM, that stores binaries. In this tutorial, we’ll walk through the basics of the Node.js buffer module and learn how to use Node.js buffer methods in our own application.
Buffer.alloc()
Buffer.write()
Buffer.byteLength()
Buffer.compare()
Buffer.concat()
buf.entries()
Buffer.fill()
Buffer.from()
buf.includes()
Buffer.isEncoding()
buf.slice()
buf.json()
To understand what buffers are and the relationship between computers and binaries, let’s review the general concept of binary code.
For computers to understand, process, and store data, data has to be converted to binary code. This is mainly because the computer processor is made of transistors, electronic machines that are activated by on (1)
and off (0)
signals.
Every piece of data sent to the computer is first converted to binary by a microprocessor before processing and outputting the result. Therefore, it’s crucial to be able to distinguish between different data types. Through a process called encoding, the computer encodes dissimilar data types differently to tell one type from another.
A binary stream is a collection of large amounts of binary data. Due to their massive size, binary streams are not sent together. Instead, they are broken into smaller pieces before sending.
When the data processing unit cannot accept any more data streams, excess data is stored in a buffer until the data processing unit is ready to receive more data.
Node.js servers most often need to read and write to the filesystem and, of course, files are stored in binaries. Node.js also deals with TCP streams, which secure connections to receivers before sending binary data in small chunks.
Streams of data being sent to the receiver need to be stored somewhere until the receiver is ready to take in more chunks of data for processing. This is where the Node.js buffer class comes into play. It handles and stores binary data outside of the V8 engine.
Let’s review the different buffer methods and learn how to use them in our Node.js application.
One cool thing about the Node.js buffer module is that you don’t need to import it into your application before using its methods. Let’s review some important Node.js buffer methods that you should know.
Buffer.alloc()
The Buffer.alloc()
method creates a new buffer of any size. When you use this method, you assign the size of the buffer in bytes. The expression below creates a buffer with a byte size of 6
:
const buf = Buffer.alloc(6); console.log(buf); // This will print <Buffer 00 00 00 00 00 00>
Buffer.write()
The Buffer.write()
method writes a string to the buffer, which can be useful when you need to stream strings in the form of buffers. You can write a string to a buffer using the below method:
const buf = Buffer.alloc(100); // Creating a new Buffer const len = buf.write("Hello world!"); // Writing to the Buffer // len is now 12
The Buffer.write()
function returns the length of the string, which is stored in the buffer.
Buffer.byteLength()
You can check the length of a buffer object with the Buffer.byteLength()
method. The code below demonstrates how to create a buffer, attach a size to it, and check the size of the buffer you just created:
var buf = Buffer.alloc(6); //check the length of buffer created var buffLen = Buffer.byteLength(buf); //print buffer length console.log(buffLen); // This will print <6>
Buffer.compare()
The Buffer.compare()
method enables you to compare two buffer objects to check whether they are equal. This method returns -1
, 0
, or 1
, depending on the result of the comparison.
You can compare buffer objects with the Buffer.compare()
method as seen below:
var buf1 = Buffer.from('xyz'); var buf2 = Buffer.from('xyz'); var a = Buffer.compare(buf1, buf2); //This will return 0 console.log(a); var buf1 = Buffer.from('x'); var buf2 = Buffer.from('y'); var a = Buffer.compare(buf1, buf2); //This will return -1 console.log(a); var buf1 = Buffer.from('y'); var buf2 = Buffer.from('x'); var a = Buffer.compare(buf1, buf2); //This will return 1 console.log(a);
Buffer.concat()
Just like string concatenation, you can join two or more buffer objects into one object. You can also get the length of the new object:
var buffer1 = Buffer.from('x'); var buffer2 = Buffer.from('y'); var buffer3 = Buffer.from('z'); var arr = [buffer1, buffer2, buffer3]; /*This will print buffer, !concat [ <Buffer 78>, <Buffer 79>, <Buffer 7a> ]*/ console.log(arr); //concatenate buffer with Buffer.concat method var buf = Buffer.concat(arr); //This will print <Buffer 78 79 7a> concat successful console.log(buf);
buf.entries()
With buf.entries()
, you can return a loop of indexes and bytes from the content of a buffer object, which is used to know the position and size of buffer contents:
var buf = Buffer.from('xyz'); for (a of buf.entries()) { /*This will print arrays of indexes and byte of buffer content \\[ 0, 120 \][ 1, 121 \][ 2, 122 ]*/ console.log(a); }
Buffer.fill()
The Buffer.fill()
method enables you to create a buffer, allocate a size, and fill it with a specified value. The expression below shows how to use the Buffer.fill()
method:
const b = Buffer.alloc(10).fill('a'); console.log(b.toString()); // This will print aaaaaaaaaa
Buffer.from()
The buffer.from()
method enables you to create a new buffer from any object, like strings, buffer, arrays, and ArrayBuffer()
. All you have to do is specify the object you want to create a buffer from.The syntax for using this method is Buffer.from(object[, offsetOrEncoding[,length]])
.
The offset or encoding parameter is optional; it’s used for specifying string encoding while converting to buffer. If you don’t specify the encoding parameter when creating buffers from a string, the buffer will be created with a default encoding of utf8
.
You can also specify the number of bytes to expose with the length parameter, especially when creating buffer from ArrayBuffer
. While creating a buffer from an array, the array bytes should fall between the range of 0
and 255
. Otherwise, the array entries will be shortened to fit in.
The example below shows how to create a buffer from strings, arrays, and ArrayBuffer()
using the buffer.from()
method:
// Create a buffer from a string var mybuff = Buffer.from("Nigeria"); //Print Buffer Created console.log(mybuff); // Create a buffer from a buffer // Create buffer from string var mybuff = Buffer.from("Nigeria"); // Create buffer from the first buffer created var buff = Buffer.from(mybuff); // Print out final buffer created. console.log(buff); // create a buffer from an array const buf = Buffer.from([0x62, 0x75, 0x66, 0x66, 0x65, 0x72]); // Create a buffer from an arraybuffer const ab = new ArrayBuffer(10); // Specify offset and length const buf = Buffer.from(ab, 0, 2); console.log(buff);
buff.includes()
If you want to determine whether a buffer object contains any values, you can use the buff.includes()
method. With this method, you can search buffers to ascertain whether they contain expressions you wish to search for. The method returns a boolean true
or false
depending on whether a value is found:
const buf = Buffer.from('this is a buffer'); console.log(buf.includes('this')); // This will print true console.log(buf.includes(Buffer.from('a buffer example'))); // This will print false
Buffer.isEncoding()
To tell binaries apart, they must be encoded. You can use the Buffer.isEncoding()
method to confirm whether a particular character encoding type is supported:
console.log(Buffer.isEncoding('hex')); // This will print true console.log(Buffer.isEncoding('utf-8')); // This will print true console.log(Buffer.isEncoding('utf/8')); // This will print false console.log(Buffer.isEncoding('hey')); // This will print false
buf.slice()
Just like the slice method in JavaScript, buf.slice()
is used to create a new array from selected elements in an array. When you slice an array, you create a new array with a list of elements you choose in the slice:
var a = Buffer.from('uvwxyz'); var b = a.slice(2,5); console.log(b.toString()); //This will print wxy
Buffer swap is used to swap the byte order of a buffer object. This method can also be used for fast endianness conversion.
You can use buf.swap16()
, buf.swap32()
, and buf.swap64()
to swap the byte order of a 16-bit, 32-bit, and 64-bit buffer object, respectively:
const buf1 = Buffer.from([0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8]); console.log(buf1); // This will print <Buffer 01 02 03 04 05 06 07 08> //swap byte order to 16 bit buf1.swap16(); console.log(buf1); // This will print <Buffer 02 01 04 03 06 05 08 07> //swap byte order to 32 bit buf1.swap32(); console.log(buf1); // This will print <Buffer 03 04 01 02 07 08 05 06> //swap byte order to 64 bit buf1.swap64(); console.log(buf1); // This will print <Buffer 06 05 08 07 02 01 04 03>
buf.json()
The buf.json()
method returns a JSON version of the buffer object:
const buf = Buffer.from([0x1, 0x2, 0x3, 0x4, 0x5, 0x6, 0x7, 0x8]); console.log(buf.toJSON()); // This will print {"type":"Buffer", data:[1,2,3,4,5,6,7,8]}
With the Buffer offset Read method, you can determine the number of bytes to skip before starting to read an unassigned integer from a buffer object. The number of bytes to skip is determined by the offset you attach to this method.
For example, you can read 64-bit double, 32-bit float, 16-bit integer, and 32-bit integer. According to the Node.js method you use, the result could be in little endian or big endian. You can also read 8-bit integer with buf.readInt8()
.
The Buffer offset write method is used for writing a specified byte to a buffer object according to the offset attached to the method.
This method can write 64-bit double, 32-bit float, 8-bit integer, 16-bit integer, and 32-bit integer in little endian or big endian depending on the method used. The value in each method should correspond to the integer specified by the method. For instance, buf.writeFloatBE()
should have a value of 32-bit float.
When you execute a program, it’s essential that you understand what’s going on under the hood. You should now have a solid, foundational understanding of the relationship between binaries, streams, buffer, and how Node.js buffer works. You should also understand why you need to use the Node.js buffer class and the various Node.js buffer methods.
We didn’t get a chance to cover every single method available in the Node.js buffer class. You can also create a buffer object with allocUnsafe()
method, but the buffer created will be nonzero-filled.
If you want to compare buffer objects, the Buffer.compare()
method is a good fit for you, but buf.equals()
returns true or false after comparison instead of 1
, -1
, and 0
. You can read more about the Node.js buffer module in the Node.js docs.
Deploying a Node-based web app or website is the easy part. Making sure your Node instance continues to serve resources to your app is where things get tougher. If you’re interested in ensuring requests to the backend or third-party services are successful, try LogRocket.
LogRocket is like a DVR for web and mobile apps, recording literally everything that happens while a user interacts with your app. Instead of guessing why problems happen, you can aggregate and report on problematic network requests to quickly understand the root cause.
LogRocket instruments your app to record baseline performance timings such as page load time, time to first byte, slow network requests, and also logs Redux, NgRx, and Vuex actions/state. Start monitoring for free.
Would you be interested in joining LogRocket's developer community?
Join LogRocket’s Content Advisory Board. You’ll help inform the type of content we create and get access to exclusive meetups, social accreditation, and swag.
Sign up nowLearn how to manage memory leaks in Rust, avoid unsafe behavior, and use tools like weak references to ensure efficient programs.
Bypass anti-bot measures in Node.js with curl-impersonate. Learn how it mimics browsers to overcome bot detection for web scraping.
Handle frontend data discrepancies with eventual consistency using WebSockets, Docker Compose, and practical code examples.
Efficient initializing is crucial to smooth-running websites. One way to optimize that process is through lazy initialization in Rust 1.80.