Multithreading in Flutter using Dart isolates

Flutter is a great framework for writing cross-platform code that works across many devices and ecosystems. Such apps tend to have familiar patterns, like fetching data from the network, serializing it, and showing the user the result in the form of a UI.

Developers working on mobile platforms are likely aware that some expensive operations within this flow either need to be asynchronous or should run on background threads because overloading the main thread, which handles meta tasks like the event loop, will result in some janky behavior.

The Flutter framework provides nice APIs in the form of Streams and Futures to build out asynchronous behavior, but these are not completely sufficient to avoid jank. In this article, we’ll learn how to leverage multithreading in Flutter to run things in the background and keep the load off our main thread.

We’ll cover the following topics, including two methods of creating isolates in Flutter:

• Concurrency vs. asynchrony in Flutter
• What are Flutter isolates?
• Method 1: Using compute
• Method 2: Using Isolate.spawn
• Reusing Flutter isolates
• Exploring the flutter_isolate package
• Integrating isolates with stream APIs
• Flutter isolate best practices

Concurrency vs. asynchrony in Flutter

In Flutter, you can introduce asynchronous behavior using async/await for your functions and Stream APIs. However, the concurrency of your code depends on the underlying threading infrastructure that Flutter provides.

Understanding Flutter’s threading infrastructure

Firstly, Flutter maintains a set of thread pools at a VM level. These pools are used when we need to perform certain tasks, such as Network I/O.

Secondly, rather than exposing threads, Flutter provides a different concurrency primitive called isolates. In Flutter, the entire UI and most of your code run on what’s called the root isolate.

What are Flutter isolates?

An isolate is an abstraction on top of threads. It is similar to an event loop, with a few differences:

• An isolate has its own memory space
• It cannot share mutable values with other isolates
• Any data transmitted between isolates is duplicated

An isolate is meant to run independently of other isolates. This offers a lot of benefits to the Dart VM, one of which is that garbage collection is easier.

One thing to keep in mind about creating parent isolates that, in turn, create child isolates is that the child isolates will terminate if the parent does. Regardless of the hierarchy, the parent isolate cannot access the memory of the child isolate.

There are also a few components that are usually associated with isolates:

• A ReceivePort: This is used by the isolate to receive data. Another instance of this can also be used by the parent isolate to send data to the spawned isolate
• A control port: This is a special port that allows its owner to have capabilities such as pausing or terminating the isolate
• Capability: These are object instances used for isolate authentication, i.e., whenever we wish to send control port commands like pause or terminate, we also need the corresponding instances of Capability that were used when the isolate was created, without which the command would fail

Method 1: Using compute

As mentioned above, there are a couple of ways to create isolates in Flutter. One of the easiest is to use the compute function. This will execute our code in a different isolate and return the results to our main isolate.

Let’s say we have a class called Person, which we wish to deserialize from a JSON object:

class Person {
  final String name;
  Person(this.name);
}

We can add the deserializing code as follows:

Person deserializePerson(String data) {
 // JSON decoding is a costly thing its preferable  
 // if we did this off the main thread
  Map<String, dynamic> dataMap = jsonDecode(data);
  return Person(dataMap["name"]);
}

Now, to use it with the compute function, all we need to do is:

Future<Person> fetchUser() async {
  String userData = await Api.getUser();
  return await compute(deserializePerson, userData);
}

This would internally spawn an isolate, run the decoding logic in it, and return the result to our main isolate. This is suitable for tasks that are infrequent or one-offs, since we cannot reuse the isolate.

Method 2: Using Isolate.spawn

This method is one of the elementary ways to work with isolates, and it should come as no surprise that the compute method also uses this under the hood.

Here’s what our deserialization code looks like:

Future<Person> fetchUser() async {
      ReceivePort port = ReceivePort();
      String userData = await Api.getUser();
      final isolate = await Isolate.spawn<List<dynamic>>(
          deserializePerson, [port.sendPort, userData]);
      final person = await port.first;
      isolate.kill(priority: Isolate.immediate);
      return person;
}

void deserializePerson(List<dynamic> values) {
    SendPort sendPort = values[0];
    String data = values[1];
    Map<String, dynamic> dataMap = jsonDecode(data);
    sendPort.send(Person(dataMap["name"]));
}

One of the first things we should do is create an instance of ReceivePort. This allows us to listen to the response of the isolate.

The spawn function takes in two parameters:

1. A callback that is invoked within the new isolate (in our case, deserializePerson)
2. The parameter that deserializePerson takes

We combine both the port and the serialized data into a list and send it across. Next, we use sendPort.send to return the value to the main isolate and await the same with port.first. Finally, we kill the isolate to complete the cleanup.

Reusing Flutter isolates

While the previous example is best used for a single-shot task, we can easily reuse the isolate we created above by setting up two ports for bidirectional communication, and sending more data to deserialize while listening to the port stream for the results.

To do that, all we need to do is make some changes to our deserializing function:

void deserializePerson(SendPort sendPort) {
  ReceivePort receivePort = ReceivePort();
  sendPort.send(receivePort.sendPort);
  receivePort.listen((message) {
    Map<String, dynamic> dataMap = jsonDecode(message);
    sendPort.send(Person(dataMap["name"]));
  });
}

As you can see, the first item emitted by our function is a corresponding port, which the calling function can use to send data continuously to our new isolate.

Note that SendPort supports a limited number of data types — including lists, maps, SendPort, and TransferrableTypedData — apart from the primitive data types.

This method would work well in cases where we frequently need to do repetitive tasks in the background, like decoding JSON from an API. With these changes, we can send new responses and obtain the deserialized response from the same isolate.

Exploring the flutter_isolate package

Bidirectional communication enables us to reuse isolates, but more often than not, we wish to implement the equivalent of a thread pool, i.e., instantiate a set of isolates once and then reuse them as desired.

Flutter’s isolate package offers us several utility tools to help achieve this, and one of the most useful is the LoadBalancer API. This class lets us create and manage a pool of isolates. It automatically delegates a task to a free isolate when it receives it.

To use it, all we need to do is to include the package in our pubspec.yaml, like so:

 class="language-yaml hljs">dependencies:
  isolate: 2.1.1

and then update our UserService class to make use of the runner:

class UserService{
  LoadBalancer? balancer;

  Future<Person> fetchUser() async {
    String userData = await Api.getUser();
    balancer ??= await LoadBalancer.create(5, IsolateRunner.spawn);
    return await balancer!.run(deserializeJson , userData, load: 1);
  }

  Person deserializeJson(String data) {
    Map<String, dynamic> dataMap = jsonDecode(data);
    return Person(dataMap["name"]);
  }
}

Here, we’ve created a pool of five isolates and reuse them for subsequent decoding tasks. The balancer’s run function also takes an optional parameter called load, which is an integer representing the load the decoding task would have on the isolate.

We can also use this method if we wish to run tasks where one is computationally more expensive than others. For example, consider the following code:

LoadBalancer? balancer;

Future<Person> fetchInitialUsers() async {
  String userData = await Api.getUsers(count: 5);
  balancer ??= await LoadBalancer.create(2, IsolateRunner.spawn);
  return await balancer!.run(deserializeUsers, userData, load: 1);
}

Future<Person> fetchSecondaryUsers() async {
  String userData = await Api.getUsers(count: 15);
  balancer ??= await LoadBalancer.create(2, IsolateRunner.spawn);
  return await balancer!.run(deserializeUsers, userData, load: 3);
}

Notice how we’ve assigned the load for fetchInitialUsers as 1 and fetchSecondaryUsers as 3 — these indicate a number proportional to the number of users being deserialized. When we initially call fetchInitialUsers, the first isolate will run the deserialization; at the same time, if fetchSecondaryUsers is called, the load balancer will observe that the first isolate is busy with its assigned load of 1 and transfer it to the second isolate. If it’s free, with a load of 0, the second isolate will run with a load of 3. The balancer ensures any new task is queued to the isolate with the lowest load.

We can use load balancers when we have recurring tasks that need a few isolates. One example is an image loader that needs to downsize images based on a target view size — we can use a balancer pool to queue all downsizing tasks. This prevents it from overwhelming the main isolate and also avoids penalties associated with spawning new isolates too frequently.

Integrating isolates with stream APIs

Flutter offers an asyncMap operator to integrate our existing streams with isolates.

For example, if we are operating on chunked data from a file — which is generally done as a stream for memory efficiency — the file-read stream can be hooked to a load-balanced isolate in this fashion to run the code in the background.

The recipient of the stream can then collate the data on the UI/main isolate. This is similar to how we’d switch threads in reactive programming.

We can use this within an existing stream as follows.

//let's say we have to compute an md5 of a string along with a salt
encryptionSaltStream()
.asyncMap((salt) =>
 // Assuming our load balancer is already created   
    balancer!.run(generateMd5, salt.toString() + userData)
);

This would be a good use case where we wish to scale down a set of images. We can emit the URLs as a stream, read the file(s) in the isolate, scale them in the background, and then collect the updated file path in the receiver.

Flutter isolate best practices

While it may seem beneficial to create as many isolates as we want, spawning isolates have a cost that varies across devices. It’s important to understand that isolates work wonderfully when it comes to things like image manipulation, but the cost sometimes can’t be justified for simpler use cases.

Another thing to note is that data between isolates is duplicated, which creates an overhead when dealing with larger datasets and puts a heavier burden on memory. To avoid this, Flutter offers TransferrableTypedData, which acts as a byte wrapper that can be transferred between isolates without the overhead. Be sure to use this if you plan to process large files in your isolates.

Conclusion

In this article, we’ve learned about isolates and how they help bring concurrency into our code to keep our UI thread free of unnecessary overheads. It’s still important to profile our apps since isolate-spawning attracts a fair bit of cost and may not be a good use case.

You can explore the complete examples in the GitHub repo I created.