Futures - Isolates - Event Loop

Introduction

I recently received a couple of questions related to the notions of Future, async, await, Isolate and parallel processing.

Along with these questions, some people experienced troubles with the sequence order of the processing of their code.

I thought that it would be quite useful to take the opportunity of an article to cover these topics and remove any ambiguity, mainly around the notions of asynchronous and parallel processings.

Dart is a Single Threaded language

First things first, everyone needs to bear in mind that Dart is Single Thread and Flutter relies on Dart.

IMPORTANT

Dart executes one operation at a time, one after the other meaning that as long as one operation is executing, it cannot be interrupted by any other Dart code.

In other words, if you consider a purely synchronous method, the latter will be the only one to be executed until it is done.

void myBigLoop(){
  for (int i = 0; i < 1000000; i++){
      _doSomethingSynchronously();
  }
}

In the example above, the execution of the myBigLoop() method will never be interrupted until it completes. As a consequence, if this method takes some time, the application will be "blocked" during the whole method execution.

The Dart execution model

Behind the scene, how does Dart actually manage the sequence of operations to be executed?

In order to answer this question, we need to have a look at the Dart code sequencer, called the Event Loop.

When you start a Flutter (or any Dart) application, a new Thread process (in Dart language = "Isolate") is created and launched. This thread will be the only one that you will have to care for the entire application.

So, when this thread is created, Dart automatically

1. initializes 2 Queues, namely "MicroTask" and "Event" FIFO queues;
2. executes the main() method and, once this code execution is completed,
3. launches the Event Loop

During the whole life of the thread, a single internal and invisible process, called the "Event Loop", will drive the way your code will be executed and in which sequence order, depending on the content of both MicroTask and Event Queues.

The Event Loop corresponds to some kind of infinite loop, cadenced by an internal clock which, at each tick, if no other Dart code is being executed, does something like the following:

void eventLoop(){
  while (microTaskQueue.isNotEmpty){
      fetchFirstMicroTaskFromQueue();
      executeThisMicroTask();
      return;
  }

  if (eventQueue.isNotEmpty){
      fetchFirstEventFromQueue();
      executeThisEventRelatedCode();
  }
}

As we can see the MicroTask Queue has precedence over the Event Queue but what are those 2 queues used for?

MicroTask Queue

The MicroTask Queue is used for very short internal actions that need to be run asynchronously, right after something else completes and before giving the hand back to the Event Queue.

As an example of a MicroTask you could imagine having to dispose a resource, right after it has been closed. As the closure process could take some time to complete, you could write something like this:

MyResource myResource;

    ...

    void closeAndRelease() {
        scheduleMicroTask(_dispose);
        _close();
    }

    void _close(){
        // The code to be run synchronously
        // to close the resource
        ...
    }

    void _dispose(){
        // The code which has to be run
        // right after the _close()
        // has completed
    }

This is something that most of the time, you will not have to work with. As an example, the whole Flutter source code references the scheduleMicroTask() method only 7 times.

It is always preferable to consider using the Event Queue.

Event Queue

The Event Queue is used to reference operations that result from

• external events such as
  • I/O;
  • gesture;
  • drawing;
  • timers;
  • streams;
  • ...
• futures

In fact, each time an external event is triggered, the corresponding code to be executed is referenced into the Event Queue.

As soon as there is no longer any micro task to run, the Event Loop considers the first item in the Event Queue and will execute it.

It is very interesting to note that Futures are also handled via the Event Queue.

Futures

A Future corresponds to a task that runs asynchronously and completes (or fails) some point in time in the future.

When you instantiate a new Future:

• an instance of that Future is created and recorded in an internal array, managed by Dart;
• the code that needs to be executed by this Future is directly pushed into the Event Queue;
• the future instance is returned with a status (=incomplete);
• if any, the next synchronous code is executed (NOT the code of the Future)

The code referenced by the Future will be executed like any other Event, as soon as the Event Loop will pick it up from the Event loop.

When that code will be executed and will complete (or fail) its then() or catchError() will directly be executed.

In order to illustrate this, let's take the following example:

void main(){
  print('Before the Future');
  Future((){
      print('Running the Future');
  }).then((_){
      print('Future is complete');
  });
  print('After the Future');
}

If we run this code, the output will be the following:

Before the Future
After the Future
Running the Future
Future is complete

This is totally normal as the flow of execution is the following:

1. print('Before the Future')
2. add "(){print('Running the Future');}" to the Event Queue;
3. print('After the Future')
4. the Event Loop fetches the code (referenced in bullet 2) and runs it
5. when the code is executed, it looks for the then() statement and runs it

Something very important to keep in mind:

A Future is NOT executed in parallel but following the regular sequence of events, handled by the Event Loop

Async methods

When you are suffixing the declaration of a method with the async keyword, Dart knows that:

• the outcome of the method is a Future;
• it runs synchronously the code of that method up to the very first await keyword, then it pauses the execution of the remainder of that method;
• the next line of code will be run as soon as the Future, referenced by the await keyword, will have completed.

This is very important to understand this since many developers think that await pauses the execution if the whole flow until it completes but this is not the case. They forget how the Event Loop works...

To better illustrate this statement, let's take the following sample and let's try to figure out the outcome of its execution.

void main() async {
  methodA();
  await methodB();
  await methodC('main');
  methodD();
}

methodA(){
  print('A');
}

methodB() async {
  print('B start');
  await methodC('B');
  print('B end');
}

methodC(String from) async {
  print('C start from $from');
  
  Future((){                // <== This code will be executed some time in the future
    print('C running Future from $from');
  }).then((_){
    print('C end of Future from $from');
  });

  print('C end from $from');  
}

methodD(){
  print('D');
}

The correct sequence is the following:

1. A
2. B start
3. C start from B
4. C end from B
5. B end
6. C start from main
7. C end from main
8. D
9. C running Future from B
10. C end of Future from B
11. C running Future from main
12. C end of Future from main

Now, let's consider that "methodC()" from the code above would correspond to a call to a server which might take uneven time to respond. I believe it is obvious to say that it might become very difficult to predict the exact flow of execution.

If your initial expectation from the sample code would have been to only execute methodD() at the end of everything, you should have written the code, as follows:

void main() async {
  methodA();
  await methodB();
  await methodC('main');
  methodD();  
}

methodA(){
  print('A');
}

methodB() async {
  print('B start');
  await methodC('B');
  print('B end');
}

methodC(String from) async {
  print('C start from $from');
  
  await Future((){                  // <== modification is here
    print('C running Future from $from');
  }).then((_){
    print('C end of Future from $from');
  });
  print('C end from $from');  
}

methodD(){
  print('D');
}

This gives the following sequence:

1. A
2. B start
3. C start from B
4. C running Future from B
5. C end of Future from B
6. C end from B
7. B end
8. C start from main
9. C running Future from main
10. C end of Future from main
11. C end from main
12. D

The fact of adding a simple await at the level of the Future in methodC() changes the whole behavior.

Also very important to keep in mind:

An async method is NOT executed in parallel but following the regular sequence of events, handled by the Event Loop, too.

A last example I wanted to show you is the following.
What will be the output of running method1 and the one of method2? Will they be the same?

void method1(){
  List<String> myArray = <String>['a','b','c'];
  print('before loop');
  myArray.forEach((String value) async {
    await delayedPrint(value);
  });  
  print('end of loop');
}

void method2() async {
  List<String> myArray = <String>['a','b','c'];
  print('before loop');
  for(int i=0; i<myArray.length; i++) {
    await delayedPrint(myArray[i]);
  }
  print('end of loop');
}

Future<void> delayedPrint(String value) async {
  await Future.delayed(Duration(seconds: 1));
  print('delayedPrint: $value');
}

Answer:

Did you see the difference and the reason why their behavior is not the same?

The solution resides in the fact that method1 uses a function forEach() to iterate the array. Each time it iterates, it invokes a new callback which is flagged as an async (thus a Future). It executes it until it gets to the await then it pushes the remainder of the code into the Event queue. As soon as the iteration is complete, it executes the next statement "print('end of loop')". Once done, the Event Loop will process the 3 callbacks.

As regard method2, everything runs inside the same code "block" and therefore runs (in this example), line after line.

As you can see, even in a code that looks like very simple, we still need to keep in mind how the Event Loop works...

Multi-Threading

Therefore, how could we run parallel codes in Flutter? Is this possible?

Yes, thanks to the notion of Isolates.

What is an Isolate?

An Isolate corresponds to the Dart version of the notion of Thread, as already explained earlier.

However, there is a major difference with the usual implementation of "Threads" and this is why they are named "Isolates".

"Isolates" in Flutter do not share memory. Interaction between different "Isolates" is made via "messages" in terms of communication.

Each Isolate has its own "Event Loop"

Each "Isolate" has its own "Event Loop" and Queues (MicroTask and Event). This means that the code runs inside in an Isolate, independently of another Isolate.

Thanks to this, we can obtain parallel processing.

How to launch an Isolate?

Depending on the needs you have to run an Isolate, you might need to consider different approaches.

1. Low-Level solution

This first solution does not use any package and fully relies on the low-level API, offered by Dart.

1.1. Step 1: creation and hand-shaking

As I said earlier, Isolates do not share any memory and communicate via messages, therefore, we need to find a way to establish this communication between the "caller" and the new isolate.

Each Isolate exposes a port which is used to convey a message to that Isolate. This port is called "SendPort" (I personally find the name is bit misleading since it is a port aimed at receiving/listening, but this is the official name).

This means that both "caller" and "new isolate" need to know the port of each other to be able to communicate. This hand-shaking process is shown here below:

//
// The port of the new isolate
// this port will be used to further
// send messages to that isolate
//
SendPort newIsolateSendPort;

//
// Instance of the new Isolate
//
Isolate newIsolate;

//
// Method that launches a new isolate
// and proceeds with the initial
// hand-shaking
//
void callerCreateIsolate() async {
    //
    // Local and temporary ReceivePort to retrieve
    // the new isolate's SendPort
    //
    ReceivePort receivePort = ReceivePort();

    //
    // Instantiate the new isolate
    //
    newIsolate = await Isolate.spawn(
        callbackFunction,
        receivePort.sendPort,
    );

    //
    // Retrieve the port to be used for further
    // communication
    //
    newIsolateSendPort = await receivePort.first;
}

//
// The entry point of the new isolate
//
static void callbackFunction(SendPort callerSendPort){
    //
    // Instantiate a SendPort to receive message
    // from the caller
    //
    ReceivePort newIsolateReceivePort = ReceivePort();

    //
    // Provide the caller with the reference of THIS isolate's SendPort
    //
    callerSendPort.send(newIsolateReceivePort.sendPort);

    //
    // Further processing
    //
}

CONSTRAINTS

The "entry point" of an isolate MUST be a top-level function or a STATIC method.

1.2. Step 2: submission of a Message to the Isolate

Now that we have the port to be used to send a message to the Isolate, let's see how to do it:

//
// Method that sends a message to the new isolate
// and receives an answer
// 
// In this example, I consider that the communication
// operates with Strings (sent and received data)
//
Future<String> sendReceive(String messageToBeSent) async {
    //
    // We create a temporary port to receive the answer
    //
    ReceivePort port = ReceivePort();

    //
    // We send the message to the Isolate, and also
    // tell the isolate which port to use to provide
    // any answer
    //
    newIsolateSendPort.send(
        CrossIsolatesMessage<String>(
            sender: port.sendPort,
            message: messageToBeSent,
        )
    );

    //
    // Wait for the answer and return it
    //
    return port.first;
}

//
// Extension of the callback function to process incoming messages
//
static void callbackFunction(SendPort callerSendPort){
    //
    // Instantiate a SendPort to receive message
    // from the caller
    //
    ReceivePort newIsolateReceivePort = ReceivePort();

    //
    // Provide the caller with the reference of THIS isolate's SendPort
    //
    callerSendPort.send(newIsolateReceivePort.sendPort);

    //
    // Isolate main routine that listens to incoming messages,
    // processes it and provides an answer
    //
    newIsolateReceivePort.listen((dynamic message){
        CrossIsolatesMessage incomingMessage = message as CrossIsolatesMessage;

        //
        // Process the message
        //
        String newMessage = "complemented string " + incomingMessage.message;

        //
        // Sends the outcome of the processing
        //
        incomingMessage.sender.send(newMessage);
    });
}

//
// Helper class
//
class CrossIsolatesMessage<T> {
    final SendPort sender;
    final T message;

    CrossIsolatesMessage({
        required this.sender,
        this.message,
    });
}

1.3. Step 3: destruction of the new Isolate

When you do no longer need the new Isolate instance, it is a good practice to release it, the following way:

//
// Routine to dispose an isolate
//
void dispose(){
    newIsolate?.kill(priority: Isolate.immediate);
    newIsolate = null;
}

1.4. Special note - Single-Listener Streams

You might have certainly noticed that we are using Streams to communicate between the "caller" and the new isolate. These Streams are of type: "Single-Listener" Streams.

2. One-shot computation

If you only need to run some piece of code to do some specific job and do not need to interact with that Isolate once the job is done, there exists a very convenient Helper, called compute.

This function:

• spawns an Isolate,
• runs a callback function on that isolate, passing it some data,
• returns the value, outcome the callback,
• and kills the Isolate at the end of the execution of the callback.

CONSTRAINTS

The "callback" function MUST be a top-level function and CANNOT be a closure or a method of a class (static or not).

3. IMPORTANT LIMITATION

At time of writing this article, it is important to note that

Platform-Channel communication ARE ONLY SUPPORTED by the main isolate. This main isolate corresponds to the one which is created when your application is launched.

In other words, Platform-Channel communication is not possible via instances of isolates you programmatically create...

There is however a work-around... Please refer to this link for a discussion on this topic.

When should I use Futures and Isolates?

Users will evaluate the quality of an application based on different factors, such as:

• features
• look
• user friendliness
• ...

Your application could meet all of these factors but if the user experiences lags in the course of some processing, it is most likely that this will go against you.

Therefore, here are some hints you should systematically take into consideration in your developments:

1. If pieces of codes MAY NOT be interrupted, use a normal synchronous process (one method or multiple methods that call each other);
2. If pieces of codes could run independently WITHOUT impacting the fluidity of the application, consider using the Event Loop via the use of Futures;
3. If heavy processing might take some time to complete and could potentially impact the fluidity of the application, consider using Isolates.

In other words, it is advisable to try to use as much as possible the notion of Futures (directly or indirectly via async methods) as the code of these Futures will be run as soon as the Event Loop has some time. This will give the user the feeling that things are being processed in parallel (while we now know it is not the case).

Another factor that could help you decide whether to use a Future or an Isolate is the average time it takes to run some code.

• If a method takes a couple of milliseconds => Future
• If a processing might take several hundreds of milliseconds => Isolate

Here are some good candidates for Isolates:

• JSON decoding

  decoding a JSON, result of an HttpRequest, might take some time => use compute

• encryption

  encryption might be very consuming => Isolate

• image processing

  processing an image (cropping, e.g.) does take some time to complete => Isolate

• load an image from the Web

  in this case, why not delegating this to an Isolate which will return the complete image, once fully loaded?

Conclusions

I think that understanding how the Event Loop works is essential.

It is also important to keep in mind that Flutter (Dart) is Single-Thread therefore, in order to please the users, developers must make sure that the application will run as smoothly as possible. Futures and Isolates are very powerful tools that can help you achieve this goal.

Stay tuned for new articles and meanwhile... I wish you a happy coding !