Leveraging the Event Loop for Blazing-Fast Applications! [eng]

Hi Cristina, thank you for the introduction. So hello, everyone, and welcome to my talk, as mentioned, "Leveraging the Event Loop for Blazing Fast Applications." I'm Michael DiPrisco, known online as Cadiman. I'm from Italy, currently a senior backend developer at Jointly. This is the only good photo I'll ever use in my lifetime, so thanks to my wedding photographer for it. I'm also a LogRocket Content Advisory Board member, and I like to call myself a waster of NPM storage because I'm an open source contributor, mainly in the JavaScript field (NPM), and a SQL pool requester. I'm not the focus of this talk, so let's go ahead and start our journey, as we have a lot to cover.

Let's kick off with a really bad joke, but someone had to make it in a JavaScript conference. I promise you'll learn something, and I'm sacrificing my credibility for just this joke. At least we'll try to learn something today. So let's dive into our table of contents. What are we going to talk about today? First things first, what is the event loop? We'll briefly touch on the event loop in JavaScript. Our second topic will be the Microtask queue. Specifically, we'll take a deep dive into the Microtask queue. The third part will be a live demo where we'll apply what we've learned to a simple application.

I don't have both the expertise and the time to extensively discuss the event loop and connect all the dots about how it works. So please excuse me for taking shortcuts and abstracting many concepts. Today, we'll focus on how the event loop works at a high level and mainly concentrate on the usage of the Microtask queue to provide performance improvements in our web applications. If you want to know more about the event loop and how it works in different runtimes (more about runtimes in a couple of minutes), please search for the awesome presentation by Jake Archibald in "the loop" or the incredible "create your own JavaScript runtime" by Eric Wendel.

Now, let's start with a couple of premises. Going back one step before starting our journey, the first question we need to answer is: Is JavaScript single-threaded? Yes and no, but yes. There has been a lot of debate over the years about JavaScript being single or multi-threaded and how this language works. Yet the answer still is yes, JavaScript is single-threaded. Web workers and service workers in browsers or child processes and forks in Node and Deno are APIs provided by those runtimes. However, in the ECMAScript specification, let's say the set of rules governing the language, doesn't allow for multi-threading, yet it can allow you to execute non-blocking operations even in scenarios where multi-threading would be the only answer. Now, onto our second question. As we're going to talk about a specific runtime, let's understand what a runtime essentially is.

Being a non-compiled language, JavaScript runs in some sort of container and a program that reads and runs code. The container must mainly do two things. First, parse, convert, and execute your code. Second, provide some objects, prototypes, functions, and APIs for your code to interact with. The first part, involved in parsing, converting, and executing your code, is called the engine. The latter, providing you the APIs and some additional functionalities, is the runtime.

Let's discuss the most famous duo of runtimes in the JavaScript world: browsers and Node.js. We'll primarily focus on the Chrome browser to simplify things. Both the Chrome browser and the Node runtime share the same engine, V8, yet their runtimes differ significantly. Node.js provides structures such as streams, buffers, libraries like FS or PATH, and APIs like the file system ones and the require method, for example. On the other hand, the browser provides DOM objects, the window object, web APIs, etc. So there are different APIs and functionalities provided.

Take console.log as an example. What if I told you that V8, the engine responsible for both the Chrome implementation and the Node.js environment, doesn't know what console.log is? That's correct. It's part of the web API in the case of a browser runtime like Chrome and a different implementation in the Node.js source code. For Node.js, you can find a nice implementation of the console in the official library at github.com/Node.js/lib/internal/console.constructor.js.

Each runtime implements its flavor of the event loop, usually leveraging key concepts we'll discuss shortly. This talk will be based on the browser runtime for ease of demonstration. The browser implementation differs from the Node one due to how some queues in the event loop interact with the loop itself. In Node, oversimplifying it, we have one event loop per process, while in the browser, we have one per agent. An agent, in this context, refers to a Chrome tab. If a tab blocks at the event loop, you can still close that tab and continue navigating in other tabs. Blocking the main thread of one tab doesn't affect the other tabs. Now, considering how many runtimes are there, we mentioned Chrome and Node, but there are many others. As I like to say, at least one more than when I started talking. The most common ones are Node.js, Dino, BAN, WorkerD from Cloudflare, and browsers' own implementations.

Now, let's tackle our main question. Now that we know what a runtime is and that JavaScript is single-threaded, let's understand how the event loop allows non-blocking input-output, ensuring a non-blocking main thread while still leveraging functionalities that might typically be part of a multi-threaded language. Credit to Lydia Hally for creating an empowering gif explaining how the microtask queue and the macro task queue work and their order of execution. As you can see, we have the call stack, the microtask queue (emptied every time a task is being executed), and the macro task following later. The microtask queue is usually filled with things like process.nextTick, promise callbacks, and async functions, with queueMicrotask being our main topic.

As for the macro task queue, we have mainly setTimeout, setInterval, and setImmediate. So what's the difference between the microtask queue and the macro task queue? It's not just the order in which tasks are executed, and we'll delve into this later. Consider the event loop's main parts: the heap or memory heap, storing objects; the stack or call stack, an ordered list of operations to be executed; and queues, mainly the micro and macro queues, which hold callbacks waiting for execution.

Let's simplify it: queues vary between runtimes but are usually filled with timeouts, asynchronous calls, and event lists. These are tasks that are not immediately required to be executed at that exact moment, so they don't have to be placed directly in the call stack. Now, let's focus on three crucial concepts of the event loop, as they'll be essential for the live demo later. The first concept is run-to-completion operations. Run-to-completion means that every portion of our code, such as an execution block or a function, is executed till its end before another element of the call stack is executed. You might ask about concurrency in JavaScript, as it's often mentioned. However, be aware that JavaScript has no way of executing concurrent code on the main thread. This is crucial. The main thread can do something similar thanks to the worker implementation.

As mentioned earlier, while JavaScript can spawn processes and use different threads (as seen in the case of workers), the main thread executing the primary block of your code, which is responsible for keeping your application in line and your script alive, is not concurrent in any way. Asynchronous operations do not imply concurrency; they are two distinct concepts. This distinction is a key feature of JavaScript compared to many other languages. The event loop's implementation allows JavaScript to be single-threaded but non-blocking. However, it cannot be concurrent, and queues play a crucial role in freeing up the thread.

Now, let's dive deeper into the concept that "timeout isn't guaranteed." When you set a timeout, it only guarantees that the callback will be executed at least after the specified duration. However, it could take longer, especially if the main thread is blocked. So, if precise timing is crucial, it's better to use timestamp checks instead of relying solely on timeout calls. A timestamp check involves some part of your application checking an expiration timestamp before executing a callback. Returning to the primary concept of the event loop, which is to maintain a non-blocked main thread, let's explore an example where we intentionally attempt to block it. The demonstration involves a GIF with two buttons: one for an I.O. blocking function and another for a non-I.O. blocking function. The I.O. blocking function triggers an infinite loop, while the non-I.O. blocking function incorporates a setTimeout of zero milliseconds but still calls the infinite loop function. The results show that the non-I.O. blocking function allows the UI to remain responsive, whereas the I.O. blocking function freezes everything.

Examining the blocking code execution, the I.O. blocking function contains an infinite loop within a while loop. Despite the loop being seemingly empty, it still contributes to the stack, leading to the blockage of the event loop. On the other hand, the non-blocking function uses a setTimeout with zero milliseconds, scheduling the infinite loop in the future. This ensures that the event loop can progress to the next tick, preventing the blockage of the main thread even though the loop never ends. It illustrates a non-I.O. blocking operation. A notable point regarding the macro task queue in the event loop is that it is often referred to as a set, denoting a different data structure than a simple queue. The reason for this terminology is not explicitly mentioned.

Because, as we just mentioned, the event loop grabs the first runnable task in a set instead of dequeuing the first task in a list. So, it's a little different. It's really a simple difference, but it's worth mentioning. And this is the macro task queue. But what about the micro task queue? So, what is the micro task queue? After all this talk about the event loop and its queues, or as we learn, its sets, let's talk about the micro task queue, which is, well, a queue and not a set, unlike what we said before about the usual tasks being implemented, the usual queues being implemented in an event loop. This is because the micro task queue can only be filled with the runnable tasks provided by us, the developers interacting with the event loop. So, they are runnable by definition right when we put them into the queue. So, there's no need to take the first runnable one, as they are all effectively in the micro task queue. The micro task queue acts in a specific part of the event loop and is probably one of the most misunderstood parts of the whole event loop thing because this queue effectively acts as soon as the call stack is empty. So, in the context of a browser, it usually happens right before the event loop tick is completed, and it is almost starting to re-render.

I say "usually" because there are certain specific cases when this queue loses some priority. But this is not the topic of this talk. So, let's say, in an unusual context, it actually leverages the moment between the call stack emptying and the rendering time, effectively acting as a last step before starting the page rendering phase. So, after all this talk, what are we building today? So, I promised you a live demo. This will be our third part. We have four steps to victory. So, the first is project scaffolding. What do we mean by project scaffolding? A simple HTML page with nothing particular inside it, with a simple title and a couple of things inside it.

A basic signal implementation. So, let's take a minute to talk about what I personally mean as a signal. So, what are we going to build today? And a couple of things we will discover. We will leverage JS classes or ECMAScript classes, as we want to say, even if they are just mainly syntactic sugar, because they are simpler, of course. Everyone not aware of the prototype pattern can still follow this implementation without having questions. And a signal is, in this case, mainly one-way data binding from JavaScript to the HTML. So, we will just provide a class in which the objects can attach to a DOM element. And after attaching to that DOM element, can leverage some JavaScript functionalities to change effectively the HTML markup. So, the third step will be a bare-bone benchmark. So, a simple benchmark where we will track the time needed for 100,000 updates, one million updates. We will probably go for a million just to have some wow effect in the end.

And for... it's a secret. No, I'm just kidding. The whole talk is about what we do here. So, we will implement the microtask queue, and we will try to understand how much of a difference it can make. So, making our application effectively blazing fast. So, if you want to see some code, follow me. So, we have a simple VS Code implementation now. So, nothing hard here. We will create an index.html file, simple enough. And we will go with the help of Emmet for a simple HTML5 page. My signal implementation. We'll have a GitHub repository later. So, please don't worry if you can't follow the code line by line because there will be a different implementation provided later. So, let's start by creating a simple paragraph. So, let's put an id on it and let's call it the main paragraph. And let's, with the help of Emmet, of course, let's do some lorem ipsum. OK, now let's create a script tag.

I know it's not the best way to do this, but please, I want to keep it simple. So, let's go by creating our class, which will be called signal. OK, and now let's start by putting a couple of things. So, an initial element, so a DOM element attached to it and a value. So, I know we can leverage private properties and private methods for classes, but I wanted to keep it simple. So, you will not look at something like this. We will just simply prepend an underscore for our, let's say, private properties. So, let's go for the constructor. And we will, of course, have an element and a value being passed to our constructor and we will put it inside. OK, this.dot.l equals l and this.dot.value equals value. OK, so here we are. Now our simple signal is doing mainly nothing, but we can start instructing it and creating it. So, signal equals to new signal. I don't trust copilot enough, so I will write it by myself, and we have plenty of time. So, let's try to use it. Main paragraph. OK, here we are.

And then we will pass a value. Let's say, hello world. Of course, yeah, we don't need this because we are going to effectively control in a one-way data binding way the content of our main paragraph. So, let's remove it. Then we will need a rendering function. Of course, because we need to bring in the inner HTML of this element. Of course, I'm keeping it simple. As I said before, later I will provide some slides, a slide with a QR code you can follow to look at a different implementation, but we will talk about it later. So, please try to keep it simple with me. OK, so our rendering function just brings the inner HTML following the value being passed. OK, the internal value. So, let's call this.dot.render in our constructor. Now that we have all this scaffolding ready, let's go live. I will move the Chrome browser below. OK. I hope you can see it.

I will zoom it in a lot. OK, here we are. So we have a simple implementation. Now we have our "hello world" being printed on the console. Of course, I cannot leverage this console to change my text. So let's say ABC, and then we can call the render function. But of course, we want to make it faster. So we will provide some magic inside this implementation. So let's start by providing an API to the users using, to the other developers using our signal. Let's provide a magic getter for our values, which will return this.dot.underscoreValue, and a magic setter with a val being passed. So this.dot.value equals val and this.dot.render. OK, so what changed now?

Let's take a minute to talk about it. So our effect will be this one. So I can change the value. Let's see what we are missing here. `this.dot.value = val`. OK, `this.dot.render`. OK, seems good. `Signal.value = ABC`. OK, you can see that our value is changing and our DOM is being updated. So our DOM node element is attached, and we can control it with our signal. We are using some JavaScript magic, some object-oriented programming magic. Of course, as I said, a prototype implementation would be better. We will talk about it later. But we have the scaffolding needed for our application. So now let's move to our third step. So we have the scaffolding. We have our basic signal implementation.

And now we can move to the part where we do some benchmarking. So let's try to... OK, `const start = new Date().getTime`. Of course, we can just do this. In a Node.js environment, we could use a better and more accurate API, which is the performance one. But as always, we are keeping it simple. So let's go with the `const start = Date.now()`. Then we will just try to go for 100,000 updates. And of course, we are just moving our signal value. We are just changing it, calculating our end time. And then, let's say, `timeTaken = end - start` milliseconds. OK, so let's save it. `Date.getTime` is not a function. Yeah, sure, because it's not... Yeah, we don't need `Date.getTime`, but we need `Date.now()`. That's what we need. So let's do it also for the end, `Date.now()`. And here we go. OK, `timeTaken`, 135 milliseconds. OK, so now let's try to clutter this up a little. OK, so maybe create our... Let's say paragraph one, paragraph two and paragraph three.

So let's go with our `signal1`, `paragraph1`, `2` and `3`. So we are just trying to bring in some latency, some work for our main thread to do. OK, so that we can better leverage the DMAker task queue later. So let's say we are moving to this approach. OK, `hello world1`, `hello world2`, and `hello world3`. OK, let's go. `timeTaken`, 394, circa 400 milliseconds. So let's improve this number. So let's say we are doing a million operations, three times. So three million operations. As you can see, my input is... My main thread is blocked. So if I try to update it again, I cannot copy and paste anything inside the page until it's done. So now that it is done, you can see the cursor changing. I will do it again. So as you can see, everything is blocked on my site. And then I can.

OK, so we have four seconds effectively where our event loop is just doing its job and blocking our main thread. So we need to fix this, of course, because we cannot wait four seconds for it to render everything. OK, so let's put something here. So let's go back to our signal implementation and let's move our implementation from an immediately rendered mechanism to a queue mechanism. So is it queued? So this will be a flag, of course. `this.dot.queue = false`. And as soon as we now try to move the new value, we will effectively keep updating our internal value because, for example, and you will see this later, we can have some event listeners, for example, that have to be triggered when the value changes. Yet our rendering phase shouldn't be bored with three million operations every time. OK, so if this is not queued, `this.dot.queue = true`. And then `window.queueMicrotask`.

And we can call a function inside here. So `this.dot.queued = false`. `this.dot.render()`. So before looking at the result of this operation, of course, we have to remove `this.dot.render` from here. Let's talk about what we did here. As we said, let's zoom in a little. OK, so we have added a `queued` property, putting it to false, of course, in our constructor. Rendering, of course, this didn't change because the render phase is always the same. So we have an inner HTML in `this.dot.value`. So that's it. But we changed our setter. So, of course, we just say, have we already queued this operation for our next rendering phase? If so, don't do anything. We could just do it in a different way. So it does, if `this.dot.queued`, continue. But we don't care. OK, so if this is not queued, let's put it in our queue. So let's bring this flag on true and then leverage the `window.queueMicrotask`.

So this is exactly the API we need. And this is the only thing we are looking for. So a simple function being executed when our `queueMicrotask` will bring our item in. This will be effectively our callback. So we put it again into a non-queued state and then we will render. So are we ready to look at the difference? As we said before, let's go back to our implementation so we can have a look at how this worked before. OK, let's re-render this one. 3.7 seconds. Let's do it again. So this is mainly the average, 3 and X. 3.8. So now let's go to our implementation. And if I didn't do anything wrong, 61 milliseconds. So we brought 50X improvement in our simple application. Of course, you could ask, why should I do 3 million operations inside a single rendering? But that's not the question you should do. You should ask yourself, in a simple life cycle of my applications, how many times do I need to re-render something on my screen?

So how many times will this render function be effectively called? If you look at a simple active application, with some animations, with some interactions with our user, of course, you can see there are dozens of updates being done. Of course, having a 50X improvement in performances is really awesome when these dozens of updates become hundreds, thousands, and maybe millions. Because if your application is really high in animations and interactions, you can easily reach a million interactions. So a million updates using this signal implementation. This is something... And of course, remember that everything that allows you to improve performances without any disadvantage should be done. So this is a rule we all should try to follow.

So we saw some code, and now we can say, wow. So if what I said earlier was well explained and this result is what many of you might have expected. If not, please, as I said, go to the... Look at the Archibald and Eric Wender's job. So our last question is, why don't we always use the Microtask EQ? So as we said, this is a 50X improvement in performances. This is awesome. It's easy to implement, and we can do it right away in every application. OK, so why aren't we effectively doing this? Well, all that glitters is not gold. So despite the enormous improvements in performances we can have by leveraging the Microtask EQ, it doesn't mean it's always the best solution.

As we said earlier, this EQ effectively acts before the event loop starts a new tick. OK, so effectively acting as a last step. So if you bring in too many tasks, we will still find ourselves blocking the main thread. What I mean by that, so let's try to look at our safe loop implementation and try to change the behavior by not doing a `setTimeout`, but a `Promise.resolve`. `Promise.resolve` means resolve it immediately. So put it inside the Microtask EQ. And then call the `safeLoop` function. I need a glass of water, so please excuse me. OK, here we are. So `Promise.resolve` effectively puts it into the Microtask EQ and then it executes it immediately. So as soon as our call stack is empty, we call the Microtask EQ. So we say the event loop asks the Microtask EQ, do you have anything to bring into my stack? Yeah, of course, I have the `safeLoop` function. The `safeLoop` function is executed, puts another execution of the `safeLoop` function, the callback, inside the Microtask EQ and then empties itself. Now the event loop asks the same question. OK, so I'm done. Can I do a next tick or is there something inside the Microtask EQ? Yeah, there is this `safeLoop`. So we continuously clutter our main thread just by doing one single operation. So we will effectively create an infinite loop.

OK, so as we started with a bad joke, I thought it would be nice to end with an even worse one. So let's say the event loop is the only infinite loop you will ever want in your app. So please be aware of one thing. We will go back a little because we need to do this. Remember, even if in this GIF, this really awesome GIF, the Microtask EQ acts effectively right before the Macrotask EQ, this is not exactly the correct order of execution because one is inside the current tick of the event loop and the other one, the Macrotask EQ, is effectively one of the first things being done after the tick of the event loop.

So effectively, using `setInterval`, `setTimeout`, and `setImmediate`, we are not creating a block in our main thread. And of course, I can agree with you that `process.nextTick` and `setImmediate` are the worst names for these kinds of functions because effectively the `process.nextTick` acts in the current tick of the event loop and the `setImmediate` isn't immediate but acts in the next tick of the event loop. Yet this is the specification, so we have to stick with it. So let's go. We have to repeat our animations all over.

So I hope it's all clear and why you should leverage the Macrotask EQ and the Microtask EQ. And if you have some questions, of course, please provide them. But first, if you want to see a complete implementation of a signal leveraging the Microtask EQ, please find it following this QR code or search Super Simple Signal on my GitHub profile. Remember, I am Cadiman, so C-A-D-I-E-N-V-A-N, and you can find the Super Simple Signal implementation, which doesn't leverage the class implementation, but a different one, a prototype-based one. So, ready for your questions.