How JavaScript works under the hood (Part 2)

Hi, welcome back.

If you are coming from part one of this guide you will find this part a breeze. But if not, I will strongly recommend you to check out Part 1 which serves as a base for this guide. But you can also read and gain something from this article alone. This guide is completely independent of the previous one.

So far we have learned in Part 1

• A little history of JavaScript
• JavaScript Runtime, which is basically a special environment provided by the browser or context we run our code in. This environment provides us with objects, APIs, and other components so our code can interact with the outside world and get executed.
• Components of runtime environment; like JavaScript engine, Web APIs, Callback queue, and Eventloop.
• JavaScript engine, which again consists of Callstack or Execution Stack and the Heap.
• Execution Context, which is basically a special environment for the execution of JavaScript code, contains the code that is currently running and everything that aids in its execution. This special environment is called the Execution context.
• Scoping and Temporal Dead Zone, we learned how functions can access declarations from their parent scope via Lexical Scoping. We briefly discussed TDZ as well.

In this article, we will be focussing more on the remaining three major components of JavaScript runtime in the context of the browser;

• Web APIs
• Callback queue
• The Eventloop

The drawback of JS single-threaded nature

As we all know JavaScript is single-threaded in nature, as it has only one heap and one stack. The next program has to sit and wait until the current program finishes execution, heap, and Callstack get cleared and the new program starts executing.

But what if, the currently executing task is taking so long, what if our current execution context is requesting some data from the server(slow server), definitely it would take some time. In this case, the Callstack queue will be stuck as JavaScript only executes one task at a time. All the execution contexts that are next to be executed will keep waiting until our current guilty Execution context is resolved. How we can handle this sort of behavior. How can we schedule some tasks, or park some expensive tasks so that we can keep our application going? How can we make our synchronous JavaScript asynchronous?

This is where Web APIs, Callback queue, and Eventloop come to the rescue.

Web APIs

First of all, we should know that Web APIs are not part of the JavaScript engine, but they are part of Web browsers. They provide our code with some extra features so our code can interact with the outside world. Modern web browsers have a lot of web APIs which we can use in our applications. We often use web APIs to carry out these noble purposes in our application:

• Dom Manipulation: Probably the most commonly used web API among JavaScript developers is the DOM API which enables us to manipulate DOM and CSS styles associated with DOM elements. It gives us the power to add nodes to DOM, remove them or apply styles to them dynamically.
• Graphics Manipulations: Web APIs like Canvas API and Web Graphics Library API let you draw and manipulate graphics in the canvas element. They let you programmatically add or modify graphics. The Canvas API provides a means for drawing graphics via JavaScript and the HTML <canvas> element. Among other things, it can be used for animation, game graphics, data visualization, photo manipulation, and real-time video processing. The Canvas API largely focuses on 2D graphics.
• Asynchronous JavaScript: As we already know originally JavaScript is single-threaded synchronous in nature. But we can make it asynchronous with Web APIs like XMLHttpRequest and its promise-based replacement, the Fetch API. We can enable our JavaScript code to carry out other vital tasks until a response from the server is returned. We will talk about asynchronous JavaScript more in a while. For now, let's just move forward.
• Work with Multimedia: Web APIs like Audio API and HTMLMediaElement provide us control over audio and video on the web. We can pause, play, forward our media or add certain effects to the stream.
• Work with Files: The File API enables web applications to access files and their contents. Web applications can access files when the user makes them available, either using a file <input> element or via drag and drop.

All of the event listeners, AJAX calls, and timer functionalities sit in the Web APIs container until an event is triggered.

Before deep diving into the Callback queue, we should first discuss Callback functions.

What are Callback functions?

The callback function is a normal function provided as an argument to another normal function, waiting to be called inside the recipient function at a suitable time.

Let me explain it to you. The callback is not a special kind of function. It's just a normal function that is passed to some other function, and that function handles the invocation (calling) of our callback function. See this Callback function supplied to click the eventlistner;

As you can see we are providing a callback function to the listener. This function is not called right away but when a click happens. So the callback function is sitting and waiting somewhere in the JavaScript engine for its trigger. When a user interacts and clicks, this function gets called. So the place where this function is sitting and waiting is the Callback queue.

At first callback functions were used to handle asynchronous tasks in JavaScript. They did pretty well but handling more complex became like a hell. Because when a callback function is also an asynchronous function and receives another callback, syntax became really messy which we call The Callback Hell these days.

Well, that is not our topic for today, you can check out callbacks here.

The Callback Queue

The callback is a data structure that holds callback functions sent by Web APIs. Callback functions sit here until the Callstack is empty, and the Eventloop transfers them to Callstack.

As we all know JavaScript is a single-threaded synchronous language natively, and It runs your pieces of code sequentially. Line after line. If Line 13 is taking longer than expected, our program would freeze and would not attempt to execute Line 14 until 13 is completed. This is the problem that the Callback queue solves. Have a look at this setTimeout Web API:

setTimeout Web API executes a provided callback function after specified time. Learn more about of setTimeout here.

If we consider JavaScript as a single-threaded, synchronous programming language what output we would expect from this above code:

Expected

First of all, 1 will be logged to the console as a result of console.log(1).

Then, after waiting for 1 second (1000ms), the callback function will be executed logging 2 to the console.

And finally, the program will end after logging 3 to the console as a result of console.log(3)

Output

First of 1 logs as expected.

Unexpectedly, 3 logs to the console next. Apparently, the JavaScript engine skipped setTimeout and moved on to the third statement console.log(3).

Then after 1 second 2 logs to the console.

Explanation

Let me explain this behavior; at first, JavaScript encountered the console.log() function and stored it in the Execution stack or Callstack for execution. Functional Execution Context(FEC) gets created to execute this special built-in function(method), and it gets executed and fades out of the Callstack.

Secondly, as a second statement, the JavaScript engine encounters setTimeout Web API. As we all know, setTimeout is not a feature of JavaScript itself. First of all, setTimeout is added to Callstack. But it is not executed here, being a web API, it creates a task inside the browser and pops out of the Callstack.

JavaScript engine keeps executing code, again encounters console.log(3) function and creates Functional Execution Context to handle it. Very similar to statement 1 console.log(1) it gets executed and as a result, pops out of the main Execution stack.

After the specified time in setTimeout API, which in our case is 1000ms, the browser sends this task(our callback function) to the callback queue.

At this point Execution Context is empty, and our callback function is sitting in the Callback queue waiting for the opportunity of execution. Here a very important question arises, What next, and What connects the callback queue and the Callstack? You guessed it right, the event loop.

The Eventloop

The event loop is a special mechanism in JavaScript which keeps an eye on the Callstack and the Callback Queue. Its main purpose is to transport callback functions from the Callback queue to the Callstack for execution, once it is empty.

The Callback queue works in a FIFO(first in, first out) manner. Mean the oldest callback will be sent to the Callstack first. You can imagine the Callback queue simply as an array, which adds new callbacks with Array.push() method to the end of the queue and takes the first callback out using Array.shift() method.

The event loop takes callbacks from the callback queue one by one and keeps doing this until the Callback queue is also empty.

As soon the Callstack was empty, the event loop looked in the Callback queue for any pending callback function. But will not find one. Probably you know the reason. The rest of our program will a be executed in few milliseconds but the browser sends our callback function(task) to callback queue after a specified time. 
So what next? Did our program fail here? Of course not. As clear from the name, the event loop is a loop and keeps checking again and again.

After 1000ms our callback(task) is sent to the callback queue by the browser. At this point in time, the Callstack is already empty, and immediately event loop transfers our task to the Callstack, where it gets executed similar to any other program.

Hopefully, most of the concepts will be clear in your mind by now. If you still have some confusion, let's take a couple of more examples.

This program is exactly the same as the previous one with only one change, Delay time is set to 0 for setTimeout. Can you guess what would be the output of this code?

Well if you guessed 1 2 3, this is not the correct answer. Let me explain how?

The first statement console.log(1) will be executed exactly as we have already discussed.

The twist comes in the second statement, setTimeout. We have set a 0ms delay time, and we expect it to run immediately. And then 3rd statement should be executed. But this is not the case here.

When setTimeout is encountered and added to Callstack, and being a web API, it adds a task to the browser and fades out of execution context. As we haven't added any delay, immediately the browser sends our task to the callback queue. 

At the very moment, the JavaScript engine is executing the 3rd statement of our program console.log(3). As Callbacks are only transported to the main Callstack after it is empty. Our callback has to sit and wait inside the Callback queue until our main Callstack is empty.

As soon FEC of console.log(3) finishes execution, the Callstack is empty, The event loop comes into play, checks the Callback queue for any pending scheduled tasks, and guess what finds one. The eventloop transfers our callback to the Callstack and it gets executed.

So this way output is 1 3 2 not 1 2 3. Reason is the fact that the eventloop only transfers callbacks from the Callback queue to Callstack after it is empty.

Challenge for you

I hope your concepts regarding Callback queue, Web APIs, and eventloop are clear now. So I have a little to do for you. Consider this code and predict the output. Sounds good?

Take your time, make notes let me know your first assessment in the comment section.

Let's do it together as well.

Step1

console.log(1) is added to the main Calllstack and executed. Pops out of the execution stack.

Step2

setTimeout no 1 is added to the Callstack, it adds a task to the browser and pops out of the stack.

At this stage, the browser container has 1 task(setTimeout 1), with a timer of 1000ms.

Step3

setTimeout no 2 is added to the Callstack next, and adds a task to the browser and pops out of the stack.

At this stage, the browser container has 2 tasks, task1(setTimeout1) and task2(setTimeout2) with a timer of 1000ms and 0ms respectively.

Here is the twist. We know that task2 was added after task1, but task2 has no delay at all 0ms. So task2 will be sent to the callback queue before the task1.

Let's update our state, at this point we have only 1 task left in the browser container(task1), and we have one callback function(task2) in the callback queue. So far only 1 statement was executed console.log(1).

Step4

Then comes the 3rd setTimeout API. It is added to the Callstack, it adds a task to the browser. This task3 has a 500ms delay. This means it will be sent to the Callback queue after half a second.

So far, we have 1 callback function(task2) sitting in the Callback queue, and 2 tasks, task1, and task3 waiting in the browser container with a delay of 1000ms and 500ms respectively.

Step5

Statement 5 in the main script is executed immediately and popped out of the Callstack.

Now we have executed 2 statements, 2 tasks are waiting in the browser container, and 1 task sitting in the callback queue.

Step6

Here comes out 4th and final setTimeout. It has a delay set to 250ms. It is added to the Callstack and pops out after adding another task to the browser. At this point in time, we have 3 tasks in the browser container;

• Task1: Added by setTimeout1 with a timer of 1000ms.
• Task3: Added by setTimeout1 with a timer of 500ms.
• Task4: Added by setTimeout1 with a timer of 250ms.

As we know the task is sent to the callback queue after the time specified. So task 4 will go first, followed by task3, and task 1 will be added to the callback queue at the last.

State of Callback queue

As our Callstack is empty at this point, the Eventloop will start transferring callbacks to Callstack 1 by 1. This will happen first in, first out manner. Mean the oldest callback function will be sent to Callstack first.

So, first of all, task 2 will be transferred to the Callstack. It will be executed by logging 3 to the console.

Then, task 4 is transferred resulting in the logging of 6 to the console.

Then task 3 and at last task 1 logging 4 and 2 to the console respectively.

So do we get, 1 5 3 6 4 2. This is the magic key. Hopefully, you got the same.

Credits and Motivations

• Elegant guide on JavaScript Runtime Environment by Gemma Croad.

Wrapping Up
So this was part 2 of JavaScript under the hood. Hopefully, you got some value from this guide, and if you did please let me know in the comment section. This will give me another level of motivation, so I keep writing articles like this.

Till then, Stay safe, and try to keep others safe.

See you soon💓