Official_Node.js -- Event

The Node.js Event Loop

The Event Loop is one of the most important aspects to understand about Node.js.

Why is this so important? Because it explains how Node.js can be 1️⃣ asynchronous and have 2️⃣ non-blocking I/O, and so it explains basically the “killer app” of Node.js, the thing that made it this successful.

The Node.js JavaScript code runs on a single thread. There is just 👨‍🎤one thing happening at a time.

This is a limitation that’s actually very helpful, as it simplifies a lot how you program without worrying about concurrency issues.

You just need to pay attention to how you write your code and avoid anything that could block the thread, like synchronous network calls or infinite loops.

In general, in most browsers there is an event loop for every browser tab, to 🚀1️⃣make every process isolated and 2️⃣avoid a web page with infinite loops or heavy processing to block your entire browser.

The environment manages 👪multiple concurrent event loops

for example, to handle API calls, Web Workers run in their own event loop as well.

You mainly need to be concerned that your code will ⭐️ run on a single event loop, and write code with this thing in mind to avoid blocking it.

Blocking the event loop

Any JavaScript code that takes too long to return back control to the event loop will block the execution of any JavaScript code in the page, even block the UI thread, and the user cannot click around, scroll the page, and so on.

Almost all the I/O primitives in JavaScript are non-blocking. Network requests, filesystem operations, and so on. Being blocking is the exception, and this is why JavaScript is based so much on callbacks, and more recently on promises and async/await.

The call stack

The call stack is a LIFO (Last In, First Out) stack.

The event loop continuously checks the call stack to see if there’s any function that needs to run.

While doing so, it adds any function call it finds to the call stack and executes each one in order.

You know the error stack trace you might be familiar with, in the debugger or in the browser console? The browser looks up the function names in the call stack to inform you which function originates the current call:

A simple event loop explanation

const bar = () => console.log('bar')

const baz = () => console.log('baz')

const foo = () => {
  console.log('foo')
  bar()
  baz()
}

foo()

The event loop on every iteration looks 🤔if there’s something in the call stack, and executes it

until the call stack is empty.

Queuing function execution

The above example looks normal, there’s nothing special about it: JavaScript finds things to execute, runs them in order.

Let’s see how to defer a function until the stack is clear.

The use case of setTimeout(() => {}, 0) is to call a function, but execute it once every other function in the code has executed.

const bar = () => console.log('bar')

const baz = () => console.log('baz')

const foo = () => {
    console.log('foo')
    setTimeout(bar, 0)
    baz()
}

foo()

The Message Queue

When setTimeout() is called, the Browser or Node.js starts the ⏲timer. Once the timer expires, the callback function is put in the Message Queue.

The Message Queue is also where

user-initiated events like click or keyboard events, or fetch responses are queued 在此排队

before your code has the opportunity to react to them. Or also DOM events like onLoad.

The loop gives priority to the call stack, and it first processes everything it finds in the call stack, and once there’s nothing in there, it goes to pick up things in the message queue.

We don’t have to wait for functions like setTimeout, fetch or other things to do their own work, because they are provided by the browser, and they live on their own threads. For example, if you set the setTimeout timeout to 2 seconds, you don’t have to wait 2 seconds - the wait happens elsewhere.

ES6 Job Queue

ECMAScript 2015 introduced the concept of the Job Queue, which is used by Promises (also introduced in ES6/ES2015). It’s a way to execute the result of an async function as soon as possible, rather than being put at the end of the call stack.

Promises that resolve before the current function ends will be executed right after the current function.

I find nice the analogy of a rollercoaster ride at an amusement park: the message queue puts you at the back of the queue, behind all the other people, where you will have to wait for your turn, while the job queue is the fastpass ticket that lets you take another ride right after you finished the previous one.

const bar = () => console.log('bar')

const baz = () => console.log('baz')

const foo = () => {
    console.log('foo')
    setTimeout(bar, 0)
    new Promise((resolve, reject) =>
                resolve('should be right after baz, before bar')
               ).then(resolve => console.log(resolve))
    baz()
}

foo()

foo
baz
should be right after baz, before bar
bar

That’s a big difference between Promises (and Async/await, which is built on promises) and plain old asynchronous functions through setTimeout() or other platform APIs.

Understanding process.nextTick()

As you try to understand the Node.js event loop, one important part of it is process.nextTick().

Every time the event loop takes (complete) a full trip, we call it a tick.

When we pass a function to process.nextTick(), we instruct the engine to invoke this function at the end of the current operation, before the next event loop tick starts:

process.nextTick(() => {
    //do something
})

The event loop is busy processing the current function code.

When this operation ends, the JS engine runs all the functions passed to nextTick calls during that operation.

It’s the way we can tell the JS engine to process a function asynchronously (after the current function), but as soon as possible, not queue it.

Calling setTimeout(() => {}, 0) will execute the function at the end of next tick, much later than when using nextTick() which prioritizes the call and executes it just before the beginning of the next tick.

Use nextTick() when you want to make sure that in the next event loop iteration that code is already executed.

Understanding setImmediate()

When you want to execute some piece of code asynchronously, but as soon as possible, one option is to use the setImmediate() function provided by Node.js:

setImmediate(() => {
    //run something
})

Any function passed as the setImmediate() argument is a callback that’s executed in the next iteration of the event loop.

How is setImmediate() different from setTimeout(() => {}, 0) (passing a 0ms timeout), and from process.nextTick()?

A function passed to process.nextTick() is going to be executed on the 👊 current iteration of the event loop, after the current operation ends. This means it will always execute before setTimeout and setImmediate.

A setTimeout() callback with a 0ms delay is very similar to setImmediate(). The execution order will depend on various factors, but they will be both run in the next iteration of the event loop.

Discover JavaScript Timers

setTimeout()

When writing JavaScript code, you might want to delay the execution of a function.

This is the job of setTimeout. You specify a callback function to execute later, and a value expressing how later you want it to run, in milliseconds:

setTimeout(() => {
  // runs after 2 seconds
}, 2000)

setTimeout(() => {
  // runs after 50 milliseconds
}, 50)

This syntax defines a new function. You can call whatever other function you want in there, or you can pass an existing function name, and a set of parameters:

const myFunction = (firstParam, secondParam) => {
  // do something
}

// runs after 2 seconds
setTimeout(myFunction, 2000, firstParam, secondParam)

setTimeout returns the timer id. This is generally not used, but you can store this id, and clear it if you want to delete this scheduled function execution:

const id = setTimeout(() => {
  // should run after 2 seconds
}, 2000)

// I changed my mind
clearTimeout(id)

Zero delay

If you specify the timeout delay to 0, the callback function will be executed as soon as possible, but after the current function execution:

setTimeout(() => {
  console.log('after ')
}, 0)

console.log(' before ')

will print before after.

This is especially useful to avoid blocking the CPU on intensive tasks and let other functions be executed while performing a heavy calculation, by queuing functions in the scheduler.

Some browsers (IE and Edge) implement a setImmediate() method that does this same exact functionality, but it’s not standard and unavailable on other browsers. But it’s a standard function in Node.js.

setInterval()

setInterval is a function similar to setTimeout, with a difference: instead of running the callback function once, it will run it forever, at the specific time interval you specify (in milliseconds):

setInterval(() => {
  // runs every 2 seconds
}, 2000)

The function above runs every 2 seconds unless you tell it to stop, using clearInterval, passing it the interval id that setInterval returned:

const id = setInterval(() => {
  // runs every 2 seconds
}, 2000)

clearInterval(id)

It’s common to call clearInterval inside the setInterval callback function, to let it auto-determine if it should run again or stop. For example this code runs something unless App.somethingIWait has the value arrived:

const interval = setInterval(() => {
  if (App.somethingIWait === 'arrived') {
    clearInterval(interval)
    return
  }
  // otherwise do things
}, 100)

console.log("first");

setImmediate(() => {
    //run something
    console.log("setImmediate");
})

setTimeout(() => {
    console.log('setTimeout_1000 ')
}, 1000)

setTimeout(() => {
    console.log('setTimeout_0 ')
}, 0)

process.nextTick(() => {
    //do something
    console.log("process nextTick()");
})

console.log('end of the current event loop ')

first
end of the current event loop
process nextTick()
setTimeout_0
setImmediate
setTimeout_1000

JavaScript Asynchronous Programming and Callbacks

Asynchronicity in Programming Languages

Computers are asynchronous by design.

Asynchronous means that things can happen independently of the main program flow.

In the current consumer computers, every program runs for a specific time slot and then it stops its execution to let another program continue their execution. This thing runs in a cycle so fast that it’s impossible to notice. We think our computers run many programs simultaneously, but this is an illusion (except on multiprocessor machines).

Programs internally use interrupts, a signal that’s emitted to the processor to gain the attention of the system.

I won’t go into the internals of this, but just keep in mind that it’s normal for programs to be asynchronous and halt their execution until they need attention, allowing the computer to execute other things in the meantime. When a program is waiting for a response from the network, it cannot halt the processor until the request finishes.

Normally, programming languages are synchronous and some provide a way to manage asynchronicity in the language or through libraries. C, Java, C#, PHP, Go, Ruby, Swift, and Python are all synchronous by default. Some of them handle async by using threads, spawning a new process.

JavaScript

JavaScript is synchronous by default and is single threaded. This means that code cannot create new threads and run in parallel.

Lines of code are executed in series, one after another, for example:

const a = 1
const b = 2
const c = a * b
console.log(c)
doSomething()

But JavaScript was born inside the browser, its main job, in the beginning, was to respond to user actions, like onClick, onMouseOver, onChange, onSubmit and so on.

How could it do this with a synchronous programming model?

The answer was in its 🌳 environment. The browser provides a way to do it by providing a set of APIs that can handle this kind of functionality.

More recently, Node.js introduced a non-blocking I/O environment to extend this concept to file access, network calls and so on.

Callbacks

You can’t know when a user is going to click a button. So, you define an event handler for the click event.

This event handler accepts a function, which will be called when the event is triggered:

document.getElementById('button').addEventListener('click', () => {
    //item clicked
})

This is the so-called callback.

A callback is a simple function that’s ⭐️passed as a value to another function, and will only be executed when the event happens.

We can do this because JavaScript has first-class functions, which can be 1️⃣assigned to variables and 2️⃣passed around to other functions (called higher-order functions)

It’s common to wrap all your client code in a load event listener on the window object, which runs the callback function only when the page is ready

window.addEventListener('load', () => {
  //window loaded
  //do what you want
})

Callbacks are used everywhere, not just in DOM events.

One common example is by using timers:

setTimeout(() => {
  // runs after 2 seconds
}, 2000)

XHR requests also accept a callback, in this example by assigning a function to a property that will be called when a particular event occurs (in this case, the state of the request changes):

const xhr = new XMLHttpRequest()
xhr.onreadystatechange = () => {
  if (xhr.readyState === 4) {
    xhr.status === 200 ? console.log(xhr.responseText) : console.error('error')
  }
}
xhr.open('GET', 'https://yoursite.com')
xhr.send()

Handling errors in callbacks

How do you handle errors with callbacks? One very common strategy is to use what Node.js adopted: the first parameter in any callback function is the error object: error-first callbacks

If there is no error, the object is null. If there is an error, it contains some description of the error and other information.

fs.readFile('/file.json', (err, data) => {
    if (err) {
        //handle error
        console.log(err)
        return
    }

    //no errors, process data
    console.log(data)
})

The problem with callbacks

Callbacks are great for simple cases!

However every callback adds a level of nesting, and when you have lots of callbacks, the code starts to be complicated very quickly:

window.addEventListener('load', () => {
  document.getElementById('button').addEventListener('click', () => {
    setTimeout(() => {
      items.forEach(item => {
        //your code here
      })
    }, 2000)
  })
})

This is just a simple 4-levels code, but I’ve seen much more levels of nesting and it’s not fun.

How do we solve this?

Alternatives to callbacks

Starting with ES6, JavaScript introduced several features that help us with asynchronous code that do not involve using callbacks: Promises (ES6) and Async/Await (ES2017).

Understanding JavaScript Promises

let done = false

const isItDoneYet = new Promise((resolve, reject) => {
    if (done) {
        const workDone = 'Here is the thing I built'
        resolve(workDone)
    } else {
        const why = 'Still working on something else'
        reject(why)
    }
})

const checkIfItsDone = () => {
    isItDoneYet
        .then(ok => {
        console.log(ok)
    })
        .catch(err => {
        console.log("l am error");
        console.error(err)
    })
}

checkIfItsDone()

A promise is commonly defined ⭐️ as a proxy for a value that will eventually become available.

Promises are one way to ⭐️deal with asynchronous code, without getting stuck in callback hell.

Promises have been part of the language for years (standardized and introduced in ES2015), and have recently become more integrated, with async and await in ES2017.

Async functions use promises behind the scenes, so understanding how promises work is fundamental to understanding how async and await work.

How promises work, in brief

Once a promise has been called, it will start in a pending state. This means that the calling function continues executing, while the promise is pending until it resolves, giving the calling function whatever data was being requested.

The created promise will eventually end in a resolved state, or in a rejected state, calling the respective callback functions (passed to then and catch) upon finishing.

Which JS APIs use promises?

In addition to your own code and libraries code, promises are used by standard modern Web APIs such as:

• the Battery API
• the Fetch API
• Service Workers

It’s unlikely that in modern JavaScript you’ll find yourself not using promises, so let’s start diving right into them.

---

Creating a promise

The Promise API exposes a Promise constructor, which you initialize using new Promise():

let done = true

const isItDoneYet = new Promise((resolve, reject) => {
  if (done) {
    const workDone = 'Here is the thing I built'
    resolve(workDone)
  } else {
    const why = 'Still working on something else'
    reject(why)
  }
})

As you can see, the promise checks the done global constant, and if that’s true, the promise goes to a resolved state (since the resolve callback was called); otherwise, the reject callback is executed, putting the promise in a rejected state. (If none of these functions is called in the execution path, the promise will remain in a pending state)

Using resolve and reject, we can communicate back to the caller what the resulting promise state was, and what to do with it. In the above case we just returned a string, but it could be an object, or null as well. Because we’ve created the promise in the above snippet, it has already started executing. This is important to understand what’s going on in the section Consuming a promise below.

A more common example you may come across is a technique called Promisifying. This technique is a way to be able to use a classic JavaScript function that takes a callback, and have it return a promise:

const fs = require('fs')

const getFile = (fileName) =>   
  return new Promise((resolve, reject) => {
    fs.readFile(fileName, (err, data) => {
      if (err) {
        reject(err)  // calling `reject` will cause the promise to fail with or without the error passed as an argument
        return        // and we don't want to go any further
      }
      resolve(data)
    })
  })
}

getFile('/etc/passwd')
.then(data => console.log(data))
.catch(err => console.error(err))

In recent versions of Node.js, you won’t have to do this manual conversion for a lot of the API. There is a promisifying function available in the util module that will do this for you, given that the function you’re promisifying has the correct signature.

---

Consuming a promise

In the last section, we introduced how a promise is created.

Now let’s see how the promise can be consumed or used.

const isItDoneYet = new Promise(/* ... as above ... */)
//...

const checkIfItsDone = () => {
  isItDoneYet
    .then(ok => {
      console.log(ok)
    })
    .catch(err => {
      console.error(err)
    })
}

Running checkIfItsDone() will specify functions to execute when the isItDoneYet promise resolves (in the then call) or rejects (in the catch call).

---

Chaining promises

A promise can be returned to another promise, creating a chain of promises.

A great example of chaining promises is the Fetch API, which we can use to get a resource and

queue a chain of promises to execute when the resource is fetched.

The Fetch API is a promise-based mechanism, and calling fetch() is equivalent to defining our own promise using ⭐️new Promise().

Example of chaining promises

const status = response => {
    if (response.status >= 200 && response.status < 300) {
        return Promise.resolve(response)
    }
    return Promise.reject(new Error(response.statusText))
}

const json = response => response.json()

fetch('/todos.json')
    .then(status)    // note that the `status` function is actually **called** here, and that it **returns a promise***
    .then(json)      // likewise, the only difference here is that the `json` function here returns a promise that resolves with `data`
    .then(data => {  // ... which is why `data` shows up here as the first parameter to the anonymous function
    console.log('Request succeeded with JSON response', data)
})
    .catch(error => {
    console.log('Request failed', error)
})

node-fetch is minimal code for window.fetch compatible API on Node.js runtime.

In this example, we call fetch() to get a list of TODO items from the todos.json file found in the domain root, and we create a chain of promises.

Running fetch() returns a response, which has many properties, and within those we reference:

• status, a numeric value representing the HTTP status code
• statusText, a status message, which is OK if the request succeeded

response also has a json() method, which returns a promise that will resolve with the content of the body processed and transformed into JSON.

So given those promises, this is what happens: the first promise in the chain is a function that we defined, called status(), that checks the response status and if it’s not a success response (between 200 and 299), it rejects the promise.

This operation will cause the promise chain to skip all the chained promises listed and will skip directly to the catch() statement at the bottom, logging the Request failed text along with the error message.

If that succeeds instead, it calls the json() function we defined. Since the previous promise, when successful, returned the response object, we get it as an input to the second promise.

In this case, we return the data JSON processed, so the third promise receives the JSON directly:

.then((data) => {
    console.log('Request succeeded with JSON response', data)
})

and we simply log it to the console.

---

Handling errors

In the example, in the previous section, we had a catch that was appended to the chain of promises.

When anything in the chain of promises fails and raises an error or rejects the promise, the control goes to the nearest catch() statement down the chain.

new Promise((resolve, reject) => {
    throw new Error('Error')
}).catch(err => {
    console.error(err)
})

// or

new Promise((resolve, reject) => {
    reject('Error')
}).catch(err => {
    console.error(err)
})

Cascading errors

If inside the catch() you raise an error, you can append a second catch() to handle it, and so on.

new Promise((resolve, reject) => {
  throw new Error('Error')
})
  .catch(err => {
    throw new Error('Error')
  })
  .catch(err => {
    console.error(err)
  })

---

Orchestrating promises

Promise.all()

If you need to synchronize different promises, Promise.all() helps you define a list of promises, and execute something when they are all resolved.

Example:

const f1 = fetch('/something.json')
const f2 = fetch('/something2.json')

Promise.all([f1, f2])
  .then(res => {
    console.log('Array of results', res)
  })
  .catch(err => {
    console.error(err)
  })

The ES2015 destructuring assignment syntax allows you to also do

Promise.all([f1, f2]).then(([res1, res2]) => {
  console.log('Results', res1, res2)
})

You are not limited to using fetch of course, any promise can be used in this fashion.

Promise.race()

Promise.race() runs when the first of the promises you pass to it resolves, and it runs the attached callback just once, with the result of the first promise resolved.

Example:

const first = new Promise((resolve, reject) => {
  setTimeout(resolve, 500, 'first')
})
const second = new Promise((resolve, reject) => {
  setTimeout(resolve, 100, 'second')
})

Promise.race([first, second]).then(result => {
  console.log(result) // second
})

Common errors

Uncaught TypeError: undefined is not a promise

If you get the Uncaught TypeError: undefined is not a promise error in the console, make sure you use new Promise() instead of just Promise()

UnhandledPromiseRejectionWarning

This means that a promise you called rejected, but there was no catch used to handle the error. Add a catch after the offending then to handle this properly.

Modern Asynchronous JavaScript with Async and Await

⭐️Introduction

JavaScript evolved in a very short time from callbacks to promises (ES2015), and since ES2017 asynchronous JavaScript is even simpler with the async/await syntax.

Async functions are a combination of promises and generators, and basically, they are a higher level abstraction over promises.

Let me repeat: ⭐️ async/await is built on promises.

Prepending the async keyword to any function means that the function will ⭐️return a promise.

await : get a Promise , convert to a value of return or exception

Why were async/await introduced?

They reduce the boilerplate around promises, and the “don’t break the chain” limitation of chaining promises.

When Promises were introduced in ES2015, they were meant to solve a problem with asynchronous code, and they did, but over the 2 years that separated ES2015 and ES2017, it was clear that promises could not be the ultimate solution.

Promises were introduced to solve the famous callback hell problem, but they introduced complexity on their own, and syntax complexity.

They were good primitives around which a better syntax could be exposed to the developers, so when the time was right we got async functions.

They make the code look like it’s synchronous, but it’s asynchronous and non-blocking behind the scenes.

How it works

An async function⭐️returns a promise, like in this example:

const doSomethingAsync = () => {
    return new Promise(resolve => {
        setTimeout(() => resolve('I did something'), 3000)
    })
}

When you want to call this function you prepend await, and the calling code will stop until the promise is resolved or rejected.

One caveat: the client function must be defined as async. Here’s an example:

const doSomething = async () => {
    console.log(await doSomethingAsync())
}

A quick example

This is a simple example of async/await used to run a function asynchronously:

const doSomethingAsync = () => {
    return new Promise(resolve => {
        setTimeout(() => resolve('I did something'), 3000)
    })
}

const doSomething = async () => {
    console.log(await doSomethingAsync())  
}

console.log('Before')
doSomething()
console.log('After')

Promise all the things

Prepending the async keyword to any function means that the function will return a promise.

Even if it’s not doing so explicitly, it will internally make it return a promise.

This is why this code is valid:

const aFunction = async () => {
  return 'test'
}

aFunction().then(alert) // This will alert 'test'

and it’s the same as:

const aFunction = () => {
  return Promise.resolve('test')
}

aFunction().then(alert) // This will alert 'test'

The code is much simpler to read

As you can see in the example above, our code looks very simple. Compare it to code using plain promises, with chaining and callback functions.

And this is a very simple example, the major benefits will arise when the code is much more complex.

For example here’s how you would get a JSON resource, and parse it, using promises:

const getFirstUserData = () => {
    return fetch('/users.json') // get users list
        .then(response => response.json()) // parse JSON
        .then(users => users[0]) // pick first user
        .then(user => fetch(`/users/${user.name}`)) // get user data
        .then(userResponse => userResponse.json()) // parse JSON
}

getFirstUserData()

And here is the same functionality provided using await/async:

const getFirstUserData = async () => {
    const response = await fetch('/users.json') // get users list
    const users = await response.json() // parse JSON
    const user = users[0] // pick first user
    const userResponse = await fetch(`/users/${user.name}`) // get user data
    const userData = await userResponse.json() // parse JSON
    return userData
}

getFirstUserData()

Multiple async functions in series

Async functions can be chained very easily, and the syntax is much more readable than with plain promises:

const promiseToDoSomething = () => {
    return new Promise(resolve => {
        setTimeout(() => resolve('I did something'), 10000)
    })
}

const watchOverSomeoneDoingSomething = async () => {
    const something = await promiseToDoSomething()
    return something + '\nand I watched'
}

const watchOverSomeoneWatchingSomeoneDoingSomething = async () => {
    const something = await watchOverSomeoneDoingSomething()
    return something + '\nand I watched as well'
}

watchOverSomeoneWatchingSomeoneDoingSomething().then(res => {
    console.log(res)
})

Easier debugging

Debugging promises is hard because the debugger will not step over asynchronous code.

Async/await makes this very easy because to the compiler it’s just like synchronous code.

The Node.js Event emitter

If you worked with JavaScript in the browser, you know how much of the interaction of the user is handled through events: mouse clicks, keyboard button presses, reacting to mouse movements, and so on.

On the backend side, Node.js offers us the option to build a similar system using the events module.

This module, in particular, offers the EventEmitter class, which we’ll use to handle our events.

You initialize that using

const EventEmitter = require('events')
const eventEmitter = new EventEmitter()

This object exposes, among many others, the on and emit methods.

• emit is used to trigger an event
• on is used to add a callback function that’s going to be executed when the event is triggered

For example, let’s create a start event, and as a matter of providing a sample, we react to that by just logging to the console:

eventEmitter.on('start', () => {
    console.log('started')
})

When we run

eventEmitter.emit('start')

the event handler function is triggered, and we get the console log.

You can pass arguments to the event handler by passing them as additional arguments to emit():

eventEmitter.on('start', number => {
    console.log(`started ${number}`)
})

eventEmitter.emit('start', 23)

Multiple arguments:

eventEmitter.on('start', (start, end) => {
  console.log(`started from ${start} to ${end}`)
})

eventEmitter.emit('start', 1, 100)

The EventEmitter object also exposes several other methods to interact with events, like

• once(): add a one-time listener
• removeListener() / off(): remove an event listener from an event
• removeAllListeners(): remove all listeners for an event

You can read all their details on the events module page at https://nodejs.org/api/events.html