Node.js Async Tutorial

I’ve been doing a lot of backend development in Node.js recently. Node.js runs on a single threaded event loop and leverages asynchronous calls for doing various things, like I/O operations. While other languages will send a database query and wait there for the result to come back, Node.js will not. When you send a database query, Node.js will continue executing the code that comes after it, then jump back when the result is available.

This is a powerful concept that enables gains in efficiency, but occasionally requires a bit more work on your end to deal with certain situations. One of those situations, which I’ve run into quite frequently, is the need to wait for a number of asynchronous operations to finish before executing additional code.

For example, maybe you have an array of items that you want to save to your database. Once they’re all saved, you want to execute a function that calculates some stats.

Your first thought might be to do something like this:

// Loop through some items
items.forEach(function (item) {
  // Call asynchronous function, often a save() to DB
  item.someAsyncCall()
})

// At this point, we've fired a bunch of async calls
// but they're probably not all done executing yet

// This function is meant to be executed once all the async
// calls above are done, but we don't know if/when they are,
// and therein lies the problem with this approach
doSomethingOnceAllAreDone()

As you can see in the comments above, there’s an issue here. You may (and probably will) execute doSomethingOnceAllAreDone() before everything above it is actually done.

One of the best solutions to this problem is to use the node-async package which includes a number of functions for dealing with situations like this. I’m going to show you how to resolve this issue using two different node-async features.

async.each()

First, we’ll look at the async.each() function. This is the simpler solution to the problem. The function takes an array of items, then iterates over them calling a wrapper function which accepts the item as an argument. When all the calls are complete, you specify a final function to be called.

// Include the async package
// Make sure you add "async" to your package.json
async = require('async')

// 1st para in async.each() is the array of items
async.each(
  items,
  // 2nd param is the function that each item is passed to
  function (item, callback) {
    // Call an asynchronous function, often a save() to DB
    item.someAsyncCall(function () {
      // Async call is done, alert via callback
      callback()
    })
  },
  // 3rd param is the function to call when everything's done
  function (err) {
    // All tasks are done now
    doSomethingOnceAllAreDone()
  }
)

async.parallel()

The solution above works well if you simply need to iterate over a collection, but what if we have a more complex situation? Rather than iterating over a collection, async.parallel() allows you to push a bunch of (potentially unrelated) asynchronous calls into an array. Once we have the array populated, we execute all the tasks inside it, then call a function when we’re done.

// Include the async package
// Make sure you add "async" to your package.json
async = require('async')

// Array to hold async tasks
var asyncTasks = []

// Loop through some items
items.forEach(function (item) {
  // We don't actually execute the async action here
  // We add a function containing it to an array of "tasks"
  asyncTasks.push(function (callback) {
    // Call an async function, often a save() to DB
    item.someAsyncCall(function () {
      // Async call is done, alert via callback
      callback()
    })
  })
})

// At this point, nothing has been executed.
// We just pushed all the async tasks into an array.

// To move beyond the iteration example, let's add
// another (different) async task for proof of concept
asyncTasks.push(function (callback) {
  // Set a timeout for 3 seconds
  setTimeout(function () {
    // It's been 3 seconds, alert via callback
    callback()
  }, 3000)
})

// Now we have an array of functions doing async tasks
// Execute all async tasks in the asyncTasks array
async.parallel(asyncTasks, function () {
  // All tasks are done now
  doSomethingOnceAllAreDone()
})

Conclusion

Asynchronous functionality is available in lots of programming languages, not just Javascript. If you’re new to the concept, it can take a little time to wrap your head around the challenges. Luckily for Node.js developers, the node-async package provides a ton of useful features for dealing with those challenges. The examples above solve some of the simpler situations, but only scratch the surface. Check out the node-async repo for more code and examples.

I’ve uploaded full (executable) code for the examples in this post to GitHub.