Detailed explanation of asynchronous iterators in nodejs

Detailed explanation of asynchronous iterators in nodejs

Preface

Asynchronous iterators have been around since Node.js v10.0.0, and they have been gaining traction in the community lately. In this article, we will discuss the role of asynchronous iterators and also address the question of what they might be used for.

What are asynchronous iterators?

So what are asynchronous iterators? They are effectively asynchronous versions of the iterators that were available previously. When we don't know the value and final status of the iteration, we can use asynchronous iterators, and finally we get a promise that can be resolved to a {value:any, done:boolean} object. We also got a for-await-of loop to help us loop over asynchronous iterators. Just like a for-of loop is for synchronous iterators.

const asyncIterable = [1, 2, 3];
asyncIterable[Symbol.asyncIterator] = async function*() {
  for (let i = 0; i < asyncIterable.length; i++) {
    yield { value: asyncIterable[i], done: false }
  }
  yield { done: true };
};

(async function() {
  for await (const part of asyncIterable) {
    console.log(part);
  }
})();

As opposed to a regular for-of loop, a for-await-of loop will wait for each promise it receives to resolve before moving on to the next one.

There aren't many structures that currently support asynchronous iteration other than streams, but the notation can be manually added to any iterable structure as shown here.

As an asynchronous iterator stream

Asynchronous iterators are very useful when processing streams. Readable, Writable, Duplex, and Transform streams all support asynchronous iterators.

async function printFileToConsole(path) {
  try {
    const readStream = fs.createReadStream(path, { encoding: 'utf-8' });

    for await (const chunk of readStream) {
      console.log(chunk);
    }

    console.log('EOF');
  } catch(error) {
    console.log(error);
  }
}

If you write your code this way, you don't have to listen to the data and end events as you iterate to get each chunk of data, and the for-await-of loop ends when the stream itself ends.

Calling an API with paging functionality

You can also use asynchronous iteration to easily fetch data from sources that use pagination. To do this, we also need a way to reconstruct the response body from the stream that the Node https request method gives us. You can also use asynchronous iterators here, since https requests and responses are streams in Node:

const https = require('https');

function homebrewFetch(url) {
  return new Promise(async (resolve, reject) => {
    const req = https.get(url, async function(res) {
      if (res.statusCode >= 400) {
        return reject(new Error(`HTTP Status: ${res.statusCode}`));
      }

      try {
        let body = '';

        /*
          Instead of res.on listening for data in the stream,
          We can use for-await-of and append the data chunk
          to the rest of the response body*/
        for await (const chunk of res) {
          body += chunk;
        }
    
        // Handle the case where there is no body if (!body) resolve({});
        // We need to parse the body to get the json since it is a string const result = JSON.parse(body);
        resolve(result);
      } catch(error) {
        reject(error)
      }
    });

    await req;
    req.end();
  });
}

We will make a request to the Cat API to get some cat pictures in groups of 10. We will also add a 7 second delay between requests, with a maximum page count of 5, to avoid overloading the cat API.

We will also add a 7 second delay between requests and max pages 5 to avoid overloading the cat API, as that would be catastrophic.

function fetchCatPics({ limit, page, done }) {
  return homebrewFetch(`https://api.thecatapi.com/v1/images/search?limit=${limit}&page=${page}&order=DESC`)
    .then(body => ({ value: body, done }));
}

function catPics({ limit }) {
  return {
    [Symbol.asyncIterator]: async function*() {
      let currentPage = 0;
      // Stop after 5 pages
      while(currentPage < 5) {
        try {
          const cats = await fetchCatPics({ currentPage, limit, done: false });
          console.log(`Fetched ${limit} cats`);
          yield cats;
          currentPage++;
        } catch(error) {
          console.log('There has been an error fetching all the cats!');
          console.log(error);
        }
      }
    }
  };
}

(async function() {
  try {
    for await (let catPicPage of catPics({ limit: 10 })) {
      console.log(catPicPage);
      // Wait for 7 seconds between requests
      await new Promise(resolve => setTimeout(resolve, 7000));
    }
  } catch(error) {
    console.log(error);
  }
})()

This way, we'll automatically fetch a full page of cat pictures every 7 seconds for you to enjoy.

A more common way to navigate between pages is to implement next and previous methods and expose them as controls:

function actualCatPics({ limit }) {
  return {
    [Symbol.asyncIterator]: () => {
      let page = 0;
      return {
        next: function() {
          page++;
          return fetchCatPics({ page, limit, done: false });
        },
        previous: function() {
          if (page > 0) {
            page--;
            return fetchCatPics({ page, limit, done: false });
          }
          return fetchCatPics({ page: 0, limit, done: true });
        }
      }
    }
  };
}

try {
    const someCatPics = actualCatPics({ limit: 5 });
    const { next, previous } = someCatPics[Symbol.asyncIterator]();
    next().then(console.log);
    next().then(console.log);
    previous().then(console.log);
} catch(error) {
  console.log(error);
}

As you can see, asynchronous iterators are very useful when you want to fetch pages of data or do things like infinite scrolling on your application's UI.

These features have been in browsers for a while, available in Chrome v63+, Firefox v57+, and Safari v11.1. But it is currently not available in IE and Edge.

The above is a detailed explanation of the asynchronous iterator in nodejs. For more information about nodejs asynchronous iterator, please pay attention to other related articles on 123WORDPRESS.COM!

You may also be interested in:
  • NodeJs high memory usage troubleshooting actual combat record
  • Detailed explanation of using Nodejs built-in encryption module to achieve peer-to-peer encryption and decryption
  • Detailed explanation of nodejs built-in modules
  • Nodejs module system source code analysis
  • A brief discussion on event-driven development in JS and Nodejs
  • How to use module fs file system in Nodejs
  • Summary of some tips for bypassing nodejs code execution
  • Nodejs Exploration: In-depth understanding of the principle of single-threaded high concurrency
  • Nodejs error handling process record
  • How to use nodejs to write a data table entity class generation tool for C#

<<:  Installation tutorial of mysql8.0rpm on centos7

>>:  Detailed tutorial on installing MySQL 8.0 from source code on CentOS 7.4

Recommend

Pure CSS to achieve the effect of picture blinds display example

First, let me show you the finished effect Main i...

Vue interpretation of responsive principle source code analysis

Table of contents initialization initState() init...

Summary of common commands for building ZooKeeper3.4 middleware under centos7

1. Download and decompress 1. Introduction to Zoo...

JavaScript+HTML to implement student information management system

Table of contents 1. Introduction 2. Rendering 3....

More Ways to Use Angle Brackets in Bash

Preface In this article, we will continue to expl...

How to install mysql on centos and set up remote access

1. Download the mysql repo source $ wget http://r...

Install mysql5.7.17 using RPM under Linux

The installation method of MySQL5.7 rpm under Lin...

Detailed installation and use of virtuoso database under Linux system

I've been researching some things about linke...

Mini Program to Implement Calculator Function

This article example shares the specific code of ...

Detailed explanation of the frame and rules attributes of the table in HTML

The frame and rules attributes of the table tag c...

Docker installation tomcat dubbo-admin instance skills

1. Download the tomcat image docker pull tomcat:8...

MySQL detailed explanation of isolation level operation process (cmd)

Read uncommitted example operation process - Read...

Steps to change mysql character set to UTF8 under Linux system

Table of contents 1. Check the MySQL status in th...