Detailed explanation of asynchronous iterators in nodejs

Detailed explanation of asynchronous iterators in nodejs

Preface

Asynchronous iterators have been around since Node.js v10.0.0, and they have been gaining traction in the community lately. In this article, we will discuss the role of asynchronous iterators and also address the question of what they might be used for.

What are asynchronous iterators?

So what are asynchronous iterators? They are effectively asynchronous versions of the iterators that were available previously. When we don't know the value and final status of the iteration, we can use asynchronous iterators, and finally we get a promise that can be resolved to a {value:any, done:boolean} object. We also got a for-await-of loop to help us loop over asynchronous iterators. Just like a for-of loop is for synchronous iterators.

const asyncIterable = [1, 2, 3];
asyncIterable[Symbol.asyncIterator] = async function*() {
  for (let i = 0; i < asyncIterable.length; i++) {
    yield { value: asyncIterable[i], done: false }
  }
  yield { done: true };
};

(async function() {
  for await (const part of asyncIterable) {
    console.log(part);
  }
})();

As opposed to a regular for-of loop, a for-await-of loop will wait for each promise it receives to resolve before moving on to the next one.

There aren't many structures that currently support asynchronous iteration other than streams, but the notation can be manually added to any iterable structure as shown here.

As an asynchronous iterator stream

Asynchronous iterators are very useful when processing streams. Readable, Writable, Duplex, and Transform streams all support asynchronous iterators.

async function printFileToConsole(path) {
  try {
    const readStream = fs.createReadStream(path, { encoding: 'utf-8' });

    for await (const chunk of readStream) {
      console.log(chunk);
    }

    console.log('EOF');
  } catch(error) {
    console.log(error);
  }
}

If you write your code this way, you don't have to listen to the data and end events as you iterate to get each chunk of data, and the for-await-of loop ends when the stream itself ends.

Calling an API with paging functionality

You can also use asynchronous iteration to easily fetch data from sources that use pagination. To do this, we also need a way to reconstruct the response body from the stream that the Node https request method gives us. You can also use asynchronous iterators here, since https requests and responses are streams in Node:

const https = require('https');

function homebrewFetch(url) {
  return new Promise(async (resolve, reject) => {
    const req = https.get(url, async function(res) {
      if (res.statusCode >= 400) {
        return reject(new Error(`HTTP Status: ${res.statusCode}`));
      }

      try {
        let body = '';

        /*
          Instead of res.on listening for data in the stream,
          We can use for-await-of and append the data chunk
          to the rest of the response body*/
        for await (const chunk of res) {
          body += chunk;
        }
    
        // Handle the case where there is no body if (!body) resolve({});
        // We need to parse the body to get the json since it is a string const result = JSON.parse(body);
        resolve(result);
      } catch(error) {
        reject(error)
      }
    });

    await req;
    req.end();
  });
}

We will make a request to the Cat API to get some cat pictures in groups of 10. We will also add a 7 second delay between requests, with a maximum page count of 5, to avoid overloading the cat API.

We will also add a 7 second delay between requests and max pages 5 to avoid overloading the cat API, as that would be catastrophic.

function fetchCatPics({ limit, page, done }) {
  return homebrewFetch(`https://api.thecatapi.com/v1/images/search?limit=${limit}&page=${page}&order=DESC`)
    .then(body => ({ value: body, done }));
}

function catPics({ limit }) {
  return {
    [Symbol.asyncIterator]: async function*() {
      let currentPage = 0;
      // Stop after 5 pages
      while(currentPage < 5) {
        try {
          const cats = await fetchCatPics({ currentPage, limit, done: false });
          console.log(`Fetched ${limit} cats`);
          yield cats;
          currentPage++;
        } catch(error) {
          console.log('There has been an error fetching all the cats!');
          console.log(error);
        }
      }
    }
  };
}

(async function() {
  try {
    for await (let catPicPage of catPics({ limit: 10 })) {
      console.log(catPicPage);
      // Wait for 7 seconds between requests
      await new Promise(resolve => setTimeout(resolve, 7000));
    }
  } catch(error) {
    console.log(error);
  }
})()

This way, we'll automatically fetch a full page of cat pictures every 7 seconds for you to enjoy.

A more common way to navigate between pages is to implement next and previous methods and expose them as controls:

function actualCatPics({ limit }) {
  return {
    [Symbol.asyncIterator]: () => {
      let page = 0;
      return {
        next: function() {
          page++;
          return fetchCatPics({ page, limit, done: false });
        },
        previous: function() {
          if (page > 0) {
            page--;
            return fetchCatPics({ page, limit, done: false });
          }
          return fetchCatPics({ page: 0, limit, done: true });
        }
      }
    }
  };
}

try {
    const someCatPics = actualCatPics({ limit: 5 });
    const { next, previous } = someCatPics[Symbol.asyncIterator]();
    next().then(console.log);
    next().then(console.log);
    previous().then(console.log);
} catch(error) {
  console.log(error);
}

As you can see, asynchronous iterators are very useful when you want to fetch pages of data or do things like infinite scrolling on your application's UI.

These features have been in browsers for a while, available in Chrome v63+, Firefox v57+, and Safari v11.1. But it is currently not available in IE and Edge.

The above is a detailed explanation of the asynchronous iterator in nodejs. For more information about nodejs asynchronous iterator, please pay attention to other related articles on 123WORDPRESS.COM!

You may also be interested in:
  • NodeJs high memory usage troubleshooting actual combat record
  • Detailed explanation of using Nodejs built-in encryption module to achieve peer-to-peer encryption and decryption
  • Detailed explanation of nodejs built-in modules
  • Nodejs module system source code analysis
  • A brief discussion on event-driven development in JS and Nodejs
  • How to use module fs file system in Nodejs
  • Summary of some tips for bypassing nodejs code execution
  • Nodejs Exploration: In-depth understanding of the principle of single-threaded high concurrency
  • Nodejs error handling process record
  • How to use nodejs to write a data table entity class generation tool for C#

<<:  Installation tutorial of mysql8.0rpm on centos7

>>:  Detailed tutorial on installing MySQL 8.0 from source code on CentOS 7.4

Recommend

What are the differences between sql and mysql

What is SQL? SQL is a language used to operate da...

Detailed explanation of how to configure static IP in Centos8

After installing centos 8, the following error wi...

Detailed explanation of JavaScript to monitor route changes

Table of contents history pushState() Method push...

js to realize simple shopping cart function

This article example shares the specific code of ...

A brief introduction to MySQL storage engine

1. MySql Architecture Before introducing the stor...

A summary of detailed insights on how to import CSS

The development history of CSS will not be introd...

Detailed explanation of react setState

Table of contents Is setState synchronous or asyn...

Detailed steps to install MySql 5.7.21 in Linux

Preface The most widely used database in Linux is...

How to configure domestic sources in CentOS8 yum/dnf

CentOS 8 changed the software package installatio...

Mysql Chinese sorting rules description

When using MySQL, we often sort and query a field...

Install redis and MySQL on CentOS

1|0MySQL (MariaDB) 1|11. Description MariaDB data...

Detailed explanation of how to use the Vue date time picker component

This article example shares the specific code of ...