Solve the problem that await does not work in forEach

1. Introduction

A few days ago, I encountered a pit when using for traversal in a project, and it took me a day to solve it. Just remember it here.

2. Problem

First, let me introduce a very simple topic: given an array, print it out every 1s. Here I paste the code I started in the project. (Of course, this has nothing to do with the business)

const _ = require('lodash');
const echo = async (i) => {
  setTimeout(() => {
    console.log('i===>', i);
  }, 5000);
}
let arrs = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const task = async () => {
  _.forEach(arrs, async (i) => {
    await echo(i);
  })
}
const run = async () => {
  console.log('run-start====>date:', new Date().toLocaleDateString())
  await task() ;
  console.log('run-end====>date:', new Date().toLocaleDateString())
}
(async () => {
  console.log('start...')
  await run();
  console.log('end...')
})()
// start...
// run-start====>date: 2018-8-25
// run-end====>date: 2018-8-25
// end...
// i ===> 1
// i ===> 2
// i ===> 3
// i ===> 4
// i ===> 5
// i ===> 6
// i ===> 7
// i ===> 8
// i ===> 9

The above code and output have been given. It is strange that the await here has no effect. At first, because I added the business, there was a problem with my business code, and then I extracted the code, but it still didn’t work. At that time, I really doubted await.

Finally, the answer to the question is given:

lodash's forEach and [].forEach do not support await. If you must execute await while traversing, you can use for-of

Here is the correct code:

const _ = require('lodash');
const echo = async (i) => {
  return new Promise((resolve,reject)=>{
    setTimeout(() => {
      console.log('i===>', i,new Date().toLocaleTimeString());
      resolve(i) ;
    }, 2000);
  })
}
let arrs = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10];
const task = async () => {
  // _.forEach(arrs, async (i) => {
  // await echo(ji) ;
  // })
  // arrs.forEach(async (i )=> {
  // await echo(i);
  // });
  for (const i of arrs) {
    await echo(i) ;
  }
}
const run = async () => {
  console.log('run-start====>date:', new Date().toLocaleDateString())
  await task() ;
  console.log('run-end====>date:', new Date().toLocaleDateString())
}
(async () => {
  console.log('start...')
  await run();
  console.log('end...')
})()
// Output start...
run-start====>date: 2018-8-26
i===> 1 20:51:29
i===> 2 20:51:31
i===> 3 20:51:33
i===> 4 20:51:35
i===> 5 20:51:37
i===> 6 20:51:39
i===> 7 20:51:42
i===> 8 20:51:44
i===> 9 20:51:46
i===> 10 20:51:48
run-end====>date: 2018-8-26
end...

Conclusion

When solving a problem, sometimes you can use the process of elimination. For example, in this example, we know that the await mechanism must be fine. If there is a problem, it will definitely not be my turn to test it. So the remaining problem can only be the cause of the for traversal.

Because I implemented it with lodash at the beginning, I might wonder if lodash's forEach did not do (or did redundant) await processing. At this time, I could try another way. In general, it's a matter of experience.

Supplement: Problems encountered when using async/await in forEach

1. Problem Description

A few days ago, I encountered a JavaScript asynchronous problem in the project:

There is a set of data, each of which needs to be processed asynchronously, and it is hoped that the processing is synchronous.

The code description is as follows:

// Generate data const getNumbers = () => {
 return Promise.resolve([1, 2, 3])
}
// Asynchronous processing const doMulti = num => {
 return new Promise((resolve, reject) => {
  setTimeout(() => {
   if (num) {
    resolve(num * num)
   } else {
    reject(new Error('num not specified'))
   }
  }, 2000)
 })
}
// Main function const main = async () => {
 console.log('start');
 const nums = [1, 2, 3];
 nums.forEach(async (x) => {
  const res = await doMulti(x);
  console.log(res);
 });
 console.log('end');
};
// Execute main();

In this example, forEach iterates over each number and executes the doMulti operation. The result of the code execution is: first, start and end are printed immediately. After 2 seconds, 1, 4, and 9 are output at once.

This result is somewhat different from what we expected. We hope to perform asynchronous processing every 2 seconds and output 1, 4, and 9 in sequence. So the current code should be executed in parallel, but we expect it to be executed serially.

Let's try replacing the forEach loop with a for loop:

const main = async () => {
 console.log('start');
 const nums = await getNumbers();
 for (const x of nums) {
  const res = await doMulti(x);
  console.log(res);
 }
 console.log('end');
};

The execution result is exactly as expected: the output is: start, 1, 4, 9, end.

2. Problem Analysis

The ideas are the same, but the traversal methods used are different. Why does this happen? I searched for the polyfill for forEach on MDN. Reference MDN-Array.prototype.forEach():

// Production steps of ECMA-262, Edition 5, 15.4.4.18
// Reference: http://es5.github.io/#x15.4.4.18
if (!Array.prototype.forEach) {
 Array.prototype.forEach = function(callback, thisArg) {
  var T, k;
  if (this == null) {
   throw new TypeError(' this is null or not defined');
  }
  // 1. Let O be the result of calling toObject() passing the
  // |this| value as the argument.
  var O = Object(this);
  // 2. Let lenValue be the result of calling the Get() internal
  // method of O with the argument "length".
  // 3. Let len be toUint32(lenValue).
  var len = O.length >>> 0;
  // 4. If isCallable(callback) is false, throw a TypeError exception. 
  // See: http://es5.github.com/#x9.11
  if (typeof callback !== "function") {
   throw new TypeError(callback + ' is not a function');
  }
  // 5. If thisArg was supplied, let T be thisArg; else let
  // T is undefined.
  if (arguments.length > 1) {
   T = thisArg;
  }
  // 6. Let k be 0
  k = 0;
  // 7. Repeat, while k < len
  while (k < len) {
   var kValue;
   // a. Let Pk be ToString(k).
   // This is implicit for LHS operands of the in operator
   // b. Let kPresent be the result of calling the HasProperty
   // internal method of O with argument Pk.
   // This step can be combined with c
   // c. If kPresent is true, then
   if (k in O) {
    // i. Let kValue be the result of calling the Get internal
    // method of O with argument Pk.
    kValue = O[k];
    // ii. Call the Call internal method of callback with T as
    // the this value and argument list containing kValue, k, and O.
    callback.call(T, kValue, k, O);
   }
   // d. Increase k by 1.
   k++;
  }
  // 8. return undefined
 };
}

From setp 7 in the polyfill above, we can simply understand the following steps:

Array.prototype.forEach = function (callback) {
 // this represents our array
 for (let index = 0; index < this.length; index++) {
  // We call the callback for each entry
  callback(this[index], index, this);
 };
};

This is equivalent to a for loop executing this asynchronous function, so it is executed in parallel, resulting in all the output results at once: 1, 4, 9.

const main = async () => {
 console.log('start');
 const nums = await getNumbers();
 // nums.forEach(async (x) => {
 // const res = await doMulti(x);
 // console.log(res);
 // });
 for (let index = 0; index < nums.length; index++) {
  (async x => {
   const res = await doMulti(x)
   console.log(res)
  })(nums[index])
 }
 console.log('end');
};

3. Solution

Now, we have analyzed the problem clearly. In the previous solution, we used the for-of loop instead of forEach. In fact, we can also modify forEach:

const asyncForEach = async (array, callback) => {
 for (let index = 0; index < array.length; index++) {
  await callback(array[index], index, array);
 }
}
const main = async () => {
 console.log('start');
 const nums = await getNumbers();
 await asyncForEach(nums, async x => {
  const res = await doMulti(x)
  console.log(res)
 })
 console.log('end');
};
main();

IV. Eslint Issues

At this time, Eslint reported another error: no-await-in-loop. Regarding this point, the official Eslint document https://eslint.org/docs/rules/no-await-in-loop also explains it.

Good writing:

async function foo(things) {
 const results = [];
 for (const thing of things) {
  // Good: all asynchronous operations are started immediately.
  results.push(bar(thing));
 }
 // Now that all the asynchronous operations are running, here we wait until they all complete.
 return baz(await Promise.all(results));
}

Bad writing:

async function foo(things) {
 const results = [];
 for (const thing of things) {
  // Bad: each loop iteration is delayed until the entire asynchronous operation completes
  results.push(await bar(thing));
 }
 return baz(results);
}

In fact, there is no good or bad difference between the above two ways of writing. The results of these two ways of writing are completely different. The "good writing method" recommended by Eslint has no order when performing asynchronous operations, while the "bad writing method" has order. The specific writing method to be used should be determined based on business needs.

Therefore, in the When Not To Use It section of the document, Eslint also mentioned that if sequential execution is required, we can disable this rule:

In many cases the iterations of a loop are not actually independent of each-other. For example, the output of one iteration might be used as the input to another. Or, loops may be used to retry asynchronous operations that were unsuccessful. Or, loops may be used to prevent your code from sending an excessive amount of requests in parallel. In such cases it makes sense to use await within a loop and it is recommended to disable the rule via a standard ESLint disable comment.

The above is my personal experience. I hope it can give you a reference. I also hope that you will support 123WORDPRESS.COM. If there are any mistakes or incomplete considerations, please feel free to correct me.

You may also be interested in: