Serial and parallel operations in JavaScript

Serial and parallel operations in JavaScript

1. Introduction

This article describes the solutions for asynchronous functions, serial execution, and parallel execution in es5 and es6 in js . Examples have been used in combination with serial and parallel.

2. es5 method

Before es6 came out, the nodejs community already had a promise solution to deal with callback hell. If there are multiple asynchronous functions, how should the execution sequence be arranged? How can all asynchronous functions be executed faster and then proceed to the next step? Here comes the problem of serial execution and parallel execution of js.

3. Asynchronous function serial execution

var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];

function async(arg, callback) {
  console.log('Parameter is ' + arg +', return result after 1 second');
  setTimeout(function () { callback(arg * 2); }, 1000);
}

function final(value) {
  console.log('Completed: ', value);
}

function series(item) {
  if(item) {
    async(item, function(result) {
      results.push(result);
      return series(items.shift()); // recursively execute all data });
  } else {
    return final(results[results.length - 1]);
  }
}

series(items.shift());

4. Parallel execution of asynchronous functions

The above functions are executed one by one, and the next one is executed after the previous one is finished, which is similar to async and await in es6 (collectively referred to as es6 after es5). Is there something like promise.all that can execute all in parallel?

We can write:

var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];

function async(arg, callback) {
  console.log('Parameter is ' + arg +', return result after 1 second');
  setTimeout(function () { callback(arg * 2); }, 1000);
}

function final(value) {
  console.log('Completed: ', value);
}

items.forEach(function(item) {// loop complete async(item, function(result){
    results.push(result);
    if(results.length === items.length) {// Determine whether the number of completed functions is equal to the number of functions to be executed final(results[results.length - 1]);
    }
  })
});

5. Combination of serial and parallel execution of asynchronous functions

If many asynchronous data (hundreds of them) are executed in parallel, and each asynchronous data contains a lot of (https) request data, it is bound to cause insufficient TCP connections or accumulation of countless call stacks leading to memory overflow. Therefore, it is not easy to execute too much data in parallel, so a combination of parallel and serial methods emerged.

The code can be written as follows:

var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];
var running = 0;
var limit = 2;

function async(arg, callback) {
  console.log('Parameter is ' + arg +', return result after 1 second');
  setTimeout(function () { callback(arg * 2); }, 1000);
}

function final(value) {
  console.log('Completed: ', value);
}

function launcher() {
  while(running < limit && items.length > 0) {
    var item = items.shift();
    async(item, function(result) {
      results.push(result);
      running--;
      if (items.length > 0) {
        launcher();
      } else if(running == 0) {
        final(results);
      }
    });
    running++;
  }
}

launcher();

6. es6 method

es6 naturally comes with serial and parallel execution methods. For example, serial can use async and await (explained in the previous article), and parallel can use promise.all and so on. Then for combining serial and parallel operations and limiting the number of concurrent promise all calls, the community also has some solutions, such as

tiny-async-pool, es6-promise-pool, p-limit

Simple encapsulation of a promise all concurrent number limit solution function

function PromiseLimit(funcArray, limit = 5) { // Execute 5 data concurrently let i = 0;
  const result = [];
  const executing = [];
  const queue = function() {
    if (i === funcArray.length) return Promise.all(executing);
    const p = funcArray[i++]();
    result.push(p);
    const e = p.then(() => executing.splice(executing.indexOf(e), 1));
    executing.push(e);
    if (executing.length >= limit) {
      return Promise.race(executing).then(
        () => queue(),
        e => Promise.reject(e)
      );
    }
    return Promise.resolve().then(() => queue());
  };
  return queue().then(() => Promise.all(result));
}

use:

// Test code const result = [];
for (let index = 0; index < 10; index++) {
  result.push(function() {
    return new Promise((resolve, reject) => {
      console.log("start" + index, new Date().toLocaleString());
      setTimeout(() => {
        resolve(index);
        console.log("End" + index, new Date().toLocaleString());
      }, parseInt(Math.random() * 10000));
    });
  });
}

PromiseLimit(result).then(data => {
  console.log(data);
});

Modify the test code and add random failure logic

// Modify the test code to fail or succeed randomly const result = [];
for (let index = 0; index < 10; index++) {
  result.push(function() {
    return new Promise((resolve, reject) => {
      console.log("start" + index, new Date().toLocaleString());
      setTimeout(() => {
        if (Math.random() > 0.5) {
          resolve(index);
        } else {
          reject(index);
        }
        console.log("End" + index, new Date().toLocaleString());
      }, parseInt(Math.random() * 1000));
    });
  });
}
PromiseLimit(result).then(
  data => {
    console.log("success", data);
  },
  data => {
    console.log("failed", data);
  }
);

7. async and await combined with promise all

async function PromiseAll(promises,batchSize=10) {
 const result = [];
 while(promises.length > 0) {
   const data = await Promise.all(promises.splice(0,batchSize));
   result.push(...data);
 }
return result;
}

There are two problems with this writing:

  • 1. promises are created before calling Promise.all , promise are actually executed
  • 2. Your implementation must wait for the previous batchSize個promise resolve before running the next batchSize , that is, promise all must succeed.

The improvements are as follows:

async function asyncPool(array,poolLimit,iteratorFn) {
  const ret = [];
  const executing = [];
  for (const item of array) {
    const p = Promise.resolve().then(() => iteratorFn(item, array));
    ret.push(p);

    if (poolLimit <= array.length) {
      const e = p.then(() => executing.splice(executing.indexOf(e), 1));
      executing.push(e);
      if (executing.length >= poolLimit) {
        await Promise.race(executing);
      }
    }
  }
  return Promise.all(ret);
}

use:

const timeout = i => new Promise(resolve => setTimeout(() => resolve(i), i));
return asyncPool( [1000, 5000, 3000, 2000], 2,timeout).then(results => {
    ...
});

This is the end of this article about serial and parallel operations in JavaScript asynchronous operations. For more relevant content about serial and parallel operations in JavaScript asynchronous operations, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • Summary of several common processing methods for JavaScript asynchronous operations
  • Solve the problem of using $q.all to sort the order of Angularjs asynchronous operation background requests
  • Detailed explanation of asynchronous operation examples of ES6 javascript
  • async/await and promise (asynchronous operation problem in nodejs)
  • Thinkjs page jump synchronous and asynchronous operation
  • JavaScript-Solve the asynchronous operation of mongoose data query
  • A simple Node.js asynchronous operation manager sharing

<<:  HTML css js implements Tab page sample code

>>:  How to change fixed positioning of child elements to absolute positioning by CSS3 transform

Recommend

Full HTML of the upload form with image preview

The upload form with image preview function, the ...

Modify the default data directory of MySQL 8.0 (quick operation without configuration)

Usage scenario: We use Alibaba Cloud and purchase...

How to export and import .sql files under Linux command

This article describes how to export and import ....

Detailed explanation of creating stored procedures and functions in mysql

Table of contents 1. Stored Procedure 1.1. Basic ...

Attributes and usage of ins and del tags

ins and del were introduced in HTML 4.0 to help au...

Node+Express test server performance

Table of contents 1 Test Environment 1.1 Server H...

Detailed explanation of mysql permissions and indexes

mysql permissions and indexes The highest user of...

Docker container from entry to obsession (recommended)

1. What is Docker? Everyone knows about virtual m...

Ubuntu 20.04 connects to wifi (2 methods)

I recently installed Ubuntu 20.04 and found that ...

HTML table border control implementation code

Generally, when we use a table, we always give it...

About the garbled problem caused by HTML encoding

Today a junior student asked a question. The HTML...

The iframe frame sets the white background to transparent in IE browser

Recently, I need to frequently use iframe to draw ...

Linux Centos8 Create CA Certificate Tutorial

Install Required Files Yum install openssl-* -y C...

Linux common basic commands and usage

This article uses examples to illustrate common b...

Detailed tutorial on installing mysql on centos 6.9

1. Confirm whether MySQL has been installed. You ...