Nine advanced methods for deduplicating JS arrays (proven and effective)

Preface

The general methods are not listed here, there are still many, such as double loop to determine whether they are equal, or create a new array for comparison and then push, etc. It should be noted that when using the splice method to remove elements, it may cause the array to collapse, which needs to be handled

This article introduces a variety of array deduplication methods, using a large number of high-level methods and APIs, and gives corresponding explanations and syntax. There are also many other combined calling methods. The principle logic is actually similar. Among them, the for loop can be converted to the forEach method, so I will not list them one by one here. If you have a better algorithm, you can leave me a message and ask for my advice humbly! !

Given an array [1,2,2,4,null,null,'3','abc',3,5,4,1,2,2,4,null,null,'3','abc',3,5,4] remove duplicates

let arr = [1,2,2,4,null,null,'3','abc',3,5,4,1,2,2,4,null,null,'3','abc',3,5,4] 

1. Use the unique key of the object

As we all know, the key of an object cannot be repeated, otherwise the latter will overwrite the former. Using this feature, we can deduplicate the array, traverse the array, and use each item in the array as the key value of the object.

let obj = {};
for (let i = 0; i < arr.length; i++) {
 let item = arr[i]
 if (obj[item] !== undefined) {
 arr.splice(i, 1);
 i--; // Solve the problem of array collapse after deleting elements continue;
 }
 obj[item] = item
}

// arr: [1, 2, 4, null, "3", "abc", 3, 5]

2. Swap the elements to replace the splice method

The above method has certain performance issues, that is, the deletion performance based on splice is not very good. After the current item is deleted, the index of each subsequent item must be moved forward one position. When the amount of data is large, it will definitely affect the performance.

Based on the above considerations, exchanging the positions of elements will be more efficient. If the current element is repeated, it will be exchanged with the last element of the array, and i-- can be judged again. At the same time, length--, the length of the array is operated to delete the last element of the array, so that other elements in the array will not be affected.

let obj = {};
for (let i = 0; i < arr.length; i++) {
 let item = arr[i]
 if (obj[item] !== undefined) {
 arr[i] = arr[arr.length - 1]
 arr.length--;
 i--; 
 continue;
 }
 obj[item] = item
}
// arr: [1, 2, 4, null, "3", "abc", 3, 5]

3. Array.filter + Array.indexOf

filter() method: Create a new array whose elements are all elements in the specified array that meet certain conditions. If there are no elements matching the condition, an empty array is returned.

Syntax: array.filter(function(item,index,arr))

• filter() does not detect empty arrays.
• filter() does not mutate the original array.

Principle: Returns the element whose first occurrence of item is equal to the current index

let newArr = arr.filter((item, index) => arr.indexOf(item) === index); 

// [1, 2, 4, null, "3", "abc", 3, 5]

4. Array.filter + Object.hasOwnProperty

hasOwnProperty() method: Returns a Boolean value indicating whether the object has the specified property.

Principle: Use the non-repeatable nature of the object's key name

let obj = {}

arr.filter(item => obj.hasOwnProperty(typeof item + item) ? false : (obj[typeof item + item] = true)) 

5. Array.reduce + Array.includes

reduce() method: Receives a function as an accumulator, and each value in the array is calculated from left to right and finally calculated as a value.

Syntax: arr.reduce(function(total, currValue, currIndex, arr), initValue)

reduce() will not execute the callback function for an empty array.

total: required. Initial value, or return value after calculation

currValue: required. Current element

currIndex:Optional. The index of the current element

arr : optional. The current array object.

initValue: Optional. Accumulator initial value

Calling the reduce() method on an empty array without providing an initial value will result in an error.

Calling the reduce() method with an empty array and providing an initial value will directly return the initial value without calling the callback function.

If a non-empty array calls reduce() to provide an initial value, total will be equal to the initial value, and currValue will start from the first element; if no initial value is provided, total will be equal to the first element value, and currValue will start from the second element.

let newArr = arr.reduce((accu, cur) => {
 return accu.includes(cur) ? accu : accu.concat(cur); // 1. Concatenation method// return accu.includes(cur) ? accu : [...accu, cur]; // 2. Extension operation}, []);


// [1, 2, 4, null, "3", "abc", 3, 5]

6. Array.indexOf

indexOf() method: returns the position of a specified element in an array. This method traverses the array, searches for the corresponding element and returns the index of the first occurrence of the element. If the specified element is not found, it returns -1.

let newArr = []
for (var i = 0; i < arr.length; i++) {
 if (newArr.indexOf(arr[i]) === -1) newArr.push(arr[i]) 
}

// Equivalent to forEach arr.forEach( item => newArr.indexOf(item) === -1 ? newArr.push(item) : '')

console.log(newArr) // [1, 2, 4, null, "3", "abc", 3, 5]

7. Array.includes

includes() method: used to determine whether an array contains a specified value, if so, returns true, otherwise false.

let newArr = []
for (var i = 0; i < arr.length; i++) {
 if (!newArr.includes(arr[i])) newArr.push(arr[i])
}

// Equivalent to forEach arr.forEach( item => !newArr.includes(item) ? newArr.push(item) : '')

console.log(newArr) // [1, 2, 4, null, "3", "abc", 3, 5]

8. new Set + spread operator || Array.from

ES6 provides a new data structure Set. Similar to an array, but the member values ​​are unique and there are no duplicate values.

Set itself is a constructor that can accept a data structure with an iterable interface as a parameter (such as an array, a string) for initialization.

let newArr = [...new Set(arr)]; // [1, 2, 4, null, "3", "abc", 3, 5]

let newArr = Array.from(new Set(arr)); // [1, 2, 4, null, "3", "abc", 3, 5]

let newStr = [...new Set('ababbc')].join('') // 'abc'

9. New Map

ES6 provides a new data structure Map. Similar to an object, it is also a collection of key-value pairs, but the scope of "key" is not limited to strings. Values ​​of various types (including objects) can be used as keys.

The set method sets the key value corresponding to the key name key to value, and then returns the entire Map structure. If the key already has a value, the key value will be updated, otherwise a new key will be generated.

The get method reads the key value corresponding to the key. If the key cannot be found, it returns undefined.

The has method returns a Boolean value indicating whether a key is in the current Map object.

let map = new Map();
let newStr = [];

for (let i = 0; i < arr.length; i++) {
 if (!map.has(arr[i])) {
  map.set(arr[i], true);
  newStr.push(arr[i]);
 }
}
console.log(newArr) // [1, 2, 4, null, "3", "abc", 3, 5]

Summarize

This concludes the article on nine advanced methods for JS array deduplication. For more relevant JS array deduplication content, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!