Seven ways to implement array deduplication in JS

Example: Remove duplicate elements from the following array (taking multiple data types as examples)

const arr = [1, 2, 2, 'abc', 'abc', true, true, false, false, undefined, undefined, NaN, NaN]

1. Using Set()+Array.from()

• Set object: is a collection of values, you can iterate its elements in the order they were inserted. The elements in a Set will only appear once, that is, the elements in a Set are unique.
• Array.from() method: Create a new, shallow copy of an array-like or iterable object.

const result = Array.from(new Set(arr))
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]

Note: The above method is also effective for deduplication of NaN and undefined types, because both NaN and undefined can be stored in Set, and NaN are regarded as the same value (although in js: NaN !== NaN).

2. Using two-layer loop + array splice method

The array elements are compared one by one through two loops, and then splice method is used to delete the duplicate elements. This method cannot deduplicate NaN because NaN !== NaN when comparing.

function removeDuplicate(arr) {

   let len ​​= arr.length

   for (let i = 0; i < len; i++) {

      for (let j = i + 1; j < len; j++) {
        if (arr[i] === arr[j]) {
        arr.splice(j, 1)
        len-- // Reduce the number of loops to improve performance j-- // Ensure that the value of j remains unchanged after addition}
    }
  }
   return arr
}

 const result = removeDuplicate(arr)

 console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN, NaN ]

3. Using the indexOf method of array

Create a new empty array, traverse the array that needs to be deduplicated, and store the array elements in the new array. Before storing, determine whether the array already contains the current element. If not, store it. This method also cannot remove duplicates of NaN.

indexOf() method: Returns the index of the first occurrence of a specified value in String object on which it is called, searching from fromIndex . If the value is not found, -1 is returned.

function removeDuplicate(arr) {

   const newArr = []

   arr.forEach(item => {

     if (newArr.indexOf(item) === -1) {
     newArr.push(item)
    }
 })
 return newArr // Return a new array }

const result = removeDuplicate(arr)
console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN, NaN ]

4. Use the includes method of array

The logic of this method is similar to that of indexOf method, except that it uses the includes method to determine whether duplicate elements are included.

includes() method: used to determine whether an array contains a specified value. Depending on the situation, it returns true if it contains a value, otherwise it returns false.

 function removeDuplicate(arr) {

    const newArr = []

    arr.forEach(item => {

       if (!newArr.includes(item)) {
       newArr.push(item)
      }
   })
  return newArr
 }

 const result = removeDuplicate(arr)

 console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]

Note: Why includes can detect that the array contains NaN involves the underlying implementation of includes. The following figure shows part of the code for includes implementation. When determining whether an element is included, the sameValueZero method is called for comparison. If it is NaN, isNaN() is used for conversion.

Simple test includes()'s judgment on NaN:

const testArr = [1, 'a', NaN]
console.log(testArr.includes(NaN)) // true

5. Using filter()+indexOf() of array

filter method will store the elements that meet the conditions into a new array and make judgments in combination with indexOf method.

filter() method: creates a new array containing all elements that pass the test implemented by the provided function.

 function removeDuplicate(arr) {

    return arr.filter((item, index) => {

       return arr.indexOf(item) === index
  })
}

const result = removeDuplicate(arr)

console.log(result) // [ 1, 2, 'abc', true, false, undefined ]

Note: The output here does not include NaN because indexOf() cannot judge NaN, that is, arr.indexOf(item) === index returns false. The test is as follows:

const testArr = [1, 'a', NaN]
console.log(testArr.indexOf(NaN)) // -1

6. Using Map()

The Map object is a data structure provided by JavaScript . The structure is in the form of key-value pairs. The array elements are stored as map keys, and the front-end training then combines the has() and set() methods to determine whether the keys are repeated.

Map object: used to store key-value pairs and remember the original insertion order of the keys. Any value (object or primitive) can be used as a key or a value.

function removeDuplicate(arr) {

const map = new Map()

const newArr = []

arr.forEach(item => {

  if (!map.has(item)) { // has() is used to determine whether the map contains the attribute value of item map.set(item, true) // Use set() to set item to the map and set its attribute value to true
  
    newArr.push(item)
  }
})

return newArr
}

const result = removeDuplicate(arr)

console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]

Note: You can also use Map() to deduplicate NaN because Map considers NaN to be equal to NaN, and all other values ​​are equal based on the result of the === operator.

7. Target of use

Its implementation idea is similar to Map() , mainly taking advantage of the fact that the attribute names of objects cannot be repeated.

function removeDuplicate(arr) {

   const newArr = []

   const obj = {}

   arr.forEach(item => {

       if (!obj[item]) {
       newArr.push(item)
       obj[item] = true
      }
   })

   return newArr
 }

 const result = removeDuplicate(arr)

 console.log(result) // [ 1, 2, 'abc', true, false, undefined, NaN ]

This concludes this article about seven ways to implement array deduplication in JS. For more information on JS array deduplication, please search previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope you will support 123WORDPRESS.COM in the future!