javascript remove duplicates from array

The shortest way to remove duplicates from an array is [...new Set(arr)]: a Set holds each value once, and spreading it back gives an array in first-seen order. It works for strings, numbers, booleans, null, undefined and NaN. For arrays of objects, deduplicate by a key (for example id) with a Map.

Primitive values

cat > dedupe.mjs <<'EOF'
const values = [3, "3", 1, 3, NaN, NaN, null, null, "a", "A", "a"];
console.log([...new Set(values)]);
console.log(Array.from(new Set(values)).length, "unique values");
console.log(values.filter((v, i) => values.indexOf(v) === i));
EOF
node dedupe.mjs

[
  3,   '3',  1,
  NaN, null, 'a',
  'A'
]
7 unique values
[ 3, '3', 1, null, 'a', 'A' ]

Set uses SameValueZero equality: 3 and "3" are different values, "a" and "A" are different, and NaN counts as equal to itself. The filter + indexOf version drops every NaN, because indexOf uses === and NaN === NaN is false.

Objects by key

cat > objects.mjs <<'EOF'
const rows = [
  { id: 1, name: "first" },
  { id: 2, name: "second" },
  { id: 1, name: "first again" },
];
console.log("Set on objects:", new Set(rows).size);
const keepLast = [...new Map(rows.map((r) => [r.id, r])).values()];
console.log("keep last:", keepLast);
const seen = new Set();
const keepFirst = rows.filter((r) => !seen.has(r.id) && seen.add(r.id));
console.log("keep first:", keepFirst);
EOF
node objects.mjs

Set on objects: 3
keep last: [ { id: 1, name: 'first again' }, { id: 2, name: 'second' } ]
keep first: [ { id: 1, name: 'first' }, { id: 2, name: 'second' } ]

A Set of objects removes nothing, because every object literal is a distinct reference. Building a Map keyed by id keeps the last object for each key; the filter with a seen set keeps the first.

Methods compared

Pitfalls

• Case and type differences are real differences for Set. Normalise first: [...new Set(arr.map(s => s.trim().toLowerCase()))].
• Spreading a Set needs ES2015 iteration; when TypeScript targets ES5 without downlevelIteration, use Array.from(new Set(arr)) instead.
• Calling array methods directly on a Set (new Set(arr).map(...)) fails with x is not a function: convert it back to an array first.

Confirm it worked

Compare lengths: arr.length against new Set(arr).size tells you how many duplicates there were before you remove them, and the two are equal afterwards.

FAQ