What is the most efficient method to dedupe a complex array based on the value of another property? I have found many examples that will dedupe an array or complex array but not this specific use case.
I am trying to find records with unique values in column 3 (state) with the highest number in column 2 (license)
var arrayWithDuplicates = [
["Boat", 1, "NV"],
["Car", 7, "CA"],
["Boat", 3, "NV"],
["Boat", 4, "CA"],
["Car", 5, "OR"],
["Boat", 6, "CA"],
];
Desired outcome
var outputArray = [
["Car", 7, "CA"],
["Boat", 3, "NV"],
["Car", 5, "OR"]
];
This works but not sure if with large datasets
var arrayWithDuplicates = [
["Boat", 1, "NV"],
["Car", 7, "CA"],
["Boat", 3, "NV"],
["Boat", 4, "CA"],
["Car", 5, "OR"],
["Boat", 6, "CA"],
];
let arr= arrayWithDuplicates;
let unique = []
for (let i = 0; i < arr.length; i++) {
let found = false;
for (let j = 0; j < unique.length; j++) {
if (arr[i][2] === unique[j][2]) {
found = true;
if (arr[i][1] > unique[j][1]) {
unique[j] = arr[i];
}
break;
}
}
if (!found) {
unique.push(arr[i])
}
}
console.log(unique);
[["Boat", 3, "NV"], ["Boat", 7, "CA"], ["Car", 5, "OR"]]
You can see the performance of the proposed solutions: https://jsbench.me/eskxxcwnhn/1
One way to solve your problem would be to use a Map instance to hold the most relevant value. Then replace it if you encounter another. Then, when you are done iterating, you take the values that are present in the Map instance.
const arrayWithDuplicates = [
["Boat", 1, "NV"],
["Car", 7, "CA"],
["Boat", 3, "NV"],
["Boat", 4, "CA"],
["Car", 5, "OR"],
["Boat", 6, "CA"],
];
const lookup = new Map();
for (const record of arrayWithDuplicates) {
const key = record[2];
if (!lookup.has(key)) {
lookup.set(key, record);
continue;
}
const other = lookup.get(key);
if (record[1] > other[1]) {
// Iteration order is based on insertion order. By removing the
// current value first, the new value will be placed at the end.
// If you don't care about the order, deletion can be omitted.
lookup.delete(key);
lookup.set(key, record);
}
}
const result = Array.from(lookup.values());
console.log(result);Note that the following code sequence can be a fairly heavy operation:
lookup.delete(key);
lookup.set(key, record);
Due to the fact that it rearranges the iteration order of the Map contents. This is only done get the result in the order you are looking for. If the order of the resulting items is irrelevant, then you should remove the lookup.delete(key) call to improve the execution speed.
Although using Map instances might produce some execution overhead for small collections. The improved lookup speed really shines when collections get larger.
Using Sort and Reduce
const arrayWithDuplicates = [ ["Boat", 1, "NV"], ["Car", 7, "CA"], ["Boat", 3, "NV"], ["Boat", 4, "CA"], ["Car", 5, "OR"], ["Boat", 6, "CA"], ];
let deduped = [...arrayWithDuplicates]; // take a copy
deduped.sort((a, b) => { // sorts in place
if (a[0] < b[0]) return 1; // sort on names
if (a[0] > b[0]) return -1;
return b[1] - a[1]; // sort on second element
})
// reduce into an object keyed on state
deduped = Object.values( // take only the values from the object
deduped.reduce((acc, cur) => {
const state = cur[2];
if (!acc[state]) acc[state] = cur;
return acc;
},{}))
console.log(deduped)