Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

302
Views
How to check for duplicate data sets in JSON payload using JavaScript?

I would like to find duplicate data sets from below payload using combination of 'NAME', 'ID'. If a set exist more than 3 times, I need to return NAME, ID of duplicated data set.

{
"Test": "1",
"value": [
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "EFGH",
        "ID": "5678",
        "ACTIVE": "true"
    },
    {
        "NAME": "EFGH",
        "ID": "5678",
        "ACTIVE": "true"
    },
    {
        "NAME": "EFGH",
        "ID": "5678",
        "ACTIVE": "true"
    },
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "IJKL",
        "ID": "91011",
        "ACTIVE": "true"
        }
    ]
}

Expected output:

["ABCD:1234", "EFGH:5678"]
about 4 years ago · Juan Pablo Isaza
3 answers
Answer question

0

Try this. You can improve this further by performance wise, if you work around a bit.

  var data = {
 "Test": "1",
 "value": [
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "EFGH",
        "ID": "5678",
        "ACTIVE": "true"
    },
    {
        "NAME": "IJKL",
        "ID": "91011",
        "ACTIVE": "true"
    }
   ]
 };
 var objArray = data.value;
 var duplicates = []; // duplicates will be stored here.
 for(var i=0, iLen = objArray.length; i<iLen;i++){
    var obj = objArray[i];
        var filtered = objArray.filter(function(arrVal) {
        return arrVal.NAME === obj.NAME && arrVal.ID === obj.ID ;
    });
    var dup = obj.NAME + ":" + obj.ID;
    if(filtered.length>=3 && duplicates.indexOf(dup) < 0) {
        duplicates.push(dup);
    }

 }
about 4 years ago · Juan Pablo Isaza Report

0

this questuon was answered multiple times on SO. You create array from the json and compare the elements of the array.

How to remove all duplicates from an array of objects?

about 4 years ago · Juan Pablo Isaza Report

0

According to your sample output I believe it's "at least 3 times" rather than "more than 3 times".

Below snippet can product the expected output with the sample data.

const data = {
"Test": "1",
"value": [
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "EFGH",
        "ID": "5678",
        "ACTIVE": "true"
    },
    {
        "NAME": "EFGH",
        "ID": "5678",
        "ACTIVE": "true"
    },
    {
        "NAME": "EFGH",
        "ID": "5678",
        "ACTIVE": "true"
    },
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "ABCD",
        "ID": "1234",
        "ACTIVE": "true"
    },
    {
        "NAME": "IJKL",
        "ID": "91011",
        "ACTIVE": "true"
        }
    ]
};

const occurrence = {}; // key: count
const result = [];
for (const item of data.value) {
  const key = `${item.NAME}:${item.ID}`;
  occurrence[key] = (occurrence[key] || 0) + 1;
  if (occurrence[key] >= 3) {
    result.push(key);
  }
}

console.log(result);
about 4 years ago · Juan Pablo Isaza Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!