EDIT: I believe I was going down the wrong path with trying to remove duplicates and needed to clear out the array instead.

Here is example code. I am trying to find the best place to clear out the parent.subdirectories array before each iteration.

const mainArray = [{Id:1,Name:"test",SubDirectoryCount:10,Files:[]}];
const directories = [{Id:56,Name:"Client 2",SubDirectoryCount:1,ParentId:1,Files:[]},{Id:53,Name:"Client 3",SubDirectoryCount:1,ParentId:1,Files:[]},{Id:50,Name:"Client 4",SubDirectoryCount:1,ParentId:1,Files:[]},{Id:48,Name:"Client 5",SubDirectoryCount:1,ParentId:53,Files:[]},{Id:47,Name:"Client 5",SubDirectoryCount:1,ParentId:53,Files:[]},{Id:49,Name:"Client 5",SubDirectoryCount:1,ParentId:53,Files:[]}];

// create flat list
const tmp = mainArray.concat(directories);
tmp.forEach(directory => {
  const parent = tmp.find(item => item.Id === directory.ParentId);

  if(parent) {
      parent.SubDirectories = parent.SubDirectories || [];
      parent.SubDirectories.push(directory)
  }
});

console.log(mainArray)

My attempt but, it removes top items before rebuilding JSON.

  tmp.forEach((directory) => {
    const parent = tmp.find((item) => item.Id === directory.ParentFk);

    if (parent) {
      parent.SubDirectories = parent.SubDirectories || [];

      const found = parent.SubDirectories.some((el) => el.Id === directory.Id);
      if (!found) {
        parent.SubDirectories.push(directory);
      } else {
        parent.SubDirectories = [];
      }
    }
  }, []);