I've developed a multi user app where I can login and upload a csv file. The data is extracted from the csv file into objects (about 10000) and then the data/object(s) is/are inserted into mongodb. I've tried differnt mongodb insert option. The fastest is the bulk.insert option. It inserts about 200 objects per second. Following is my bulk.insert code:
const bulk = db.collection('mycollection').initializeUnorderedBulkOp();
bulk.insert(csvData);
bulk.execute();
But the issue with this bulk insert is that while data is inserted to mongodb I cannot view other pages (the pages that read data from the same db). Also, other users cannot login to the website. The website has just one db with different collections.
I tried normal insert option also. Following is the code:
await db.collection("mycollection").insert(csvData);
With this option, other users can login, I can view other pages. But, the insert speed is some 9 times slower (About 24 inserts per second).
With bulk operation, it seems, while mongodb is busy with one query, it does not respond to other queries. Please suggest me a way to increase insert speed and at the same time make the db accessible to others.