Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

197
Views
What happens when adding a new shard in the cluster using hashed shard?

I'm wondering if the following function is supported in the MongoDB. When adding a new shard to the existing sharding cluster which is currently using hashed sharding, some documents in the old shards should be migrated to the new shard.

Question: Does mongodb can handle this process automatically?

Thank you in advance.

over 4 years ago · Santiago Trujillo
1 answers
Answer question

0

Yes. Naively you would imagine an implementation taking a hashcode of a key field, doing a modulo of the number of nodes in a cluster, and putting the value on that node. Adding a node/shard to the cluster would change your modulo'd value, and all data would be scrambled, so adding a node would not be possible.

To work around this issue, distributed systems do it a bit differently. The hash code, instead of mapping to a physical machine, maps to what in mongodb is called a "chunk." In apache spark this would be a partition. There are more chunks than nodes, but many rows of data map to one "chunk."

As an example, you could imagine if you have ten million rows and 5 hosts, you could take the hashcode modulo 100 so that your ten million rows have 100 unique keys with which to deterministically select a node in the cluster, then you manage a mapping that chooses node1 for keys 1-20, node2 for keys 21-40, etc. When you add a fifth node, a rebalancing process can move "chunks" from one node to another, and update the mapping table. Since the mapping table is only recording node locations for a hundred chunks, and not every row, this is manageable.

Mongodb uses a background process to monitor how many chunks are on a node, and move them as needed.

"Chunks" in mongodb: https://docs.mongodb.com/v3.2/core/sharding-data-partitioning/

Reblanacing: https://docs.mongodb.com/v3.2/core/sharding-balancer-administration/

over 4 years ago · Santiago Trujillo Report
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!