There is a pool of number 1 - 160 000 000.
When create a obj, need to allocate one number to the obj. There is some rules
Also user sometimes will specify one number to use for the obj creation.
Below are some solutions, each have its own problems, So I expect some better solution
Please note that we use mongo DB here. I do not want to change database because this one issue.
Generate a big table(collection) with 160,000,000 items. The structure of the collection is
number,allocated
When allocate number, use find_one_and_update method to update one record,change the allocated from false to true
problem for this solution is that generate a collection of 160,000,000 is too heavy
Similar to solution 1 except we do not generate 160,000,000 at one time. Instead we generate 1000 each time. When this 1000 records is run out, we generate another 1000
The problem is that user can specify number sometimes. For example, we generate 1000 records in the collection, but use want to use number 5000 instead. So this is the problem now because we did not generate it
Each time we create an obj, we generate a random number within 1-160,000,000 to this obj and save it in the db.
It is hard to avoid that the random number you generated is not used previously
The usual way to do this is to have a (Sharded) Atomic counter. The counter initially has a value of zero. When there is need of an index, an API should be called which will atomically increment this counter and give out its old value.
While this will likely be much faster than the approaches you have mentioned, this may still not be quick enough depending on your needs. The bottleneck in the above situation is the single lock typically used while making the increment atomic. This is not ideal in some distributed situations.
Using Sharded Counters:
The usual way to increase performance in such distributed scenarios is to have sharded counters:
1..160,000,000 into N disjoint ranges).N threads / processes / entities / machines with N different locks.The above will increase performance N-fold and will likely scale to your application needs.
Some interesting reading on sharded counters is at this link.
Note that if you want to use a random number generation (Solution 3), you could optimize looking for existence of a key using Bloom Filters. This may be sufficient depending on your performance needs.