I have a table with the default capacities i.e. 5 RCUs and 5 WCUs. According to the docs, this would result in Dynamo DB creating only one partition.
Table Structure:
I have some item_types with one or two item ids and some with 100,000. We have around 10 million records in total.
Am trying to understand if there is only one partition, how will this create the problem of hot keys? What is a hot key in general?
I know it's an old answer, but I found some useful informations.
As described in Partitions and Data distributions:
DynamoDB allocates additional partitions to a table in the following situations:
- If you increase the table's provisioned throughput settings beyond what the existing partitions can support.
- If an existing partition fills to capacity and more storage space is required.
This means that you can't assume how many partitions you are using. Actually, DynamoDB's docs are never talking about physical partitions. They instead focus on the partitionKey of a table.
If you dig more in the page there are detailed explanation about how dynamoDB uses the partionKey to hash the logical/physical partition.
How to use a partitionKey to avoid hot key?
As described in Designing Partition Keys to Distribute Your Workload Evenly:
The partition key portion of a table's primary key determines the logical partitions in which a table's data is stored. This in turn affects the underlying physical partitions. Provisioned I/O capacity for the table is divided evenly among these physical partitions. Therefore a partition key design that doesn't distribute I/O requests evenly can create "hot" partitions that result in throttling and use your provisioned I/O capacity inefficiently.
That oversimplified means that typically you have to design your partitionKey in order to maximize the partition/record factor.
This isn't always true: for example you can have a large number of record under the same partitionKey, that are almost never read or updated and writes to that partitionKey are rare.
In your case: if you expect to have a lot of reads/writes to the same item_type it's better to model your data differently.
More useful links:
Best Practices for Designing and Using Partition Keys Effectively
Using Write Sharding to Distribute Workloads Evenly