In the Cassandra compound PK approach, the first index is hashed which determines the partition position, and the other indices are appended to the index which can be used for range queries. This works well in practice since the first index is typically an ID where we don’t do range queries on, whereas subsequent columns can be columns where range queries are useful ((user_id, upload_timestamp)).
Methods
- Key range partitioning:
- Here, keys are sorted and a partition owns all the keys from some minimum up to some maximum.
- Range queries are possible with sorting but there is a risk of hotspots
- Partitions are rebalanced by splitting a range into 2 sub ranges when the partition gets too big
- Hash partitioning:
- A hash function is applied to each key and a partition owns a range of hashes
- This disallows range queries but is better for distributing load
- There are hybrid approaches for using things like compound keys where you use an identifier as a hash index for distributing and then a secondary index for range queries as mentioned above
Rebalancing
The main strategy here is to create many more partitions than nodes and when new nodes join, they take a partition from each node until everything is balanced. The entire partitions moves with this strategy. Partitions can also be split and reassigned if they get too large with dynamic partitioning.
Partitioning vs Secondary Index
- Document-partitioned indexes are when secondary indexes are stored in the same partition as the primary key and value
- This means that only one partition needs to be updated on a write but a range query requires a scatter gather
- Term-partitioned indexes are when the secondary indexes are partitioned separately
- This has the opposite pattern where on write, several partitions need to be written to, but a range query only queries one partition
How do we know what partition to send a user to? See Request Routing