Re-partitioning a state store in a stream processor requires deleting a range of keys.
Consider a DB with key range from [1 - 1000]. Splitting this into two might result in two DBs, one with [1-500], and another with [501-1000].
This can be achieved with following steps
- Clone the DB to DB-1 and DB-2
- Delete ranges [501-1000] from DB-1 and ranges [1-500] from DB-2.
- DB-1 and DB-2 are the new partitions.
Step 2 can be a metadata only operation if we supported delete range that updates the manifest with deleted ranges.
A compaction can then do the actual deletion of keys.
Re-partitioning a state store in a stream processor requires deleting a range of keys.
Consider a DB with key range from [1 - 1000]. Splitting this into two might result in two DBs, one with [1-500], and another with [501-1000].
This can be achieved with following steps
Step 2 can be a metadata only operation if we supported delete range that updates the manifest with deleted ranges.
A compaction can then do the actual deletion of keys.