Releases: facebook/rocksdb
Releases · facebook/rocksdb
RocksDB 4.6.1
Public API Changes
- Change default of BlockBasedTableOptions.format_version to 2. It means default DB created by 4.6 or up cannot be opened by RocksDB version 3.9 or earlier.
- Added strict_capacity_limit option to NewLRUCache. If the flag is set to true, insert to cache will fail if no enough capacity can be free. Signature of Cache::Insert() is updated accordingly.
- Tickers [NUMBER_DB_NEXT, NUMBER_DB_PREV, NUMBER_DB_NEXT_FOUND, NUMBER_DB_PREV_FOUND, ITER_BYTES_READ] are not updated immediately. The are updated when the Iterator is deleted.
- Add monotonically increasing counter (DB property "rocksdb.current-super-version-number") that increments upon any change to the LSM tree.
New Features
- Add CompactionPri::kMinOverlappingRatio, a compaction picking mode friendly to write amplification.
- Deprecate Iterator::IsKeyPinned() and replace it with Iterator::GetProperty() with prop_name="rocksdb.iterator.is.key.pinned"
RocksDB 4.5.1
Public API Changes
- Add a new perf context level between kEnableCount and kEnableTime. Level 2 now does not include timers for mutexes.
- Statistics of mutex operation durations will not be measured by default. If you want to have them enabled, you need to set Statistics::stats_level_ to kAll.
- DBOptions::delete_scheduler and NewDeleteScheduler() are removed, please use DBOptions::sst_file_manager and NewSstFileManager() instead
New Features
- ldb tool now supports operations to non-default column families.
- Add kPersistedTier to ReadTier. This option allows Get and MultiGet to read only the persited data and skip mem-tables if writes were done with disableWAL = true.
- Add DBOptions::sst_file_manager. Use NewSstFileManager() in include/rocksdb/sst_file_manager.h to create a SstFileManager that can be used to track the total size of SST files and control the SST files deletion rate.
RocksDB 4.4.1
New Features
- Dynamic slow-down on writes. When slow down on writes is triggered, RocksDB will dynamically optimize the slow-down per write based on the estimated pending compaction bytes. If the number of pending compaction bytes keeps increasing, then RocksDB will slow down on writes more to make compaction able to catch up to avoid write stall.
- Introduce CompactionJobInfo::compaction_reason, this field include the reason to trigger the compaction.
- Added a new parameter --path to ldb tool. --path accepts the name of either MANIFEST, SST or a WAL file. Either --db or --path can be used when calling ldb.
Public API Changes
- Deprecate options.soft_rate_limit and add options.soft_pending_compaction_bytes_limit.
- If options.max_write_buffer_number > 3, writes will be slowed down when writing to the last write buffer to delay a full stop.
- Increase default options.delayed_write_rate to 2MB/s.
- Change names in CompactionPri and add a new one.
RocksDB 4.3.1
New Features
- CompactionFilter has new member function called IgnoreSnapshots which allows CompactionFilter to be called even if there are snapshots later than the key.
- RocksDB will now persist options under the same directory as the RocksDB database on successful DB::Open, CreateColumnFamily, DropColumnFamily, and SetOptions.
- Introduce LoadLatestOptions() in rocksdb/utilities/options_util.h. This function can construct the latest DBOptions / ColumnFamilyOptions used by the specified RocksDB intance.
- Introduce CheckOptionsCompatibility() in rocksdb/utilities/options_util.h. This function checks whether the input set of options is able to open the specified DB successfully.
Public API Changes
- When options.db_write_buffer_size triggers, only the column family with the largest column family size will be flushed, not all the column families.
RocksDB 4.2
New Features
- Introduce CreateLoggerFromOptions(), this function create a Logger for provided DBOptions.
- Add GetAggregatedIntProperty(), which returns the sum of the GetIntProperty of all the column families.
- Add MemoryUtil in rocksdb/utilities/memory.h. It currently offers a way to get the memory usage by type from a list rocksdb instances.
Public API Changes
- CompactionFilter::Context includes information of Column Family ID
- The need-compaction hint given by TablePropertiesCollector::NeedCompact() will be persistent and recoverable after DB recovery. This introduces a breaking format change. If you use this experimental feature, including NewCompactOnDeletionCollectorFactory() in the new version, you may not be able to directly downgrade the DB back to version 4.0 or lower.
- TablePropertiesCollectorFactory::CreateTablePropertiesCollector() now takes an option Context, containing the information of column family ID for the file being written.
- Remove DefaultCompactionFilterFactory.
RocksDB 4.1
New Features
- Added single delete operation as a more efficient way to delete keys that have not been overwritten.
- Added experimental AddFile() to DB interface that allow users to add files created by SstFileWriter into an empty Database, see include/rocksdb/sst_file_writer.h and DB::AddFile() for more info.
- Added support for opening SST files with .ldb suffix which enables opening LevelDB databases.
- CompactionFilter now supports filtering of merge operands and merge results.
Public API Changes
- Added SingleDelete() to the DB interface.
- Added AddFile() to DB interface.
- Added SstFileWriter class.
- CompactionFilter has a new method FilterMergeOperand() that RocksDB applies to every merge operand during compaction to decide whether to filter the operand.
- We removed CompactionFilterV2 interfaces from include/rocksdb/compaction_filter.h. The functionality was deprecated already in version 3.13.
RocksDB 4.0
New Features
- Added support for transactions. See include/rocksdb/utilities/transaction.h for more info.
- DB::GetProperty() now accepts "rocksdb.aggregated-table-properties" and "rocksdb.aggregated-table-properties-at-levelN", in which case it returns aggregated table properties of the target column family, or the aggregated table properties of the specified level N if the "at-level" version is used.
- Add compression option kZSTDNotFinalCompression for people to experiment ZSTD although its format is not finalized.
- We removed the need for LATEST_BACKUP file in BackupEngine. We still keep writing it when we create new backups (because of backward compatibility), but we don't read it anymore.
Public API Changes
- Removed class Env::RandomRWFile and Env::NewRandomRWFile().
- Renamed DBOptions.num_subcompactions to DBOptions.max_subcompactions to make the name better match the actual functionality of the option.
- Added Equal() method to the Comparator interface that can optionally be overwritten in cases where equality comparisons can be done more efficiently than three-way comparisons.
- Previous 'experimental' OptimisticTransaction class has been replaced by Transaction class.
RocksDB 4.1
New Features
- Added single delete operation as a more efficient way to delete keys that have not been overwritten.
- Added experimental AddFile() to DB interface that allow users to add files created by SstFileWriter into an empty Database, see include/rocksdb/sst_file_writer.h and DB::AddFile() for more info.
- Added support for opening SST files with .ldb suffix which enables opening LevelDB databases.
- CompactionFilter now supports filtering of merge operands and merge results.
Public API Changes
- Added SingleDelete() to the DB interface.
- Added AddFile() to DB interface.
- Added SstFileWriter class.
- CompactionFilter has a new method FilterMergeOperand() that RocksDB applies to every merge operand during compaction to decide whether to filter the operand.
- We removed CompactionFilterV2 interfaces from include/rocksdb/compaction_filter.h. The functionality was deprecated already in version 3.13.
RocksDB 3.13.1
New Features
- RollbackToSavePoint() in WriteBatch/WriteBatchWithIndex
- Add NewCompactOnDeletionCollectorFactory() in utilities/table_properties_collectors, which allows rocksdb to mark a SST file as need-compaction when it observes at least D deletion entries in any N consecutive entries in that SST file. Note that this feature depends on an experimental NeedCompact() API --- the result of this API will not persist after DB restart.
- Add DBOptions::delete_scheduler. Use NewDeleteScheduler() in include/rocksdb/delete_scheduler.h to create a DeleteScheduler that can be shared among multiple RocksDB instances to control the file deletion rate of SST files that exist in the first db_path.
Public API Changes
- Deprecated WriteOptions::timeout_hint_us. We no longer support write timeout. If you really need this option, talk to us and we might consider returning it.
- Deprecated purge_redundant_kvs_while_flush option.
- Removed BackupEngine::NewBackupEngine() and NewReadOnlyBackupEngine() that were deprecated in RocksDB 3.8. Please use BackupEngine::Open() instead.
- Deprecated Compaction Filter V2. We are not aware of any existing use-cases. If you use this filter, your compile will break with RocksDB 3.13. Please let us know if you use it and we'll put it back in RocksDB 3.14.
- Env::FileExists now returns a Status instead of a boolean
- Add statistics::getHistogramString() to print detailed distribution of a histogram metric.
- Add DBOptions::skip_stats_update_on_db_open. When it is on, DB::Open() will run faster as it skips the random reads required for loading necessary stats from SST files to optimize compaction.
RocksDB 3.12.1
New Features
- Added experimental support for optimistic transactions. See include/rocksdb/utilities/optimistic_transaction.h for more info.
- Added a new way to report QPS from db_bench (check out --report_file and --report_interval_seconds)
- Added a cache for individual rows. See DBOptions::row_cache for more info.
- Several new features on EventListener (see include/rocksdb/listener.h):
- OnCompationCompleted() now returns per-compaciton job statistics, defined in include/rocksdb/compaction_job_stats.h.
- Added OnTableFileCreated() and OnTableFileDeleted().
- Add compaction_options_universal.enable_trivial_move to true, to allow trivial move while performing universal compaction. Trivial move will happen only when all the input files are non overlapping.
Public API Changes
- EventListener::OnFlushCompleted() now passes FlushJobInfo instead of a list of parameters.
- DB::GetDbIdentity() is now a const function. If this function is overridden in your application, be sure to also make GetDbIdentity() const to avoid compile error.
- Move listeners from ColumnFamilyOptions to DBOptions.
- Add max_write_buffer_number_to_maintain option
- DB::CompactRange()'s parameter reduce_level is changed to change_level, to allow users to move levels to lower levels if allowed. It can be used to migrate a DB from options.level_compaction_dynamic_level_bytes=false to options.level_compaction_dynamic_level_bytes.true.
- Change default value for options.compaction_filter_factory and options.compaction_filter_factory_v2 to nullptr instead of DefaultCompactionFilterFactory and DefaultCompactionFilterFactoryV2.
- If CancelAllBackgroundWork is called without doing a flush after doing loads with WAL disabled, the changes which haven't been flushed before the call to CancelAllBackgroundWork will be lost.
- WBWIIterator::Entry() now returns WriteEntry instead of
const WriteEntry& - options.hard_rate_limit is deprecated.
- When options.soft_rate_limit or options.level0_slowdown_writes_trigger is triggered, the way to slow down writes is changed to: write rate to DB is limited to to options.delayed_write_rate.
- DB::GetApproximateSizes() adds a parameter to allow the estimation to include data in mem table, with default to be not to include. It is now only supported in skip list mem table.
- DB::CompactRange() now accept CompactRangeOptions instead of multiple paramters. CompactRangeOptions is defined in include/rocksdb/options.h.
- CompactRange() will now skip bottommost level compaction for level based compaction if there is no compaction filter, bottommost_level_compaction is introduced in CompactRangeOptions to control when it's possbile to skip bottommost level compaction. This mean that if you want the compaction to produce a single file you need to set bottommost_level_compaction to BottommostLevelCompaction::kForce.
- Add Cache.GetPinnedUsage() to get the size of memory occupied by entries that are in use by the system.
- DB:Open() will fail if the compression specified in Options is not linked with the binary. If you see this failure, recompile RocksDB with compression libraries present on your system. Also, previously our default compression was snappy. This behavior is now changed. Now, the default compression is snappy only if it's available on the system. If it isn't we change the default to kNoCompression.
- We changed how we account for memory used in block cache. Previously, we only counted the sum of block sizes currently present in block cache. Now, we count the actual memory usage of the blocks. For example, a block of size 4.5KB will use 8KB memory with jemalloc. This might decrease your memory usage and possibly decrease performance. Increase block cache size if you see this happening after an upgrade.
- Add BackupEngineImpl.options_.max_background_operations to specify the maximum number of operations that may be performed in parallel. Add support for parallelized backup and restore.