Add cudf::memory_resources for separate temporary/output allocation control - #15
Draft
karthikeyann wants to merge 2 commits into
Draft
Add cudf::memory_resources for separate temporary/output allocation control#15karthikeyann wants to merge 2 commits into
karthikeyann wants to merge 2 commits into
Conversation
…d output allocations This large-scale refactoring introduces the cudf::memory_resources class that enables fine-grained control over memory allocation by separating temporary (intermediate) allocations from output (returned) allocations. Key changes: - Add cudf::memory_resources class in utilities/memory_resource.hpp with: * Two-argument constructor for explicit output and temporary MRs * Single-argument constructor for backward API compatibility * get_output_mr() and get_temporary_mr() accessor methods * Implicit conversion from rmm::device_async_resource_ref - Update 562 files across the codebase: * 197 public API headers converted to accept cudf::memory_resources * 365+ implementation files updated to use resources.get_temporary_mr() * All Thrust exec_policy calls now include memory resource parameter * All rmm::device_uvector and device_buffer allocations updated - Add validation support: * LIBCUDF_ERROR_ON_CURRENT_DEVICE_RESOURCE_REF environment variable * Enables strict checking that resources are threaded through all code paths * Helpful error messages for debugging - Create comprehensive test suite: * memory_resources_tests.cpp with 20+ test cases * memory_resources_validation_tests.cpp with 10+ validation tests * Tests cover constructors, separate pools, tracking, validation mode, edge cases - Add extensive documentation: * Implementation plan and design decisions * Refactoring summary with patterns used * Detailed list of targeted fixes applied * Test documentation and debugging guide * Validation script for verifying completeness Patterns updated throughout codebase: - rmm::device_async_resource_ref mr → cudf::memory_resources resources - cudf::get_current_device_resource_ref() → resources.get_temporary_mr() - rmm::exec_policy(stream) → rmm::exec_policy(stream, resources.get_temporary_mr()) - Function calls: pass entire resources object, not just get_output_mr() Maintains backward API compatibility through implicit conversion while enabling new functionality to use separate memory pools for optimization and profiling. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Owner
Author
|
/ok to test |
…urces-refactoring # Conflicts: # cpp/src/binaryop/compiled/binary_ops.cu # cpp/src/copying/copy.cu # cpp/src/copying/purge_nonempty_nulls.cu # cpp/src/copying/sample.cu # cpp/src/copying/shift.cu # cpp/src/datetime/datetime_ops.cu # cpp/src/dictionary/detail/concatenate.cu # cpp/src/dictionary/detail/merge.cu # cpp/src/dictionary/remove_keys.cu # cpp/src/dictionary/search.cu # cpp/src/dictionary/set_keys.cu # cpp/src/filling/repeat.cu # cpp/src/filling/sequence.cu # cpp/src/groupby/hash/hash_compound_agg_finalizer.cu # cpp/src/groupby/sort/group_collect.cu # cpp/src/groupby/sort/group_correlation.cu # cpp/src/groupby/sort/group_count.cu # cpp/src/groupby/sort/group_count_scan.cu # cpp/src/groupby/sort/group_histogram.cu # cpp/src/groupby/sort/group_m2.cu # cpp/src/groupby/sort/group_merge_lists.cu # cpp/src/groupby/sort/group_nth_element.cu # cpp/src/groupby/sort/group_nunique.cu # cpp/src/groupby/sort/group_quantiles.cu # cpp/src/groupby/sort/group_rank_scan.cu # cpp/src/groupby/sort/group_replace_nulls.cu # cpp/src/groupby/sort/group_std.cu # cpp/src/groupby/sort/sort_helper.cu # cpp/src/hash/md5_hash.cu # cpp/src/hash/murmurhash3_x64_128.cu # cpp/src/hash/murmurhash3_x86_32.cu # cpp/src/hash/xxhash_32.cu # cpp/src/hash/xxhash_64.cu # cpp/src/interop/to_arrow_host.cu # cpp/src/io/avro/reader_impl.cu # cpp/src/io/comp/compression.cu # cpp/src/io/comp/nvcomp_adapter.cu # cpp/src/io/csv/csv_gpu.cu # cpp/src/io/csv/durations.cu # cpp/src/io/json/json_column.cu # cpp/src/io/json/json_tree.cu # cpp/src/io/json/nested_json_gpu.cu # cpp/src/io/json/read_json.cu # cpp/src/io/orc/writer_impl.cu # cpp/src/io/parquet/writer_impl.cu # cpp/src/io/text/multibyte_split.cu # cpp/src/io/utilities/data_casting.cu # cpp/src/join/filtered_join.cu # cpp/src/join/join_utils.cu # cpp/src/join/mixed_join_semi.cu # cpp/src/join/sort_merge_join.cu # cpp/src/labeling/label_bins.cu # cpp/src/lists/combine/concatenate_list_elements.cu # cpp/src/lists/combine/concatenate_rows.cu # cpp/src/lists/contains.cu # cpp/src/lists/copying/concatenate.cu # cpp/src/lists/copying/copying.cu # cpp/src/lists/count_elements.cu # cpp/src/lists/dremel.cu # cpp/src/lists/explode.cu # cpp/src/lists/extract.cu # cpp/src/lists/interleave_columns.cu # cpp/src/lists/reverse.cu # cpp/src/lists/segmented_sort.cu # cpp/src/lists/sequences.cu # cpp/src/lists/set_operations.cu # cpp/src/lists/stream_compaction/apply_boolean_mask.cu # cpp/src/lists/utilities.cu # cpp/src/merge/merge.cu # cpp/src/partitioning/partitioning.cu # cpp/src/partitioning/round_robin.cu # cpp/src/quantiles/quantile.cu # cpp/src/quantiles/tdigest/tdigest.cu # cpp/src/quantiles/tdigest/tdigest_aggregation.cu # cpp/src/reductions/all.cu # cpp/src/reductions/any.cu # cpp/src/reductions/max.cu # cpp/src/reductions/min.cu # cpp/src/reductions/minmax.cu # cpp/src/reductions/nth_element.cu # cpp/src/reductions/scan/ewm.cu # cpp/src/reductions/scan/rank_scan.cu # cpp/src/reductions/scan/scan_exclusive.cu # cpp/src/reductions/scan/scan_inclusive.cu # cpp/src/reductions/segmented/counts.cu # cpp/src/reductions/segmented/nunique.cu # cpp/src/replace/clamp.cu # cpp/src/replace/nans.cu # cpp/src/replace/nulls.cu # cpp/src/reshape/byte_cast.cu # cpp/src/reshape/interleave_columns.cu # cpp/src/rolling/detail/rolling_collect_list.cu # cpp/src/round/round.cu # cpp/src/search/contains_scalar.cu # cpp/src/search/search_ordered.cu # cpp/src/sort/is_sorted.cu # cpp/src/sort/rank.cu # cpp/src/sort/segmented_sort.cu # cpp/src/stream_compaction/distinct_count.cu # cpp/src/stream_compaction/distinct_helpers.cu # cpp/src/stream_compaction/stable_distinct.cu # cpp/src/stream_compaction/unique.cu # cpp/src/stream_compaction/unique_count.cu # cpp/src/stream_compaction/unique_count_column.cu # cpp/src/strings/attributes.cu # cpp/src/strings/capitalize.cu # cpp/src/strings/char_types/char_types.cu # cpp/src/strings/convert/convert_booleans.cu # cpp/src/strings/convert/convert_datetime.cu # cpp/src/strings/convert/convert_durations.cu # cpp/src/strings/convert/convert_fixed_point.cu # cpp/src/strings/convert/convert_floats.cu # cpp/src/strings/convert/convert_hex.cu # cpp/src/strings/convert/convert_integers.cu # cpp/src/strings/convert/convert_ipv4.cu # cpp/src/strings/convert/int_cast.cu # cpp/src/strings/copying/concatenate.cu # cpp/src/strings/copying/copying.cu # cpp/src/strings/copying/shift.cu # cpp/src/strings/like.cu # cpp/src/strings/repeat_strings.cu # cpp/src/strings/reverse.cu # cpp/src/strings/scan/scan_inclusive.cu # cpp/src/strings/search/find.cu # cpp/src/strings/search/find_multiple.cu # cpp/src/strings/slice.cu # cpp/src/strings/split/partition.cu # cpp/src/strings/split/split.cu # cpp/src/strings/split/split_re.cu # cpp/src/strings/strip.cu # cpp/src/strings/utilities.cu # cpp/src/strings/wrap.cu # cpp/src/structs/scan/scan_inclusive.cu # cpp/src/text/deduplicate.cu # cpp/src/text/detokenize.cu # cpp/src/text/jaccard.cu # cpp/src/text/minhash.cu # cpp/src/text/ngrams_tokenize.cu # cpp/src/text/normalize.cu # cpp/src/text/stemmer.cu # cpp/src/text/tokenize.cu # cpp/src/text/vocabulary_tokenize.cu # cpp/src/text/wordpiece_tokenize.cu # cpp/src/transform/mask_to_bools.cu # cpp/src/transform/one_hot_encode.cu # cpp/src/unary/cast_ops.cu # cpp/src/unary/math_ops.cu
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces the
cudf::memory_resourcesclass that enables fine-grained control over memory allocation by separating temporary (intermediate) allocations from output (returned) allocations throughout libcudf.Motivation
Currently, all libcudf APIs use a single memory resource for both output data and temporary allocations. This limits the ability to:
Changes
Core Infrastructure
cudf::memory_resourcesinutilities/memory_resource.hppoutput_mrandtemporary_mrget_output_mr()andget_temporary_mr()rmm::device_async_resource_reffor API compatibilityRefactoring Scope
cudf::memory_resourcesresources.get_temporary_mr()for temporary allocationsdevice_uvectoranddevice_bufferupdated throughoutValidation Infrastructure
LIBCUDF_ERROR_ON_CURRENT_DEVICE_RESOURCE_REFvalidate_refactoring.sh- all checks pass ✅Testing
MEMORY_RESOURCES_TESTS_README.mdDocumentation
API Changes
Backward Compatible ✅
Existing code continues to work due to implicit conversion: