This repository was archived by the owner on Oct 10, 2025. It is now read-only.
Rel scan selection optimizations - #4558
Merged
Merged
Conversation
ray6080
approved these changes
Nov 22, 2024
ray6080
left a comment
Contributor
There was a problem hiding this comment.
LGTM! @andyfengHKU can you also take a quick look into the changes regarding to null mask and aggregation functions and see if there are other places we can apply the changes in similar pattern?
andyfengHKU
approved these changes
Nov 22, 2024
| ~ValueVector() = default; | ||
|
|
||
| template<class Func> | ||
| void forEachNonNull(Func&& func) const { |
Contributor
There was a problem hiding this comment.
Can u open an issue to propagate this to unary, binary & ternary evaluator and assign the issue to @acquamarin
…if possible bitset access is relatively slow, and may not be necessary There are also some optimizations possible to take advantage of not having to scan the SelectionVector's buffer (though I'm not sure if we're actually taking advantage of it right now). And I avoided the unnecessary creation/destruction of a std::shared_ptr in RelTableScanState::setNodeIDVectorToFlat by using getSelVectorUnsafe instead of getSelVectorShared.
benjaminwinger
force-pushed
the
rel-scan-selection-opt
branch
from
November 22, 2024 16:06
0a69983 to
7174804
Compare
Benchmark ResultMaster commit hash:
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #4558 +/- ##
=======================================
Coverage 87.36% 87.36%
=======================================
Files 1349 1349
Lines 56587 56601 +14
Branches 7079 7068 -11
=======================================
+ Hits 49436 49451 +15
+ Misses 6979 6978 -1
Partials 172 172 ☔ View full report in Codecov by Sentry. 🚨 Try these New Features:
|
benjaminwinger
force-pushed
the
rel-scan-selection-opt
branch
from
November 22, 2024 22:25
7174804 to
7b20c24
Compare
Benchmark ResultMaster commit hash:
|
ray6080
pushed a commit
that referenced
this pull request
Dec 18, 2024
* Set the SelectionVector to STATIC_FILTERED when scanning cached rels if possible bitset access is relatively slow, and may not be necessary There are also some optimizations possible to take advantage of not having to scan the SelectionVector's buffer (though I'm not sure if we're actually taking advantage of it right now). And I avoided the unnecessary creation/destruction of a std::shared_ptr in RelTableScanState::setNodeIDVectorToFlat by using getSelVectorUnsafe instead of getSelVectorShared. * Optimize some operator code * Removed an unnecessary std::shared_ptr copy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The biggest optimization here is avoiding using the bitset in
CSRNodeGroup::tryScanCachedTuplesif it's known that all values are selected.In some unfiltered rel table scans this improved performance of
CSRNodeGroup::tryScanCachedTuplesby about 4x, for a total of about a 15% on some queries I'd benchmarked which were mostly scans.E.g. 'MATCH (v:N)-[e:E]->(v2:N) RETURN SUM(v.ID);' on the ldbc graph500-30 dataset (with 128 threads) went from ~2.65s to ~2.25s
I've also:
std::shared_ptrcopies (which are immediately destroyed).ValueVector::forEachNonNullto consolidate some code in the aggregate functions