Skip to content
This repository was archived by the owner on Oct 10, 2025. It is now read-only.

Add projected graph with table droping tests - #5073

Merged
andyfengHKU merged 2 commits into
masterfrom
add-projected-graph-tx-tests
Mar 20, 2025
Merged

andyfengHKU merged 2 commits into
masterfrom
add-projected-graph-tx-tests

Conversation

@andyfengHKU

@andyfengHKU andyfengHKU commented Mar 19, 2025

Copy link
Copy Markdown
Contributor

Description

Throw proper error message if any table from projected graph is dropped afterwards. We now store a string version when creating projected graph and on-the-fly bind it for any gds/vector index query.

@andyfengHKU
andyfengHKU requested a review from ray6080 March 19, 2025 12:19
@github-actions

Copy link
Copy Markdown

Benchmark Result

Master commit hash: f8f1517ff8f82d5c6f2a4ed54dad9d6e74d25f8b
Branch commit hash: 1d41d98c91020ca3d60067c7cdd6cc84ebb9e4e6

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 733.59 725.67 7.92 (1.09%)
aggregation q28 6587.26 6581.62 5.63 (0.09%)
filter q14 134.04 125.63 8.42 (6.70%)
filter q15 135.26 129.30 5.96 (4.61%)
filter q16 349.34 341.21 8.14 (2.38%)
filter q17 454.83 445.49 9.33 (2.09%)
filter q18 1968.51 1994.05 -25.54 (-1.28%)
filter zonemap-node 98.21 89.33 8.88 (9.94%)
filter zonemap-node-lhs-cast 97.46 89.82 7.64 (8.51%)
filter zonemap-node-null 96.78 90.71 6.06 (6.68%)
filter zonemap-rel 5847.05 5774.07 72.98 (1.26%)
fixed_size_expr_evaluator q07 686.94 682.75 4.18 (0.61%)
fixed_size_expr_evaluator q08 972.10 966.94 5.16 (0.53%)
fixed_size_expr_evaluator q09 972.96 963.20 9.76 (1.01%)
fixed_size_expr_evaluator q10 262.58 256.12 6.46 (2.52%)
fixed_size_expr_evaluator q11 264.36 255.77 8.59 (3.36%)
fixed_size_expr_evaluator q12 242.13 233.03 9.10 (3.91%)
fixed_size_expr_evaluator q13 1564.14 1563.00 1.15 (0.07%)
fixed_size_seq_scan q23 119.86 114.00 5.86 (5.14%)
join q29 727.60 694.26 33.34 (4.80%)
join q30 1573.76 1584.47 -10.71 (-0.68%)
join q31 6.42 5.51 0.91 (16.46%)
join SelectiveTwoHopJoin 48.96 48.82 0.14 (0.29%)
ldbc_snb_ic q35 10.58 10.32 0.27 (2.58%)
ldbc_snb_ic q36 99.98 93.25 6.72 (7.21%)
ldbc_snb_is q32 5.51 5.24 0.27 (5.12%)
ldbc_snb_is q33 11.58 11.99 -0.41 (-3.45%)
ldbc_snb_is q34 1.16 1.18 -0.02 (-1.98%)
multi-rel multi-rel-large-scan 1778.56 1698.53 80.03 (4.71%)
multi-rel multi-rel-lookup 11.58 11.04 0.54 (4.91%)
multi-rel multi-rel-small-scan 195.79 200.83 -5.05 (-2.51%)
order_by q25 139.42 130.01 9.40 (7.23%)
order_by q26 458.30 448.21 10.09 (2.25%)
order_by q27 1413.56 1406.05 7.51 (0.53%)
recursive_join recursive-join-bidirection 273.88 302.97 -29.09 (-9.60%)
recursive_join recursive-join-dense 5108.18 7154.89 -2046.72 (-28.61%)
recursive_join recursive-join-path 22321.46 23072.60 -751.14 (-3.26%)
recursive_join recursive-join-sparse 629.21 634.47 -5.27 (-0.83%)
recursive_join recursive-join-trail 5731.05 7055.52 -1324.48 (-18.77%)
scan_after_filter q01 177.23 168.48 8.75 (5.19%)
scan_after_filter q02 162.67 153.98 8.69 (5.65%)
shortest_path_ldbc100 q37 85.46 87.20 -1.74 (-1.99%)
shortest_path_ldbc100 q38 327.17 242.00 85.17 (35.19%)
shortest_path_ldbc100 q39 55.36 61.01 -5.65 (-9.26%)
shortest_path_ldbc100 q40 361.99 351.73 10.26 (2.92%)
var_size_expr_evaluator q03 2140.50 2137.31 3.19 (0.15%)
var_size_expr_evaluator q04 2256.21 2283.26 -27.05 (-1.18%)
var_size_expr_evaluator q05 2627.99 2738.81 -110.82 (-4.05%)
var_size_expr_evaluator q06 1383.35 1368.27 15.09 (1.10%)
var_size_seq_scan q19 1459.76 1441.43 18.34 (1.27%)
var_size_seq_scan q20 2761.65 2678.92 82.73 (3.09%)
var_size_seq_scan q21 2308.06 2257.67 50.38 (2.23%)
var_size_seq_scan q22 130.07 126.07 4.00 (3.17%)

#include "storage/index/hnsw_index_utils.h"
#include "storage/index/index_utils.h"
#include "storage/storage_manager.h"
#include <function/gds/gds.h>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#include <function/gds/gds.h>
#include "function/gds/gds.h"


catalog::NodeTableCatalogEntry* nodeTableEntry = nullptr;
const graph::GraphEntry* graphEntry = nullptr;
// const graph::GraphEntry* graphEntry = nullptr;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove this line.

Comment thread src/function/gds/gds.cpp
return bindGraphEntry(context, context.getGraphEntrySetUnsafe().getEntry(name));
}

GraphEntry GDSFunction::bindGraphEntry(ClientContext& context, const ParsedGraphEntry& entry) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be part of GDSFunction? I don't think hnsw should be aware of gds function. Consider moving it to a common class that both gds and hnsw share.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add one more test that a table was dropped, but then another table was created with the same name but not satisfying the requirement of a projected graph?

@andyfengHKU
andyfengHKU force-pushed the add-projected-graph-tx-tests branch from 0ca6626 to 519e569 Compare March 20, 2025 06:33
@github-actions

Copy link
Copy Markdown

Benchmark Result

Master commit hash: 035d769c19ada3231292d11782e25fb91e9abd30
Branch commit hash: f347bf1b66bb14223ae597535acb7bc40c02d74d

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 737.25 727.73 9.52 (1.31%)
aggregation q28 6582.89 6604.17 -21.28 (-0.32%)
filter q14 126.90 128.43 -1.52 (-1.19%)
filter q15 130.77 126.70 4.08 (3.22%)
filter q16 348.83 345.41 3.42 (0.99%)
filter q17 452.09 452.68 -0.59 (-0.13%)
filter q18 1924.43 1949.36 -24.93 (-1.28%)
filter zonemap-node 90.63 90.02 0.60 (0.67%)
filter zonemap-node-lhs-cast 89.43 90.50 -1.08 (-1.19%)
filter zonemap-node-null 89.13 89.69 -0.55 (-0.62%)
filter zonemap-rel 5788.22 5744.03 44.19 (0.77%)
fixed_size_expr_evaluator q07 683.34 698.19 -14.85 (-2.13%)
fixed_size_expr_evaluator q08 979.49 981.13 -1.64 (-0.17%)
fixed_size_expr_evaluator q09 979.39 983.84 -4.45 (-0.45%)
fixed_size_expr_evaluator q10 257.18 267.85 -10.67 (-3.98%)
fixed_size_expr_evaluator q11 257.96 267.95 -10.00 (-3.73%)
fixed_size_expr_evaluator q12 234.32 245.41 -11.09 (-4.52%)
fixed_size_expr_evaluator q13 1596.61 1584.68 11.94 (0.75%)
fixed_size_seq_scan q23 110.76 119.30 -8.54 (-7.16%)
join q29 736.79 740.95 -4.16 (-0.56%)
join q30 1617.34 1540.55 76.79 (4.98%)
join q31 6.83 6.94 -0.11 (-1.54%)
join SelectiveTwoHopJoin 47.31 46.39 0.92 (1.99%)
ldbc_snb_ic q35 10.19 10.83 -0.64 (-5.92%)
ldbc_snb_ic q36 113.13 87.01 26.11 (30.01%)
ldbc_snb_is q32 3.20 3.14 0.06 (1.83%)
ldbc_snb_is q33 14.23 15.02 -0.79 (-5.26%)
ldbc_snb_is q34 1.19 1.27 -0.08 (-6.16%)
multi-rel multi-rel-large-scan 1756.76 2193.26 -436.50 (-19.90%)
multi-rel multi-rel-lookup 6.63 5.96 0.67 (11.28%)
multi-rel multi-rel-small-scan 206.51 206.27 0.25 (0.12%)
order_by q25 127.73 131.38 -3.65 (-2.78%)
order_by q26 457.25 459.82 -2.57 (-0.56%)
order_by q27 1384.62 1436.37 -51.75 (-3.60%)
recursive_join recursive-join-bidirection 279.52 294.70 -15.18 (-5.15%)
recursive_join recursive-join-dense 7165.75 7029.06 136.69 (1.94%)
recursive_join recursive-join-path 23522.45 23512.55 9.89 (0.04%)
recursive_join recursive-join-sparse 639.85 625.41 14.44 (2.31%)
recursive_join recursive-join-trail 7114.56 7010.56 104.00 (1.48%)
scan_after_filter q01 170.85 174.52 -3.67 (-2.10%)
scan_after_filter q02 156.19 157.97 -1.79 (-1.13%)
shortest_path_ldbc100 q37 90.60 93.86 -3.27 (-3.48%)
shortest_path_ldbc100 q38 311.31 257.21 54.10 (21.04%)
shortest_path_ldbc100 q39 61.94 42.27 19.68 (46.55%)
shortest_path_ldbc100 q40 415.90 388.87 27.03 (6.95%)
var_size_expr_evaluator q03 2140.92 2121.33 19.59 (0.92%)
var_size_expr_evaluator q04 2227.38 2234.13 -6.75 (-0.30%)
var_size_expr_evaluator q05 2739.98 2718.97 21.02 (0.77%)
var_size_expr_evaluator q06 1392.93 1359.25 33.68 (2.48%)
var_size_seq_scan q19 1443.42 1454.39 -10.97 (-0.75%)
var_size_seq_scan q20 2651.73 2724.23 -72.51 (-2.66%)
var_size_seq_scan q21 2301.84 2299.94 1.90 (0.08%)
var_size_seq_scan q22 127.43 128.33 -0.90 (-0.70%)

@andyfengHKU
andyfengHKU force-pushed the add-projected-graph-tx-tests branch from 1cd3bb6 to 364df19 Compare March 20, 2025 07:15
@github-actions

Copy link
Copy Markdown

Benchmark Result

Master commit hash: 035d769c19ada3231292d11782e25fb91e9abd30
Branch commit hash: 45da066dcdb961811a2d4d57be3868c6606ffef5

Query Group Query Name Mean Time - Commit (ms) Mean Time - Master (ms) Diff
aggregation q24 739.36 727.73 11.63 (1.60%)
aggregation q28 6576.00 6604.17 -28.17 (-0.43%)
filter q14 127.70 128.43 -0.73 (-0.57%)
filter q15 131.40 126.70 4.71 (3.72%)
filter q16 347.22 345.41 1.81 (0.52%)
filter q17 453.53 452.68 0.85 (0.19%)
filter q18 1915.31 1949.36 -34.05 (-1.75%)
filter zonemap-node 90.91 90.02 0.88 (0.98%)
filter zonemap-node-lhs-cast 88.40 90.50 -2.10 (-2.32%)
filter zonemap-node-null 88.27 89.69 -1.42 (-1.58%)
filter zonemap-rel 5769.75 5744.03 25.72 (0.45%)
fixed_size_expr_evaluator q07 692.91 698.19 -5.28 (-0.76%)
fixed_size_expr_evaluator q08 979.34 981.13 -1.80 (-0.18%)
fixed_size_expr_evaluator q09 981.64 983.84 -2.21 (-0.22%)
fixed_size_expr_evaluator q10 258.99 267.85 -8.86 (-3.31%)
fixed_size_expr_evaluator q11 260.97 267.95 -6.99 (-2.61%)
fixed_size_expr_evaluator q12 236.95 245.41 -8.46 (-3.45%)
fixed_size_expr_evaluator q13 1591.40 1584.68 6.72 (0.42%)
fixed_size_seq_scan q23 113.20 119.30 -6.09 (-5.11%)
join q29 784.12 740.95 43.17 (5.83%)
join q30 1592.81 1540.55 52.26 (3.39%)
join q31 6.26 6.94 -0.68 (-9.76%)
join SelectiveTwoHopJoin 48.56 46.39 2.18 (4.69%)
ldbc_snb_ic q35 10.83 10.83 0.00 (0.02%)
ldbc_snb_ic q36 97.63 87.01 10.61 (12.20%)
ldbc_snb_is q32 3.93 3.14 0.79 (25.31%)
ldbc_snb_is q33 15.20 15.02 0.18 (1.19%)
ldbc_snb_is q34 1.68 1.27 0.42 (33.00%)
multi-rel multi-rel-large-scan 1762.29 2193.26 -430.97 (-19.65%)
multi-rel multi-rel-lookup 5.83 5.96 -0.13 (-2.21%)
multi-rel multi-rel-small-scan 178.15 206.27 -28.12 (-13.63%)
order_by q25 133.99 131.38 2.61 (1.99%)
order_by q26 460.79 459.82 0.97 (0.21%)
order_by q27 1407.83 1436.37 -28.54 (-1.99%)
recursive_join recursive-join-bidirection 305.86 294.70 11.16 (3.79%)
recursive_join recursive-join-dense 7190.00 7029.06 160.95 (2.29%)
recursive_join recursive-join-path 23274.43 23512.55 -238.12 (-1.01%)
recursive_join recursive-join-sparse 643.25 625.41 17.84 (2.85%)
recursive_join recursive-join-trail 7100.11 7010.56 89.55 (1.28%)
scan_after_filter q01 171.47 174.52 -3.05 (-1.75%)
scan_after_filter q02 156.54 157.97 -1.44 (-0.91%)
shortest_path_ldbc100 q37 90.89 93.86 -2.98 (-3.17%)
shortest_path_ldbc100 q38 312.98 257.21 55.77 (21.68%)
shortest_path_ldbc100 q39 59.11 42.27 16.84 (39.84%)
shortest_path_ldbc100 q40 397.88 388.87 9.02 (2.32%)
var_size_expr_evaluator q03 2128.37 2121.33 7.05 (0.33%)
var_size_expr_evaluator q04 2219.65 2234.13 -14.48 (-0.65%)
var_size_expr_evaluator q05 2757.41 2718.97 38.45 (1.41%)
var_size_expr_evaluator q06 1402.67 1359.25 43.42 (3.19%)
var_size_seq_scan q19 1456.63 1454.39 2.24 (0.15%)
var_size_seq_scan q20 2692.30 2724.23 -31.94 (-1.17%)
var_size_seq_scan q21 2308.10 2299.94 8.16 (0.35%)
var_size_seq_scan q22 127.59 128.33 -0.74 (-0.58%)

@andyfengHKU
andyfengHKU merged commit a573d70 into master Mar 20, 2025
@andyfengHKU
andyfengHKU deleted the add-projected-graph-tx-tests branch March 20, 2025 09:17
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants