Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Redis 7.0.3 crashed by signal: 11, si_code: 128 while using search #5031

Open
adihegde23 opened this issue Sep 18, 2024 · 9 comments
Open
Assignees
Labels

Comments

@adihegde23
Copy link

adihegde23 commented Sep 18, 2024

Describe the bug
Redis cluster seems to be crashing while performing a search operation from the stack .

REDIS BUG REPORT
=== REDIS BUG REPORT START: Cut & paste starting from here ===
1152207:M 12 Sep 2024 07:21:09.634 # Redis 7.0.3 crashed by signal: 11, si_code: 128
1152207:M 12 Sep 2024 07:21:09.634 # Accessing address: (nil)
1152207:M 12 Sep 2024 07:21:09.634 # Crashed running the instruction at: 0x7f631e1b765e

------ STACK TRACE ------
EIP:
/usr/local/bin/redis-modules-RediSearchv2.6.12_Ubuntuv22.so(+0x1b765e)[0x7f631e1b765e]

Backtrace:
/lib/x86_64-linux-gnu/libc.so.6(+0x42520)[0x7f631fa42520]
/usr/local/bin/redis-modules-RediSearchv2.6.12_Ubuntuv22.so(+0x1b765e)[0x7f631e1b765e]
/usr/local/bin/redis-modules-RediSearchv2.6.12_Ubuntuv22.so(+0x1e7d2f)[0x7f631e1e7d2f]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](moduleNotifyKeyspaceEvent+0xec)[0x556a29fe6d2c]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](notifyKeyspaceEvent+0x32)[0x556a29fce2b2]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](RM_NotifyKeyspaceEvent+0x32)[0x556a29fdb352]
/usr/local/bin/redis-modules-RedisJSONv2.4.7_Ubuntuv22.so(+0x10c052)[0x7f6311d0c052]
/usr/local/bin/redis-modules-RedisJSONv2.4.7_Ubuntuv22.so(+0x4a6d7)[0x7f6311c4a6d7]
/usr/local/bin/redis-modules-RedisJSONv2.4.7_Ubuntuv22.so(+0xf3b47)[0x7f6311cf3b47]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](RedisModuleCommandDispatcher+0x4a)[0x556a29fe368a]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](call+0xdf)[0x556a29f3e14f]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](processCommand+0x79d)[0x556a29f3f51d]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](processInputBuffer+0xe6)[0x556a29f58996]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](readQueryFromClient+0x2f8)[0x556a29f5c998]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](+0x14eb60)[0x556a2a010b60]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](aeProcessEvents+0x1ea)[0x556a29f34f3a]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](aeMain+0x1d)[0x556a29f352fd]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](main+0x330)[0x556a29f308c0]
/lib/x86_64-linux-gnu/libc.so.6(+0x29d90)[0x7f631fa29d90]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80)[0x7f631fa29e40]
/usr/local/bin/redis-server 127.0.0.1:26379 [cluster](_start+0x25)[0x556a29f30fa5]

------ REGISTERS ------
1152207:M 12 Sep 2024 07:21:09.634 # 
RAX:00007f62ff00c18c RBX:0000000000000001
RCX:0000000000000000 RDX:00007f62ff00c19c
RDI:01f4000c01080156 RSI:0000000020010000
RBP:00007f630c2ac1a0 RSP:00007ffceaca2480
R8 :00007ffcead80000 R9 :00ea9b65d951caa7
R10:00007ffcead7e080 R11:00007ffcead7e080
R12:0000000000000000 R13:00007ffceaca2530
R14:00007f62fe8e0610 R15:0000000000000001
RIP:00007f631e1b765e EFL:0000000000010206
CSGSFS:002b000000000033
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca248f) -> 0000000000000000
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca248e) -> 00007f62fe8e0610
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca248d) -> 00007f631f463b28
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca248c) -> 00007ffceaca2520
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca248b) -> 0000000000002000
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca248a) -> 0000000000003fff
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2489) -> 00007f631e1e7d2f
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2488) -> 00007ffceaca2530
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2487) -> 00007f62fe8e0610
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2486) -> 00007f62fe8e0610
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2485) -> 00007f631f463b28
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2484) -> 00007f630c2acd60
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2483) -> 0000000000002000
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2482) -> 0000000000000000
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2481) -> 00007f62fe8e0610
1152207:M 12 Sep 2024 07:21:09.635 # (00007ffceaca2480) -> 00007f631f463b28

------ INFO OUTPUT ------
# Server
redis_version:7.0.3
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:2826e8c26ad19d13
redis_mode:cluster
os:Linux 6.2.0-1018-azure x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:11.4.0
process_id:1152207
process_supervised:systemd
run_id:f3adfc7ccdfc50ebe875971c7ef2cbfa65a9f1c3
tcp_port:26379
server_time_usec:1726125669633553
uptime_in_seconds:524607
uptime_in_days:6
hz:10
configured_hz:10
lru_clock:14849637
executable:/usr/local/bin/redis-server
config_file:/etc/redis/redis.conf
io_threads_active:0

# Clients
connected_clients:27
cluster_connections:10
maxclients:25000
client_recent_max_input_buffer:20480
client_recent_max_output_buffer:20504
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:112989344
used_memory_human:107.76M
used_memory_rss:180723712
used_memory_rss_human:172.35M
used_memory_peak:131251040
used_memory_peak_human:125.17M
used_memory_peak_perc:86.09%
used_memory_overhead:28791904
used_memory_startup:2733144
used_memory_dataset:84197440
used_memory_dataset_perc:76.37%
allocator_allocated:115340152
allocator_active:155561984
allocator_resident:162471936
total_system_memory:8315179008
total_system_memory_human:7.74G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:4572839936
maxmemory_human:4.26G
maxmemory_policy:allkeys-lru
allocator_frag_ratio:1.35
allocator_frag_bytes:40221832
allocator_rss_ratio:1.04
allocator_rss_bytes:6909952
rss_overhead_ratio:1.11
rss_overhead_bytes:18251776
mem_fragmentation_ratio:1.60
mem_fragmentation_bytes:67765336
mem_not_counted_for_evict:2048
mem_replication_backlog:23108296
mem_total_replication_buffers:23108008
mem_clients_slaves:0
mem_clients_normal:191904
mem_cluster_links:10880
mem_aof_buffer:2048
mem_allocator:jemalloc-5.2.1
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:7
rdb_bgsave_in_progress:0
rdb_last_save_time:1726125153
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
rdb_saves:291
rdb_last_cow_size:2670592
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:2420
aof_enabled:1
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:0
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:1
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:1695744
module_fork_in_progress:0
module_fork_last_cow_size:0
aof_current_size:23176198
aof_base_size:885645
aof_pending_rewrite:0
aof_buffer_length:0
aof_pending_bio_fsync:0
aof_delayed_fsync:0

# Stats
total_connections_received:45202
total_commands_processed:4357843
instantaneous_ops_per_sec:5
total_net_input_bytes:102167150
total_net_output_bytes:2256527212
total_net_repl_input_bytes:0
total_net_repl_output_bytes:23020625
instantaneous_input_kbps:0.07
instantaneous_output_kbps:0.02
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:1
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:6240
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:679562
keyspace_misses:836
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:3116
total_forks:313
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:33552
dump_payload_sanitizations:0
total_reads_processed:2786613
total_writes_processed:2290588
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:2096
reply_buffer_expands:1776

# Replication
role:master
connected_slaves:1
slave0:ip=10.12.72.140,port=6379,state=online,offset=32424892651,lag=1
master_failover_state:no-failover
master_replid:df299e2cf44ef0c21d141fb57f7e07ceb9543fb1
master_replid2:49a5227797250730a48f8ecadb44e649bae6a1dd
master_repl_offset:32424892651
second_repl_offset:32401872027
repl_backlog_active:1
repl_backlog_size:415236096
repl_backlog_first_byte_offset:32401872027
repl_backlog_histlen:23020625

# CPU
used_cpu_sys:542.618821
used_cpu_user:1384.187093
used_cpu_sys_children:4.653139
used_cpu_user_children:36.887668
used_cpu_sys_main_thread:523.625587
used_cpu_user_main_thread:1300.265560

# Modules
module:name=search,ver=20612,api=1,filters=0,usedby=[],using=[ReJSON],options=[]
module:name=ReJSON,ver=20407,api=1,filters=0,usedby=[search],using=[],options=[handle-io-errors]

# Commandstats
cmdstat_config|set:calls=94,usec=11821,usec_per_call=125.76,rejected_calls=0,failed_calls=15
cmdstat_config|get:calls=44527,usec=7706564,usec_per_call=173.08,rejected_calls=0,failed_calls=0
cmdstat_del:calls=1437,usec=33089,usec_per_call=23.03,rejected_calls=0,failed_calls=0
cmdstat_json.set:calls=49386,usec=3430827,usec_per_call=69.47,rejected_calls=33510,failed_calls=0
cmdstat_set:calls=1783,usec=26057,usec_per_call=14.61,rejected_calls=0,failed_calls=0
cmdstat_cluster|myid:calls=524506,usec=1527547,usec_per_call=2.91,rejected_calls=0,failed_calls=0
cmdstat_cluster|nodes:calls=4,usec=339,usec_per_call=84.75,rejected_calls=0,failed_calls=0
cmdstat_cluster|info:calls=44497,usec=2105467,usec_per_call=47.32,rejected_calls=0,failed_calls=0
cmdstat_cluster|slots:calls=734667,usec=26950163,usec_per_call=36.68,rejected_calls=0,failed_calls=0
cmdstat_sdiff:calls=7,usec=29040,usec_per_call=4148.57,rejected_calls=0,failed_calls=0
cmdstat_exists:calls=291,usec=1046,usec_per_call=3.59,rejected_calls=0,failed_calls=0
cmdstat_info:calls=44530,usec=15164334,usec_per_call=340.54,rejected_calls=1,failed_calls=0
cmdstat_sadd:calls=3828,usec=24707,usec_per_call=6.45,rejected_calls=0,failed_calls=0
cmdstat_client|setname:calls=45015,usec=103829,usec_per_call=2.31,rejected_calls=0,failed_calls=0
cmdstat_hmset:calls=694,usec=19643,usec_per_call=28.30,rejected_calls=0,failed_calls=0
cmdstat_FT.DROPINDEX:calls=5,usec=3603,usec_per_call=720.60,rejected_calls=0,failed_calls=2
cmdstat_search.CLUSTERREFRESH:calls=524506,usec=42608820,usec_per_call=81.24,rejected_calls=0,failed_calls=0
cmdstat_ping:calls=1572371,usec=2289672,usec_per_call=1.46,rejected_calls=1,failed_calls=0
cmdstat_psync:calls=1,usec=80,usec_per_call=80.00,rejected_calls=0,failed_calls=0
cmdstat_dbsize:calls=2,usec=4,usec_per_call=2.00,rejected_calls=0,failed_calls=0
cmdstat_FT.SEARCH:calls=897,usec=4154026,usec_per_call=4631.02,rejected_calls=0,failed_calls=1
cmdstat_latency|latest:calls=44480,usec=83578,usec_per_call=1.88,rejected_calls=0,failed_calls=0
cmdstat_latency|histogram:calls=44480,usec=26774719,usec_per_call=601.95,rejected_calls=0,failed_calls=0
cmdstat__FT.DROPINDEX:calls=11,usec=632,usec_per_call=57.45,rejected_calls=0,failed_calls=7
cmdstat_FT.CREATE:calls=2,usec=2721,usec_per_call=1360.50,rejected_calls=0,failed_calls=0
cmdstat_auth:calls=45192,usec=246818,usec_per_call=5.46,rejected_calls=0,failed_calls=0
cmdstat_get:calls=295,usec=1578,usec_per_call=5.35,rejected_calls=3,failed_calls=3
cmdstat_hgetall:calls=16224,usec=141143,usec_per_call=8.70,rejected_calls=0,failed_calls=0
cmdstat_hset:calls=568,usec=11032,usec_per_call=19.42,rejected_calls=0,failed_calls=0
cmdstat_replconf:calls=521887,usec=815965,usec_per_call=1.56,rejected_calls=0,failed_calls=0
cmdstat_module|load:calls=2,usec=157,usec_per_call=78.50,rejected_calls=0,failed_calls=2
cmdstat_slowlog|get:calls=44478,usec=281052,usec_per_call=6.32,rejected_calls=0,failed_calls=0
cmdstat_slowlog|len:calls=44478,usec=76287,usec_per_call=1.72,rejected_calls=0,failed_calls=0
cmdstat_smembers:calls=4,usec=2834,usec_per_call=708.50,rejected_calls=0,failed_calls=0
cmdstat__FT.CREATE:calls=6,usec=1665,usec_per_call=277.50,rejected_calls=0,failed_calls=0
cmdstat__FT.SEARCH:calls=2688,usec=4013964,usec_per_call=1493.29,rejected_calls=0,failed_calls=6

# Errorstats
errorstat_ERR:count=21
errorstat_MOVED:count=33510
errorstat_NOAUTH:count=2
errorstat_Syntax:count=4
errorstat_Unknown:count=9
errorstat_WRONGTYPE:count=3
errorstat_idx_audits_:count=1
errorstat_idx_facilities_:count=2

# Latencystats
latency_percentiles_usec_config|set:p50=3.007,p99=3555.327,p99.9=3588.095
latency_percentiles_usec_config|get:p50=163.839,p99=346.111,p99.9=1187.839
latency_percentiles_usec_del:p50=13.055,p99=83.455,p99.9=274.431
latency_percentiles_usec_json.set:p50=50.175,p99=372.735,p99.9=667.647
latency_percentiles_usec_set:p50=14.015,p99=30.079,p99.9=44.031
latency_percentiles_usec_cluster|myid:p50=3.007,p99=6.015,p99.9=17.023
latency_percentiles_usec_cluster|nodes:p50=83.455,p99=102.399,p99.9=102.399
latency_percentiles_usec_cluster|info:p50=46.079,p99=84.479,p99.9=428.031
latency_percentiles_usec_cluster|slots:p50=34.047,p99=77.311,p99.9=139.263
latency_percentiles_usec_sdiff:p50=4653.055,p99=5144.575,p99.9=5144.575
latency_percentiles_usec_exists:p50=3.007,p99=8.031,p99.9=15.039
latency_percentiles_usec_info:p50=323.583,p99=720.895,p99.9=2932.735
latency_percentiles_usec_sadd:p50=5.023,p99=27.007,p99.9=59.135
latency_percentiles_usec_client|setname:p50=2.007,p99=11.007,p99.9=27.007
latency_percentiles_usec_hmset:p50=28.031,p99=50.175,p99.9=59.135
latency_percentiles_usec_FT.DROPINDEX:p50=720.895,p99=1327.103,p99.9=1327.103
latency_percentiles_usec_search.CLUSTERREFRESH:p50=78.335,p99=162.815,p99.9=288.767
latency_percentiles_usec_ping:p50=1.003,p99=3.007,p99.9=10.047
latency_percentiles_usec_psync:p50=80.383,p99=80.383,p99.9=80.383
latency_percentiles_usec_dbsize:p50=2.007,p99=2.007,p99.9=2.007
latency_percentiles_usec_FT.SEARCH:p50=1212.415,p99=45875.199,p99.9=63176.703
latency_percentiles_usec_latency|latest:p50=2.007,p99=4.015,p99.9=22.015
latency_percentiles_usec_latency|histogram:p50=561.151,p99=1327.103,p99.9=3997.695
latency_percentiles_usec__FT.DROPINDEX:p50=15.039,p99=181.247,p99.9=181.247
latency_percentiles_usec_FT.CREATE:p50=729.087,p99=1810.431,p99.9=1810.431
latency_percentiles_usec_auth:p50=5.023,p99=16.063,p99.9=49.151
latency_percentiles_usec_get:p50=5.023,p99=13.055,p99.9=84.479
latency_percentiles_usec_hgetall:p50=9.023,p99=19.071,p99.9=42.239
latency_percentiles_usec_hset:p50=15.039,p99=35.071,p99.9=187.391
latency_percentiles_usec_replconf:p50=2.007,p99=3.007,p99.9=9.023
latency_percentiles_usec_module|load:p50=74.239,p99=83.455,p99.9=83.455
latency_percentiles_usec_slowlog|get:p50=6.015,p99=14.015,p99.9=24.063
latency_percentiles_usec_slowlog|len:p50=1.003,p99=9.023,p99.9=18.047
latency_percentiles_usec_smembers:p50=325.631,p99=1212.415,p99.9=1212.415
latency_percentiles_usec__FT.CREATE:p50=259.071,p99=385.023,p99.9=385.023
latency_percentiles_usec__FT.SEARCH:p50=255.999,p99=16908.287,p99.9=23592.959

# Cluster
cluster_enabled:1

# Keyspace
db0:keys=39663,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=129 addr=10.12.72.127:52320 laddr=10.12.72.129:6379 fd=51 name= age=524599 idle=9 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=123 addr=10.12.72.110:47850 laddr=10.12.72.129:6379 fd=37 name= age=524602 idle=524602 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2
id=502851 addr=10.240.160.17:14232 laddr=10.12.72.129:6379 fd=64 name= age=79564 idle=279 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=json.set user=default redir=-1 resp=2
id=162 addr=10.238.160.31:40550 laddr=10.12.72.129:6379 fd=57 name= age=524574 idle=13 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=592999 addr=10.12.72.13:56784 laddr=10.12.72.129:6379 fd=34 name= age=23 idle=22 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2
id=132 addr=10.238.160.39:58798 laddr=10.12.72.129:6379 fd=53 name= age=524597 idle=26 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=143 addr=10.12.72.129:47866 laddr=10.12.72.129:6379 fd=55 name= age=524588 idle=2 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=20 addr=10.12.72.127:52306 laddr=10.12.72.129:6379 fd=45 name= age=524606 idle=524606 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2
id=136 addr=10.12.72.110:47864 laddr=10.12.72.129:6379 fd=54 name= age=524595 idle=14 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=188 addr=10.238.160.30:45790 laddr=10.12.72.129:6379 fd=59 name= age=524551 idle=3 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=203 addr=10.12.72.128:52162 laddr=10.12.72.129:6379 fd=60 name= age=524539 idle=11 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=577646 addr=10.240.96.118:20310 laddr=10.12.72.129:6379 fd=62 name= age=13494 idle=142 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=json.set user=default redir=-1 resp=2
id=181 addr=10.238.160.36:49134 laddr=10.12.72.129:6379 fd=58 name= age=524557 idle=15 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=126 addr=10.12.72.140:60452 laddr=10.12.72.129:6379 fd=49 name= age=524600 idle=44 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=10 addr=10.12.72.140:41204 laddr=10.12.72.129:6379 fd=26 name= age=524607 idle=1 flags=S db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=1 omem=20504 tot-mem=42776 events=r cmd=replconf user=default redir=-1 resp=2
id=11 addr=10.12.72.140:60440 laddr=10.12.72.129:6379 fd=31 name= age=524607 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2
id=18 addr=10.12.72.128:36726 laddr=10.12.72.129:6379 fd=43 name= age=524606 idle=45 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=_FT.SEARCH user=default redir=-1 resp=2
id=131 addr=10.12.72.13:33090 laddr=10.12.72.129:6379 fd=52 name= age=524598 idle=6 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=19 addr=10.12.72.140:60444 laddr=10.12.72.129:6379 fd=44 name= age=524606 idle=524606 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=auth user=default redir=-1 resp=2
id=24 addr=10.238.160.24:41206 laddr=10.12.72.129:6379 fd=48 name= age=524605 idle=45 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=577632 addr=10.22.38.162:23477 laddr=10.12.72.129:6379 fd=61 name= age=13504 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=1008 qbuf-free=19466 argv-mem=977 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=23281 events=r cmd=json.set user=default redir=-1 resp=2
id=12 addr=10.12.72.127:52290 laddr=10.12.72.129:6379 fd=32 name= age=524607 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2
id=532077 addr=10.12.72.13:33876 laddr=10.12.72.129:6379 fd=47 name= age=53762 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2
id=150 addr=10.238.160.33:44580 laddr=10.12.72.129:6379 fd=56 name= age=524583 idle=31 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=cluster|slots user=default redir=-1 resp=2
id=21 addr=10.12.72.129:40350 laddr=10.12.72.129:6379 fd=46 name= age=524606 idle=2 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2
id=13 addr=10.12.72.128:54002 laddr=10.12.72.129:6379 fd=33 name= age=524607 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2
id=15 addr=10.12.72.129:40334 laddr=10.12.72.129:6379 fd=41 name= age=524606 idle=412 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=1800 events=r cmd=_FT.SEARCH user=default redir=-1 resp=2
id=16 addr=10.12.72.110:49242 laddr=10.12.72.129:6379 fd=40 name= age=524606 idle=2 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=1024 rbp=7 obl=0 oll=0 omem=0 tot-mem=22272 events=r cmd=ping user=default redir=-1 resp=2

------ CURRENT CLIENT INFO ------
id=577632 addr=10.22.38.162:23477 laddr=10.12.72.129:6379 fd=61 name= age=13504 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=1008 qbuf-free=19466 argv-mem=977 multi-mem=0 rbs=1024 rbp=0 obl=0 oll=0 omem=0 tot-mem=23281 events=r cmd=json.set user=default redir=-1 resp=2
argv[0]: '"JSON.SET"'
argv[1]: '"prod:auditqueue:20603538"'
argv[2]: '"."'
argv[3]: '"{\"requestId\":20603538,\"requestDate\":\"Sep 12, 2024\",\"auditType\":\"FOLLOW UP AUDIT\",\"supplierName\":\"YAKJIN TRADING CORPORATION\",\"fa"'
1152207:M 12 Sep 2024 07:21:09.635 # key 'prod:auditqueue:20603538' found in DB containing the following object:
1152207:M 12 Sep 2024 07:21:09.635 # Object type: 5
1152207:M 12 Sep 2024 07:21:09.635 # Object encoding: 0
1152207:M 12 Sep 2024 07:21:09.635 # Object refcount: 1

------ MODULES INFO OUTPUT ------
@meiravgri
Copy link
Collaborator

Hi @adihegde23,
Could you please attach the full log including modules info output?

Were you able to observe a consistent crash when issuing this command with these specific parameters?
Also, please note that the latest version for 2.6 major is 2.6.22. It might be a bug that was already solved.

@adihegde23
Copy link
Author

Hi @meiravgri , Well thats all that was printed in the logs. before it could print entire bug report redis was restarted.

@meiravgri
Copy link
Collaborator

stack translate:

0x1b765e is in Indexes_DeleteMatchingWithSchemaRules (/build/src/spec.c:2779).
0x1e7d2f is in HashNotificationCallback (/build/src/notifications.c:101).

Did you try using latest 2.6?

@adihegde23
Copy link
Author

@meiravgri we didnt try using the latest but i have one more crash report this time a detailed one i will upload it .
redis.log

@meiravgri
Copy link
Collaborator

meiravgri commented Sep 26, 2024

Can you please provide steps to reproduce?
Thanks

@adihegde23
Copy link
Author

@meiravgri couple of questions ..

1> how did you do that stack translate ?
0x1b765e is in Indexes_DeleteMatchingWithSchemaRules (/build/src/spec.c:2779).
0x1e7d2f is in HashNotificationCallback (/build/src/notifications.c:101).

2> CURRENT CLIENT INFO section in bug report shows FT.CREATE command , which has some issue. does this mean that client crashed while running this instruction?

3> Steps to repoduce are unknown yet.

@meiravgri
Copy link
Collaborator

Hello @adihegde23

  1. It's an internal script.
  2. Yes, this is the command that caused the crash.
  3. Please share when you have more details.

In addition, I recommend upgrading to the latest version.

@adihegde23
Copy link
Author

@meiravgri surely i will share more details when i have . currently looking to fix that faulty queries and monitor .. anything else i can do to get to root this issue ? attaching GDB etc?

@muvelus
Copy link

muvelus commented Oct 22, 2024

@meiravgri Is there a way or a configuration parameter to detect Syntax errors and prevent crashes in Search module?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants