-
Notifications
You must be signed in to change notification settings - Fork 96
Description
I have a tokumx database(single server). Today this db started crashing after I restarted it.( press ctrl + c to shutdown database)
[root@xxxxx bin]# ./tokumx --port 28005 --dbpath /data/tokulogdb
Thu Oct 29 19:59:30.905 nssize is a deprecated parameter
Thu Oct 29 19:59:30.908 [initandlisten] TokuMX starting : pid=11554 port=28005 dbpath=/data/tokulogdb 64-bit host=xxxxx
Thu Oct 29 19:59:30.908 [initandlisten] TokuMX mongod server v2.0.2-mongodb-2.4.10, using TokuKV rev unknown
Thu Oct 29 19:59:30.908 [initandlisten] git version: unknown
Thu Oct 29 19:59:30.908 [initandlisten] build info: Linux vps-centos5-x64-05.ci.percona.com 2.6.32-042stab106.6 #1 SMP Mon Apr 20 14:48:47 MSK 2015 x86_64 x86_64 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49
Thu Oct 29 19:59:30.908 [initandlisten] options: { dbpath: "/data/tokulogdb", port: 28005 }
Thu Oct 29 19:59:30.910 [initandlisten] [tokumx] startup
Thu Oct 29 19:59:30.911
Thu Oct 29 19:59:30.911 ================================================================================
Thu Oct 29 19:59:30.911 Fatal error detected
Thu Oct 29 19:59:30.911 ================================================================================
Thu Oct 29 19:59:30.911
Thu Oct 29 19:59:30.911 About to gather debugging information, please include all of the following along
Thu Oct 29 19:59:30.912 with logs from other servers in the cluster in a bug report.
Thu Oct 29 19:59:30.912
Thu Oct 29 19:59:30.912 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.912 Version info:
Thu Oct 29 19:59:30.912
Thu Oct 29 19:59:30.912 tokumxVersion: 2.0.2
Thu Oct 29 19:59:30.912 gitVersion: unknown
Thu Oct 29 19:59:30.912 tokukvVersion: unknown
Thu Oct 29 19:59:30.912 tokumxAuditVersion: unknown
Thu Oct 29 19:59:30.913 sysInfo: Linux vps-centos5-x64-05.ci.percona.com 2.6.32-042stab106.6 #1 SMP Mon Apr 20 14:48:47 MSK 2015 x86_64 x86_64 x86_64 GNU/Linux BOOST_LIB_VERSION=1_49
Thu Oct 29 19:59:30.913 loaderFlags:
Thu Oct 29 19:59:30.913 compilerFlags: -fPIC -fno-strict-aliasing -ggdb -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -pipe -Wnon-virtual-dtor -Woverloaded-virtual -Wno-unused-local-typedefs -fno-builtin-memcmp -O3
Thu Oct 29 19:59:30.913 debug: false
Thu Oct 29 19:59:30.913
Thu Oct 29 19:59:30.913 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.913 Simple stacktrace:
Thu Oct 29 19:59:30.914
0xb62596 0x847dd4 0x849eff 0x6f284e 0x7f5c0d9f4130 0x7f5c0e24e3ac 0x7f5c0e2dfae0 0x7865ef 0x6fa097 0x6fa73f 0x6da40c 0x7f5c0ca08af5 0x6f1fad
./tokumx(_ZN5mongo15printStackTraceERSo+0x26) [0xb62596]
./tokumx() [0x847dd4]
./tokumx(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x1f) [0x849eff]
./tokumx(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x24e) [0x6f284e]
/lib64/libpthread.so.0(+0xf130) [0x7f5c0d9f4130]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1a3ac) [0x7f5c0e24e3ac]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0xabae0) [0x7f5c0e2dfae0]
./tokumx(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x38f) [0x7865ef]
./tokumx(_ZN5mongo14_initAndListenEi+0x407) [0x6fa097]
./tokumx(_ZN5mongo13initAndListenEi+0xf) [0x6fa73f]
./tokumx(main+0x2bc) [0x6da40c]
/lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5c0ca08af5]
./tokumx() [0x6f1fad]
Thu Oct 29 19:59:30.918
Thu Oct 29 19:59:30.918 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.919 TokuKV engine backtrace:
Thu Oct 29 19:59:30.919
Thu Oct 29 19:59:30.919 Backtrace: (Note: toku_do_assert=0x0x7f5c0e02cca0)
Thu Oct 29 19:59:30.922 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokuportability.so(Z27db_env_do_backtrace_errfuncPFvPKviPKczES0+0x1f) [0x7f5c0e02e55f]
Thu Oct 29 19:59:30.922 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1447d) [0x7f5c0e24847d]
Thu Oct 29 19:59:30.922 ./tokumx() [0x847eed]
Thu Oct 29 19:59:30.922 ./tokumx(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x25) [0x849f05]
Thu Oct 29 19:59:30.922 ./tokumx(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x24e) [0x6f284e]
Thu Oct 29 19:59:30.923 /lib64/libpthread.so.0(+0xf130) [0x7f5c0d9f4130]
Thu Oct 29 19:59:30.923 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1a3ac) [0x7f5c0e24e3ac]
Thu Oct 29 19:59:30.923 /root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0xabae0) [0x7f5c0e2dfae0]
Thu Oct 29 19:59:30.923 ./tokumx(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x38f) [0x7865ef]
Thu Oct 29 19:59:30.923 ./tokumx(_ZN5mongo14_initAndListenEi+0x407) [0x6fa097]
Thu Oct 29 19:59:30.923 ./tokumx(_ZN5mongo13initAndListenEi+0xf) [0x6fa73f]
Thu Oct 29 19:59:30.923 ./tokumx(main+0x2bc) [0x6da40c]
Thu Oct 29 19:59:30.923 /lib64/libc.so.6(__libc_start_main+0xf5) [0x7f5c0ca08af5]
Thu Oct 29 19:59:30.924 ./tokumx() [0x6f1fad]
Thu Oct 29 19:59:30.924 Engine status function not available
Thu Oct 29 19:59:30.924 Memory usage:
Arena 0:
system bytes = 0
in use bytes = 0
Total (incl. mmap):
system bytes = 0
in use bytes = 0
max mmap regions = 0
max mmap bytes = 0
Backtrace: (Note: toku_do_assert=0x0x7f5c0e02cca0)
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokuportability.so(_Z19db_env_do_backtraceP8_IO_FILE+0x1b)[0x7f5c0e02c19b]
./tokumx[0x847eed]
./tokumx(_ZN5mongo13dumpCrashInfoERKNS_10StringDataE+0x25)[0x849f05]
./tokumx(_ZN5mongo24abruptQuitWithAddrSignalEiP7siginfoPv+0x24e)[0x6f284e]
/lib64/libpthread.so.0(+0xf130)[0x7f5c0d9f4130]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0x1a3ac)[0x7f5c0e24e3ac]
/root/tokumx-e-2.0.2-linux-x86_64/bin/../lib64/libtokufractaltree.so(+0xabae0)[0x7f5c0e2dfae0]
./tokumx(_ZN5mongo7storage7startupEPNS_16TxnCompleteHooksEPNS0_14UpdateCallbackE+0x38f)[0x7865ef]
./tokumx(_ZN5mongo14_initAndListenEi+0x407)[0x6fa097]
./tokumx(_ZN5mongo13initAndListenEi+0xf)[0x6fa73f]
./tokumx(main+0x2bc)[0x6da40c]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f5c0ca08af5]
./tokumx[0x6f1fad]
Engine status function not available
Memory usage:
Arena 0:
system bytes = 0
in use bytes = 0
Total (incl. mmap):
system bytes = 0
in use bytes = 0
max mmap regions = 0
max mmap bytes = 0
Thu Oct 29 19:59:30.931
Thu Oct 29 19:59:30.931 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.931 Crash reason:
Thu Oct 29 19:59:30.931
Thu Oct 29 19:59:30.932 Invalid access at address: 0x7f5c187d6073 from thread: initandlisten
Thu Oct 29 19:59:30.932
Thu Oct 29 19:59:30.932 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.932 Process info:
Thu Oct 29 19:59:30.932
Thu Oct 29 19:59:30.932 OS: Linux CentOS Linux release 7.1.1503 (Core) Kernel 3.10.0-229.7.2.el7.x86_64 x86_64
Thu Oct 29 19:59:30.932 NCPU: 4
Thu Oct 29 19:59:30.932 VIRT: 196 MB
Thu Oct 29 19:59:30.933 RES: 196 MB
Thu Oct 29 19:59:30.933 PHYS: 7566 MB
Thu Oct 29 19:59:30.933 RLIMIT_CORE: 0 (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_CPU: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_DATA: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_FSIZE: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_NOFILE: 65535 (soft), 65535 (hard)
Thu Oct 29 19:59:30.933 RLIMIT_STACK: 8388608 (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 RLIMIT_AS: unlimited (soft), unlimited (hard)
Thu Oct 29 19:59:30.933 _SC_OPEN_MAX: 65535
Thu Oct 29 19:59:30.934 _SC_PAGESIZE: 4096
Thu Oct 29 19:59:30.934 _SC_PHYS_PAGES: 1937060
Thu Oct 29 19:59:30.934 _SC_AVPHYS_PAGES: 261335
Thu Oct 29 19:59:30.934 _SC_NPROCESSORS_CONF: 4
Thu Oct 29 19:59:30.934 _SC_NPROCESSORS_ONLN: 4
Thu Oct 29 19:59:30.934
Thu Oct 29 19:59:30.934 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.934 Parsed server options:
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 dbpath: "/data/tokulogdb"
Thu Oct 29 19:59:30.935 port: 28005
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.935 Filesystem information:
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 Information for dbpath "/data/tokulogdb":
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 type magic: 0x0000EF53
Thu Oct 29 19:59:30.935 type: unknown
Thu Oct 29 19:59:30.935 bsize: 4096
Thu Oct 29 19:59:30.935 blocks: 128982382
Thu Oct 29 19:59:30.935 bfree: 24651634
Thu Oct 29 19:59:30.935 bavail: 18093951
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.935 Current operations in progress:
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.935 { opid: 0, active: false, op: "none", ns: "", query: {}, desc: "initandlisten", threadId: "0x7f5c0e965840", rootTxnid: 0, waitingForLock: false, lockStats: { timeLockedMicros: {}, timeAcquiringMicros: {} } }
Thu Oct 29 19:59:30.935
Thu Oct 29 19:59:30.936 --------------------------------------------------------------------------------
Thu Oct 29 19:59:30.936 OpDebug info:
Thu Oct 29 19:59:30.936
Thu Oct 29 19:59:30.936 none keyUpdates:0 0ms
Thu Oct 29 19:59:30.936
Thu Oct 29 19:59:30.936 Got signal: 11 (Segmentation fault).
Segmentation fault