Skip to content

bad_alloc with leveldb driver #13

Description

@donhcd

I tried doing a batch index of 100k small documents and got this:

2016-01-20T03:50:03.179441003Z terminate called after throwing an instance of 'std::bad_alloc'
2016-01-20T03:50:03.181435150Z   what():  std::bad_alloc
2016-01-20T03:50:03.186041100Z SIGABRT: abort
2016-01-20T03:50:03.188148518Z PC=0x7f066b5b7cc9 m=4
2016-01-20T03:50:03.188171572Z signal arrived during cgo execution
2016-01-20T03:50:03.188181591Z
2016-01-20T03:50:03.188188687Z goroutine 14 [syscall, locked to thread]:
2016-01-20T03:50:03.188828071Z runtime.cgocall(0x404400, 0xc820050c98, 0xc800000000)
2016-01-20T03:50:03.189058384Z  /usr/local/go/src/runtime/cgocall.go:120 +0x11b fp=0xc820050c50 sp=0xc820050c20
2016-01-20T03:50:03.190074963Z github.com/jmhodges/levigo._Cfunc_leveldb_writebatch_put(0x7f065c000910, 0xc82042d000, 0x13, 0xc82042d013, 0xb)
2016-01-20T03:50:03.190125547Z  github.com/jmhodges/levigo/_obj/_cgo_gotypes.go:872 +0x35 fp=0xc820050c98 sp=0xc820050c50
2016-01-20T03:50:03.190141271Z github.com/jmhodges/levigo.(*WriteBatch).Put(0xc9849200f0, 0xc82042d000, 0x13, 0x1000, 0xc82042d013, 0xb, 0xfed)
2016-01-20T03:50:03.190148213Z  /go/src/github.com/jmhodges/levigo/batch.go:51 +0x6e fp=0xc820050cc8 sp=0xc820050c98
2016-01-20T03:50:03.190419659Z github.com/blevesearch/blevex/leveldb.(*Batch).Set(0xc8b550c020, 0xc82042d000, 0x13, 0x1000, 0xc82042d013, 0xb, 0xfed)
2016-01-20T03:50:03.190619153Z  /go/src/github.com/blevesearch/blevex/leveldb/batch.go:24 +0x61 fp=0xc820050d08 sp=0xc820050cc8
2016-01-20T03:50:03.190880265Z github.com/blevesearch/bleve/index/upside_down.(*UpsideDownCouch).batchRows(0xc820088b00, 0x7f066d9ce0a8, 0xc8cc5e5f50, 0xc98e71c000, 0x52661a, 0x651200, 0xc93045a000, 0x249fb, 0x2d600, 0xc820051240, ...)
2016-01-20T03:50:03.190970751Z  /go/src/github.com/blevesearch/bleve/index/upside_down/upside_down.go:167 +0x862 fp=0xc820051188 sp=0xc820050d08
2016-01-20T03:50:03.191209473Z github.com/blevesearch/bleve/index/upside_down.(*UpsideDownCouch).Batch(0xc820088b00, 0xc8d7206470, 0x0, 0x0)
2016-01-20T03:50:03.191233138Z  /go/src/github.com/blevesearch/bleve/index/upside_down/upside_down.go:823 +0xc1b fp=0xc820051690 sp=0xc820051188
2016-01-20T03:50:03.191242246Z github.com/blevesearch/bleve.(*indexImpl).Batch(0xc820108af0, 0xc8e3ee36e0, 0x0, 0x0)
2016-01-20T03:50:03.191987345Z  /go/src/github.com/blevesearch/bleve/index_impl.go:310 +0x122 fp=0xc8200516f8 sp=0xc820051690
... application code stack trace

for now I've worked around this by splitting my batch into smaller batches, but it'd be nice if the driver (or bleve?) could handle doing this for me

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions