Skip to content

gemini - load chunk error with LSF scheduler #918

Open

Description

@Fazulur

Dear Team,

I tried to run gemini database using LSF as scheduler and specifying tmp directory to some other location where I have more space.
Command:

gemini load --cores 64 --scheduler lsf --queue normal --tempdir /home/fazulur/tmp/ -t snpEff -v test.vcf.gz test.db

gemini load_chunks function is trying to create chunks in the below path which is not exists. temp directory path is /home/fazulur/tmp/ and my current working directory is /home/fazulur/create-gemini-db/.
/home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db

Here is the Error:

2019-07-17 08:53:41.998 [IPClusterStop] Stopping cluster [pid=5089] with [signal=2]
Traceback (most recent call last):
File "$TOOl_DIR/gemini/v0.30.1/tools/bin/gemini", line 7, in
gemini_main.main()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1249, in main
args.func(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 204, in load_fn
gemini_load.load(parser, args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 47, in load
load_ipython(args)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 99, in load_ipython
chunks = load_chunks_ipython(grabix_file, args, view)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 339, in load_chunks_ipython
chunk_dbs = view.map(load_chunk, chunk_steps, [load_args] * total_chunks)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-137>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 52, in sync_results
ret = f(self, *args, **kwargs)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-136>", line 2, in map
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 37, in save_ids
ret = f(self, *args, **kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/view.py", line 1114, in map
return pf.map(*sequences)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 299, in map
return self(*sequences, __ipp_mapping=True)
File "<$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/decorator.pyc:decorator-gen-119>", line 2, in call
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 80, in sync_view_results
return f(self, *args, **kwargs)
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/remotefunction.py", line 285, in call
return r.get()
File "$TOOl_DIR/gemini/v0.30.1/data/anaconda/lib/python2.7/site-packages/ipyparallel/client/asyncresult.py", line 169, in get
raise self.exception()
ipyparallel.error.CompositeError: one or more exceptions from call to method: load_chunk
[18:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 1 123256 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 1 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk0.db' returned non-zero exit status 1
[24:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 123257 246512 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 123257 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk1.db' returned non-zero exit status 1
[22:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 246513 369768 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 246513 /home/fazulur/tmp///home/fazulur/create-gemini-db/test.vcf.chunk2.db' returned non-zero exit status 1
[38:apply]: CalledProcessError: Command 'grabix grab test.vcf.gz 369769 493024 | gemini load_chunk -v - -t snpEff --skip-gerp-bp --skip-cadd --skip-info-string --tempdir /home/fazulur/tmp -o 369769 /home/fazulur/tmp//home/fazulur/create-gemini-db/test.vcf.chunk3.db' returned non-zero exit status 1
.... 46 more exceptions ...

Could you please help me to resolve this issue.

Thanks In Advance
Fazulur Rehaman

Activity

  1. changed the title [-]gemini - load chunk error due to incorrect path[/-] [+]gemini - load chunk error with LSF scheduler[/+] on Jul 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions