Error in rule spades --No module named 'distutils'-- | Error! The Snakemake workflow aborted. #41
Description
Hi, akcorut:
Thanks for the pipeline.
After running the pipeline, it aborted with the following errors: 1)Error in rule spades 2)Error! The Snakemake workflow aborted.
==>cat -n run2.log | grep -A 4 -E "Error|error"
29434 Error in rule spades:
29435 jobid: 1334
29436 input: results/fetch_reads_with_kmers/Gly/merged_reads/Gly_reads_with_kmers.merged.sorted.R1.fastq, results/fetch_reads_with_kmers/Gly/merged_reads/Gly_reads_with_kmers.merged.sorted.R2.fastq
29437 output: results/assemble_reads_with_kmers/Gly/assembly/contigs.fasta, results/assemble_reads_with_kmers/Gly/assembly/assembly_graph_with_scaffolds.gfa, results/assemble_reads_with_kmers/Gly/assembling_source_reads.done
29438 log: logs/assemble_reads/Gly/assemble_reads.spades.log (check log file(s) for error details)
29439 conda-env: /disk/disk1/xuan/kmerFlow_451/.snakemake/conda/93fe6492c51dbf0e4f6310c59c861ef8_
29440 shell:
29441
29442 spades.py --careful --only-assembler -t 8 --pe1-1 results/fetch_reads_with_kmers/Gly/merged_reads/Gly_reads_with_kmers.merged.sorted.R1.fastq --pe1-2 results/fetch_reads_with_kmers/Gly/merged_reads/Gly_reads_with_kmers.merged.sorted.R2.fastq -o results/assemble_reads_with_kmers/Gly/assembly 2> logs/assemble_reads/Gly/assemble_reads.spades.log
----------
30244 Exiting because a job execution failed. Look above for error message
30245 Complete log: .snakemake/log/2024-11-07T113757.764885.snakemake.log
30246 Checking if all required files are provided...
30247 All required files are present!
30248 Error! The Snakemake workflow aborted.
30249 ERROR:root:Error running Snakemake: Command '['snakemake', '--use-conda', '--conda-frontend', 'conda', '--cores', '16', '--snakefile', '/disk/disk2/work/xuan/anaconda3/envs/kgwasflow/lib/python3.11/site-packages/workflow/Snakefile', '--rerun-triggers', 'mtime', 'params', 'input', 'software-env', 'code']' returned non-zero exit status 1.
For the first errr: 1) Error in rule spades
I checked the file "assemble_reads.spades.log" and it indicated "No module named 'distutils'"
==>cat logs/assemble_reads/Gly/assemble_reads.spades.log
/disk/disk1/xuan/kmerFlow_451/.snakemake/conda/93fe6492c51dbf0e4f6310c59c861ef8_/share/spades/spades_pipeline/support.py:508: SyntaxWarning: invalid escape sequence '\d'
return [atoi(c) for c in re.split("(\d+)", text)]
Traceback (most recent call last):
File "/disk/disk1/xuan/kmerFlow_451/.snakemake/conda/93fe6492c51dbf0e4f6310c59c861ef8_/bin/spades.py", line 26, in <module>
import support
File "/disk/disk1/xuan/kmerFlow_451/.snakemake/conda/93fe6492c51dbf0e4f6310c59c861ef8_/share/spades/spades_pipeline/support.py", line 23, in <module>
from distutils.version import LooseVersion
ModuleNotFoundError: No module named 'distutils'
After searching the solution for "No module named 'distutils'", it said the the problem might come with new python 3.12, ( mine is 3.11.10 ). Followed the solution:
checking if I had distutils: it showed that I had the module
==>python -m distutils.util
<frozen runpy>:128: RuntimeWarning: 'distutils.util' found in sys.modules after import of package 'distutils', but prior to execution of 'distutils.util'; this may result in unpredictable behaviour
Installing setuptools: it showed that I already have the setuptools.
==>pip install setuptools
Requirement already satisfied: setuptools in /disk/disk2/work/xuan/anaconda3/envs/kgwasflow/lib/python3.11/site-packages (75.3.0)
==>pip install --upgrade setuptools
Requirement already satisfied: setuptools in /disk/disk2/work/xuan/anaconda3/envs/kgwasflow/lib/python3.11/site-packages (75.3.0)
So I don't know how to deal with this problem, would you please kindly give me some suggestions ?
For the second error: may be not a problem after deal with the first error
I have two more questions:
1: Can I start the pipeline at middle steps ? For example, can I just start at where I met the above errors ? and how if yes?
2: How the pipeline extract reads with significant associated k-mers ?
eg:
I found a kmer in kmers/Gly_kmers_list.fa
==>cat fetch_kmers/Gly_kmers_list.fa
>kmer1_6.37946e-11
ACCCCGAAGCCACCGGCTACCAAGCCGCCCA
then I use grep above-kmer acc1.clean_1.fq.gz; I got just one read sequence
when grep above-kmer fetch_reads_with_kmers/Gly/individual_reads/acc1_reads_with_kmers_R1.fastq, I also got just one read sequence; but there are 5 reads in the file acc1_reads_with_kmers_R1.fastq.
So I was wondering how the pipeline found 4 more reads.
Thanks a lot again !