Merge pull request #124 from assemblerflow/dev

1.2.2 version release

Author: cimendes

.gitignore

@@ -0,0 +1,6 @@
+# ignore ide folders
+.idea/
+.vscode/
+
+# ignore python generated files
+*.pyc

changelog.md

@@ -2,6 +2,54 @@
 
 ## Upcoming release (`dev` branch)
 
+### Components changes
+
+- `mapping_patlas`: refactored to remove temporary files used to create
+sam and bam files and added data to .report.json. Updated databases to pATLAS
+version 1.5.2.
+- `mash_screen` and `mash_dist`: added data to .report.json. Updated databases 
+to pATLAS version 1.5.2.
+
+### New components
+
+- Added component `fasterq_dump`
+- Added component `mash_sketch_fasta`
+- Added component `mash_sketch_fastq`
+- Added component `downsample_fastq` for FastQ read sub sampling using seqtk
+- Added component `momps` for typing of Legionella pneumophila
+- Added component `split_assembly`
+- Added component `mafft`
+- Added component `raxml`
+- Added component `viral_assembly`
+- Added component `progressive_mauve`
+- Added component `dengue_typing`
+
+### Minor/Other changes
+
+- Added check for `params.accessions` that enables to report a proper
+error when it is set to `null`.
+- Added `build` option to export component parameters information in JSON format. 
+- Fixed minor issue preventing the `maxbin2` and `split_assembly` components 
+from being used multiples times in a pipeline
+- Added a catch to the `filter_poly` process for cases where the input file is empty. 
+- spades template now reports the exit code of spades' execution
+
+### Bug fixes
+
+- Removed the need for the nf process templates to have an empty line
+at the beginning of the template files.
+- Fixed issue when the `inspect` mode was executed on a pipeline directory
+with failed processes but with the work directory removed (the log files
+where no longer available).
+- Fixed issue when the `inspect` mode was executed on a pipeline without the 
+memory directory defined.
+- Fixed issue in the `inspect` mode, where there is a rare race condition between
+tags in the log and trace files.
+- Fixed bug on `midas_species` process where the output file was not being 
+linked correctly, causing the process to fail
+- Fixed bug on `bowtie` where the reference parameter was missing the pid
+- Fixed bug on `filter_poly` where the tag was missing
+
 ## 1.2.1
 
 ### Improvements

docs/user/available_components.rst

@@ -14,6 +14,9 @@ Download
 - :doc:`components/reads_download`: Downloads reads from the SRA/ENA public
   databases from a list of accessions.
 
+- :doc:`components/fasterq_dump`: Downloads reads from the SRA public databases
+  from a list of accessions, using ``fasterq-dump``.
+
 Reads Quality Control
 --------------------
 
@@ -35,6 +38,9 @@ Reads Quality Control
 
 - :doc:`components/trimmomatic`: Runs Trimmomatic on paired-end FastQ files.
 
+- :doc:`components/sample_fastq`: Subsamples fastq files up to a target coverage
+  depth.
+
 
 Assembly
 --------
@@ -86,14 +92,18 @@ Distance Estimation
 - :doc:`components/mash_dist`: Executes mash distance against a reference index
   plasmid database and generates a `JSON` for pATLAS. This component calculates
   pairwise distances between sequences (one from the database and the query
-  sequence). However if a
-  different database is provided it can use mash dist for other purposes.
+  sequence). However if a different database is provided it can use mash dist
+  for other purposes.
 
 - :doc:`components/mash_screen`: Performs mash screen against a reference index
   plasmid database and generates a JSON input file for pATLAS. This component
   searches for containment of a given sequence in read sequencing data.
-  However if a different
-  database is provided it can use mash screen for other purporses.
+  However if a different database is provided it can use mash screen for other
+  purporses.
+
+- :doc:`components/mash_sketch_fasta`: Performs mash sketch for fasta files.
+
+- :doc:`components/mash_sketch_fastq`: Performes mash sketch for fastq files.
 
 Mapping
 -------
@@ -142,3 +152,6 @@ Typing
 
 - :doc:`components/sistr`: Serovar predictions from whole-genome sequence assemblies
   by determination of antigen gene and cgMLST gene alleles.
+
+- :doc:`components/momps`: Multi-locus sequence typing for Legionella pneumophila
+  from assemblies and reads.

docs/user/components/fasterq_dump.rst

@@ -0,0 +1,50 @@
+fasterq_dump
+============
+
+Purpose
+-------
+
+This component downloads reads from the SRA public databases from a
+list of accessions. This component uses ``fasterq-dump`` from
+`NCBI sra-tools <https://github.com/ncbi/sra-tools>`_. ``fasterq-dump``
+increases the download speed in comparison from ``fastq-dump`` by
+**multi-threading** the extraction of FASTQ from SRA-accessions.
+The reads for each accession are then emitted through
+the main output of this component to any other component (or components) that
+receive FastQ data.
+
+Input/Output type
+------------------
+
+- Input type: ``accessions``
+- Output type: ``fastq``
+
+.. note::
+    The default input parameter for Accessions data is ``--accessions``.
+
+Parameters
+----------
+
+- ``option_file``: This options enables the *option-file* parameter of
+``fasterq-dump``, allowing parameters to be passed.
+- ``compress_fastq``: This options allows users to disable the compression of
+the fastq files resulting from this component. The default (``true``) behavior
+compresses the fastq files to *fastq.gz*.
+
+Published results
+-----------------
+
+- ``reads/<accession>``: Stores the reads for each provided accession.
+
+Published reports
+-----------------
+
+None.
+
+Default directives
+------------------
+
+- ``cpus``: 1
+- ``memory``: 1GB
+- ``container``: flowcraft/sra-tools
+- ``version``: 2.9.1-1

docs/user/components/mapping_patlas.rst

@@ -17,8 +17,8 @@ can be imported into `pATLAS <http://www.patlas.site/>`_.
 Input/Output type
 ------------------
 
-- Input type: ``FastQ``
-- Output type: ``JSON``
+- Input type: ``fastq``
+- Output type: ``json``
 
 
 Parameters
@@ -59,11 +59,11 @@ Default directives
 ------------------
 
 - ``mappingBowtie``:
-    - ``container``: tiagofilipe12/patlasflow_mapping
-    - ``version``: 1.1.2
+    - ``container``: flowcraft/mapping-patlas
+    - ``version``: 1.4.1
 - ``samtoolsView``:
-    - ``container``: tiagofilipe12/patlasflow_mapping
-    - ``version``: 1.1.2
+    - ``container``: flowcraft/mapping-patlas
+    - ``version``: 1.4.1
 - ``jsonDumpingMapping``:
-    - ``container``: tiagofilipe12/patlasflow_mapping
-    - ``version``: 1.1.2
+    - ``container``: flowcraft/mapping-patlas
+    - ``version``: 1.4.1

docs/user/components/mash_dist.rst

@@ -21,8 +21,8 @@ database than the default that is intended for plasmid searches.
 Input/Output type
 ------------------
 
-- Input type: ``Fasta``
-- Output type: ``JSON``
+- Input type: ``fasta``
+- Output type: ``json``
 
 
 Parameters
@@ -64,8 +64,8 @@ Default directives
 ------------------
 
 - ``runMashDist``:
-    - ``container``: tiagofilipe12/patlasflow_mash_screen
-    - ``version``: 1.1
+    - ``container``: flowcraft/mash-patlas
+    - ``version``: 1.4.1
 - ``mashDistOutputJson``:
-    - ``container``: tiagofilipe12/patlasflow_mash_screen
-    - ``version``: 1.1
+    - ``container``: flowcraft/mash-patlas
+    - ``version``: 1.4.1

docs/user/components/mash_screen.rst

@@ -21,8 +21,8 @@ database than the default that is intended for plasmid searches.
 Input/Output type
 ------------------
 
-- Input type: ``FastQ``
-- Output type: ``JSON``
+- Input type: ``fastq``
+- Output type: ``json``
 
 
 Parameters
@@ -59,8 +59,8 @@ Default directives
 ------------------
 
 - ``mashScreen``:
-    - ``container``: tiagofilipe12/patlasflow_mash_screen
-    - ``version``: 1.1
+    - ``container``: flowcraft/mash-patlas
+    - ``version``: 1.4.1
 - ``mashOutputJson``:
-    - ``container``: tiagofilipe12/patlasflow_mash_screen
-    - ``version``: 1.1
+    - ``container``: flowcraft/mash-patlas
+    - ``version``: 1.4.1

docs/user/components/mash_sketch_fasta.rst

@@ -0,0 +1,47 @@
+mash_sketch_fasta
+=================
+
+Purpose
+-------
+
+This component performs mash sketch for fasta input files.
+
+.. note::
+    - MASH documentation can be found `here <https://mash.readthedocs.io/en/latest/>`_.
+
+
+Input/Output type
+------------------
+
+- Input type: ``fasta``
+- Output type: ``msh``
+
+
+Parameters
+----------
+
+- ``kmerSize``: Parameter to set the kmer size for hashing. Default: 21.
+  Default: false.
+
+- ``sketchSize``: Parameter to set the number of hashes per sketch.
+  Default: 1000.
+
+
+Published results
+-----------------
+
+None.
+
+
+Published reports
+-----------------
+
+None.
+
+
+Default directives
+------------------
+
+- ``mashSketchFasta``:
+    - ``container``: flowcraft/mash-patlas
+    - ``version``: 1.4.1

docs/user/components/mash_sketch_fastq.rst

@@ -0,0 +1,55 @@
+mash_sketch_fastq
+=================
+
+Purpose
+-------
+
+This component performs mash sketch for fastq input files.
+
+.. note::
+    - MASH documentation can be found `here <https://mash.readthedocs.io/en/latest/>`_.
+
+
+Input/Output type
+------------------
+
+- Input type: ``fastq``
+- Output type: ``msh``
+
+
+Parameters
+----------
+
+- ``kmerSize``: Parameter to set the kmer size for hashing. Default: 21.
+  Default: false.
+
+- ``sketchSize``: Parameter to set the number of hashes per sketch.
+  Default: 1000.
+
+- ``minKmer``: Minimum copies of each k-mer required to pass noise filter for
+  reads. Default: 1.
+
+- ``genomeSize``: Genome size (raw bases or with K/M/G/T). If specified, will
+  be used for p-value calculation instead of an estimated size from k-mer
+  content. Default: *false*, meaning that it won't be used. If you want to use
+  it pass a number to this parameter.
+
+
+Published results
+-----------------
+
+None.
+
+
+Published reports
+-----------------
+
+None.
+
+
+Default directives
+------------------
+
+- ``mashSketchFastq``:
+    - ``container``: flowcraft/mash-patlas
+    - ``version``: 1.4.1

docs/user/components/momps.rst

@@ -0,0 +1,53 @@
+momps
+========
+
+Purpose
+-------
+
+This component performs Multi-Locus Sequence Typing (MLST) on Legionella pneumophila
+from reads and assemblies.
+
+.. note::
+    Software page: https://github.com/bioinfo-core-BGU/mompS
+
+Input/Output type
+------------------
+
+- Input type: ``fasta``
+- Output type: None
+
+.. note::
+    The default input parameter for fasta data is ``--fasta``. This process
+    also requires FastQ reads provided via the ``--fastq`` parameter.
+
+Parameters
+----------
+
+None.
+
+Published results
+-----------------
+
+- ``results/typing/momps``: Stores TSV files with the ST and allelic profiles
+  for each strain.
+
+Published reports
+-----------------
+
+None.
+
+Default directives
+------------------
+
+- ``momps``:
+    - ``container``: flowcraft/momps
+    - ``version``: 0.1.0-4
+
+Advanced
+--------
+
+Reports JSON
+^^^^^^^^^^^^
+
+``typing``:
+    - ``momps``: <typing result>