Confronting #817 head-on, we have a few different problems:

• split-paired-reads.py would drop reads that it couldn't interpret (oops)
• screed by default splits up the record name into 'name' and 'description', which breaks Casava 1.8 names
• this splitting leads to silly results -- e.g. see https://github.com/ged-lab/khmer/blob/fix/casava_formats/tests/test_scripts.py#L2316

It looks to me like the right thing to do is make sure that parse_description is always set to false
in screed iterators. This will necessitate changing about a dozen things in scripts/. @mr-c thoughts on this wrt semantic versioning?

cc @standage

Addresses #763

This fixes the splitting/interleaving issues with Casava 1.8 AFAICS.

The changes might not be fun but if the tests pass without modifications then we are violating the semantic versioning.

Sorry, clarify: yes, fix, and to heck with semver?

On Feb 21, 2015, at 8:26 PM, Michael R. Crusoe notifications@github.com wrote:

The changes might not be fun but if the tests pass without modifications then we are violating the semantic versioning.

—
Reply to this email directly or view it on GitHub.

mea culpa; ----> Then we are not violating semantic versioning if the
tests continue to pass unmodified.

On Sat Feb 21 2015 at 8:34:43 PM C. Titus Brown notifications@github.com
wrote:

Sorry, clarify: yes, fix, and to heck with semver?

On Feb 21, 2015, at 8:26 PM, Michael R. Crusoe notifications@github.com
wrote:

The changes might not be fun but if the tests pass without modifications
then we are violating the semantic versioning.

—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
#818 (comment).

To summarize my understanding: khmer has always mangled sequence record names: either by dropping descriptions or appending different pairing notation on the end.

This PR aims to bring khmer in line with expected practice: both old and new style pair naming are accepted everywhere; descriptions will no longer be dropped.

With respect to semantic versioning we have three options:

A) We declare the old behavior a horrible bug and widely publicize the correction and thus we don't increment the major version number.

or

B) We accept that the old behavior was present for such a long time and likely required workarounds that will break upon its fixing. Therefore we increment the major version to indicate this, thus following our commitment to semantic (and not 'marketing') versioning.

or

C) we hid the corrected behavior behind a switch and wait for the 2.0 release to make it the default behavior

As Titus privately pointed out to me, seeing what changes will be required to update the protocols (if anything) will inform the wisdom of option B or C.

While additional tests are needed, and it's not yet ready for a full CR, it's mostly done, I think? Thoughts welcome. I'll finish sanding down the corners tomorrow or later today, as time permits.

• add tests for filter-abund
• add tests for filter-abund-single
• resolve issues with count-median now allowing spaces in output
• add tests for do-partition
• add tests for annotate-partition and extract-partition
• add tests for filter-stoptags
• add tests for normalize-by-median
• add tests for extract-long-sequences
• add tests for fastq-to-fasta
• add tests for sample-reads-randomly

OK, so it's a slightly bigger issue than I thought :).

• file issue to update file formats to proper CSV, with headers, for khmer 2.0.

• Is it mergable?
• Did it pass the tests?
• If it introduces new functionality in scripts/ is it tested?
  Check for code coverage.
• Is it well formatted? Look at make pep8, make diff_pylint_report,
  make cppcheck, and make doc output. Use make format and manual
  fixing as needed.
• Did it change the command-line interface? Only additions are allowed
  without a major version increment. Changing file formats also requires a
  major version number increment.
• Is it documented in the ChangeLog?
• Was a spellchecker run on the source code and documentation after
  changes were made?

Ready for review, w00t.

Additional issues/bugs found & fixed:

• removed no longer needed test-data files;
• interleave-reads.py used to happily interleave the same file twice; now it errors out.

LGTM!

w00t!