release MeTA v2.4.2

Bug fixes
- Properly shuffle documents when doing an even-split classification test
- Make forward indexer listen to `indexer-num-threads` config option.
- Use correct number of threads when deciding block sizes for
    `parallel_for`
- Add workaround to `filesystem::remove_all` for Windows systems to avoid
    spurious failures caused by virus scanners keeping files open after we
    deleted them
- Fix invalid memory access in `gzstreambuf::underflow`

release MeTA v2.4.1

Bug fixes
- Eliminate excess warnings on Darwin about double preprocessor definitions
- Fix issue finding `config.h` when used as a sub-project via
    add_subdirectory()

release MeTA v2.4.0

New features
- Add a minimal perfect hashing implementation for `language_model`, and unify
  the querying interface with the existing language model.
- Add a CMake `install()` command to install MeTA as a library (issue meta-toolkit#143). For
  example, once the library is installed, users can do:

    find_package(MeTA 2.4 REQUIRED)

    add_executable(my-program src/my_program.cpp)
    target_link_libraries(my-program meta-index) # or whatever other libs you
    need from MeTA
- Feature selection functionality added to `multiclass_dataset` and
  `binary_dataset` and views (issues meta-toolkit#111, meta-toolkit#149 and PR meta-toolkit#150 thanks to @siddshuk).

    auto selector = features::make_selector(*config, training_vw);
    uint64_t total_features_selected = 20;
    selector->select(total_features_selected);
    auto filtered_dset = features::filter_dataset(dset, *selector);

- Users can now, similar to `hash_append`, declare standalone functions in the
  same scope as their type called `packed_read` and `packed_write` which will be
  called by `io::packed::read` and `io::packed::write`, respectively, via
  argument-dependent lookup.

Bug fixes
- Fix edge-case bug in the succinct data structures
- Fix off-by-one error in `lm::diff`

Enhancements
- Added functionality to the `meta::hashing` library: `hash_append` overload for
  `std::vector`, manually-seeded hash function
- Further isolate ICU in MeTA to allow CMake to `install()`
- Updates to EWS (UIUC) build guide
- Add `std::vector` operations to `io::packed`
- Consolidated all variants of chunk iterators into one template
- Add MeTA's citation to the README!

Release MeTA v2.3.0

New features
- Forward and inverted indexes are now stored in one directory. **To make
    use of your existing indexes, you will need to move their
    directories.** For example, a configuration that used to look like the
    following

    dataset = "20newsgroups"
    corpus = "line.toml"
    forward-index = "20news-fwd"
    inverted-index = "20news-inv"

    will now look like the following

    dataset = "20newsgroups"
    corpus = "line.toml"
    index = "20news-index"

    and your folder structure should now look like

    20news-index
    ├── fwd
    └── inv

    You can do this by simply moving the old folders around like so:

    mkdir 20news-index
    mv 20news-fwd 20news-index/fwd
    mv 20news-inv 20news-index/inv

- `stats::multinomial` now can report the number of unique event types
    counted (`unique_events()`)
- `std::vector` can now be hashed via `hash_append`.

Bug fixes
- Fix rounding bug in language model-based rankers. This bug caused
    severely degraded performance for these rankers with short queries. The
    unit tests have been improved to prevent such a regression in the
    future.

Enhancements
- The bundled ICU version has been bumped to ICU 57.1.
- MeTA will now attempt to build its own version of ICU on Windows if it
    fails to find a suitable ICU installed.
- CI support for GCC 6.x was added for all three major platforms.
- CI support also uses a fixed version of LLVM/libc++ instead of trunk.

Release MeTA v2.2.0

New features
- Parallelized versions of PageRank and Personalized PageRank have been
  added. A demo is available in `wiki-page-rank`; see the website for
  more information on obtaining the required data.
- Add a disk-based streaming minimal perfect hash function library. A
  sub-component of this is a small memory-mapped succinct data structure
  library for answering rank/select queries on bit vectors.
- Much of our CMake magic has been moved into a separate project included
  as a submodule: https://github.com/meta-toolkit/meta-cmake, which can
  now be used in other projects to simplify initial build system
  configuration.

Bug fixes
- Fix parameter settings in language model rankers not being range checked
  (issue meta-toolkit#134).
- Fix incorrect incoming edge insertion in `directed_graph::add_edge()`.
- Fix `find_first_of` and `find_last_of` in `util::string_view`.

Enhancements
- `forward_index` now knows how to tokenize a document down to a
  `feature_vector`, provided it was generated with a non-LIBSVM analyzer.
- Allow loading of an existing index where its corpus is no longer
  available.
- Data is no longer shuffled in `batch_train`. Shuffling the data
  causes horrible access patterns in the postings file, so the data
  should instead shuffled before indexing.
- `util::array_view`s can now be constructed as empty.
- `util::multiway_merge` has been made more generic. You can now specify
  both the comparison function and merging criteria as parameters, which
  default to `operator<` and `operator==`, respectively.
- A simple utility classes `io::mifstream` and `io::mofstream` have been
  added for places where a moveable `ifstream` or `ofstream` is desired
  as a workaround for older standard libraries lacking these move
  constructors.
- The number of indexing threads can be controlled via the configuration
  key `indexer-num-threads` (which defaults to the number of threads on
  the system), and the number of threads allowed to concurrently write to
  disk can be controlled via `indexer-max-writers` (which defaults to 8).

Release MeTA v2.1.0

New features
- Add the [GloVe algorithm](http://www-nlp.stanford.edu/pubs/glove.pdf) for
  training word embeddings and a library class `word_embeddings` for loading and
  querying trained embeddings. To facilitate returning word embeddings, a simple
  `util::array_view` class was added.
- Add simple vector math library (and move `fastapprox` into the `math`
  namespace).

Bug fixes
- Fix `probe_map::extract()` for `inline_key_value_storage` type; old
  implementation forgot to delete all sentinel values before returning the
  vector.
- Fix incorrect definition of `l1norm()` in `sgd_model`.
- Fix `gmap` calculation where 0 average precision was ignored
- Fix progress output in `multiway_merge`.

Enhancements
- Improve performance of `printing::progress`. Before, `progress::operator()` in
  tight loops could dramatically hurt performance, particularly due to frequent
  calls to `std::chrono::steady_clock::now()`. Now, `progress::operator()`
  simply sets an atomic iteration counter and a background thread periodically
  wakes to update the progress output.
- Allow full text storage in index as metadata field. If `store-full-text =
  true` (default false) in the corpus config, the string metadata field
  "content" will be added. This is to simplify the creation of full text
  metadata: the user doesn't have to duplicate their dataset in `metadata.dat`,
  and `metadata.dat` will still be somewhat human-readable without large strings
  of full text added.
- Allow `make_index` to take a user-supplied corpus object.

Miscellaneous
- ZLIB is now a required dependency.
- Switch to just using the standalone `./unit-test` instead of `ctest`. There
  aren't really many advantages for us to using CTest at this point with the new
  unit test framework, so just use our unit test executable.

release MeTA v2.0.1

Bug fixes
- Fix issue where `metadata_parser` would not consume spaces in string
    metadata fields. Thanks to @Hopsalot on the forum for the bug report!
- Fix build issue on OS X with Xcode 6.4 and `clang` related to their
    shipped version of `string_view` lacking a const `to_string()` method

Enhancements
- The `./profile` executable ensures that the file exists before operating on
  it. Thanks to @domarps for the PR!
- Add a generic `util::multiway_merge` algorithm for performing the
    merge-step of an external memory merge sort.
- Build with the following Xcode versions on Travis CI:
  * Xcode 6.1 and OS X 10.9 (as before)
  * Xcode 6.4 and OS X 10.10 (new)
  * Xcode 7.1.1 and OS X 10.10 (new)
  * Xcode 7.2 and OS X 10.11 (new)

Merge branch 'develop' for MeTA 2.0.0 release

release MeTA version 1.3.8

release MeTA v1.3.7

Bug fixes
- Fix inconsistent behavior of `utf::segmenter` (and thus `icu_tokenizer`) for
    different locales. Thanks @CanoeFZH and @tng-konrad for helping debug
    this!

Enhancements
- Allow for specifying the language and country for locale generation in
    setting up `utf::segmenter` (and thus `icu_tokenizer`)
- Allow for suppression of `<s>` and `</s>` tags within `icu_tokenizer`,
    mostly useful for information retrieval experiments with unigram words.
    Thanks @HusseinHazimeh for the suggestion!
- Add a `default-unigram-chain` filter chain preset which is suitable for
    information retrieval experiments using unigram words. Thanks
    @HusseinHazimeh for the suggestion!