Commits

Commit history

1. Commits on Sep 24, 2017

   • Clean up tests

     jdkato

     committed
     8857f6bView commit details

     Copy full SHA for 8857f6b

     View code at this pointBrowse repository at this point
2. Commits on Jul 29, 2017

   • Update link to TreebankWordTokenizer source

     jdkato

     authored
     54de340View commit details

     Copy full SHA for 54de340

     View code at this pointBrowse repository at this point

   • Remove TODOs from GoDoc

     jdkato

     committed
     66b3c11View commit details

     Copy full SHA for 66b3c11

     View code at this pointBrowse repository at this point
3. Commits on Jul 25, 2017

   • Drop matloob/regexp

     jdkato

     committed
     4d68d1bView commit details

     Copy full SHA for 4d68d1b

     View code at this pointBrowse repository at this point
4. Commits on Jun 4, 2017

   • Only call `FindAllStringSubmatchIndex` once

     Show description for d62723e
     jdkato

     committed
     d62723eView commit details

     Copy full SHA for d62723e

     View code at this pointBrowse repository at this point
5. Commits on Jun 3, 2017

   • More caching in abbreviationReplacer

     Show description for 7f2adc9
     jdkato

     committed
     7f2adc9View commit details

     Copy full SHA for 7f2adc9

     View code at this pointBrowse repository at this point

   • Cache compiled regexps in abbreviationReplacer

     Show description for b95f760
     jdkato

     committed
     b95f760View commit details

     Copy full SHA for b95f760

     View code at this pointBrowse repository at this point

   • Use matloob.io/regexp

     Show description for f869725
     jdkato

     committed
     f869725View commit details

     Copy full SHA for f869725

     View code at this pointBrowse repository at this point
6. Commits on May 23, 2017

   • Update comments

     jdkato

     committed
     002177eView commit details

     Copy full SHA for 002177e

     View code at this pointBrowse repository at this point

   • Lowercase `rule`

     jdkato

     committed
     dfbb852View commit details

     Copy full SHA for dfbb852

     View code at this pointBrowse repository at this point

   • Leave `rule.sub` early for non-matches

     jdkato

     committed
     8e5dfdaView commit details

     Copy full SHA for 8e5dfda

     View code at this pointBrowse repository at this point

   • Leave `search` early if we can

     jdkato

     committed
     825d5d8View commit details

     Copy full SHA for 825d5d8

     View code at this pointBrowse repository at this point

   • Precompile boundaries

     Show description for 673dacf
     jdkato

     committed
     673dacfView commit details

     Copy full SHA for 673dacf

     View code at this pointBrowse repository at this point
7. Commits on May 22, 2017

   • Add benchmarks

     jdkato

     committed
     3d706efView commit details

     Copy full SHA for 3d706ef

     View code at this pointBrowse repository at this point

   • Update comments

     jdkato

     committed
     e78bf70View commit details

     Copy full SHA for e78bf70

     View code at this pointBrowse repository at this point

   • Add the PragmaticSegmenter

     Show description for 3f81395
     jdkato

     committed
     3f81395View commit details

     Copy full SHA for 3f81395

     View code at this pointBrowse repository at this point
8. Commits on May 12, 2017

   • Use modified PunktSentenceTokenizer in TextToWords

     jdkato

     committed
     6bc03bbView commit details

     Copy full SHA for 6bc03bb

     View code at this pointBrowse repository at this point
9. Commits on May 11, 2017

   • Add TestNewRegexpTokenizer

     jdkato

     committed
     f0741b2View commit details

     Copy full SHA for f0741b2

     View code at this pointBrowse repository at this point

   • More tests

     jdkato

     committed
     1ffb04fView commit details

     Copy full SHA for 1ffb04f

     View code at this pointBrowse repository at this point

   • Add TestTextToWords

     jdkato

     committed
     35f5a69View commit details

     Copy full SHA for 35f5a69

     View code at this pointBrowse repository at this point

   • Fix example tests

     jdkato

     committed
     21c571fView commit details

     Copy full SHA for 21c571f

     View code at this pointBrowse repository at this point

   • Move doc examples to tokenize_test

     jdkato

     committed
     f118987View commit details

     Copy full SHA for f118987

     View code at this pointBrowse repository at this point
10. Commits on May 10, 2017

    • Clean up Godoc

      jdkato

      committed
      fca9981View commit details

      Copy full SHA for fca9981

      View code at this pointBrowse repository at this point
11. Commits on May 9, 2017

    • Update licenses

      jdkato

      committed
      80fa9daView commit details

      Copy full SHA for 80fa9da

      View code at this pointBrowse repository at this point

    • Remove "-" from WordBoundaryTokenizer

      jdkato

      committed
      370dc0fView commit details

      Copy full SHA for 370dc0f

      View code at this pointBrowse repository at this point

    • Improve PunktSentenceTokenizer

      jdkato

      committed
      78e9229View commit details

      Copy full SHA for 78e9229

      View code at this pointBrowse repository at this point
12. Commits on Apr 11, 2017

    • tag: add license to aptag

      jdkato

      committed
      c591293View commit details

      Copy full SHA for c591293

      View code at this pointBrowse repository at this point

    • summarize: Update `TreebankWordTokenizer` citation

      jdkato

      committed
      74efe5dView commit details

      Copy full SHA for 74efe5d

      View code at this pointBrowse repository at this point
13. Commits on Apr 10, 2017

    • tokenize: Describe `TreebankWordTokenizer` better

      Show description for 07d1363
      jdkato

      committed
      07d1363View commit details

      Copy full SHA for 07d1363

      View code at this pointBrowse repository at this point
14. Commits on Apr 7, 2017

    • tokenize: Match Unicode in WordBoundaryTokenizer

      jdkato

      committed
      ea6d9acView commit details

      Copy full SHA for ea6d9ac

      View code at this pointBrowse repository at this point

    • tokenize: wrap sentences into a `ProseTokenizer`

      jdkato

      committed
      4fed1f5View commit details

      Copy full SHA for 4fed1f5

      View code at this pointBrowse repository at this point

    • tokenize: Define ProseTokenizer interface

      jdkato

      committed
      625de72View commit details

      Copy full SHA for 625de72

      View code at this pointBrowse repository at this point

    • tokenize: Fix gometalinter errors

      jdkato

      committed
      3191360View commit details

      Copy full SHA for 3191360

      View code at this pointBrowse repository at this point

    • tokenize: Clarify Treebank input

      jdkato

      authored
      e09179eView commit details

      Copy full SHA for e09179e

      View code at this pointBrowse repository at this point
15. Commits on Apr 6, 2017

    • tokenize: Add `RegexpTokenizer`

      jdkato

      committed
      dbc4e4eView commit details

      Copy full SHA for dbc4e4e

      View code at this pointBrowse repository at this point