Skip to content

Greedy option seems inconsistent #97

Open
Open
Greedy option seems inconsistent#97
Labels
questionFurther information is requested

Description

@dysby
dysby
opened on May 26, 2023

Hi,
using your library version: 0.9.1

I found inconsistent behavior when using greedy option.
See example below, where I was expecting the lemmatized versions of the text to be equal when we force greedy option.

>>> text_lemmatizer("fire crew", lang="en")
['fire', 'crow']
>>> text_lemmatizer("fire crews", lang="en", greedy=True)
['fire', 'crew']
>>> text_lemmatizer(" ".join(text_lemmatizer("fire crews", lang="en", greedy=True)), lang="en")
['fire', 'crow']

Thanks,

Activity

  1. adbar commented on May 30, 2023

    @adbar
    adbar
    on May 30, 2023
    Owner

    Hi @dysby, good catch!

    My guess would be that the results are cached internally, which affects the results of text_lemmatizer(). In any case it is worth looking further into the issue.

  2. adbar
    added
    questionFurther information is requested
    on May 30, 2023
  3. dysby
    closed this as completedon May 30, 2023
  4. dysby
    reopened this on May 30, 2023

  5. dysby commented on May 30, 2023

    @dysby
    dysby
    on May 30, 2023
    Author

    I think it has to do with minimum word length in simplemma.py#L495 at latest release 0.9.1.

    Not sure if recent code does the same.

  6. adbar
    added this to the v1.0 milestone on Jun 21, 2023
  7. adbar
    removed this from the v1.0 milestone on May 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    questionFurther information is requested

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions