Skip to content

MLX-LM having invalid Tokenizer #2250

@TomLucidor

Description

@TomLucidor

Issue Description

This resolved issue with MLX-LM ml-explore/mlx-lm#683 and ml-explore/mlx-lm#682 (comment)

Steps to reproduce the issue

using one-liners like ramalama --runtime=mlx serve hf://ExaltedSlayer/nvidia-nemotron-3-nano-30b-a3b-mlx-mxfp4 or ramalama --runtime=mlx serve hf://noctrex/Nemotron-3-Nano-30B-A3B-MXFP4_MOE-GGUF

Describe the results you received

Traceback (most recent call last):
  File "/opt/homebrew/bin/mlx_lm.server", line 7, in <module>
    sys.exit(main())
             ~~~~^^
  File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/server.py", line 1578, in main
    run(args.host, args.port, ModelProvider(args))
                              ~~~~~~~~~~~~~^^^^^^
  File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/server.py", line 393, in __init__
    self.load(self.cli_args.model, draft_model_path="default_model")
    ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/server.py", line 421, in load
    model, tokenizer = load(
                       ~~~~^
        self.cli_args.model,
        ^^^^^^^^^^^^^^^^^^^^
        adapter_path=adapter_path,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^
        tokenizer_config=tokenizer_config,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/utils.py", line 326, in load
    tokenizer = load_tokenizer(
        model_path, tokenizer_config, eos_token_ids=config.get("eos_token_id", None)
    )
  File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/utils.py", line 279, in load_tokenizer
    return _load_tokenizer(
        model_path, tokenizer_config_extra, eos_token_ids=eos_token_ids
    )
  File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/tokenizer_utils.py", line 463, in load
    AutoTokenizer.from_pretrained(model_path, **kwargs),
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/transformers/models/auto/tokenization_auto.py", line 1153, in from_pretrained
    raise ValueError(
        f"Tokenizer class {tokenizer_class_candidate} does not exist or is not currently imported."
    )
ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.

Describe the results you expected

Having the one-liner work so I can use --verbose for speed tests

ramalama info output

IDK

Upstream Latest Release

Yes

Additional environment details

Ramalama along with MLX and MLX-LM are installed with HomeBrew

Additional information

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions