-
Notifications
You must be signed in to change notification settings - Fork 287
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Issue Description
This resolved issue with MLX-LM ml-explore/mlx-lm#683 and ml-explore/mlx-lm#682 (comment)
Steps to reproduce the issue
using one-liners like ramalama --runtime=mlx serve hf://ExaltedSlayer/nvidia-nemotron-3-nano-30b-a3b-mlx-mxfp4 or ramalama --runtime=mlx serve hf://noctrex/Nemotron-3-Nano-30B-A3B-MXFP4_MOE-GGUF
Describe the results you received
Traceback (most recent call last):
File "/opt/homebrew/bin/mlx_lm.server", line 7, in <module>
sys.exit(main())
~~~~^^
File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/server.py", line 1578, in main
run(args.host, args.port, ModelProvider(args))
~~~~~~~~~~~~~^^^^^^
File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/server.py", line 393, in __init__
self.load(self.cli_args.model, draft_model_path="default_model")
~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/server.py", line 421, in load
model, tokenizer = load(
~~~~^
self.cli_args.model,
^^^^^^^^^^^^^^^^^^^^
adapter_path=adapter_path,
^^^^^^^^^^^^^^^^^^^^^^^^^^
tokenizer_config=tokenizer_config,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
)
^
File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/utils.py", line 326, in load
tokenizer = load_tokenizer(
model_path, tokenizer_config, eos_token_ids=config.get("eos_token_id", None)
)
File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/utils.py", line 279, in load_tokenizer
return _load_tokenizer(
model_path, tokenizer_config_extra, eos_token_ids=eos_token_ids
)
File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/mlx_lm/tokenizer_utils.py", line 463, in load
AutoTokenizer.from_pretrained(model_path, **kwargs),
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/mlx-lm/0.29.0/libexec/lib/python3.14/site-packages/transformers/models/auto/tokenization_auto.py", line 1153, in from_pretrained
raise ValueError(
f"Tokenizer class {tokenizer_class_candidate} does not exist or is not currently imported."
)
ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.
Describe the results you expected
Having the one-liner work so I can use --verbose for speed tests
ramalama info output
IDKUpstream Latest Release
Yes
Additional environment details
Ramalama along with MLX and MLX-LM are installed with HomeBrew
Additional information
No response
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working