Studio: keep Spark-TTS LoRA checkpoints exportable by danielhanchen · Pull Request #13424 · unslothai/unsloth

danielhanchen · 2026-10-12T13:39:17Z

Exporting a Spark-TTS LoRA trained in Studio failed in every format (Merged, LoRA safetensors, GGUF) with Unrecognized model in unsloth/Spark-TTS-0.5B. Should have a model_type key in its config.json. Loading the same checkpoint for inference failed the same way.

Cause

Spark-TTS trains from <snapshot>/LLM: the repo root holds BiCodec and config.yaml, not a transformers config.json. When the model was picked from the local cache, Studio pins the load to the snapshot and afterwards restores the Hub identity so saved metadata does not carry a local path. That restore matched any path inside the snapshot, so <snapshot>/LLM was rewritten to unsloth/Spark-TTS-0.5B, dropping the subfolder. adapter_config.json then named a root PEFT cannot load.

Changes

utils/models/model_identity.py: only rewrite a value that is the snapshot root itself. A path inside a snapshot keeps its path (the same thing an unpinned Spark-TTS run already records).
core/inference/audio_codecs.py: spark_tts_adapter_load_target, used by the export and inference Spark-TTS LoRA loads. Checkpoints already saved with the bare repo id load through a temporary copy whose adapter_config.json points at LLM/; the user's checkpoint is never modified, and anything else loads exactly as before. Without this, every Spark-TTS checkpoint trained since the pinned-snapshot flow landed would need retraining.

Evidence (real Studio code paths, `UnslothTrainer.load_model` with the snapshot pin, LoRA, `save_pretrained`, then `ExportBackend.load_checkpoint`)

case	main	this PR
adapter base saved after a pinned load	`unsloth/Spark-TTS-0.5B`	`.../snapshots/<sha>/LLM`
export load of that new checkpoint	`Unrecognized model ... model_type`	loads, merged export OK
export of a checkpoint saved by main (legacy)	`Unrecognized model ... model_type`	loads; merged q_proj differs from base by the LoRA delta (0.00315, same as a fresh checkpoint); LoRA export OK; original checkpoint untouched
inference load of the legacy checkpoint	`Unrecognized model ... model_type`	loads as `PeftModelForCausalLM`
unpinned training run	already worked	unchanged

Tests: tests/test_spark_tts_adapter_base.py (5 new; the identity case fails on main). Related backend suites (57 files touching export, identity, bicodec): same 6 failures on main and on this branch (missing fastmcp in the test env), no new failures.

Restoring the Hub identity after a pinned-snapshot load rewrote <snapshot>/LLM to the bare repo id, dropping the LLM subfolder, so adapter_config.json named a repo root with no config.json and every export (and inference load) of a Spark-TTS LoRA failed with 'Unrecognized model ... Should have a model_type key'. Only rewrite the snapshot root itself, and load checkpoints saved before this fix through a temporary copy whose adapter base points at LLM/.

chatgpt-codex-connector · 2026-10-12T13:39:31Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-12T14:14:41.744756Z	`447b2f7`	Manual request

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

danielhanchen · 2026-10-12T13:39:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bdb1462ae9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-12T13:47:05Z

+    # A path inside the snapshot (Spark-TTS loads <snapshot>/LLM) is not the repo root, so the repo id would lose the
+    # subfolder and point PEFT at a root with no config.json (#13411).
+    if not _cache_path_matches_repo(value, repo_id) or not _is_snapshot_root(value):


Keep a portable base identity for Spark adapters

When pinned Spark-TTS training loads <snapshot>/LLM, this condition now preserves that machine-specific cache path in base_model_name_or_path, so every newly saved or exported LoRA depends on the original snapshot directory. If the adapter is uploaded, copied to another machine, or the cache is evicted, loading fails; spark_tts_adapter_load_target cannot recover it because _spark_tts_llm_dir immediately declines every base ending in /LLM. Preserve enough stable repository/subfolder identity to re-resolve the base, or teach the compatibility loader to reconstruct and download the repository from a missing snapshot path.

Useful? React with 👍 / 👎.

Not changing: unpinned Spark-TTS runs on main already record this same absolute /LLM path, and no single string is loadable by PEFT for a repo subfolder. An evicted base is a separate, pre-existing case (audio type detection already fails on it before this loader is reached).

chatgpt-codex-connector · 2026-10-12T13:47:05Z

+        adapter_config = json.loads(
+            (checkpoint / "adapter_config.json").read_text(encoding = "utf-8")
+        )


Support remote Spark adapter identifiers

When inference is given a Hugging Face LoRA identifier such as owner/spark-lora, ModelConfig supports and classifies it as a remote adapter, but this helper treats the identifier as a local filesystem path. Reading owner/spark-lora/adapter_config.json therefore raises, the broad exception handler yields the original identifier unchanged, and a legacy adapter whose base is the bare Spark repository still reaches FastModel.from_pretrained and fails with the same missing-root-config.json error this change is intended to fix. Download or otherwise resolve the remote adapter config before constructing the patched staging view.

Useful? React with 👍 / 👎.

Not reachable for the case this fixes: legacy adapters could never be exported or pushed from Studio (the export load is exactly what failed), so a Hub-hosted adapter with the bare Spark base would have to be uploaded by hand.

chatgpt-codex-connector · 2026-10-12T13:47:05Z

+            snapshot_download(
+                base,
+                allow_patterns = ["config.json", "LLM/*"],
+                token = False if hf_token is False else (hf_token or None),


Honor the adapter's pinned base revision

For a legacy adapter whose adapter_config.json records a non-default revision, this download ignores that field and resolves the base repository's current default revision instead. The staged config then points to that local snapshot, so the downstream loader cannot apply the adapter's revision; after the Spark base repository changes, export or inference can silently combine the LoRA with different base weights and produce incorrect results. Pass the recorded revision into snapshot_download when resolving the LLM directory.

Useful? React with 👍 / 👎.

Studio's Spark-TTS training never passes a revision to the loader, so the saved adapter_config.json always has revision null; there is no recorded revision to honor.

chatgpt-codex-connector · 2026-10-12T13:47:05Z

+    llm = None
+    try:
+        adapter_config = json.loads(
+            (checkpoint / "adapter_config.json").read_text(encoding = "utf-8")


Accept BOM-prefixed adapter configuration files

When a local adapter_config.json has a UTF-8 BOM, the existing discovery path accepts it via utf-8-sig and correctly routes the checkpoint as a Spark LoRA, but this new read uses plain utf-8; json.loads then raises on the BOM, the exception is swallowed, and the unpatched legacy checkpoint is handed to the loader and fails on the bare Spark repository root. Read this file with utf-8-sig, as the other adapter-config readers in model_config.py already do.

Useful? React with 👍 / 👎.

PEFT writes adapter_config.json without a BOM, and the legacy file comes straight from Studio's own save; a BOM-prefixed hand-edited config is outside the case this fixes.

danielhanchen · 2026-10-12T13:50:49Z

@codex review

danielhanchen · 2026-10-12T13:51:03Z

@codex review

chatgpt-codex-connector · 2026-10-12T13:53:23Z

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: bdb1462ae9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

So its tests import without torch and run on every CI runner.

danielhanchen · 2026-10-12T14:10:32Z

@codex review

danielhanchen · 2026-10-12T14:10:45Z

@codex review

chatgpt-codex-connector · 2026-10-12T14:14:40Z

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 447b2f73a3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

for more information, see https://pre-commit.ci

danielhanchen self-assigned this Oct 12, 2026

danielhanchen mentioned this pull request Oct 12, 2026

[staging CI] unslothai/unsloth#13424 Etherll/unsloth#266

Closed

chatgpt-codex-connector Bot reviewed Oct 12, 2026

View reviewed changes

unslothai deleted a comment from chatgpt-codex-connector Bot Oct 12, 2026

danielhanchen added 2 commits October 12, 2026 06:58

Tighten the Spark-TTS adapter loader docstring

702e426

Move the Spark-TTS adapter loader to a torch-free module

447b2f7

So its tests import without torch and run on every CI runner.

danielhanchen mentioned this pull request Oct 12, 2026

[staging CI] unslothai/unsloth#13424 Imagineer99/unsloth#354

Closed

danielhanchen mentioned this pull request Oct 12, 2026

[staging CI] unslothai/unsloth#13424 Imagineer99/unsloth#358

Closed

[pre-commit.ci] auto fixes from pre-commit.com hooks

5e6cd5f

for more information, see https://pre-commit.ci

danielhanchen merged commit 071e87e into main Oct 12, 2026
0 of 42 checks passed

danielhanchen deleted the studio-spark-tts-export-13411 branch October 12, 2026 15:47

danielhanchen mentioned this pull request Oct 12, 2026

[Bug] Export fails for fine-tuned unsloth/SparkTTS-0.5B: Unrecognized model (missing model_type in config.json) #13411

Closed

2 of 5 tasks

Conversation

danielhanchen commented Oct 12, 2026

Cause

Changes

Evidence (real Studio code paths, UnslothTrainer.load_model with the snapshot pin, LoRA, save_pretrained, then ExportBackend.load_checkpoint)

Uh oh!

chatgpt-codex-connector Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

danielhanchen commented Oct 12, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

danielhanchen commented Oct 12, 2026

Uh oh!

danielhanchen commented Oct 12, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 12, 2026

Uh oh!

danielhanchen commented Oct 12, 2026

Uh oh!

danielhanchen commented Oct 12, 2026

Uh oh!

chatgpt-codex-connector Bot commented Oct 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Evidence (real Studio code paths, `UnslothTrainer.load_model` with the snapshot pin, LoRA, `save_pretrained`, then `ExportBackend.load_checkpoint`)

chatgpt-codex-connector Bot commented Oct 12, 2026 •

edited

Loading