support training decision ("System One") models with SFTTrainer #7623

Feature request

Decision models (Cloudflare Clef / Clef-flash, Perplexity pplx-decider, TypeSafe Jev, community Kev / JevK5 / Laya, Unsloth FastDecisionModel) are causal LMs + LoRA that answer a typed question (noul yes/no, choice, ordinal score) with a probability distribution over a fixed option set. No generation: one forward pass, logits at the answer position restricted to the option tokens, softmax.

Shared recipe: label-smoothed CE over restricted logits (+ Brier), Gaussian soft targets for ordinal levels, option permutation / none-of-the-above augmentation, temperature scaling.

Not sure if DecisionTrainer is a better approach. The recipe I hacked right now is a ~300-line SFTTrainer subclass, but a third of it exists only to bypass private APIs. Proposed primitives:

1. SFTConfig.extra_columns: list[str] — dataset columns kept through remove_unused_columns and the default collator (today: override _set_signature_columns_if_needed + collator).
2. A documented compute_loss contract or loss_fn hook, plus a public way to log extra metrics (today: self._metrics[mode]).
3. Brier / ECE metrics as shared utilities (also covers Add reward functions to support RLCR #3871).
4. An examples/ recipe: Clef-style decision model with SFTTrainer.

Non-goals: new trainer, decision-specific dataset format, head module, GRPO changes (for a one-token policy the expected reward is closed-form; sampling is strictly worse).

Motivation

These models are shipping from Cloudflare, Perplexity and TypeSafe, and Unsloth already supports them; TRL cannot express the objective without private-API overrides. The blockers (pass-through columns, candidate-restricted loss, calibration metrics) are generic and hit anyone training a custom loss on SFTTrainer.

I read #4223, #4374, #7182 / #7254 and #5485 first: a standalone trainer would overlap ~90% with SFTTrainer and fail the usage/maintenance bar, and a "decision head" abstraction has the same risk as Judge. Small primitives plus a recipe match the "prototype outside, upstream on traction" path.

Related: #3871.

Your contribution

Happy to write PRs and work on an end to end implementation