Apertus model support #4018

Summary

I'd like to add inference support for Apertus. Before opening a PR, I wanted to check whether it's welcome and agree on scope. I couldn't find an existing issue or PR about it.

Architecture

Apertus is already supported in transformers (models/apertus, ApertusForCausalLM). It's a Llama-style decoder (GQA, RoPE, RMSNorm, no biases) with three differences:

• MLP: non-gated, down_proj(xIELU(up_proj(x))). xIELU has per-layer parameters (alpha_p, alpha_n, plus beta/eps buffers) stored in the checkpoint.
• Attention: q_norm/k_norm RMSNorm over head_dim, applied to each head before RoPE, as in qwen3.rs.
• Naming: the layer norms are attention_layernorm / feedforward_layernorm.

Proposed scope (first PR)

Following the layout of recent model PRs (e.g. #3400, #3372):

• candle-transformers/src/models/apertus.rs, loading HF safetensors via VarBuilder
• candle-examples/examples/apertus/ with main.rs and a README

Possible follow-ups

• A quantized variant.
• Apertus 1.5's text backbone (same architecture)
• Apertus 1.5 (multimodal). Its transformers support is still in review (Apertus 1.5: add model transformers#47662), so I'd wait for the weight layout to settle.

Questions

1. Would you accept this in candle-transformers?
2. Should xIELU live in the model file, or would you prefer it added to candle-nn's activations?
3. Should the quantized variant come in the same PR or a separate one?

I'm happy to do the implementation.