Skip to content
mmh3-orgPublic

About

A fast inference engine optimized to MiniMax H3 video and image.

Topics

Resources

Stars

42 stars

Watchers

0 watching

Forks

Repository files navigation

mmh3

mmh3 is an inference engine dedicated to the MiniMax H3 family, written in Rust with CUDA and Metal kernels and running without PyTorch or ComfyUI. It generates videos with audio with MiniMax H3 and images with MiniMax H3 Image. The tokenizer, the Qwen3-VL text encoder, the diffusion transformer, the samplers and the VAEs are written in this repository and tuned for the shapes of these models.

Videos run on one machine or distributed across several running mmh3 worker, splitting every diffusion step between them over RDMA where there is a path for it and ordinary sockets where there is not. The machine that hands the work out runs none of it, so it needs no GPU of its own.

The backends are CUDA on Linux and Windows (WSL) and Metal on macOS, each with its own kernels. Each page says what that backend needs.

Getting started

With what your backend needs installed, CUDA or Metal, run these commands from the repository directory:

make
make download-models
make generate

make builds mmh3 with Metal on macOS and CUDA on Linux and Windows (WSL). make download-models downloads the models into models, and make generate writes a video with audio from a sample prompt to out.mp4. See Usage for the other settings.

These commands download MiniMax H3 Image and write an image to out.png. See Image for its settings and for editing images.

tools/download-models.sh --image
target/release/mmh3 image --models models --out out.png \
  --prompt "A red panda in a knitted scarf sits on a mossy log in a misty pine forest at dawn."

These settings generate a 4096×4096 image in about 24 s on a DGX Spark, in three steps of the PDMD LoRA on a half-size canvas and one at full size, which can change the details and the composition. See Image for what they change.

tools/download-models.sh --image --pdmd --latent-upscaler
target/release/mmh3 image --models models --out out.png --width 4096 --height 4096 \
  --steps 3 --lora minimax_h3_pdmd_2step_lora_rank128_bf16.safetensors \
  --draft-steps 3 --renoise-sigma 0.5 \
  --latent-upscaler minimax_h3_latent_upscaler_3d_conv_v1_bf16.safetensors \
  --attention-precision int8-fp8 \
  --prompt "A red panda in a knitted scarf sits on a mossy log in a misty pine forest at dawn."

Performance

These are the times of a complete generation, including model loading and encoding, with warm model files. Performance and accuracy has the times of each stage.

Videos on CUDA on Linux and Windows (WSL)

make generate uses FastVideo's 4-step FastH3 as a patch on the base DiT, with its video sparse attention, INT8/FP8 attention and the INT8 video VAE, and writes MP4 with NVENC H.264 and AAC. Compared with mmh3's defaults, these can change image details and the composition. mmh3 also runs four other few-step models with the settings on their pages.

On a DGX Spark, a complete generation took:

Model Steps Time per step Time
FastH3 4 14 s 80 s
lightx2v Turbo LoRA 4 14 s 81 s
DMAD 4 14 s 80 s
TaoMate-H3 3 14 s 66 s
PDMD 2 14 s 52 s

Each generated a 1344×768, 124-frame video, 5.2 seconds at 24 fps.

With draft steps, the first steps run on a canvas of half the width and height, which settles the layout and the motion in a quarter of the tokens. With the first two of the Turbo LoRA's four steps as draft steps, a generation took 61.5 s rather than 83.2 s.

Videos on Metal on macOS

make generate uses the same 4-step FastH3 patch with its video sparse attention, in a smaller and shorter clip, and writes MP4 with VideoToolbox H.264 and AAC. The DiT, the text encoder and the video VAE's transformer run on the GPU's matrix units with INT8 products and FP16 attention, on the Neural Accelerators of an M5 or later. A Mac whose GPU cannot hold a whole model keeps as many layers as fit and reads the others from the disk as they run.

A complete generation took:

Mac Memory Time per step Time
M4 Max 128 GB 27 s 153 s
M6 24 GB 12 s 67 s

Each generated a 672×384, 73-frame video, 3.0 seconds at 24 fps. On the M6, the 2-step PDMD LoRA generated a video of the same size in 45 s, and the 4-step DMAD LoRA in 76 s.

Images on CUDA

On a DGX Spark, a 4096×4096 image took:

Settings Time per step Time
Faster settings: 3 PDMD steps as draft steps, INT8/FP8 attention 7.4 s 24 s
The same with NVFP4 linear layers 6.1 s 23 s
Defaults, six steps, BF16 attention 8.8 s 63 s

The times per step are those of a step at full size, of which the faster settings run one. A server that keeps the models loaded takes 15 s with the faster settings. Image has the times of other settings.

Images on Metal

A 4096×4096 image took:

Mac Settings Time per step Time
M6, 24 GB Faster settings: 3 PDMD steps as draft steps, INT8/FP16 attention 59 s 140 s
M6, 24 GB Defaults, FP16 attention 49 s 337 s
M5 Max, 128 GB 2 draft steps enlarged bilinearly, Sol-Attn and FP16 attention 14 s 89 s
M5 Max, 128 GB Defaults, FP16 attention 24 s 165 s

On the M6, the DiT's blocks that do not fit are read from the disk as they run. The M5 Max is a 14-inch MacBook Pro in High Power Mode on AC power, whose GPU clock moved its times by two thirds to one and a half times. Image has the times of other settings.

Status

Videos

  • Text to video with audio (T2VA) works end to end, one video at a time. Native MP4 and WebM generation and complete video/audio decoding have been verified on GB10.
  • Videos from a first frame, a last frame or both (FL2VA) work too.
  • Videos from reference images, sounds and clips (Ref2VA) work with the ref2va DiT, reading reference clips from MP4 files.

Images

  • Images of about 16 megapixels from text, and edits of up to 16 reference images (Image), work with mmh3 image on CUDA and have been checked against diffusers' pipeline. They work on Metal too, where editing needs macOS 26.

Machines

  • NVIDIA GPUs with CUDA. It is developed on a DGX Spark (GB10, sm_121) and builds for sm_120f, so it should also run on RTX PRO 6000 and RTX 50 series GPUs, which have not been tested yet. The build also targets Ada (sm_89), which runs on an RTX 4090.
  • Apple silicon with Metal on macOS 15 or later, tested on an M4 Max and an M6. Text to video with audio works with LoRAs, patches such as FastH3's, and Sol-Attn and VSA sparse attention, and a Mac with 24 GB runs it by reading the layers that do not fit from the disk. Videos from frames and references work on macOS 26. See Metal for its current limits.
  • Distributed generation of videos works on both backends and between them, and from a machine with neither. Shares follow what each machine measures itself to do. Between two DGX Sparks a 768p run takes about a quarter less, and a Mac that hands every step to a Spark rather than taking one itself generates about twenty-five times faster than it does alone.
  • An HTTP server takes a video or an image as a form and keeps its models loaded between the generations it runs, so the second one against a warm server starts at its first step. It has no authentication of its own and waits on loopback.

Documentation

docs/README.md lists the documentation pages.

License

mmh3 is released under the MIT License. See LICENSE, and NOTICE for third-party material and credits. The model weights are not part of this repository and come under their own licenses.

About

A fast inference engine optimized to MiniMax H3 video and image.

Topics

Resources

Stars

42 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages