MiniMax-H3 - SGLang Documentation

1. Quick start

Install with uv pip install "sglang[diffusion]" --prerelease=allow, then choose a verified recipe below. Setup changes the deployment; Server and Request expose orthogonal startup and sampling choices.

The generated Server command already includes the recommended encoder policy. Change a Server option only for a deliberate trade-off; Request options do not reload the model.

The Docker form installs the platform-specific diffusion extra from the source bundled in the image. For conditioned requests, set Host media directory under Variables; the builder mounts it read-only at /data/minimax-h3. AMD currently offers the Python form, while NVIDIA also offers Docker. To use ModelScope through the same normal sglang serve path, prefix the copied command with SGLANG_USE_MODELSCOPE=true and replace the model path with MiniMax/MiniMax-H3. Keep the selected variant and topology flags unchanged. For platform-specific installation details, see the SGLang Diffusion installation guide.

2. Model capabilities

MiniMax-H3 is a native joint video-and-audio model for text-to-video-and-audio, first/last-frame control, and multimodal reference conditioning. Its main strength is producing the picture and stereo soundtrack together, so speech, music, ambient sound, and visible events can stay aligned without a separate audio-generation pass. Choose H3 when synchronized audiovisual output or reference-driven generation matters more than a lightweight deployment. The released recipe targets a 768-pixel short edge at 24 fps for 4–15 seconds, and its capabilities are split across two checkpoint partitions; serving every mode therefore requires separate FL2VA and Ref2VA deployments.

Task	`task` value	Conditioning
Text to video and audio	`t2va`	Text prompt only
First/last frame to video and audio	`fl2va`	First frame, last frame, or both
Reference to video and audio	`ref2va`	Image, video, and audio references

Video-to-video (V2V) is a supported ref2va use case, not a fourth task value. Run the Ref2VA partition and provide a video reference in conditions. Use the selected Hub’s root model ID: MiniMaxAI/MiniMax-H3 on Hugging Face or MiniMax/MiniMax-H3 on ModelScope. Select the checkpoint variant with --model-variant: fl2va serves both t2va and fl2va, while ref2va serves reference-conditioned requests. SGLang owns the checkpoint-directory mapping; do not point --model-path at a manually downloaded subdirectory.

Review the license and usage terms in the MiniMax-H3 model card before production or commercial use. SGLang support does not grant additional model usage rights.

3. Deployment details

The builder accepts legal custom GPU counts and topologies, marking them Unverified until the exact recipe has completed end-to-end validation. Static H3 head or partition violations disable Copy before they reach sglang serve. For a four-card H200 host, keep the full BF16/FP32 model resident by default. The model fits without FSDP, so this path avoids the per-block parameter all-gathers of the memory-oriented FSDP profile:

4×H200 resident

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --ulysses-degree 4 \
  --encoder-parallel auto \
  --performance-mode speed \
  --port 30010

Pure Ulysses4 is also the faster measured topology on H200, not just a capacity default. The 4×H100 TP2 + Ulysses2 recipe below fits on 141 GB H200 cards, but it replaces the Ulysses all-to-all exchange with two per-block tensor-parallel all-reduces and measured slower end-to-end, at about 30 GB lower peak memory per GPU. See the H200 topology comparison in the Benchmarks section for the measured numbers; treat TP2 + Ulysses2 on H200 as a deliberate memory trade, not a latency default. For 4×H100 80 GB, balance the large packed activation with resident weight sharding. TP2 + Ulysses2 was the fastest measured lossless topology while the Qwen encoder still folds across all four GPUs:

4×H100 fastest

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --tp-size 2 \
  --ulysses-degree 2 \
  --encoder-parallel auto \
  --performance-mode speed \
  --port 30010

Pure Ulysses4 could not keep the full pipeline resident on 80 GB H100s. Use --tp-size 4 --ulysses-degree 1 when lower resident memory matters more than the last few percent of latency. FSDP remains a verified capacity option, but its per-block weight all-gathers do not make it the H100 speed default:

4×H100 FSDP capacity

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --ulysses-degree 4 \
  --encoder-parallel auto \
  --performance-mode speed \
  --use-fsdp-inference true \
  --port 30010

For a two-card RTX 5090 host, use TP2 and keep 20 DiT blocks resident. Layerwise placement is lossless: it changes parameter placement and transfer scheduling, not the BF16/FP32 denoising or VAE math. This is the fastest measured 32 GB operating point:

2×RTX 5090 fastest lossless

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 2 \
  --tp-size 2 \
  --ulysses-degree 1 \
  --encoder-parallel auto \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder,vae \
  --dit-offload-prefetch-size 1 \
  --dit-layerwise-resident-layers 20 \
  --enable-torch-compile false \
  --port 30010

The DiT residency and prefetch knobs apply only to the repeatedly executed DiT blocks. The text encoder and the video VAE decoder blocks use one-layer prefetch with zero resident layers. The video VAE encoder stays resident because its indexed down blocks cannot host executable layerwise hooks; the roughly 577 MiB audio VAE also stays resident because offloading it only adds transfer overhead. This exact recipe was validated on 2× RTX 5090 (32 GB each) and a 377 GiB host; use a 384 GiB-class machine. The latency and memory comparison is collected in the benchmark section below. For a single 24 GB consumer card (RTX 4090), stream the DiT and text encoder and quantize DiT linear layers online with kitchen_int8. Keep vae out of --layerwise-offload-components: putting the VAE decoder in layerwise offload re-streams about 9 GiB on each of 167 decode tiles. Default attention stays fa (exact). Approximate backends are opt-in; see Attention Backends. Install comfy-kitchen first (pip install comfy-kitchen).

1×RTX 4090 24GB

sglang generate \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --quantization kitchen_int8 \
  --attention-backend fa \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder \
  --dit-offload-prefetch-size 1 \
  --dit-layerwise-resident-layers 0 \
  --enable-torch-compile false \
  --prompt "A cat walking on a sunny beach, gentle waves." \
  --save-output

The same flags work on sglang serve. Drop --quantization for the BF16 baseline; everything else stays identical. GPU peak stays about 18 GB either way because streaming offload is set by the offload buffers and VAE decode, not the weight dtype.

Pre-quantized GGUF transformer

Use --transformer-weights-path to replace only the DiT with a GGUF file; the base repository continues to provide the text encoder, VAEs, scheduler, and tokenizers. Do not also pass --quantization gguf.

1×RTX 5090 Q4_K_M

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --transformer-weights-path \
    leejet/MiniMax-H3-GGUF/minimax_h3_fl2va-Q4_K_M.gguf \
  --attention-backend fa \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder \
  --dit-offload-prefetch-size 1 \
  --dit-layerwise-resident-layers 0 \
  --enable-torch-compile false \
  --port 30010

The loader also recognizes pruned checkpoints that replace the timestep MLP with adaln_t_table. Repositories containing both FL2VA and Ref2VA variants need a full file reference:

Pruned FL2VA Q4_K

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --transformer-weights-path \
    unsloth/MiniMax-H3-GGUF/minimax_h3_fl2va_pruned-Q4_K.gguf \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder \
  --enable-torch-compile false \
  --port 30010

The linear adapter reuses SRT’s GGUF type definitions and CUDA dequantization, then runs the native GEMM. SRT’s fused MMVQ/MMQ kernels target the low-token LLM regime and are slower at diffusion sequence lengths. TP is supported when each row-parallel input shard remains GGML-block aligned; incompatible degrees fail during model construction. FSDP, LoRA merging, and the separate MiniMax-H3 AdaLN cache flags are not compatible with packed GGUF weights. The first launch downloads the model through the selected Hub. If the Hugging Face repository requires authentication, export a Hugging Face token in the server environment. For MiniMax-H3, --performance-mode speed deliberately keeps the DiT eager. The current torch.compile path changes the model’s numerical output, so it is not enabled implicitly by any recommended lossless preset. An explicit --enable-torch-compile true remains available for controlled experiments, but it should not be used to generate consistency ground truth.

AdaLN-pruned safetensors transformers

Comfy-Org/MiniMax-H3 publishes smaller DiT-only checkpoints that replace the original AdaLN branches with an interpolated curve table. Select one file explicitly; the base model still supplies the text encoder and VAEs.

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --transformer-weights-path \
    Comfy-Org/MiniMax-H3/diffusion_models/minimax_h3_fl2va_pruned_bf16.safetensors \
  --num-gpus 4 \
  --tp-size 2 \
  --ulysses-degree 2 \
  --performance-mode speed \
  --port 30010

The pruned checkpoint is approximate and is therefore rejected by quality="high", which remains limited to the audited official BF16 DiT. The Comfy pruned_fp8_scaled FL2VA and Ref2VA files are also supported. Their per-layer markers are detected automatically; do not add --quantization:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --transformer-weights-path \
    Comfy-Org/MiniMax-H3/diffusion_models/minimax_h3_fl2va_pruned_fp8_scaled.safetensors \
  --num-gpus 4 \
  --tp-size 2 \
  --ulysses-degree 2 \
  --performance-mode speed \
  --port 30010

SGLang uses its native static-activation FP8 linear path when the checkpoint stores input scales, and automatically uses dynamic activation scaling for Comfy FP8 exports that omit them. The checkpoint above marks fc2 for full-precision matrix multiplication, so SGLang retains its FP8 storage but materializes and scales one compute-dtype fc2 matrix for each call. This preserves the checkpoint’s mixed execution contract and low resident weight memory, but that part is slower than a fully quantized FP8 GEMM. TP, Ulysses/Ring sequence parallelism, and component/layerwise offload are supported; FSDP inference is rejected. The pruned_int8_convrot files use the same override path and are detected automatically. Install comfy-kitchen, replace the FP8 filename above with minimax_h3_fl2va_pruned_int8_convrot.safetensors, and still omit --quantization. SGLang loads their serialized INT8 weights and row scales into the fused ConvRot kernel without requantizing them. TP1/2/4 and sequence or layerwise offload are supported; TP8 violates the checkpoint’s 256-element ConvRot group boundary, and FSDP is rejected. Self-describing community MXFP8 files use the same --transformer-weights-path <repo/file.safetensors> form and need no quantization flag. SGLang reads their global format metadata, keeps unmarked H3 layers in their original dtype, and reuses SRT’s MXFP8 dense kernels. The selected SRT backend must support MXFP8 on the target GPU; FSDP is rejected for this mixed per-layer layout. W4A8 ConvRot DiT files use the same flagless flow. With comfy-kitchen>=0.2.27, pass a file such as starsfriday/MiniMax-H3-w4a8/minimax_h3_fl2va_pruned_w4a8_mixed.safetensors to --transformer-weights-path; SGLang reads the serialized per-layer metadata and packed INT4 tensors automatically. Do not add --quantization. TP remains subject to each row-parallel shard preserving the checkpoint’s ConvRot group boundary, and FSDP is rejected. W4A4 ConvRot files are also detected from their layer metadata. Install a current comfy-kitchen, then pass a full or pruned FL2VA / Ref2VA file such as Merserk/MiniMax-H3-INT4-ConvRot/minimax_h3_fl2va_pruned_int4_convrot.safetensors to --transformer-weights-path. The packed weights stay INT4 and the runtime honors each layer’s activation mode; omit --quantization. Mixed exports use the same command: SGLang dispatches each marked layer to its serialized W4A4 or INT8 ConvRot kernel instead of applying one global method. Comfy NVFP4 DiTs reuse SGLang’s ModelOpt NVFP4 backend, which requires CUDA compute capability 10.0 or newer. Pass a pruned FL2VA / Ref2VA file such as Abiray/Minimax-H3-nvfp4-INT4-INT8-Convrot/MiniMax_H3_FL2VA_pruned_nvfp4.safetensors to --transformer-weights-path and omit --quantization. SGLang infers the packed group size and Comfy scale layout from the checkpoint; FSDP is rejected. Mixed files may mark selected linears as int8_tensorwise or dynamic/static FP8; SGLang dispatches those layers to their serialized Kitchen INT8 or native FP8 path automatically.

Advanced: precomputed AdaLN cache

The model card notes that about 13B H3 parameters are AdaLN branches whose outputs can be precomputed for inference. The public base checkpoint contains the original branches, not a ready-to-use cache. SGLang therefore keeps the standard path as the default.

This is an experimental deployment path. It is intentionally disabled unless you provide an explicitly generated cache; end-to-end numerical and peak-memory validation remains required before using it in production.

When an inference-only deployment has a fixed sampling schedule, build a cache from the already materialized transformer directory on CUDA, then pass it to the usual sglang serve command. This does not alter the denoising formula: the cache stores the BF16 outputs of the original AdaLN linears.

Command

python -m sglang.multimodal_gen.tools.build_minimax_h3_adaln_cache \
  --transformer-path "$TRANSFORMER_PATH" \
  --model-variant fl2va \
  --mode t2va \
  --num-inference-steps 50 \
  --flow-shift 12 \
  --audio-flow-shift 3 \
  --output /models/minimax-h3-fl2va-adaln-50step.safetensors

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --minimax-h3-adaln-cache-path /models/minimax-h3-fl2va-adaln-50step.safetensors \
  --num-gpus 4 \
  --tp-size 2 \
  --ulysses-degree 2 \
  --port 30010

$TRANSFORMER_PATH is the FL2VA/transformer or Ref2VA/transformer directory in the normal SGLang/Hugging Face snapshot; the builder never downloads a second copy. A cache only covers the scheduler settings used to create it, including its mode, step count, flow shifts, and condition noise values. SGLang rejects a request outside that coverage instead of silently changing conditioning. Cache mode supports the matching unquantized checkpoint only.

Serve MiniMax-H3 on Ascend NPUs

For Ascend NPU, follow the NPU installation guide before starting the server. The Ascend commands below explicitly enable the Cache-DiT configuration used for the reported performance measurements. Remove these SGLANG_CACHE_DIT_* variables to use lossless denoising. See the Ascend NPU topology comparison in the Benchmarks section for the measured eight- and four-NPU latency. The measured latency configuration also passes --dit-cpu-offload false to keep the transformer resident on the NPUs. Omit this flag when lower device memory usage is more important than avoiding CPU-to-NPU transfer latency. For an eight-NPU host, the validated topology is TP2 + SP4 with Laser Attention. Use Ascend Flash Attention by replacing laser_attn with fa.

8-NPU

SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
HCCL_BUFFSIZE=256 sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-type diffusion \
  --model-variant fl2va \
  --dit-cpu-offload false \
  --num-gpus 8 \
  --tp-size 2 \
  --sp-degree 4 \
  --attention-backend laser_attn \
  --port 30088 \
  --component-residency text_encoder=layerwise-offload

For a four-NPU host, use TP2 + SP2:

4-NPU

SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=2 \
SGLANG_CACHE_DIT_BN=1 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.4 \
SGLANG_CACHE_DIT_MC=4 \
SGLANG_CACHE_DIT_TAYLORSEER=true \
SGLANG_CACHE_DIT_TS_ORDER=2 \
HCCL_BUFFSIZE=256 sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-type diffusion \
  --model-variant fl2va \
  --dit-cpu-offload false \
  --num-gpus 4 \
  --tp-size 2 \
  --sp-degree 2 \
  --attention-backend laser_attn \
  --port 30088 \
  --component-residency text_encoder=layerwise-offload

4. Generate video and audio

MiniMax-H3 uses the asynchronous OpenAI-compatible video endpoint. Choose a generation mode below, submit a job, poll its status, and then download the completed MP4.

T2VA
FL2VA
V2V
Multimodal Ref2VA

MiniMax-H3 supports output durations from 4 through 15 seconds, inclusive. The following request keeps the verified 5-second profile at a 768-pixel short edge. MiniMax-H3 resolves the aligned output canvas and frame count from target.

Command

video_id=$(
  curl -sS -X POST http://127.0.0.1:30010/v1/videos \
    -H "Content-Type: application/json" \
    -d '{
      "model": "MiniMaxAI/MiniMax-H3",
      "prompt": "At night, while their owner sleeps in a bedroom, three cats march in loudly playing tiny brass instruments, then abruptly file out.",
      "seconds": 5,
      "task": "t2va",
      "conditions": [],
      "target": {
        "short_edge": 768,
        "aspect_ratio": "16:9",
        "duration_seconds": 5.0
      },
      "num_outputs_per_prompt": 1,
      "num_inference_steps": 50,
      "flow_shift": 12.0,
      "audio_flow_shift": 3.0,
      "seed": 1101
    }' |
    jq -r '.id'
)

while true; do
  status=$(curl -sS "http://127.0.0.1:30010/v1/videos/${video_id}" | jq -r '.status')
  [ "$status" = "completed" ] && break
  [ "$status" = "failed" ] && exit 1
  sleep 1
done

curl -sS -L "http://127.0.0.1:30010/v1/videos/${video_id}/content" \
  -o minimax-h3-t2va.mp4

The output contract is an MP4 containing H.264 video at 24 fps and one AAC stereo audio stream at 32 kHz.

For fl2va, provide one or two image conditions with role keyframe. The supported frame-index sets are [0], [-1], and [0, -1].The following request uses one server-local first frame. Use frame_index: -1 for a last frame, or include both entries for first-and-last conditioning.Choose FL2VA when the supplied image should be the actual first or last frame of the generated clip. Use image-based Ref2VA instead when the image should guide identity, style, or composition without being preserved as an endpoint; Ref2VA may recompose or crop the reference.

Command

curl -sS -X POST http://127.0.0.1:30010/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-H3",
    "prompt": "The supplied frame continues with calm, natural motion and synchronized ambient sound.",
    "seconds": 5,
    "task": "fl2va",
    "conditions": [
      {
        "type": "image",
        "uri": "file:///data/minimax-h3/first-frame.png",
        "role": "keyframe",
        "frame_index": 0
      }
    ],
    "target": {
      "short_edge": 768,
      "aspect_ratio": "auto",
      "duration_seconds": 5.0
    },
    "num_outputs_per_prompt": 1,
    "num_inference_steps": 50,
    "flow_shift": 12.0,
    "audio_flow_shift": 3.0,
    "seed": 2101
  }'

V2V uses the reference-conditioning weights. Launch the server with --model-variant ref2va, keep the request task set to ref2va, and provide a video reference in conditions. There is no separate v2v task value.Use type: "video" when the input may be silent. If the file has a soundtrack, H3 also uses it as an audio reference. Use type: "video_audio" only when both streams are required; that form rejects an input without audio. The prompt tag for the visual stream is <Video 1>; an available soundtrack is exposed as <Audio 1>.

Ref2VA treats the input video as reference material, not as a pixel-aligned edit source. It can resynthesize or reorder motion and cuts, and it does not expose a denoising-strength control. Do not rely on it to preserve every source frame or exact timing.

Set conditions[].start_time_seconds to select a segment from a longer source. The default is 0. SGLang seeks the visual stream and soundtrack to the same offset, then decodes at most the requested target duration in one pass; the source is not re-encoded into an intermediate clip.

Command

curl -sS -X POST http://127.0.0.1:30010/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-H3",
    "prompt": "Follow the motion and appearance of <Video 1>, changing the setting to a moonlit bedroom while preserving coherent timing.",
    "seconds": 5,
    "task": "ref2va",
    "conditions": [
      {
        "type": "video",
        "uri": "file:///data/minimax-h3/input.mp4",
        "role": "reference",
        "start_time_seconds": 35.0
      }
    ],
    "target": {
      "short_edge": 768,
      "aspect_ratio": "16:9",
      "duration_seconds": 5.0
    },
    "num_outputs_per_prompt": 1,
    "num_inference_steps": 50,
    "flow_shift": 12.0,
    "audio_flow_shift": 3.0,
    "seed": 4101
  }'

Use conditions[].uri for H3 V2V. The generic top-level video_path, video_url, and video_reference upload fields are not lowered into H3 reference conditions.

For ref2va, first launch the reference-conditioning capability with --model-variant ref2va, then provide conditions with role reference. Image, video, and audio references can be combined. Material tags in the prompt use the one-based order for each modality.An image condition here is semantic reference material rather than a pixel-aligned first frame. Use the FL2VA tab when animating a screenshot from that exact starting composition.

Command

curl -sS -X POST http://127.0.0.1:30010/v1/videos \
  -H "Content-Type: application/json" \
  -d '{
    "model": "MiniMaxAI/MiniMax-H3",
    "prompt": "Use <Picture 1> as the visual subject and <Audio 1> as the sound reference, with coherent natural motion.",
    "seconds": 5,
    "task": "ref2va",
    "conditions": [
      {
        "type": "image",
        "uri": "file:///data/minimax-h3/reference.png",
        "role": "reference"
      },
      {
        "type": "audio",
        "uri": "file:///data/minimax-h3/reference.mp3",
        "role": "reference"
      }
    ],
    "target": {
      "short_edge": 768,
      "aspect_ratio": "auto",
      "duration_seconds": 5.0
    },
    "num_outputs_per_prompt": 1,
    "num_inference_steps": 50,
    "flow_shift": 12.0,
    "audio_flow_shift": 3.0,
    "seed": 3101
  }'

Poll and download any conditioned request with the same job-status and content endpoints used in the T2VA example. Server-local file:// URIs must refer to files visible inside the SGLang server environment.

5. LoRA recipes

H3 accepts both native fused adapters and standard Diffusers/PEFT adapters. Native adapters target modules such as blocks.*.attn.qkv_proj; PEFT adapters may instead provide separate to_q, to_k, and to_v projections and the default adapter namespace. SGLang normalizes both layouts. The following FL2VA adapters have distinct purposes:

Recipe	Repository and pinned file	Request setting	Prompt requirement
Recommended speed/quality balance	`larryvrh/MiniMax-H3-Turbo-Lora`, `minimax_h3_turbo_v4_step600_ema.safetensors`	`num_inference_steps: 9` (8 denoiser evaluations), `lora_scale: 1.0`	None
Most aggressive speed preset (standard PEFT layout)	`lightx2v/Minimax-h3-Turbo`, `minimax_h3_fl2v_turbo_4step_v0.1.safetensors`	`num_inference_steps: 5` (4 denoiser evaluations), `lora_scale: 1.0`, `lora_alpha: 8`	None
Realistic people style	`fal/MiniMax-H3-Realism-People-LoRA`, `h3-realism-people-t2v-i2v-r2v.safetensors`	Keep the normal `num_inference_steps: 50` schedule; start with `lora_scale: 0.7`	Include `r34l1sm` in the prompt

The H3 request field controls the number of sigma grid points, including the terminal zero; the denoising loop therefore runs one fewer model evaluation. This is why an adapter described as 8-step uses 9, and a 4-step adapter uses 5, in the request. All three use the same launch shape. Pinning the filename is required for repositories that publish multiple revisions, and is also recommended for a reproducible single-file recipe:

Command

LORA_REPO=larryvrh/MiniMax-H3-Turbo-Lora
LORA_FILE=minimax_h3_turbo_v4_step600_ema.safetensors
LORA_NAME=h3-turbo-v4
LORA_SCALE=1.0
LORA_ALPHA_ARGS=()
# LightX2V only: LORA_ALPHA_ARGS=(--lora-alpha 8)

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --ulysses-degree 4 \
  --performance-mode speed \
  --lora-path "$LORA_REPO" \
  --lora-weight-name "$LORA_FILE" \
  --lora-nickname "$LORA_NAME" \
  --lora-scale "$LORA_SCALE" \
  "${LORA_ALPHA_ARGS[@]}" \
  --lora-merge-mode auto \
  --port 30010

auto merges an adapter into ordinary resident weights to avoid per-step LoRA matmuls, but keeps the dynamic path for FSDP-sharded weights where a full gather can increase peak memory. Use dynamic when one resident server must switch repeatedly between base and LoRA output. Use the filename, scale, and request schedule from the table together. The 4-evaluation LightX2V recipe is the more aggressive latency/quality tradeoff. Its checkpoint has rank 128 but omits the training alpha from both the file and repository metadata, so --lora-alpha 8 is required to reproduce the author’s reference implementation. Start with the Larry 8-evaluation recipe when preserving fine visual detail is more important than minimum latency. These adapters were trained for the FL2VA partition and apply to t2va or fl2va requests. Do not use them with the separate ref2va weights unless the adapter author explicitly provides Ref2VA-compatible weights. Also avoid stacking a distilled adapter with quality: "high": both alter denoising, and that combination has not been quality-validated.

LoRAs trained for a pruned or structurally modified ComfyUI graph are not automatically compatible with the native H3 weights. Use only adapters whose architecture and target modules match the full native H3 checkpoint.

6. Sampling and output controls

MiniMax-H3 supports more than one output per prompt. The video API accepts num_outputs_per_prompt (or OpenAI-compatible n) from 1 through 10. Offline generation accepts --num-outputs-per-prompt N; --num-outputs N is the short alias. A scalar seed is expanded deterministically as seed + output_index, so the outputs do not reuse the same noise. Same-prompt fan-out reuses text conditioning. On the verified 2× RTX 5090 recipe, a 5-step two-output request completed in 155.39 seconds versus 78.11 seconds for one output, while producing two distinct valid MP4 files. The independent denoise and decode passes remain sequential on this 32 GB profile to keep peak memory bounded; the grouped path adds essentially no orchestration overhead. Use server replicas when lower wall-clock latency for many variants matters more than per-server memory efficiency. For example, set "num_outputs_per_prompt": 2 in any request above. After the job completes, download both outputs by selecting each zero-based variant:

Command

video_id="<completed-job-id>"
for variant in 0 1; do
  curl -sS -L \
    "http://127.0.0.1:30010/v1/videos/${video_id}/content?variant=${variant}" \
    -o "minimax-h3-${variant}.mp4"
done

Choose the quality level

quality is a request-scoped sampling parameter with two validated levels:

"lossless" (default): the exact reference path. Output is bit-exact against the reference implementation and the CI ground truth.
"high": the audited accelerated path. Quality is guaranteed (the audited Cache-DiT configuration measures SSIM 0.931 / PSNR 28.16 dB against lossless), but output is no longer bit-identical to the reference.

One resident server serves both levels; a quality: "high" request mounts its audited Cache-DiT policy at the batch boundary, and a later quality: "lossless" request removes the hooks before denoising. Start the validated server once:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 4 \
  --tp-size 1 \
  --sp-degree 4 \
  --ulysses-degree 4 \
  --ring-degree 1 \
  --encoder-parallel auto \
  --performance-mode speed \
  --use-fsdp-inference false \
  --enable-torch-compile false \
  --port 30010

Then choose a request level:

lossless (default)
high

Native denoising with no feature-cache approximation. This is the default; omitting the field is equivalent.

Request field

{
  "quality": "lossless"
}

The audited accelerated path. Use it when you can trade bit-exactness for latency while keeping output closest to the same-seed lossless trajectory.

Request field

{
  "quality": "high"
}

The measured trade-off is:

`quality`	Mean inference latency	Speedup	SSIM vs lossless	PSNR vs lossless	Expected trade-off
`lossless`	75.10 s	1.00×	1.000	exact	Native reference path
`high`	53.70 s	1.40×	0.931	28.16 dB	Smallest same-seed visual change

These numbers use 1344×768, 124-frame, 24 fps T2VA with 50 inference steps, video flow shift 12, audio flow shift 3, and three fixed prompt/seed pairs on 4×H200. The prompts cover a quiet detailed scene, fast multi-subject action, and a moving close-up portrait. inference_time_s is averaged across the three prompts; the quiet-scene point is itself the mean of two repeats. SSIM and PSNR compare decoded, frame-aligned output with the lossless result for the same prompt and seed. They measure trajectory deviation, not absolute perceptual quality: the high path can produce a different but still plausible realization. It also changes the joint audio-video denoise trajectory, while these two metrics cover video only. quality: "high" currently accepts only the exact workload and 4×H200 deployment above; other hardware, task modes, request shapes, step counts, or flow shifts fail before denoising. Offline generation uses the same level name, for example sglang generate --quality high.

quality selects a model sampling level and can change generated content. output_quality controls only output-file compression; it is a separate field.

For manually tuned Cache-DiT experiments outside that validated path, omit the request quality field and set the process-wide environment controls directly. An explicit quality: "lossless" request overrides those controls and restores native denoising:

Command

SGLANG_CACHE_DIT_ENABLED=true \
SGLANG_CACHE_DIT_FN=1 \
SGLANG_CACHE_DIT_BN=0 \
SGLANG_CACHE_DIT_WARMUP=4 \
SGLANG_CACHE_DIT_RDT=0.12 \
SGLANG_CACHE_DIT_MC=2 \
sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --performance-mode speed \
  --port 30010

Cache-DiT skips selected block computation and is approximate. It cannot be combined with FSDP inference or DiT layerwise offload. Breakable CUDA graph execution takes precedence and leaves Cache-DiT disabled. Tune the cache thresholds only after comparing both video and audio quality on the target task profile. A real B200 request has completed, but the quality: "high" path above remains fail-closed to the audited 4×H200 workload.

7. Feature contracts and advanced recipes

The generated command already contains the recommended topology and encoder setting. Use the detailed reference below only when applying an optional override or checking its installation, topology limits, and validation evidence.

Lossless runtime
Attention backends
Online quantization
Encoder scheduling

The recommended speed launch already combines resident components with Ulysses sequence parallelism. Validation status below applies only to the listed hardware and topology; it is not inherited by a similar GPU family.

Feature	Validation status	Notes
Ulysses sequence parallelism	Verified: 8× B200, 4× H200, 4× H100, and Ulysses1/2/4/8 on MI300X and MI355X	Use `--ulysses-degree`. Combine with Ring for cross-node scaling; see the next row.
Ring sequence parallelism (cross-node)	Verified: 2 nodes of 8× H200 each (Ulysses8 × Ring2)	Use `--ring-degree` together with `--nnodes`/`--node-rank`/`--dist-init-addr`. Ring shards the sequence across nodes while Ulysses shards heads within a node; H3’s packed multi-segment attention only supports Ring across the node boundary, not within a single node’s Ulysses group. Requires `--encoder-parallel replicate` — `auto`’s fold decision is not node-boundary aware. See the benchmark section below.
Tensor parallelism	Verified: B200 TP2 + Ulysses4; H100 TP2 + Ulysses2 and TP4 + Ulysses1	`--tp-size` may be combined with Ulysses when the TP-local head count remains divisible by the Ulysses degree. On 4×H100, TP2 + Ulysses2 is the measured speed default.
FSDP inference	Verified: 4× B200 and 4× H100 + Ulysses4	Preserves H3’s mixed BF16/FP32 parameter policy. B200 completed the exact eager comparison; H100 completed consecutive real requests at about 57 GB peak memory per GPU.
Resident components	Verified: B200, H200, 4×H100 with TP, and 1/2/4/8× MI300X and MI355X	This is the recommended single-request latency path when the complete workload fits.
CPU and layerwise offload	Verified: 2× RTX 5090 TP2; 1× RTX 4090 24 GB	The 5090 lossless recipe keeps 20 DiT blocks plus both VAE encoders resident, streams the remaining DiT blocks, text encoder, and video VAE decoder blocks, and leaves the small audio VAE resident. The 4090 recipe streams DiT and the text encoder with zero resident DiT layers and omits `vae` from `--layerwise-offload-components`.
Breakable CUDA graph	Verified: B200 Ref2VA, opt-in	Matching eager output was observed for the captured signature, without a measured speedup. Re-capture for other shapes and reference sets.
`torch.compile`	Measured: H200, opt-in	Steady-state benefit was below measurement noise, while startup increased and numerical output changed. Do not use it for consistency ground truth.

The verified parallel, placement, and matching-signature BCG paths keep the BF16/FP32 weights and denoising math. torch.compile is the exception called out above. Always use the eager BF16/FP32 launch when producing CI consistency ground truth.For the validated 1344×768 Ref2VA profile, use a 5504-row text bucket so both the server warmup and reference-conditioned requests share the captured signature:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --performance-mode speed \
  --enable-breakable-cuda-graph true \
  --warmup-resolutions 1344x768 \
  --bcg-text-buckets 5504 \
  --port 30010

BCG is lossless for a matching captured signature, but capture reserves extra GPU memory. Re-measure the live H3 text length before reusing this bucket for a different task profile, reference set, resolution, or prompt template.

Leave --attention-backend unset for the platform default. Use --attention-backend fa only for an explicit FlashAttention comparison.SageAttention uses quantized attention math and is not a consistency mode. To select H3’s native packed-varlen Sage path, install the dependency and add --attention-backend sage_attn. On Hopper, install the upstream SM90 binding fix rather than the PyPI 2.2.0 build:

Command

pip install --force-reinstall \
  git+https://github.com/thu-ml/SageAttention.git@d9704247a5139ab4c03bf7fc6b35cc0e2cbb5ea4 \
  --no-build-isolation

The backend is a server-wide default. Use --component-attention-backends only when a measured component needs a different kernel, and keep the platform default for every component not named in the override.

On the verified 8× B200 topology, quantize the BF16 transformer at server load:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --performance-mode speed \
  --quantization fp8 \
  --port 30010

H3 automatically keeps its video/audio patch projections, timestep MLP, and final video/audio heads in FP32. All other linear layers have stable full module prefixes, so additional layers can be kept unquantized:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --encoder-parallel auto \
  --quantization fp8 \
  --quantization-ignored-layers blocks.0.attn token_refiner \
  --port 30010

Online FP8 is approximate and is not a consistency ground-truth mode. It can be combined with Cache-DiT, but the two approximations compound. Validate visual quality, audio quality, memory use, and latency on the target workload. This recipe is limited to the resident B200 and B300 topologies used for real H3 validation runs.

On a single 24 GB card, use kitchen_int8 instead of FP8. It quantizes the four GEMMs per DiT block online from the Hub BF16 weights (data-free, no calibration) and dispatches them through comfy_kitchen.int8_linear. Quantization happens after H3’s grouped qkv reorder, so do not load an externally pre-quantized INT8 checkpoint here.

Command

sglang generate \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --quantization kitchen_int8 \
  --attention-backend fa \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder \
  --dit-offload-prefetch-size 1 \
  --dit-layerwise-resident-layers 0 \
  --enable-torch-compile false \
  --prompt "A cat walking on a sunny beach, gentle waves." \
  --save-output

fa keeps exact attention. For a faster, approximate DiT path, use --attention-backend sol_attn with --attention-backend-config dense_backend=sage_attn,dense_steps=10 and --component-attention-backends text_encoder=torch_sdpa,transformer=sol_attn. See Quantization and Attention Backends.

kitchen_int8 changes Linear numerics. sol_attn / sage_attn also change the attention algorithm. Neither is a consistency ground-truth mode. The BF16 path is unchanged when comfy-kitchen is not installed.

The Qwen3-VL text encoder can be replaced independently of the DiT. To reduce its resident memory, point the text-encoder component at the serialized FP8 checkpoint used in validation:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --component-paths.text_encoder Qwen/Qwen3-VL-32B-Instruct-FP8 \
  --num-gpus 4 \
  --tp-size 2 \
  --ulysses-degree 2 \
  --performance-mode speed \
  --port 30010

--text-encoder-path is accepted as a shorter alias. No separate quantization flag is required: SGLang reads the checkpoint’s quantization_config and fails closed if the native encoder does not support that format. The language linear layers use FP8 while embeddings, normalization, and the vision tower remain BF16. This is an approximate serve-time choice and is incompatible with the strict quality="high" deployment contract.The mixed W4A8 community encoder follows the same component-path contract:

Overlay

--component-paths.text_encoder \
  Winnougan/MiniMax-H3-INT4_Convrot_ComfyUI/qwen3vl_32b_minimax_h3-w4a8_convrot.safetensors

Install comfy-kitchen>=0.2.27 and omit --quantization. SGLang automatically loads its W4A8 language linears and tensorwise INT8 embedding; the unmarked vision tower remains BF16.W4A4 Qwen3-VL files use the same overlay, for example:

Overlay

--component-paths.text_encoder \
  Merserk/MiniMax-H3-INT4-ConvRot/qwen3vl_32b_minimax_h3_int4_convrot.safetensors

This checkpoint keeps its unmarked embedding and vision tower in their source precision; no explicit component quantization option is needed.The official Comfy NVFP4-AWQ encoder uses the same flagless overlay:

Overlay

--component-paths.text_encoder \
  Comfy-Org/MiniMax-H3/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors

SGLang auto-detects its row-wise INT8 embedding, NVFP4 language linears, and AWQ input pre-scales. The weights stay compressed at rest; each active linear is dequantized for a BF16/FP16 matrix multiplication, so this path primarily reduces resident memory rather than encoder latency. Do not add a component or transformer quantization option.The same component option accepts a self-describing Quanto qint8 file without an additional quantization flag:

Overlay

--component-paths.text_encoder \
  DeepBeepMeep/MiniMax-H3/Qwen3-VL-32B-Instruct/Qwen3-VL-32B-Instruct-layer50_quanto_bf16_int8.safetensors

This variant keeps the declared language and vision linear weights in qint8 storage, then dequantizes only the active matrix for BF16/FP16 linear math. Use it as a memory option, not as an INT8 throughput claim. The embedded Quanto map is the selector; adding --quantization would describe a different operation.Qwen3-VL GGUF files use the same component option:

Overlay

--component-paths.text_encoder \
  DeepBeepMeep/MiniMax-H3/Qwen3-VL-32B-Instruct/qwen3vl-32B-MiniMax-H3-Q4_K_M.gguf

The file selects its own Q4/Q6 layout; do not add --quantization. SGLang keeps the aligned vocabulary table and language linears in packed GGML storage. The smaller Comfy vision tower, whose flattened blocks can cross row and TP shard boundaries, is restored one matrix at a time to BF16 during loading. This CUDA-only memory option supports encoder TP and layerwise placement, but not encoder FSDP. Like the Quanto variant, it is approximate and is not a throughput claim.

The picker explicitly writes --encoder-parallel auto in every single-node recipe. At the default request batch size of one, H100/H200/B200/B300 servers with peer-to-peer access fold the Qwen encoder across otherwise idle Ulysses ranks. A pure-TP recipe keeps the encoder inside its TP group, while a PCIe-only host can avoid an expensive world fold. Keep auto unless one of the cases below applies.Encoder DP is a throughput policy for compatible request batches. It requires TP1 and DiT DP1, replicates the encoder weights, and does not improve a batch of one:

Overlay

--encoder-parallel dp \
--batching-max-size 2

The cross-node picker recipe already uses replication because the automatic fold decision is not node-boundary aware:

Overlay

--encoder-parallel replicate

8. Configuration notes

MiniMax-H3 produces the canonical 24 fps output; request duration is expressed through target.duration_seconds.
target.duration_seconds must be between 4 and 15 seconds, inclusive. The command picker defaults to the verified 5-second profile.
Use a 768-pixel short edge for the released quality recipe. The aligned output dimensions are derived from target.aspect_ratio.
flow_shift controls video diffusion and audio_flow_shift controls audio diffusion.
V2V uses task: "ref2va" with a video or video_audio reference; it is served by the Ref2VA partition and is not a separate public task value.
conditions[].start_time_seconds selects a non-negative offset for a video reference. Its visual and audio streams are always sought together.
Ref2VA condition order is semantic and must match the one-based material tags in the prompt. For Ref2VA, target.aspect_ratio: "auto" resolves to the model’s 16:9 fallback rather than inheriting a reference asset’s geometry.
The distilled pipeline uses a single denoising branch, so CFG parallelism does not apply. Do not enable it: --enable-cfg-parallel true or --cfg-parallel-size greater than 1 is rejected instead of duplicating the positive branch. Explicitly disabling CFG, or setting its size to 1, remains a valid no-op.
The released visual VAE quality recipe uses overlapping tiled decode. SGLang keeps that recipe by default and distributes complete tiles across the decode group; this changes scheduling, not the computation inside each tile.
H3 rejects --vae-config.parallel-decode-mode spatial and spatial_shard: validation found output mismatches. Use the default released tiled recipe.
Keep the default --encoder-parallel auto. With the server’s default batching_max_size of 1, single-node H100/H200/B200/B300 recipes with peer-to-peer access fold the Qwen text encoder over otherwise idle Ulysses ranks. This is separate from DiT tensor parallelism. A pure-TP recipe already shards the encoder over its TP group and does not add a world fold.
For throughput-oriented serving, select DP (batched throughput). The picker pairs --encoder-parallel dp with an editable --batching-max-size greater than 1. Encoder DP stays inside each DiT replica and composes with encoder TP: the H100 TP2 + Ulysses2 recipe has two TP-sharded encoder copies that can split a batch, while the RTX 5090 pure-TP2 recipe has one encoder copy and therefore no additional batch-DP degree. It provides no benefit for a batch of one and is not bitwise-identical to the unsplit deployment.
Use explicit Fold to prioritize single-request latency and encoder memory on a measured high-bandwidth single-node topology. Use Replicate as the compatibility path when folding or encoder DP is unsuitable.
--use-fsdp-inference true shards only the DiT. MiniMax-H3 preserves the original FP32 dtype of its patch, time, and output projections during FSDP all-gather, so this path does not trade numerical correctness for memory. On 4×H100, prefer TP2 + Ulysses2 for speed; use FSDP as an explicit capacity policy rather than assuming it is faster.
speed keeps model components resident, while auto applies the model-aware 120 GiB residency threshold. memory prioritizes avoiding OOM and includes the executable VAE decoder in its default layerwise set. A measured recipe with sufficient headroom can opt into --component-residency vae=resident; the 2×H100 CI recipe does this because the VAE’s 4.8 GiB/GPU cost avoids repeated decoder transfers during tiled decode. DiT residency and prefetch knobs remain scoped to the DiT. Use speed only after confirming that the complete target workload fits.
Breakable CUDA graph execution is an explicit opt-in, not part of the recommended speed preset. It requires --enable-breakable-cuda-graph, every served size in --warmup-resolutions, and --bcg-text-buckets that cover the live H3 condition sequence. The validated 1344×768 Ref2VA recipe uses 5504; other task profiles and reference sets may need a different value. It preserves eager output for matching captured signatures, but graph capture consumes additional GPU memory and may provide little latency benefit when Ulysses attention and collectives dominate, so benchmark it on the target topology before enabling it.

9. Benchmarks

The picker exposes resident and FSDP profiles on NVIDIA datacenter GPUs. GPU counts are properties of the selected recipes, not a claim that every platform requires that many GPUs. The detailed tables below report performance only for the configurations with collected measurements:

Hardware	Default resident recipe	Other profile or topology
B300	8× Ulysses8 resident	8× FSDP + Ulysses8; the 8-GPU sweep is not a minimum-GPU claim.
B200	8× Ulysses8 resident	4× FSDP + Ulysses4
H200	4× Ulysses4 resident	4× FSDP + Ulysses4; 4× TP2 + Ulysses2; 2 nodes × 8× Ulysses8×Ring2 cross-node
H100	4× TP2 + Ulysses2 resident	4× TP4 + Ulysses1; 4× FSDP + Ulysses4
Ascend NPU	8 NPUs, TP2 + SP4, Laser Attention	4 NPUs, TP2 + SP2, Laser Attention
MI300X / MI355X	8× Ulysses8 resident	1×, 2×, and 4× scaling runs
RTX 5090	2× TP2 + layerwise offload	—
RTX 4090 24 GB	1× layerwise offload + `kitchen_int8`	Approximate attention backends are opt-in

Ascend NPU topology comparison

Both topologies used Laser Attention and the explicit Cache-DiT configuration from the Ascend launch commands, with --dit-cpu-offload false keeping the DiT resident. The measured workload was one 5-second T2VA request at 1344×768, 124 frames, 24 fps, and 50 inference steps.

NPU count	Topology	End-to-end latency
8	TP2 + SP4	55.07 s
4	TP2 + SP2	103.57 s

These are individual end-to-end measurements for each topology, not averages. The eight-NPU topology had 46.8% lower end-to-end latency than the four-NPU topology.

B300 precision and encoder placement

A 12-configuration sweep on a single 8× B300 host, covering both checkpoint partitions, both transformer precisions, and all three text-encoder placements. It answers one question — how long does one request take, and how much memory does it need.

What was measured

Hardware. 8× NVIDIA B300 SXM6, single node. Model. MiniMaxAI/MiniMax-H3, both released weight partitions. Serve command. Exactly the recipe the picker emits for B300, plus the one or two overlay flags under test:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant fl2va \
  --num-gpus 8 \
  --ulysses-degree 8 \
  --performance-mode speed \
  --host 0.0.0.0 \
  --port 30010

The swept axes are --model-variant (fl2va / ref2va), --quantization (unset for BF16 / fp8), and --encoder-parallel (auto / fold / replicate). Nothing else differs between the 12 servers. This is a single-request latency sweep (batching_max_size: 1), so encoder DP is intentionally excluded: it cannot distribute a batch of one. Use the DP for a request batch setting above for a compatible multi-request deployment; the table below does not claim a measured H3 DP speedup. Driver.

Command

python3 -m sglang.multimodal_gen.benchmarks.bench_serving \
  --host 127.0.0.1 --port 30010 \
  --model MiniMaxAI/MiniMax-H3 \
  --dataset vbench --task text-to-video \
  --num-prompts 1 --max-concurrency 1 \
  --warmup-requests 1 --warmup-inference-steps 50 \
  --extra-body '{"task":"t2va","conditions":[],"target":{"short_edge":768,"aspect_ratio":"16:9","duration_seconds":5.0},"seconds":5,"flow_shift":12.0,"audio_flow_shift":3.0}'

Workload

Property	Value
Output duration	5.167 s
Resolution	1344×768
Frames	124 @ 24 fps
Denoising steps	50
`flow_shift` / `audio_flow_shift`	12.0 / 3.0
Requests in flight	1 (`--max-concurrency 1`, server at `batching_max_size: 1`)
Requests measured	1 per cell, after 1 warmup request

Results

Weights	Precision	Encoder	Load	Warmup	Latency	Peak/GPU
FL2VA	BF16	auto	118.1 s	29.65 s	19.04 s	83,578 MB
FL2VA	BF16	fold	114.0 s	28.72 s	19.04 s	83,578 MB
FL2VA	BF16	replicate	116.0 s	28.33 s	19.04 s	124,158 MB
FL2VA	FP8	auto	116.0 s	27.16 s	18.03 s	51,926 MB
FL2VA	FP8	fold	116.0 s	25.99 s	18.04 s	51,926 MB
FL2VA	FP8	replicate	118.0 s	27.97 s	18.04 s	92,506 MB
Ref2VA	BF16	auto	114.0 s	38.69 s	29.12 s	83,968 MB
Ref2VA	BF16	fold	118.0 s	36.58 s	29.13 s	83,968 MB
Ref2VA	BF16	replicate	116.0 s	35.17 s	29.13 s	124,490 MB
Ref2VA	FP8	auto	124.0 s	34.30 s	27.12 s	52,816 MB
Ref2VA	FP8	fold	112.0 s	34.44 s	27.12 s	52,816 MB
Ref2VA	FP8	replicate	116.0 s	33.42 s	27.12 s	93,396 MB

H200 topology comparison

The same four-card H200 host completed both lossless resident placements with the standard 1344×768, 5-second, 50-step T2VA request (fixed prompt and seed, eager BF16/FP32, back-to-back runs on an otherwise idle host). Latency is the warmed-up request; the first pair uses the default warmup request, the second pair adds --warmup-resolutions 1344x768 so warmup already covers the served resolution:

Topology	Warmup	Denoise	Decode	E2E	Peak/GPU
Ulysses4	default	79.04 s	3.77 s	84.14 s	94,288 MB
TP2 + Ulysses2	default	81.17 s	2.97 s	85.51 s	63,490 MB
Ulysses4	`--warmup-resolutions 1344x768`	71.73 s	1.32 s	74.38 s	94,290 MB
TP2 + Ulysses2	`--warmup-resolutions 1344x768`	75.52 s	1.29 s	78.33 s	63,490 MB

Ulysses4 stays the H200 latency default: 5.0 % faster end-to-end than TP2 + Ulysses2 once warmup covers the served resolution (1.6 % with the default warmup, where first-request cold start masks the topology gap). TP2 + Ulysses2 shards the DiT weights and holds peak memory about 30 GB per GPU lower, which is why it remains the 80 GB H100 recipe. Matching the warmup request to the served resolution removes the cold first-request cost on both topologies (about 10 s end-to-end on this workload).

H200 cross-node scaling

Long references and long durations grow the packed sequence length, and Ulysses alone cannot scale sequence parallelism past the GPU count of one node without either violating head-count divisibility or exposing all-to-all traffic across the slower inter-node link. H3 combines node-local Ulysses with cross-node Ring: Ring’s point-to-point KV rotation is designed to overlap with attention compute, which fits a slower cross-node link better than an all-to-all does. Hardware. 2 nodes × 8× NVIDIA H200 SXM, same cluster, InfiniBand between nodes. Serve command. The cross-node cell the picker emits for H200, run identically on both nodes with --node-rank set to 0 and 1:

Command

sglang serve \
  --model-path MiniMaxAI/MiniMax-H3 \
  --model-variant ref2va \
  --num-gpus 16 \
  --nnodes 2 \
  --node-rank {{NODE_RANK}} \
  --dist-init-addr {{NODE0_IP}}:20000 \
  --sp-degree 16 \
  --ulysses-degree 8 \
  --ring-degree 2 \
  --encoder-parallel replicate \
  --performance-mode speed \
  --host 0.0.0.0 \
  --port 30010

What was measured. A controlled denoise-stage comparison on identical hardware: 8× H200 single-node (Ulysses8, no Ring) versus the same 16-GPU cross-node command above (Ulysses8 × Ring2), holding prompt, seed, and step count fixed:

Task	Single-node (Ulysses8)	Cross-node (Ulysses8 × Ring2)	Change
T2VA denoise/step	0.749 s	0.477 s	−36.3%
Ref2VA/V2V denoise/step	2.572 s	1.494 s	−41.9%

The gain grows with sequence length because Ring’s per-hop communication cost stays roughly constant while attention compute grows quadratically with sequence length, so V2V’s longer packed sequence benefits more than T2VA’s shorter one. With the point-to-point KV rotation pipelined against attention compute, one V2V request’s full denoise stage completed in 68.1–68.3 seconds versus 128.6 seconds on the single-node 8-GPU baseline (−47.0%), with byte-identical output to the unpipelined cross-node path. Cross-node determinism was confirmed separately: the same request run twice against the same cross-node deployment produced byte-identical output. A cross-node run’s output is not expected to bit-match a single-node run of the same prompt and seed — Ring’s online-softmax merge across hops accumulates floating-point operations in a different order than single-node attention, which is an expected source of bit-level difference, not a correctness regression.

--encoder-parallel auto’s fold decision is not yet node-boundary aware and attempts to fold the text encoder across nodes, which crashes the Ref2VA reference-conditioned encoder. Always pass --encoder-parallel replicate explicitly for cross-node H3 deployments.

H100 topology comparison

The same four-card H100 host completed three lossless placements. TP2 with Ulysses2 was the fastest; TP4 used the least memory:

Topology	Pipeline latency	Peak/GPU
TP2 + Ulysses2	13.25 s	66.04 GB
FSDP + Ulysses4	13.36 s	57.01 GB
TP4 + Ulysses1	13.86 s	49.80 GB

RTX 5090 capacity run

The verified two-card RTX 5090 host used TP2 with layerwise offload. The full 50-step, 1344×768, 5-second request completed in 559.67 seconds: 525.05 seconds of denoising and 33.61 seconds of decoding, with a 26.3 GiB sampled peak per GPU.

DiT settings	5-step denoise	Inference	Peak/GPU	Result
prefetch 1, resident 20	43.48 s	78.11 s	26.3 GiB	Selected recipe
prefetch 2, resident 20	43.37 s	78.06 s	27.5 GiB	No measurable gain
Ulysses2, prefetch 2, resident 10	Did not reach warmup	—	—	Rejected

Consumer GPU tuning

On consumer hardware the binding question is not which card you have but how much host RAM sits behind it. H3’s weights are about 108 GB — 61.73 GB of DiT and 46.18 GB of text encoder — so no consumer configuration holds them all, and where the shortfall lands decides the throughput. The command — most consumer machines need exactly one flag beyond the model:

consumer single GPU, lossless

sglang serve --model-path MiniMaxAI/MiniMax-H3 --model-variant fl2va \
  --layerwise-offload-components dit,text_encoder,vae

With 16 GB of VRAM or more, add --layerwise-resident-layers video_vae=36 for the 13 s decode; with ~96 GB of host RAM and 16 GB+ of VRAM, add --dit-layerwise-resident-layers 4 for the 6 s step. That is the whole flag surface. The builder at the top of this page has consumer cards and a Host RAM selector: pick your budget and it emits this command with your tier’s measured expectations attached as comments. The table below is the same data in one view. Two budgets, and what each one buys

	12 GB VRAM + 32 GB host	host free, VRAM 16 GB
Recipe	A	B
Peak VRAM	≤ 12 GiB	≤ 16 GiB (OOMs at 12)
Host anonymous (must fit)	24.5 GiB	116.7 GB pinned
Denoise, 864×480 / 124 frames / 20 NFE	16.8 - 18.7 s/it	6.01 s/it
Runs at all	yes	yes

The left column is one configuration measured twice, at 318.94 s and 356.37 s; the 12% spread tracked host load on a shared machine, so treat smaller differences than that as unresolved. The right column is 120.92 s at a 16 GiB allocator cap. Four resident DiT layers is what Recipe B buys its speed with, and it is also why 12 GiB is not enough for it. Read the host row carefully, because the two numbers are not the same kind of memory. Anonymous host memory — pinned buffers and pageable copies — has to fit, and the kernel cannot reclaim it. Page cache backing a file mapping is droppable, so it does not count against the budget even though it shows up in VmRSS; use RssAnon from /proc/<pid>/status when checking. Likewise measure VRAM with torch.cuda.set_per_process_memory_fraction and let the allocator fail, rather than reading nvidia-smi, which reports the caching allocator’s reserved pool and overstates the requirement. Inside 32 GB the weights cannot be pinned, so each denoise step copies about 60 GiB from the checkpoint mapping, and a mapped source is synchronous however the copy is requested: the driver stages it through its own buffer, so the transfer neither overlaps compute nor runs at pinned bandwidth. That is where the step goes, and giving the host room to pin the weights instead is what takes it to 6.01 s. Two caveats on the constrained number, both from instrumenting the run rather than from arithmetic. The machine it was measured on has 2 TB of host memory, so the kernel kept all 107.7 GiB of mapped checkpoint pages resident: major faults across a whole request were 6, and read_bytes was zero. Nothing was read from disk. A real 32 GB host cannot cache 107.7 GiB, so it will fault and re-read, and should be expected to be slower than the figures here rather than equal to them — an NVMe is a requirement, not a recommendation. Measure your own machine with major faults (/proc/<pid>/stat) on the worker process, not on the launcher, which holds no weights. Recipe A — fits 12 GB VRAM + 32 GB host

12 GB + 32 GB, lossless

PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True \
sglang serve --model-path MiniMaxAI/MiniMax-H3 --model-variant fl2va \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder,vae \
  --layerwise-resident-layers video_vae=36

Recipe B — host memory is free (the fast path)

unconstrained host, lossless

sglang serve --model-path MiniMaxAI/MiniMax-H3 --model-variant fl2va \
  --performance-mode memory \
  --layerwise-offload-components dit,text_encoder,vae \
  --dit-layerwise-resident-layers 4 \
  --layerwise-resident-layers video_vae=36

Recipe B pins ~112 GB of host memory (DiT 61.56 GB, text encoder 46.18 GB, VAE ~4.5 GB in its decode dtype). Do not reach for it on a 32 GB machine. What not to change, and why

video_vae=36 holds every decoder block for the decode only — residency arms at the decoder’s first block and releases when it finishes, so the denoise still runs on an empty card. It fits 12 GB because decoder weights are held in their decode compute dtype (fp16) from load, which halves them to ~4.9 GiB; the rounding was already part of every output (the decode computes in fp16 autocast), so the result is bit-identical, and the decode drops from 60 s streamed (or 209 s on a busy host) to ~10 s. The expandable_segments line stays: the decode sits close enough to the cap that fragmentation otherwise tips it over.
Leave --enable-torch-compile off, as elsewhere on this page. Layerwise offload rebinds param.data on every layer, so compiled graphs do not get the benefit they would on resident weights.
Recipe A’s flags are what the automatic policy should choose on its own. Until the model declares its own placement, --performance-mode memory plus the explicit component list is what makes it happen; pass them.

Reading the startup log The server prints the memory decisions it made; checking three lines against your budget catches a mis-set machine in the first minute instead of the first request.

Layerwise offload: host memory available: N GiB — what the runtime sees after loading, not your DIMM size. On a 32 GB host expect single digits here; a much larger number means another process’s memory accounting (or a container limit) is in play.
leaving N GiB of weights on the checkpoint mapping — the expected line on a 32 GB host: the DiT streams from the checkpoint file. If instead the log reports pinned weights, the runtime decided your host has room — which is faster, and means the 32 GB figures above do not apply to you.
Loaded video_vae: ... host mmap vs host pageable — where the VAE landed (decoder weights are ~4.9 GiB once held in their decode dtype). Loaded <component> lines carry the same buckets for every component.

If a request dies after the denoise finishes, it is the decode colliding with the cap: keep the expandable_segments line, and if it persists drop to video_vae=24 and take the partially streamed decode. Against ComfyUI, on the same weights Same unpruned bf16 checkpoints, same card, same sampler settings (cfg 1.0, euler_ancestral, sigma shift 12.0/3.0, seed 1101), 864×480 / 124 frames / 20 NFE: When host memory is free, the engines are close and sglang is ahead:

	denoise	host anonymous	peak VRAM
sglang, Recipe B	6.01 s/it	116.7 GB pinned	≤ 16 GiB
ComfyUI KSampler	6.58–6.59 s/it	116.5 GiB	13048 MiB

Inside 12 GB, both engines run these weights, and one measurement convention matters on each side. ComfyUI’s memory manager reads system RAM and adapts, so the rows below patch psutil to a pretend host size — the same convention the sglang rows use. Its --reserve-vram is also soft: told to keep 12 GiB free it still peaked at 13.5 GiB, a figure a real 12 GB card cannot give it, so both engines here run under the same hard allocator cap (set_per_process_memory_fraction), where its peak stays at 12.1–12.3 GiB. Under that cap, Recipe A wins the whole request at every host size:

12 GB VRAM, both engines hard-capped	sglang Recipe A (TE + denoise + decode)	ComfyUI, bf16 (warm)
32 GB host	12.4 + 212.8 + 9.4 ≈ 235 s	276–302 s
48 GB host	15.8 + 192.1 + 10.0 ≈ 218 s	246–267 s
64 GB host	7.5 + 162.4 + 10.3 ≈ 180 s	194–195 s

Same GPU, same load window, unpruned bf16 checkpoints, outputs verified. All figures are anchored at 480P — activations grow with the pixel count, so at 768P drop the resident DiT layers to 0 first, then video_vae to 24 if the decode still collides. And the host convention holds the weights in page cache; a physical 32 GB machine re-reads them from disk each step, so the page cache cannot hold the per-step weight sweep, so every step re-reads it from disk and the drive becomes the denoise clock: a real desktop 4090 with a 990 Pro measured 38 s/step, reading 52.9 GB per step (faulted sequentially, so almost none of it shows in majflt — measure read_bytes, not major faults). Two things cut that read directly: resident DiT layers (~1 GB/step each — on a physically small host raise them as far as VRAM allows, the opposite of the capped-host guidance above), and more RAM (64 GB caches the sweep and returns to the quoted times). The VRAM axis holds too: capped at 16 GiB the same recipe wins ~250 vs 292–301 s, and at 24 GiB (with --dit-layerwise-resident-layers 6 — measured at a 22 GiB cap so a desktop’s own allocations fit; a headless card can raise it to 10 for under 1% more) ~8.5 s/step vs ComfyUI’s 249–260 s requests. Four changes carry it: the VAE staying on its checkpoint mapping (#35862, root fix #35946), per-layer pinning with net-cost accounting (#35867), the courier thread that ships still-mapped layers through pinned slots (#35882), and decoder weights held in their decode dtype from load (#35967) — which is what lets video_vae=36 fit and turns the decode from the slowest stage (54–96 s streamed) into the fastest (~10 s, faster than ComfyUI’s own 15–25 s). Output equivalence is bit-level: the fp16-held decode reproduced the fp32-store run’s video byte for byte, and the audio stream is bit-identical. Stage by stage under the cap: text encoding is even (both stream the same 48 GB Qwen3VL), the denoise leads at 32–48 GB hosts and sits within run-to-run variance of ComfyUI at 64 GB (162 vs 159 s), and the decode leads everywhere. Two ComfyUI notes that still matter: --fast-disk measured no faster than its default here, and stacking --novram --cache-none --disable-pinned-memory made things strictly worse (69.1 GiB anonymous, 750 s requests) — the adaptive default is the right configuration on a small host. The path ComfyUI ships for 12 GB cards uses minimax_h3_fl2va_pruned_int8_convrot and qwen3vl_32b_minimax_h3_nvfp4_awq, i.e. an int8 DiT and an NVFP4 text encoder, and its pruned bf16 file is 40.2 GB against the unpruned 66.3 GB. Those are different weights, so it is not a like-for-like comparison with the recipes above.

RTX 4090 24 GB single-GPU run

One RTX 4090 D 24 GB completed the 1344×768, 107-frame, 20-NFE T2VA workload (euler, torch.compile and step caching disabled) with DiT and text-encoder layerwise offload. Same process: load → warmup (seed 0) → timed (seed 42); only the timed pass is reported. GPU peak stayed about 18 GB.

Config	Timed e2e	Denoise	vs BF16	PSNR vs BF16
BF16 + FlashAttention	405.6 s	370.2 s	1.00×	—
`kitchen_int8` + FA	303.3 s	273.7 s	1.34×	24.81 dB
`kitchen_int8` + `sol_attn`	223.9 s	203.9 s	1.81×	24.44 dB
`kitchen_int8` + `sage_attn`	174.9 s	154.2 s	2.32×	23.51 dB
`kitchen_int8` + Sage→Sol hybrid	163.8 s	143.1 s	2.48×	23.04 dB

kitchen_int8 + FA changes Linear numerics only. The sol_attn / sage_attn / hybrid rows also change the attention algorithm, so speed and pixel fidelity rank in opposite orders there. Default remains kitchen_int8 + fa.

AMD Instinct task and scaling runs

The AMD recipes keep the released BF16/FP32 precision policy and use AITER packed attention. The picker emits the fastest measured topology, 8 GPUs with Ulysses degree 8. All runs below completed full H.264/AAC decoding and representative-frame inspection.

Hardware	Task	Denoise	Decode	Peak/GPU
MI355X	T2VA	55.2907 s	9.5344 s	97,444 MB
MI355X	FL2VA	53.7978 s	9.4477 s	96,922 MB
MI355X	Ref2VA	41.3812 s	6.8247 s	94,518 MB
MI300X	T2VA	167.4878 s	25.3244 s	97,272 MB
MI300X	FL2VA	150.2311 s	12.5684 s	96,750 MB
MI300X	Ref2VA	107.6232 s	11.3768 s	94,268 MB

The task matrix used 8 GPUs and 50 denoising steps. The scaling matrix uses one 1344×768, 209-frame T2VA request and changes only the GPU count and matching Ulysses degree:

Hardware	GPUs	Denoise	Decode	Peak/GPU
MI355X	8	55.2907 s	9.5344 s	97,444 MB
MI355X	4	104.2294 s	11.1824 s	103,350 MB
MI355X	2	223.0246 s	15.5330 s	115,250 MB
MI355X	1	288.7968 s	24.0472 s	137,676 MB
MI300X	8	167.4878 s	25.3244 s	97,272 MB
MI300X	4	297.3727 s	26.5067 s	103,436 MB
MI300X	2	585.5401 s	29.4909 s	115,010 MB
MI300X	1	978.0886 s	36.0142 s	137,626 MB

For a measured lower-count AMD deployment, set both --num-gpus and --ulysses-degree to 4, 2, or 1. AITER packed attention matched segment-wise BF16 SDPA at cosine similarity 0.9999991655 on MI355X and 0.9999991059 on MI300X.

​1. Quick start

​2. Model capabilities

​3. Deployment details

​Pre-quantized GGUF transformer

​AdaLN-pruned safetensors transformers

​Advanced: precomputed AdaLN cache

​Serve MiniMax-H3 on Ascend NPUs

​4. Generate video and audio

​5. LoRA recipes

​6. Sampling and output controls

​Choose the quality level

​7. Feature contracts and advanced recipes

​8. Configuration notes

​9. Benchmarks

​Ascend NPU topology comparison

​B300 precision and encoder placement

​What was measured

​Results

​H200 topology comparison

​H200 cross-node scaling

​H100 topology comparison

​RTX 5090 capacity run

​Consumer GPU tuning

​RTX 4090 24 GB single-GPU run

​AMD Instinct task and scaling runs

1. Quick start

2. Model capabilities

3. Deployment details

Pre-quantized GGUF transformer

AdaLN-pruned safetensors transformers

Advanced: precomputed AdaLN cache

Serve MiniMax-H3 on Ascend NPUs

4. Generate video and audio

5. LoRA recipes

6. Sampling and output controls

Choose the quality level

7. Feature contracts and advanced recipes

8. Configuration notes

9. Benchmarks

Ascend NPU topology comparison

B300 precision and encoder placement

What was measured

Results

H200 topology comparison

H200 cross-node scaling

H100 topology comparison

RTX 5090 capacity run

Consumer GPU tuning

RTX 4090 24 GB single-GPU run

AMD Instinct task and scaling runs