NVIDIA’s TensorRT LLM VisualGen combines NVFP4 quantization, fused kernels, and multi-GPU parallelism to cut diffusion latency.

NVIDIA published a tutorial showing how FLUX 3 Image Klein Base and its distilled variant run through TensorRT LLM VisualGen on Blackwell GPUs. Combined optimizations reached up to 3.46x faster text-to-image and 4.45x faster image-to-image generation at 2048×2048 compared with uncompiled BF16 inference on a single B200.

Black Forest Labs trained FLUX 3 Image Klein on NVIDIA GB200 NVL72 systems. The model pairs Qwen3-VL-4B-Instruct for text conditioning with a diffusion transformer and an autoencoder, and it ships in Base and distilled checkpoints that trade latency against fidelity. NVIDIA VisualGen adds NVFP4 quantization, fused PrimsTS GEMM kernels, and torch.compile support.

Quantized attention and caching give operators concrete knobs. VC-Attention FP8 with a 0.005 repair budget delivered 1.18x speedups at 2048×2048 with LPIPS scores of 0.086 and 0.151, while Cache-DiT step caching nearly halved Base latency and hit 1.94x at 2048×2048 with an LPIPS of 0.033. Sol-Attention sparse attention added 1.30x faster denoising and decoding for Base at 12 dense steps.

Multi-GPU scaling with CFG and Ulysses parallelism across eight B200 GPUs cut median text-to-image latency from 14.90 to 3.00 seconds for Base and from 2.73 to 1.08 seconds for the distilled model. Builders can run the FLUX 3 example end to end with the provided YAML configurations and inspect the runtime APIs for their own pipelines.

What matters

• Combined optimizations speed text-to-image work by 3.46x and image-to-image work by 4.45x at 2048×2048.
• Eight B200 GPUs cut median text-to-image latency for the Base model from 14.90 seconds to 3.00 seconds.
• The tutorial ships YAML configs and optimization APIs, and builders can file issues on the TensorRT GitHub.

Why it matters

The tutorial ships YAML configs and optimization APIs, and builders can file issues on the TensorRT GitHub.

This GenAI News article was prepared in original wording using reporting and materials published by NVIDIA Developer Blog. Source reference: https://developer.nvidia.com/blog/speed-up-image-generation-with-flux-3-image-klein-on-nvidia-blackwell-gpus/.

Drafted by the GenAI News review pipeline.