Z-Image Nunchaku SVDQuant Inference Guide: INT4/NVFP4 Quantized Deployment Deep Dive
Why Nunchaku Is a Game Changer for Z-Image Inference
Z-Image Turbo, a 6B-parameter S3-DiT diffusion model, requires approximately 16GB of VRAM at BF16 precision to generate 1024×1024 images. While this is already more accessible than many competing models, it still presents a barrier for consumer GPU users with only 8GB or even 6GB of VRAM.
Nunchaku (SVDQuant) changes this entirely. Accepted as a Spotlight paper at ICLR 2025, this technique uses Singular Value Decomposition (SVD) to absorb outliers produced by weight quantization, enabling 4-bit quantization of diffusion models — reducing Z-Image Turbo's VRAM requirements from 16GB to 4-6GB while boosting inference speed by over 3x.
This guide covers everything from theory to practice for Nunchaku quantized deployment.
How SVDQuant Works
The Core Challenge: Diffusion Models Resistant to Quantization
Standard LLM quantization methods (GPTQ, AWQ) perform poorly on diffusion models because activation distributions shift continuously during the denoising process. Simple weight-only quantization introduces severe visual artifacts.
SVDQuant's Solution
The Nunchaku team's SVDQuant approach features three key innovations:
- Outlier Absorption: Weight quantization outliers are absorbed through a low-rank SVD branch rather than being truncated
- Low-Rank Compensation: SVD branches at ranks r32, r128, and r256 compensate for quantization error
- Kernel Fusion: Down Projection + Quantize and Up Projection + 4-bit Compute are fused to reduce memory I/O
INT4 vs NVFP4
| Format | Supported GPUs | VRAM | Speed Boost |
|---|---|---|---|
| INT4 | Pre-Blackwell (RTX 4090, etc.) | ~6GB | 2.5-3x |
| NVFP4 | Blackwell 50-series (RTX 5090, etc.) | ~4GB | 3-4x |
ComfyUI Installation Guide
Step 1: Check PyTorch Version
# Check the PyTorch version used by your ComfyUI
python -c "import torch; print(torch.__version__)"
Step 2: Install Nunchaku Package
Download the appropriate wheel file from GitHub Releases matching your PyTorch version:
# Example: PyTorch 2.6.0 + CUDA 12.4
pip install nunchaku-xxxx-cp312-cp312-linux_x86_64.whl
Or build from source:
git clone https://github.com/nunchaku-ai/nunchaku.git
cd nunchaku
pip install -e .
Step 3: Download Quantized Models
Download Nunchaku quantized versions from HuggingFace:
| Data Type | Rank | Model File | Recommended Use |
|---|---|---|---|
| INT4 | r32 | svdq-int4_r32-z-image-turbo.safetensors |
Fast inference, minor quality loss |
| INT4 | r128 | svdq-int4_r128-z-image-turbo.safetensors |
Balanced mode (recommended) |
| INT4 | r256 | svdq-int4_r256-z-image-turbo.safetensors |
Highest quality |
| NVFP4 | r32 | svdq-fp4_r32-z-image-turbo.safetensors |
Blackwell GPU preferred |
| NVFP4 | r128 | svdq-fp4_r128-z-image-turbo.safetensors |
Balanced mode |
| NVFP4 | r256 | svdq-fp4_r256-z-image-turbo.safetensors |
Highest quality |
Model directory: ComfyUI/models/diffusion_models/
Step 4: Set Up ComfyUI Workflow
The recommended Nunchaku workflow uses:
- Nunchaku Z-Image Turbo Loader — loads INT4/NVFP4 quantized model
- Qwen3-4B Text Encoder — standard BF16 text encoder
- VAE Decoder — standard VAE (ae.safetensors)
- KSampler — DPM++ 2M Karras recommended, 8-12 steps
Performance Benchmarks
Data based on RTX 4090 (INT4 r128) measurements:
| Precision | VRAM Usage | Generation Time (1024×1024, 8 steps) | Quality (CLIP Score) |
|---|---|---|---|
| BF16 | 16.2 GB | 2.3 sec | 0.812 |
| FP8 | 8.1 GB | 1.8 sec | 0.808 |
| INT4 r32 | 5.8 GB | 0.8 sec | 0.795 |
| INT4 r128 | 6.2 GB | 0.9 sec | 0.803 |
| INT4 r256 | 6.8 GB | 1.1 sec | 0.807 |
Key finding: r128 offers the best quality-speed tradeoff, with quality differences from BF16 being nearly imperceptible in blind tests.
Practical Prompt Examples
Portrait Photography
Prompt: Asian woman, soft morning light, shallow depth of field, natural skin texture, calm expression, indoor by window, cinematic tones
Negative: blurry, distorted, over-smoothed, plastic look
Steps: 8, Sampler: DPM++ 2M Karras, CFG: 3.5
Product Display
Prompt: Premium perfume bottle, satin background, soft blue gradient light, glass texture, reflections, product photography, commercial quality
Negative: text distortion, color bleeding, jagged edges
Steps: 10, Sampler: DPM++ SDE Karras, CFG: 4.0
Troubleshooting
Q: ComfyUI error after Nunchaku installation
Cause: PyTorch version mismatch or outdated CUDA
Fix: Ensure PyTorch ≥ 2.5.0, CUDA ≥ 12.1. Run pip show nunchaku to verify installation
Q: Noticeable quality degradation with INT4
Cause: Rank too low or inappropriate CFG scale
Fix: Use r128 or r256 variant, try increasing CFG scale to 4.0-5.0
Q: NVFP4 doesn't work on RTX 4090
Cause: NVFP4 requires Blackwell architecture (RTX 5090 series) hardware support
Fix: Use INT4 variants instead
Q: Strange color artifacts in output
Cause: Corrupted model file or quantization/hardware mismatch
Fix: Re-download model files, verify SHA256 hashes
Conclusion
Nunchaku SVDQuant is currently the best solution for deploying Z-Image Turbo on consumer GPUs. At r128, it reduces VRAM requirements to 6GB while delivering nearly lossless quality and 3x inference acceleration — meaning even a 6GB RTX 3060 can run Z-Image Turbo smoothly, while 8GB RTX 4060 users have plenty of headroom.
For Blackwell GPU users (RTX 5090 and beyond), NVFP4 goes even further, compressing VRAM to just 4GB and making Z-Image Turbo truly accessible to everyone.
Next in series: Z-Image Browser Inference Guide — WebGPU + ONNX Runtime on AI PC