Hi! Steppe-Wan2.1 is my new generative latent upscaler for Wan 2.1 latent space, which also include Krea 2, Qwen Image (don't confuse with Qwen Image 2.1, it has other VAE) and some other models. It is set of model of different scales trained from scratch on a large dataset of images and video (millions of samples, ~300 GPU hours) to perform 2x super-resolution task directly in latent space. Unlike the "traditional" two stage approach where a first deterministic latent upscaler makes blurry latents and then large generative model fixes it through some denoising steps, this model's output can be decoded directly (pic 3). This is much faster and also keeps more information from low resolution latents while achieving best fidelity, reconstruction and some other metrics (pic 4, 5).
- ComfyUI nodes: https://github.com/tensorforger/ComfyUI-Steppe (or simply search "steppe" in manager)
- Hugging Face demo space (you can try it in browser on your images): https://huggingface.co/spaces/TensorForger/Steppe
- Weights: https://huggingface.co/TensorForger/Steppe-Wan2.1 (ComfyUI node downloads required weights automatically)
Key features:
- Support images and videos.
- For video it can do both spatial and temporal upscaling, even both in a single forward pass
- You can control the output sharpness through High Frequency Features Conditioning (HFF), pic 6. It works like a "prompt" describing how sharp details should be in the output. The model doesn't take prompt by the way, it understand image content from input low resolution sample alone (pic 7).
- There are three models in this release: multi-step 200M parameter base model (best control, can use CFG over the HFF), it's distilled single step version (also 200M, best for most tasks) and light 20M distilled single step model (for extremely high resolution upscaling or real-time video upscaling).
- Support different aspect ratios, resolutions and video lengths because it it trained on there different combinations.
Deeper implementation details:
Development codebase: https://github.com/tensorforger/STDev
- The model's architecture was discovered through long agentic autoresearch loop (pic 9). Claude Opus 5.5 run ~120 experiments trying everything starting from convolutional UNets with different configurations, transformers, various hybrid architectures, training losses, objectives, batch and lr schedules... It was inspired with https://github.com/karpathy/autoresearch, but adapted to the task and run in parallel on 8 GPUs. The agent's task was to achieve best patch_fid metric while having 30 minutes of RTX5090 compute. Then best runs was scaled to 2 hour training, best in total was scaled up and used for base model pretrain. You can read the logs of autoresearch in dev repository.
- The resulted architecture is some combination of Swin transformer with local window attention, convolutional layers, two input low resolution sample paths (learned and not) and some actually novel training stability improvements.
- The model is local both in space and time: the window attention can be computed in chunks and temporal path is fully causal (so it can be used for frame by frame streaming). These optimizations reduce VRAM usage and they are implemented in ComfyUI nodes.
- To reproduce benchmarks see dev repository.
Now some thoughts about all this
It is quite surprising how many different approaches are there to get high resolution image from generated latents. Conceptually, they are "Decode, then Upscale", "Upscale, then Decode" and "Upscale during Decode". "Decode, then Upscale" hits several problems of expensive high resolution compute and sensitivity to AI-generated artifacts. Any small mistake of decoder becomes larger. "Upscale during Decode" is quite interesting. It is spacepxl/Wan2.1-VAE-upscale2x and Nvidia-Pid. I can't say it is bad or something, it is a good direction too. And if the latency is critical for you spacepxl/Wan2.1-VAE-upscale2x is still a great alternative. Nvidia-Pid is just bad) It completely changes structure and has huge color shift, so it was not even benchmarked here. "Upscale, then Decode" can also be done in many ways. I already mentioned "traditional" approach above. But there is another one: fuse upscaler with large generative model. Examples are "Hildegard" for Flux.2 and new Nvidia's Open-H3-Regenerate-2K. My model is another point in this space, and I believe this is a good direction.