r/sdforall • u/Royal_Being_5084 • 22h ago
Discussion Comparing GPU options after RunPod’s cold starts and availability got unworkable
Comparing GPU options after RunPod’s cold starts and availability got unworkable
We’d been running our image gen on RunPod for a while, and for most of that time it was fine. Eventually, though, the waiting became more annoying than the price.
Cold starts on bigger models are real, and it’s not just me. RunPod’s own writeup showed a 32B model with a median cold start of around 324 seconds on default settings, dropping to about 91 seconds after four configuration changes. A lot of that time is spent on the model side: loading weights, torch.compile, CUDA graph capture, and everything else that happens after the machine comes up.
Storage was another problem. Your network volume is tied to one datacenter, which means GPU availability in that particular location becomes the limiting factor. If the card you need is out of stock there, you’re either waiting for capacity or copying a few hundred gigs of data somewhere else. That kind of defeats the point of having everything ready to go.
For bursty workloads, none of this matters that much. You spin up, run your jobs, and shut everything down. It became a problem once we started running things closer to all day. Keeping workers warm solves some of the waiting, but then you’re paying for idle GPUs around the clock.
So I started looking at alternatives.
Vast seems to be the cheapest by a pretty wide margin, but you’re renting from marketplace hosts, so hardware quality and availability can vary.
Lambda and CoreWeave seem like more predictable options if you just want rented GPU capacity without dealing with individual hosts.
Then there’s the option of owning the hardware. We looked at B3IQ, where you buy the machine and they rack and manage it for you. The obvious downside is that you’re paying for the entire box whether you’re using it or not. They also help arrange offtake contracts, so supposedly you can rent out unused compute instead of letting it sit idle.
What I’m still trying to figure out is whether owning the hardware actually solves the cold start and availability problems, or whether you’re just trading those for hardware failures, maintenance, and the risk that offtake demand doesn’t hold up.