r/CUDA • • 4h ago

Sm_120 RTX 5070 causal conv1d and Ssm wheels

2 Upvotes

RTX 5070/5080/5090 (sm_120, compute 12.0) currently fails with "no kernel image is available" because setup.py only builds for 70;80;90.

I built working wheels for Blackwell:

\- Patch: TORCH_CUDA_ARCH_LIST=12.0 and replace sm_70/sm_80/sm_90 with sm_120 in setup.py

\- Tested on 5070 WSL2 + torch 2.11.0+cu128 + py3.10

\- Wheels here for anyone stuck: https://github.com/Rapa2026/mamba-ssm-5070-blackwell-wheels

Happy to PR the sm_120 fix if you want it.

Test: Mamba2(d_model=64).cuda() -> OK


r/CUDA • • 9h ago

Replaced most inline PTX with MLIR ops, cutting asm regions from 499 to 81 across 20 kernels.

8 Upvotes

While building a high-level language with a rich type system, a core goal is reducing the need for low-level escape hatches like inline asm.

In glm-vx(GLM-5.3 serving engine with kernels in Vx-lang, checkpoint loading, and streaming HTTP), we replaced most inline PTX with MLIR ops, cutting asm regions from 499 to 81 across 20 kernels.

When LLVM sees the surrounding code, it can simplify logic and cut spills thus generating better code.

e.g., A bf16 store. The epilogue of the GEMM converts each f32 result to bf16 and stores 16 bits. The asm converted into a 16-bit register and then widened it to 32 bits with cvt.u32.u16, only for the store to use the low 16 bits again. Now the store takes the 16-bit register, and fp8_gemm_m64 has 64 fewer instructions:

// before
{ .reg .b16 h; cvt.rn.bf16.f32 h, %r393; cvt.u32.u16 %r199, h; }
st.global.b16  [%rd65], %r199;

// after
cvt.rn.bf16.f32  %rs1, %r327;
st.global.b16    [%rd74], %rs1;

But when hoisting/reordering hurts (like %laneid or mma scheduling on H200), we have no option to keep the asm. Link to the full writeup in comments.


r/CUDA • • 22h ago

why it's so hard to make SASS assembler?

Thumbnail redplait.blogspot.com
14 Upvotes

summary:

  • only ~80% of ELF sections currently known
  • ~50% of EIATTRs
  • instruction scheduler is highly gpu specific and can't be implemented based on llvm machine scheduler. And np-hard