r/vulkan • u/PhysicsDisastrous462 • 8h ago
Hierarchos-Native v0.2.5 is finally out! P3 WMMA update
so, after quite a bit of work since the v0.2.0 release, i've finally managed to get v0.2.5 published!
for anyone who followed the previous release, v0.2.0 was where i first introduced the opt-in RDNA3 WMMA implementation. since then i've been working on improving the Full FP16 path, specifically its numerical accuracy and performance, without having to sacrifice the execution structure that was already working.
the biggest change in this release is something i've been calling P3.
previously, the Full FP16 WMMA kernel merged its eight accumulator-bank partials using multiple intermediate FP16 additions. this meant that the result was rounded to FP16 several times along the way. P3 changes that final merge so all eight partials are summed in FP32, with just one final rounding to FP16.
the actual two-wave workgroup structure, K-slice stream, bank assignment, staging, cooperative multiplication and barriers are all preserved. i wanted to improve the accuracy of the final result without having to redesign the kernel all over again.
some of the results from the new qualification:
- 37/37 numerical gates passed across all eight qualification seeds.
- 32/32 preserved K=96 outputs were bit-identical to the previously qualified P3 GPU results.
- REF64F16 maximum error improved from
0.03035259to0.02593803. - 31 of 32 cases improved, with zero cases getting worse.
- The maximum tile16 distance is
0.03125, meeting the existing hard numerical limit.
i also ran another full performance sweep. this time using 24 paired measurement blocks per workload, GPU timestamps, and corrected 95% confidence intervals.
the measured improvements against the preserved Main kernel baseline were:
2048×768×2048— 1.0243×512×2048×768— 1.1246×512×768×768— 1.1163×128×8192×768— 1.2722×
all four protected workloads passed independently, including the lower bounds of their confidence intervals. the largest measured improvement was around 27%.
just to clarify, these are per-dispatch GPU GEMM measurements, not end-to-end training speedups. i don't want to imply that an entire training run is suddenly 27% faster when that isn't what was measured.
there were some other changes too.
i tightened up the production dispatch admission so the profitability predicate is checked against each actual batch, rather than relying only on the maximum batch size supplied when creating the projection. the existing 16,000,000 MAC profitability floor has not changed.
i also went through the actual portable binaries and verified that they dispatch the P3 kernel when an eligible workload is supplied. the packaged trainer produced 8 cooperative P3 dispatches, and the packaged logits executable produced another 3. disabling WMMA resulted in zero cooperative dispatches, as expected.
the numerical comparison for the packaged logits test covered 262,144 finite elements, with a maximum absolute difference of 0.000738330 and RMS difference of 0.000122237.
the new release is available as a portable Windows x64 package too.
i've included seven executables, and the package was built with a statically linked MSVC runtime. there's no need to install Rust, Cargo, Python, PyTorch, Visual Studio, or the MSVC redistributable just to run it.
you will still need a compatible Vulkan-capable GPU driver and the Vulkan runtime loader. model weights aren't included.
one thing i want to make clear is that WMMA is still opt-in globally. the default is still off, and you have to explicitly select full-fp16 to use the P3 path. P3 is simply the default implementation within that mode. the Mixed precision path is separate and hasn't been replaced.
the hardware qualification was performed on my AMD Radeon Graphics adapter, PCI ID 1002:15bf (RDNA3 / Phoenix). i haven't qualified this on every RDNA3 GPU, RDNA3.5, RDNA4, Intel or NVIDIA hardware, so please don't assume universal compatibility.
there are still things i haven't been able to test, including independent clean-machine/second-device acceptance, GUI interaction, Windows 10 acceptance, and code signing. the executables are currently unsigned.
anyway, i'm really happy to finally have this one out. v0.2.0 was the first public release of the WMMA implementation, and v0.2.5 is a pretty substantial improvement to that foundation. a lot of time went into checking that the numerical improvements were real, that the performance was actually measurable, and that the distributed binaries were doing what they were supposed to do.
Release: https://github.com/necat101/Hierarchos-Native/releases/tag/v0.2.5
Source: https://github.com/necat101/Hierarchos-Native
thank you to everyone who's been following the project, testing things, reporting issues, or just keeping an eye on development. it genuinely means a lot! thank you so much everyone!! :3 also, sorry for the AI posts lately, I've been trying to save as much time as possible to work on the project itself due to live events getting tough! this one is completely human written! :3
