r/LocalLLaMA • u/Aggressive-Sign-8167 • 2d ago
Question | Help Tencent Hy Team & Shunyu Yao Here! Ask Us Anything about Hy4 & Hy4 flash!

Fresh off launching Hy4 and Hy4 Flash, the Hy research team is here to discuss how they built the models, boosted efficiency, and made them better for everyday workflows.
Our participants today:
Blog: https://hy.tencent.ai/research/hy4-flash https://hy.tencent.ai/research/hy4
HF: https://huggingface.co/tencent/Hy4-flash https://huggingface.co/tencent/Hy4
345
Upvotes
View all comments
Show parent comment
21
u/edwardbai 1d ago
Good question. We went back and forth between GQA and MLA for quite a while. MLA has significantly higher compute intensity than GQA, which made spec decoding on inference GPUs compute-bound. However, at the same KV cache size, MLA delivers better model quality than GQA. Fortunately, with the introduction of DSA, we were able to bring down MLA's compute intensity during decoding. That ultimately tipped the balance, and we decided to go with MLA.