Hy4 open source: total parameter 77 billion, context 1 million
Hy4 preview, new model using MoE structure, total parameter 770B, 49B, every activation, and extension to 1M. Compared to the total parameter of 295B for Hy3, the active parameter of 21B and the context of 256K, the size of the generation has more than doubled and the context has been expanded to about four times. The call did not focus solely on generic running points, but on real tasks such as strengthening codes, offices, games and research. The model, for example, can be built on a zero-on-one basis from the Three.js 3D website, and Demo, which produces a playable game in Unity, can process 72 financial files at a time to check duplicate claims, amounts over limits and budget anomalies. Hy4 is also training and iterative with telecommunication products such as WorkBuddy and CodeBuddy. More specifically, Hy4 has started to participate in its own research and development. It was stated that it would help to optimize training methods, data strategies, assessment systems and lower-level algorithms by proposing its own programmes, operating experiments and continuing to adapt to the results. The code, log and feedback from the experiment will then enter the next cycle of R & D, which has resulted in an initial self-improvement cycle. Hy4 preview has access to WorkBuddy, CodeBuddy, Yuanbao, Ima, TokenHub and OpenRouter. Before that, in the financial statements, Hy4 predicted that it would be bigger than Hy3 and used real product feedback as an important source of model training。
View Details