Hy4 open source: total parameter 77 billion, context 1 million

The hybrid Hy4 model, which was released by telecommunication, uses MOE structures, with a total parameter of 77 billion, with 4.9 billion activated each time, and the context expanded to 1 million. Compared to Hy3, the overall parameters of Hy4 have doubled and the long context has increased approximately fourfold. The call highlighted the ability of the model to be applied to real tasks such as code, office, games and scientific research, such as the creation of the Three-D website from the zero line, the production of games demo in Unity, and the processing of 72 financial documents to check duplicate claims and budget anomalies. Hy4 is also co-training and iterative with the products of the communicators, such as workbuddy and codebuddy, to form an initial self-improvement cycle. Hy4 preview has been connected to a number of telecommunication products, which were previously forecasted in the financial statements that Hy4 would be larger than Hy3 and used real product feedback as an important source of model training。