A research team has proposed a new AI model compression method that seeks to break the common trade-off between "the smaller the model, the weaker its performance". Multiverse Company presented on August 25 at Hugging Face blog that they compress the OpenAI Open GPT-OSS 120B to 60 billion parameters and press further to the 4-bit expression, the model performed better than the corresponding 60B full accuracy version in multiple tests.
7 of 9 tests won
The research team named it Quantification-Aware Health. According to its disclosure, the compressed 60B model contained 7 of the 9 baseline tests that exceeded the "unquantified " version of 60B, i.e., a corresponding model that was usually considered to be of higher quality.
That doesn't mean that the compressed version has completely exceeded the original 120B large model. The article mentions that the original version of 120B is still stronger in most comparisons, but that the compressed version crosses a previously unusual threshold: Better local test results can still be achieved with lower RAM and fewer parameters.
- Original model: GPT-OSS 120B
- Compressed size: 60B parameters
- Memory indicates: 4-bit
The point is that the subject changed.
It is common practice to downsize the large model and then allow smaller models to imitate this "median version". The problem is that this intermediate version itself has lost some of its capacity during the compression process. The small model so trained is usually only close to this “failed answer”.
The approach has changed the training objectives. The research team did not allow the small model to continue the condensed intermediate version, but rather directly allowed it to learn the results from the original, uncompressed large model. In other words, compression steps are no longer just performance losses, but are being used as an additional supervisory training opportunity.
More attractive for local deployment
If this result can be replicated on more models, it is primarily at the deployment end. The smaller size of the parameter and the lower bit width mean that both the visibility and the power required to run the model will decline. According to the article, this “rehabilitated” version requires only about a quarter of the memory of the original model, while reducing the number of parameters by half.
This would have a direct impact on the ability of the model to move from the data centre to the desktop device and even further to the mobile end. For laboratories with limited budgets, independent developers and locally deployed users, the decline in reasoning costs is in itself an important change.
Open-source weights published
Multiverse Company indicates that the compressed Hypernova-60B has been published in Hugging Face in open weight and can be downloaded by the user. However, the compression tools used to generate this model are still proprietary and the complete process is not fully open at present.
Additional information:The test, which currently covers only GPT-OSS, has not been extended to other mainstream model families such as Llama, Qwen or Mistral, and the interoperability of the methodology remains to be followed up.