Alibaba Cloud contributes to the open-source community

Author: Isha Jain

Alibaba Cloud has announced its latest contribution to the open-source community by open-sourcing its seven billion parameter Large Language Models (LLM), Qwen-7B and Qwen-7B-Chat, through its AI model community, ModelScope, and the collaborative AI platform, Hugging Face.

The company has introduced its proprietary LLM, Tongyi Qianwen, earlier this year in April. This model, capable of generating human-like content in both Chinese and English, has different model sizes, including seven billion and above parameters.

In an effort to democratise AI technologies, the models’ code, model weights, and documentation will be freely accessible to academics, researchers and commercial institutions worldwide. For commercial uses, the models will be free to use for companies with fewer than 100 million monthly active users. Programs with more users can request for a license. 

“By open-sourcing our proprietary large language models, we aim to promote inclusive technologies and enable more developers and SMEs to reap the benefits of generative AI,” says Jingren Zhou, CTO of Alibaba Cloud Intelligence. “As a determined long-term champion of open-source initiatives, we hope that this open approach can also bring collective wisdom to further help open-source communities thrive.”

The Qwen-7B was pre-trained on over two trillion tokens, including Chinese, English and other multilingual materials, code and mathematics, covering general and professional fields. Its context length reaches 8K. In training, this model was aligned with human instructions. Both models can be deployed on cloud and on-premises infrastructures. This enables users to fine-tune the models and build their own high quality generative models effectively and cost efficiently.

The pre-trained Qwen-7B model distinguished itself in the Massive Multi-task Language Understanding (MMLU) benchmark, scoring a notable 56.7, outperforming other major models with similar scales or even some larger size models. This benchmark assesses a text model’s multitask accuracy across 57 varied tasks, encompassing fields such as elementary mathematics, computer science and law. Moreover, it achieved the highest score among models with equivalent parameters in the leader board of C-Eval, a comprehensive Chinese evaluation suite for foundational models. It covers 52 subjects in four major specialities including humanities, social sciences, STEM and others. Additionally, it reached outstanding performance on benchmarks of mathematics and code generation, such as GSM8K and HumanEval. 

Click here for latest data centre news.