Elon Musk’s artificial intelligence (AI) venture, xAI, made its large language model (LLM) Grok-1 available in open source on March 17. Last week, the billionaire revealed on his social media platform X (formerly known as Twitter) that the AI chatbot will be open-sourced. Now, it is available for researchers and developers. Notably, the xAI developers stated that only the pre-trained LLM has been released in the public domain, which means while the weights and network architecture can be accessed to build on top of it, Grok does not come with any training data.
Announcing the open release, xAI stated in a blog post, “We are releasing the base model weights and network architecture of Grok-1, our large language model. Grok-1 is a 314 billion parameter Mixture-of-Experts model trained from scratch by xAI.” The AI firm also noted that the LLM is being provided in open source under the Apache 2.0 licence. Interested individuals can access the AI model from GitHub.
As per a report by VentureBeat, the Apache 2.0 licence includes commercial usage, as well as modification and redistribution. This means developers can build on top of the LLM, fine-tune it for specific applications, and sell it. However, the model cannot be trademarked and developers will have to mention any changes made to the original code.
But while a commercial licence is being provided, the version of Grok-1 that has been made available is from October 2023, before the model was trained using data from X and it gives a unique personality to the Grok chatbot. This means any researchers or developers intending to use the model will have to procure the data themselves.
xAI stated that the open-sourced Grok-1 is a 314 billion parameter Mixture-of-Experts (MoE) model. The parameter size is much larger than other LLMs in the public domain, such as Meta LLaMa 2 or the Mistral 8x7B. The larger the parameter size, the bigger the context window of the AI. This enables it to provide more contextually accurate and thorough responses.
Grok was unveiled on November 3, 2023, as a chatbot designed to compete with the major players in the segment. It was made available to a limited set of users who purchased the X Premium+ subscription. Musk also said at the time that Grok will get a standalone app.