Stability AI, a renowned name in the AI industry, has recently announced the release of two new large language models (LLMs), namely FreeWilly1 and FreeWilly2. These models are a significant contribution to the AI community, as they are designed to advance research and promote open access.

The Birth of FreeWilly1 and FreeWilly2

These new models are based on Meta’s LLaMA and LLaMA 2 open-source models. However, they have been trained on a smaller dataset that includes synthetic data. The names FreeWilly1 and FreeWilly2 are a playful nod to the “Orca” AI training methodology developed by Microsoft researchers.

Performance and Capabilities

Despite being trained on just 10% of the size of the original Orca dataset, which equates to 600,000 data points, these models demonstrate exceptional performance. They excel in intricate reasoning, linguistic subtleties, and answering complex questions in specialized domains like law and mathematics. Their performance is comparable to, and in some cases even exceeds, that of ChatGPT on GPT-3.5.

Environmental Impact

The smaller dataset not only contributes to their performance but also makes these models more environmentally friendly. They use less energy and have a lower carbon footprint, making them a sustainable choice in the world of AI.

Addressing the Issue of Model Collapse

One of the concerns with LLMs is the potential for “model collapse,” where LLMs trained on AI-generated data perform worse than those trained on human-generated data. Stability AI has addressed this issue by training the FreeWilly models using synthetic data generated by other LLMs. The results showed that the FreeWilly models still performed well, indicating that synthetic data may be a viable solution to model collapse and the use of copyrighted or proprietary data.

Open Access and Licensing

FreeWilly1 and FreeWilly2 are released under a non-commercial license. This means they are intended to advance research and promote open access in the AI community, rather than being used for business purposes. Researchers and developers can access the weights for FreeWilly2 as-is, while the weights for FreeWilly1 are released as deltas over the original model.

Conclusion

Stability AI’s release of FreeWilly1 and FreeWilly2 sets new standards in open access LLMs. These models are empowering natural language understanding and enabling complex tasks, making them a significant contribution to the AI community.

FAQs

1. What are FreeWilly1 and FreeWilly2?

FreeWilly1 and FreeWilly2 are large language models released by Stability AI. They are based on Meta’s LLaMA and LLaMA 2 open-source models but have been trained on a smaller dataset that includes synthetic data.

2. What is the performance of these models?

Despite being trained on a smaller dataset, these models demonstrate exceptional performance. They excel in intricate reasoning, linguistic subtleties, and answering complex questions in specialized domains like law and mathematics.

3. How does Stability AI address the issue of model collapse?

Stability AI has addressed the issue of model collapse by training the FreeWilly models using synthetic data generated by other LLMs. The results showed that the FreeWilly models still performed well.

4. Can these models be used for commercial purposes?

No, FreeWilly1 and FreeWilly2 are released under a non-commercial license. They are intended to advance research and promote open access in the AI community, rather than being used for business purposes.

5. How can researchers and developers access these models?

Researchers and developers can access the weights for FreeWilly2 as-is, while the weights for FreeWilly1 are released as deltas over the original model.