[ad_1]
Ask anyone in the open source AI community, and they’ll tell you that the gap between them and big private companies is more than just computing power. Ai2 fixes that, first with fully open source databases and models, and now with an open and easily adaptable post-training system for converting “raw” large language models (LLMs) into usable models.
Contrary to what many believe, “basic” language models do not emerge from the training process ready to go. Pre-training is of course necessary, but it is by no means sufficient. Some even think that pre-training may soon no longer be the most important part of it all.
This is because the post-training process is increasingly shown to be where real value can be created. This is where the model was formed from a giant, all-knowing network that would produce Holocaust denial talking points as easily as it would produce cookie recipes. You generally don’t want that!
Companies are conservative about their post-training systems, because while anyone can surf the web and make a model using the latest methods, making that model useful to, say, a therapist or research analyst is an entirely different challenge.
Ai2 (formerly the Allen Institute for AI) has spoken publicly about the lack of openness in ostensibly “open” AI projects, such as Meta’s Llama. While the model is truly free and anyone can use and modify it, the sources and process of making the prototype and how it is trained for public use remain carefully guarded secrets. It’s not bad – but it’s also not really ‘open’.
Ai2, on the other hand, is committed to being as open as possible, from exposing its data collection, curation, cleaning, and other pipelines to the exact training methods it uses to produce LLMs like OLMo.
But the simple fact is that few developers have the skills to run their own LLM courses to begin with, and even fewer can do subsequent training the way Meta, OpenAI or Anthropic do – partly because they don’t know how, but also because it’s complicated. Technically and time consuming.
Fortunately, Ai2 wants to democratize this aspect of the AI ecosystem as well. That’s where Tülu 3 comes in. It’s a huge improvement over the previous, more primitive post-workout process (called, you guessed it, Tülu 2). In the nonprofit’s tests, this resulted in scores on par with the more advanced “open” models available. It is based on months of experimentation, reading, interpretation of what adults insinuate, and lots of repeated training sessions.
Basically, Tülu 3 covers everything from choosing the topics you want your model to care about — for example, downplaying multilingual abilities but using mathematics and programming — to taking them through a long system of data curation, reinforcement learning, fine-tuning and preference from Adjusting to tweaking a bunch of meta parameters and other training processes that I couldn’t adequately describe to you. Hopefully, the result will be a much more capable model that focuses on the skills you need.
But the real point is to take another game out of the private companies’ toy box. Previously, if you wanted to create a custom-trained MBA, it was very difficult to avoid using the resources of a major company one way or another, or hiring an intermediary to do the work for you. Not only is this expensive, but it also involves risks that some companies cannot afford.
For example, research and medical services companies: Sure, you can use OpenAI’s API, or talk to Scale or someone else to customize an internal model, but both involve third-party companies in sensitive user data. If it’s unavoidable, just bite the bullet – but if not? For example, if a research organization releases an integrated pre- and post-training system that you can implement on-premises? This may be a better alternative.
Ai2 uses this himself, and it’s the best endorsement one can give. Although the test results they are publishing today use Llama as the base model, they are planning to release a model trained on the basis of OLMo and Tülu 3 soon which should offer further improvements over the baseline and is also fully open source, tip to the tail.
If you’re interested in how the model is currently performing, Try the live demo.
[ad_2]