Is AI models Big Tech’s new “business moat” that will keep all competitors away?
By: Jan Olsen, Connectivity and IOT thoughts
AI models are changing the world in many ways, from enabling personalized advertising and recommendation systems to driving medical research and advancing autonomous vehicles. However, the question of whether AI models will completely change the world is more complex.
On the one hand, AI models have the potential to transform many industries, from healthcare to transportation to finance, by improving efficiency, accuracy, and productivity. They can also help solve some of the world’s most pressing challenges, such as climate change and disease outbreaks.
On the other hand, AI models are not a magic bullet, and they are not without limitations and risks. They are only as good as the data they are trained on, and biases in that data can lead to flawed decision-making. There are also concerns about the impact of AI on employment and job displacement, as well as the potential for AI models to perpetuate social inequalities and discrimination.
Moreover, there is a limit to what AI models can achieve. They are excellent at performing certain tasks, such as image and speech recognition, but they lack human-like creativity, empathy, and intuition. There are also ethical and moral considerations surrounding the development and use of AI, particularly in areas such as military applications and surveillance.
So, it is clear that AI models have already changed the world in many ways and will continue to do so in the future. However, they are not a panacea, and some limitations and risks need to be carefully considered. The key is to approach the development and use of AI models with caution, responsibility, and a commitment to ensuring that they are used for the benefit of society as a whole.
How big an advantage is AI for Big Tech against smaller competitors?
The argument so far is that replicating large AI models like ChatGPT and other big-tech AI models is technically possible. Still, it would require a significant investment of resources and expertise.
A logical approach (that has worked well in other business segments) is to replicate these AI models would be to use open-source software and publicly available datasets to train smaller versions of the models. This would allow businesses, developers (and researchers) to experiment with the models and better understand how they work.
Another approach being tried is using cloud-based computing resources to train the models. Cloud computing providers like Amazon Web Services, Microsoft Azure, and Google Cloud Platform offer powerful computing resources that can be used to train large AI models. However, so far this still requires a significant investment of resources, as the cost of using these resources can quickly add up.
Some efforts are underway to develop more efficient and scalable AI models that can be trained on smaller datasets and using fewer computing resources. For example, researchers are exploring techniques like transfer learning, which involves reusing pre-trained models for new tasks, and model compression, which involves reducing the size of the models by removing unnecessary parameters.
Ultimately right now the story goes – replicating large AI models like ChatGPT and other big tech AI models requires “a deep understanding of the underlying algorithms and techniques”, as well as access to significant computing resources and data. While it is technically possible to replicate these models, it remains a challenging and resource-intensive task. Or so they want us to believe…
Enter MPT-7B-StoryWriter, and now big tech is in trouble (their words, not mine)
MPT-7B-StoryWriter is a revolutionary new LLM model that BEATS GPT-4 with an INSANE 65K+ token limit! (This means the new open-source AI model can read a book like The Great Gatsby in one go and generate an epilogue of the book in about 20 seconds (about 150k words per minute). Most of the proprietary models cant perform even close to this, as can be seen in the sources.
The new AI model sets were created from scratch by MosaicML in only 9,5 days and trained for approx. 200k$. The new AI reportedly learns very quickly and produces significantly better results than some of the big-tech AI models that are supposed to be worth billions of USD.
And the new open-source AI model is:
- Licensed for commercial use(unlike LLaMA).
- Trained on a large amount of data(1T tokens like LLaMA vs. 300B for Pythia, 300B for OpenLLaMA, and 800B for StableLM).
- Prepared to handle extremely long inputs thanks to ALiBi (we trained on up to 65k inputs and can handle up to 84k vs. 2k-4k for other open source models).
- Optimized for fast training and inference(via FlashAttention and FasterTransformer)
- Equipped with highly efficient open-source training code.
And the best part, since it is open source, anyone can install these new game-changing on something as simple as your own computer and test their super impressive capabilities yourself.
MPT-7B is a language model that is based on the GPT-3 architecture, similar to the ChatGPT model. However, the MosaicML team has made several improvements to the architecture and training process that they claim to make the model more efficient and effective.
One of the key innovations of the MosaicML approach is their use of multi-party training, which involves training the model on multiple datasets simultaneously. This allows the model to learn from a more diverse set of data and can help improve its performance.
In addition, the MosaicML team has developed a more efficient training process that allows the model to be trained on fewer data and with fewer computing resources than some of the big tech models.
The MPT-7B AI model is the next step forward in the commercial “AI war” and an excellent example of how open-source development can lead to innovations and advancements in AI research and development that quickly may leave all the advantages Big Tech thinks they have behind. By sharing their code and techniques with the broader community, MosaicML is helping to democratize access to AI technology and accelerate progress in the field.
Did anybody say, open office, Linux, and 100th of thousands of open source projects built by volunteer developers and code sharing among millions of contributors?
Against the open source environment, even giants like Google, Microsoft, Facebook, etc will quickly lose, and it seems like they know it…
INTERNAL GOOGLE DOCUMENT SAYS OPEN SOURCE AI WILL OUTCOMPETE GOOGLE AND OPENAI
Quote ”According to the document, after the open-source community got their hands on the leaked AI model, motivated and highly knowledgeable individuals set to work to take a fairly basic model to new levels where it could begin to compete with the offerings by OpenAI and Google. Major innovations are the scaling issues, allowing these LLMs to work on far less powerful systems (like a laptop or even smartphone)”.
And there we have it, the open-source environment “sees” 1000th solutions to a problem, tries 100th and adapts the best, and lets everyone learn from it. No secret sauce etc.
And from the leaked AI model, we now know there were indeed fairly simple solutions that massively cut down the effort and resources required to train a new AI model, and hence a lot of Big Tech competitive advances from the “large data sets” were gone.
As the leaked document phrases, it, “Google and by extension, OpenAI do not have a ‘secret sauce that makes their approaches better than anything the wider community can come up with”
And therefore, the proprietary AI Models by Google, OpenAI, Meta, Microsoft, and others will soon cease to be relevant, as the open-source community is right now steamrolling them into fine, digital dust.
Business by the speed of your network just hit Big Tech, and the crazy valuation for proprietary AI companies in the multi-billion-dollar range is about to crash.
Conclusion: There will be no moat in AI models for Big Tech
More in the links and the video below
Sources
https://www.youtube.com/watch?v=O9Y_ZdsuKWQ
