OpenAI’s CEO, Sam Altman, mentioned that the development of GPT-4 involved a large team and nearly all of OpenAI’s resources. However, retraining GPT-4 with only five people is now feasible due to significant advancements. The introduction of GPT-4.5 in February was hailed as OpenAI’s most powerful model yet, with breakthroughs that would facilitate the reconstruction of GPT-4. In a recent podcast episode, Altman posed a question to three key engineers involved in GPT-4.5, asking about the minimum team size required to retrain GPT-4 from scratch today. The team agreed that with the insights gained from GPT-4.5, the retraining process could now be managed by a team of five to ten individuals. This represents a stark contrast to the hundreds of people that were initially involved in building GPT-4. The shift towards more efficient data processing and algorithms has made rebuilding GPT-4 significantly simpler. Altman also highlighted that OpenAI is no longer constrained by computing power, signaling a significant development in their model-building capabilities. The surge in investment from tech giants like Microsoft, Amazon, Google, and Meta, in addition to OpenAI’s successful funding round, will further enhance the company’s computing capabilities. Looking ahead, the focus is on improving data efficiency to achieve the next leaps in scale and capability.
The article highlights how data can become a limiting factor. The individual mentioned that further progress will necessitate the development of new algorithms to maximize the potential of existing data. For more details, refer to the original piece on Business Insider.
