The rapid advancement of artificial intelligence has long been anchored in the belief that larger models equate to greater power. However, a transformative shift is on the horizon as the industry begins to reconsider this foundational assumption.
With rising operational costs, many users are now exploring the potential of smaller, more economical AI models. This emerging trend of cost-effective AI solutions is anticipated to reshape the industry, though its full impact remains to be seen.
Brian Armstrong, co-founder of Coinbase, predicts a significant transition, suggesting that a majority of AI tasks may soon migrate to these affordable models. He asserts, "Demand for intelligence is near infinite, but 80% of workloads will be running on 99% cheaper models within 12-18 months." Only a fraction, about 20%, will still rely on the latest generation models where maximum performance is crucial.
If Armstrong's forecast holds true, it would mark a pivotal change for the AI sector. Traditionally, companies have focused on quality, often opting for the most advanced models. However, if tasks can be effectively managed by less expensive models without compromising quality, it could lead to a substantial economic transformation within the industry, particularly affecting major players like OpenAI and Anthropic as they approach their IPOs.
Initial experiments indicate that, under the right conditions, cheaper models can deliver comparable results to their larger counterparts. For instance, legal AI startup Harvey recently demonstrated that it could cut inference costs by threefold while maintaining quality, partnering with Fireworks AI to optimize its processes.
Gabe Pereyra, co-founder of Harvey, emphasizes that while quality remains paramount in legal services, the definition of quality is evolving. It now involves selecting the most efficient model that yields accurate results rather than defaulting to the most powerful option.
This trend is often framed as a battle between major labs and open-weight models, but the real distinction lies between large and small models. Transitioning from a model like GPT-5.5 to a smaller alternative like GPT-5.4-mini could yield similar performance while significantly reducing costs.
Currently, a competitive price war is unfolding between in-house models from large labs and independently served open-weight models. The crucial question remains: which small model will prevail?
While it may seem intuitive to minimize computational resources, this approach contradicts the scaling-first mentality that has dominated the industry. Historically, labs have prioritized training the most compute-intensive models, driven by investor subsidies that encouraged the use of cutting-edge technology.
As token prices rise and funding slows, enterprises are now facing unprecedented cost pressures. Whether this will lead to a widespread adoption of smaller models remains uncertain; companies might opt to economize by reducing usage or scaling back on less promising projects instead.
Nonetheless, should it be proven that most applications can operate effectively on smaller models, it could significantly dampen the demand for intensive inference and prompt a reevaluation of the justification for training high-end models.