The escalating demand for computational power to support AI models is rapidly intensifying. However, the industry faces two significant challenges: sourcing the right chips and integrating them into data centers to start generating revenue.
General Compute, a pioneering neocloud company specializing in AI processing power for inference--when models actively respond to user queries--has developed solutions to these challenges. Their innovative approach has attracted a $15 million seed investment, valuing the company at $60 million, led by FUSE VC with contributions from Carya Venture Partners and Village Global Ventures.
Identifying the ideal chip is crucial. While GPUs have been in high demand, experts are recognizing that they may not be the optimal choice for running trained AI models. The inference phase demands different computational capabilities, prompting the design of new chip architectures. Nvidia's recent $20 billion acquisition of Groq and Cerebras' $57 billion IPO highlight this shift in the industry.
With supply constraints at these leading companies, General Compute's co-founders, CEO Finn Puklowski and CTO Jason Goodison, have turned to SambaNova. This Intel-backed chipmaker specializes in inference and is poised to make waves with its upcoming chip release. SambaNova's new architecture promises enhanced flexibility and increased memory for context storage during inference, potentially outperforming existing GPUs and specialized chips from competitors like Groq and Cerebras. Puklowski anticipates that the new chips will achieve 600 to 700 tokens per second, compared to about 250 tokens per second for GPUs.
General Compute has already placed an order for $300 million worth of SN50 chips, positioning itself as the inaugural neocloud to deploy this technology.
These innovative chips also address the second major challenge of data center integration. Unlike traditional water-cooled systems, SambaNova's chips are air-cooled and energy-efficient, allowing for installation in existing facilities without the need for extensive infrastructure upgrades.
Puklowski is actively pursuing colocation agreements, enabling General Compute to install its hardware in third-party facilities. This strategy extends beyond data center providers to include partnerships with crypto miners seeking to repurpose their infrastructure.
Last week, General Compute launched its cloud service, claiming to be the fastest in executing MiniMax 2.7, a notable open-source large language model (LLM).
Venture investor Joe Hassleman, who invested in Groq early on, has established Evercrest Partners, focusing on the AI sector, with General Compute as his inaugural investment. He draws parallels between SambaNova's collaboration with General Compute and other successful partnerships in the industry, emphasizing the importance of a diverse customer base for growth.
The future of AI architecture remains uncertain, with inference clouds representing a shift toward a landscape where multiple models coexist, and speed and cost become paramount. Innovations like General Compute's offerings could redefine how AI models are utilized, making processes faster and more efficient.