Chip challengers try to break Nvidia’s grip on AI market

Stay informed with free updates

Simply sign up to the Artificial intelligence myFT Digest — delivered directly to your inbox.

Nvidia’s rivals are mobilising in an effort to break the company’s stranglehold on the AI chip market, raising hundreds of millions of dollars and rolling out new products as they look to share the spoils of a boom in artificial intelligence technology.

Cerebras, d-Matrix and Groq are among a group of smaller companies aiming to take a slice of the multibillion-dollar AI chip market from Nvidia, which has so far dominated the first wave of investment with its graphics processing units, or GPUs.

They are riding a wave of expectation that demand for artificial intelligence “inference” — the compute power needed for models such as OpenAI’s ChatGPT and Google’s Gemini to generate responses to queries — will grow exponentially as chatbots and other generative AI applications become more popular.

Nvidia’s Hopper GPUs, which are well suited to the highly resource-intensive task of training top AI models, have become one of the world’s hottest commodities.

Cerebras, d-Matrix and Groq are focusing instead on cheaper, more specialised chips designed for running AI models.

On Tuesday Cerebras announced its new “Cerebras Inference” platform, based on its CS-3 chip, which is the size of a dinner plate. Cerebras claims its solution is 20 times faster than Nvidia’s current generation of Hopper chips at AI inference, at a fraction of the price. Cerebras cites tests run by benchmarking analysis provider Artificial Analysis.

“The way you beat the 800lb gorilla is by bringing a vastly better product to market,” Cerebras chief executive Andrew Feldman told the Financial Times. “In my experience, better products usually win, and we’ve taken meaningful customers from [Nvidia].”

The CS-3 chip shuns the use of a separate high-bandwidth memory chip, which is used by Nvidia. Instead it offers an alternative architecture with memory built directly into the chip wafer.

Limitations on memory bandwidth, Feldman said, are a fundamental constraint on the inference speed of an AI chip. The combination of logic and memory into a single large chip delivers results that are “orders of magnitude faster”, he said.

d-Matrix, founded by Sid Sheth in 2019, is also kicking off a new funding round less than a year after it raised $110mn in a series B funding round led by Singapore’s state-owned fund Temasek. The company is aiming to raise $200mn or more later this year or early next, according to Sheth. d-Matrix is early in the fundraising process and said the ultimate figure raised could change.

d-Matrix is planning a full-scale launch of its own chip platform, Corsair, at the end of this year. Sheth said the company was pairing its products with open software such as Triton, which competes with Nvidia’s Cuda, a widely used software platform that offers the tools for developers to build AI applications and optimises the performance of its chips.

Nvidia’s biggest customers are backing the use of open software such as Triton. “App developers don’t like to be held to one particular tool,” Sheth said, and “people are getting wise that Nvidia has a stranglehold with Cuda on the training side”.

Groq, another AI inference competitor led by a former founding member of Google’s tensor processing unit team, raised $640mn this month from investors led by BlackRock Private Equity Partners, at a valuation of $2.8bn.

One venture capitalist cautioned that despite the hype around the sector, semiconductor start-ups had had a challenging time breaking into the market.

Chipmaker Graphcore was bought by SoftBank last month for just above $600mn, less than the roughly $700mn that the company had raised in venture capital since it was founded in 2016, according to people familiar with the deal.

Groq and Cerebras were also founded in 2016. “There has been a near insatiable desire from public investors to find and back the next Nvidia,” said Peter Hébert, co-founder and managing partner at venture firm Lux Capital. “This isn’t just about chasing the latest trend. The momentum is also benefiting several VC-funded chip start-ups that have been toiling away for nearly a decade.”