Nvidia CEO Jensen Huang recently stated that the company's new line of chips is now being manufactured at full capacity. He claimed these processors provide five times the AI computing power of Nvidia's prior chips for running applications like chatbots.
In a speech at the Consumer Electronics Show in Las Vegas, the leader of the world's most valuable company revealed new details about its chips, which will arrive later this year and which Nvidia executives told Reuters are already in the company's labs being tested by AI firms, as Nvidia faces increasing competition from rivals as well as its own customers.
The Vera Rubin platform, made up of six separate Nvidia chips, is expected to debut later this year, with the flagship server containing 72 of the company's graphics units and 36 of its new central processors. Huang showed how they can be strung together into "pods" with more than 1,000 Rubin chips and said they could improve the efficiency of generating what are known as "tokens" - the fundamental unit of AI systems - by 10 times.
To get the new performance results, however, Huang said the Rubin chips use a proprietary kind of data that the company hopes the wider industry will adopt. "This is how we were able to deliver such a gigantic step up in performance, even though we only have 1.6 times the number of transistors," Huang said.
While Nvidia still dominates the market for training AI models, it faces far more competition - from traditional rivals such as Advanced Micro Devices as well as customers like Alphabet's Google - in delivering the fruits of those models to hundreds of millions of users of chatbots and other technologies.
Much of Huang's speech focused on how well the new chips would work for that task, including adding a new layer of storage technology called "context memory storage" aimed at helping chatbots provide snappier responses to long questions and conversations.
Nvidia also touted a new generation of networking switches with a new kind of connection called co-packaged optics. The technology, which is key to linking together thousands of machines into one, competes with offerings from Broadcom and Cisco Systems.
