Intel Gaudi 3 AI Chip Unveiled; Airtel, Infosys, Krutrim Among Customers

Intel has taken the wraps off its Gaudi 3 AI accelerator and has outlined what it calls an Open Systems Strategy to promote its scalable and secure solutions across AI workloads. Gaudi 3 promises up to 50 percent better performance at AI inference, 50 percent faster training, and 40 percent better power efficiency than Nvidia’s wildly popular H100 processor, while also being competitive with the next-generation H200.  Intel hopes to score wins in the datacentre market by making Gaudi 3 more easily available than Nvidia’s offerings, which until recently have had a nearly year-long waiting list due to high demand and constrained supply.

Gaudi 3 will be available to OEMs in Q2 2024, with general availability to datacentre customers following later this year. Dell, HPE, Lenovo, and Supermicro are already confirmed as customers. Speaking at the company’s Vision 2024 AI conference, Intel CEO Pat Gelsinger laid out the company’s AI strategy, stating ““Innovation is advancing at an unprecedented pace, all enabled by silicon – and every company is quickly becoming an AI company”. 

Intel also announced that Bharti Airtel and Indian AI startup Krutrim, founded by Ola’s Bhavish Aggarwal, are amongst its customers who are already using currently available Gaudi offerings. Airtel is described as leveraging AI with its rich telecom data to “turbo charge” customers’ experiences and drive new revenue streams. Infosys will be using Gaudi accelerators as well as Intel’s Xeon and Core Ultra CPUs to “accelerate business value using generative AI technologies”.

Krutrim, which became India’s first AI unicorn and the fastest startup to achieve unicorn status this January, is already leveraging Gaudi 2 to pre-train its foundational AI models. The company aims to power genAI assistant functionality in more than 10 local languages, at an unprecedented price point. Other companies named as Intel’s customers include Bosch, IBM, NielsenIQ, and Roboflow.

The Gaudi 3 accelerator is aimed at enterprise-scale generative AI workflows. It is said to be capable of 4X the compute power of its predecessor, Gaudi 2, in certain tasks, as well as 1.5X better memory bandwidth and twice the networking bandwidth to allow for massively scaled-up compute clusters. According to Intel, there is high demand for cost-effective, scalable and easily available AI solutions, and its open industry standards approach offers businesses more flexibility and higher return on investment. 

Also read: Intel Showcases Make In India Products At Its Tech Summit

On the hardware level, each Gaudi 3 accelerator is built around a heterogeneous computing architecture with 64 programmable Tensor processor cores and eight matrix multiplication engines (MMEs) which can all work in parallel across multiple data types. Each chip can leverage 128GB of HBMe2 memory and 96MB of onboard static RAM, which Intel says is ideal for AI datasets such as large language models and multimodal models. Memory bandwidth is 3.7TBps. 

Every Gaudi 3 accelerator also has 24 200Gbps Ethernet ports allowing for massively scaled compute clusters comprising thousands of nodes, without the need for a proprietary interconnect fabric.  

Intel also teased its next-gen datacentre GPU, codenamed Falcon Shores, which will integrate Gaudi and Intel Xe GPU technology. The Xeon 6 CPU series, set to ship in Q2 2024, will integrate heterogenous cores in some segments, promising massive gains in power efficiency.

Enterprise AI will need to be easy to develop and scale, while ensuring the security of confidential data. According to the company, enterprises have reached an AI inflection point, with 80 percent expected to use generative AI by 2026, driving projected spends upwards of $150 billion this decade. Intel expects to ship 100 million AI accelerators by the end of 2025, including consumer PCs with Core Ultra CPUs.