Google Introduces Ironwood as its Most Powerful, Capable, and Energy Efficient TPU for the Age of Inference

At the Google Cloud Next 25 conference, Google has introduced its Ironwood AI accelerator chip. It is the brand’s 7th generation TPU (Tensor Processing Unit), and is revealed to be its most powerful and scalable custom AI accelerator chip so far. Notably, it is also said to be the first TPU that has been designed specifically for inference.

Read more about it below.

Google’s Ironwood AI Accelerator Chip Introduced

The Ironwood AI accelerator chip has been made to meet the complex communication and computational demands of ‘thinking’ models (LLMs, MoEs, advanced reasoning tasks) that require massive parallel processing as well as efficient memory access. Low-latency high-bandwidth inter-chip communication (ICI) network has been used in order to support coordinated and synchronous communication at full TPU pod scale. Also, two size variants are available for Ironwood – 256GB chip configuration and 9,216GB chip configuration.

Speaking more about its specs and features, when using the 9,216GB chip per pod, 42.5 Exaflops in total is offered. When compared with El Capitan – the world’s largest supercomputer that offers 1.7 Exaflops per pod, this is 24x more compute power. Each individual chip also has a peak compute of 4,614 TFLOPS. Additionally, it features an enhanced SparseCore too. Furthermore, ML runtime that has been developed by Google DeepMind ensures efficient distribution of computing power across multiple TPU chips.

Significant performance gain while focusing on power efficiency and which allows AI workloads to run cost effectively (2x performance per watt when compared to Trillium – 6th gen TPU), increased HBM capacity (offers 192GB per chip, 6x more than Trillium), improved HBM bandwidth (7.2 TBps per chip, 4.5x more than Trillium), and enhanced ICI bandwidth (1.2 Tbps bidirectional, 1.5x more than Trillium) are notably Ironwood’s key features.

 

 

To sum up, Ironwood with its increased computational power, ICI networking advancements and reliability, memory capacity, and improved power efficiency fulfills the AI demands of tomorrow, marking a unique breakthrough in this age of inference.