AWS says Elastic Compute Cloud Trainium2 instances, UltraServers now available

Amazon Web Services said that the new Amazon Elastic Compute Cloud Trn2 instances and Trn2 UltraServers, the” most powerful” EC2 compute options for ML training and inference, are now available.. Powered by the second generation of AWS Trainium chips, the Trn2 instances are 4x faster, offer 4x more memory bandwidth, and 3x more memory capacity than the first-generation Trn1 instances. Trn2 instances offer 30-40% better price performance than the current generation of GPU-based EC2 P5e and P5en instances. In addition to the 16 Trainium2 chips, each Trn2 instance features 192 vCPUs, 2 TiB of memory, and 3.2 Tbps of Elastic Fabric Adapter v3 network bandwidth, which offers up to 50% lower latency than the previous generation. The Trn2 UltraServers, which are a completely new compute offering, feature 64x Trainium2 chips connected with a high-bandwidth, low-latency NeuronLink interconnect, for peak inference and training performance on frontier foundation models. For the largest models that require even more compute, Trn2 UltraServers allow customers to scale training beyond the limits of a single Trn2 instance, reducing training time, accelerating time to market, and enabling rapid iteration to improve model accuracy. Trn2 UltraServers are a completely new EC2 offering that use ultra-fast NeuronLink interconnect to connect four Trn2 servers together into one giant server. With new Trn2 UltraServers, customers can scale up their generative AI workloads across 64 Trainium2 chips. For inference workloads, customers can use Trn2 UltraServers to improve real-time inference performance for trillion-parameter models in production. Together with AI safety and research company Anthropic, AWS is building an EC2 UltraCluster of Trn2 UltraServers, named Project Rainier, which will scale out distributed model training across hundreds of thousands of Trainium2 chips interconnected with third-generation, low-latency petabit scale EFA networking-more than five times the number of exaflops that Anthropic used to train their current generation of leading AI models. When completed, it is expected to be the world’s largest AI compute cluster reported to date available for Anthropic to build and deploy its future models on.