Jamie Spencer
Amazon is gearing up to unveil its latest artificial intelligence (AI) chips as it intensifies its push to reduce dependency on Nvidia and maximize returns from its substantial semiconductor investments. The Big Tech giant’s cloud computing arm, Amazon Web Services (AWS), has ramped up spending on custom chips to enhance data center efficiency and reduce costs for both itself and its customers.
This initiative is led by Annapurna Labs, a startup that Amazon acquired in 2015 for $350 million. The lab’s latest breakthrough, Trainium 2, is expected to be made widely available soon. Notably, Trainium 2 is already being tested by major players like Anthropic, backed by $4 billion from Amazon, along with Databricks, Deutsche Telekom, and Japanese companies Ricoh and Stockmark.
Targeting Nvidia’s AI Market Dominance
AWS aims to provide an alternative to Nvidia’s stronghold in the AI processor market. “We want to be absolutely the best place to run Nvidia,” said Dave Brown, AWS’s vice-president of compute and networking services. “But at the same time we think it’s healthy to have an alternative.” Amazon’s existing chip, Inferentia, is already positioned as a more cost-effective option, offering 40% lower operational costs for running AI models.
Brown highlighted the significant savings that come from reduced cloud computing costs, particularly for large-scale machine learning projects. “When you save 40 per cent on tens of millions of dollars, it does,” he said, emphasizing the scale of potential customer savings.
A Push for Efficiency and Cost Reduction
Amazon plans to allocate approximately $75 billion to capital spending in 2024, with most of it going toward technology infrastructure. This figure marks a substantial increase from 2023’s $48.4 billion. AWS’s ambitions align with broader trends among tech giants like Microsoft and Google, which are also investing heavily in custom chip development to support their AI growth strategies.
Rami Sinno, Annapurna’s director of engineering, highlighted Amazon’s comprehensive approach to AI infrastructure. “It’s not [just] about the chip, it’s about the full system,” Sinno said. Amazon’s commitment to building a vertically integrated system, from chip design to server implementation, underscores the complexity and scale of its operations. “It’s really hard to do what we do at scale. Not too many companies can,” Sinno added.
The Challenge of Competing with Nvidia
Despite Amazon’s efforts, Nvidia remains dominant in the AI chip market. In its second fiscal quarter of 2024, Nvidia generated $26.3 billion in revenue from AI data center chip sales — a figure matching Amazon’s entire AWS revenue for the same period, only a fraction of which comes from customers using Annapurna’s infrastructure.
Chip consultant Patrick Moorhead of Moor Insights & Strategy noted that while independent benchmarks offer initial insights, the true measure of performance comes from real-world deployments. Amazon’s claims of significant performance improvements between Trainium 1 and Trainium 2 have been met with cautious optimism. “People appreciate all of the innovation that Nvidia brought, but nobody is comfortable with Nvidia having 90 per cent market share,” Moorhead said. “This can’t last for long.”
Industry-Wide Trends and the Future of AI Chips
Amazon is not alone in its quest for AI chip autonomy. Other tech leaders, including Microsoft, Meta, and even OpenAI, are all developing custom chips as part of a broader trend towards vertical integration and cost control. “Everybody from OpenAI to Apple is looking to build their own chips,” said Daniel Newman of The Futurum Group, emphasizing the push for lower costs, higher margins, and greater control over supply chains.
G Dan Hutcheson, an analyst at TechInsights, pointed out that Amazon’s chips have the advantage of power efficiency and could drive down operational costs by making their data centers more energy-efficient. While Nvidia’s GPUs are likened to versatile station wagons, Amazon’s tailored chips act more like compact cars optimized for specific functions, providing flexibility and customization in their services.
Amazon’s launch of Trainium 2 signals its determination to carve out a larger share of the AI infrastructure market, challenging Nvidia’s dominance. Although AWS has yet to make a significant dent in Nvidia’s stronghold, its investments in custom chips reflect a long-term vision of diversification, efficiency, and cost savings. As the competition in AI chip technology intensifies, consumers stand to benefit from greater choice and innovation.
