Why smarter AI models could drive up compute prices 10x
Why smarter AI models could drive up compute prices 10x
Podcast11 min 18 sec
Listen to Episode
Note: AI-generated summary based on third-party content. Not financial advice. Read more.
Quick Insights

Investors should consider buying GOOGL as Alphabet aggressively secures scarce high-end hardware like GB200s and GB300s to maintain its dominant AI market position. Capitalize on semiconductor manufacturing bottlenecks by investing in TSMC and ASML, which control the constrained supply of leading-edge N3 nodes and EUV lithography machines. Compute spot prices are currently more than 40% higher than their February trough, creating strong pricing power for infrastructure suppliers. Avoid software companies that rely heavily on raw compute without proprietary model efficiency, as they risk being priced out by soaring compute costs.

Detailed Analysis

AI Compute Infrastructure and Semiconductor Sector

  • AI frontier labs (such as Anthropic, OpenAI, and Google) are experiencing massive revenue growth scaling at roughly 10x year-over-year, while available lab compute capacity is only scaling at approximately 3x year-over-year.
  • To bridge the gap between rapid revenue growth and constrained compute supply, compute spot prices are rising, lab margins are expanding, and a greater percentage of compute is being shifted toward inference rather than training.
  • Spot prices for compute are currently more than 40% higher than the February trough, and premier hardware bundles like GB200s and GB300s are commanding a 2x premium over standard spot prices when rented at scale with high security.
  • Hardware growth bottlenecks are severely limiting compute supply elasticity:
    • Moore's Law contributes about 1.4x to annual compute growth, which is difficult to accelerate.
    • New semiconductor fabrication plant construction is bottlenecked through 2030 and beyond by the availability of ASML EUV machines.
    • AI wafer allocation at TSMC on leading-edge N3 nodes is projected to max out as AI goes from 60% to 86% capacity utilization, exhausting the shift away from smartphones and PCs.
  • Companies that create the most efficient models will capture massive pricing power and higher profit margins because more efficient models economize on scarce and expensive compute tokens.

Takeaways

  • Companies supplying AI hardware, advanced GPUs, and semiconductor manufacturing equipment stand to benefit from inelastic demand and surging prices for compute resources.
  • Investors should monitor semiconductor manufacturing constraints, particularly the output of ASML EUV lithography machines and leading-edge node allocations at TSMC, as key limiters to compute scalability.
  • Software and AI application businesses that rely heavily on raw compute without proprietary model efficiency risk getting priced out as compute costs escalate.

Anthropic (Private)

  • Anthropic's revenue has grown by 10x year-over-year for three consecutive years and is projected to reach $100 billion to $150 billion by the end of the year, up from $9 billion at the end of last year.
  • The company's inference profit margins have expanded significantly, moving from 40% in the middle of last year to upwards of 80%.
  • Anthropic rents massive clusters of specialized hardware (such as GB200s and GB300s) at substantial premiums to maintain security and the scale needed to train next-generation frontier models.
  • The strong economies of scale in frontier model training allow a one-time training cost to be amortized across a massive user base, driving exponential revenue expansion relative to compute inputs.

Takeaways

  • Anthropic exhibits hyper-growth characteristics with rapidly expanding profit margins, though its operations are heavily dependent on securing scarce compute infrastructure.
  • The company's heavy reinvestment of compute into model training indicates a strategic focus on achieving artificial general intelligence (AGI) rather than pivoting purely to cloud-based inference services.

Alphabet Inc. / Google (GOOGL)

  • Google is actively securing massive compute allocations, evidenced by large-scale rentals including 110,000 GPUs (a blend of GB200s and GB300s) at a cost of $900 million a month.
  • The pricing structure for these enterprise-scale compute rentals is roughly 2x the standard spot price per hour, reflecting the high value Google places on securing reliable, highly secure compute capacity for AI development.

Takeaways

  • Big tech infrastructure leaders like Google are willing to pay significant premiums to lock down scarce, high-end AI hardware.
  • High capital expenditures on compute highlight the intense competitive pressure among major tech platforms to maintain leadership in AI capabilities.
Ask about this postAnswers are grounded in this post's content.
Episode Description
This is a video recording of a post I wrote last week. If you want to read the original you can check it out here. Thanks to Mercury for sponsoring this video. Mercury’s built-in AI, Command, helps me close my books and saves me a bunch of time. At the end of each month, Command categorizes my transactions and provides its rationale for every choice: I just review, fix anything that’s off, and approve... and then Mercury syncs everything to QuickBooks. Get started at mercury.com/command Get full access to Dwarkesh Podcast at www.dwarkesh.com/subscribe
About Dwarkesh Podcast
Dwarkesh Podcast

Dwarkesh Podcast

By Dwarkesh Patel

Deeply researched interviews <br/><br/><a href="https://www.dwarkesh.com?utm_medium=podcast">www.dwarkesh.com</a>