Anjney Midha's Plan to Radically Lower the Price of Compute
Anjney Midha's Plan to Radically Lower the Price of Compute
45 days agoOdd LotsBloomberg
Podcast50 min 21 sec
Listen to Episode
Note: AI-generated summary based on third-party content. Not financial advice. Read more.
Quick Insights

Investors should maintain exposure to NVIDIA (NVDA) as it captures 80% of AI R&D spend, but should diversify into AMD and custom silicon providers as labs seek supply chain independence. To capitalize on the "compute bottleneck," look for companies like AMP PBC that provide software-defined translation layers between different chip types to improve hardware utilization. The most immediate disruption is occurring in Software Engineering and Material Science, making companies in these "verifiable" fields high-conviction targets for AI-driven productivity gains. Beyond software, shift focus toward the physical constraints of AI by investing in electrical grids, energy turbines, and data center real estate. Prioritize investments in "frontier" labs like Anthropic or Amazon (AMZN) that demonstrate high algorithmic efficiency and have secured massive infrastructure partnerships.

Detailed Analysis

Anthropic

  • Anthropic is identified as one of the three primary "frontier" AI labs alongside OpenAI and DeepMind.
  • Anjney Midha was an early angel investor in the company, participating in a $100 million initial round after 25 venture capital firms rejected the opportunity.
  • The company eventually secured a $4 billion compute and capital partnership with Amazon, highlighting that infrastructure is the primary requirement for modern AI labs.
  • Anthropic is noted for its efficiency, achieving state-of-the-art results with fewer than 5,000 employees compared to much larger teams at competitors like Google.

Takeaways

  • Infrastructure as a Moat: The success of Anthropic underscores that access to massive compute (GPUs) is currently the most significant barrier to entry for "frontier" AI development.
  • Efficiency Over Scale: Investors should look for labs that produce high-quality models with smaller teams, as this suggests superior algorithmic efficiency and better long-term margins.

AMP PBC

  • A Public Benefit Corporation founded by Midha to solve the "compute bottleneck."
  • The company is building a "Grid for Compute," acting as an Independent System Operator (ISO) similar to an electricity grid.
  • Software-Defined Compute: AMP uses software to standardize fragmented hardware (Nvidia, AMD, etc.) into a fungible resource.
  • Utilization Gains: While average data centers run at ~70% utilization, AMP aims for 95%+ by reallocating unused "pockets" of compute from long-term leases to other researchers.

Takeaways

  • Standardization Opportunity: As the AI market matures, companies that provide "translation layers" between different chip types (Nvidia vs. AMD) will be critical for reducing costs.
  • Economic Shift: The current model of "long-term leases" for GPUs is inefficient, with effective prices often 10x higher than marketed rates due to wastage. AMP represents a shift toward a "pay-for-what-you-use" utility model.

NVIDIA (NVDA) & AMD (AMD)

  • Nvidia currently captures approximately 80 cents of every dollar spent by AI labs on R&D.
  • There is a growing trend toward AMD chips and custom silicon (like Microsoft’s MAI and Maia chips) as companies seek "supply chain independence" and better margins.
  • Compute prices for long-term rentals have increased 2x between January and June 2024, indicating that demand is still "perpendicular" (extremely steep).

Takeaways

  • Margin Pressure: While Nvidia is the current leader, the high cost of their chips is forcing major tech companies to develop in-house silicon, which could pose a long-term threat to Nvidia's pricing power.
  • Secondary Market Value: The 2x markup in rental rates suggests a massive secondary market for compute capacity that is currently underserved.

AI Infrastructure & Energy

  • The discussion highlights a shift from digital platforms to physical constraints: power usage, transformers, and data center real estate.
  • Verifiable Feedback: AI progress is fastest in fields with "verifiable" loops, such as Software Engineering (unit tests) and Material Science (robotics/physics testing).
  • The "Jagged Frontier": AI is not a single race; there are multiple frontiers (video, coding, chat, science).

Takeaways

  • Investment Theme: Physical Constraints: Investors should look beyond software to the "physicality" of AI—companies involved in electrical grids, energy turbines, and specialized real estate for data centers.
  • Sector Focus: AI will likely disrupt Coding and Structured Knowledge Work (spreadsheets/accounting) much faster than creative or subjective fields because the outputs are easier for the AI to "verify" and learn from.
  • Corporate Strategy: There is a bifurcation between CEOs who use the tools (and understand their limitations) and those who outsource their understanding. Companies with "technically literate" leadership are likely to see better ROI on AI spend.
Ask about this postAnswers are grounded in this post's content.
Episode Description
Anjney Midha wrote the first check to Anthropic. He teaches a viral course at Stanford on how AI works. And he was, until recently, a partner at a16z. In other words, he is AI-industry royalty. Midha's new project is AMP PBC, a company that believes it can radically lower the price of compute. To accomplish that, he is working on building a compute grid that turns GPUs into a standardized utility. But right now, compute is too fragmented. It's too heterogeneous. And given the way contracts are structured, he says that labs are being forced to spend money on capacity that often goes unused. In other words, small labs are forced to pay up for big, long-term contracts, even though their own demand (particularly during model training) may be very spiky. On this episode, Midha explains how the market for compute currently works and why he believes there's a software solution that could significantly improve compute utilization. He also tells us why he does not anticipate one company will emerge as the dominate player and that instead we'll have a wide range of models, each optimally used in specific applications. Read more: Amazon Says Its Data Centers Use 2.5 Billion Gallons of Water Oracle Falls Most in Six Months on Mounting Data Center Costs Only http://Bloomberg.com subscribers can get the Odd Lots newsletter in their inbox each week, plus unlimited access to the site and app. Subscribe at  bloomberg.com/subscriptions/oddlots Subscribe to the Odd Lots Newsletter Join the conversation: discord.gg/oddlots See omnystudio.com/listener for privacy information.
About Odd Lots
Odd Lots

Odd Lots

By Bloomberg

<p>Bloomberg's Joe Weisenthal and Tracy Alloway explore the most interesting topics in finance, markets and economics. Join the conversation every Monday and Thursday.</p>