Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]
Neil Movva - Making AI 10x Cheaper - [Invest Like the Best, EP.488]
Podcast1 hr 18 min
Listen to Episode
Note: AI-generated summary based on third-party content. Not financial advice. Read more.
Quick Insights

Investors should maintain core exposure to NVIDIA (NVDA) for premium real-time AI workloads and Taiwan Semiconductor Manufacturing Company (TSM) for indispensable advanced chip manufacturing leverage. Capitalize on critical hardware bottlenecks by investing in memory suppliers like Micron Technology (MU), SK Hynix, and Samsung, which stand to profit from persistent shortages in High Bandwidth Memory (HBM) and DRAM. Consider Advanced Micro Devices (AMD) as a compelling value play as major tech firms adopt cost-effective chip alternatives to diversify away from premium GPU pricing. Hold Intel Corporation (INTC) as a long-term strategic hedge against potential Asian semiconductor supply chain disruptions. Finally, expand into the emerging AI Inference Infrastructure and flexible power generation sectors as data center demand shifts toward low-cost, decentralized computing for background AI tasks.

Detailed Analysis

NVIDIA Corporation (NVDA)

  • NVIDIA holds market dominance in high-performance GPUs, driven by hardware innovations like the NVLink interconnect and whole-rack systems like the NVL72 (Grace Blackwell).
    • NVLink is considered critical for low-latency, real-time chatbot inference because it allows seamless communication across multiple chips.
    • The company maintains strong pricing power and carefully allocates chip supply across key customers and emerging cloud partners rather than selling compute directly to compete with them.
  • A contrarian perspective highlights that raw performance-per-watt gains for fundamental compute operations are leveling off across new chip architectures (from Hopper to Blackwell to Rubin).
    • High hardware costs may encourage developers to use alternative, cheaper silicon for background and batch-processing tasks where real-time speed is not critical.

Takeaways

  • Near-term demand and pricing power remain robust for real-time and frontier AI workloads.
  • Long-term margins could face pressure if the AI market shifts toward cost-effective, background inference workloads that do not require premium interconnect technologies.

Advanced Micro Devices, Inc. (AMD)

  • AMD chips present a strong value proposition in terms of raw compute power (FLOPS) per dollar compared to NVIDIA.
    • Major technology companies, including Meta and OpenAI, have increasingly purchased AMD hardware for their computing fleets.
    • The primary historical limitation has been software and kernel optimization rather than the underlying silicon hardware.

Takeaways

  • AMD is well-positioned as a secondary compute supplier as software layers mature and enterprise buyers look to diversify away from premium GPU pricing.

Memory Manufacturers: Micron Technology (MU), SK Hynix, Samsung

  • High Bandwidth Memory (HBM) and dynamic random-access memory (DRAM) are among the most severe supply chain bottlenecks for AI hardware.
    • Unlike on-chip memory (SRAM), DRAM offers significantly higher storage capacity required for long conversation histories and dynamic context (KV cache), but has lower data bandwidth.
    • Memory manufacturers remain cautious regarding massive capital expenditure expansions due to historical memory industry boom-and-bust cycles.

Takeaways

  • Memory suppliers benefit from tight supply dynamics and high structural demand as AI models demand larger context windows and higher memory capacity per accelerator.

Taiwan Semiconductor Manufacturing Company (TSM)

  • TSMC is the primary manufacturing chokepoint for leading-edge logic dies and advanced packaging across almost all major AI chip designers.
    • Foundries maintain strict manufacturing controls to minimize variation across chip quality, which adds significant time and cost.
    • A contrarian view suggests that geopolitical supply shock risks may be slightly overstated, as Western manufacturing alternatives trail TSMC's efficiency by roughly 2x at worst rather than insurmountable margins.

Takeaways

  • TSMC retains an essential role and pricing leverage across the global semiconductor ecosystem, though packaging capacity remains a primary constraint.

Intel Corporation (INTC)

  • Intel serves as the primary domestic foundry alternative in the West for advanced semiconductor manufacturing.
    • In a scenario where access to Asian fabrication capacity is disrupted, domestic processes from companies like Intel are estimated to be roughly 2x behind in performance-per-watt rather than entirely obsolete.

Takeaways

  • Intel holds long-term strategic optionality as a hedge against supply chain concentration and geopolitical disruption in overseas foundries.

AI Inference Infrastructure & Energy Sector (Investment Theme)

  • The AI market is transitioning from a speculative training phase to an inference phase, where compute spend is directly tied to ongoing product usage and immediate business utility.
  • Autonomous AI agents running long-duration tasks (such as deep research, automated coding, and continuous cybersecurity testing) prioritize low token cost over instant response latency.
  • This dynamic unlocks new infrastructure models:
    • Distributed Micro-Data Centers: Instead of massive 100+ megawatt sites, operators can utilize decentralized 1 MW sites that require less power grid infrastructure.
    • Flexible Power & Lower Uptime: Workloads that run in the background can tolerate lower uptime (80% to 95%), making them suitable for low-cost, intermittent renewable energy like solar and wind without expensive backup generators.

Takeaways

  • Capital expenditures in AI infrastructure are likely to broaden from centralized megawatt facilities into decentralized, lower-cost data centers and flexible power generation assets.
  • Companies focused on driving down the unit cost of AI inference through software and energy efficiency represent an emerging growth segment in enterprise technology.
Ask about this postAnswers are grounded in this post's content.
Episode Description
My guest today is Neil Movva, founder of Sail. Sail is building what Neil calls a token factory, an inference company designed for a specific kind of future, one where AI agents run in the background for hours or days at a time rather than answering a human in real time.  In that world, latency matters less and cost matters more, and Neil has built the whole company around driving the cost of a token as low as it can possibly go. What makes this conversation special is that it is one of the most detailed tours I have ever done through the full stack of intelligence, the software, the chips, and the power, and how all three connect.  Along the way we cover the trade-off between speed and cost that lives inside every GPU, his scavenger strategy for buying the chips and power nobody else wants, his contrarian view on Nvidia, and why the premium the frontier labs charge for being three to six months ahead may not last.  Please enjoy my conversation with Neil Movva. For the full show notes, transcript, and links to mentioned content, check out the episode page ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠here⁠⁠⁠⁠⁠.  ----- Become a Colossus member to get our quarterly print magazine and private audio experience, including exclusive profiles and early access to select episodes. Subscribe at ⁠colossus.com/subscribe⁠. ----- ⁠Ramp’s⁠ mission is to help companies manage their spend in a way that reduces expenses and frees up time for teams to work on more valuable projects. Go to⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ ⁠ramp.com/invest⁠⁠ to sign up for free and get a $250 welcome bonus. ----- Trusted by thousands of businesses, ⁠Vanta⁠ continuously monitors your security posture and streamlines audits so you can win enterprise deals and build customer trust without the traditional overhead. Invest Like the Best listeners get a special offer of $1,000 off Vanta when you go to ⁠vanta.com/invest⁠.  ----- WorkOS⁠ is the infrastructure B2B and AI-native companies use to sell to enterprise. It covers everything enterprise security requires: SSO, SCIM, RBAC, Audit Logs, AI governance, and more. Trusted by 2,000+ fast-growing companies, including OpenAI, Anthropic, Cursor, and Vercel. ----- Rogo is the AI platform for finance. They're building agents for Wall Street that are trained to understand how bankers and investors actually do work: from diligence and modeling, to turning analysis into deliverables. To learn more, visit rogo.ai/invest. ----- ⁠Ridgeline⁠ has built a complete, real-time, modern operating system for investment managers. It handles trading, portfolio management, compliance, customer reporting, and much more through an all-in-one real-time cloud platform. Visit⁠ ⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠ridgeline.ai⁠. ----- Editing and post-production work for this episode was provided by The Podcast Consultant. Timestamps: (00:00:00) Welcome to Invest Like The Best (00:02:20) Neil Movva (00:03:22) Building a Token Factory (00:05:32) The Rise of Long-Running Agents (00:08:47) Deep Research and Cybersecurity (00:15:12) The Full Stack of Intelligence (00:20:03) Throughput Versus Latency (00:24:58) The Future of AI Chips (00:33:19) Why Transformers Work (00:36:43) The Future of Data (00:44:05) The Market for AI Chips (00:47:56) Is the AI Boom Different? (00:51:08) Reinventing the Data Center (00:56:43) Scavenging Power (01:01:04) Where Compute Is Most Inefficient (01:07:02) Open Versus Closed Models (01:10:37) A Trillion Tokens a Day (01:12:42) The Contrarian Case on NVIDIA (01:14:38) Advice for AI Hardware Founders
About Invest Like the Best with Patrick O'Shaughnessy
Invest Like the Best with Patrick O'Shaughnessy

Invest Like the Best with Patrick O'Shaughnessy

By Colossus | Investing & Business Podcasts

Conversations with the best investors and business leaders in the world. We explore their ideas, methods, and stories to help you better invest your time and money. Hear stock market and boardroom insights you can't find anywhere else. If you're a professional investor, CEO, entrepreneur, or business strategist, this is for you. Explore all our episodes and learn more at https://www.joincolossus.com