Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology
Naveen Rao: 4D Computing, AI's Energy Wall & Beating Biology
11 hours agoAll-In Podcast@allin
YouTube22 min 35 sec
Watch on YouTube
Note: AI-generated summary based on third-party content. Not financial advice. Read more.
Quick Insights

Investors should allocate to power producers and grid infrastructure companies immediately, as electricity access is becoming the primary bottleneck for AI expansion over the next three years.

While NVIDIA (NVDA) retains short-term dominance through its software ecosystem, expect hyperscalers to diversify toward specialized low-power architectures over the medium term to cut massive data center energy expenses.

Alphabet Inc. (GOOGL) remains a high-volume leader in enterprise and consumer AI delivery, though its massive power requirements make rising electricity prices a crucial factor for operational margins.

Watch early-stage next-generation AI silicon and 4D computing developers such as Unconventional AI, which are targeting commercial, high-efficiency data center rollouts within two years.

In the private market, track Databricks as a premier pre-IPO asset, with enterprise AI model training and infrastructure now driving roughly 25% of its recurring revenue.

Detailed Analysis

AI Energy & Data Center Infrastructure

  • The artificial intelligence industry is rapidly heading toward a severe energy constraint, with energy availability replacing GPUs as the primary bottleneck for data center expansion.
    • Approximately 50% of the cost of serving an AI token (such as a query on ChatGPT) is directly tied to energy consumption, with the remainder going toward capital expenditures and facilities.
    • Global data center power capacity is currently under 100 gigawatts (GW), with the United States accounting for about 40 GW.
    • As AI models scale and usage grows, the industry risks hitting an "energy wall" within approximately three years.
    • AI is projected to become at least a $1 trillion market by 2030, but closing the gap between exponential AI compute demand and linear energy growth will require major hardware efficiency breakthroughs.

Takeaways

  • Investors should monitor power producers, energy contract providers, and grid infrastructure companies, as secure access to electricity is now the defining factor for AI data center development.
  • Hardware and software architectures that reduce power consumption per token will capture substantial economic value as traditional data center capacity hits electrical grid limits.

Next-Generation AI Silicon & 4D Computing

  • Traditional "von Neumann" computer architecture—which separates compute and memory—is fundamentally inefficient because most energy is spent moving data back and forth rather than processing it.
    • High-end computing systems like GPUs move nearly 30 trillion bits in and out of memory per second, whereas the human cortex operates on only 16 billion bits per second while running on roughly 20 watts.
    • Moore's Law has largely ended, meaning standard silicon shrinking no longer provides historical efficiency and performance gains.
    • Startup Unconventional AI has demonstrated early physical silicon for a "dynamical computer" (4D computing) that merges compute and memory into single units, running an image generation model at roughly 500 nanojoules per image compared to millijoules on standard GPUs.
    • The company is targeting a 1,000x improvement in power efficiency within three and a half years, with plans to deliver a commercial data center rack product within two years.
    • According to Jevons' Paradox, reducing the underlying cost and energy of compute by orders of magnitude is likely to exponentially increase overall demand, potentially enabling localized micro-data centers and edge robotics.

Takeaways

  • Long-term investors in AI hardware should watch for emerging non-von Neumann and analog/dynamical computing architectures that could eventually disrupt traditional GPU dominance in inference workloads.
  • As efficiency improves, compute infrastructure may shift away from centralized gigawatt facilities toward distributed, localized data centers and embedded robotics.

Alphabet Inc. (GOOGL)

  • Alphabet (Google) serves as a primary example of massive AI operational scale and its corresponding power footprint.
    • Google processes over 3.2 quadrillion AI tokens per month.
    • At an estimated conservative consumption rate of 10 joules per token, this scale alone requires approximately 12 gigawatts of dedicated power capacity.

Takeaways

  • Google's massive token volume demonstrates market leadership in consumer and enterprise AI delivery, but also highlights significant operational exposure to rising energy costs and power availability constraints.

NVIDIA Corporation (NVDA)

  • NVIDIA (referenced via CEO Jensen Huang and standard GPU architecture) represents the current market standard for AI acceleration.
    • Current GPU architectures rely on high data transfer rates between memory and compute, making power efficiency a growing hurdle at hyperscale.
    • Alternative architectures face significant switching barriers, requiring new software stacks (such as Python-based frameworks) to translate existing models without relying on traditional matrix multiplication (MatMul) or standard CUDA operations.

Takeaways

  • While NVIDIA retains a dominant moat due to existing software ecosystems, extreme power constraints at data centers will incentivize hyperscalers to explore specialized low-power architectures for AI inference over the medium to long term.

Databricks (Private)

  • Databricks expanded its footprint in enterprise AI infrastructure through its 2023 acquisition of Naveen Rao's previous startup, MosaicML.
    • The infrastructure built to train and scale large language models now represents roughly one-quarter (25%) of Databricks' total revenue.

Takeaways

  • Databricks continues to establish itself as a core player in enterprise AI model training and deployment, adding significant recurring revenue diversification ahead of any potential public offering.
Ask about this postAnswers are grounded in this post's content.
Video Description
(0:00) Welcome Naveen Rao (4:45) Is energy really the problem? The cost of a token, power contracts & the gap to close (10:24) Cutting out the middleman: abstractions, dynamical systems & a new kind of machine (19:40) Chamath joins: the path to product, porting existing models & building the team Thanks to our partners for making this possible! IREN is a vertically integrated AI Cloud platform, delivering data centers, compute and software for AI training and inference. https://iren.com/ Oracle connects the data, applications, and infrastructure that turn AI into business outcomes—with the flexibility, choice, and control to optimize as AI evolves. http://oracle.com/ai EY helps tech innovators scale from startup to exit to megacap. You build the future. We’ll handle the rest. http://www.ey.com Meta believes the future is for everyone. We're focused on giving every person the tools to reach their full potential and making sure the benefits of technology are distributed to all. http://www.meta.com Keel Infrastructure owns the power, land, and connectivity that HPC and AI run on - backed by secured energy assets and established grid interconnections across North America. https://keelinfra.com/ Airwallex - Agentic Global Business Accounts. Open local accounts in 70+ countries to accept payments, earn yield, pay globally, and manage spend. http://airwallex.com PayPal has been revolutionizing commerce globally for more than 25 years. Creating innovative experiences that make moving money, selling, and shopping simple, personalized, and secure, PayPal empowers consumers and businesses in approximately 200 markets to join and thrive in the global economy. For more information, visit https://www.paypal.com Google for Startups connects founders with the right people, products, and best practices to help startups build faster and go further. https://startup.google.com/ Explore ideas, industries, and technologies worth understanding with Chamath every week on Learn with Me: https://research.socialcapital.com/allin Follow the besties: https://x.com/chamath https://x.com/Jason https://x.com/DavidSacks https://x.com/friedberg Follow on X: https://x.com/theallinpod Follow on Instagram: https://www.instagram.com/theallinpod Follow on TikTok: https://www.tiktok.com/@allin Follow on LinkedIn: https://www.linkedin.com/company/allinpod Intro Music Credit: https://rb.gy/tppkzl https://x.com/yung_spielburg #allin #tech #news
About All-In Podcast
All-In Podcast

All-In Podcast

By @allin

Chamath Palihapitiya, Jason Calacanis, David Sacks & David Friedberg cover all things economic, tech, political, social & poker.