The shift from AI training to inference (running models) is creating a massive opportunity for specialized hardware that outperforms general-purpose GPUs. Investors should look toward TSMC (TSM), which remains the essential manufacturing partner for next-generation AI chips like those from Etched and big-tech hyperscalers. While NVIDIA (NVDA) remains the industry benchmark, its high margins are under threat from application-specific chips (ASICs) designed to run "Transformer" architectures more efficiently. Consider exposure to the "Token Economy" by focusing on companies that maximize "tokens per watt," as energy availability is now the primary bottleneck for AI scaling. By 2027, the market is expected to pivot from "per-seat" software pricing to "per-inference" costs, benefiting high-growth AI agent platforms like Perplexity and Anthropic.

Etched (Private Company)

Etched is a hardware startup founded in 2023 by Gavin Uberti and Rob Locken. The company is building specialized chips and full-rack systems specifically designed for AI inference (running models) rather than training. They have raised over $800 million and report over $1 billion in customer demand.

• The "Sohu" Chip: Their first product is a specialized chip designed to run AI models faster and cheaper than general-purpose GPUs.
• Vertical Integration: Unlike many competitors, Etched is building the entire stack: the chip, the boards, power delivery, interconnects, and the full server rack.
• Technical Bets:
  • Low Voltage Inference: They run chips at under half the voltage of traditional GPUs (similar to Bitcoin miners), which prevents "thermal throttling" and allows for much higher performance density.
  • Cluster Scale Memory: They built a custom interconnect stack to allow multiple chips to communicate with 5x lower latency than current industry standards (like NVIDIA’s Blackwell).
  • Architecture Specificity: While GPUs are general-purpose, Etched chips are hard-coded for the "Transformer" architecture (the math behind ChatGPT), allowing for massive efficiency gains.

Takeaways

• Inference is the Future: The founders argue that while training was the first wave, inference will become the largest market in the world as billions of users begin using AI agents concurrently.
• Efficiency over Versatility: For investors, the lesson is that "general purpose" hardware (like NVIDIA GPUs) may eventually lose market share to "application-specific" hardware (ASICs) like Etched as AI workloads become standardized.
• Speed as a Moat: Etched emphasizes "wall clock time." If a complex AI task takes a month on current hardware, Etched aims to compress that into days, which could accelerate scientific and medical breakthroughs.

NVIDIA (NVDA)

The discussion frequently uses NVIDIA as the benchmark for current industry standards, while highlighting the limitations of general-purpose GPUs for specific AI tasks.

• The "NVIDIA Tax": Hyperscalers (Google, Meta, Microsoft) are building their own chips (TPUs, Maya, etc.) primarily to avoid the high costs of buying NVIDIA hardware.
• Hardware Retrofitting: The founders argue that current GPUs were designed before the ChatGPT era and are being "retrofitted" for modern AI, leaving room for new architectures to outperform them.
• Interconnect Latency: The transcript notes that NVIDIA's Blackwell chips have a point-to-point latency of about 4,000 nanoseconds, which Etched claims to significantly reduce.

Takeaways

• Dominance but Vulnerability: While NVIDIA remains the gold standard, its "general purpose" nature is its potential weakness. Specialized startups and internal big-tech chips are targeting NVIDIA’s high margins in the inference market.
• Talent Migration: Etched has successfully recruited "legends" from NVIDIA (including the creator of the HGX/DGX systems), suggesting a shift in where top-tier hardware talent is moving.

Investment Theme: The "Token" Economy

The podcast introduces a shift in how productivity and economic value should be measured in the AI era.

• Tokens as a Commodity: The founders view "tokens" (units of AI text/thought) as a commodity that will eventually have economies of scale similar to iPhones or cars.
• GDP per Megawatt: A future metric for national success may be "agents per megawatt"—how much AI work a country can generate from its energy grid.
• The $100 Billion Data Center: The founders predict the rise of "mega token factories"—single data centers costing $100 billion—designed to serve a handful of massive models to billions of people.

Takeaways

• Energy is the Bottleneck: Power availability is the primary constraint on AI growth. Companies that can provide more "tokens per watt" will have a significant competitive advantage.
• Agentic Workforce: By 2027, the founders predict there will be more AI agents in the workforce than humans, shifting the focus of software companies from "per seat" pricing to "per token" or "per inference" costs.

Mentioned Companies & Platforms

The following companies were mentioned as key players in the AI infrastructure and software ecosystem:

• TSMC (TSM): Highlighted as an essential partner with superior customer service and yield optimization. Etched uses TSMC's 4nm process.
• Hyperscaler Chips: Mentioned as competitors to NVIDIA/Etched, including Google’s TPU, Meta’s MTIA, Microsoft’s Maya, and OpenAI’s rumored Jalapeno project.
• AI Software/Agents: Cursor, Anysphere, Perplexity, and Anthropic were noted as high-growth teams driving the demand for cheaper and faster inference.
• Infrastructure Tools: WorkOS (enterprise readiness), Vanta (compliance), and Ridgeline (investment management software) were highlighted as essential tools for the modern AI/Finance stack.

Takeaways

• Supply Chain Diversification: Etched intentionally built its first-gen product on a different supply chain (4nm) than NVIDIA’s next-gen Rubens (3nm) to avoid zero-sum competition for manufacturing capacity.
• The "Existential" Advantage: The founders argue that specialized startups (like Etched) may build better chips than Big Tech (Google/Meta) because the chip is the startup's only product, whereas for Big Tech, it is just a cost-saving measure.