20VC: Who Wins the Inference Market | Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute | The Real Limits of Post-Training and RL with Erik Bernhardsson, Modal CEO
20VC: Who Wins the Inference Market | Has Open-Source LLM Inference Peaked? | What Happens to the Cost of Compute | The Real Limits of Post-Training and RL with Erik Bernhardsson, Modal CEO
Podcast1 hr 2 min
Listen to Episode
Note: AI-generated summary based on third-party content. Not financial advice. Read more.
Quick Insights
  • For public-market exposure, NVIDIA (NVDA) is the clearest high-conviction name discussed, supported by its durable software and developer ecosystem; no price target or entry price was provided.
  • Treat AI inference and compute as long-term growth themes, but favor providers that pair infrastructure with software and services rather than relying on GPU or token resale alone.
  • Approach capital-intensive AI infrastructure cautiously: prioritize companies with committed customers, strong capacity utilization, and a credible path to positive cash flow, especially given the noted risk of a 20%–30% market pullback.
  • Modal, OpenRouter, and TurboPuffer were discussed favorably but are private or lack public investment details, so they are not directly actionable public-stock trades.
Detailed Analysis

AI Inference and Compute Infrastructure

  • The CEO sees inference as a potentially enormous market, comparing it to the next generation of cloud computing. He expects demand to keep expanding as AI becomes cheaper and more widely used.
  • He expects the market to split into specialized services, including low-latency and batch inference, model training, post-training, sandboxes, and agent tools.
  • He argued that providers selling only compute or tokens risk becoming commodities, with little customer switching cost and pressure on prices. A broader product suite could help providers differentiate and earn better margins.
  • He said current inference-provider gross margins are often around 25%–40% and suggested a well-developed platform might reach 40%–60%, while arguing that traditional 90% software margins are unlikely across today’s AI infrastructure stack.
  • GPU availability and pricing remain important variables. Modal’s CEO said GPU lead times were around three to four months, that prices had risen, and that hardware prices should eventually fall—but the timing could be anywhere from one to five years.
  • The CEO expects both open-source and frontier-model inference markets to grow substantially. He pushed back on the view that open-source inference has peaked, estimating that both markets could grow 5x–20x over the next 12 months.

Takeaways

  • The discussion is bullish on long-term AI compute demand, but growth in the market does not guarantee strong returns for every provider. Look for businesses that combine compute with useful software and services rather than relying only on token or GPU resale.
  • Monitor GPU costs, capacity needs, customer demand, and margins together. Heavy investment in infrastructure can create cash-flow pressure even when a company reports positive operating results.

Modal (Private Company)

  • The episode’s host described Modal as having reached $1 billion in ARR, raised $750 million in a round led by Accel, and been valued at $15.75 billion. The CEO said the company had grown 15x over the prior 12 months and was growing 30%–40% month over month.
  • The CEO said Modal was profitable on a P&L basis but had negative cash flow because it was spending heavily to secure GPUs and expand capacity.
  • Modal’s strategy is to combine inference with products such as sandboxes, training, post-training, and other tools for building AI applications. Its sandbox product reportedly reached a $300 million run rate, about one-third of company revenue.
  • The CEO described its infrastructure—such as its scheduler, file system, and container runtime—as difficult to replicate. He also emphasized aggregating demand across customers to use GPU capacity more efficiently.
  • He acknowledged that Modal could have invested earlier in LLM inference, initially viewing it as a potential commodity market. The company later increased its team in the area.

Takeaways

  • The discussion presents Modal as a high-growth private AI infrastructure company, but it is not a publicly traded investment. Its stated valuation and growth figures were discussed on the show and should not be treated as independently verified.
  • The investment case depends on sustained demand, successful expansion into adjacent products, and managing GPU-related cash needs. The CEO’s comments also highlight execution risk: expanding product breadth requires strict prioritization.

NVIDIA (NVDA) and the AI Chip Ecosystem

  • The CEO expects NVIDIA to continue growing while specialized chip companies also find opportunities in particular workloads. He mentioned AMD, Cerebras, Etched, Olex, and Fractile as examples of other chip players.
  • He argued that NVIDIA’s advantage is broader than CUDA alone. In his view, its ecosystem—including software support, kernel optimization, and compatibility with popular AI tools—keeps NVIDIA first in many developers’ workflows.
  • He said non-NVIDIA accelerators have been expected to gain ground for years, but that NVIDIA has repeatedly maintained its position. He also noted that AMD, Google TPUs, and Amazon Trainium are among the alternatives he considers promising but still behind.
  • When asked about a reported 30% decline in Cerebras’ stock, the CEO said he did not follow the stock market and offered no explanation.

Takeaways

  • The discussion is broadly positive on NVIDIA’s ecosystem strength, while recognizing that specialized chips and alternative accelerators may also grow.
  • The transcript does not provide a valuation, price target, or company-specific analysis for NVIDIA or the other chipmakers. It also does not explain Cerebras’ reported share-price decline, so that figure alone does not establish an investment case.

Nebius (NBIS) and CoreWeave (CRWV)

  • The CEO described neocloud providers as both potential partners and competitors, noting that some are trying to move up from GPU infrastructure into inference services.
  • He expressed skepticism that infrastructure providers can easily reproduce Modal’s software stack, citing the difficulty of building systems that schedule large numbers of tasks and manage GPUs and sandboxes at scale.
  • He praised Nebius and CoreWeave for their execution, particularly their financial engineering: securing customer commitments, raising capital, buying chips, and delivering capacity.
  • He characterized the neocloud business as heavily dependent on financing and capital management.

Takeaways

  • The transcript recognizes strong execution at both companies but does not make a direct stock recommendation or compare their valuations.
  • For investors assessing neoclouds, the discussion points to financing, customer commitments, debt, and the ability to keep capacity utilized as key areas to examine.

Open-Source Models and Specialized AI Companies

  • The CEO expects open-source models to remain important, especially for companies that want to fine-tune models, control model behavior, or avoid changes to an external provider’s model.
  • He said frontier models may be cheap and capable, but argued that specialized models can serve large individual use cases as the overall market expands.
  • He cited Jev as an example of a company/model focused on tasks such as classification and regression, and said he saw potential for a large business built around specialized models. The transcript does not provide a ticker or additional financial details.
  • He also mentioned Poolside and Reflection in connection with NVIDIA supporting U.S.-built open-source models. He argued that the expense of training models can make NVIDIA’s support valuable to the broader ecosystem.
  • The CEO said Europe has substantial technical talent but may struggle to concentrate capital and talent in a few companies at the scale seen in the United States or China.

Takeaways

  • The discussion favors a diversified AI-model ecosystem rather than a single winner: frontier models, open-source models, and specialized models may all have room to grow.
  • For specialized-model businesses, assess whether the model’s performance and customer demand are durable, and whether the company can capture value rather than relying solely on a model that may be replicated.

OpenRouter and Model Routing

  • The CEO said model routing can be a successful standalone business today and noted that he had personally invested in OpenRouter.
  • He praised OpenRouter’s growth and brand and said he would have encouraged the company to continue building rather than accept an acquisition offer.
  • He also cautioned that routing may eventually work best as part of a broader toolkit that includes model evaluation, fine-tuning, and controls for latency and throughput.

Takeaways

  • The interview is positive on OpenRouter’s business momentum, but the longer-term question is whether routing can remain differentiated as a standalone product.
  • Investors considering this area can look for evidence that a routing provider is building durable customer relationships or expanding into adjacent services.

AI Coding Environments and Data Services

  • The CEO said he would consider starting a company that sells coding environments to frontier AI labs, describing strong current demand and potential for attractive cash flow.
  • He also warned that these businesses may be less durable over the long term and questioned whether they fit the expectations of venture-backed companies seeking very large, lasting outcomes.
  • He said providers can broaden their role by adding implementation, evaluation, monitoring, and enterprise services—not just selling datasets.
  • He mentioned investing in multiple companies in the area but did not name them in this portion of the discussion.

Takeaways

  • The transcript suggests a possible opportunity in infrastructure and data products that help train or evaluate coding models.
  • Near-term demand may be strong, but the CEO’s warning is important: assess whether a company can develop lasting customer value and repeatable revenue beyond a current wave of lab spending.

TurboPuffer

  • The CEO named TurboPuffer as the one company he would choose from his angel portfolio in the quick-fire question.
  • He described it as a vector-database business and said he saw potential to expand into adjacent areas such as databases, data warehouses, and other querying systems.
  • He did not give a valuation, financial figures, or price target.

Takeaways

  • The CEO’s view is that TurboPuffer could grow beyond a specialized database product if it successfully expands into adjacent data infrastructure.
  • This is a founder’s positive opinion, not a detailed investment analysis; the transcript does not address valuation, competition, or financial performance.

AI Software Development and Coding Agents

  • The CEO was strongly bullish on software engineering, arguing that tools that make developers more productive tend to increase demand for software rather than eliminate it.
  • He said Modal spent roughly $500,000 per month on coding agents and that its engineers use them extensively.
  • He expects coding agents to consume more compute as they take on more work, while acknowledging that the timeline and degree of adoption remain uncertain.
  • He argued that software engineering involves understanding requirements and human workflows, not simply translating English into code.

Takeaways

  • The discussion supports a long-term growth thesis for software development tools and the compute required to run them.
  • Adoption, spending efficiency, and whether agent usage expands into more workflows are key things to watch; the interview does not provide a specific public-company recommendation.

AI Drug Discovery

  • The CEO described AI-aided drug design as an exciting potential application and expressed interest in AI-generated medicines for chronic conditions.
  • He noted that progress could take time because of the FDA and other regulatory processes.

Takeaways

  • AI drug discovery is presented as a long-term opportunity, not an imminent revenue forecast.
  • Regulatory timelines are a specific risk raised in the discussion, so investors should distinguish technical progress from the time required to deliver approved products.

AI Venture Funding and Market Risk

  • The CEO said he expects a possible 20%–30% pullback in the stock market, while remaining bullish on the long-term AI opportunity.
  • He specifically warned against companies that depend on continual equity financing and could struggle if venture capital pulls back.
  • He said companies should raise each round as if it might be their last and maintain a credible path to commercialization and revenue.
  • He also described AI infrastructure as capital-intensive: GPU purchases and capacity expansion can produce negative cash flow even where the underlying business is profitable on a P&L basis.

Takeaways

  • The interview’s long-term optimism is paired with a clear near-term financing risk. Prefer businesses with meaningful customer revenue and a path to self-funding over those reliant on repeated fundraising.
  • A market pullback could affect private-company valuations and access to capital even if demand for AI continues to grow.
Ask about this postAnswers are grounded in this post's content.
Episode Description
Erik Bernhardsson is the Co-Founder and CEO of Modal, the AI infrastructure platform for training models, running inference and powering AI agents. Modal has raised over $1.2 billion, including a recent $750 million Series D led by Accel at a $15.75BN valuation. The company has surpassed $1BN in ARR, growing more than 15x year-on-year. Previously, Erik built Spotify's original music recommendation system. AGENDA:  05:30 - Why wait until your mid-30s to start a company? 06:40 - Is founding a startup really like "eating glass"? 08:00 - How to Raise $1BN with No Pitch Deck 08:45 - The danger of raising too much venture capital 09:40 - How much cash is Modal burning on GPUs? 10:30 - Do early-stage startups actually have moats? 13:30 - Who wins the inference market in the next 5 years? 15:30 - Are 90% software gross margins dead in AI? 17:45 - The "open-air drug market" for GPUs 21:30 - Will GPU hardware prices ever collapse? 22:15 - Are we about to overbuild data centers? 23:45 - Can custom silicon dethrone NVIDIA's monopoly? 24:30 - Is CUDA really the moat everyone claims it is? 25:20 - Are Neo Cloud providers partners or mortal enemies? 26:35 - Nebius vs. CoreWeave: who built the better business? 27:25 - Open-source vs. frontier models: which takes the lead? 28:20 - Can Europe ever build a competitive frontier model? 30:20 - The bull case for a $100B verticalized model company 37:15 - Modal's biggest resource allocation mistake 38:00 - Has open-source LLM inference already peaked? 41:30 - The real limits of post-training and RL 44:20 - How much does Modal spend every month on coding agents? 45:50 - The massive gold rush in selling data environments to labs 47:20 - Should enterprises own their own intelligence? 49:50 - Is model routing a defensible standalone business? 53:20 - $20M+ compensation packages for exceptional new grads 54:20 - The #1 portfolio company Erik would bet on 55:10 - What Erik changed his mind on most this year
About The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch
The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

The Twenty Minute VC (20VC): Venture Capital | Startup Funding | The Pitch

By Harry Stebbings

The Twenty Minute VC (20VC) interviews the world's greatest venture capitalists with prior guests including Sequoia's Doug Leone and Benchmark's Bill Gurley. Once per week, 20VC Host, Harry Stebbings is also joined by one of the great founders of our time with prior founder episodes from Spotify's Daniel Ek, Linkedin's Reid Hoffman, and Snowflake's Frank Slootman. If you would like to see more of The Twenty Minute VC (20VC), head to www.20vc.com for more information on the podcast, show notes, resources and more.