Can Open Models Keep Up With the AI Frontier? | Nathan Lambert
Can Open Models Keep Up With the AI Frontier? | Nathan Lambert
Podcast59 min 54 sec
Listen to Episode
Note: AI-generated summary based on third-party content. Not financial advice. Read more.
Quick Insights
  • Consider Meta (META) for public-market AI exposure, but treat its positive trajectory as a thesis to verify through actual product launches and business results; a rumored new open-weight model is unconfirmed.
  • Monitor AI data, evaluation, and reinforcement-learning infrastructure as a potential growth theme, prioritizing providers that demonstrate measurable model improvements and strong quality controls.
  • Track AI coding and enterprise applications for evidence of cost-effective performance on specific tasks; customized or open models may appeal to businesses seeking privacy and control.
  • OpenAI, Anthropic, and several tooling startups are private, so they are not directly available as public-stock investments; the discussion offered no price targets or specific buy recommendations.
Detailed Analysis

OpenAI and Anthropic (Private)

  • The guest described OpenAI and Anthropic as the clearest bets for working at the AI frontier, citing their advantages in compute, data, talent, and speed.
  • OpenAI was credited with pushing the reasoning-model paradigm; Anthropic was described as having a long-standing focus on coding that helped position it for the surge in AI coding tools.
  • Frontier labs are spending heavily on training and inference. The discussion noted that AI coding has become a major source of revenue, but did not provide company-specific financial figures or forecasts.
  • The guest expects frontier labs to reach valuable markets early and serve models efficiently. However, the discussion also questioned how much of the broader knowledge-work market any one lab can capture.

Takeaways

  • These are private companies, so they are not directly accessible through public-market stock tickers.
  • For investors evaluating private AI exposure, the discussion points to compute access, model performance, inference economics, and revenue from real-world uses such as coding as important factors to monitor.
  • The transcript gives no price targets or specific buy recommendations.

Alphabet (GOOGL, GOOG)

  • Google was named among the major closed frontier labs with significant compute, data, and talent resources.
  • The guest noted that it can be difficult for outsiders to judge which large labs are making the strongest progress; Google was included among companies that could still be at the frontier.

Takeaways

  • Alphabet offers public-market exposure to a major AI developer, but the episode did not make a specific investment case for Alphabet or discuss its valuation.
  • Track whether Google can translate its AI resources into competitive products and efficient model deployment; the transcript does not establish how successful it will be.

Meta Platforms (META)

  • The guest said Meta had undergone a major reset and was again on a positive trajectory.
  • Meta was discussed as a company for which open-weight models may complement other revenue streams. The speakers also mentioned rumors that Meta might release another open-weight model, but did not confirm that it would.
  • The host cited Mark Zuckerberg’s earlier argument for open AI as a strong explanation of Meta’s strategy.

Takeaways

  • Meta’s AI strategy may benefit from open-weight distribution if it supports the company’s broader business, but the episode offered no forecast for financial returns.
  • Treat the potential model release as unconfirmed and assess the strategy through actual product launches and business results.

Chinese Open-Model Developers (Moonshot AI, Kimi, GLM and Others)

  • The guest described Chinese AI developers as strong in technical talent but more constrained by access to compute than leading U.S. labs.
  • Releasing open models was presented as a practical way for Chinese startups to gain visibility and reach customers, rather than necessarily an altruistic choice.
  • The guest said open models can trail closed frontier models by only a few months, while cautioning that the extent to which distillation explains that progress is uncertain.
  • Chinese developers were described as capable of strong pre-training and post-training work. Their slower access to compute remains a disadvantage when scaling large training or inference runs.

Takeaways

  • The open-model ecosystem may create opportunities for businesses that need control over model weights, private data, or customized fine-tuning.
  • Evaluate claims about model progress carefully: the guest said public research does not establish that distillation is the dominant explanation for Chinese models’ performance.
  • Compute limitations, uncertain economics, and reliance on rapidly changing model capabilities are relevant risks discussed in the episode.

AI Data and Reinforcement-Learning Environments

  • The guest argued that high-quality data and environments where models can act are important drivers of frontier progress, especially in fields where results are difficult to verify.
  • The discussion described labs purchasing data and environments, and suggested that demand could continue as companies seek better model behavior across areas such as biology, health care, legal work, and robotics.
  • The guest also warned that some data products have quality problems. Poorly designed environments can lead to reward hacking, where a model finds a way to score well without completing the intended task.
  • The data industry was described as having room to improve its quality controls and operational security.

Takeaways

  • Data, evaluation, and RL-environment providers are a potential AI infrastructure opportunity, but the episode does not identify a specific public company to invest in.
  • When assessing companies in this area, examine whether their data produces measurable improvements, how they test for flawed environments, and how they protect sensitive information.
  • Demand may be attractive, but uneven data quality and reward hacking are explicit risks raised in the discussion.

AI Coding and Enterprise Applications (Cursor, Decagon, Harvey)

  • Cursor was cited as an example where a customized model and planning setup could perform comparably to using two frontier models on a long-horizon coding task, at lower cost.
  • Decagon and Harvey were mentioned as startups working with model fine-tuning and specialized applications.
  • More broadly, the guest said businesses may adopt open models when they need to use private data or want to run models on their own hardware.

Takeaways

  • The discussion supports monitoring AI applications that can show cost-effective performance on specific business tasks, rather than relying only on general model benchmarks.
  • Open models could be useful where companies need customization or control, but adoption can take time as customers identify use cases, build evaluations, and establish a return on investment.
  • The transcript provides no company-specific investment recommendations or financial forecasts.

Open-Model Tooling and Research (Thinking Machines, Prime Intellect, Hugging Face, EleutherAI, Trillium)

  • The speakers discussed a growing ecosystem for post-training, model fine-tuning, and open research infrastructure.
  • Thinking Machines was described as having released a model that was not its strongest effort, but as having subsequently released a second model and built Tinker, which the guest said helped the company get going.
  • Prime Intellect, Hugging Face, and EleutherAI were mentioned in connection with model research and tooling.
  • Trillium is a newly launched nonprofit research lab. Its plans include documenting how AI models are used in AI research and releasing open environments for high-value tasks.

Takeaways

  • Open tooling and research infrastructure may support a broader ecosystem of model developers and startups, but several organizations discussed are private, nonprofit, or not identified as publicly traded.
  • For commercial opportunities, look for tools that help customers fine-tune, evaluate, or deploy models; for nonprofit projects such as Trillium, the transcript frames the value as shared research infrastructure rather than a commercial investment.
  • The guest emphasized that post-training is a fast-moving area where getting the right techniques and talent in place can matter substantially.

AI Compute and Inference Infrastructure

  • The guest identified compute, data, and talent as core inputs to model progress, and said compute is a major resource advantage for leading frontier labs.
  • Power and chip availability were discussed as constraints, while improved inference efficiency and lower model-serving costs were described as important opportunities.
  • The guest expects models to become more efficient over time, but said it remains uncertain whether lower costs will lead to higher margins for providers or expand usage enough to offset falling prices.

Takeaways

  • AI compute and inference efficiency are important investment themes, but the transcript does not name a specific publicly traded chip company or provide a security recommendation.
  • Consider both sides of the economics: limited compute can constrain model developers, while efficiency gains may reduce costs but also put pressure on pricing and margins.
  • The episode does not provide price targets or a timeline for returns.

Cybersecurity Preparedness

  • The guest said the pace of AI capability progress was less concerning to him than what he characterized as inadequate preparation in public cybersecurity and related areas.
  • He warned that AI-related risks are already spreading and argued that society needs more research and visibility into how models are used and where they fail.

Takeaways

  • Cybersecurity preparedness is a risk and policy theme highlighted by the discussion, but no cybersecurity company or specific investment was named.
  • Investors may monitor how AI adoption affects security needs, while recognizing that the transcript does not identify particular beneficiaries or quantify the opportunity.
Ask about this postAnswers are grounded in this post's content.
Episode Description
a16z Partner Lisha Li sits down with AI researcher and Trillium founder Nathan Lambert to ask how durable the gap really is between frontier AI labs and open models. Nathan argues that OpenAI and Anthropic have structural advantages in compute, capital, and the ability to pioneer new training paradigms, but open-model labs, particularly in China, have proven remarkably capable of following close behind. They unpack how much of that comes from distillation versus technical talent, why compute remains a major constraint, and how data, RL environments, and post-training are increasingly shaping model progress.  They also discuss Nathan’s new nonprofit research lab, Trillium, and its plans to open up parts of AI research that are increasingly difficult to study from the outside. That includes releasing RL environments and documenting how AI is actually being used to automate AI research, with the goal of demystifying recursive self-improvement and giving researchers and policymakers a clearer picture of what frontier systems can and cannot do. Resources: Learn more about Trillium: https://trilliumlabs.org/  Follow Nathan Lambert on X: https://x.com/natolambert  Nathan Lambert’s Newsletter: https://www.interconnects.ai/  Follow Lisha Li on X: https://x.com/lishali88   Stay Updated: Find a16z on YouTube: YouTube Find a16z on X Find a16z on LinkedIn Listen to the a16z Show on Spotify Listen to the a16z Show on Apple Podcasts Follow our host: https://twitter.com/eriktorenberg Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures. Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.
About The a16z Show
The a16z Show

The a16z Show

By Andreessen Horowitz

The a16z Podcast discusses tech and culture trends, news, and the future – especially as ‘software eats the world’. It features industry experts, business leaders, and other interesting thinkers and voices from around the world. This podcast is produced by Andreessen Horowitz (aka “a16z”), a Silicon Valley-based venture capital firm. Multiple episodes are released every week; visit a16z.com for more details and to sign up for our newsletters and other content as well!