Why We Need New AI Benchmarks, Which Industries Survive AI, and Recursive Learning Timelines | #218
Why We Need New AI Benchmarks, Which Industries Survive AI, and Recursive Learning Timelines | #218
Podcast1 hr 21 min
Listen to Episode
Note: AI-generated summary based on third-party content. Not financial advice. Read more.
Quick Insights

Evaluate all potential investments on their AI strategy, as companies that fail to adapt by 2026 face significant disruption. Prioritize "picks and shovels" investments in companies that provide essential data infrastructure, cleaning, and security services for AI adoption. Consider a pairs trade by favoring innovative fintech companies over legacy banks that are slow to modernize their technology. Be cautious with long-term holdings in traditional legal, accounting, and BPO firms whose business models are threatened by automation. Finally, favor companies that use a hybrid "human-in-the-loop" AI model over those claiming to fully replace their human workforce.

Detailed Analysis

Investment Theme: AI Adoption & Disruption

  • The central theme of the podcast is that all companies must become AI companies by 2026 or face massive disruption and potentially become irrelevant. The term used was that companies who fail to adapt will be "cooked."
  • The disruption will not impact all industries equally. Sectors heavily reliant on knowledge work and documentation are most at risk and have the most to gain.
  • A major challenge for established companies is that their competition isn't just other large players, but new, agile AI-native startups that are built from the ground up with an AI-first approach.
  • The key tension in the market is whether startups can get distribution before large incumbent companies can build the technology.

Takeaways

  • Investors should evaluate companies based on their AI strategy and their ability to execute it. A company without a clear, actionable AI plan for the near future should be considered a higher-risk investment.
  • Look for "picks and shovels" plays in the AI space. These are companies that provide the essential tools and services (like data cleaning, AI implementation, and consulting) that all other businesses will need to adopt AI. The guest's company, Invisible Technologies, is an example of this type of business model.
  • Be wary of companies that simply announce they are "pivoting to AI" without concrete plans. The discussion emphasizes that successful implementation requires a focused approach, starting with 2-3 high-impact use cases and proving them out with pilot programs.

Sector: Legal, Accounting, and Business Process Outsourcing (BPO)

  • These sectors were explicitly identified as being highly susceptible to disruption by AI.
  • Much of the work in these fields, such as producing Non-Disclosure Agreements (NDAs) or standardized venture funding documents, is seen as repetitive "commodity information" that can be easily automated.
  • The podcast mentions a startup, Harvey, as an example of a company that could replace the work of traditional law firms.
  • While high-end, bespoke advisory work (e.g., complex M&A legal advice) is expected to persist, the lower and middle tiers of these professions are at significant risk.

Takeaways

  • Consider investing in AI-native startups that are disrupting the legal and accounting spaces. These companies have the potential for high growth as they automate commoditized tasks.
  • Be cautious about long-term investments in traditional BPO firms or law/accounting firms that do not have a clear and aggressive AI adoption strategy. Their business models are fundamentally threatened.

Sector: Banking & Fintech

  • The banking sector is described as a "really interesting one to look at."
  • Most large, established banks are running on very old technology, with application footprints often being north of 20 years old. This makes them vulnerable to disruption.
  • Newer, fast-moving fintech companies like Revolut are approaching the market with modern, AI-native technology stacks.
  • The core question for the industry is whether these emerging fintechs can capture market share (distribution) faster than the big banks can modernize their technology.

Takeaways

  • There may be a "pairs trade" opportunity here: going long on innovative fintech companies that are leveraging AI effectively, while being cautious or underweight on legacy banking institutions that are slow to adapt.
  • When evaluating investments in the financial sector, pay close attention to the age of a company's tech stack and the pace of its AI implementation.

Case Study: Klarna's Customer Service AI

  • Klarna is discussed as a cautionary tale. The company announced it was moving to a fully AI-powered, "agentic" contact center, claiming it did the work of 700 full-time agents and would save $40 million a year.
  • However, 8-12 months later, Klarna announced it was rolling back the initiative and returning to human agents.
  • The key insight is that a "100% AI" approach is often flawed. Complex issues and customers who simply want to speak to a person create problems. The ideal model is a hybrid "human-in-the-loop" system where AI handles simple tasks and escalates complex ones to humans.

Takeaways

  • Be skeptical of companies that announce a complete, immediate replacement of human roles with AI, especially in customer-facing positions. This often sounds more like a "PR exercise" than a sustainable strategy.
  • The more viable and successful model is a hybrid one. Look for companies that are intelligently integrating AI to assist and augment their human workforce, not replace it entirely. This "human-in-the-loop" approach is a key feature of successful AI deployment.

Investment Opportunity: AI Benchmarking

  • A significant opportunity was identified in the creation of new benchmarks to measure AI performance on specific, narrow tasks.
  • Current public benchmarks (like for coding) are too broad for most business applications. What's needed are "thousands of new narrow benchmarks" for every industry vertical and labor category (e.g., a specific benchmark for AI in title insurance).
  • The podcast suggests that an individual or company that creates and "owns" the definitive benchmark for a specific niche can become an "instant star" and a leader in that field.
  • This is because once a reliable benchmark exists, it becomes much easier to train and validate AI models for that specific task.

Takeaways

  • This is a ground-floor investment opportunity. Look for or consider creating startups focused on developing and owning hyper-specific evaluation standards ("evals") for AI in niche industries.
  • Companies that become the standard for benchmarking in a vertical will have a significant competitive advantage, as they will define what "good" looks like and attract others who want to train models against their standard.

Investment Opportunity: Data Infrastructure & Security

  • A recurring point is that the biggest obstacle to AI adoption is not the AI models themselves, but the data. Most companies do not have "clean data."
  • Before AI can be used effectively, data must be structured, cleaned, and made accessible. This is a major undertaking for most businesses.
  • A significant risk for companies is data privacy. When using third-party AI models (like from OpenAI), proprietary company data can be sent "straight into the cloud," which is a major concern for companies in sensitive industries like finance (Jane Street was the example) and healthcare.
  • There is a need for a "layer of protection" between a company's internal data and the large public AI models.

Takeaways

  • Invest in companies that specialize in data management for the AI era. This includes data cleaning, data structuring, and creating secure, "walled-off" environments for companies to use AI without exposing their proprietary data.
  • Companies that can solve the data privacy and security problem for enterprises will be in high demand. This could involve on-premise solutions or specialized, secure cloud instances.
Ask about this postAnswers are grounded in this post's content.
Episode Description
Get access to metatrends 10+ years before anyone else - https://qr.diamandis.com/metatrends   Matthew Fitzpatrick is the CEO at Invisible Technologies Learn about Invisible Salim Ismail is the founder of OpenExO Dave Blundin is the founder & GP of Link Ventures Dr. Alexander Wissner-Gross is a computer scientist and founder of Reified – My companies: Apply to Dave's and my new fund:https://qr.diamandis.com/linkventureslanding      Go to Blitzy to book a free demo and start building today: https://qr.diamandis.com/blitzy   Grab dinner with MOONSHOT listeners: https://moonshots.dnnr.io/ _ Connect with Peter: X Instagram Connect with Matthew Linkedin  Connect with Dave: X LinkedIn Connect with Salim: X Join Salim's Workshop to build your ExO  Connect with Alex Website LinkedIn X Email Listen to MOONSHOTS: Apple YouTube – *Recorded on December 16th, 2025 *The views expressed by me and all guests are personal opinions and do not constitute Financial, Medical, or Legal advice. Learn more about your ad choices. Visit megaphone.fm/adchoices
About Moonshots with Peter Diamandis
Moonshots with Peter Diamandis

Moonshots with Peter Diamandis

By PHD Ventures

Tracking the future of technology and how it impacts humanity. Named by Fortune as one of the “World’s 50 Greatest Leaders,” Peter H. Diamandis, MD, is a founder, investor, advisor, and best-selling author. Join Peter on his mission to uplift humanity through technology. Follow Peter on X - https://x.com/PeterDiamandis