#496 – FFmpeg: The Incredible Technology Behind Video on the Internet
#496 – FFmpeg: The Incredible Technology Behind Video on the Internet
Podcast4 hr 23 min
Listen to Episode
Quick Insights

The most compelling growth opportunity lies in Kyber, a new ultra-low latency streaming protocol targeting a "glass-to-glass" delay of just 4 milliseconds. Investors should monitor this technology as it becomes the foundational infrastructure for high-stakes industries like remote surgery, cloud gaming, and humanoid robotics. For exposure to the broader shift in digital media, prioritize companies adopting the AV1 codec, such as Google (GOOGL), Amazon (AMZN), and Netflix (NFLX), to benefit from 30-50% higher efficiency and the elimination of expensive patent licensing fees. As hardware performance gains slow, high conviction should be placed on firms specializing in "extreme software optimization" and low-level engineering to extend the lifecycle of existing data centers. While FFmpeg and VLC are non-profit entities, they represent critical "invisible infrastructure" whose stability is essential for the valuation of every major tech platform globally.

Detailed Analysis

FFmpeg (Open Source Multimedia Framework)

FFmpeg is the "invisible backbone" of the internet, powering video processing for YouTube, Netflix, Chrome, Discord, and VLC. • It is estimated that 90% of all video processing workflows globally involve FFmpeg. • The project is built and maintained by a small core group of approximately 10–15 volunteer engineers and has seen contributions from over 3,000 people worldwide. • Technical Capabilities: * It can decode, encode, transcode, stream, and play almost any audio/video format ever created. * It operates at a low level, utilizing C and Handwritten Assembly to maximize hardware performance. * It includes a legendary command-line tool that acts as a "programming language" for video manipulation (cutting, fading, layering, etc.).

Takeaways

Investment Theme: FFmpeg represents the critical "open-source infrastructure" that trillion-dollar tech companies rely on. While not a tradable stock, its health is vital for the stability of the digital economy. • Efficiency Gains: The project prioritizes "extreme optimization." For example, the David (AV1) decoder uses 240,000 lines of assembly to allow high-def video to play on older hardware without needing expensive new chips. • Risk Factor: The project faces "maintainer burnout." Trillion-dollar corporations often expect free, urgent support from these volunteers without providing proportional financial backing.


VLC Media Player (VideoLAN)

• A legendary open-source media player with over 6 billion downloads. • It is famous for its ability to play "anything you throw at it," including broken files, obscure game codecs, and physical media like DVDs and VHS (via capture cards). • Business Ethics: Lead developer Jean-Baptiste Kempf famously turned down tens of millions of dollars in advertising and "spyware" deals to keep the software free, clean, and ad-free for users. • Architecture: VLC is built on a modular system, allowing it to support thousands of features (like the "puzzle filter" or ASCII art mode) without bloating the core engine.

Takeaways

Brand Power: The "Traffic Cone" logo is one of the most recognized icons in software, often searched for simply as "Cone Player." • Sustainability: VLC operates as a non-profit. Its success proves that high-quality, mission-critical software can thrive outside of traditional corporate structures. • Cross-Platform Utility: It supports everything from Windows XP to the latest iOS and even OS/2, extending the lifecycle of older hardware.


Kyber (Ultra-Low Latency Streaming)

• A new open-source startup founded by Jean-Baptiste Kempf focused on "Glass-to-Glass" latency. • The Goal: Achieving a 4-millisecond delay between a camera capturing an image and a screen displaying it over the internet. • Use Cases: * Robotics: Remote control (teleoperation) of humanoid robots, rovers, and drones. * Cloud Gaming: Reducing lag to make games feel local. * Remote Surgery: Enabling experts to operate machines from across the globe. • Business Model: Uses a Dual License (AGPL for open source/hobbyists and a commercial license for companies that want to keep their code private).

Takeaways

Investment Opportunity: This represents the next frontier of "Real-Time" internet. As the world moves toward autonomous vehicles and remote robotics, the underlying streaming protocol (Kyber) becomes a high-value asset. • Technical Edge: Kyber uses the QUIC protocol (based on UDP) to send video, audio, and control commands (mouse/keyboard) on a single "wire" while accounting for "clock drift" to keep everything synchronized.


Video Codecs & Standards (AV1, H.264, VVC)

H.264 (AVC): The current king of video, used in almost all MP4 files. • AV1: A newer, royalty-free codec backed by Google, Amazon, and Netflix. It is roughly 30-50% more efficient than H.264, meaning higher quality at lower data costs. • H.266 (VVC): The next generation of paid/licensed codecs, offering even higher compression. • X264: The gold-standard open-source encoder that made high-definition video accessible to the general public.

Takeaways

The "Patent Minefield": Multimedia is heavily patented. Companies like Apple and Google are moving toward AV1 to avoid paying hundreds of millions in licensing fees to patent pools. • Hardware vs. Software: While many believe new codecs require new chips (hardware), projects like David prove that elite software engineering can enable new standards on old devices. • Future Trend: "Volumetric Video" and "3D/Spatial Audio" are the next major integration points for these frameworks as VR/AR (Apple Vision Pro, Meta Quest) matures.


Investment Themes: The Value of Low-Level Engineering

The "Lost Art" of Assembly: The podcast highlights that handwritten Assembly language can be 10x to 60x faster than standard code (C/C++). • Hardware Limits: As "Moore’s Law" slows down (chips aren't getting faster as quickly), the only way to get more performance is through extreme software optimization. • Economic Impact: Every cycle saved in a video decoder like David translates to billions of dollars saved in electricity and battery life across the billions of devices using it daily.

Ask about this postAnswers are grounded in this post's content.
Episode Description
Jean-Baptiste Kempf is lead developer of VLC and president of VideoLAN. Kieran Kunhya is a longtime FFmpeg contributor, codec engineer, and the person behind the now-infamous FFmpeg account on X. https://lexfridman.com/sponsors/ep496-sc Transcript: https://lexfridman.com/ffmpeg-transcript CONTACT LEX: Feedback – give feedback to Lex: https://lexfridman.com/survey AMA – submit questions, videos or call-in: https://lexfridman.com/ama Hiring – join our team: https://lexfridman.com/hiring Other – other ways to get in touch: https://lexfridman.com/contact EPISODE LINKS: https://x.com/FFmpeg https://ffmpeg.org/ https://www.videolan.org/ https://x.com/videolan https://jbkempf.com/ https://www.linkedin.com/in/jbkempf/ https://github.com/jbkempf https://x.com/kierank_ https://bit.ly/3OORhmC https://github.com/kierank SPONSORS: Larridin: Measure AI adoption in your business. https://larridin.com Blitzy: AI agent for large enterprise codebases. https://blitzy.com/lex BetterHelp: Online therapy and counseling. https://betterhelp.com/lex Fin: AI agent for customer service. https://fin.ai/lex LMNT: Zero-sugar electrolyte drink mix. https://drinkLMNT.com/lex Perplexity: AI-powered answer engine. https://perplexity.ai/ OUTLINE: (00:00) – Introduction (03:00) – Sponsors, Comments, and Reflections (10:48) – Weirdest things VLC opens (15:12) – How video playback works (24:33) – Video codecs and containers (35:20) – FFmpeg explained (56:20) – Linus Torvalds (1:00:59) – Turning down millions to keep VLC ad-free (1:15:17) – FFmpeg & Google drama (1:34:31) – FFmpeg developers (1:41:08) – VLC and FFmpeg (1:45:42) – History of FFmpeg (1:48:59) – Reverse engineering codecs (2:02:14) – FFmpeg testing (2:06:21) – Assembly code (handwritten) (2:30:39) – Rust programming language (2:39:55) – FFmpeg and Libav fork (2:48:17) – Open source burnout (2:56:04) – x264 and internet video (3:09:20) – Video compression basics (3:16:17) – CIA and fake VLC (3:26:52) – Ultra low latency streaming (3:44:20) – AV2 codec and video patents (3:54:12) – VLC backdoors (4:04:27) – Video archiving (4:11:04) – Future of FFmpeg and VLC
About Lex Fridman Podcast
Lex Fridman Podcast

Lex Fridman Podcast

By Lex Fridman

Conversations about science, technology, history, philosophy and the nature of intelligence, consciousness, love, and power. Lex is an AI researcher at MIT and beyond.