Fal Pivots to Camera Controls as AI Video Clears Real-Time Barrier
Jennifer Li, Gorkem Yurtseven, Batuhan Taskaya
With real-time inference eliminating speed and cost bottlenecks, commercial adoption now hinges on deterministic spatial and camera controls required by Hollywood studios rather than faster prompt generation.
AI-assisted summary of The Next Frontier of AI Video Is Control — original source. Check the original for full context.
Source published:
Key ideas
- fal released H3 Max, a post-trained version of MiniMax's Hailuo open-weight video model that pairs custom hardware kernels with reinforcement learning pipelines to achieve a 35x latency reduction over standard baseline endpoints without degrading quality. (0:00)
- The public variant H3 Max Turbo renders a 5-second video clip in roughly 1.5 seconds at half the cost of H3 Max, crossing the real-time threshold and shifting engineering priorities from raw speed to deterministic controllability. (10:00)
- H3 Max Director enables up to 60 minutes of continuous action-controlled video by combining a short-term memory buffer of compressed raw frames for the trailing two minutes with an evolving system prompt for long-term coherence. (20:00)
- Hollywood studios have become fal's fastest-growing segment, adopting modular point solutions—such as feeding low-resolution Blender 3D blockouts into H3 Max—to achieve near 100% controllability over camera coordinates, lighting, and lip synchronization. (28:15)
Detailed notes
## Background Published on September 17, 2026, by a16z with hosts and guests Jennifer Li, Gorkem Yurtseven, and Batuhan Taskaya, this episode explores what happens when generative video reaches real-time operational speeds. The discussion centers on fal's technical post-training and kernel co-design behind H3 Max, the emergence of continuous streaming video with temporal memory, and the industry-wide shift from chasing raw inference latency to achieving precise creative control.
## TL;DR The fal team explains how combining low-level kernel optimizations with post-training and reinforcement learning yielded H3 Max, a model that renders five seconds of video in roughly 1.5 seconds. Reaching this sub-real-time threshold unlocked continuous streaming experiences—such as fal Live and H3 Max Director—which maintain temporal scene memory across extended generations through compressed latent history and dynamic system prompts. With generation latency effectively solved, the core frontier has shifted toward deterministic controllability, allowing Hollywood visual effects artists and creators to direct scenes using structured camera coordinates, Blender 3D blockouts, motion retargeting, and voice inputs.
## Atmosphere & style The conversation carries the fast-paced, pragmatic energy of engineers fresh off a successful breakthrough, blending startup war stories with deep systems architecture. Jennifer Li guides the dialogue with crisp, industry-attuned prompts, while Gorkem and Batuhan speak with unvarnished technical candor about late-night 24-hour development sprints and the gap between research lab hype and studio production realities. Listening feels like sitting inside an intense post-launch debrief where cutting-edge compute limits, practical Hollywood workflows, and rapid infrastructure experimentation collide.
## Bulletin - `00:00` fal released H3 Max, a post-trained version of MiniMax's Hailuo open-weight video model that pairs custom hardware kernels with reinforcement learning pipelines to achieve a 35x latency reduction over standard baseline endpoints without degrading quality. - `10:00` The public variant H3 Max Turbo renders a 5-second video clip in roughly 1.5 seconds at half the cost of H3 Max, crossing the real-time threshold and shifting engineering priorities from raw speed to deterministic controllability. - `20:00` H3 Max Director enables up to 60 minutes of continuous action-controlled video by combining a short-term memory buffer of compressed raw frames for the trailing two minutes with an evolving system prompt for long-term coherence. - `28:15` Hollywood studios have become fal's fastest-growing segment, adopting modular point solutions—such as feeding low-resolution Blender 3D blockouts into H3 Max—to achieve near 100% controllability over camera coordinates, lighting, and lip synchronization.
## Key takeaways - `00:00` Generative media and coding agents have achieved 'token market fit', characterized by individual power users productively consuming roughly $10,000 worth of compute per month. - `05:00` Pure systems engineering reaches a ceiling of 2x to 3x speedups under hardware roofline constraints; exceeding this requires co-designing model weights through post-training and RL alongside custom low-level CUDA kernels. - `05:00` Video inference is a multi-stage pipeline—encompassing LLM prompt expansion, latent diffusion denoising, and VAE decoding—where custom kernels pushed diffusion Model Flops Utilization (MFU) from 30%–40% up to 70%–80%. - `05:00` Architectural silicon upgrades like NVIDIA Blackwell yield an isolated 2x to 3x wall-clock improvement, but software-model co-design is required alongside top-tier hardware to achieve sub-real-time streaming. - `10:00` Most frontier video models run strictly on single 8-GPU nodes to avoid heavy inter-node communication latency penalties. - `10:00` With generation times down to 1.5 seconds for a 5-second clip, speed is no longer the critical bottleneck; the engineering focus has pivoted entirely to controllability via LoRAs for lip sync, camera paths, and styles. - `15:20` Naive sequential streaming by feeding the final frame of clip N into clip N+1 suffers from visual amnesia and rapidly drifting scene continuity. - `17:45` Continuous video generation requires overlapping generation windows where the first segment directly attends to preceding latents, overcoming the exponential compute cost of long-context temporal self-attention via custom caching. - `25:12` H3 Max Director retains scene coherence across up to 60 minutes by using compressed raw video frames for the trailing two-minute window and an evolving system prompt for macro-level narrative state tracking. - `22:38` Creators are coupling real-time voice interfaces with continuous video engines to dynamically direct virtual scenes, actor actions, and camera movements on the fly. - `28:15` Professional VFX workflows achieve near-total visual control by blocking 3D scenes and camera trajectories in Blender—often scripted via LLMs—and passing low-resolution renders as references into H3 Max. - `30:00` Deterministic camera control is achieved by training models on structured JSON spatial coordinates over time rather than imprecise natural language prompts. - `33:48` Mainstream film studios require modular, surgical point solutions that integrate into existing editing pipelines—such as clip extensions, camera tweaks, and relighting—rather than end-to-end video generation from a text prompt. - `35:28` Enterprise adoption has accelerated by offering secure, isolated post-training environments for private IP alongside US-hosted instances of international open-weight models to satisfy data sovereignty regulations.
## Detailed walkthrough ### `00:00` Token Market Fit and the Breakthrough of H3 Max
The generative video landscape has hit an inflection point where inference speed fundamentally alters consumer and creative use cases. Opening the conversation, Jennifer Li and the fal team—co-founder Gorkem Yurtseven and Head of Engineering Batuhan Taskaya—highlight that generative media has achieved what they call "token market fit," sitting alongside coding agents as the only domains where single power users can productively consume thousands of dollars worth of compute every month. Gorkem explains:
> “Generative media is, I would say, along with the coding agent market, what we call is token market fit.”
Discussing fal’s release of H3 Max (their post-trained version of MiniMax's Hailuo open-weight video model), Jennifer recalls that the initial benchmark charts were so skewed that H3 Max sat completely isolated on the far side of the graph. Gorkem reveals that the charts were drawn on a logarithmic scale simply to fit H3 Max alongside competing models; in linear time, the gap was even more dramatic. Public variants like H3 Max Turbo can render a 5-second video clip in roughly 1.5 seconds, transforming generative video from an offline rendering step into an interactive, real-time medium.
The decision to post-train MiniMax's open-weight model represented a natural pivot for fal. Although fal has historically operated as an inference-serving platform for frontier AI labs, proprietary API agreements never allowed them to build optimizations directly into the underlying model weights. When MiniMax released Hailuo (H3)—the first genuinely capable, next-generation open-source foundation video model with support for reference inputs and familiar diffusion architectures—the team recognized a unique opportunity to co-design the model weights and runtime systems together.
- Token market fit is defined as an application where an individual user productively consumes roughly $10,000 worth of tokens a month doing daily work. - The generative video industry has faced severe compute bottlenecks since around April, with platforms growing strictly as fast as they can provision hardware. - H3 Max reduces generation costs and hardware bottlenecks by roughly an order of magnitude, liberating GPU capacity across the entire ecosystem.
---
`05:00` Co-Designing Kernels, RL Pipelines, and Compound Optimizations
Batuhan Taskaya explains why traditional inference optimization reaches a fundamental ceiling when restricted to hardware alone. Pure systems-level engineering—such as custom CUDA kernels, memory layout tuning, and operator fusion—typically yields a 2x to 3x speedup on fixed weights before hitting hardware roofline constraints. To breach that threshold and achieve a 35x latency reduction compared to standard baseline endpoints without degrading quality, fal paired low-level engineering with model post-training and reinforcement learning (RL) pipelines.
> “And this new set of post-training-related optimizations with system slash model co-design enables us to go beyond that roofline by an order of magnitude.”
Reducing inference steps in diffusion architectures (for example, stepping down from 50 sampling steps to 20 steps) normally causes severe visual artifacts and perceptual degradation. To circumvent this trade-off, fal ran RL and post-training runs specifically to boost the intrinsic visual quality of intermediate checkpoints. When the team subsequently applied step-reduction techniques, the resulting model matched or exceeded the baseline model's original ELO quality score while running over ten times faster.
```mermaid flowchart TD accTitle: End-to-End Generative Video Inference Pipeline accDescr: Pipeline showing Prompt Expansion, Diffusion Latent Generation, and VAE Latent-to-Pixel Decoding stages.
subgraph LLM_Stage["Prompt Expansion"] P["User Prompt"] --> EXP["Frontier LLM Expansion<br/>(Low batch size, un-cached)"] end
subgraph Diff_Stage["Diffusion Core (H3 Max)"] EXP --> LAT["Diffusion Denoiser<br/>(20 steps post-trained via RL)"] OPT["Custom Hardware Kernels<br/>(70-80% Theoretical MFU)"] -.-> LAT end
subgraph Decode_Stage["Pixel Reconstruction"] LAT --> VAE["VAE Latent Decoding<br/>(+ Optional Upscaler)"] VAE --> OUT["Final Output Video<br/>(5s video in ~1.5s)"] end ```
Batuhan points out that video inference is not a monolithic forward pass, but a multi-stage pipeline where every stage must be individually optimized: 1. **Prompt Expansion via LLMs:** A large language model reformats short user inputs into the dense descriptive prompts expected by diffusion backbones. Unlike conversational agents that rely on high KV-cache hit rates, this is single-shot processing operating at tiny batch sizes without caching. 2. **Latent Video Diffusion:** The primary denoising process where fal replaced standard runtimes with custom kernels, pushing Model Flops Utilization (MFU) from typical industry levels of 30%–40% up to 70%–80% of theoretical hardware limits. 3. **VAE Decoding & Upscaling:** Decoding the compact latent representations back into raw pixel grids, followed by optional high-resolution spatial upscalers.
Addressing next-generation hardware like NVIDIA's Blackwell (GB200) versus Hopper (H100), Batuhan notes that architectural upgrades provide an isolated 2x to 3x wall-clock improvement. Because the cost of top-tier silicon scales in tandem with its speed, architectural hardware upgrades alone do not change overall cost efficiency. However, deploying co-designed models on Blackwell becomes essential for sub-real-time streaming: rendering five seconds of coherent video in under two seconds requires pairing extreme software optimizations directly with next-generation chips.
`10:00` Single-Node Hardware Limits, H3 Max Turbo, and the Shift to Controllability
The conversation turns to the physical infrastructure constraints governing current generative video. Today, the vast majority of video models operate strictly within a single-node setup of eight GPUs. Pushing beyond eight GPUs introduces heavy inter-node communication overhead that rapidly diminishes efficiency. MiniMax's H3 endpoints and similar frontier models are served by distributing workloads in parallel across those eight local GPUs. When Jennifer Li asks whether more performance can be extracted by sacrificing narrow edge cases or restructuring parallelism, the team explains that they originally planned to brand their optimized release as "H3 Turbo." However, because its generation quality noticeably exceeded the baseline model, they opted for the name H3 Max to signal an uncompromised breakthrough.
A mere week after that launch, the fal engineering team managed to double throughput again while preserving roughly the 97th percentile of quality. This led to the public rollout of H3 Max Turbo, an endpoint capable of rendering a five-second video in just 1.5 seconds at half the operational cost. While there is a slight, discernible dip in visual fidelity compared to standard H3 Max, the team argues that generative video has crossed an inflection point where raw inference speed is no longer the critical bottleneck. In the team's view, cutting latency from 1.5 seconds down to sub-second speeds yields diminishing returns compared to solving the model's steerability.
Instead of shaving off additional milliseconds, fal’s focus has shifted entirely to controllability. They started with basic text-to-video, expanded into image-to-video, and then introduced reference conditioning, which quickly became the standard operational mode for users. Open-sourcing model weights enables a broader ecosystem to flourish, allowing the team to train targeted LoRA adapters for specialized capabilities such as synchronized lip sync, dynamic camera movements, and distinct visual styles.
As Gorkem Yurtseven summarizes the shift in priority: > “Today, these models are so cheap and so fast that I don't think people need any faster or any cheaper.”
He further argues that the engineering mandate must now center on creative predictability: > “Like let's fix the speed and let's try to push for quality and controllability of these models, which is like, you know, what we have been pushing in the past two or three weeks.”
- Majority of frontier video architectures run exclusively on single 8-GPU nodes to avoid inter-node interconnect latency penalties. - H3 Max Turbo generates a 5-second video in approximately 1.5 seconds, running 2x faster and at half the cost of H3 Max while maintaining 97th-percentile quality. - Ongoing post-training and adaptation work includes reference conditioning, lip-sync modules, camera-angle LoRAs, and artistic style LoRAs.
`12:35` Evals Too Good to Be True, Slack Records, and the 24-Hour Development Loop
When fal first completed the post-training and distillation pipeline for H3 Max, the initial benchmark numbers were startling. Having spent tens of thousands of dollars on internal evaluations, the performance jumps were so dramatic that leadership hesitated to announce them publicly without external verification. The team feared the metrics looked "too good to be true" and held back the launch until third-party validations corroborated that the speed-ups did not compromise foundational visual coherence.
The moment the model was deployed internally, it triggered a company-wide creative frenzy. A newly formed world model accelerator team immediately jumped on building interactive WebRTC live video pipelines, while frontend engineers began rapidly hacking together live consumer demos. fal operates as a globally distributed organization, primarily anchored in San Francisco but spanning multiple international time zones. The launch initiated an uninterrupted 24-hour development cycle: engineers would grind for 16 to 17 hours, hand off their progress to teammates waking up on the other side of the planet, and sustain that momentum across three to four continuous days, setting internal company records for Slack message volume.
Reflecting on how quickly the team self-organizes around transformative performance jumps, Gorkem Yurtseven observes: > “This happens at fal once in every couple of months where, like, the whole company gets hold of something and the creativity just explodes and or a different optimization, LoRA, whatever it might.”
```mermaid flowchart TD accTitle: Distributed 24-Hour Development Loop accDescr: Diagram showing the continuous development handoff between global time zones and concurrent product workstreams at fal.
subgraph Cycle ["24-Hour Continuous Development"] SF["San Francisco Engineers<br/>(16–17 Hour Shifts)"] Handoff{"Global Handoff"} Global["International Team<br/>(Picks Up Live Branches)"] SF -->|"End of Day Push"| Handoff Handoff -->|"Wake Up & Deploy"| Global Global -->|"Morning Handoff"| SF end
subgraph Streams ["Parallel Product Hackathons"] WMA["World Model Accelerator Team<br/>(WebRTC Live Video Engine)"] FE["Frontend Team<br/>(fal Live Interactive Interfaces)"] ML["Core ML Team<br/>(Continuous Memory Conditioning)"] end
Cycle --> Streams ```
`15:20` The Leap to Infinite Streams: Rehan, levelsio, and Naive Frame Chaining
The immediate public unlock of real-time speed showed up across social media through spontaneous community and internal hacks. Over the weekend of the model release, fal engineer Rehan set up an automated script directly on his laptop that fed the last frame of an H3 Max clip back into the next prompt, broadcasting an ongoing narrative stream straight to Twitch. Almost simultaneously, indie developer Pieter Levels (known on Twitter as levelsio) reached out with a dedicated web interface built to ingest fal's streaming endpoints to generate an endless, self-directed video broadcast.
Both Rehan’s stream and levelsio’s setup relied on sequential frame-to-video stitching: the model generated a standalone clip, extracted its final frame, and used that single static frame as the visual prompt for the subsequent segment. While impressive, this approach suffered from complete amnesia. The model had zero awareness of context beyond the single boundary frame, meaning continuity dissolved quickly if characters turned away or lighting shifted.
In parallel with these external experiments, fal's core ML team was developing a native continuous video engine with deep temporal memory. Instead of discarding context at every clip boundary, this internal system retained up to two minutes of historical scene state, enabling coherent causal interactions—such as subjects turning to look at an actor entering a room. At the same time, another internal crew was building "fal Live" to expose continuous low-latency generation over WebRTC. Three independent, viral streams emerged simultaneously from the same underlying speed milestone.
Describing the difference between simple frame daisy-chaining and true continuous temporal coherence, Gorkem Yurtseven explains: > “So like you're in a scene and when you direct the model or someone else enters the room, it actually like everyone looks at that person entering and the scene is continuous.”
- Rehan's Twitch demo and Pieter Levels' (levelsio) web stream demonstrated naive sequential streaming by taking the final frame of clip $N$ as the starting prompt for clip $N+1$. - Sequential frame chaining lacks scene memory; changes across generations accumulate drift and narrative incoherence. - fal simultaneously had three independent efforts go viral on Twitter: Rehan's stream, levelsio's infinite generator, and fal's internal native WebRTC stream (fal Live).
`17:45` Overcoming the Real-Time Barrier and Scaling Temporal Attention
For months, the fal research team had been oriented toward world models—specifically, action-driven, real-time continuous video streams where user inputs immediately steer ongoing visual generation. Historically, every open-weight and proprietary video model was bottlenecked behind the real-time threshold: generating a five-second video chunk reliably took significantly longer than five seconds of wall-clock compute time. Even if low-latency streaming infrastructure was in place to multiplex outputs to concurrent users, the underlying inference models could not keep pace with real-world playback.
Once H3 Max crossed that barrier by rendering five seconds of high-fidelity video in under two seconds, the machine learning and inference optimization teams were able to repurpose architectures they had previously benchmarked unsuccessfully. To achieve continuity without hard clip boundaries, the architecture was modified to generate overlapping windows: generating a 10-second segment where the first 5 seconds directly attend to and anchor on the preceding segment's latent frames.
The principal technical bottleneck in extending this window is that full self-attention across long sequences of spatio-temporal video tokens scales exponentially in compute. Retaining an active two-minute memory window required aggressive attention-masking and memory-caching optimizations to keep latency within the real-time budget. By resolving these compute limits, the engine preserves spatial context, character identities, and environmental details across hundreds of sequentially generated frames without degrading inference throughput.
As Batuhan Taskaya emphasizes regarding the computational toll of long-context video conditioning: > “And you need to do extremely clever optimizations because attending to a two minute video is just extremely, extremely compute intensive.”
```mermaid flowchart LR accTitle: Sliding Window Video Memory Architecture accDescr: Diagram illustrating how continuous video generation attends to historical latent frames over a sliding 2-minute memory window.
subgraph PriorContext ["Temporal History (Up to 2 Minutes)"] H1["Latent Window T-2"] H2["Latent Window T-1"] H1 --> H2 end
subgraph ActiveChunk ["Overlapping Generation (10s Chunk)"] Attn["Temporal Self-Attention Engine<br/>(Optimized Sparse / KV-Cached)"] Overlap["Attended Prior Frames<br/>(First 5s Anchor)"] NewGen["Newly Synthesized Frames<br/>(Next 5s Generation)"] Overlap --> NewGen end
H2 -->|"Context Conditioning"| Attn Attn --> Overlap NewGen -->|"Appended to Latent Buffer"| Next["Continuous WebRTC Stream"] ```
- Video models previously failed continuous streaming because their real-time factor was strictly below 1.0 (generating $X$ seconds took $>X$ seconds). - H3 Max inverted this constraint, creating excess compute headroom to run overlapping sliding-window generations. - Instead of naive 5-second isolated chunks, the model generates 10-second windows where the initial 5 seconds attend directly to preceding latents. - Full video self-attention across two minutes of dense video representations incurs exponential compute costs, requiring custom KV-caching and memory optimizations to maintain sub-second latency.
`20:00` H3 Max Director and fal Live
The team rolled out H3 Max Director, making it publicly accessible as what they describe as the only model capable of generating up to 60 minutes of continuous, action-controlled video. The architecture retains fine-grained details across scene transitions: a user can prompt an office setting with someone working, let the model imagine freely for 30 seconds, and then instruct a woman to walk through the door. The model updates the scene immediately while keeping the environment coherent—the camera can pan back, and the original person remains seated in the exact same state.
To demonstrate this capability publicly, fal launched a live interactive website (fal.live). The setup functioned like an interactive, crowdsourced stream where viewers could vote in chat on what would happen next in the scene. The team introduced themed channels—ranging from complete chaos to distinct cartoon aesthetics—leveraging the model’s large memory buffer for styles and concepts. As Gorkem Yurtseven explains, “And I think it's the only model that can generate up to 60 minutes, continuous videos that is action control.”
```mermaid flowchart TD accTitle: H3 Max Director Memory Architecture accDescr: Flow diagram showing how short-term compressed video tokens and a long-term evolving system prompt maintain coherence.
subgraph Input ["Real-Time User Direction"] A["Director Prompt / Voice Input"] end
subgraph Memory ["Dual-Layer Memory Buffer"] B["Short-Term Window (< 2 mins)<br/>Compressed Raw Video Frames<br/>Last 4–8 Scenes (15–30s each)"] C["Long-Term Window (Up to 60 mins)<br/>Continuously Evolving System Prompt<br/>World Coherence & Entity Tracking"] end
subgraph Output ["Continuous Stream Generation"] D["H3 Max Video Inference Engine"] E["Real-Time Continuous Video Stream"] end
A --> D D --> B B --> C C --> D D --> E ```
`22:38` Rapid Market Adoption and Real-Time Voice Directing
Jennifer Li observes that unlike large language models, which tend to show a linear compounding progression of intelligence, video models feel like they go through months of quiet bubbling before an abrupt burst where base models, latency optimizations, and controllability converge. Gorkem notes that within three weeks of its debut, H3 Max surged to become the most popular video model on the fal platform, outpacing the runner-up by more than double in sheer usage volume. Its low cost and rapid generation speed have driven third-party platforms to adopt it as their default interface.
Creative technologists and digital IP creators are leveraging these capabilities in unexpected ways. fal is currently collaborating with creators who produce AI-generated episodic shows on TikTok and Instagram, training custom LoRAs on their specific character and aesthetic styles to broadcast live versions of their series. Furthermore, creators are combining real-time speech interfaces—such as ChatGPT voice mode and Whisper—with the continuous streaming model. Instead of typing static prompts, users can speak out loud to the video while it is actively playing, directing camera angles and character movements on the fly like a director on a physical movie set. Gorkem highlights this shift in market traction: “Yeah, it's been like almost three weeks since we released H3 Max and already it is the most popular video model on the platform, on the fal platform by like double almost, like more than double in terms of like volume.”
- H3 Max volume doubled that of any competing video model hosted on fal within its first three weeks. - Creative engineers are actively pairing real-time audio models with H3 Max Director to command virtual scenes by voice while they render. - Production teams are utilizing LoRA style fine-tuning to translate recorded AI social media franchises into continuous live broadcasts.
`25:12` The Mechanics of Scene Memory and Coherence
When asked how the model remembers narrative and physical context over long durations, Gorkem clarifies that the memory is handled primarily through raw video rather than plain text descriptions. Because a model cannot attend over raw, uncompressed full-resolution video without hitting memory bottlenecks, it feeds on an aggressively compressed representation of its own past generations. For anything occurring within the trailing two-minute window—roughly the previous four to eight scenes, assuming a scene lasts 15 to 30 seconds—the model retains granular visual details directly from those compressed frames.
Beyond the two-minute mark and up through an entire 60-minute stream, the architecture relies on a gradually evolving system prompt. This state-tracking prompt updates dynamically in the background to summarize structural milestones, tracking characters and overall world rules to prevent drift across an hour-long broadcast. As Gorkem puts it: “And then on top of that, there's like a continuously evolving, gradually evolving system prompt that like, keeps remembering the overall coherence of the world.”
`26:48` Hardware Footprints and Pipeline Heterogeneity
Addressing the economic realities of serving continuous video streams, the conversation turns to GPU footprints and cost optimization. While industry attention is fixated on the infrastructure scale required for next-generation LLMs, video pipelines present equal systems complexity because their individual stages exhibit vastly different compute profiles. Different modules within the generative video pipeline run far more efficiently on tailored hardware configurations rather than on a uniform cluster of high-end enterprise accelerators.
To drive down serving costs even further, the fal team is working to disaggregate the pipeline across heterogeneous infrastructure, with near-term plans to offload specific pipeline components directly to consumer hardware. In this hybrid split, power users could run local inference modules on their personal machines at home while offloading the heavy directorial control plane to fal’s cloud infrastructure. Gorkem outlines this engineering trajectory: “So if you were to do this even more efficient, let's call it maybe even cheaper. We would probably run different parts of the pipeline in different types of hardware.”
`28:15` Professional Workflows: Blender, LLMs, and Absolute Control
Jennifer pivots to high-end Hollywood production and visual effects (VFX), noting that real creative adoption hinges on deterministic control rather than unconstrained generation. Professional animators and visual effects artists have rapidly embraced a hybrid pipeline that pairs traditional 3D software like Blender with H3 Max. In this setup, artists first block out their camera moves, lighting, and primitive geometry inside Blender, rendering out a low-resolution wireframe or rough animation pass.
Feeding this low-res 3D render into H3 Max as an image or video reference provides visual effects teams with near-total control over the final composition, textures, and camera paths. This workflow gained immediate momentum as creators began scripting complex 3D Blender scenes directly through LLMs like GPT-4o / Astra, immediately piping the geometric references into H3 Max to output photorealistic final shots. Explaining why this approach is becoming an industry standard, Gorkem observes: “Basically, you render a low resolution of your scene, what you want to do using Blender, once you add that video as a reference to an AI model, you basically get close to 100% controllability.”
`30:00` Precision Controllability: From Camera Paths to Lip Sync
Using a large language model to generate a 3D Blender scene and feeding that downstream into a video model like H3 Max unlocks near-total controllability for studio creators. Because H3 Max generates frames so quickly, professionals can iterate through multiple setups in parallel. While the fal team initially focused on raw speed, latency reduction, and baseline visual fidelity, their immediate roadmap has shifted toward granular controllability so studio creators can output assets matching strict production requirements. Rather than settling for 80% to 90% prompt adherence, the platform is introducing specialized point solutions. Among these are a lip-synchronization model that takes an audio track and an image or video reference to lock mouth shapes frame-by-frame, and motion-retargeting controls that transfer real dance or stunt movements onto AI-generated characters.
```mermaid flowchart LR accTitle: Controllability conditioning pipeline accDescr: Pipeline showing input prompts, 3D trajectory JSON, audio, and motion conditioning entering the post-trained base model to output reliable video.
subgraph Inputs["Structured Inputs"] A["Blender / 3D Layout"] B["JSON Trajectory: T0, T1 coordinates"] C["Audio Track & Reference"] D["Motion Capture Data"] end
subgraph Engine["Post-Trained Architecture"] E["H3 Max Base Model"] F["Targeted Conditioning Layers"] E --- F end
subgraph Output["Studio-Grade Output"] G["Deterministic Video Stream"] end
Inputs --> Engine Engine --> Output ```
A major breakthrough is deterministic camera control. Instead of relying on descriptive natural language prompts that leave movement to chance, users supply structured spatial coordinates over time. Providing a JSON object defining exact positions—such as `(0, 0, 0)` at time $T_0$ transitioning to a specific 90-degree angle at time $T_1$—forces the model to treat the trajectory as absolute ground truth without hallucinating camera shifts. Because the underlying model has deep spatial representations, post-training aligns it to strictly respect these camera paths, even allowing creators to reconstruct coherent 3D scenes from a single visual input.
- Video models cannot reliably follow camera instructions out of the box; they require dedicated post-training pipelines to enforce strict coordinate adherence. - Conditioning mechanisms have evolved sequentially: 1. *Text-to-video*: Simple prompt inputs, but zero character consistency. 2. *Image-to-video*: Conditioning on an initial frame or edited image, though unable to introduce new characters midway through a clip. 3. *Reference-to-video*: Passing starting keyframes, character sheets, and dedicated character voice profiles. 4. *Directorial control layers*: Explicit timestamped camera trajectories and lighting vectors (specifying direction, angle, and intensity across timestamps).
`33:48` The Hollywood Influx and Studio Point Solutions
The entertainment industry has quickly emerged as fal’s fastest-growing customer segment, marking a dramatic shift from a year prior when studio adoption was virtually zero. Gorkem Yurtseven explains that mainstream studios are not looking to push a single button to generate an entire film from scratch. Instead, professional filmmakers need precise, surgical point solutions that slot neatly into established visual effects and editing pipelines: extending an existing clip by several frames, altering a camera move after a shoot, or adjusting scene lighting in post-production. At their recent industry showcase, Amazon MGM Studios introduced their "NARA" tool, which relies heavily on fal's underlying infrastructure.
A persistent disconnect exists between the broad objectives of frontier AI research labs and the operational realities of film crews on a set. Consumer interfaces prioritize loose prompt-based generation, but Hollywood directors communicate using precise technical terminology: focal lengths, camera angles, blocking, and lighting schemas. fal bridges this gap by creating reusable post-training infrastructure and toolkits that can be adapted to open-weight models as well as frontier closed-source models. As Gorkem puts it:
> “What Hollywood needs and what the creators actually need and what the research labs are working on, there’s a little bit of a disconnect there.”
`35:28` Overcoming Enterprise Barriers: US Hosting, IP Protections, and Gen Media
Beyond raw technical capabilities, studio enterprise adoption has historically faced severe legal and regulatory hurdles around copyright safety and data governance. fal addressed these concerns by building an infrastructure layer that enables studios to fine-tune and condition models using their proprietary intellectual property safely inside isolated environments. Furthermore, they resolved data residency and sovereignty requirements by offering US-hosted deployments of top international open models, including ByteDance's Seaweed (Seedance), removing the final compliance obstacle for major domestic entertainment brands.
Reflecting on the evolving landscape, Gorkem and Jennifer Li highlight the shifting demographic at their Generative Media Conference. The inaugural event attracted mostly consumer-facing AI experimenters and a handful of curious Hollywood observers. The second edition has been heavily populated by major legacy studios and specialized spin-off AI production companies building fully AI-native productions:
> “Everything is ready and we believe they’re going to 10x, 100x their AI usage in the coming months.”
- Studios require isolated, secure post-training systems so that private franchise IP remains fully protected while training custom character checkpoints. - US-based hosting for foreign models (such as Chinese open-weight architectures) resolves stringent enterprise compliance and security demands. - Studio production units are pivoting away from viewing AI as a disruptive threat toward embedding modular generative point solutions across production pipelines.
## Notable quotes > `33:48` What Hollywood needs and what the creators actually need and what the research labs are working on, there’s a little bit of a disconnect there. — Gorkem Yurtseven
> `28:15` Basically, you render a low resolution of your scene, what you want to do using Blender, once you add that video as a reference to an AI model, you basically get close to 100% controllability. — Gorkem Yurtseven
## Mentioned fal, Jennifer Li, Gorkem Yurtseven, Batuhan Taskaya, H3 Max, H3 Max Turbo, H3 Max Director, MiniMax, Hailuo, NVIDIA, Hopper (H100), Blackwell (GB200), Rehan, Pieter Levels, levelsio, Twitch, WebRTC, fal Live, ChatGPT voice mode, Whisper, TikTok, Instagram, Blender, GPT-4o, Astra, Amazon MGM Studios, NARA, ByteDance, Seaweed (Seedance), Generative Media Conference
## Video description a16z General Partner Jennifer Li sits down with fal co-founder Gorkem Yurtseven and Head of Engineering Batuhan Taskaya to discuss what changes when generative video becomes fast enough to run in real time. They unpack the technical work behind H3 Max, fal’s post-trained version of MiniMax’s open-weight video model, and how combining model post-training with systems and hardware optimization significantly reduced generation time while maintaining quality. That speed has enabled experiments with continuous video, including streams that can remember previous scenes and respond to new directions while they’re running. They also discuss why the next challenge may be less about speed and more about control, from camera movement and lighting to characters, motion, and lip sync. And they explore what those capabilities could mean for professional creative workflows, where artists and studios need predictable tools rather than simply generating a video from a prompt.
[Source](https://mgln.ai/e/1344/afp-848985-injected.calisto.simplecastaudio.com/3f86df7b-51c6-4101-88a2-550dba782de8/episodes/3cafbcc2-e9d6-40ae-adcb-adee4b50432e/audio/128/default.mp3?aid=rss_feed&awCollectionId=3f86df7b-51c6-4101-88a2-550dba782de8&awEpisodeId=3cafbcc2-e9d6-40ae-adcb-adee4b50432e&feed=JGE3yC0V)