Middle-earth Live

I wanted to know whether an AI movie could get better without forgetting what it had already solved.

The starting point was Andrej Karpathy's Middle-earth experiment. He gave Opus 5 the opening paragraph of The Lord of the Rings and a large token budget, then let it spend roughly two hours writing 5,500 lines of Three.js that procedurally staged the story. His larger point stayed with me: LLMs make strange, highly specific worlds cheap enough to attempt, but worlds also expose a major weakness. A model can generate an enormous system and still struggle to watch, play, and audit what it made.

Middle-earth Live asks what comes after that first feat of generation. Instead of treating the generated world as a one-off artifact, I turned the world into a production substrate: something persistent, inspectable, and capable of carrying accepted decisions into the next revision.

Most generative video workflows behave like rerolls. A good character, camera move, or piece of geography can disappear in the next attempt because the model has to rediscover the whole scene. Middle-earth Live is my attempt to build a different production loop: one persistent world, short versioned cuts, explicit continuity checks, and a record of what each revision is allowed to change.

The project is a private adaptation studio and continuous broadcast of The Lord of the Rings. The broadcast shows the latest approved sequence. Behind it, the production system can revise one short section at a time without throwing away staging, timing, camera, or world state that already works.

One World, Five Beats

The first proof is deliberately small: Episode 001, Cut 001, a reproducible 78-second movement from morning at Bag End to Bilbo's disappearance at the birthday party.

Five typed story beats drive the episode clock. They control the camera, daylight, fog, lanterns, party setup, cast, captions, and narration. Play, pause, seek, beat jumps, and Reset all resolve from the same state. Reset does not approximate the opening frame; it returns to the same seed, camera, world values, and cast.

The source world is intentionally simple. Bag End, the Party Tree, the road, Gandalf's cart, the party platform, Bilbo, Frodo, Gandalf, and the crowd are readable low-poly forms rather than finished art. Their job is to make geography and story action unambiguous before an expensive model touches the image.

Three.js Is Truth

The central rule is simple: Three.js owns story truth, and the neural renderer owns appearance.

The source scene decides who is present, where they stand, what the camera sees, when the light changes, and the exact frame where Bilbo vanishes. The neural pass can change texture, atmosphere, and cinematic finish, but it is not allowed to rewrite the event. Every rendered frame remains traceable to an episode time, source frame, world sample, seed, and renderer configuration.

I used Codex to operate the production loop around that contract: capture exact browser frames, hash the source sequence, prepare the local renderer, run the GPU jobs through WSL, assemble the outputs, and build review sheets. ElevenLabs narration is generated offline, stored as one editable clip per beat, and synchronized to the same story clock. No provider key reaches the browser.

The First Neural Continuity Gate

The latest experiment covers the 14 seconds around Bilbo's disappearance. The system captured a deterministic 336-frame source control, then rendered two overlapping eight-second windows through StreamDiffusionV2 with a Wan causal video-to-video model. Both windows used the same seed and preservation settings.

The assembly process compared every frame in the two-second overlap and chose a seam at story time 71.4375 seconds. That metric found a plausible join, but it did not decide whether the sequence passed. The final 224-frame candidate was decoded into 14 lossless review sheets and inspected frame by frame.

Bilbo remains singular before the cue and is absent afterward. The ring appears and clears on schedule. The platform stays empty through the final frame, the crowd does not reset, and the join between the independently rendered windows is not the sequence's largest visual discontinuity.

The result passed its continuity gate as a local review candidate. It did not pass as final movie imagery. The neural side is still dark, soft, and blocky. That distinction matters: a coherent artifact is useful evidence, but it is not automatically a better cut.

The Next Test Is Pre-Registered

The next experiment is not a general request to make the image prettier. Gate 2C changes one variable: neural noise scale moves from 0.45 to 0.60. The source window, prompt, seed, model, dimensions, frame rate, and post-processing remain fixed.

The accepted Gate 2B window becomes the parent and comparison baseline. Before running the GPU job, the test records its temporal-stability and source-fidelity floors, then requires a complete 128-frame visual review. A candidate must look meaningfully better in at least two areas such as exposure, material detail, and live-action realism without duplicating Bilbo, weakening the disappearance, or making the action harder to read.

Gate 2C is currently prepared but has not run. The runner allows one explicit local attempt and no automatic retry. That constraint turns the next render into an experiment with a falsifiable result rather than another untracked generation.

What It Proves

The current build proves that one short scene can have a deterministic performance, synchronized narration, a reproducible neural input, resumable local rendering, and a review surface that refuses to present an unverified file as canonical.

It does not yet prove the larger system. There is no autonomous adaptation, critique, or promotion loop. Cut 001 is still the baseline. Gate 2C now tests whether appearance can improve without losing continuity. After that, the larger test is whether later cuts can repair a named failure while preserving everything the parent already got right. If Cut 006 is only the sixth reroll, the project has retries. If it inherits accepted decisions and improves under comparison, it has production memory.

Middle-earth Live is a private, noncommercial adaptation experiment. Everything shown here comes from original code, geometry, narration, and local renders. The project does not publish book text, film footage, actor likenesses, or a public stream.