Middle-earth Live

I wanted to know whether an AI-made film could be improved one revision at a time without losing what already worked.

It started from Andrej Karpathy's Middle-earth experiment, where Opus 5 generated a large 3D world in Three.js from the opening of The Lord of the Rings. That was a single generation. I wanted something I could keep working on, where the staging, timing, camera, and state of the world carry over from one revision to the next.

With most AI video tools, fixing a shot means generating the whole thing again. Here the 3D world stays the same between versions, the cuts are short and each one is saved as a version, and continuity gets checked, so every new pass starts from what already works.

A 78-second test

Episode 001 runs 78 seconds, from morning at Bag End to Bilbo's disappearance at the birthday party, and plays out exactly the same way every time. It is split into five story beats, and each beat sets the camera, daylight, fog, lanterns, who is on screen, captions, and narration. Reset puts the seed, the camera, the world settings, and the cast back where they started.

The low-poly world is simple on purpose. It makes clear where everything is and what happens before an AI model changes how it looks.

The 3D scene decides what happens

The Three.js scene decides who is there, where they stand, what the camera sees, and the exact frame where Bilbo vanishes. The AI renderer can change how things look, but it can't change what happens. Every rendered frame can be traced back to the 3D frame it came from, its seed, and the renderer settings.

Codex does the production work around that: it captures frames from the 3D scene, prepares the renders that run locally, assembles the output, and builds review sheets. The ElevenLabs narration is one editable clip per story beat, timed to the same clock as the scene.

Bilbo's disappearance

The first AI rendering test covers the 14 seconds around Bilbo's disappearance. I rendered two overlapping eight-second sections with StreamDiffusionV2, joined them into a 224-frame candidate, and reviewed it frame by frame.

Before the cue there is one Bilbo, and after it there are none. The platform stays empty, the crowd doesn't reset, and continuity holds across the join between the two sections. It passed its continuity check, but it doesn't look like finished film yet. It's still dark, soft, and blocky.

The next test changes one setting: the neural noise scale goes from 0.45 to 0.60, and the source, prompt, seed, model, and post-processing stay the same. It's set up but hasn't run yet.

The current build proves that one short scene can play the same way every time, keep its narration in sync, give the AI renderer the same input on every run, and be reviewed frame by frame. It doesn't yet prove a system that can adapt a story on its own. The bigger test is whether a later cut can fix one specific problem without losing what already works.

Middle-earth Live is a private, noncommercial adaptation experiment. Everything shown here comes from original code, 3D models, narration, and renders made locally. It doesn't publish any book text, film footage, or actor likenesses, and there is no public stream.