Inside our release pipeline: the strict MP4 muxer

Every release on Autoretto goes through fifteen steps before it lands on a creator's channel. The step I want to talk about is the in-process MP4 muxer. It sits midway through the pipeline. Upstream, other stages have generated the audio track, the video frames, and the metadata. Downstream, the quality and policy gates are waiting. The muxer's job is to take those separate pieces and make one MP4 file that can be uploaded.

What does it actually receive? The manifest from the orchestrator lists the paths. There is a video track that comes from the render stage. It is often a silent file because the audio is separate. There is an audio file from Suno after normalization. There is a JPEG for the thumbnail, but that is not muxed into the video; thumbnail is handled later. The muxer gets a JSON block with the title, description, tags, and a channel identifier. It also gets the exact time base for the source video and the sample rate for the audio, because these need to match.

The real work is a few commands. We use a container that has ffmpeg and ffprobe built in. The muxer runs ffmpeg with a map that joins video and audio streams. It sets the output to H.264 and AAC, because that is the combination YouTube handles well. It sets faststart so the file plays before it has fully downloaded. It writes the metadata by setting the tags. It also sets the seed for the bitstream so the packet ordering is clean. After ffmpeg returns a zero exit code, the muxer does not celebrate. It starts the verification.

Verification is where the step earns its keep. The muxer creates a sidecar file with the expected checksum from an earlier stage. Then it uses ffprobe to read the file's actual streams. It checks that there is exactly one video stream and one audio stream. It checks the codecs. It checks that the duration of the video and audio differ by no more than one frame. Why one frame? Because the source audio might have a length that does not perfectly divide by the video frame rate. A one frame gap is fine, but anything larger means a desync.

Then it decodes the first few frames and the last few frames. It does that to catch a file that is there but empty. It reads the moov atom from the end of the file and confirms the ftyp box is present. It also runs a small script that looks for the metadata keys it wrote. If any check fails, the muxer throws away the file and marks the step failed. It does not try to repair the file. Repairing a broken mux hides the actual issue upstream, and the next release would fail the same way.

A failed mux triggers a rerun of the previous stage with a backoff timer. That is not automatic in the sense of a loop. The orchestrator records the failure and waits for a fresh attempt. The muxer itself logs the exact error from ffmpeg and from its validation checks. That log goes to a central store so we can see patterns. The release does not move to the quality gate until the muxer gives it a clean receipt.

The receipt is a small JSON document. It lists the verified duration, the codecs, the resolution, the audio sample rate, and a SHA-256 hash of the final file. The next stages use that hash to confirm the file has not changed on disk. When the scheduler later picks up the release for upload, it compares the hash before it sends the bytes. That way the file that gets uploaded is exactly the one the muxer verified.

That might sound like overkill for a step that simply wraps video in a container. But in an autonomous system, no one is watching the logs in real time. The verification is the substitute for human eyes. If the step cannot prove the file is good, it does not exist. That is why we wrote the muxer to be strict. A corrupt MP4 that slips through could get scheduled to YouTube and fail after being uploaded, or worse, pass but show a broken video to viewers. We do not want that. So the muxer is strict, and it has to prove itself before it hands off to the next step.