2026-07-20

Browser-native MP4 in NearVid: WebCodecs, a spike that found no H.264, and why we shipped Mediabunny

The last post ended with a promise: NearVid's GPL-free ffmpeg.wasm core can't re-encode to MP4/MOV with H.264 — no royalty-free encoder exists to bundle — but a separate, independent pipeline using the browser's own WebCodecs API covers real MP4 output instead, "a topic for another post, not this one." This is that post. It's a two-stage story: an exploratory spike that found the codec we needed completely unavailable in its own test environment, and a production integration that shipped anyway — using a different library than the spike used, for reasons the spike itself surfaced.

What the spike set out to check

The idea is simple to state: VideoEncoder/VideoDecoder (video) and AudioEncoder/AudioDecoder (audio) are browser-native WebCodecs APIs that hand off to whatever H.264/AAC implementation the browser itself ships — hardware or a vendor-licensed software encoder. NearVid never bundles that codec code, so the GPL and patent questions that rule out an MP4 path in the ffmpeg.wasm engine don't apply here. The spike's job was to find out whether that mechanism actually works, in a real browser, with real files — not to assume it from the spec.

What it actually found: H.264 was completely absent, in this sandbox

The spike ran in real Playwright Chromium (the literal engine version installed in the sandbox, not a description of one) and called VideoEncoder.isConfigSupported() against nine H.264/HEVC codec-string combinations: H.264 Baseline (avc1.42001f, tested with default, prefer-software, and prefer-hardware acceleration hints), H.264 Main (avc1.4d0028), H.264 High at two levels (avc1.640028, avc1.64001f), H.264 High at 1080p (avc1.640033), and HEVC Main (hvc1.1.6.L93.B0). Every single one came back supported: false. A VP9 control string (vp09.00.10.08), tested the same way, came back true — so this wasn't a broken test harness, it was a real, specific absence. Audio told the same story: AAC-LC (mp4a.40.2) and MP3 both reported supported: false; only Opus reported true.

isConfigSupported() is a capability query, and queries can be wrong (more on that below), so the spike didn't stop there. It demuxed a real H.264 source file (fixtures/src.mp4) with mp4box.js, pulled out an actual H.264 sample, and fed it directly to a real VideoDecoder. The result wasn't a polite "unsupported" flag — it was an immediate runtime error, "Unsupported configuration. Check isConfigSupported() prior to calling configure().", firing on the error callback before the decoder could do anything. The capability flag and the real behavior agreed: no H.264 in this specific browser build, full stop.

Why (researched, not confirmed)

The spike's own §1-5 is explicit about the difference between a finding and a hypothesis, and this section is a hypothesis. Chrome's software H.264 encoder is Cisco's OpenH264, and in a normal, Google-branded Chrome install that component isn't compiled in statically — it's fetched at runtime by Chrome's Component Updater. Playwright bundles "Chromium for Testing," an open-source build that doesn't go through that update channel, so the working hypothesis is that the OpenH264 component (and possibly other licensed codec components) simply never gets installed in that build. That's a plausible, researched explanation, not something the spike reproduced directly — there was no official Google-branded Chrome binary available in that sandbox to test against, so "does real Chrome actually have it" stayed an open question, deliberately left unconfirmed rather than assumed.

What it proved instead: the pipeline itself is real

Absence of H.264 in one build doesn't mean the mechanism is untested — it means the codec had to be swapped for one this browser actually supports, and the exact same pipeline shape run for real. The spike built a full chain: demux the source with mp4box.js, decode the extracted samples with VideoDecoder/AudioDecoder, re-encode with VideoEncoder/AudioEncoder using VP9 video + Opus audio (the codecs that actually returned supported: true), mux the result into MP4 with mp4-muxer, and load the resulting bytes into a real <video> element to confirm it actually played. It did: 90 video samples decoded and re-encoded (first chunk type "key", all chunks non-zero length), 151 audio samples decoded and re-encoded, a muxed file that started with a valid ftyp box, and — the part that matters most — real playback: duration 3.04s matching the source, resolution 640×360 matching the source, and playbackAdvanced: true, meaning currentTime actually moved forward during play() rather than the check just confirming the file's headers parsed. That's the load-bearing claim of the whole spike: the demux→decode→encode→mux→play mechanism genuinely works, end to end, with real data. Swapping the codec strings from vp09.xx to avc1.xx on a browser that actually has H.264 is expected to be the same code path — but that specific swap was never run for real in this sandbox, and the spike says so plainly instead of asserting it.

Two real bugs, hit while actually running the code

Both of these only surfaced because the pipeline was actually built and executed, not just designed on paper:

• mp4box.js extraction options set too late. Calling setExtractionOptions()/start() after the file-loading promise resolved left the sample count at zero. The reason: appendBuffer() parses the whole file synchronously as it's fed in, so by the time extraction options are set afterward, parsing has already finished and there's nothing left to extract into. The fix was to call setExtractionOptions()/start() inside the onReady callback, before parsing completes.
• A video.src assignment that never ran before the timeout. The playback-verification code originally set video.src = blobUrl after an await new Promise(...) that was itself waiting on the video's loadedmetadata event — a wait that can obviously never resolve if the source is never assigned. The bug is exactly what it looks like once you say it out loud: await defers everything written after it until the promise settles, so code meant to kick the promise off has to run inside the executor, immediately after the event listener is registered — not after the awaited line.

Neither bug is exotic. Both are the specific, mundane kind of mistake that only shows up when code is actually run against real files rather than reasoned about in the abstract — which is exactly why the spike ran it for real instead of stopping at "the API shape looks right."

From spike to production: why Mediabunny, not mp4box.js + mp4-muxer

The spike used mp4box.js (demux) and mp4-muxer (mux) because they were the obvious, permissively licensed libraries for the job — and while it was running npm install, it hit a real, unprompted warning: npm warn deprecated mp4-muxer@5.2.2: This library is superseded by Mediabunny. Please migrate to it. The package's own README repeats it: mp4-muxer is deprecated, no longer maintained, no new features or bug fixes. The spike noted this and flagged Mediabunny as the right production choice, without installing or testing it — that verification was left for later, on purpose, rather than claimed without having run it.

Production (packages/near-video-engine/mp4-engine.js) did that verification and switched to Mediabunny for three concrete reasons documented in that file's header comment, not just because the old library was deprecated:

• Wider container support. mp4box.js only understands ISOBMFF (MP4/MOV). Mediabunny's Input reads MP4, MOV, WebM, MKV, and MPEG-TS natively, which widens the set of source files this pipeline can demux without any extra work.
• A real smoke-test, not just a capability flag, on Firefox. Mediabunny's canEncodeVideo()/canEncodeAudio() call the standard isConfigSupported() checks — but on Firefox specifically, they also run an actual one-frame encode as a smoke test, because Mediabunny's own issue tracker found isConfigSupported() can lie there. That's not a hypothetical: the spike's own §5-2 cites a real, filed bug, Mozilla Bugzilla #1918769, "WebCodecs VideoDecoder fails on h.264 video frame" — a case where the capability flag said yes and the real decode failed, almost exactly the same flag-vs-reality mismatch the spike's own §1-4 dug into for H.264 in this sandbox's Chromium.
• No unbundled-vendor build step. Every worker Mediabunny spawns is a self-contained Blob-URL worker (the worker function is serialized into a Blob at runtime, not loaded from a separate file via new Worker(new URL(...))). Unlike @ffmpeg/ffmpeg (see the first post's engine, and its own build-process workaround), Mediabunny needs no special copy-unbundled step in build.mjs — it's a completely ordinary esbuild dependency.

Mediabunny is MPL-2.0 licensed — not GPL, and, like the BSD/MIT libraries the spike checked, fine for closed-source, commercial use.

Choices that aren't defaults, made on purpose

Two decisions in mp4-engine.js are easy to miss unless you know what they're guarding against:

• H.264 Baseline profile, not Mediabunny's own default. Mediabunny defaults to High profile (avc1.64...) when you don't specify a codec string. Production pins avc1.42001f (Baseline, level 3.1) instead — the exact string the spike itself tested in §1-2 — because the research in §1-5 traces Chrome's software H.264 encoder specifically to OpenH264, which only implements Constrained Baseline profile. Probing (or encoding) with a higher profile could report unsupported and silently hide the MP4 feature on exactly the software-encode-only devices where users need it most. The same codec string is used for both the capability probe and the real encode — probing one config and encoding with a more demanding one would just reintroduce the flag-vs-reality gap Mediabunny already works around for Firefox.
• Both video AND audio encode support are required, not just video. checkMp4EncodeSupport() only reports supported: true if both canEncodeVideo() and canEncodeAudio() succeed. A "MP4 output" that silently dropped the audio track because AAC wasn't available would misrepresent the "H.264 + AAC" feature actually promised to users.

And one thing that's deliberately absent: nothing here ever branches on the user agent string. Every gate is a real isConfigSupported()/canEncodeVideo()/ canEncodeAudio() call, exactly as both the spike and the production module insist — because a UA sniff would be exactly the kind of assumption this whole investigation exists to avoid making.

What this doesn't cover

This is an additional, conditional pipeline layered on top of the existing engine, not a replacement for it. The GPL-free ffmpeg.wasm engine from the first post is still the only path to WebM and GIF output, and it's still what handles any container Mediabunny can't read. Mediabunny's real, verified format coverage is MP4/MOV/WebM/MKV/MPEG-TS — AVI, WMV, and 3GP are not in its supported list. canReadForMp4() simply doesn't offer the MP4 option for files in those containers, rather than silently routing them through ffmpeg.wasm first as an intermediate step and then re-encoding a second time through WebCodecs. That two-stage route was considered and rejected: chaining two heavy pipelines together and taking a second lossy re-encode pass for a single output is a worse trade than just not offering MP4 for that one file — the visitor still has trim, WebM, and GIF for it either way. And even where MP4 output is offered, availability is never universal or guaranteed — it depends on the specific browser and device, checked live, every time, not assumed from a browser name.

The honest summary

The spike's own verdict was CONDITIONAL-GO, not GO, and that distinction mattered: it proved the demux→decode→encode→mux→play mechanism for real, with a real file, catching two real bugs along the way — but it could not prove H.264 availability itself, because the one browser build available to test with didn't have it. The researched explanation (OpenH264 via Chrome's Component Updater) is plausible and cited, not confirmed. Production shipped anyway, on the strength of the proven mechanism plus runtime feature detection that fails safe when the codec genuinely isn't there — and along the way it upgraded from the spike's mp4box.js + mp4-muxer combo to Mediabunny, for concrete, documented reasons: wider format support, a real smoke test on a browser known to lie about its own capabilities, and deliberate profile and dual-track checks that guard against exactly the failure modes this whole investigation was built to catch.