I've been working with video encoding, subtitles and basically anything that is "layer things on top of videos" for many years now, and lately I started wondering if there was a more efficient way to do it without relying on the usual methods and libraries. Basically, every time you paint stuff over a video, you create the overlays with 'some' tool (something that generates cards as PNGs, or a headless browser for more complex animated designs.. or other techniques), and then use good old ffmpeg to merge the composition together, or eventually use ffmpeg's complex filtergraphs with all their quirks and limitations if you're desperate, but nobody does that anymore.
But what if ffmpeg could do proper composition, natively? That's probably a bit out of scope for them, so I set out to start rewriting ffmpeg's CLI in Rust with some extras.. because, why not. Of course, the codecs still comes from libav*, but everything above it is new: a planner that figure out the cheapest way to produce the output (frames with nothing drawn on them aren't re-encoded), a compositor with layers, keyframes, masks and transitions, and a layout engine for the part of CSS that is most commonly used: flexbox, gradients, shadows, clip paths, blend modes, and @keyframes played as written.
Now I have something that paints pretty animated graphics natively over videos very quickly, and without using playwright or chrominum. You can see some benchmarks here against two of the most common methods used to achieve this here: https://github.com/geneva-render/geneva/blob/main/docs/bench...
As you probably guessed, this was all written by Fable/Opus under my guidance. Just stating it plainly so you can decide whether to trust it or not. This is just the engine, and I have a couple of ideas on how to move it forward, but I'd like to hear some feedback before carrying on blindly, so feel free to reply here or get in touch.
Some rendered examples are in the repo and here: https://genevarender.com