The Tech:
GPU Heavy: It uses decord and PyTorch for scene analysis. I’m calculating action density and spectral flux locally to find hooks before hitting an LLM.
Local Audio: I’m using ChatterBox locally for TTS to avoid recurring costs and privacy leaks.
Rendering: Final assembly is offloaded to NVENC.
Looking for Collaborators: I’m currently looking for PRs specifically around: Intelligent Auto-Zoom: Using YOLO/RT-DETR to follow the action in a 9:16 crop.
Voice Engine Upgrades: Moving toward ChatterBoxTurbo or NVIDIA's latest TTS.
It's fully dockerized, and also has a makefile. Would love some feedback on the pipeline architecture!This is the first sentence in your features section, so it is not strange if users don't understand if this tool is running locally or not.
Still a cool tool though! Although it seems partly AI generated.
Regardless, we need more tools like this to speed social media towards death.
I think that sounds a little too convenient and idealistic to be what really happens, but I did find the concept to be a potential positive to what's happening around it. Facebook is already a good portion of the way there, being stuffed with bots consuming stolen or AI content from other bots, with confused elderly people in the middle.
I did smth similar 4 years ago with YOLO ultralytics.
Back then I used chat messsges spike as one of several variables to detect highs and fails moments. It needed a lot a human validation but was so fun.
Keep going