byterover has been doing something similar for a while. amp was initially doing a variation of this and then pivoted. I built a similar tool about 9 months ago and then abandoned it.
The approach seems tempting but there's something off about it I think I might have figure out.