In the first 10 years of the rclone project we received about 20 security disclosures through GitHub. We had to deal with over 40 in the last month! That has taken a huge amount of my time, even using AI tools to triage and come up with fixes for review.
The hit rate for those security disclosures is pretty good - about 75% of them have a nugget of something which needs looking at. The configurations for rclone have got increasingly unlikely so I'm hoping they will dry up eventually.
I was considering just merging the fixes straight to master just to make my life easier rather than holding a dozen independent security fixes on branches and merging them at the point release and hoping not to have too many conflicts to fix up. I've decided to stick with the process for the moment.
GitHub assigns CVEs for the advisories. Before the AI apocalypse they took 2-3 days for an assignment but now it they are running at 3-4 weeks so I have to send the point releases out with CVE-PENDING in the changelog which isn't ideal.
Not sure what the solution is, but it is definitely a problem for us.
A strange bottleneck; anyone know why that would be so slow?
- grab a group of (related) bugs/defects/vulns
- fix them on a branch like bug-batch-XXX
- run that group through the verification, landing in main, CI/CD flow to amortize process cost
- repeat as needed to process backlog
My experience is that process often has irreducible time (eg, two days due to reviews by various parties); but that time slot can be shared between several bugs in a single PR — especially if you have several related to the same feature.
No matter how good AI gets at fixing bugs we'll never fix them when there's no will to fix things. Software will never be good if there's no will to make good software. The problem has always been about will. To many better products. It's insane that in a time where we can do better on speed and quality we still choose speed and tell ourselves it's velocity
I'm hoping one of the unintended side effect of it being essentially free to find and exploit (and fix) software bugs is that companies become less cavalier about shipping bugs in their software. Unlike most of the industry I don't believe "bugs are inevitable." Bugs are a choice developers make when they're rushing and careless and when all of their incentives are to ship quickly. You can ship bug-free software but it takes (or used to take) a really long time and a lot of care, care that commercial software developers just don't ever seem to muster.
Maybe when their software is getting 0wned over and over and 30 security issues are published a day, they'll start caring and taking their time.
The worst one was where I fixed a datetime bug and although it had been sending out false alerts, I was asked to dry run the 5 lines of code I changed, like a coding interview. In all this pressure I forgot what the code was meant to do, and was dismissed and asked to set up another meeting with an explanation of all the various cases that could happen...
Very low signal information, preemptively trying to cover every rebuttal despite nobody ever planning on making one, in the few times someone does he plays devils advocate endlessly
Like bro just let us babysit these agents, everything’s going to happen
I do agree with the author's ideas, though; most of these are things that should have been done much sooner, and I suppose it's good in a sense that there is a forcing factor now.
True, but it used to take days or weeks of research, testing, and RE to get those PoCs.
Today the entire chain - reading commits, RE patch binaries, building exploit, scripting exploit scan, $profit - can be fully automated and happen in minutes or hours.
“I’m told there’s a path traversal exploit in this package. Can you find it?” - probably a reasonably high chance of it finding one, even if you just made that rumor up.
Add to that the danger of supply-chain attacks where you don't even want automatic updates.
LLMs are extremely good at finding corner-case vulnerabilities in C bindings even within a memory safe language; see for example the fixes in an OCaml crypto library here: https://discuss.ocaml.org/t/the-series-of-mirage-crypto-rele...