I talked about that directly in the post. We didn't want to just challenge everyone. We use JS challenges but we try to use them sparingly. The rest of the ops team and I were fighting to stay up but it never got so bad that it was a choice between complete outage and using the Under Attack mode.

Author here. This was not a misconfigured data scraper. We see those every week[1]. This attack wasn't scraping useful content. It was almost entirely 404s and 302s and pulled virtually zero real docs. It specifically looked for URLs not served by the CDN and when it found a pattern, did millions of variations of it. Whether built by an AI or not, it was designed to cause outages and financial damage from autoscaling. However, as others have suggested, we may have been a test run for a real target.

[1] https://about.readthedocs.com/blog/2024/07/ai-crawlers-abuse...

> Most of their docs hosting is static/easily CDN cached

The article says

> and it purposefully attacked areas that bypassed caching

So that doesn't work. Also, it seems that they were trying to cause financial harm, not to take down the infrastructure but to make it costly for the org itself. That's smart.

Could have been a live-fire exercise by a nation state.

I'm the author of the blog. I don't know. Internally, we were half joking that we were going to get ransom notice, but we never did.

The only thing that sort of correlates with this attack is that before it started, we began rolling out some slightly more aggressive rate limits one by one. This was mostly because anytime any new "company" thinks they're going to catchup with Claude/OpenAI, they scrape us very aggressively (and they're not respectful about it). My guess is that the attackers behind this attack were already probing us (they were) and they thought the window of opportunity might be closing.

Could be testing in preparation for attacking something more critical?

Either testing for something bigger OR demonstrating their power to a 3rd party with minimal real disruption