I think the big cloud companies (AWS) figured out that they could scrape compute-intensive pages in order to drive up their customers' spend. Getting hammered? Upgrade to more-expensive instances. Not using cloud yet? We'll force you to.
The other possibility is cloudflare punishing anybody who isn't using it.
Probably a combination of these two things. Whoever's behind this has ungodly supplies of cheap bandwidth -- more than any AI company does. It's a cloud company.
> Whoever's behind this has ungodly supplies of cheap bandwidth -- more than any AI company does. It's a cloud company.
Most of the major cloud companies are themselves also AI companies, so I don't think the “cloud companies are artificially driving up compute spend” hypothesis is mutually exclusive with the “AI companies are doing a very bad job at scraping” hypothesis.
I think the big cloud companies (AWS) figured out that they could scrape compute-intensive pages in order to drive up their customers' spend. Getting hammered? Upgrade to more-expensive instances. Not using cloud yet? We'll force you to.
The other possibility is cloudflare punishing anybody who isn't using it.
Probably a combination of these two things. Whoever's behind this has ungodly supplies of cheap bandwidth -- more than any AI company does. It's a cloud company.