...and here I am, running a blog that if it gets 15k hits a second, it won't even bat an eye, and I could run it on a potato. Probably because I don't serve hundreds of megabytes of garbage to visitors. (The preview image is also controllable iirc, so just, like, set it to something reasonably sized.)
map $http_user_agent $badagent {
default 0;
# list of AI crawler user agents in "~crawler 1" format
}
if ($badagent) {
rewrite ^ /gpt;
}
location /gpt {
proxy_pass https://courses.cs.washington.edu/courses/cse163/20wi/files/lectures/L04/bee-movie.txt;
}
...is a wonderful thing to put in my nginx config. (you can try curl -Is -H "User-Agent: GPTBot" https://chronicles.mad-scientist.club/robots.txt | grep content-length: to see it in action ;))
That would result in those fediverse servers theoretically requesting 333333 * 114MB = ~38Gigabyte/s.
On the other hand, if the site linked would not serve garbage, and would fit like 1Mb like a normal site, then this would be only ~325mb/s, and while that's still high, it's not the end of the world. If it's a site that actually puts effort into being optimized, and a request fits in ~300kb (still a lot, in my book, for what is essentially a preview, with only tiny parts of the actual content loaded), then we're looking at 95mb/s.
If said site puts effort into making their previews reasonable, and serve ~30kb, then that's 9mb/s. It's 3190 in the Year of Our Lady Discord. A potato can serve that.
It's not. It just doesn't get enough hits for that 86k to matter. Fun fact: most AI crawlers hit /robots.txt first, they get served a bee movie script, fail to interpret it, and leave, without crawling further. If I'd let them crawl the entire site, that'd result in about two megabytes of traffic. By serving a 86kb file that doesn't pass as robots.txt and has no links, I actually save bandwidth. Not on a single request, but by preventing a hundred others.
It's about 5 times longer than previous releases were maintained for, and is an experiment. If there's a need for a longer term support branch, there will be one. It's pointless to start maintaining an 5+ year branch with 0 users and a handful of volunteers, none of whom are paid for doing the maintenance.
got him ( lemy.lol )
Is Mastodon's Link-Previewing Overloading Servers ? ( news.itsfoss.com )
The blog Its FOSS has 15,000 followers for its Mastodon account — which they think is causing problems:...
Forgejo v7.0 is now available ( forgejo.org )