This Server, UGH

[Easy 6]

This is not a finger pointing thread, DaFace has been clear that he is hamstrung when this happens... but isn't it just weird AF that it never happens on some random Tuesday?

Nope, its ALWAYS gotta be a Saturday or Sunday

It's simply uncanny...

[DaFace]

Originally Posted by ChiefsCountry

Still get this when I login

I think I found the issue with that one. Let me know if you still see the error.

[GloryDayz]

Originally Posted by DaFace

If you're curious, it turns out that "m.chiefsplanet.com" still works for the old mobile theme.

Yikes, but still more colorful than we used to have to enter data into torpedoes and missiles back in the day.

[suzzer99]

https://jan.wildeboer.net/2025/04/We...Botnet-Part-2/

Botnet Part 2: The Web is Broken

I guess you have all heard about the growing problem of AI companies trying to aggressively collect whatever data they can get their hands on to train their models. This has caused an explosive surge in web crawlers relentlessly hitting servers big and small. But who runs these crawlers? Turns out — it could be you!

1. Those stealthy botnets - How I found out about a not so new class of botnets
2. The Web is Broken - Certain companies recruit app developers to create botnets by injecting “network sharing” SDKs into their apps. These botnets then use the network bandwidth of unsuspecting users of said apps to crawl the web, brute-force mail servers and other nasty things.

So there is a (IMHO) shady market out there that gives app developers on iOS, Android, MacOS and Windows money for including a library into their apps that sells users network bandwidth. Infatica1 is just one example, there are many more.

I am 99% sure that these companies cause what effectively are DDoS attacks by aggressive AI crawlers that many webmasters have to deal with since months. This business model should simply not exist. Apple, Microsoft and Google should act.



What these companies then sell to their customers is network access through the devices/PCs that have an app with this SDK installed2. They are proud to tell you how you can funnel your (AI) web scraping etc through millions of rotating, residential and mobile IP addresses. Exactly the pattern we see hitting our servers.





There are many
Now, again, this company is just one of many selling similar services. And they all promise that they carefully check what commands their customers send to the (IMHO) infected apps on your phone and PC. Yeah, I am sure they “do no evil”. And when they do, they can claim it’s not their problem because they are merely the proxy. Again, IMHO, a shady business model.

Trend Micro did some research on these companies back in 2023 and it confirms my suspicions. And I guess with AI scraping this kind of business is booming.

But IMHO (In My Humble Opinion) this also explains the explosion of bot traffic that really cripples a lot of smaller services (like my forgejo instance, that I had to make non-public).

So if you as an app developer include such a 3rd party SDK in your app to make some money — you are part of the problem and I think you should be held responsible for delivering malware to your users, making them botnet members.

Unfortunately it is next to impossible for normal users to detect the inclusion of such shady SDKs and the network traffic they cause. Not even mentioning how hard this is for admins of (small) web servers.

I already blogged about this back in February 2025 but I think it is better to put what I have learned since then in this new post. I guess it won’t be my last on this topic.

See for yourself!

If you want to feel really dirty, go to https://proxyway.com/reviews?e-filte...roxy-providers for a collection of reviews on these services. It’s a huge market and I am 100% convinced that “AI” web scraping is currently the biggest “growth” driver for these companies.

And when I see that quite some of them rely on injecting SDKs into 3rd party apps to “extend” their “Reach” and fill their pools of “residential proxies”, I would call out these companies for distributing malware and creating botnets. But that’s just my personal opinion. I am sure they are all legit.



My conclusion

I am now of the opinion that every form of web-scraping should be considered abusive behaviour and web servers should block all of them. If you think your web-scraping is acceptable behaviour, you can thank these shady companies and the “AI” hype for moving you to the bad corner.

Thank you for your time and interest! I hope it helps you understand why web crawlers have become a real problem and how this is more and more an attack on the foundation of the Web as it was intended to be. This “residential proxy” business is just one part of this. And we webmasters/admins can only try to block. It is getting more and more difficult to keep up with these waves. Thanks “AI”, I guess?