1408
you are viewing a single comment's thread
view the rest of the comments
[-] RelativeArea1@sh.itjust.works 160 points 1 week ago* (last edited 1 week ago)

this is some fucking stupid situation, we somewhat got a faster internet and these bots messing each other are hogging the bandwidth.

[-] dual_sport_dork@lemmy.world 15 points 1 week ago* (last edited 1 week ago)

Especially since the solution I cooked up for my site works just fine and took a lot less work. This is simply to identify the incoming requests from these damn bots -- which is not difficult, since they ignore all directives and sanity and try to slam your site with like 200+ requests per second, that makes 'em easy to spot -- and simply IP ban them. This is considerably simpler, and doesn't require an entire nuclear plant powered AI to combat the opposition's nuclear plant powered AI.

In fact, anybody who doesn't exhibit a sane crawl rate gets blocked from my site automatically. For a while, most of them were coming from Russian IP address zones for some reason. These days Amazon is the worst offender, I guess their Rufus AI or whatever the fuck it is tries to pester other retail sites to "learn" about products rather than sticking to its own domain.

Fuck 'em. Route those motherfuckers right to /dev/null.

[-] Flagstaff@programming.dev 4 points 1 week ago

Geez, that's a lot of requests!

[-] dual_sport_dork@lemmy.world 1 points 1 week ago

It sure is. Needless to say, I noticed it happening.

[-] morrowind@lemmy.ml 1 points 1 week ago

Cloudflare offers that too, but you can't always tell

[-] desktop_user 0 points 1 week ago

the only problem with that solution being applied to generic websites is schools and institutions can have many legitimate users from one IP address and many sites don't want a chance to accidentally block one.

[-] dual_sport_dork@lemmy.world 3 points 1 week ago

This is fair in those applications. I only run an ecommerce web site, though, so that doesn't come into play.

[-] ininewcrow@lemmy.ca 9 points 1 week ago

It's what I've been saying about technology for the past decade or two .... we've hit an upper limit to our technological development ... that limit is on individual human greed where small groups of people or massively wealthy people hinder or delay any further development because they're always trying to find ways to make money off it, prevent others from making money off it, monopolize an area or section of society .... capitalism is literally our world's bottleneck and it's being choked off by an oddly shaped gold bar at this point.

[-] drmoose@lemmy.world 3 points 1 week ago

Lol website traffic accounts for like 1% of bandwidth budget. 1 netflix movie is like 20k web pages.

this post was submitted on 21 Mar 2025
1408 points (100.0% liked)

Technology

68187 readers
3259 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related news or articles.
  3. Be excellent to each other!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
  9. Check for duplicates before posting, duplicates may be removed
  10. Accounts 7 days and younger will have their posts automatically removed.

Approved Bots


founded 2 years ago
MODERATORS