AI crawlers run up a website's hosting bill

[Brainspore]

@jlw

Web crawlers for search have long used a surprising amount of bandwidth

There are a lot more web crawlers these days than there used to be.