Recently I have been paying closer than usual attention to my apache server logs, and noticed a few new services that are visiting much more frequently than I think they need to. It’s really not necessary to come every 30 minutes, is it? No, it’s not. A spider service should use a more intelligent method of checking a site. For example, maybe only come once per day. Then if the site regularly has multiple postings on a day, increase to checking more frequently. But starting out by checking every 30 minutes is overkill, and bordering on abuse. Another annoying thing is that the spider in question checked my robots.txt file once, and is now reading my feed (and not even my Feedburner feed) every 30 minutes. I’ve emailed them, if I don’t get a response by Friday evening, I’ll simply block their IP address range.