MelessaLogue353

I have a couple of web sites, and just not too long ago (this past month), I've noticed that one particular of them is using what I would think about excessive bandwidth. Digging a tiny deeper, it appears that bots are employing all the bandwidth. This is just my personal internet site ? I have a travel weblog in the principal domain and a loved ones history internet site in a subdomain. In the past, I have employed on typical 400mb per month of bandwidth. This month I've had to boost the bandwidth to 1.5GB, but it is almost certainly going to go over (it's at present at 1.38GB). These two websites are not huge, and do not get stacks of hits, mostly just friends and family members. The household history website has a few huge images, but absolutely nothing excessive. Searching at awstats this month, the weblog internet site has used 150mb of bandwidth + 480mb of bot bandwidth (380mb of that is msnbot). The family history web site has used 55mb of bandwidth + 650mb of bot bandwidth (620mb of that is googlebot). Most robots identify themselves by a custom user agent in the request headers. Which can simply be blocked with htaccess. There are a number of great articles on this. Let me know if you have any troubles, as it really is a matter of identifying the offending bots/crawlers and banning them as per your need to have. I have employed google tools to tell it to not frequent the internet site as significantly about a week ago, but it does not seem to have produced a various. Brief of telling the robots to bugger off completely via the robots.txt file (it is just my private web site, but it is nonetheless nice to be listed in google!), is there anything else I can do? I presently use this on all my sites, essentially it blocks all bad user agents , negative bots and scrappers, Not only can it save your content material from becoming mass harvested but will also save you a tiny bandwidth because of less bots operating about your website. Hope it helps can tell you that Google drags in a fantastic a lot of spiders due to advertizing, particularly if you are making use of Adsense on your web site along with ising the a variety of ads from the Google ad network partners - these partners also send their bots to test your traffic sources and what adverts to place on your internet site - Google has been hitting tough lately due to the fact of the algorithm tweaks and Adsense getting had a lucrative month in the terms of the amount of new advertisers on board. Even if you slow down the crawl rate, you will still see a large chunk of bandwidth disappear. The bots are way also intermittent to make accurate adjustments unless you wish to block them. There's a new wordpress plugin that can aid with this! I've gotten a couple of emails re a item becoming sold to get rid of or "spank" the bad spiders that are taking up lots of bandwidth and not adding value to your business, freeing up space for real visitors and not causing a difficulty with hosting limits. It's named Spyder spanker at that name .com if you want to see the sales page. Anyway, I am not positive if this is some thing valuable that I need to have or not. I do see a lot of spider activity in my stats, but I constantly believed that was sort of good b/c it means they are crawling my internet sites and hopefully indexing them. The big danger is stealing your bandwidth. Some of the spiders sent by spammers will hammer your website as fast as they can, slowing down response for your human guests. filter spiders wordpress