It’s not new that publishers often complaint about their websites, ads, and campaigns not performing up-to-the-mark. They feel ‘non-human traffic’ AKA ‘bot traffic’ is one of the reasons. About 78% publishers report bot traffic on their sites, yet only 38.4% purchase traffic.
Despite publishers know the reason, the problem still continues to persist as they don’t know why and how bot traffic has been plaguing their efforts. So at the end, where does this unawareness lead to?
Revenue at risk…
Bot traffic or non-human traffic is yet another addition to the entire ad fraud nexus. Hence, it’s high time publishers start making an effort to understand:
- what bot traffic is
- what does it want
- where does it come from
- what will it take
What is Bot Traffic?
The term ‘bot + traffic’ per se makes it clear to understand一the visitors coming to your website who are not even human. Whether yours is a big, popular website or a new one, a certain percentage of bots will pay you a visit. Whether you know it or not…
Internet bots or web robots are automated to visit premium websites; and appear as targetable humans (audience). Some bots perform repetitive tasks like copying, ad clicking, posting comments, or anything malware-causing.
Data has it that almost 29% of a website traffic is bot traffic. Which also means 29% budget is being spent on processing artificial pageviews/ad clicks; eventually leading to high (poor) bounce rate (more coming up on this).
An acceptable bounce rate of a website ranges from 45-65%. Normally, such a figure would appear too unimpressive. However, publishers, advertisers, and marketers have become accustomed to this range of bounce rate. Why?
Website owners know every bit of their traffic cannot be real. Holistically, almost 50% of the web traffic is bot traffic. In 2016, bot traffic accounted for 51.8% web traffic. That’s insane and mind-wrecking.
Six Types of Bots to Watch Out From
These bots make fraudulent ad clicks. This is the most threatening bot type for web publishers; especially if you follow the PPC model. Consequently, the analytics data gets skewed and budget gets eroded.
These bots also tamper with the analytics-generated user engagement data. However, instead of ad click count, they add to fake download count. In case where a free ebook download is your end conversion, these bots are likely to mess up your conversion data.
This is the most common bot type which disrupts user engagement with distribution of unwarranted content viz. spam comments, phishing emails, ads, unusual website redirects, negative SEO against competitors, etc.
Perhaps the most despised type, these bots mine individual or business data. They steal information like email address from websites, forums, chat-rooms, and others.
These bots visit a website with malicious intent一stealing your content. They are made by third-party scrapers who are employed by competitors in order to steal content, product catalog, and prices. The stolen content is then repurposed to publish elsewhere.
These bots appear as genuine visitors who intent to bypass online security measures. It’s mostly these bots who are responsible for attacks like distributed denial of service. They’re also the ones who inject spyware on your site or appear as fake search engines.
FYI, not all bots are bad. The good one are created to perform operational tasks like old data scraping, content hygiene, data capturing, etc. Some good bots are backlink checker bots, monitoring bots, social network bots, feedfetcher bots, search engine crawler bots.
How to Identify Bot Traffic?
Bot traffic is hitting websites every hour. Even while you’re here reading about it. In the beginning of this article, I brought up this point about publishers not understanding why and how bot traffic affects their efforts. Also that they don’t know how to deal with it. So let’s begin with understanding now.
#1 Recheck Page Load Speed
You may have conducted this test a week ago. As we know, these test results look a tad different after every short interval. But, the next time when you conduct a page-load speed test and notice a considerable fall (without any major changes that happened to your site), chances are you’ve been hit by bot traffic.
There could be many reasons of a slow loading site. However, in case of detecting bot traffic, checking the page-load speed is the first step. It’s possible that a whole lot of bots are together trying to strain your servers and take them offline.
#2 Keep Tab On Certain Metrics
In you notice a sudden rise in your traffic count and bounce rate at the same time, your site is probably being visited by bot traffic. Here, high traffic count means a high number of bots or high frequency of same bots coming to your site again and again.
And high bounce rate means non-human traffic who visit for no purpose and just leave without exploring more webpages. A suddenly changed session-duration behavior also indicates bot traffic.
Let’s say your site usually serves long-form content, hence your average session duration lies between two to five minutes. However, if you see an expected dip, bot traffic could be the reason. Alongside these, there are also other common metrics you should keep looking at.
#3 Verify Traffic Sources And IP Address
Not just metrics, even some data sources can act as buzzers for bot traffic. Regular and high number of visits from same IP addresses emphasize the fact you’re getting bot traffic. Tools like Deep Log Analyzer can help dig into endless raw server logs and blacklist the offending IPs.
Odd traffic sources are next. Suppose most of your traffic comes from US region and Asian countries. A sudden addition in traffic coming via Arab (non-english) countries may be one of the indications.
All of this can be checked using website analytics tools like Google Analytics. If you’re new to GA, it’s advised to get acquainted with the platform first, and then move ahead with understanding the use cases with bot traffic identification.
#4 Test For Content Duplication
Your content is the heart of your website. And with the invasion of bots, it might be at risk. To detect bot traffic, keep checking for duplicate content to ensure no scraper bots have visited your site and stolen from you.
Tools/platforms like SiteLiner, Duplichecker, CopyScape are handy to use and find if your content is repurposed and used elsewhere.
How to Stop Bot Traffic?
Detecting bot traffic once, immediately calls for stopping bot traffic once and for all. Bots are like viruses hitting your website, skewing your set systems, stealing your data, and more. But thankfully, there are methods which can help you shield against it. Here you go:
- Buy traffic only from known sources. To ensure purchased yet safe traffic, many publishers practice traffic arbitrage to ensure high yielding PPC/CPM based campaigns.
- Place robots.txt to keep selective bots away from crawling your web pages. Publishers might also want to ensure the crawler settings to prevent troubles in AdSense ads.
- Execute distributed denial of service (DDOS). Publishers having a list of offensive IP addresses leverage DDOS protection to deny those visit requests on their website.
- Add CAPTCHA on sign-up or download forms. Many publishers and premium websites place CAPTCHA to prevent download or spam bots.
- Examine server error log files. As bots try to overrun servers, thoroughly examining the server error logs helps find and fix website errors caused by bots.
How to Filter Out Bot Traffic From Google Analytics?
As understood by now, bot traffic really messes up with data. Also, as per the above listed methods, stopping bot traffic requires timely attention. But thankfully, there’s a simple way to filter out bot traffic from Google Analytics to at least prevent the data damage.
Here are the steps which won’t even take 30 seconds:
- Visit Google Analytics Admin Panel.
- Navigate to View Settings in the View tab.
- Scroll slightly downward to spot Bot Filtering checkbox.
- Apply Check in the check box , if unchecked.
- Hit Save.
Just so you know, this filtration of bot traffic out from your Google Analytics a/c ensures all types of recognized bots (the ones mentioned above) steer clear of your data. However, the method may not be able to debar unidentified or a new type of bots.
Why Not Ignore Bot Traffic?
Not making this up. Right on the day I was working on this article, the very next day I encountered with Spam Bots (Spam comments on my blog). Reading or writing about bot traffic, or spam bots in particular, did not seem scary until I saw this:
My WordPress showed I’ve received seven spam comments. The treatment to this is ‘mark these comments as spam.’ Now notice how the first three comments are received on the same date. You’ll also notice this date is very near to the date of publish of this article.
After this perfectly-timed incident, I really wish you (publishers) understand how bot traffic affects your overall efforts. And you should start caring about it because your:
- site and ads are also being hurt by ad fraud disguised as bot traffic.
- precious data/analytics is getting skewed, despite you knowing about it.
- website load time and overall performance might be deteriorating.
- website is getting vulnerable to botnets, DDOS attacks, and bad SEO.
- CPC and revenue are getting severely affected by fake clicks.
Surprisingly, only 5% publishers work with a dedicated fraud-patrol professional. This means, publishers are getting affected by ad fraud methods (bot traffic being one), but they’re doing pretty much nothing about it.
Why? Do they lack knowledge on how to stop bot traffic? They don’t have the bandwidth or expertise to execute the required steps? Or maybe they just don’t bother…
Whatever may be the case for publishers, the growing intervention of bot traffic is calling for prevention methods to be applied immediately, and hence save efforts and ROI.