Bot Filter

Overview

AdGlare has a built-in bot filter to detect crawlers and spiders. As we're living in a world where about half of the internet's traffic is caused by bots, it's important to filter them in order to avoid skewing your reports.

Type of Bots

Two types of bots can be detected: GIVT (General Invalid Traffic) and SIVT (Sophisticated Invalid Traffic).

• GIVT Traffic is detected by analyzing the user-agent string. For example, Googlebot, AHrefs Bot and PageSpeed bots all have a user-agent string that shows who they are. By matching the user-agent string against a list of known bots, GIVT traffic can be detected.
• SIVT traffic refers to more advanced forms of invalid traffic. It requires more sophisticated methods to detect and filter such bots, as they intentionally present themselves as human traffic. This may be a warning sign of fraudulent activity. To not make fraudsters wiser than they are, I won't disclose AdGlare's algorithms used to detect SIVT traffic here.

Filtering Bot Traffic

By default, ads will be shown to bots. Their impressions and clicks will be counted. You can generate a Non-Human Traffic report to see the number of events that were logged by bot traffic. If you're a publisher, you can adjust the invoice for your advertiser accordingly by not charging for bot traffic.

Alternative, you can filter bot traffic from your reports as soon as event data is sent to your ad server. Bot traffic still load ads, but their impressions and clicks will simply be discarded before it hits the report cluster. The advantage is that your reports are much cleaner as bot traffic is not included in your metrics.

Disclaimer

Bot detection is a specialized field that requires significant expertise. AdGlare's bot filter represents our best effort to identify and exclude bot traffic from your reports, but it's not guaranteed to be 100% accurate.

For a more sophisticated solution, you may want to try Datadome.co.