# Bot filtering

> How Datacenter keeps bots, crawlers and automated browsers out of your numbers, why hosting traffic still counts as people, and why they are never billed.

Source: https://datacenter.so/docs/bot-filtering
Section: Start here

A lot of web traffic is not people. Datacenter filters it in a few layers, so your numbers stay close to real visits.

## What is dropped

These events are never written:

- A user agent that names a crawler, a link preview, an uptime monitor, a headless browser or an HTTP library such as `curl`. An empty user agent and a bare `Mozilla/5.0` count as bots too.
- Requests from verified bots, such as search engine crawlers.
- Browsers that report `navigator.webdriver`, which most automation tools do. The script sends nothing from them.
- Browsers that cannot send a beacon. Real browsers today all can.

## What is marked

What gets through is written with a set of flags, so it can be judged again later.

| Flag | Means |
|---|---|
| Unknown user agent | The user agent names no browser and no operating system |
| Bot score | The network's bot score says it is likely automated |
| Hosting network | The request came from a known cloud or hosting network |

The dashboard leaves out events with either of the first two flags.

## Why hosting traffic still shows

Many real people browse through a VPN or a privacy relay. Those often run on cloud and hosting networks. If we dropped every visit from one, we would undercount real people. So a hosting network flag on its own does not remove a visit from your stats.

It does keep it off your bill. Pageviews and clicks from a hosting network never count toward your monthly allowance or extra use, so a script on a cloud server cannot run up what you pay.

## Pageviews need a visible tab

The script waits until the tab has been visible for 1.5 seconds before it counts a pageview. Tabs that open in the background and close unseen are not counted. That also stops many simple bots that load a page and leave at once.