AdsCrawlr

What this crawler does

AdsCrawlr makes a small number of ordinary GET requests to publicly available resources on a website — primarily its /ads.txt file and its homepage — to understand how the site monetizes advertising. We use this to identify publishers who might benefit from additional advertising demand, for business-to-business outreach.

It does not log in, submit forms, access private areas, or attempt anything beyond fetching public pages. It is not an attack, a vulnerability scan, or a content scraper.

How to identify it

Every request carries this User-Agent:

Mozilla/5.0 (compatible; AdsCrawlr/1.0; +https://crawler.adscrawlr.com)

How we behave

• We honor robots.txt for page fetches.
• Read-only: only GET requests to public URLs.
• Low volume and rate-limited — we do not hammer sites.
• We skip government, military, banking and other sensitive infrastructure.

Opt out

To stop AdsCrawlr from visiting your site, add this to your robots.txt — it takes effect on our next visit:

User-agent: AdsCrawlr
Disallow: /

Prefer email, or have a question or complaint? Contact abuse@adnation.io and we will remove your domain promptly.