Technical · Free tool
Robots.txt checker
See which search engines and AI crawlers a site allows, and which sitemaps it declares.
Free · No sign-up to see your score · We only read public pages
What it checks
Robots.txt Checker: what you get
- Full robots.txt
- Allowed / blocked for Googlebot, Bingbot and 9 AI crawlers
- Declared sitemaps
Real checks, no invented numbers
Everything in the result comes from your live page or from Google’s own APIs. We never make up authority, traffic or ranking figures.
Safe and private
We only read publicly available pages. Results are stored privately and shared only through your report link.
Built by an SEO team
The same checks our team uses for clients in the USA, Canada and Australia.
Frequently asked questions
Does robots.txt stop a page from being indexed?
No — it stops crawling. A blocked URL can still appear in search if other sites link to it. Use a noindex tag to keep a page out of the index.
Want our team to fix it for you?
Get a clear plan and a written quote from the Forelskets seo services team.
