Robots.txt / sitemap checker
Your browser sends GET requests directly to the entered website, transmitting the URL and connection information to that site. CORS or network errors may prevent reading. Use public URLs only.
How to use this tool
Send direct browser GET requests to robots.txt or sitemap URLs, show HTTP status, and count literal Sitemap: lines or <url> tags.
Read public robots.txt or sitemap HTTP status and simple entry counts.
💡 3 Quick Steps to Get Started
Enter a public HTTP/HTTPS URL, not pasted robots or XML contents.
A site URL reads root robots.txt and sitemap.xml; a file URL is read directly.
Inspect HTTP status. XML counts exact <url> occurrences; robots counts Sitemap: declarations only.
When it helps
A site URL checks same-origin /robots.txt and /sitemap.xml. A robots.txt or .xml URL checks only that exact address.
How it works
The tool uses browser fetch without the GoGo Tools backend. XML counting matches only case-sensitive literal <url> strings, while robots.txt counting matches case-insensitive Sitemap: at the start of a line. It does not parse XML, sitemap indexes, compressed files, or robots directives.
Worked example
Example: entering https://example.com attempts https://example.com/robots.txt and https://example.com/sitemap.xml in sequence.
Limits and data handling
If a cross-origin server does not allow CORS, the browser cannot read the body and the tool shows error text without creating a manual link. There is no custom timeout or cache, and no parsing of sitemap indexes, namespaces, compressed sitemaps, or robots rules.
Your browser sends GET requests directly to the target site, so that site receives a network request from you. The URL is not sent to the GoGo Tools backend, and analytics events do not contain the URL or response body.
Robots.txt / sitemap checker guide
Workflow and limits
Enter a public HTTP/HTTPS URL, not pasted robots or XML contents.
A site URL reads root robots.txt and sitemap.xml; a file URL is read directly.
Inspect HTTP status. XML counts exact <url> occurrences; robots counts Sitemap: declarations only.
Your browser sends GET directly to the site, transmitting the URL and connection information. CORS or network errors may block reading.
No Allow/Disallow rule analysis, XML validation or Google indexing inspection is performed. Crawl blocking does not ensure deindexing; noindex must remain crawlable.
Feedback
How can we improve this tool?
Send a suggestion or bug report for this tool.
FAQ
Robots.txt / sitemap checker: How are target URLs selected?
A general site URL ignores its path and checks same-origin /robots.txt and /sitemap.xml. If the pathname ends in robots.txt or .xml, only the full entered URL is checked.
Robots.txt / sitemap checker: What does the result show?
Each readable response shows its target URL, HTTP status, and a count of Sitemap: lines or literal <url> tags. This text count does not prove that directives, XML, or listed URLs are valid.
Robots.txt / sitemap checker: What CORS and format limits apply?
If a cross-origin server does not allow CORS, the browser cannot read the body and the tool shows error text without creating a manual link. There is no custom timeout or cache, and no parsing of sitemap indexes, namespaces, compressed sitemaps, or robots rules.
Robots.txt / sitemap checker: Where is the entered URL sent?
Your browser sends GET requests directly to the target site, so that site receives a network request from you. The URL is not sent to the GoGo Tools backend, and analytics events do not contain the URL or response body.
