Please specify URL to crawl. (example)

About

Quick robots.txt checker to test which robots are disallowed or allowed to index/archive contents. recognizes robots.txt and <meta> tag specified by Google.

Bookmarklet: test robots.txt

Parmalink: http://forbiddenrobots.usb0.net/