Robots.txt Generator

Robots.txt Generator

All Robots

Crawl Delay

Sitemap URL

Search Engine Robots Override defaults

Google

Google Image

Google Mobile

Bing/MSN

Yahoo

Baidu

Alexa/Archive

Naver

Yahoo MM

Yahoo Blogs

Ask/Teoma

GigaBlast

DMOZ

Nutch

MSN PicSearch

Disallow Folders (with trailing /)

About Robots.txt

What is Robots.txt?

Controls crawler access
Prevents indexing of private pages
Directs to sitemap

Best Practices

Block admin folders
Include sitemap URL
Test before deploying

Credit Card Validator

JavaScript Beautifier

Online Ping Website Tool

Server Status Checker

Search

Advertisement

300*250

Popular Tools

YouTube Thumbnail Downloader

Image Resizer

Image to Base64

Find Facebook ID

Password Generator

Favicon Generator

Word Counter

QR Code Generator

JavaScript Obfuscator & Deobfuscator