How to configure robots.txt to allow everything?

Question

My robots.txt in Google Webmaster Tools shows the following values:

User-agent: *
Allow: /

What does it mean? I don\'t have enough

Answer 1

I understand that this is fairly old question and has some pretty good answers. But, here is my two cents for the sake of completeness.

As per the official documentation, there are four ways, you can allow complete access for robots to access your site.

Clean:

Specify a global matcher with a disallow segment as mentioned by @unor. So your /robots.txt looks like this.

User-agent: *
Disallow:

The hack:

Create a /robots.txt file with no content in it. Which will default to allow all for all type of Bots.

I don't care way:

Do not create a /robots.txt altogether. Which should yield the exact same results as the above two.

The ugly:

From the robots documentation for meta tags, You can use the following meta tag on all your pages on your site to let the Bots know that these pages are not supposed to be indexed.

In order for this to be applied to your entire site, You will have to add this meta tag for all of your pages. And this tag should strictly be placed under your HEAD tag of the page. More about this meta tag here.