How to add `nofollow, noindex` all pages in robots.txt?

Question

I want to add nofollow and noindex to my site whilst it\'s being built. The client has request I use these rules.

I am aware of

Answer 1

There is a non-standard Noindex field, which Google (and likely no other consumer) supported as experimental feature.

Following the robots.txt specification, you can’t disallow indexing nor following links with robots.txt.

For a site that is still in development, has not been indexed yet, and doesn’t get backlinks from pages which may be crawled, using robots.txt should be sufficient:

# no bot may crawl 
User-agent: *
Disallow: /

If pages from the site are already indexed, and/or if other pages which may be crawled link to it, you have to use noindex, which can not only be specified in the HTML, but also as HTTP header:

X-Robots-Tag: noindex, nofollow