Precedence of X-Robots-Tag header vs robots meta tag

Question

I\'ve placed the following Header in my vhost config:

Header set X-Robots-Tag \"noindex, nofollow\"

The goal here is to just disab

Answer 1

I am not sure if a definitive answer can be given to the question, as the behavior may be implementation-dependent (on the robot side).

However, I think there is reasonable evidence that X-Robots-Tag will take precedence over . See :

One significant difference between the X-Robots-Tag and the robots meta directive is:

• X-Robots-Tag is part of the HTTP protocol header.
• is part of the HTML document header.

Therefore the the X-Robots-Tag belongs to HTTP protocol layer, while belongs to the HTML protocol layer.

As they belong to a different protocol layer, they will not be parsed simultaneously by the (robot) client getting the page: The HTTP layer will be parsed first, and the HTML in a later step.

(Also, it should be noted that X-Robots-Tag and are not suppported by all robots. Google and Yahoo/Bing suppport both, but according to this some support only , others support neither.)

Summary :

• if supported by the robot, X-Robots-Tag will be processed first ; restrictions (noindex, nofollow) apply (and is ignored).
• else, directive applies.