Robots.txt for multiple domains

Question

We have different domains for each language

1. www.abc.com
2. www.abc.se
3. www.abc.de

And then we have different sitemap.xml for each

Answer 1

I'm using the following solution in .htaccess after all domain redirects and www to non-www redirection.

# Rewrite URL for robots.txt
RewriteRule ^robots\.txt$ robots/%{HTTP_HOST}.txt [L]

Create a new directory in your root called robots. Create a text file filled with the specific robots information for every domain.

• /robots/abc.com.txt
• /robots/abc.se.txt
• /robots/abc.de.txt