How to no index specific URLS?

Question

I was searching around on how to no index specific URLs but I havent found any specific info on the following.

By adding the below

Answer 1

You can add this rule in a robots.txt file:

Disallow: www.test.com/REF-123*

Answer 2

By using robots.txt, you can disallow crawling.

By using meta-robots (or the HTTP header X-Robots-Tag), you can disallow indexing.

If you intend to forbid indexing, you shouldn’t disallow the URLs in robots.txt, otherwise bots will never know that you don’t want these URLs to be indexed.

In case you want to disallow crawling, you could use this robots.txt:

User-agent: *
Disallow: /REF-123

This would apply to all URLs whose paths start with REF-123 (case-sensitive!).

In case you want to disallow indexing, you could add to all these pages this meta element

<meta name="robots" content="noindex">

or send the corresponding HTTP header X-Robots-Tag:

X-Robots-Tag: noindex