What Is The X-robots Tag?

The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used in an robots meta tag can also be specified as an X-Robots-Tag. Here's an example of an HTTP response with an X-Robots-Tag instructing crawlers not to index a page.
 
WebGenius said:
The X-Robots-Tag can be used as an element of the HTTP header response for a given URL. Any directive that can be used in an robots meta tag can also be specified as an X-Robots-Tag. Here's an example of an HTTP response with an X-Robots-Tag instructing crawlers not to index a page.
Click to expand...


Ok, Please share some sample code, Thanks.......
 
Sample code: robots meta tag

HTML: 
<!DOCTYPE html>
<html><head>
<meta name="robots" content="noindex" />
(…)
</head>
<body>(…)</body>
</html>
 
First, let me be clear that if you are using above robots.txt and robots meta tag for blocking your content, then there is no need to use this one. TheX-Robots-Tagcan be used as an element of the header response for a given URL of the web page
 
The X-Robots-Tag can be used as an element of the HTTP header response for a given URL of the web page. Benefits of Using it:
  • You can use it where the usage of robots meta tags is not possible, for example: non-html files like Image, Video, and Flash can be blocked.
  • You can add the X-Robots-Tag to a site’s HTTP responses using .htaccess and httpd.conf files.
  • It is Global, which means an X-Robots-Tag with HTTP responses allows you to specify crawling directives that are applied globally across a site.
  • You can use regular expressions for high quality flexibility for complex url or content.
  • Like Robots Meta Tag, you can use other attributes for your page like nosnippet, noodp, notranslate etc.
 
You must log in or register to reply here.
Back
Top