What is Robots.txt?

thank you buddy

 

Robots.txt is the tool to carwl all weblinks and reach faster in all search engines.

 

Robots.txt file is a text file which is uploaded on the root of the website. The main task of this file is to instruct search engine crawler that which site they have to visit and which are not.

 

Robots.txt is the text file tool to crawl all web pages faster to index them for all top search engines

 

It is great when search engines frequently visit your site and index your content but often there are cases when indexing parts of your online content is not what you want. For instance, if you have two versions of a page (one for viewing in the browser and one for printing), you'd rather have the printing version excluded from crawling, otherwise you risk being imposed a duplicate content penalty.

 

Robots.txt to identify and analysis the websites to crawl faster in reach all search engines.

 

Technically perfect answer amisha. Answer says a lot words in just two lines.

 

It helps to restrict the particular web pages from crawling of search engines.

 

we make use of the robots text file to tell the search engine about which page we dont want to crawl by the search engine bots.

 

Robots.txt means it states that which file or a folder should be viewed or which should not.

 

Robot.txt is used in on page so as to hide some coding from the google bots....

 

Robot text are used by the programmers a part of coding from the bots of google when the content is crawled by search engine bots

 

Robots.txt is a text file which we upload on our website to tell the various search engines about which part of our website we dont to crawl or index.
It takes the following format:
User Agent: Search engine spider
Disallow: link of our website

 

what is robots.txt?
>It is an html tag placed on the source of a webpage which directs search engine spiders which files to crawl on or not.