|
Attention Webmasters: ALL listings are available for spidering. If you do not wish for your
site to be spidered then it will be necessary to write a
robots.txt file for the
site. Or you can include meta tags in each page between the <head> ... </head> lines:
<meta name="Robots" content="noindex"> To exclude page from being spidered
|