source:
ruby/trunk/terminology/public/robots.txt@
209
Last change on this file since 209 was 164, checked in by , 15 years ago | |
---|---|
File size: 204 bytes |
Rev | Line | |
---|---|---|
[164] | 1 | # See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file |
2 | # | |
3 | # To ban all spiders from the entire site uncomment the next two lines: | |
4 | # User-Agent: * | |
5 | # Disallow: / |
Note:
See TracBrowser
for help on using the repository browser.