Difference between revisions of "Robots.txt"

From Cyborg Anthropology
Jump to: navigation, search
(Created page with '===Definition=== "Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web. ===External Links=== http://www…')
 
Line 1: Line 1:
 
===Definition===
 
===Definition===
"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web.
+
"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web.  
  
 
===External Links===
 
===External Links===
Line 6: Line 6:
  
 
[[Category:Book Pages]]
 
[[Category:Book Pages]]
[[Category:Unfinished]]
+
[[Category:Finished]]

Revision as of 00:04, 5 June 2011

Definition

"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web.

External Links

http://www.javascriptkit.com/howto/robots.shtml