Robots.txt

From Cyborg Anthropology
Revision as of 12:33, 1 February 2011 by Caseorganic (Talk | contribs)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Definition

"Robots.txt" is a regular text file that through its name, has special meaning to the majority of "honorable" robots on the web.

External Links

http://www.javascriptkit.com/howto/robots.shtml