WWW robots should be identified
Steve Putz (putz@parc.xerox.com)
Thu, 15 Jul 1993 11:01:31 PDT
In the interim while we are trying to to figure out how automatic indexing of the web should work, if anyone is running any kind of robot for walking the web (i.e. http servers), please publish your robot's IP address to www-talk and comp.infosystems.www. That way I can distinguish these accesses when extracting statistics from my http server log files.
Thanks,
Steve Putz
Xerox PARC
<http://pubweb.parc.xerox.com:80/hypertext/people/StevePutz.html>