Re: searchable index of the web

mkgray@athena.mit.edu
Wed, 30 Jun 93 15:05:45 EDT

I have written a perl script that wanders the WWW collecting URLs, keeping
tracking of where it's been and new hosts that it finds. Eventually,
after hacking up the code to return some slightly more useful information
(currently it just returns URLs), I will produce a searchabe index of this.
There is a complete list of all the sites it has found at

<a href="http://www.mit.edu:8001/afs/sipb/user/mkgray/ht/comprehensive.html">
A complete list of sites found by the W4 (World Wide Web Wanderer)
</a>

I'll announce here when we get this index properly running, however it probably
won't be until sometime in August, as I am going on vacation. Until then...

Matthew Gray
mkgray@athena.mit.edu

Visit the SIPB WWW Plexus Server. URL: http://www.mit.edu:8001/