An Intranet with Secure Hosts


You want to index your secure internal web servers, one UNIX and one Windows NT, both of which host CGI scripts to dynamically create web pages. You also want complete logging of all messages.

vspider -cmdfile /verity/vspider/intra.cmd

where intra.cmd consists of:

-collection icd.coll
-start http://sigma.verity.com:8015 -start http://colt.verity.com
-domain verity.com -cgiok
-auth /usr/secure/intranet/authfile.txt
-casesen -indexers 5 -maxindmem 2048 -submitsize 2000 -trace

Case-specific Options

Option
Reason
-auth
In order to gain access to the secured paths on your web servers, you must supply login information. The can read and then pass this information from an authentication file.
-cgiok
You must explicitly enable the 's ability to parse documents served by CGI scripts.
-trace
You must explicitly state a higher logging level than the default verbose, and in this case the most complete level is trace.

Unnecessary Options for this Case

Option
Reason
-nofollow, -unlimited
You want to index links, but only on those servers which are within your domain. Therefore -nofollow cannot be used to stop from following links, and -unlimited would undermine your desire to only follow links to hosts within your domain.
-abspath, -prefixmap
These options only affect indexing file systems.





Copyright © 1998, Verity, Inc. All rights reserved.