Webbot

2007年2月1日 (四) 00:17的版本

Webbot - the Libwww Robot

The webbot is a very fast Web walker with support for regular expressions, SQL logging facilities, and many other features. The webbot comes with the libwww codebase. It can be used to check links, find bad HTML, map out a web site, download images, etc.

http://www.w3.org/Robot/