Package: spider

A Web spidering library: handles robots.txt, scraping, finding more links, and doing it all over again.

Available downloads

Most recent version: spider-0.5.4.tar.gz

Usage in debian/watch file

Use the following in your debian/watch file:

version=4
https://gemwatch.debian.net/spider .*/spider-(.*).tar.gz