This document describes Scrapy . For development docs, go here.
TODO: introduction
TODO: describe purpose of rules
TODO: describe purpose of extractors & processors
TODO: plenty of examples
TODO: describe crawlspider
TODO: describe class
TODO: describe spider rules
TODO: describe Rules class
TODO: describe extractors purpose
TODO: describe base extractor
TODO: describe sgml extractor
TODO: describe xpath request extractor
TODO: describe request processors
TODO: describe proc
TODO: describe unique
TODO: describe filter domain
TODO: describe filter url
TODO: describe matchers
TODO: describe base matcher
TODO: describe url matcher
TODO: describe UrlListMatcher
TODO: describe url list matcher