When a URI is crawled, a ToeThread will execute a series of processors on it.
The processors are split into 5 distinct chains that are exectued in sequence:
Pre-fetch processing chain
Fetch processing chain
Extractor processing chain
Write/Index processing chain
Post-processing chain
The processors are split into 5 distinct chains that are exectued in sequence:
Pre-fetch processing chain
Fetch processing chain
Extractor processing chain
Write/Index processing chain
Post-processing chain