diff options
| author | ale <ale@incal.net> | 2019-01-02 09:53:42 +0000 |
|---|---|---|
| committer | ale <ale@incal.net> | 2019-01-02 09:53:42 +0000 |
| commit | c5ec7eb826bfd08aa6e8dd880efa15930f78ba19 (patch) | |
| tree | 7c7d5fcfc55922cf78a97001b7ca4b879b747d28 /crawler.go | |
| parent | 3518feaf05fcb7f745975851c6684a63532ff19a (diff) | |
| download | crawl-c5ec7eb826bfd08aa6e8dd880efa15930f78ba19.tar.gz crawl-c5ec7eb826bfd08aa6e8dd880efa15930f78ba19.zip | |
Add multi-file output
The output stage can now write to size-limited, rotating WARC files
using a user-specified pattern, so that output files are always
unique.
Diffstat (limited to 'crawler.go')
0 files changed, 0 insertions, 0 deletions