diff options
| author | ale <ale@incal.net> | 2017-12-19 10:16:08 +0000 |
|---|---|---|
| committer | ale <ale@incal.net> | 2017-12-19 10:16:08 +0000 |
| commit | df800e154f3265f43a3758ac5071caba026ae585 (patch) | |
| tree | 4ddb94ea6ee6c06390860bc31e48f97d9a076caa /README.md | |
| parent | b06b0cd46156918b8a9ab3f60328c22dc60582ef (diff) | |
| download | crawl-df800e154f3265f43a3758ac5071caba026ae585.tar.gz crawl-df800e154f3265f43a3758ac5071caba026ae585.zip | |
Provide better defaults for command-line options
Defaults that are more suitable to real-world site archiving.
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 4 |
1 files changed, 2 insertions, 2 deletions
@@ -34,8 +34,8 @@ The crawling scope is controlled with a set of overlapping checks: prefix is implicitly ignored) * maximum crawling depth can be controlled with the *--depth* option * resources related to a page (CSS, JS, etc) will always be fetched, - even if on external domains, if the *--include-related* option is - specified + even if on external domains, unless the *--exclude-related* option + is specified If the program is interrupted, running it again with the same command line from the same directory will cause it to resume crawling from |