| Age | Commit message (Collapse) | Author | |
|---|---|---|---|
| 2022-02-14 | client, crawl: fix/simplify net.Dialer overrides | Jordan | |
| 2022-02-10 | client, crawl: --bind, support making outbound requests from a particular ↵ | Jordan | |
| address | |||
| 2020-08-23 | Allow setting DNS overrides using the --resolve option | ale | |
| 2018-08-31 | Improve error checking | ale | |
| Detect write errors (both on the database and to the WARC output) and abort with an error message. Also fix a bunch of harmless lint warnings. | |||
| 2017-12-19 | Use a global http.Client with sane settings | ale | |
 |
index : crawl | |
| A simple recursive web crawler which stores content in the WARC/1.0 format | Jordan |
| aboutsummaryrefslogtreecommitdiff |