| Age | Commit message (Expand) | Author |
|---|---|---|
| 2020-02-17 | Propagate the link tag through redirects | ale |
| 2019-01-20 | Refactor Handlers in terms of a Publisher interface | ale |
| 2018-08-31 | Add a simple test for the full WARC crawler | ale |
| 2018-08-31 | Explicitly delegate retry logic to handlers | ale |
| 2017-12-19 | Add tags (primary/related) to links | ale |
 |
index : crawl | |
| A simple recursive web crawler which stores content in the WARC/1.0 format | Jordan |
| aboutsummaryrefslogtreecommitdiff |
path: root/crawler_test.go