| Mode | Name | Size | |
|---|---|---|---|
| -rw-r--r-- | .gitlab-ci.yml | 221 | logplain |
| -rw-r--r-- | LICENSE | 1101 | logplain |
| -rw-r--r-- | Makefile | 345 | logplain |
| -rw-r--r-- | README.md | 3875 | logplain |
| d--------- | analysis | 77 | logplain |
| -rw-r--r-- | client.go | 1946 | logplain |
| d--------- | cmd | 64 | logplain |
| -rw-r--r-- | crawler.go | 10577 | logplain |
| -rw-r--r-- | crawler_test.go | 1283 | logplain |
| d--------- | debian | 139 | logplain |
| -rwxr-xr-x | gen-ignores.py | 1095 | logplain |
| -rw-r--r-- | go.mod | 444 | logplain |
| -rw-r--r-- | go.sum | 6683 | logplain |
| -rw-r--r-- | ignore_patterns.go | 45040 | logplain |
| -rw-r--r-- | queue.go | 3189 | logplain |
| -rw-r--r-- | renovate.json | 107 | logplain |
| -rw-r--r-- | scope.go | 3981 | logplain |
| -rw-r--r-- | scope_test.go | 1709 | logplain |
| d--------- | vendor | 113 | logplain |
| d--------- | warc | 111 | logplain |
 |
index : crawl | |
| A simple recursive web crawler which stores content in the WARC/1.0 format | Jordan |
| aboutsummaryrefslogtreecommitdiff |