| Age | Commit message (Collapse) | Author |
|---|
|
In order to do this we have to plumb it through the queue and the
Handler interface, but it should allow fetches of the resources
associated with a page via the IncludeRelatedScope even if it's behind
a redirect.
|
|
Detect write errors (both on the database and to the WARC output) and
abort with an error message.
Also fix a bunch of harmless lint warnings.
|
|
|
|
The native Go implementation of LevelDB.
|
|
The queuing code now performs proper lease accounting, and it will not
return a URL twice if the page load is slow.
|
