diff options
| author | ale <ale@incal.net> | 2020-02-17 21:39:06 +0000 |
|---|---|---|
| committer | ale <ale@incal.net> | 2020-02-17 21:40:29 +0000 |
| commit | 533f472553d6db42a1ae704285e33f53cf90f81d (patch) | |
| tree | 122c472cc685e567d25794357c90ff92b7165b1c /crawler_test.go | |
| parent | fec78595f9986cb908ef1ff61cfb3a5828986456 (diff) | |
| download | crawl-533f472553d6db42a1ae704285e33f53cf90f81d.tar.gz crawl-533f472553d6db42a1ae704285e33f53cf90f81d.zip | |
Propagate the link tag through redirects
In order to do this we have to plumb it through the queue and the
Handler interface, but it should allow fetches of the resources
associated with a page via the IncludeRelatedScope even if it's behind
a redirect.
Diffstat (limited to 'crawler_test.go')
| -rw-r--r-- | crawler_test.go | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/crawler_test.go b/crawler_test.go index 0ad469b..fa81c2f 100644 --- a/crawler_test.go +++ b/crawler_test.go @@ -33,7 +33,7 @@ func TestCrawler(t *testing.T) { ) var crawledPages int - h := HandlerFunc(func(p Publisher, u string, depth int, resp *http.Response, err error) error { + h := HandlerFunc(func(p Publisher, u string, tag, depth int, resp *http.Response, err error) error { crawledPages++ next := fmt.Sprintf(srv.URL+"/page/%d", crawledPages) log.Printf("%s -> %s", u, next) |