summaryrefslogtreecommitdiff
path: root/searx/engines
AgeCommit message (Collapse)Author
2023-10-13[fix] hackernews keyerror problemHackurei
2023-10-12[fix] imgur - incorrect wikidata idHackurei
2023-10-12[fix] ddg-lite & ddg-extra: don't send empty vqd valueMarkus Heiser
DDG's bot detection is sensitive to the vqd value. For some search terms (such as extremely long search terms that are often sent by bots), no vqd value can be determined. If SearXNG cannot determine a vqd value, then no request should go out to DDG (WEB): a request with a wrong vqd value leads to DDG temporarily putting SearXNG's IP on a block list. Requests from IPs in this block list run into timeouts. Not sure, but it seems the block list is a sliding window: to get my IP rid from the bot list I had to cool down my IP for 1h (send no requests from that IP to DDG). Since such issues can't reproduce in a local instance I tested this patch 24h on my public SearXNG instance: There are still errors (rare), but the reliability is still 100%. Related: - https://github.com/searxng/searxng/pull/2922 - https://github.com/searxng/searxng/pull/2923 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-10[fix] ddg-lite vqd value: some search terms do not have a vqd valueMarkus Heiser
Some search terms do not have results and therefore no vqd value BTW: remove a leftover from 9197efa Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-10[fix] duckduckgo lite engine: set HTTP header 'Referer'Markus Heiser
We have had problems with this before, the bot protection from ddg-lite seems to have included this referer in the rating [1][2]. From reverse engineering: - The Referer ``https://google.com/`` was set in commt 257dc7d6c4 --> DDG lite does not like this referer anymore! - The 'Referer' header is only set on second and follow up pages but not on the first page - The vqd value is not needed on the first page, the ddg-lite client sets this value only on follow up pages / this can help to reduce the vqd requests from SearXNG. Related to 'Referer' header & ddg requests: [1] https://github.com/searxng/searxng/pull/2161 [2] https://github.com/searxng/searxng/pull/2081 Closes: https://github.com/searxng/searxng/issues/2796 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-09[mod] yacy: use official instance by default and fix crashesBnyro
2023-10-09[feat] implement hackernews engine - news.ycombinator.comHackurei
2023-10-09[fix] matrixrooms add proper MRS integrationAine
Related: - https://github.com/searxng/searxng/issues/2918
2023-10-09[feat] duckduckgo: support for videos and newsBnyro
2023-10-07[fix] kickass: crash when no resultsBnyro
2023-10-07[mod] piped: always show video length if availableBnyro
2023-10-06[feat] engine: implementation of mastodonBnyro
2023-10-05[mod] engine - simplify region & lang handling, make filters configurableMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-05[feat] engine: implementation of radio-browser.infoBnyro
2023-10-01[typo] solved a typo in yahoo error message.Jinyuan Huang
2023-10-01[fix] Bug: Yahoo results for simplified Chinese search sometimes have the ↵Jinyuan Huang
first character cut off #2866 Co-authored-by: Blair Noctis <n@sail.ng>
2023-10-01[fix] emojipedia: fix engineBnyro
2023-10-01[fix] Revision of the Bing enginesMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-10-01[fix] engine - bing fix search, pagination, remove safesearchjazzzooo
2023-09-30[feat] engine: implementation of pinterestBnyro
2023-09-30[fix] matrixrooms.info: pagination not working properlyBnyro
2023-09-30[fix] engine - moviepilot instead of thumbnail use img_srcMarkus Heiser
Instead of thumbnail use img_src in the result item, otherwise the "movies" categories looks clunky. Related: - https://github.com/searxng/searxng/commit/b4e0d2eedc94df2369a7661f9134fcfd8fd68af4#r128785388 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-30[mod] tagesschau: add option to only use tagesschau urlsBnyro
2023-09-30[feat] engine: implementation of matrixrooms.infoBnyro
2023-09-29[mod] add movies category for tmdb, imdb and moviepilotBnyro
2023-09-28[fix] engine - currency fix and simplifyjazzzooo
2023-09-28[fix] bilibili new api usedJinyuan Huang
2023-09-27[fix] engine - kickass update url, fix parsing, use multiple mirrorsjazzzooo
2023-09-27[fix] engine brave - fetch traits (modified settings menu)Markus Heiser
Brave has changed it settings menu fundamental. Region codes are no longer in the HTML page, we have to read the regional codes from a JS: https://cdn.search.brave.com/serp/v2/_app/immutable/chunks/parameters.734c106a.js Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-27[fix] engine annas archive - fetch traits (modified xpath selectors)Markus Heiser
Anna’s Archive has cleaned up their languages, available file extensions and changed the HTML form. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-24[mod] remove twitter (#2843)Émilien (perso)
2023-09-22[feat] engine: implementation of imgurBnyro
2023-09-22[fix] internet archive scholar: crash when there's no titleBnyro
2023-09-22Revert "[fix] engine - duckduckgo vqd edge-case"Markus Heiser
This reverts commit 102502a4f09e78682cd4f030605be394bc33282c.
2023-09-21[feat] engine: implementation of chefkoch.deBnyro
2023-09-21[fix] engine - brave don't show adsjazzzooo
2023-09-21[fix] engine - google images error when no resultsjazzzooo
2023-09-21[feat] engine: implementation of google icons/material design iconsBnyro
2023-09-21[feat] engine: implementation of wallhavenBnyro
2023-09-20[fix] engine - duckduckgo vqd edge-casejazzzooo
2023-09-19[fix] engine qwant (web-lite) - ignore advertising addsMarkus Heiser
Closes: https://github.com/searxng/searxng/issues/2812 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-19wikipedia wikidata infobox + disable wikisource (#2806)Émilien (perso)
Co-authored-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-18[feat] engine: implementation of internet archive scholarBnyro
2023-09-18[fix] spellingjazzzooo
2023-09-18[fix] engine - qwant delivers only 5 pages maximumMarkus Heiser
all qwant engines (incl qwant-lite) delivers only 5 pages maximum Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-18[fix] engine - qwant wrong error typejazzzooo
2023-09-17[fix] qwant: subsequent fix of commit d9dbcedebMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-17[feat] implementation of qwant lite for web searchMarkus Heiser
Related: https://github.com/searxng/searxng/issues/2719 Replace: https://github.com/searxng/searxng/pull/2748 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2023-09-17[feat] engine: implemenation of moviepilot (de)Bnyro
2023-09-16[fix] tagesschau videosjazzzooo
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion