| Age | Commit message (Collapse) | Author |
|---|
|
Add a search engine for core.ac.uk
|
|
It is the whole engine definition that should be comment out : without the
api_key nothing works.
Signed-off-by: Markus Heiser <markus@darmarit.de>
|
|
- add to list of pylint scripts
- add debug log messages
- move API key int `settings.yml`
- improved readability
- add some metadata to results
Signed-off-by: Markus Heiser <markus@darmarit.de>
|
|
|
|
|
|
|
|
Replace Makefile boilerplate by shell scripts
|
|
|
|
[mod] update /stats
|
|
|
|
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
|
|
[CI] JamesIves/github-pages-deploy-action COMMIT_MESSAGE
|
|
remove rocket symbol from the commit message
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Suggested-by: https://github.com/JamesIves/github-pages-deploy-action/pull/576
|
|
[CI] JamesIves/github-pages-deploy-action SINGLE_COMMIT
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Renamed 'test.sh' to 'test.shell' seems to solve this issue on my ubu204::
$ make test.sh
...
TEST robot
...
INSTALL gecko.driver
...
Running 7 tests
cat test.sh >test
cat: test.sh: Datei oder Verzeichnis nicht gefunden
make: *** [<eingebaut>: test] Fehler 1
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
ERROR: When pyenv build fails, the pyenv command runs into a recursion [1]:
....
BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV ERROR: Cannot install idna==3.1 and requests[socks]==2.25.1 because these package versions have conflicting dependencies.
BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV
BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV The conflict is caused by:
[1] https://github.com/return42/searx-next/pull/4#issuecomment-821999497
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Replaces the make targets with the bash scripts
Signed-off-by: Markus Heiser <markus@darmarit.de>
|
|
note: in further patches script 'manage' will replace 'manage.sh'
pyenv.* :
assert : build virtualenv if not exists
intsall : developer install of searx into virtualenv
unintsall : uninstall developer installation
cmd ... : run command ... in virtualenv
OK : test if virtualenv is OK
pypi.upload:
Upload python packages to PyPi (to test use pypi.upload.test)
pybuild :
Build python packages at ./${PYDIST}
pyclean :
delete virtualenv and intermediate py files
test.* :
pylint : lint PYLINT_FILES, searx/engines, searx & tests
pep8 : pycodestyle (pep8) for all files except PYLINT_FILES
unit : run unit tests
coverage : run unit tests with coverage
robot : run robot test
clean : clean intermediate test stuff
node.* :
env : download & install npm dependencies locally
clean : drop npm installations
buildenv :
rebuild ./utils/brand.env
data.* :
all : update searx/languages.py and ./data/*
languages : update searx/data/engines_languages.json & searx/languages.py
useragents: update searx/data/useragents.json with the most recent versions of Firefox.
themes.* :
all : build all themes
oscar : build oscar theme
simple : build simple theme
bootstrap : less compile bootstrap.min.css CSS
babel.compile :
pybabel compile ./searx/translations
docs.* :
html : build HTML documentation
gh-pages : deploy on gh-pages branch
autobuild : autobuild HTML documentation while editing
prebuild : build reST include files (./${DOCS_BUILD}/includes)
clean : clean documentation build
docker.build [push] :
build (and push) docker image
gecko.driver :
download & install geckodriver if not already installed (required for
robot_tests)
Signed-off-by: Markus Heiser <markus@darmarit.de>
|
|
Error::
$ utils/lxc.sh --help
ERROR: missing command lxc
...
This breaks also docs build::
$ make docs
...
WARNING: Unexpected return code 42 from command '../utils/lxc.sh --help'
...
Signed-off-by: Markus Heiser <markus@darmarit.de>
|
|
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Implement a boilerplate to manage performance optimized virtualenv builds.
Shell scripts can use (e.g.) 'pyenv.cmd' to execute command in the virtualenv
without having to worry about whether and how the environment is provided. ::
pyenv.cmd which python
..../local/py3/bin/python
pyenv.cmd which pip
..../local/py3/bin/pip
If pyenv.cmd released multiple times the installation will only rebuild if the
function 'pyenv.OK' fails. Function 'pyenv.OK' make some test to validate that
the virtualenv exists and works as expected. The check also fails if
requirements listed requirements-dev.txt and requirements.txt has been edited.
Among these tests 'pyenv.OK' calls 'pyenv.check' which implements a python
script that validate the python installation. Here is an example how a
'pyenv.check' implementation could look like::
pyenv.check() {
cat <<EOF
import yaml
print('import yaml --> OK')
EOF
}
Signed-off-by: Markus Heiser <markus@darmarit.de>
|
|
Fix issues reported by `make test.sh`
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
|
|
|
|
searxng brand
|
|
|
|
Add a search engine for Springer Nature
|
|
[enh] add engine using API from "The Art Institute of Chicago"
|
|
Signed-off-by: Markus Heiser <markus@darmarit.de>
|
|
Springer Nature is a global publisher dedicated to providing service to research
community [1] with official API [2].
To test this PR, first get your API key following this page:
https://dev.springernature.com/signup
In searx/engines/springer.py at line 24, add this API key. I left my own key,
commented out in the line aboce. Feel free to use it, if needed.
[1] https://www.springernature.com/
[2] https://dev.springernature.com/
|
|
|
|
[fix] youtube - send CONSENT Cookie to not be redirected
|
|
[fix] Fix typo in morty installation instruction
|
|
[enh] add DOI resolver from sci-hub / replace default DOI
|
|
|
|
The new sci-hub URLs are comming from @aurora-vasiliev [1].
[1] https://github.com/searx/searx/pull/2706
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
In the EU there exists a "General Data Protection Regulation" [1] aka GDPR (BTW:
very user friendly!) which requires consent to tracking. To get the consent
from the user, youtube requests are redirected to confirm and get a CONSENT
Cookie from https://consent.youtube.com
This patch adds a CONSENT Cookie to the youtube request to avoid redirection.
[1] https://en.wikipedia.org/wiki/General_Data_Protection_Regulation
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
Reported-by: https://github.com/searx/searx/issues/2774
|
|
[drop] plugin HTTPS rewrite - rule base is to old (update is to big)
|
|
The usefulness of the _HTTPS rewrite_ plugin is questionable:
- the 36 rule files have not been updated since 2015 [1]
- actual there are 23760 rule files in the https-everywhere repo [2]
For the first, we can remove this plugin. For a complete new implementation, it
might be good to know that there is a project "https-everywhere : Privacy for
Pythons" [3]
related: https://github.com/return42/searx-next/issues/8
[1] https://github.com/return42/searx-next/tree/d187a1d/searx/plugins/https_rules
[2] https://github.com/EFForg/https-everywhere/tree/master/src/chrome/content/rules
[3] https://github.com/jayvdb/https-everywhere-py
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
|
|
Metrics
|
|
|
|
|
|
* display the median time instead of the average.
* add a "Reliability" column (sum up the metrics and the checker results).
* the "selected language", "SafeSearch", "Time range" values are displayed as "broken" when the checker tests fail.
|
|
Some error won't stop the engine:
* additional HTTP redirects for example
* some invalid results
secondary=True allows to flag these errors as not important.
|
|
|
|
Report to the user suspended engines.
searx.search.processor.abstract:
* manages suspend time (per network).
* reports suspended time to the ResultContainer (method extend_container_if_suspended)
* adds the results to the ResultContainer (method extend_container)
* handles exceptions (method handle_exception)
|
