summaryrefslogtreecommitdiff
path: root/utils
AgeCommit message (Collapse)Author
2021-04-08Allow overriding env vars SEARX_SETTINGS_PATH,UWSGI_SETTINGS_PATH3nprob
2021-03-05[mod] LXC switch to Fedora 33 / Fedora 31 reached its EOLMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-05Merge pull request #2600 from dalf/searx-extraAlexandre Flament
Add searx_extra package
2021-03-04Add searx_extra packageAlexandre Flament
Split the utils directory into: * searx_extra contains update scripts, standalone_searx.py * utils contains the files to build and setup searx.
2021-03-04Merge pull request #2618 from thezeroalpha/masterAlexandre Flament
Fix security vulnerabilities in provided nginx configuration
2021-03-04Merge pull request #2623 from return42/fix-centos7Alexandre Flament
[fix] add package `which` to CentOS-7 boilerplate
2021-03-04Merge pull request #2619 from return42/drop-ubu1604Alexandre Flament
[mod] Drop Ubuntu 16.04 (Xenial Xerus) support
2021-03-04Merge pull request #2620 from return42/fix-gitAlexandre Flament
[fix] support git versions <v2.22
2021-03-04Use $host in nginx morty.conf templateAlex Balgavy
2021-03-03[fix] add package 'which' to CentOS-7 boilerplateMarkus Heiser
Newer CentOS-7 images from https://images.linuxcontainers.org do no longer include the which command. Issue: $ sudo -H ./utils/lxc.sh cmd searx-centos7 ./utils/filtron.sh install all INFO: [searx-centos7] ./utils/filtron.sh install all ... Install Go in user's HOME ------------------------- download and install go binary .. ... -bash: line 1: which: command not found -->|ERROR - Go Installation not found in PATH!?! -bash: line 2: which: command not found Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03[fix] utils/serax.sh create_pyenv() - drop duplicate 'pip install .'Markus Heiser
The wrong and unnecessary `pip install .` is executed in /usr/local/searx and is responsible for the error message: ERROR: File "setup.py" not found. Directory cannot be installed in editable mode: /usr/local/searx The correct pip-install comes right after changing to `cd ${SEARX_SRC}`. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03[fix] support git versions <v2.22Markus Heiser
LTS distros like Ubuntu 18.04 do not ship a up-to-date version of git.:: $ sudo -H ./utils/lxc.sh cmd searx-ubu1804 git --version ... git version 2.17.1 The option `--show-current` was added in git v2.22, the alternative to this option is:: git rev-parse --abbrev-ref HEAD Issue when using option `--show-current`:: [searx-ubu1804] Clone searx sources [searx-ubu1804] ------------------- [searx-ubu1804] [searx-ubu1804] error: unknown option `show-current' [searx-ubu1804] usage: git branch [<options>] [-r | -a] [--merged | --no-merged] .... Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03[mod] Drop Ubuntu 16.04 (Xenial Xerus) supportMarkus Heiser
EOL of Ubuntu 16.04 (Xenial Xerus) is in April 2021 but we do not support 16.04 since a longer time. Issues are comming from dependencies (e.g. pip):: [searx-ubu1604] |searx| SyntaxError: invalid syntax [searx-ubu1604] |searx| Traceback (most recent call last): [searx-ubu1604] |searx| File "/usr/local/searx/searx-pyenv/bin/pip", line 7, in <module> [searx-ubu1604] |searx| from pip._internal.cli.main import main [searx-ubu1604] |searx| File "/usr/local/searx/searx-pyenv/lib/python3.5/site-packages/pip/_internal/cli/main.py", line 60 [searx-ubu1604] |searx| sys.stderr.write(f"ERROR: {exc}") [searx-ubu1604] |searx| ^ [searx-ubu1604] |searx| SyntaxError: invalid syntax Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-03-03Fix security vulnerabilities in suggested nginx configurationAlex Balgavy
The suggested configurations for nginx found in the documentation and templates lead to vulnerabilities allowing host spoofing [1] and path traversal [2], as reported by Gixy [3]. This commit fixes those issues. [1] https://github.com/yandex/gixy/blob/master/docs/en/plugins/hostspoofing.md [2] https://github.com/yandex/gixy/blob/master/docs/en/plugins/aliastraversal.md [3] https://github.com/yandex/gixy
2021-02-25remove articles number from engines_languages.jsonMarc Abonce Seguin
2021-02-24[mod] add utils/fetch_external_bangs.pyAlexandre Flament
Based on duckduckgo bangs Store bangs on a trie to allow autocomplete (not in this commit)
2021-02-23Merge pull request #2595 from dalf/update-wikidata-unitsAlexandre Flament
[mod] update wikidata_units.json and fetch_wikidata_units.py
2021-02-23[mod] update currencies.json and fetch_currencies.pyAlexandre Flament
use a sparql request on wikidata to get the list of currencies. currencies.json contains the translation for all supported searx languages. Supersede #993
2021-02-23[mod] update wikidata_units.json and fetch_wikidata_units.pyAlexandre Flament
The fetch_wikidata_units.py result won't change randomly. See comments in the script.
2021-02-02[mod] CI: minor changesAlexandre Flament
* utils/makefile.python: travis-gh-pages renamed ci-gh-pages
2021-02-01Merge pull request #2500 from dalf/github-action-dataAlexandre Flament
[enh] every Sunday, call utils/fetch_*.py scripts and create a PR automatically
2021-02-01Merge pull request #2514 from return42/fix-gh-pagesAlexandre Flament
[fix] Makefile target gh-pages & flatten history of branch gh.pages
2021-02-01[fix] re-add 'pip-exe' target - partial revert 9b48ae47Markus Heiser
Target pip-exe is a prerequisite of the targets: - pyinstall - pyuninstall and was accidentally deleted in commit 9b48ae47. HINT: do not confuse pyinstall with penvinstall pyinstall & pyuninstall Installing into user's HOME using pip from OS, therefore the message is needed. pyenvinstall & pyenvuninstall Installing into virtualenv (./local) using pip which is provided by prerequisite 'pyenv' in the virtualenv. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-02-01[mod] more robust make pyenv / make pyenvinstallAlexandre Flament
"make pyenv" ensures that ./local/py3/bin/python is an executable
2021-02-01Merge pull request #2494 from return42/rm-fabfileAlexandre Flament
[fix] remove Fabric file
2021-01-29[fix] Makefile target gh-pages & flatten history of branch gh.pagesMarkus Heiser
1. This patch fixes error: rm -rf gh-pages/ make V=1 gh-pages make[1]: Leaving directory '/800GBPCIex4/share/searx' [ -d "gh-pages/.git" ] || git clone gh-pages fatal: repository 'gh-pages' does not exist 2. The gh-page build has been moved to ./build/gh-pages this also affects 'travis-gh-pages' 3. The gh-pages commit messages now includes a ref to the repository and commit 4. Since a gh-pages history has only the drawback that the reposetory grows fast, this patch also flattens the history: cd build/gh-pages/; git log --oneline bash: cd: build/gh-pages/: Datei oder Verzeichnis nicht gefunden 026126be (HEAD -> gh-pages, origin/gh-pages) make gh-pages: from https://github.com/return42/searx.git@71d66979c2935312e0aed7fc7c3cf6199fbe88a2 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-24[mod] utils/fetch_languages.py: write files at the right locationAlexandre Flament
2021-01-24[fix] lxc.sh - SC2034: ubu2010_boilerplate appears unused.Markus Heiser
$ make test.sh In utils/lxc.sh line 42: ubu2010_boilerplate="$ubu1904_boilerplate" ^-----------------^ SC2034: ubu2010_boilerplate appears unused. Verify use (or export if used externally). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-22[fix] remove Fabric fileMarkus Heiser
The fabfile.py has not been updated since 5 years. I also asked [1] if someone still use Fabric wtihout any response. Lets drop outdated Fabric file. [1] https://github.com/searx/searx/discussions/2400 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-19[mod] checker : replace pycld3 by langdetectAlexandre Flament
pycld3 requires the native library cld3 langdetect is a pure python package
2021-01-16Merge pull request #2473 from return42/fix-setup.pyAlexandre Flament
[fix] setup.py requires pyyaml installed
2021-01-16[fix] setup.py requires pyyaml installedMarkus Heiser
pip install -e . ... Obtaining file:///usr/local/searx/searx-src ERROR: Command errored out with exit status 1: command: /usr/local/searx/searx-pyenv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/usr/local/searx/searx-src/setup.py'"'"'; __file__='"'"'/usr/local/searx/searx-src/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'rn'"'"', '"'"'n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-vzer91m2 cwd: /usr/local/searx/searx-src/ Complete output (9 lines): Traceback (most recent call last): File "<string>", line 1, in <module> File "/usr/local/searx/searx-src/setup.py", line 10, in <module> from searx.version import VERSION_STRING File "/usr/local/searx/searx-src/searx/__init__.py", line 19, in <module> import searx.settings_loader File "/usr/local/searx/searx-src/searx/settings_loader.py", line 8, in <module> import yaml ModuleNotFoundError: No module named 'yaml' ---------------------------------------- ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-14[enh] engines: add about variableAlexandre Flament
move meta information from comment to the about variable so the preferences, the documentation can show these information
2021-01-12[enh] add searx.sharedAlexandre Flament
shared dictionary between the workers (UWSGI or werkzeug) scheduler: run a task once every x seconds (UWSGI or werkzeug)
2021-01-12[enh] add checkerAlexandre Flament
2021-01-11[fix] utils/build_env.py: make sure to always use searx/settings.ymlAlexandre Flament
ignore SEARX_SETTINGS_PATH
2021-01-11[fix] utils/build_env.py: make sure to always use searx/settings.ymlMarkus Heiser
ignore SEARX_SETTINGS_PATH [1] https://github.com/searx/searx/pull/2408#pullrequestreview-565247895 Suggested-by: @dalf [1] Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-11[mod] get rid of searx/brand.pyMarkus Heiser
Removes module searx/brand.py and creates a namespace at searx.brand. This patch is a first 'proof of concept'. Later we can decide to remove the brand namespace entirely or not. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-11[mod] move brand options from Makefile to settings.ymlMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-10[mod] add target upload-pypi-test https://test.pypi.org/Markus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-02[mod] add Ubuntu 20.10 image to the LXC suiteMarkus Heiser
Ubuntu 20.10 image [1] is avialable from https://images.linuxcontainers.org BTW: replace builhost package 'ttf-dejavu' by more common meta package 'fonts-dejavu' ('ttf-dejavu' is no longer available in 20.10). [1] https://github.com/lxc/lxc-ci/commit/d161de601bffe4055d77b9d52a826ce5d81567d9 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-01-02[fix] remove Ubuntu 19.10 from the LXC suite (EOL)Markus Heiser
Official support for Ubuntu 19.10 'Eoan Ermine' ended on July 17, 2020. The image has been dropped [1] from https://images.linuxcontainers.org [1] https://github.com/lxc/lxc-ci/commit/d161de601bffe4055d77b9d52a826ce5d81567d9 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-27Merge pull request #2411 from dalf/update-secret-key-checkAlexandre Flament
Update secret key check
2020-12-27[perf] pylint: Use multiple processes to speed up Pylint.Markus Heiser
Specifying '-j 0' will auto-detect the number of processors available to use. Suggested-by: @dalf / https://github.com/searx/searx/pull/2421#issuecomment-751444257 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-27[mod] check secret_key when searx.webapp is importedAlexandre Flament
Without this commit the module searx checks the secret_key value. With this commit, make docs, utils/standalone_searx.py, utils/fetch_firefox_version.py works without SEARX_DEBUG=1 For reference see https://github.com/searx/searx/pull/2386
2020-12-27Merge pull request #2420 from return42/fix-sphinx-book-targetsAlexandre Flament
fix makefile `books/{name}.*` targets and add documentation about these targets
2020-12-27Merge pull request #2412 from dalf/update-buildenvAlexandre Flament
[fix] update buildenv
2020-12-26[mod] LXC_ENV_FOLDER moved from ./lxc to ./lxc-envMarkus Heiser
BTW: - add target $(LXC_ENV_FOLDER) - fix indentations Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-26[fix] utils/lib.sh: various typos in messages and commentsMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2020-12-24[fix] message to install different python versions from apt PPAMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion