summaryrefslogtreecommitdiff
path: root/manage
AgeCommit message (Collapse)Author
2021-10-04[fix] commit message of 'make weblate.push.translations'Markus Heiser
Fix `\n` issue in the commit message [1] by using multiple `-m` options [2]:: 7d9ffd680 translations [translations] update messages.pot and messages.po files\nFrom cebc0e39 - 2021-10-04 - Markus Heiser <markus.heiser@darmarIT.de> [1] https://github.com/searxng/searxng/pull/379#issuecomment-933242702 [2] https://git-scm.com/docs/git-commit#Documentation/git-commit.txt--mltmsggt Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-10-03[fix] make weblate.push.translationsAlexandre Flament
Even when there are changes, the function exit without pushing them.
2021-10-03Merge pull request #375 from dalf/searxng_extraAlexandre Flament
SearXNG: searx_extra
2021-10-02SearXNG: searxng_extraAlexandre Flament
2021-10-02SearXNG: SEARXNG_SETTINGS_PATHAlexandre Flament
2021-10-02SearXNG: environment variablesAlexandre Flament
2021-09-17[mod] manage - use pyenv.activate where it makes senseMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-17[mod] utils/lib.sh: add pyenv.activateAlexandre Flament
* initialize virtualenv (pyenv.install) * activate virtualenv close #310
2021-09-12[fix] manage help(): commands babel.* has been renamed to weblate.*Markus Heiser
In commit 97355672c the functions named babel.* has been renamed to weblate.* but it was forgotten to change it also in the help(). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-07[fix] add 'categories' to PYLINT_ADDITIONAL_BUILTINS_FOR_ENGINESMarkus Heiser
androp no longer needed (see line 591 in 7b235a1):: # pylint: disable=undefined-variable Suggested-by: @dalf https://github.com/searxng/searxng/issues/102#issuecomment-914068609 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-06Merge pull request #293 from dalf/fix-290Alexandre Flament
[fix] translation: lock weblate only when there is an actual change of messages.pot
2021-09-06[mod] one logger per engineMarkus Heiser
Suggested-by: @dalf in https://github.com/searxng/searxng/issues/98#issuecomment-849013518 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-09-06[fix] translation: lock weblate only when there is an actual change of ↵Alexandre Flament
messages.pot Close #290
2021-09-02[mod] move searx/testing.py to the tests directoryAlexandre Flament
move robot tests to tests.robot manage calls "python -m tests.robot"
2021-08-16[translations] ./manage rename shell functions to integrate weblateMarkus Heiser
Functions implemented to run *Weblate* workflows should use prefix 'weblate.': - babel.setup.translations.worktree --> weblate.translations.worktree Create git worktree ${TRANSLATIONS_WORKTREE} and checkout branch 'translations' from Weblate's counterpart (weblate) of the SearXNG (origin):: remote weblate https://weblate.bubu1.eu/git/searxng/searxng/ - babel.weblate.to.translations --> weblate.to.translations Update 'translations' branch of SearXNG (origin) with last additions from Weblate. - babel.translations.to.master --> weblate.translations.commit Update 'translations' branch of SearXNG (origin) with last additions from Weblate. Copy the changes to the master branch, compile translations and create a commit in the local branch (master) - babel.master.to.translations --> weblate.push.translations Push *translation changes* from SearXNG (origin) to Weblate's counterpart (weblate). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-16[translation] ./manage: add comments to new functions, rename jobMarkus Heiser
./mange Comment functions: - babel.setup.translations.worktree - babel.weblate.to.translations - babel.translations.to.master - babel.master.to.translations BTW: - changed some cd into pushd - removed obsolete `set -x` - remove not needed echo from `last_commit_message=$(echo ..` .github/workflows/translations-update.yml docs/dev/translation.rst Rename job to: "create PR for additons from weblate" Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-07[translations] web integrationAlexandre Flament
* make babel.translations.to.master: pull weblate updates * make babel.master.to.translations: push .pot and .po files to weblate
2021-08-02[mod] manage - implement babel commands, drop update_translations.shMarkus Heiser
In ./manage implement babel.*: - extract : extract messages from source files and generate POT file - update : update existing message catalogs from POT file - compile : compile translation catalogs into binary MO files Replace searx_extra/update/update_translations.sh by command: - ci.babel.update Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-08-01manage script: remove unused reference to utils/brand.env and .config.shAlexandre Flament
2021-07-30version based on the git repositoryAlexandre Flament
This commit remove the need to update the brand for GIT_URL and GIT_BRANCH: there are read from the git repository. It is possible to call python -m searx.version freeze to freeze the current version. Useful when the code is installed outside git (distro package, docker, etc...)
2021-07-24[fix] test.robot - do not use environment from ./utils/brand.envMarkus Heiser
Some defaults in the settings.yml are taken from the environment. By example; The manage scripts sources the ./utils/brand.env and sets SEARX_PORT environment. This enviroment *wins over* any settings in a YAML file. Whe we run a:: make test.robot The searx/settings_robot.yml is used, in this file the server settings are:: server: port: 11111 bind_address: 127.0.0.1 To get in use of the 'port: 11111' we have to unset the SEARX_PORT environment which was sourced before. The function buildenv.unset_env() can be called in all use cases where the enviroment from ./utils/brand.env is not wanted. ATM it unset the enviroment variables:: unset GIT_URL unset GIT_BRANCH unset SEARX_URL unset SEARX_PORT unset SEARX_BIND_ADDRESS Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21[fix] utils/build_env.py and add documentationMarkus Heiser
modified docs/admin/engines/settings.rst - Fix documentation and add section 'brand'. - Add remarks about **buildenv** variables. - Add remarks about settings from environment variables $SEARX_DEBUG, $SEARX_PORT, $SEARX_BIND_ADDRESS and $SEARX_SECRET modified docs/admin/installation-searx.rst & docs/build-templates/searx.rst Fix template location /templates/etc/searx/settings.yml modified docs/dev/makefile.rst Add description of the 'make buildenv' target and describe - we have all SearXNG setups are centralized in the settings.yml file - why some tasks need a utils/brand.env (aka instance's buildenv) modified manage Settings file from repository's working tree are used by default and ask user if a /etc/searx/settings.yml file exists. modified searx/settings.yml Add comments about when it is needed to run 'make buildenv' modified searx/settings_defaults.py Default for server:port is taken from enviroment variable SEARX_PORT. modified utils/build_env.py - Some defaults in the settings.yml are taken from the environment, e.g. SEARX_BIND_ADDRESS (searx.settings_defaults.SHEMA). When the 'brand.env' file is created these enviroment variables should be unset first. - The CONTACT_URL enviroment is not needed in the utils/brand.env Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-21[mod] ./manage docker.buildx make variables localMarkus Heiser
The environment variables .. - SEARX_GIT_VERSION - VERSION_GITCOMMIT - SEARX_PYTHON_VERSION - GITHUB_USER - SEARX_IMAGE_NAME - BUILD are local to the docker.buildx() function, other variables like GIT_URL are comming from the utils/brand.env file (rebuild by: make buildenv). Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-07-17[fix] ./manage script, prompt the unknown commandMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-26[mod] move functions from utils/manage_static.sh to ./manage scriptMarkus Heiser
The functions: - static.build.commit - static.build.commit.drop - static.build.restore are imported into the ./manage script. To avoid name collisions some variables and fucntions has been renamed by adding the prefix *static_*. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-22Merge pull request #157 from searxng/mod_static_buildAlexandre Flament
static build: move generated files to searx/static/themes/*/src/generated
2021-06-22[fix] pygments.less - remove pyenv_OK conditionMarkus Heiser
BTW: add pygments.less to the help message Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-22[mod] simple theme: include fontsAlexandre Flament
"npm run webfont" to build the fonts directory. It requires fontforge and ttfautohint distro packages. partial revert of commit 7137d2893f33e9158e8563a21d9586119d175269
2021-06-22[mod] make themes.all update pygments*.lessAlexandre Flament
Add a searx/static/themes/*/src/generated empty folder
2021-06-21[fix] py.build - fix small typo in the build messageMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-19[fix] node.clean - ignore npm dependencies when npm is not installedMarkus Heiser
error pattern:: $ make clean CLEAN pyenv PYENV [virtualenv] drop local/py3 CLEAN docs -- build/docs dist/docs CLEAN locally installed npm dependencies ./manage: line 318: npm: command not found ERROR: node.clean exit with error (127) make: *** [Makefile:90: node.clean] Error 127 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-17[enh] ./manage node.env - check build tools firstMarkus Heiser
The node.env build environment require npm, ttfautohint and fontforge installed in the OS. These tools can be installed by:: sudo -H ./utils/searx.sh install buildhost If one of the tools is not installed, the script node.env stops with a appropriate message. BTW: We ignore CentOS-7 as developer & build platform Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-16[mod] make node.clean: call the "clean" script from the packages.jsonAlexandre Flament
The dependencies and build process are described in the gruntfiles.js and packages.json files. It makes sense to move the clean up to these files too.
2021-06-16[mod] remove common grunt installAlexandre Flament
Before this commit, there are 3 node_modules directory: * one in . * two others in ./searx/statics/themes/* This is no desirable: * it declares the npm depdenencies in the shell script. * dependabot can't updates theses dependencies. * this is a not standard way to build a package (two different locations for the dependencies). With this commit and the PR #150 there is one unique node_modules directory per theme.
2021-06-16[mod] simple theme: remove src/less/ion.lessAlexandre Flament
This file is generated by webfont. * It is now generated as searx/static/themes/simple/ion.less * It is generated before the .less compilation. * .gitignore includes this file Add two new package depedencies: fontforge ttfautohint See utils/searx.sh
2021-06-15[fix] test.yamllint - fixed typoMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-12[yamllint] ./utils/templates/etc/searx/*.ymlMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-09[data.all] add searx_extra/update/update_osm_keys_tags.pyMarkus Heiser
Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-06-05[enh] add test.yamllint - lint yaml filesMarkus Heiser
Usage:: make test.yamllint ./manage test.yamllint test.yamllint is also added to the `test` makefile target. Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-28[docker] multiarch support: linux/amd64,linux/arm64,linux/arm/v7Alexandre Flament
make docker.buildx : build and push multiarch build. (it can't be only build) use buildx with the --cache-from and --cache-to options to cache the layers (only the last built is cached)
2021-04-27[fix] manage - fix miss usage of 'set -e'Markus Heiser
The philosophy of set -e is typically that it only exits upon uncaught errors. Here, the presence of || outside the subshell seems to tell the shell that the error inside the subshell is 'caught' and therefore set -e does not cause an exit after false [1]. The shell does not exit if the command that fails is ... part of any command executed in a && or || list except the command following the final && or ||, any command in a pipeline but the last, or if the command’s return status is being inverted with ! [2] [1] https://unix.stackexchange.com/questions/296526/set-e-in-a-subshell [2] https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html#The-Set-Builtin BTW: fix error reported by 'make test.shell' Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-26[pylint] get PYLINT_FILES from tag '# lint: pylint'Markus Heiser
These py files are linted by test.pylint(), all other files are linted by test.pep8() close: https://github.com/searxng/searxng/issues/21 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-24[mod] core.ac.uk engineMarkus Heiser
- add to list of pylint scripts - add debug log messages - move API key int `settings.yml` - improved readability - add some metadata to results Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-24Rerevert c54bf42Alexandre Flament
2021-04-23[fix] ./manage pyenv.install - die when pyenv build failsMarkus Heiser
ERROR: When pyenv build fails, the pyenv command runs into a recursion [1]: .... BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV ERROR: Cannot install idna==3.1 and requests[socks]==2.25.1 because these package versions have conflicting dependencies. BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV BUILDENV The conflict is caused by: [1] https://github.com/return42/searx-next/pull/4#issuecomment-821999497 Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>
2021-04-23[mod] replace makefile boilerplate by 'manage' scriptMarkus Heiser
Replaces the make targets with the bash scripts Signed-off-by: Markus Heiser <markus@darmarit.de>
2021-04-23[enh] implement all build task in a bash script (manage)Markus Heiser
note: in further patches script 'manage' will replace 'manage.sh' pyenv.* : assert : build virtualenv if not exists intsall : developer install of searx into virtualenv unintsall : uninstall developer installation cmd ... : run command ... in virtualenv OK : test if virtualenv is OK pypi.upload: Upload python packages to PyPi (to test use pypi.upload.test) pybuild : Build python packages at ./${PYDIST} pyclean : delete virtualenv and intermediate py files test.* : pylint : lint PYLINT_FILES, searx/engines, searx & tests pep8 : pycodestyle (pep8) for all files except PYLINT_FILES unit : run unit tests coverage : run unit tests with coverage robot : run robot test clean : clean intermediate test stuff node.* : env : download & install npm dependencies locally clean : drop npm installations buildenv : rebuild ./utils/brand.env data.* : all : update searx/languages.py and ./data/* languages : update searx/data/engines_languages.json & searx/languages.py useragents: update searx/data/useragents.json with the most recent versions of Firefox. themes.* : all : build all themes oscar : build oscar theme simple : build simple theme bootstrap : less compile bootstrap.min.css CSS babel.compile : pybabel compile ./searx/translations docs.* : html : build HTML documentation gh-pages : deploy on gh-pages branch autobuild : autobuild HTML documentation while editing prebuild : build reST include files (./${DOCS_BUILD}/includes) clean : clean documentation build docker.build [push] : build (and push) docker image gecko.driver : download & install geckodriver if not already installed (required for robot_tests) Signed-off-by: Markus Heiser <markus@darmarit.de>
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion