diff options
Diffstat (limited to 'docs/dev')
| -rw-r--r-- | docs/dev/contribution_guide.rst | 71 | ||||
| -rw-r--r-- | docs/dev/install/installation.rst | 277 | ||||
| -rw-r--r-- | docs/dev/plugins.rst | 45 | ||||
| -rw-r--r-- | docs/dev/search_api.rst | 24 | ||||
| -rw-r--r-- | docs/dev/translation.rst | 74 |
5 files changed, 491 insertions, 0 deletions
diff --git a/docs/dev/contribution_guide.rst b/docs/dev/contribution_guide.rst new file mode 100644 index 000000000..5685549eb --- /dev/null +++ b/docs/dev/contribution_guide.rst @@ -0,0 +1,71 @@ +How to contribute +----------------- + +Prime directives: Privacy, Hackability +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Searx has 2 prime directives, privacy-by-design and hackability. The +hackability comes in at least 3 levels: + +- support for search engines +- plugins for altering search behaviour +- hacking searx itself. + +Happy hacking. Observe the lack of "world domination" among the +directives, searx has no intentions for wide mass-adoption, rounded +corners, etc. The prime directive: "privacy" - deserves a seperate +chapter, as it's quite uncommon unfortunately, here it goes: + +Privacy-by-design +^^^^^^^^^^^^^^^^^ + +Searx is a privacy-respecting, hackable meta-search engine. It was born +out of the need for a privacy-respecing search facility that can be +expanded easily to maximise both its search and it's privacy protecting +capabilities. + +Consequences of Privacy-by-design are that some widely used features +work differently or not by default or at all. If some feature reduces +the privacy perserving aspects of searx, it should by default be +switched of, if implemented at all. There is enough search engines +already out there providing such features. = Since privacy-preservation +is a prime goal, if some feature does reduce the protection of searx and +is implemented, care should be taken to educate the user about the +consequences of choosing to enable this. Further features which +implement widely known features in a manner that protects privacy but +thus deviate from the users expectations should also be explained to the +user. Also if you think that something works weird with searx, maybe +it's because of the tool you use is designed in a way to interfere with +privacy respect, submiting a bugreport to the vendor of the tool that +misbehaves might be a good feedback for the vendor to reconsider his +disrespect towards his customers (e.g. GET vs POST requests in various +browsers). + +Remember the other prime directive of searx is to be hackable, so if the +above privacy concerns do not fancy you, simply fork it. + +Code +~~~~ + +Code modifications are accepted in pull requests, don't forget to add +yourself to the AUTHORS file. + +Python code follows all the pep8 standards except maximum line width +which is 120 char. + +Please be sure that the submitted code doesn't break existing tests and +follows coding conventions. + +If new functionality implemented, tests are highly appreciated. + +Translation +~~~~~~~~~~~ + +Translation currently happens on +`transifex <https://transifex.com/projects/p/searx>`__. Please do not +update translation files in the repo. + +Documentation +~~~~~~~~~~~~~ + +The main place of the documentation is this wiki, updates are welcome. diff --git a/docs/dev/install/installation.rst b/docs/dev/install/installation.rst new file mode 100644 index 000000000..2ce6a6b54 --- /dev/null +++ b/docs/dev/install/installation.rst @@ -0,0 +1,277 @@ +Installation +============ + +Step by step installation for Debian / Ubuntu with virtualenv. + +Source: https://about.okhin.fr/posts/Searx/ with some additions + +How to: `Setup searx in a couple of hours with a free SSL +certificate <https://www.reddit.com/r/privacytoolsIO/comments/366kvn/how_to_setup_your_own_privacy_respecting_search/>`__ + +Basic installation +------------------ + +For Ubuntu, be sure to have enable universe repository. + +Install packages : + +.. code:: sh + + sudo apt-get install git build-essential libxslt-dev python-dev python-virtualenv python-pybabel zlib1g-dev libffi-dev libssl-dev + +Install searx : + +.. code:: sh + + cd /usr/local + sudo git clone https://github.com/asciimoo/searx.git + sudo useradd searx -d /usr/local/searx + sudo chown searx:searx -R /usr/local/searx + +Install dependencies in a virtualenv : + +.. code:: sh + + sudo -u searx -i + cd /usr/local/searx + virtualenv searx-ve + . ./searx-ve/bin/activate + pip install -r requirements.txt + python setup.py install + +Configuration +------------- + +.. code:: sh + + sed -i -e "s/ultrasecretkey/`openssl rand -hex 16`/g" searx/settings.yml + +Edit searx/settings.yml if necessary. + +Check +----- + +Start searx : + +.. code:: sh + + python searx/webapp.py + +Go to http://localhost:8888 + +If everything works fine, disable the debug option in settings.yml : + +.. code:: sh + + sed -i -e "s/debug : True/debug : False/g" searx/settings.yml + +At this point searx is not demonized ; uwsgi allows this. + +You can exit the virtualenv and the searx user bash (enter exit command +twice). + +uwsgi +----- + +Install packages : + +.. code:: sh + + sudo apt-get install uwsgi uwsgi-plugin-python + +Create the configuration file /etc/uwsgi/apps-available/searx.ini with +this content : + +:: + + [uwsgi] + # Who will run the code + uid = searx + gid = searx + + # disable logging for privacy + disable-logging = true + + # Number of workers (usually CPU count) + workers = 4 + + # The right granted on the created socket + chmod-socket = 666 + + # Plugin to use and interpretor config + single-interpreter = true + master = true + plugin = python + + # Module to import + module = searx.webapp + + # Virtualenv and python path + virtualenv = /usr/local/searx/searx-ve/ + pythonpath = /usr/local/searx/ + chdir = /usr/local/searx/searx/ + +Activate the uwsgi application and restart : + +.. code:: sh + + cd /etc/uwsgi/apps-enabled + ln -s ../apps-available/searx.ini + /etc/init.d/uwsgi restart + +Web server +---------- + +with nginx +^^^^^^^^^^ + +If nginx is not installed (uwsgi will not work with the package +nginx-light) : + +.. code:: sh + + sudo apt-get install nginx + +Hosted at / +""""""""""" + +Create the configuration file /etc/nginx/sites-available/searx with this +content : + +.. code:: nginx + + server { + listen 80; + server_name searx.example.com; + root /usr/local/searx; + + location / { + include uwsgi_params; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + } + +Restart service : + +.. code:: sh + + sudo service nginx restart + sudo service uwsgi restart + +from subdirectory URL (/searx) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Add this configuration in the server config file +/etc/nginx/sites-available/default : + +.. code:: nginx + + location = /searx { rewrite ^ /searx/; } + location /searx { + try_files $uri @searx; + } + location @searx { + uwsgi_param SCRIPT_NAME /searx; + include uwsgi_params; + uwsgi_modifier1 30; + uwsgi_pass unix:/run/uwsgi/app/searx/socket; + } + +Enable base\_url in searx/settings.yml + +:: + + base_url : http://your.domain.tld/searx/ + +Restart service : + +.. code:: sh + + sudo service nginx restart + sudo service uwsgi restart + +disable logs +~~~~~~~~~~~~ + +for better privacy you can disable nginx logs about searx. + +how to proceed : below ``uwsgi_pass`` in +/etc/nginx/sites-available/default add + +:: + + access_log /dev/null; + error_log /dev/null; + +Restart service : + +.. code:: sh + + sudo service nginx restart + +with apache +----------- + +Add wsgi mod : + +.. code:: sh + + sudo apt-get install libapache2-mod-uwsgi + sudo a2enmod uwsgi + +Add this configuration in the file /etc/apache2/apache2.conf : + +.. code:: apache + + <Location /> + Options FollowSymLinks Indexes + SetHandler uwsgi-handler + uWSGISocket /run/uwsgi/app/searx/socket + </Location> + +Note that if your instance of searx is not at the root, you should +change ``<Location />`` by the location of your instance, like +``<Location /searx>``. + +Restart Apache : + +.. code:: sh + + sudo /etc/init.d/apache2 restart + +disable logs +------------ + +For better privacy you can disable Apache logs. + +WARNING : not tested + +WARNING : you can only disable logs for the whole (virtual) server not +for a specific path. + +Go back to /etc/apache2/apache2.conf and above ``<Location />`` add : + +.. code:: apache + + CustomLog /dev/null combined + +Restart Apache : + +.. code:: sh + + sudo /etc/init.d/apache2 restart + +How to update +------------- + +.. code:: sh + + cd /usr/local/searx + sudo -u searx -i + . ./searx-ve/bin/activate + git stash + git pull origin master + git stash apply + pip install --upgrade -r requirements.txt + sudo service uwsgi restart + diff --git a/docs/dev/plugins.rst b/docs/dev/plugins.rst new file mode 100644 index 000000000..c51fdf0eb --- /dev/null +++ b/docs/dev/plugins.rst @@ -0,0 +1,45 @@ +Plugins +------- + +Plugins can extend/replace functionality of various components inside +searx. + +example\_plugin.py +~~~~~~~~~~~~~~~~~~ + +.. code:: python + + name = 'Example plugin' + description = 'This plugin extends the suggestions with the word "example"' + default_on = False # disable by default + + js_dependencies = tuple() # optional, list of static js files + css_dependencies = tuple() # optional, list of static css files + + + # attach callback to the post search hook + # request: flask request object + # ctx: the whole local context of the post search hook + def post_search(request, ctx): + ctx['search'].suggestions.add('example') + return True + +Currently implemented plugin entry points (a.k.a hooks) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- Pre search hook (``pre_search``) +- Post search hook (``post_search``) +- Result hook (``on_result``) (is called if a new result is added (see + https\_rewrite plugin)) + +Feel free to add more hooks to the code if it is required by a plugin. + +TODO +~~~~ + +- Better documentation +- More hooks +- search hook (is called while searx is requesting results (for + example: things like math-solver), the different hooks are running + parallel) + diff --git a/docs/dev/search_api.rst b/docs/dev/search_api.rst new file mode 100644 index 000000000..cd86fd71c --- /dev/null +++ b/docs/dev/search_api.rst @@ -0,0 +1,24 @@ +Search API +========== + +Search API endpoints: ``/``, ``/search`` +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Endpoints have equivalent functionality. + +Parameters +^^^^^^^^^^ + ++------------------+----------------------------------------------------------------------------------------------------+-----------------------------+ +| Name | Description | | ++==================+====================================================================================================+=============================+ +| ``q`` | The search query, see :doc:`/user/search_syntax` | required | ++------------------+----------------------------------------------------------------------------------------------------+-----------------------------+ +| ``categories`` | Comma separated list, specifies the active search categories | optional | ++------------------+----------------------------------------------------------------------------------------------------+-----------------------------+ +| ``engines`` | Comma separated list, specifies the active search engines | optional | ++------------------+----------------------------------------------------------------------------------------------------+-----------------------------+ +| ``pageno`` | Search page number | optional (default: ``1``) | ++------------------+----------------------------------------------------------------------------------------------------+-----------------------------+ + +Both ``GET`` and ``POST`` methods are supported. diff --git a/docs/dev/translation.rst b/docs/dev/translation.rst new file mode 100644 index 000000000..0fc02a7de --- /dev/null +++ b/docs/dev/translation.rst @@ -0,0 +1,74 @@ +Translation +=========== + +run these commands in the root directory of searx + +Add new language +~~~~~~~~~~~~~~~~ + +``pybabel init -i messages.pot -d searx/translations -l it`` + +Update .po files +~~~~~~~~~~~~~~~~ + +``./utils/update-translations.sh`` + +You may have errors here. In that case, edit the +``update-translations.sh`` script to change ``pybabel`` to +``pybabel-python2`` + +After this step, you can modify the .po files. + +Compile translations +~~~~~~~~~~~~~~~~~~~~ + +``pybabel compile -d searx/translations`` + +Transifex stuff +~~~~~~~~~~~~~~~ + +Init Project +^^^^^^^^^^^^ + +.. code:: shell + + tx set --auto-local -r searx.messagespo 'searx/translations/<lang>/LC_MESSAGES/messages.po' \ + --source-lang en --type PO --source-file messages.pot --execute + +http://docs.transifex.com/developer/client/set + +*TODO: mapping between transifex and searx* + +Get translations +^^^^^^^^^^^^^^^^ + +.. code:: shell + + tx pull -a + +http://docs.transifex.com/developer/client/pull + +Upload source File +^^^^^^^^^^^^^^^^^^ + +:: + + tx push -s + +Upload all Translation +^^^^^^^^^^^^^^^^^^^^^^ + +:: + + tx push -s -t + +upload specifc Translation (only for admins) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +:: + + tx push -t -l tr + +http://docs.transifex.com/developer/client/push + +*TODO: upload empty files? (new translations)* |