[translations] web integration

* make babel.translations.to.master: pull weblate updates
* make babel.master.to.translations: push .pot and .po files to weblate

2 files changed, 70 insertions, 48 deletions

diff --git a/docs/dev/translation.rst b/docs/dev/translation.rst
index 523dcf78c..be0cf5bef 100644
--- a/docs/dev/translation.rst
+++ b/docs/dev/translation.rst
@@ -4,69 +4,46 @@
 Translation
 ===========
 
-.. _searx@transifex: https://www.transifex.com/asciimoo/searx/
+.. _weblate.bubu1.eu: https://weblate.bubu1.eu/projects/searxng/
 
-Translation currently takes place on `searx@transifex`_
+Translation takes place on `weblate.bubu1.eu`_ ( `documentation <https://docs.weblate.org/en/latest/index.html>`_ ).
 
-Requirements
-============
+New messages on the master branch are extracted and pushed to Weblate automatically.
 
-* Transifex account
+Every Friday, a GitHub workflow creates a pull request with the updated translations (messages.mo, messages.po, messages.mo files).
 
-Init Transifex project
-======================
-
-After installing ``transifex`` using pip, run the following command to
-initialize the project.
-
-.. code:: sh
-
-   ./manage pyenv.cmd tx init # Transifex instance: https://www.transifex.com/asciimoo/searx/
-
-
-After ``$HOME/.transifexrc`` is created, get a Transifex API key and insert it
-into the configuration file.
-
-Create a configuration file for ``tx`` named ``$HOME/.tx/config``.
-
-.. code:: ini
-
-    [main]
-    host = https://www.transifex.com
-    [searx.messagespo]
-    file_filter = searx/translations/<lang>/LC_MESSAGES/messages.po
-    source_file = messages.pot
-    source_lang = en
-    type = PO
+.. image:: https://weblate.bubu1.eu/widgets/searxng/-/searxng/svg-badge.svg
+   :target: https://weblate.bubu1.eu/projects/searxng/
 
+Weblate integration
+===================
 
-Then run ``tx set``:
+Weblate monitors the `translations branch <https://github.com/searxng/searxng/tree/translations>`_, not the master branch.
 
-.. code:: shell
+This branch contains only the .pot and pot files, nothing else.
 
-    ./manage pyenv.cmd tx set --auto-local \
-        -r searx.messagespo 'searx/translations/<lang>/LC_MESSAGES/messages.po' \
-        --source-lang en --type PO --source-file messages.pot --execute
+Documentation
+-------------
 
+* `wlc <https://docs.weblate.org/en/latest/wlc.html>`_
+* `pybabel <http://babel.pocoo.org/en/latest/cmdline.html>`_
+* `weblate workflow <https://docs.weblate.org/en/latest/workflows.html>`_
 
-Update translations
-===================
+Worfklow
+--------
 
-To retrieve the latest translations, pull it from Transifex.
+.. image:: translation.svg
 
-.. code:: sh
 
-   ./manage pyenv.cmd tx pull -a
-   [?] Enter your api token: ....
+wlc
+---
 
-Then check the new languages.  If strings translated are not enough, delete those
-folders, because those should not be compiled.  Call the command below to compile
-the ``.po`` files.
+All weblate integration is done by GitHub workflows, but if you want to use wlc, copy this content into `~/.config/weblate <https://docs.weblate.org/en/latest/wlc.html#wlc-config>`_ :
 
-.. code:: shell
+.. code-block:: ini
 
-   ./manage pyenv.cmd pybabel compile -d searx/translations
+  [keys]
+  https://weblate.bubu1.eu/api/ = APIKEY
 
 
-After the compilation is finished commit the ``.po`` and ``.mo`` files and
-create a PR.
+Replace `APIKEY` by `your API key <https://weblate.bubu1.eu/accounts/profile/#api>`_.
diff --git a/docs/dev/translation.svg b/docs/dev/translation.svg
new file mode 100644
index 000000000..060e58420
--- /dev/null
+++ b/docs/dev/translation.svg
@@ -0,0 +1,45 @@
+<?xml version="1.0" encoding="UTF-8" standalone="no"?><svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" contentScriptType="application/ecmascript" contentStyleType="text/css" height="968px" preserveAspectRatio="none" style="width:1008px;height:968px;background:#FFFFFF;" version="1.1" viewBox="0 0 1008 968" width="1008px" zoomAndPan="magnify"><defs><filter height="300%" id="fwj4fqzvak25j" width="300%" x="-1" y="-1"><feGaussianBlur result="blurOut" stdDeviation="2.0"/><feColorMatrix in="blurOut" result="blurOut2" type="matrix" values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .4 0"/><feOffset dx="4.0" dy="4.0" in="blurOut2" result="blurOut3"/><feBlend in="SourceGraphic" in2="blurOut3" mode="normal"/></filter></defs><g><rect fill="#FFFFFF" filter="url(#fwj4fqzvak25j)" height="320.7266" style="stroke:#000000;stroke-width:2.0;" width="981" x="10" y="116.7266"/><rect fill="#FFFFFF" filter="url(#fwj4fqzvak25j)" height="289.5938" style="stroke:#000000;stroke-width:2.0;" width="961" x="20" y="140.8594"/><rect fill="#FFFFFF" filter="url(#fwj4fqzvak25j)" height="324.7266" style="stroke:#000000;stroke-width:2.0;" width="848" x="143" y="494.5859"/><rect fill="#FFFFFF" filter="url(#fwj4fqzvak25j)" height="245.3281" style="stroke:#000000;stroke-width:2.0;" width="828" x="153" y="518.7188"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="91" x2="91" y1="56.5938" y2="908.5781"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="270" x2="270" y1="56.5938" y2="908.5781"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="514" x2="514" y1="56.5938" y2="908.5781"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="738" x2="738" y1="56.5938" y2="908.5781"/><line style="stroke:#A80036;stroke-width:1.0;stroke-dasharray:5.0,5.0;" x1="899" x2="899" y1="56.5938" y2="908.5781"/><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="30.2969" style="stroke:#A80036;stroke-width:1.5;" width="119" x="30" y="21.2969"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="105" x="37" y="41.292">master branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="30.2969" style="stroke:#A80036;stroke-width:1.5;" width="119" x="30" y="907.5781"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="105" x="37" y="927.5732">master branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="30.2969" style="stroke:#A80036;stroke-width:1.5;" width="211" x="163" y="21.2969"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="197" x="170" y="41.292">translations_update branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="30.2969" style="stroke:#A80036;stroke-width:1.5;" width="211" x="163" y="907.5781"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="197" x="170" y="927.5732">translations_update branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="30.2969" style="stroke:#A80036;stroke-width:1.5;" width="153" x="436" y="21.2969"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="139" x="443" y="41.292">translations branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="30.2969" style="stroke:#A80036;stroke-width:1.5;" width="153" x="436" y="907.5781"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="139" x="443" y="927.5732">translations branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="46.5938" style="stroke:#A80036;stroke-width:1.5;" width="153" x="660" y="5"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="120" x="676.5" y="24.9951">weblate clone of</text><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="139" x="667" y="41.292">translations branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="46.5938" style="stroke:#A80036;stroke-width:1.5;" width="153" x="660" y="907.5781"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="120" x="676.5" y="927.5732">weblate clone of</text><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="139" x="667" y="943.8701">translations branch</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="46.5938" style="stroke:#A80036;stroke-width:1.5;" width="140" x="827" y="5"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="58" x="868" y="24.9951">weblate</text><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="122" x="834" y="41.292">pending changes</text><rect fill="#FEFECE" filter="url(#fwj4fqzvak25j)" height="46.5938" style="stroke:#A80036;stroke-width:1.5;" width="140" x="827" y="907.5781"/><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="58" x="868" y="927.5732">weblate</text><text fill="#000000" font-family="sans-serif" font-size="14" lengthAdjust="spacing" textLength="122" x="834" y="943.8701">pending changes</text><rect fill="#EEEEEE" filter="url(#fwj4fqzvak25j)" height="3" style="stroke:#EEEEEE;stroke-width:1.0;" width="1001" x="0" y="87.1602"/><line style="stroke:#000000;stroke-width:1.0;" x1="0" x2="1001" y1="87.1602" y2="87.1602"/><line style="stroke:#000000;stroke-width:1.0;" x1="0" x2="1001" y1="90.1602" y2="90.1602"/><rect fill="#EEEEEE" filter="url(#fwj4fqzvak25j)" height="23.1328" style="stroke:#000000;stroke-width:2.0;" width="230" x="385.5" y="76.5938"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="211" x="391.5" y="92.6606">for each commit on master</text><path d="M10,116.7266 L314,116.7266 L314,123.7266 L304,133.7266 L10,133.7266 L10,116.7266 " fill="#EEEEEE" style="stroke:#000000;stroke-width:1.0;"/><rect fill="none" height="320.7266" style="stroke:#000000;stroke-width:2.0;" width="981" x="10" y="116.7266"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="259" x="25" y="129.7935">.github/workflows/integration.yml</text><path d="M20,140.8594 L332,140.8594 L332,147.8594 L322,157.8594 L20,157.8594 L20,140.8594 " fill="#EEEEEE" style="stroke:#000000;stroke-width:1.0;"/><rect fill="none" height="289.5938" style="stroke:#000000;stroke-width:2.0;" width="961" x="20" y="140.8594"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="267" x="35" y="153.9263">make babel.master.to.translations</text><polygon fill="#FBFB77" filter="url(#fwj4fqzvak25j)" points="711,162.9922,765,162.9922,775,173.9922,765,185.9922,711,185.9922,701,173.9922,711,162.9922" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="50" x="713" y="179.0591">wlc lock</text><polygon fill="#A80036" points="726.5,208.2578,736.5,212.2578,726.5,216.2578,730.5,212.2578" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="514.5" x2="732.5" y1="212.2578" y2="212.2578"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="47" x="521.5" y="207.1919">wlc pull</text><polygon fill="#A80036" points="749.5,237.3906,739.5,241.3906,749.5,245.3906,745.5,241.3906" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="743.5" x2="898" y1="241.3906" y2="241.3906"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="74" x="755.5" y="236.3247">wlc commit</text><polygon fill="#A80036" points="525.5,266.5234,515.5,270.5234,525.5,274.5234,521.5,270.5234" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="519.5" x2="737.5" y1="270.5234" y2="270.5234"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="196" x="531.5" y="265.4575">git merge weblate/translations</text><polygon fill="#A80036" points="502.5,310.7891,512.5,314.7891,502.5,318.7891,506.5,314.7891" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="91.5" x2="508.5" y1="314.7891" y2="314.7891"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="101" x="98.5" y="294.5903">pybabel extract</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="399" x="98.5" y="309.7231">extract messages, store messages.pot on translations branch</text><polygon fill="#A80036" points="726.5,385.3203,736.5,389.3203,726.5,393.3203,730.5,389.3203" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="514.5" x2="732.5" y1="389.3203" y2="389.3203"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="200" x="521.5" y="338.856">pybabel update (messages.po)</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="163" x="521.5" y="353.9888">git add searx/translations</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="69" x="521.5" y="369.1216">git commit</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="51" x="521.5" y="384.2544">git push</text><polygon fill="#FBFB77" filter="url(#fwj4fqzvak25j)" points="703,402.3203,773,402.3203,783,413.3203,773,425.3203,703,425.3203,693,413.3203,703,402.3203" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="66" x="705" y="418.3872">wlc unlock</text><rect fill="#EEEEEE" filter="url(#fwj4fqzvak25j)" height="3" style="stroke:#EEEEEE;stroke-width:1.0;" width="1001" x="0" y="465.0195"/><line style="stroke:#000000;stroke-width:1.0;" x1="0" x2="1001" y1="465.0195" y2="465.0195"/><line style="stroke:#000000;stroke-width:1.0;" x1="0" x2="1001" y1="468.0195" y2="468.0195"/><rect fill="#EEEEEE" filter="url(#fwj4fqzvak25j)" height="23.1328" style="stroke:#000000;stroke-width:2.0;" width="111" x="445" y="454.4531"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="92" x="451" y="470.52">every Friday</text><path d="M143,494.5859 L513,494.5859 L513,501.5859 L503,511.5859 L143,511.5859 L143,494.5859 " fill="#EEEEEE" style="stroke:#000000;stroke-width:1.0;"/><rect fill="none" height="324.7266" style="stroke:#000000;stroke-width:2.0;" width="848" x="143" y="494.5859"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="325" x="158" y="507.6528">.github/workflows/translations-update.yml</text><path d="M153,518.7188 L465,518.7188 L465,525.7188 L455,535.7188 L153,535.7188 L153,518.7188 " fill="#EEEEEE" style="stroke:#000000;stroke-width:1.0;"/><rect fill="none" height="245.3281" style="stroke:#000000;stroke-width:2.0;" width="828" x="153" y="518.7188"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="267" x="168" y="531.7856">make babel.translations.to.master</text><polygon fill="#FBFB77" filter="url(#fwj4fqzvak25j)" points="711,540.8516,765,540.8516,775,551.8516,765,563.8516,711,563.8516,701,551.8516,711,540.8516" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="50" x="713" y="556.9185">wlc lock</text><polygon fill="#A80036" points="726.5,586.1172,736.5,590.1172,726.5,594.1172,730.5,590.1172" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="514.5" x2="732.5" y1="590.1172" y2="590.1172"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="47" x="521.5" y="585.0513">wlc pull</text><polygon fill="#A80036" points="749.5,615.25,739.5,619.25,749.5,623.25,745.5,619.25" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="743.5" x2="898" y1="619.25" y2="619.25"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="74" x="755.5" y="614.1841">wlc commit</text><polygon fill="#A80036" points="525.5,644.3828,515.5,648.3828,525.5,652.3828,521.5,648.3828" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="519.5" x2="737.5" y1="648.3828" y2="648.3828"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="196" x="531.5" y="643.3169">git merge weblate/translations</text><polygon fill="#A80036" points="281.5,718.9141,271.5,722.9141,281.5,726.9141,277.5,722.9141" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="275.5" x2="513.5" y1="722.9141" y2="722.9141"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="105" x="287.5" y="672.4497">pybabel compile</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="134" x="287.5" y="687.5825">cp searx/translations</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="44" x="287.5" y="702.7153">git add</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="69" x="287.5" y="717.8481">git commit</text><polygon fill="#FBFB77" filter="url(#fwj4fqzvak25j)" points="703,735.9141,773,735.9141,783,746.9141,773,758.9141,703,758.9141,693,746.9141,703,735.9141" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="66" x="705" y="751.981">wlc unlock</text><polygon fill="#FBFB77" filter="url(#fwj4fqzvak25j)" points="175,776.0469,365,776.0469,375,795.0469,365,814.0469,175,814.0469,165,795.0469,175,776.0469" style="stroke:#A80036;stroke-width:1.0;"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="186" x="177" y="792.1138">create or update pull request</text><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="135" x="177" y="807.2466">"Update translations"</text><rect fill="#EEEEEE" filter="url(#fwj4fqzvak25j)" height="3" style="stroke:#EEEEEE;stroke-width:1.0;" width="1001" x="0" y="846.8789"/><line style="stroke:#000000;stroke-width:1.0;" x1="0" x2="1001" y1="846.8789" y2="846.8789"/><line style="stroke:#000000;stroke-width:1.0;" x1="0" x2="1001" y1="849.8789" y2="849.8789"/><rect fill="#EEEEEE" filter="url(#fwj4fqzvak25j)" height="23.1328" style="stroke:#000000;stroke-width:2.0;" width="168" x="416.5" y="836.3125"/><text fill="#000000" font-family="sans-serif" font-size="13" font-weight="bold" lengthAdjust="spacing" textLength="149" x="422.5" y="852.3794">developper's review</text><polygon fill="#A80036" points="102.5,886.5781,92.5,890.5781,102.5,894.5781,98.5,890.5781" style="stroke:#A80036;stroke-width:1.0;"/><line style="stroke:#A80036;stroke-width:1.0;" x1="96.5" x2="269.5" y1="890.5781" y2="890.5781"/><text fill="#000000" font-family="sans-serif" font-size="13" lengthAdjust="spacing" textLength="121" x="108.5" y="885.5122">merge pull request</text><!--MD5=[e3d4a82efdc738a59c5df45b59de0f06]
+@startuml
+participant "master branch" as master
+participant "translations_update branch" as translations_update
+participant "translations branch" as translations
+participant "weblate clone of\ntranslations branch" as weblate
+participant "weblate\npending changes " as weblate_change
+
+== for each commit on master ==
+
+group .github/workflows/integration.yml
+  group make babel.master.to.translations
+    hnote over weblate : wlc lock
+    translations -> weblate: wlc pull
+    weblate_change -> weblate: wlc commit
+    weblate -> translations: git merge weblate/translations
+    master -> translations: pybabel extract\nextract messages, store messages.pot on translations branch
+    translations -> weblate: pybabel update (messages.po)\ngit add searx/translations\ngit commit\ngit push
+    hnote over weblate : wlc unlock
+  end  
+end
+
+== every Friday ==
+
+group .github/workflows/translations-update.yml
+  group make babel.translations.to.master
+    hnote over weblate : wlc lock
+    translations -> weblate: wlc pull
+    weblate_change -> weblate: wlc commit
+    weblate -> translations: git merge weblate/translations
+    translations -> translations_update: pybabel compile\ncp searx/translations\ngit add\ngit commit
+    hnote over weblate : wlc unlock
+  end
+  hnote over translations_update : create or update pull request\n"Update translations"
+end
+
+==  developper's review ==
+
+translations_update -> master: merge pull request
+@enduml
+
+See
+https://plantuml.com/en/sequence-diagram
+https://www.planttext.com
+--></g></svg>