[mod] isolation of botdetection from the limiter

This patch was inspired by the discussion around PR-2882 [2].  The goals of this
patch are:

1. Convert plugin searx.plugin.limiter to normal code [1]
2. isolation of botdetection from the limiter [2]
3. searx/{tools => botdetection}/config.py and drop searx.tools
4. in URL /config, 'limiter.enabled' is true only if the limiter is really
   enabled (Redis is available).

This patch moves all the code that belongs to botdetection into namespace
searx.botdetection and code that belongs to limiter is placed in namespace
searx.limiter.

Tthe limiter used to be a plugin at some point botdetection was added, it was
not a plugin.  The modularization of these two components was long overdue.
With the clear modularization, the documentation could then also be organized
according to the architecture.

[1] https://github.com/searxng/searxng/pull/2882
[2] https://github.com/searxng/searxng/pull/2882#issuecomment-1741716891

To test:

- check the app works without the limiter, check `/config`
- check the app works with the limiter and with the token, check `/config`
- make docs.live .. and read
  - http://0.0.0.0:8000/admin/searx.limiter.html
  - http://0.0.0.0:8000/src/searx.botdetection.html#botdetection

Signed-off-by: Markus Heiser <markus.heiser@darmarit.de>

5 files changed, 35 insertions, 7 deletions

diff --git a/docs/admin/index.rst b/docs/admin/index.rst
index f335f1631..606b51c22 100644
--- a/docs/admin/index.rst
+++ b/docs/admin/index.rst
@@ -15,7 +15,7 @@ Administrator documentation
    installation-apache
    update-searxng
    answer-captcha
-   searx.botdetection
+   searx.limiter
    api
    architecture
    plugins
diff --git a/docs/admin/searx.limiter.rst b/docs/admin/searx.limiter.rst
new file mode 100644
index 000000000..c23635571
--- /dev/null
+++ b/docs/admin/searx.limiter.rst
@@ -0,0 +1,17 @@
+.. _limiter:
+
+=======
+Limiter
+=======
+
+.. sidebar:: info
+
+   The limiter requires a :ref:`Redis <settings redis>` database.
+
+.. contents::
+   :depth: 2
+   :local:
+   :backlinks: entry
+
+.. automodule:: searx.limiter
+   :members:
diff --git a/docs/admin/settings/settings_server.rst b/docs/admin/settings/settings_server.rst
index e2b4cb67d..ba0c9484d 100644
--- a/docs/admin/settings/settings_server.rst
+++ b/docs/admin/settings/settings_server.rst
@@ -36,11 +36,9 @@
 ``secret_key`` : ``$SEARXNG_SECRET``
   Used for cryptography purpose.
 
-.. _limiter:
-
 ``limiter`` :
   Rate limit the number of request on the instance, block some bots.  The
-  :ref:`limiter src` requires a :ref:`settings redis` database.
+  :ref:`limiter` requires a :ref:`settings redis` database.
 
 .. _image_proxy:
 
diff --git a/docs/conf.py b/docs/conf.py
index aa4905ef4..2ed85a800 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -2,6 +2,7 @@
 # SPDX-License-Identifier: AGPL-3.0-or-later
 
 import  sys, os
+from pathlib import Path
 from pallets_sphinx_themes import ProjectLink
 
 from searx import get_setting
@@ -13,7 +14,6 @@ project = 'SearXNG'
 copyright = 'SearXNG team'
 author = 'SearXNG team'
 release, version = VERSION_STRING, VERSION_STRING
-
 SEARXNG_URL = get_setting('server.base_url') or 'https://example.org/searxng'
 ISSUE_URL = get_setting('brand.issue_url')
 DOCS_URL = get_setting('brand.docs_url')
@@ -22,6 +22,9 @@ PRIVACYPOLICY_URL = get_setting('general.privacypolicy_url')
 CONTACT_URL = get_setting('general.contact_url')
 WIKI_URL = get_setting('brand.wiki_url')
 
+SOURCEDIR = Path(__file__).parent.parent / "searx"
+os.environ['SOURCEDIR'] = str(SOURCEDIR)
+
 # hint: sphinx.ext.viewcode won't highlight when 'highlight_language' [1] is set
 #       to string 'none' [2]
 #
diff --git a/docs/admin/searx.botdetection.rst b/docs/src/searx.botdetection.rst
index c89f1cd93..04cb81dfd 100644
--- a/docs/admin/searx.botdetection.rst
+++ b/docs/src/searx.botdetection.rst
@@ -12,8 +12,10 @@ Bot Detection
 .. automodule:: searx.botdetection
   :members:
 
-.. automodule:: searx.botdetection.limiter
-  :members:
+.. _botdetection ip_lists:
+
+IP lists
+========
 
 .. automodule:: searx.botdetection.ip_lists
   :members:
@@ -50,3 +52,11 @@ Probe HTTP headers
 
 .. automodule:: searx.botdetection.http_user_agent
   :members:
+
+.. _botdetection config:
+
+Config
+======
+
+.. automodule:: searx.botdetection.config
+  :members: