diff options
| -rw-r--r-- | docs/admin/index.rst | 1 | ||||
| -rw-r--r-- | docs/admin/searx.botdetection.rst (renamed from docs/src/searx.botdetection.rst) | 4 | ||||
| -rw-r--r-- | searx/botdetection/__init__.py | 17 |
3 files changed, 22 insertions, 0 deletions
diff --git a/docs/admin/index.rst b/docs/admin/index.rst index 583b6b428..f335f1631 100644 --- a/docs/admin/index.rst +++ b/docs/admin/index.rst @@ -15,6 +15,7 @@ Administrator documentation installation-apache update-searxng answer-captcha + searx.botdetection api architecture plugins diff --git a/docs/src/searx.botdetection.rst b/docs/admin/searx.botdetection.rst index 48261ceac..c89f1cd93 100644 --- a/docs/src/searx.botdetection.rst +++ b/docs/admin/searx.botdetection.rst @@ -19,6 +19,8 @@ Bot Detection :members: +.. _botdetection rate limit: + Rate limit ========== @@ -29,6 +31,8 @@ Rate limit :members: +.. _botdetection probe headers: + Probe HTTP headers ================== diff --git a/searx/botdetection/__init__.py b/searx/botdetection/__init__.py index fcd8e5630..74f6c4263 100644 --- a/searx/botdetection/__init__.py +++ b/searx/botdetection/__init__.py @@ -2,6 +2,23 @@ # lint: pylint """.. _botdetection src: +The :ref:`limiter <limiter src>` implements several methods to block bots: + +a. Analysis of the HTTP header in the request / can be easily bypassed. + +b. Block and pass lists in which IPs are listed / difficult to maintain, since + the IPs of bots are not all known and change over the time. + +c. Detection of bots based on the behavior of the requests and blocking and, if + necessary, unblocking of the IPs via a dynamically changeable IP block list. + +For dynamically changeable IP lists a Redis database is needed and for any kind +of IP list the determination of the IP of the client is essential. The IP of +the client is determined via the X-Forwarded-For_ HTTP header + +.. _X-Forwarded-For: + https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-Forwarded-For + X-Forwarded-For =============== |