+ +
+

Favicons (source)

+ +

Implementations for providing the favicons in SearXNG

+
+
+searx.favicons.favicon_proxy()[source]
+

REST API of SearXNG’s favicon proxy service

+
/favicon_proxy?authority=<...>&h=<...>
+
+
+
+
authority:

Domain name RFC 3986 / see favicon_url

+
+
h:

HMAC RFC 2104, build up from the server.secret_key setting.

+
+
+
+ +
+
+searx.favicons.favicon_url(authority: str) → str[source]
+

Function to generate the image URL used for favicons in SearXNG’s result +lists. The authority argument (aka netloc / RFC 3986) is usually a +(sub-) domain name. This function is used in the HTML (jinja) templates.

+
<div class="favicon">
+   <img src="{{ favicon_url(result.parsed_url.netloc) }}">
+</div>
+
+
+

The returned URL is a route to favicon_proxy REST API.

+

If the favicon is already in the cache, the returned URL is a data URL +(something like data:image/png;base64,...). By generating a data url from +the cache.FaviconCache, additional HTTP roundtripps via the +favicon_proxy are saved. However, it must also be borne in mind +that data urls are not cached in the client (web browser).

+
+ +
+

Favicons Config

+
+
+class searx.favicons.config.FaviconConfig(cfg_schema: int, cache: ~searx.favicons.cache.FaviconCacheConfig = <factory>, proxy: ~searx.favicons.proxy.FaviconProxyConfig = <factory>)[source]
+

The class aggregates configurations of the favicon tools

+
+
+classmethod from_toml_file(cfg_file: Path, use_cache: bool) → FaviconConfig[source]
+

Create a config object from a TOML file, the use_cache argument +specifies whether a cache should be used.

+
+ +
+
+cache: FaviconCacheConfig
+

Setup of the cache.FaviconCacheConfig.

+
+ +
+
+cfg_schema: int
+

Config’s schema version. The specification of the version of the schema +is mandatory, currently only version CONFIG_SCHEMA is supported. +By specifying a version, it is possible to ensure downward compatibility in +the event of future changes to the configuration schema

+
+ +
+
+proxy: FaviconProxyConfig
+

Setup of the proxy.FaviconProxyConfig.

+
+ +
+ +
+
+searx.favicons.config.CONFIG_SCHEMA: int = 1
+

Version of the configuration schema.

+
+ +
+
+searx.favicons.config.TOML_CACHE_CFG: dict[str, FaviconConfig] = {}
+

Cache config objects by TOML’s filename.

+
+ +
+
+

Favicons Proxy

+

Implementations for a favicon proxy

+
+
+class searx.favicons.proxy.FaviconProxyConfig(max_age: int = 604800, secret_key: str = '', resolver_timeout: int = 3.0, resolver_map: dict[str, str] = <factory>, favicon_path: str = '/home/runner/work/searxng/searxng/searx/static/themes/{theme}/img/empty_favicon.svg', favicon_mime_type: str = 'image/svg+xml')[source]
+

Configuration of the favicon proxy.

+
+
+favicon(**replacements)[source]
+

Returns pathname and mimetype of the default favicon.

+
+ +
+
+favicon_data_url(**replacements)[source]
+

Returns data image URL of the default favicon.

+
+ +
+
+get_resolver(name: str) → Callable | None[source]
+

Returns the callable object (function) of the resolver with the +name. If no resolver is registered for the name, None is +returned.

+
+ +
+
+max_age: int
+

HTTP header Cache-Control max-age

+
+ +
+
+resolver_map: dict[str, str]
+

The resolver_map is a key / value dictionary where the key is the name of +the resolver and the value is the fully qualifying name (fqn) of resolver’s +function (the callable). The resolvers from the python module +searx.favicons.resolver are available by default.

+
+ +
+
+resolver_timeout: int
+

Timeout which the resolvers should not exceed, is usually passed to the +outgoing request of the resolver. By default, the value from +outgoing.request_timeout setting is used.

+
+ +
+
+secret_key: str
+

By default, the value from server.secret_key +setting is used.

+
+ +
+ +
+
+searx.favicons.proxy.favicon_proxy()[source]
+

REST API of SearXNG’s favicon proxy service

+
/favicon_proxy?authority=<...>&h=<...>
+
+
+
+
authority:

Domain name RFC 3986 / see favicon_url

+
+
h:

HMAC RFC 2104, build up from the server.secret_key setting.

+
+
+
+ +
+
+searx.favicons.proxy.favicon_url(authority: str) → str[source]
+

Function to generate the image URL used for favicons in SearXNG’s result +lists. The authority argument (aka netloc / RFC 3986) is usually a +(sub-) domain name. This function is used in the HTML (jinja) templates.

+
<div class="favicon">
+   <img src="{{ favicon_url(result.parsed_url.netloc) }}">
+</div>
+
+
+

The returned URL is a route to favicon_proxy REST API.

+

If the favicon is already in the cache, the returned URL is a data URL +(something like data:image/png;base64,...). By generating a data url from +the cache.FaviconCache, additional HTTP roundtripps via the +favicon_proxy are saved. However, it must also be borne in mind +that data urls are not cached in the client (web browser).

+
+ +
+
+searx.favicons.proxy.search_favicon(resolver: str, authority: str) → tuple[None | bytes, None | str][source]
+

Sends the request to the favicon resolver and returns a tuple for the +favicon. The tuple consists of (data, mime), if the resolver has not +determined a favicon, both values are None.

+
+
data:

Binary data of the favicon.

+
+
mime:

Mime type of the favicon.

+
+
+
+ +
+
+

Favicons Resolver

+

Implementations of the favicon resolvers that are available in the favicon +proxy by default. A resolver is a function that obtains the favicon from an +external source. The resolver function receives two arguments (domain, +timeout) and returns a tuple (data, mime).

+
+
+searx.favicons.resolvers.allesedv(domain: str, timeout: int) → tuple[None | bytes, None | str][source]
+

Favicon Resolver from allesedv.com / https://favicon.allesedv.com/

+
+ +
+
+searx.favicons.resolvers.duckduckgo(domain: str, timeout: int) → tuple[None | bytes, None | str][source]
+

Favicon Resolver from duckduckgo.com / https://blog.jim-nielsen.com/2021/displaying-favicons-for-any-domain/

+
+ +
+
+searx.favicons.resolvers.google(domain: str, timeout: int) → tuple[None | bytes, None | str][source]
+

Favicon Resolver from google.com

+
+ +
+
+searx.favicons.resolvers.yandex(domain: str, timeout: int) → tuple[None | bytes, None | str][source]
+

Favicon Resolver from yandex.com

+
+ +
+
+

Favicons Cache

+

Implementations for caching favicons.

+
+
FaviconCacheConfig:

Configuration of the favicon cache

+
+
FaviconCache:

Abstract base class for the implementation of a favicon cache.

+
+
FaviconCacheSQLite:

Favicon cache that manages the favicon BLOBs in a SQLite DB.

+
+
FaviconCacheNull:

Fallback solution if the configured cache cannot be used for system reasons.

+
+
+
+
+
+class searx.favicons.cache.FaviconCache(cfg: FaviconCacheConfig)[source]
+

Abstract base class for the implementation of a favicon cache.

+
+
+abstract maintenance(force=False)[source]
+

Performs maintenance on the cache

+
+ +
+
+abstract set(resolver: str, authority: str, mime: str | None, data: bytes | None) → bool[source]
+

Set data and mime-type in the cache. If data is None, the +FALLBACK_ICON is registered. in the cache.

+
+ +
+
+abstract state() → FaviconCacheStats[source]
+

Returns a FaviconCacheStats (key/values) with information +on the state of the cache.

+
+ +
+ +
+
+class searx.favicons.cache.FaviconCacheConfig(db_type: Literal['sqlite', 'mem'] = 'sqlite', db_url: str = '/tmp/faviconcache.db', HOLD_TIME: int = 2592000, LIMIT_TOTAL_BYTES: int = 52428800, BLOB_MAX_BYTES: int = 20480, MAINTENANCE_PERIOD: int = 3600, MAINTENANCE_MODE: Literal['auto', 'off'] = 'auto')[source]
+

Configuration of the favicon cache.

+
+
+BLOB_MAX_BYTES: int
+

The maximum BLOB size in bytes that a favicon may have so that it can be +saved in the cache. If the favicon is larger, it is not saved in the cache +and must be requested by the client via the proxy.

+
+ +
+
+HOLD_TIME: int
+

Hold time (default in sec.), after which a BLOB is removed from the cache.

+
+ +
+
+LIMIT_TOTAL_BYTES: int
+

Maximum of bytes (default) stored in the cache of all blobs. Note: The +limit is only reached at each maintenance interval after which the oldest +BLOBs are deleted; the limit is exceeded during the maintenance period. If +the maintenance period is too long or maintenance is switched off +completely, the cache grows uncontrollably.

+
+ +
+
+MAINTENANCE_MODE: Literal['auto', 'off']
+

Type of maintenance mode

+
+
auto:

Maintenance is carried out automatically as part of the maintenance +intervals (MAINTENANCE_PERIOD); no external process is required.

+
+
off:

Maintenance is switched off and must be carried out by an external process +if required.

+
+
+
+ +
+
+MAINTENANCE_PERIOD: int
+

Maintenance period in seconds / when MAINTENANCE_MODE is set to +auto.

+
+ +
+
+db_type: Literal['sqlite', 'mem']
+

Type of the database:

+
+
sqlite:

cache.FaviconCacheSQLite

+
+
mem:

cache.FaviconCacheMEM (not recommended)

+
+
+
+ +
+
+db_url: str
+

URL of the SQLite DB, the path to the database file.

+
+ +
+ +
+
+class searx.favicons.cache.FaviconCacheMEM(cfg)[source]
+

Favicon cache in process’ memory. Its just a POC that stores the +favicons in the memory of the process.

+
+

Attention

+

Don’t use it in production, it will blow up your memory!!

+
+
+
+maintenance(force=False)[source]
+

Performs maintenance on the cache

+
+ +
+
+set(resolver: str, authority: str, mime: str | None, data: bytes | None) → bool[source]
+

Set data and mime-type in the cache. If data is None, the +FALLBACK_ICON is registered. in the cache.

+
+ +
+
+state()[source]
+

Returns a FaviconCacheStats (key/values) with information +on the state of the cache.

+
+ +
+ +
+
+class searx.favicons.cache.FaviconCacheNull(cfg: FaviconCacheConfig)[source]
+

A dummy favicon cache that caches nothing / a fallback solution. The +NullCache is used when more efficient caches such as the +FaviconCacheSQLite cannot be used because, for example, the SQLite +library is only available in an old version and does not meet the +requirements.

+
+
+maintenance(force=False)[source]
+

Performs maintenance on the cache

+
+ +
+
+set(resolver: str, authority: str, mime: str | None, data: bytes | None) → bool[source]
+

Set data and mime-type in the cache. If data is None, the +FALLBACK_ICON is registered. in the cache.

+
+ +
+
+state()[source]
+

Returns a FaviconCacheStats (key/values) with information +on the state of the cache.

+
+ +
+ +
+
+class searx.favicons.cache.FaviconCacheSQLite(cfg: FaviconCacheConfig)[source]
+

Favicon cache that manages the favicon BLOBs in a SQLite DB. The DB +model in the SQLite DB is implemented using the abstract class +sqlitedb.SQLiteAppl.

+

The following configurations are required / supported:

+ +
+
+maintenance(force=False)[source]
+

Performs maintenance on the cache

+
+ +
+
+set(resolver: str, authority: str, mime: str | None, data: bytes | None) → bool[source]
+

Set data and mime-type in the cache. If data is None, the +FALLBACK_ICON is registered. in the cache.

+
+ +
+
+state() → FaviconCacheStats[source]
+

Returns a FaviconCacheStats (key/values) with information +on the state of the cache.

+
+ +
+
+DB_SCHEMA: int = 1
+

As soon as changes are made to the DB schema, the version number must be +increased. Changes to the version number require the DB to be recreated (or +migrated / if an migration path exists and is implemented).

+
+ +
+
+DDL_BLOBS = 'CREATE TABLE IF NOT EXISTS blobs (\n  sha256     TEXT,\n  bytes_c    INTEGER,\n  mime       TEXT NOT NULL,\n  data       BLOB NOT NULL,\n  PRIMARY KEY (sha256))'
+

Table to store BLOB objects by their sha256 hash values.

+
+ +
+
+DDL_BLOB_MAP = "CREATE TABLE IF NOT EXISTS blob_map (\n    m_time     INTEGER DEFAULT (strftime('%s', 'now')),  -- last modified (unix epoch) time in sec.\n    sha256     TEXT,\n    resolver   TEXT,\n    authority  TEXT,\n    PRIMARY KEY (resolver, authority))"
+

Table to map from (resolver, authority) to sha256 hash values.

+
+ +
+
+SQL_DROP_LEFTOVER_BLOBS = 'DELETE FROM blobs WHERE sha256 IN ( SELECT b.sha256   FROM blobs b   LEFT JOIN blob_map bm     ON b.sha256 = bm.sha256  WHERE bm.sha256 IS NULL)'
+

Delete blobs.sha256 (BLOBs) no longer in blob_map.sha256.

+
+ +
+
+property next_maintenance_time: int
+

Returns (unix epoch) time of the next maintenance.

+
+ +
+ +
+
+class searx.favicons.cache.FaviconCacheStats(favicons: int | None = None, bytes: int | None = None, domains: int | None = None, resolvers: int | None = None)[source]
+

Dataclass wich provides information on the status of the cache.

+
+ +
+
+searx.favicons.cache.init(cfg: FaviconCacheConfig)[source]
+

Initialization of a global CACHE

+
+ +
+
+searx.favicons.cache.maintenance(force: bool = True, debug: bool = False)[source]
+

perform maintenance of the cache

+
+ +
+
+searx.favicons.cache.state()[source]
+

show state of the cache

+
+ +
+
+ + +
+