diff options
| author | Árni Dagur <arni@dagur.eu> | 2020-12-19 20:30:17 +0000 |
|---|---|---|
| committer | Árni Dagur <arni@dagur.eu> | 2020-12-19 20:30:17 +0000 |
| commit | 729d6c9d8fd2c8420f97a2a80eca2875c0af5126 (patch) | |
| tree | 114321b7ce68a3ef233016c340ab4c1c61b63152 | |
| parent | c72e181ed3b2eb2f28c4a7eb97eba10c5b91f157 (diff) | |
| parent | 8e7f24bc0c6cdb333fef1a4a72d659aaeb7bbf09 (diff) | |
| download | qutebrowser-729d6c9d8fd2c8420f97a2a80eca2875c0af5126.tar.gz qutebrowser-729d6c9d8fd2c8420f97a2a80eca2875c0af5126.zip | |
Merge branch 'master' into more-sophisticated-adblock
71 files changed, 773 insertions, 270 deletions
diff --git a/.bumpversion.cfg b/.bumpversion.cfg index fa5e2a66e..2e62dc15b 100644 --- a/.bumpversion.cfg +++ b/.bumpversion.cfg @@ -1,5 +1,5 @@ [bumpversion] -current_version = 1.14.0 +current_version = 1.14.1 commit = True message = Release v{new_version} tag = True diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8e1f1341f..fbab9ee9c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -90,7 +90,7 @@ jobs: - uses: actions/checkout@v2 - name: Set up problem matchers run: "python scripts/dev/ci/problemmatchers.py py38 ${{ runner.temp }}" - - run: tox -e py38 + - run: tox -e py tests: if: "!contains(github.event.head_commit.message, '[ci skip]')" @@ -112,6 +112,10 @@ jobs: - testenv: py38-pyqt514 os: ubuntu-20.04 python: 3.8 + ### PyQt 5.15.0 (Python 3.9) + - testenv: py39-pyqt5150 + os: ubuntu-20.04 + python: 3.9 ### PyQt 5.15 (Python 3.9, with coverage) - testenv: py39-pyqt515-cov os: ubuntu-20.04 @@ -173,14 +177,6 @@ jobs: steps: - name: Checkout repository uses: actions/checkout@v2 - with: - # We must fetch at least the immediate parents so that if this is - # a pull request then we can checkout the head. - fetch-depth: 2 - # If this run was triggered by a pull request event, then checkout - # the head of the pull request instead of the merge commit. - - run: git checkout HEAD^2 - if: ${{ github.event_name == 'pull_request' }} - name: Initialize CodeQL uses: github/codeql-action/init@v1 with: diff --git a/.github/workflows/docker.yml b/.github/workflows/docker.yml index 03510ad6e..06707eb3f 100644 --- a/.github/workflows/docker.yml +++ b/.github/workflows/docker.yml @@ -7,6 +7,7 @@ on: jobs: docker: + if: "github.repository == 'qutebrowser/qutebrowser'" runs-on: ubuntu-20.04 strategy: matrix: diff --git a/.gitignore b/.gitignore index 50c67dee4..31c4ca3b7 100644 --- a/.gitignore +++ b/.gitignore @@ -35,6 +35,7 @@ Sessionx.vim /.pytest_cache /.testmondata /.hypothesis +/.benchmarks .mypy_cache /prof /venv diff --git a/README.asciidoc b/README.asciidoc index 203327127..2e70257d3 100644 --- a/README.asciidoc +++ b/README.asciidoc @@ -125,12 +125,15 @@ The following software and libraries are required to run qutebrowser: sensitive data.** * https://www.riverbankcomputing.com/software/pyqt/intro[PyQt] 5.12.0 or newer for Python 3 -* https://pypi.python.org/pypi/setuptools/[pkg_resources/setuptools] +* https://pypi.python.org/pypi/setuptools/[pkg_resources/setuptools] (being + phased out for qutebrowser v2.0.0) * https://fdik.org/pyPEG/[pyPEG2] * http://jinja.pocoo.org/[jinja2] * http://pygments.org/[pygments] * https://github.com/yaml/pyyaml[PyYAML] * https://www.attrs.org/[attrs] +* https://importlib-resources.readthedocs.io/[importlib_resources] (on Python + 3.8 or older) The following libraries are optional: diff --git a/doc/changelog.asciidoc b/doc/changelog.asciidoc index 9cfa73806..5a0ce3938 100644 --- a/doc/changelog.asciidoc +++ b/doc/changelog.asciidoc @@ -42,6 +42,12 @@ Major changes still relying on it. The `cssutils` project is also dead upstream, with its repository being gone after Bitbucket https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket[removed Mercurial support]. +- TODO: The former dependency on the `pkg_resources` module (part of the + `setuptools` project) got dropped. +- A new dependency on the `importlib_resources` module got introduced for + Python versions up to and including 3.8. Note that the stdlib + `importlib.resources` module for Python 3.7 and 3.8 is missing the needed APIs, + thus requiring the backports for those versions as well. Removed ~~~~~~~ @@ -64,6 +70,12 @@ Added settings might stop working. As a (currently undocumented) escape hatch, this version adds a `QUTE_DARKMODE_VARIANT=qt_515_2` environment variable which can be set to get the correct behavior in (transitive) situations like this. +- New `--desktop-file-name` commandline argument, which can be used to customize + the desktop filename passed to Qt (which is used to set the `app_id` on + Wayland). +- New userscripts: + - `kodi` to play videos in Kodi + - `qr` to generate a QR code of the current URL Changed ~~~~~~~ @@ -83,14 +95,30 @@ Changed pre-selected in the prompt shown by qutebrowser. - URLs such as `::1/foo` are now handled as a search term or local file rather than IPv6. Use `[::1]/foo` to force parsing as IPv6 instead. +- The `mkvenv.py` script now runs a "smoke test" after setting up the virtual + environment to ensure it's working as expected. If necessary, the test can be + skipped via a new `--skip-smoke-test` flag. +- Both qutebrowser userscripts and Greasemonkey scripts are now additionally + picked up from qutebrowser's config directory (the `userscripts` and + `greasemonkey` subdirectories of e.g. `~/.config/qutebrowser/`) rather than only + the data directory (the same subdirectories of e.g. + `~/.local/share/qutebrowser/`). +- The `:later` command now understands a time specification like `5m` or + `1h5m2s`, rather than just taking milliseconds. Fixed ~~~~~ - With interpolated color settings (`colors.tabs.indicator.*` and `colors.downloads.*`), the alpha channel is now handled correctly. - -v1.14.1 (unreleased) +- The `format_json` userscript now uses `env` in its shebang, making it work + correctly on systems where `bash` isn't located in `/bin`. +- TODO: Due to a long-standing bug in the `pkg_resources` dependency, it caused + qutebrowser's startup to slow down by around 150ms-1s (heavily depending on + the system). Since the dependency is now removed, qutebrowser's startup time + thus improved. + +v1.14.1 (2020-12-04) -------------------- Added @@ -178,8 +206,17 @@ Fixed installed, it was suggested to install `qt5-webengine-devtools`, which does not, in fact, exist. It's now correctly suggested to install `qt5-qtwebengine-devtools` instead. +- With Qt 5.15.2, lines/borders coming from the `readability-js` userscript + were invisible. This is now fixed by changing the border color to grey (with all + Qt versions). +- Due to changes in the underlying Chromium, the + `colors.webpage.prefers_color_scheme_dark` setting broke with Qt 5.15.2. It now + works properly again. +- A bug in the `pkg_resources` module used by qutebrowser caused deprecation + warnings to appear on start with Python 3.9 on some setups. Those are now + hidden. - Minor performance improvements. -- (TODO) Fix for various functionality breaking in private windows with v1.14.0, +- Fix for various functionality breaking in private windows with v1.14.0, after the last private window is closed. This includes: * Ad blocking * Downloads diff --git a/doc/help/commands.asciidoc b/doc/help/commands.asciidoc index 3103bfa15..f60f30169 100644 --- a/doc/help/commands.asciidoc +++ b/doc/help/commands.asciidoc @@ -597,7 +597,7 @@ Syntax: +:greasemonkey-reload [*--force*]+ Re-read Greasemonkey scripts from disk. -The scripts are read from a 'greasemonkey' subdirectory in qutebrowser's data directory (see `:version`). +The scripts are read from a 'greasemonkey' subdirectory in qutebrowser's data or config directories (see `:version`). ==== optional arguments * +*-f*+, +*--force*+: For any scripts that have required dependencies, re-download them. @@ -784,12 +784,12 @@ Jump to the mark named by `key`. [[later]] === later -Syntax: +:later 'ms' 'command'+ +Syntax: +:later 'duration' 'command'+ Execute a command after some time. ==== positional arguments -* +'ms'+: How many milliseconds to wait. +* +'duration'+: Duration to wait in format XhYmZs or a number for milliseconds. * +'command'+: The command to run, with optional args. ==== note @@ -1308,7 +1308,8 @@ Note that the command is *not* run in a shell, so things like `$VAR` or `> outpu * +*-v*+, +*--verbose*+: Show notifications when the command started/exited. * +*-o*+, +*--output*+: Show the output in a new tab. * +*-m*+, +*--output-messages*+: Show the output as messages. -* +*-d*+, +*--detach*+: Whether the command should be detached from qutebrowser. +* +*-d*+, +*--detach*+: Detach the command from qutebrowser so that it continues running when qutebrowser quits. + ==== count Given to userscripts as $QUTE_COUNT. diff --git a/doc/qutebrowser.1.asciidoc b/doc/qutebrowser.1.asciidoc index 777eddc65..83e7986bc 100644 --- a/doc/qutebrowser.1.asciidoc +++ b/doc/qutebrowser.1.asciidoc @@ -62,6 +62,9 @@ show it. *--backend* '{webkit,webengine}':: Which backend to use. +*--desktop-file-name* 'DESKTOP_FILE_NAME':: + Set the base name of the desktop entry for this application. Used to set the app_id under Wayland. See https://doc.qt.io/qt-5/qguiapplication.html#desktopFileName-prop + === debug arguments *-l* '{critical,error,warning,info,debug,vdebug}', *--loglevel* '{critical,error,warning,info,debug,vdebug}':: Override the configured console loglevel diff --git a/doc/userscripts.asciidoc b/doc/userscripts.asciidoc index 2dc34402d..9bbc68ce0 100644 --- a/doc/userscripts.asciidoc +++ b/doc/userscripts.asciidoc @@ -18,7 +18,7 @@ mpv, a simple key binding to something like `:spawn mpv {url}` should suffice. Also note userscripts need to have the executable bit set (`chmod +x`) for qutebrowser to run them. -To call a userscript, it needs to be stored in your data directory under +To call a userscript, it needs to be stored in your config or data directory under `userscripts` (for example: `~/.local/share/qutebrowser/userscripts/myscript`), or just use an absolute path. diff --git a/misc/org.qutebrowser.qutebrowser.appdata.xml b/misc/org.qutebrowser.qutebrowser.appdata.xml index f0885b8ed..601093418 100644 --- a/misc/org.qutebrowser.qutebrowser.appdata.xml +++ b/misc/org.qutebrowser.qutebrowser.appdata.xml @@ -44,6 +44,7 @@ </content_rating> <releases> <!-- Add new releases here --> +<release version="1.14.1" date="2020-12-04"/> <release version="1.14.0" date="2020-10-15"/> <release version="1.13.1" date="2020-07-17"/> <release version="1.13.0" date="2020-06-26"/> diff --git a/misc/requirements/requirements-check-manifest.txt b/misc/requirements/requirements-check-manifest.txt index 8b8f6ba1a..f2a5f00dd 100644 --- a/misc/requirements/requirements-check-manifest.txt +++ b/misc/requirements/requirements-check-manifest.txt @@ -2,8 +2,7 @@ build==0.1.0 check-manifest==0.45 -packaging==20.4 +packaging==20.8 pep517==0.9.1 pyparsing==2.4.7 -six==1.15.0 toml==0.10.2 diff --git a/misc/requirements/requirements-dev.txt b/misc/requirements/requirements-dev.txt index 6bc4325cf..f58af8072 100644 --- a/misc/requirements/requirements-dev.txt +++ b/misc/requirements/requirements-dev.txt @@ -1,17 +1,17 @@ # This file is automatically generated by scripts/dev/recompile_requirements.py bump2version==1.0.1 -certifi==2020.11.8 +certifi==2020.12.5 cffi==1.14.4 chardet==3.0.4 colorama==0.4.4 -cryptography==3.2.1 +cryptography==3.3.1 github3.py==1.3.0 hunter==3.3.1 idna==2.10 jwcrypto==0.8 manhole==1.6.0 -packaging==20.4 +packaging==20.8 pycparser==2.20 Pympler==0.9 pyparsing==2.4.7 diff --git a/misc/requirements/requirements-flake8.txt b/misc/requirements/requirements-flake8.txt index d77bdbbf5..6ed02ad61 100644 --- a/misc/requirements/requirements-flake8.txt +++ b/misc/requirements/requirements-flake8.txt @@ -6,14 +6,14 @@ flake8-bugbear==20.11.1 flake8-builtins==1.5.3 flake8-comprehensions==3.3.0 flake8-copyright==0.2.2 -flake8-debugger==3.2.1 +flake8-debugger==4.0.0 flake8-deprecated==1.3 flake8-docstrings==1.5.0 flake8-future-import==0.4.6 flake8-mock==0.3 flake8-polyfill==1.0.2 flake8-string-format==0.3.0 -flake8-tidy-imports==4.1.0 +flake8-tidy-imports==4.2.0 flake8-tuple==0.4.1 mccabe==0.6.1 pep8-naming==0.11.1 diff --git a/misc/requirements/requirements-mypy.txt b/misc/requirements/requirements-mypy.txt index b564cfa18..d640851c9 100644 --- a/misc/requirements/requirements-mypy.txt +++ b/misc/requirements/requirements-mypy.txt @@ -4,12 +4,12 @@ diff-cover==4.0.1 inflect==5.0.2 Jinja2==2.11.2 jinja2-pluralize==0.3.0 -lxml==4.6.1 +lxml==4.6.2 MarkupSafe==1.1.1 mypy==0.790 mypy-extensions==0.4.3 pluggy==0.13.1 -Pygments==2.7.2 +Pygments==2.7.3 -e git+https://github.com/stlehmann/PyQt5-stubs.git@704207e90bee7b36ec9861dfa6b39f06a27c6718#egg=PyQt5_stubs typed-ast==1.4.1 typing-extensions==3.7.4.3 diff --git a/misc/requirements/requirements-pylint.txt b/misc/requirements/requirements-pylint.txt index e3856a40a..02165c497 100644 --- a/misc/requirements/requirements-pylint.txt +++ b/misc/requirements/requirements-pylint.txt @@ -1,10 +1,10 @@ # This file is automatically generated by scripts/dev/recompile_requirements.py astroid==2.3.3 # rq.filter: < 2.4 -certifi==2020.11.8 +certifi==2020.12.5 cffi==1.14.4 chardet==3.0.4 -cryptography==3.2.1 +cryptography==3.3.1 github3.py==1.3.0 idna==2.10 isort==4.3.21 diff --git a/misc/requirements/requirements-pyqt-5.15.0.txt b/misc/requirements/requirements-pyqt-5.15.0.txt new file mode 100644 index 000000000..b9ee53f65 --- /dev/null +++ b/misc/requirements/requirements-pyqt-5.15.0.txt @@ -0,0 +1,5 @@ +# This file is automatically generated by scripts/dev/recompile_requirements.py + +PyQt5==5.15.0 # rq.filter: == 5.15.0 +PyQt5-sip==12.8.1 +PyQtWebEngine==5.15.0 # rq.filter: == 5.15.0 diff --git a/misc/requirements/requirements-pyqt-5.15.0.txt-raw b/misc/requirements/requirements-pyqt-5.15.0.txt-raw new file mode 100644 index 000000000..12d6adb7d --- /dev/null +++ b/misc/requirements/requirements-pyqt-5.15.0.txt-raw @@ -0,0 +1,4 @@ +#@ filter: PyQt5 == 5.15.0 +#@ filter: PyQtWebEngine == 5.15.0 +PyQt5 == 5.15.0 +PyQtWebEngine == 5.15.0 diff --git a/misc/requirements/requirements-pyroma.txt b/misc/requirements/requirements-pyroma.txt index 1a2dbde7f..d0568a7df 100644 --- a/misc/requirements/requirements-pyroma.txt +++ b/misc/requirements/requirements-pyroma.txt @@ -1,5 +1,5 @@ # This file is automatically generated by scripts/dev/recompile_requirements.py docutils==0.16 -Pygments==2.7.2 +Pygments==2.7.3 pyroma==2.6 diff --git a/misc/requirements/requirements-qutebrowser.txt-raw b/misc/requirements/requirements-qutebrowser.txt-raw index b43ee7f6e..2d527aeef 100644 --- a/misc/requirements/requirements-qutebrowser.txt-raw +++ b/misc/requirements/requirements-qutebrowser.txt-raw @@ -5,3 +5,6 @@ PyYAML colorama attrs adblock # Optional, for improved adblocking +importlib-resources + +#@ markers: importlib-resources python_version<"3.9" diff --git a/misc/requirements/requirements-sphinx.txt b/misc/requirements/requirements-sphinx.txt index 463bb8e73..2cbf8d57d 100644 --- a/misc/requirements/requirements-sphinx.txt +++ b/misc/requirements/requirements-sphinx.txt @@ -2,19 +2,18 @@ alabaster==0.7.12 Babel==2.9.0 -certifi==2020.11.8 +certifi==2020.12.5 chardet==3.0.4 docutils==0.16 idna==2.10 imagesize==1.2.0 Jinja2==2.11.2 MarkupSafe==1.1.1 -packaging==20.4 -Pygments==2.7.2 +packaging==20.8 +Pygments==2.7.3 pyparsing==2.4.7 pytz==2020.4 requests==2.25.0 -six==1.15.0 snowballstemmer==2.0.0 Sphinx==3.3.1 sphinxcontrib-applehelp==1.0.2 diff --git a/misc/requirements/requirements-tests.txt b/misc/requirements/requirements-tests.txt index 8b4eb0fba..28fd44126 100644 --- a/misc/requirements/requirements-tests.txt +++ b/misc/requirements/requirements-tests.txt @@ -3,9 +3,9 @@ apipkg==1.5 attrs==20.3.0 beautifulsoup4==4.9.3 -certifi==2020.11.8 +certifi==2020.12.5 chardet==3.0.4 -cheroot==8.4.7 +cheroot==8.5.1 click==7.1.2 # colorama==0.4.4 coverage==5.3 @@ -15,7 +15,7 @@ filelock==3.0.12 Flask==1.1.2 glob2==0.7 hunter==3.3.1 -hypothesis==5.41.3 +hypothesis==5.43.3 icdiff==1.9.1 idna==2.10 iniconfig==1.1.1 @@ -26,17 +26,17 @@ Mako==1.1.3 manhole==1.6.0 # MarkupSafe==1.1.1 more-itertools==8.6.0 -packaging==20.4 +packaging==20.8 parse==1.18.0 parse-type==0.5.2 pluggy==0.13.1 pprintpp==0.4.0 -py==1.9.0 +py==1.10.0 py-cpuinfo==7.0.0 -Pygments==2.7.2 +Pygments==2.7.3 pyparsing==2.4.7 -pytest==6.1.2 -pytest-bdd==4.0.1 +pytest==6.2.0 +pytest-bdd==4.0.2 pytest-benchmark==3.2.3 pytest-clarity==0.3.0a0 pytest-cov==2.10.1 @@ -54,7 +54,7 @@ requests==2.25.0 requests-file==1.5.1 six==1.15.0 sortedcontainers==2.3.0 -soupsieve==2.0.1 +soupsieve==2.1 termcolor==1.1.0 tldextract==3.1.0 toml==0.10.2 diff --git a/misc/requirements/requirements-tox.txt b/misc/requirements/requirements-tox.txt index 95dbdd654..c72393868 100644 --- a/misc/requirements/requirements-tox.txt +++ b/misc/requirements/requirements-tox.txt @@ -3,11 +3,11 @@ appdirs==1.4.4 distlib==0.3.1 filelock==3.0.12 -packaging==20.4 +packaging==20.8 pluggy==0.13.1 -py==1.9.0 +py==1.10.0 pyparsing==2.4.7 six==1.15.0 toml==0.10.2 tox==3.20.1 -virtualenv==20.2.1 +virtualenv==20.2.2 diff --git a/misc/userscripts/README.md b/misc/userscripts/README.md index a17f7164c..669bfa664 100644 --- a/misc/userscripts/README.md +++ b/misc/userscripts/README.md @@ -24,7 +24,7 @@ The following userscripts are included in the current directory. - [qutedmenu](./qutedmenu): Handle open -s && open -t with bemenu. - [readability](./readability): Executes python-readability on current page and opens the summary as new tab. -- [readability-js](./readability-js): Processes the current page with the readability +- [readability-js](./readability-js): Processes the current page with the readability library used in Firefox Reader View and opens the summary as new tab. - [ripbang](./ripbang): Adds DuckDuckGo bang as searchengine. - [rss](./rss): Keeps track of URLs in RSS feeds and opens new ones. @@ -32,6 +32,9 @@ The following userscripts are included in the current directory. - [tor_identity](./tor_identity): Change your tor identity. - [view_in_mpv](./view_in_mpv): Views the current web page in mpv using sensible mpv-flags. +- [qr](./qr): Show a QR code for the current webpage via + [qrencode](https://fukuchi.org/works/qrencode/). +- [kodi](./kodi): Play videos in Kodi. [castnow]: https://github.com/xat/castnow [youtube-dl]: https://rg3.github.io/youtube-dl/ @@ -67,6 +70,8 @@ The following userscripts can be found on their own repositories. and retrieve they when you want. - [doi](https://github.com/cadadr/configuration/blob/master/qutebrowser/userscripts/doi): Opens DOIs on Sci-Hub. +- [1password](https://github.com/tomoakley/dotfiles/blob/master/qutebrowser/userscripts/1password): + Integration with 1password on macOS. [Zotero]: https://www.zotero.org/ [Pocket]: https://getpocket.com/ diff --git a/misc/userscripts/format_json b/misc/userscripts/format_json index 541408c70..8a83c25fa 100755 --- a/misc/userscripts/format_json +++ b/misc/userscripts/format_json @@ -1,4 +1,4 @@ -#!/bin/bash +#!/usr/bin/env bash set -euo pipefail # # Behavior: diff --git a/misc/userscripts/kodi b/misc/userscripts/kodi new file mode 100755 index 000000000..63fcc81fe --- /dev/null +++ b/misc/userscripts/kodi @@ -0,0 +1,111 @@ +#!/usr/bin/env bash +# +# Behavior: +# A qutebrowser userscript that plays Twitch, YouTube or Vimeo videos in Kodi via its +# API. +# +# Requirements: +# awk +# bash +# curl +# +# Kodi setup: +# Settings -> Services -> Control +# enable 'Allow remote control via HTTP' +# set Username and Password +# enable 'Allow remote control from applications on this system' +# Optional yet recommended, setup SSL within Kodi over via a proxy webserver +# +# userscript setup: +# create ~/.config/qutebrowser/kodi_rc with host and authentication information like: +# +# HOST="http://127.0.0.1:8080" +# or +# HOST="https://kodi.example.com" +# +# AUTH="user:password" +# or +# AUTH="bas64authenticationinformation" +# +# The base64 authentication is the output of +# `echo -ne "user:password" |base64 --wrap 0` +# reminder base64 is not encryption +# +# For vim users you might want to add '# vim: set nospell filetype=bash' to the +# kodi_rc file. +# +# qutebrowser setup: +# in ~/.config/qutebrowser/config.py add something like +# +# to send video link via hints: +# config.bind('X', 'hint links userscript kodi') +# to send current URL: +# config.bind('X', 'spawn --userscript kodi') +# +# troubleshooting: +# Errors detected within this userscript with have an exit of 231. All other exit +# codes will come from curl or awk. To test that the kodi_rc file is set up +# correctly, run the following command. It will display a 'It works!' notification within Kodi. +# +# source ~/.config/qutebrowser/kodi_rc ; curl --request POST "$HOST"/jsonrpc --header "Authorization: Basic $AUTH" --header "Content-Type: application/json" --data '{"id":1,"jsonrpc":"2.0","method":"GUI.ShowNotification","params":{"title":"It works!","message":"both HOST and AUTH are correct"}}' +# +# In case you miss the notification in Kodi the successful response is: +# +# {"id":1,"jsonrpc":"2.0","result":"OK"} +# +# Note, curl will display errors for some problems, but not all. + +if [[ -z "$QUTE_FIFO" ]] ; then + echo "This script is designed to run as a qutebrowser userscript, not as a standalone script." + exit 231 +fi + +# configuration loading adapted from the password_fill userscript +QUTE_CONFIG_DIR=${QUTE_CONFIG_DIR:-${XDG_CONFIG_HOME:-$HOME/.config}/qutebrowser/} +KODI_CONFIG=${PWFILL_CONFIG:-${QUTE_CONFIG_DIR}/kodi_rc} +if [[ -f "$KODI_CONFIG" ]] ; then + # shellcheck source=/dev/null + source "$KODI_CONFIG" + if [[ -z "$HOST" || -z "$AUTH" ]] ; then + echo "message-error 'HOST and/or AUTH not set in $KODI_CONFIG'" > "$QUTE_FIFO" + exit 231 + fi +else + echo "message-error '$KODI_CONFIG not found'" > "$QUTE_FIFO" + exit 231 +fi + +# get real URL from twitter links +if [[ "$QUTE_URL" =~ ^https:\/\/t\.co ]] ; then + QUTE_URL=$(curl -o /dev/null --silent --head --write-out '%{redirect_url}' "$QUTE_URL" ) +fi + +# regex from https://github.com/dirkjanm/firefox-send-to-xbmc/blob/master/webextension/main.js +if [[ "$QUTE_URL" =~ ^.*twitch.tv\/([a-zA-Z0-9_]+)$ ]] ; then + NAME="${BASH_REMATCH[1]}" + JSON='{"jsonrpc":"2.0","method":"Player.Open","params":{"item":{"file":"plugin://plugin.video.twitch/?mode=play&channel_name='$NAME'"}},"id":"2"}' + +elif [[ "$QUTE_URL" =~ ^.*twitch.tv\/videos\/([0-9]+)$ ]] ; then + NAME="${BASH_REMATCH[1]}" + JSON='{"jsonrpc":"2.0","method":"Player.Open","params":{"item":{"file":"plugin://plugin.video.twitch/?mode=play&video_id='$NAME'"}},"id":"2"}' + +elif [[ "$QUTE_URL" =~ ^.*vimeo.com\/([0-9]+) ]] ; then + NAME="${BASH_REMATCH[1]}" + JSON='{"jsonrpc":"2.0","method":"Player.Open","params":{"item":{"file":"plugin://plugin.video.vimeo/play/?video_id='$NAME'"}},"id":"2"}' + +elif [[ "$QUTE_URL" =~ ^.*youtu.be\/|v\/|u\/\w\/|embed\/|watch\?v=([^#\&\?]*).* ]] ; then + NAME="${BASH_REMATCH[1]}" + JSON='{"jsonrpc":"2.0","method":"Player.Open","params":{"item":{"file":"plugin://plugin.video.youtube/play/?video_id='$NAME'"}},"id":"2"}' +fi + +if [[ "$JSON" ]] ; then + curl \ + --request POST "$HOST"/jsonrpc \ + --header "Authorization: Basic $AUTH" \ + --header "Content-Type: application/json" \ + --data "$JSON" \ + --silent > /dev/null +else + URL=$(echo "$QUTE_URL" |awk -F/ '{print $3}') + echo "message-warning 'kodi userscript does not support this $URL'" > "$QUTE_FIFO" +fi diff --git a/misc/userscripts/qr b/misc/userscripts/qr new file mode 100755 index 000000000..84215249b --- /dev/null +++ b/misc/userscripts/qr @@ -0,0 +1,8 @@ +#!/usr/bin/env bash + +pngfile=$(mktemp --suffix=.png) +trap 'rm -f "$pngfile"' EXIT + +qrencode -t PNG -o "$pngfile" -s 10 "$QUTE_URL" +echo ":open -t file:///$pngfile" >> "$QUTE_FIFO" +sleep 1 # give qutebrowser time to open the file before it gets removed diff --git a/misc/userscripts/readability-js b/misc/userscripts/readability-js index e189e5ee4..310d1c081 100755 --- a/misc/userscripts/readability-js +++ b/misc/userscripts/readability-js @@ -57,7 +57,7 @@ const HEADER = ` table, th, td { - border: 1px solid currentColor; + border: 1px solid grey; border-collapse: collapse; padding: 6px; vertical-align: top; @@ -77,7 +77,7 @@ const HEADER = ` background-color: #dddddd; } blockquote { - border-inline-start: 2px solid #333333 !important; + border-inline-start: 2px solid grey !important; padding: 0; padding-inline-start: 16px; margin-inline-start: 24px; diff --git a/qutebrowser/__init__.py b/qutebrowser/__init__.py index b5b4b8c7c..dfb1febd4 100644 --- a/qutebrowser/__init__.py +++ b/qutebrowser/__init__.py @@ -26,7 +26,7 @@ __copyright__ = "Copyright 2014-2020 Florian Bruhin (The Compiler)" __license__ = "GPL" __maintainer__ = __author__ __email__ = "mail@qutebrowser.org" -__version__ = "1.14.0" +__version__ = "1.14.1" __version_info__ = tuple(int(part) for part in __version__.split('.')) __description__ = "A keyboard-driven, vim-like browser based on PyQt5." diff --git a/qutebrowser/app.py b/qutebrowser/app.py index f722c580f..76d52470a 100644 --- a/qutebrowser/app.py +++ b/qutebrowser/app.py @@ -96,7 +96,8 @@ def run(args): q_app = Application(args) q_app.setOrganizationName("qutebrowser") q_app.setApplicationName("qutebrowser") - q_app.setDesktopFileName("org.qutebrowser.qutebrowser") + # Default DesktopFileName is org.qutebrowser.qutebrowser, set in `get_argparser()` + q_app.setDesktopFileName(args.desktop_file_name) q_app.setApplicationVersion(qutebrowser.__version__) if args.version: @@ -491,8 +492,6 @@ def _init_modules(*, args): log.init.debug("Misc initialization...") macros.init() windowundo.init() - # Init backend-specific stuff - browsertab.init() class Application(QApplication): diff --git a/qutebrowser/browser/browsertab.py b/qutebrowser/browser/browsertab.py index dd21cd0e0..bf25455a6 100644 --- a/qutebrowser/browser/browsertab.py +++ b/qutebrowser/browser/browsertab.py @@ -83,15 +83,6 @@ def create(win_id: int, parent=parent) -def init() -> None: - """Initialize backend-specific modules.""" - if objects.backend == usertypes.Backend.QtWebEngine: - from qutebrowser.browser.webengine import webenginetab - webenginetab.init() - return - assert objects.backend == usertypes.Backend.QtWebKit, objects.backend - - class WebTabError(Exception): """Base class for various errors.""" diff --git a/qutebrowser/browser/commands.py b/qutebrowser/browser/commands.py index 34c078d89..18777e250 100644 --- a/qutebrowser/browser/commands.py +++ b/qutebrowser/browser/commands.py @@ -1057,7 +1057,8 @@ class CommandDispatcher: verbose: Show notifications when the command started/exited. output: Show the output in a new tab. output_messages: Show the output as messages. - detach: Whether the command should be detached from qutebrowser. + detach: Detach the command from qutebrowser so that it continues + running when qutebrowser quits. cmdline: The commandline to execute. count: Given to userscripts as $QUTE_COUNT. """ diff --git a/qutebrowser/browser/greasemonkey.py b/qutebrowser/browser/greasemonkey.py index 0305bd589..df8b2b0c2 100644 --- a/qutebrowser/browser/greasemonkey.py +++ b/qutebrowser/browser/greasemonkey.py @@ -41,9 +41,12 @@ from qutebrowser.misc import objects gm_manager = cast('GreasemonkeyManager', None) -def _scripts_dir(): +def _scripts_dirs(): """Get the directory of the scripts.""" - return os.path.join(standarddir.data(), 'greasemonkey') + return [ + os.path.join(standarddir.data(), 'greasemonkey'), + os.path.join(standarddir.config(), 'greasemonkey'), + ] class GreasemonkeyScript: @@ -277,18 +280,19 @@ class GreasemonkeyManager(QObject): self._run_end = [] self._run_idle = [] - scripts_dir = os.path.abspath(_scripts_dir()) - log.greasemonkey.debug("Reading scripts from: {}".format(scripts_dir)) - for script_filename in glob.glob(os.path.join(scripts_dir, '*.js')): - if not os.path.isfile(script_filename): - continue - script_path = os.path.join(scripts_dir, script_filename) - with open(script_path, encoding='utf-8-sig') as script_file: - script = GreasemonkeyScript.parse(script_file.read(), - script_filename) - if not script.name: - script.name = script_filename - self.add_script(script, force) + for scripts_dir in _scripts_dirs(): + scripts_dir = os.path.abspath(scripts_dir) + log.greasemonkey.debug("Reading scripts from: {}".format(scripts_dir)) + for script_filename in glob.glob(os.path.join(scripts_dir, '*.js')): + if not os.path.isfile(script_filename): + continue + script_path = os.path.join(scripts_dir, script_filename) + with open(script_path, encoding='utf-8-sig') as script_file: + script = GreasemonkeyScript.parse(script_file.read(), + script_filename) + if not script.name: + script.name = script_filename + self.add_script(script, force) self.scripts_reloaded.emit() def add_script(self, script, force=False): @@ -325,7 +329,7 @@ class GreasemonkeyManager(QObject): log.greasemonkey.debug("Loaded script: {}".format(script.name)) def _required_url_to_file_path(self, url): - requires_dir = os.path.join(_scripts_dir(), 'requires') + requires_dir = os.path.join(_scripts_dirs()[0], 'requires') if not os.path.exists(requires_dir): os.mkdir(requires_dir) return os.path.join(requires_dir, utils.sanitize_filename(url)) @@ -426,7 +430,7 @@ def greasemonkey_reload(force=False): """Re-read Greasemonkey scripts from disk. The scripts are read from a 'greasemonkey' subdirectory in - qutebrowser's data directory (see `:version`). + qutebrowser's data or config directories (see `:version`). Args: force: For any scripts that have required dependencies, @@ -440,7 +444,8 @@ def init(): global gm_manager gm_manager = GreasemonkeyManager() - try: - os.mkdir(_scripts_dir()) - except FileExistsError: - pass + for scripts_dir in _scripts_dirs(): + try: + os.mkdir(scripts_dir) + except FileExistsError: + pass diff --git a/qutebrowser/browser/shared.py b/qutebrowser/browser/shared.py index 193a2a0e0..9234e82d8 100644 --- a/qutebrowser/browser/shared.py +++ b/qutebrowser/browser/shared.py @@ -82,7 +82,7 @@ def javascript_confirm(url, js_msg, abort_on): raise CallSuper msg = 'From <b>{}</b>:<br/>{}'.format(html.escape(url.toDisplayString()), - js_msg) + html.escape(js_msg)) urlstr = url.toString(QUrl.RemovePassword | QUrl.FullyEncoded) ans = message.ask('Javascript confirm', msg, mode=usertypes.PromptMode.yesno, @@ -99,7 +99,7 @@ def javascript_prompt(url, js_msg, default, abort_on): return (False, "") msg = '<b>{}</b> asks:<br/>{}'.format(html.escape(url.toDisplayString()), - js_msg) + html.escape(js_msg)) urlstr = url.toString(QUrl.RemovePassword | QUrl.FullyEncoded) answer = message.ask('Javascript prompt', msg, mode=usertypes.PromptMode.text, @@ -122,7 +122,7 @@ def javascript_alert(url, js_msg, abort_on): return msg = 'From <b>{}</b>:<br/>{}'.format(html.escape(url.toDisplayString()), - js_msg) + html.escape(js_msg)) urlstr = url.toString(QUrl.RemovePassword | QUrl.FullyEncoded) message.ask('Javascript alert', msg, mode=usertypes.PromptMode.alert, abort_on=abort_on, url=urlstr) diff --git a/qutebrowser/browser/webengine/darkmode.py b/qutebrowser/browser/webengine/darkmode.py index 630a7bf9e..d067edea3 100644 --- a/qutebrowser/browser/webengine/darkmode.py +++ b/qutebrowser/browser/webengine/darkmode.py @@ -264,6 +264,16 @@ def _variant() -> Variant: def settings() -> Iterator[Tuple[str, str]]: """Get necessary blink settings to configure dark mode for QtWebEngine.""" + if (qtutils.version_check('5.15.2', compiled=False) and + config.val.colors.webpage.prefers_color_scheme_dark): + # With older Qt versions, this is passed in qtargs.py as --force-dark-mode + # instead. + # + # With Chromium 85 (> Qt 5.15.2), the enumeration has changed in Blink and this + # will need to be set to '0' instead: + # https://chromium-review.googlesource.com/c/chromium/src/+/2232922 + yield "preferredColorScheme", "1" + if not config.val.colors.webpage.darkmode.enabled: return diff --git a/qutebrowser/browser/webengine/webenginesettings.py b/qutebrowser/browser/webengine/webenginesettings.py index 1526574a7..05e7b4b68 100644 --- a/qutebrowser/browser/webengine/webenginesettings.py +++ b/qutebrowser/browser/webengine/webenginesettings.py @@ -26,25 +26,35 @@ Module attributes: import os import operator -from typing import cast, Any, List, Optional, Tuple, Union +from typing import cast, Any, List, Optional, Tuple, Union, TYPE_CHECKING from PyQt5.QtGui import QFont +from PyQt5.QtWidgets import QApplication from PyQt5.QtWebEngineWidgets import QWebEngineSettings, QWebEngineProfile -from qutebrowser.browser.webengine import spell, webenginequtescheme +from qutebrowser.browser import history +from qutebrowser.browser.webengine import (spell, webenginequtescheme, cookies, + webenginedownloads) from qutebrowser.config import config, websettings from qutebrowser.config.websettings import AttributeInfo as Attr -from qutebrowser.utils import standarddir, qtutils, message, log, urlmatch, usertypes +from qutebrowser.utils import (standarddir, qtutils, message, log, + urlmatch, usertypes, objreg) +if TYPE_CHECKING: + from qutebrowser.browser.webengine import interceptor # The default QWebEngineProfile default_profile = cast(QWebEngineProfile, None) # The QWebEngineProfile used for private (off-the-record) windows private_profile: Optional[QWebEngineProfile] = None # The global WebEngineSettings object -global_settings = cast('WebEngineSettings', None) +_global_settings = cast('WebEngineSettings', None) parsed_user_agent = None +_qute_scheme_handler = cast(webenginequtescheme.QuteSchemeHandler, None) +_req_interceptor = cast('interceptor.RequestInterceptor', None) +_download_manager = cast(webenginedownloads.DownloadManager, None) + class _SettingsWrapper: @@ -217,6 +227,26 @@ class ProfileSetter: def __init__(self, profile): self._profile = profile + self._name_to_method = { + 'content.cache.size': self.set_http_cache_size, + 'content.cookies.store': self.set_persistent_cookie_policy, + 'spellcheck.languages': self.set_dictionary_language, + } + + # WORKAROUND for https://bugreports.qt.io/browse/QTBUG-75884 + # (note this isn't actually fixed properly before Qt 5.15) + header_bug_fixed = qtutils.version_check('5.15', compiled=False) + if header_bug_fixed: + for name in ['user_agent', 'accept_language']: + self._name_to_method[f'content.headers.{name}'] = self.set_http_headers + + def update_setting(self, name): + """Update a setting based on its name.""" + try: + meth = self._name_to_method[name] + except KeyError: + return + meth() def init_profile(self): """Initialize settings on the given profile.""" @@ -267,20 +297,21 @@ class ProfileSetter: def set_persistent_cookie_policy(self): """Set the HTTP Cookie size for the given profile.""" - assert not self._profile.isOffTheRecord() + if self._profile.isOffTheRecord(): + return if config.val.content.cookies.store: value = QWebEngineProfile.AllowPersistentCookies else: value = QWebEngineProfile.NoPersistentCookies self._profile.setPersistentCookiesPolicy(value) - def set_dictionary_language(self, warn=True): + def set_dictionary_language(self): """Load the given dictionaries.""" filenames = [] for code in config.val.spellcheck.languages or []: local_filename = spell.local_filename(code) if not local_filename: - if warn: + if not self._profile.isOffTheRecord(): message.warning("Language {} is not installed - see " "scripts/dictcli.py in qutebrowser's " "sources".format(code)) @@ -295,28 +326,10 @@ class ProfileSetter: def _update_settings(option): """Update global settings when qwebsettings changed.""" - global_settings.update_setting(option) - - # WORKAROUND for https://bugreports.qt.io/browse/QTBUG-75884 - # (note this isn't actually fixed properly before Qt 5.15) - header_bug_fixed = qtutils.version_check('5.15', compiled=False) - - if option in ['content.headers.user_agent', - 'content.headers.accept_language'] and header_bug_fixed: - default_profile.setter.set_http_headers() - if private_profile: - private_profile.setter.set_http_headers() - elif option == 'content.cache.size': - default_profile.setter.set_http_cache_size() - if private_profile: - private_profile.setter.set_http_cache_size() - elif option == 'content.cookies.store': - default_profile.setter.set_persistent_cookie_policy() - # We're not touching the private profile's cookie policy. - elif option == 'spellcheck.languages': - default_profile.setter.set_dictionary_language() - if private_profile: - private_profile.setter.set_dictionary_language(warn=False) + _global_settings.update_setting(option) + default_profile.setter.update_setting(option) + if private_profile: + private_profile.setter.update_setting(option) def _init_user_agent_str(ua): @@ -328,33 +341,54 @@ def init_user_agent(): _init_user_agent_str(QWebEngineProfile.defaultProfile().httpUserAgent()) +def _init_profile(profile: QWebEngineProfile) -> None: + """Initialize a new QWebEngineProfile. + + This currently only contains the steps which are shared between a private and a + non-private profile (at the moment, only the default profile). + """ + profile.setter = ProfileSetter(profile) # type: ignore[attr-defined] + profile.setter.init_profile() + + _qute_scheme_handler.install(profile) + _req_interceptor.install(profile) + _download_manager.install(profile) + cookies.install_filter(profile) + + # Clear visited links on web history clear + history.web_history.history_cleared.connect(profile.clearAllVisitedLinks) + history.web_history.url_cleared.connect( + lambda url: profile.clearVisitedLinks([url])) + + _global_settings.init_settings() + + def _init_default_profile(): """Init the default QWebEngineProfile.""" global default_profile default_profile = QWebEngineProfile.defaultProfile() + init_user_agent() - default_profile.setter = ProfileSetter( # type: ignore[attr-defined] - default_profile) default_profile.setCachePath( os.path.join(standarddir.cache(), 'webengine')) default_profile.setPersistentStoragePath( os.path.join(standarddir.data(), 'webengine')) - default_profile.setter.init_profile() - default_profile.setter.set_persistent_cookie_policy() + + _init_profile(default_profile) def init_private_profile(): """Init the private QWebEngineProfile.""" global private_profile - if not qtutils.is_single_process(): - private_profile = QWebEngineProfile() - private_profile.setter = ProfileSetter( # type: ignore[attr-defined] - private_profile) - assert private_profile.isOffTheRecord() - private_profile.setter.init_profile() + if qtutils.is_single_process(): + return + + private_profile = QWebEngineProfile() + assert private_profile.isOffTheRecord() + _init_profile(private_profile) def _init_site_specific_quirks(): @@ -430,14 +464,33 @@ def init(): webenginequtescheme.init() spell.init() + # For some reason we need to keep a reference, otherwise the scheme handler + # won't work... + # https://www.riverbankcomputing.com/pipermail/pyqt/2016-September/038075.html + global _qute_scheme_handler + app = QApplication.instance() + log.init.debug("Initializing qute://* handler...") + _qute_scheme_handler = webenginequtescheme.QuteSchemeHandler(parent=app) + + global _req_interceptor + log.init.debug("Initializing request interceptor...") + from qutebrowser.browser.webengine import interceptor + _req_interceptor = interceptor.RequestInterceptor(parent=app) + + global _download_manager + log.init.debug("Initializing QtWebEngine downloads...") + _download_manager = webenginedownloads.DownloadManager(parent=app) + objreg.register('webengine-download-manager', _download_manager) + from qutebrowser.misc import quitter + quitter.instance.shutting_down.connect(_download_manager.shutdown) + + global _global_settings + _global_settings = WebEngineSettings(_SettingsWrapper()) + _init_default_profile() init_private_profile() config.instance.changed.connect(_update_settings) - global global_settings - global_settings = WebEngineSettings(_SettingsWrapper()) - global_settings.init_settings() - _init_site_specific_quirks() _init_devtools_settings() diff --git a/qutebrowser/browser/webengine/webenginetab.py b/qutebrowser/browser/webengine/webenginetab.py index f026b7c23..98a6bf05d 100644 --- a/qutebrowser/browser/webengine/webenginetab.py +++ b/qutebrowser/browser/webengine/webenginetab.py @@ -27,68 +27,19 @@ from typing import cast, Union from PyQt5.QtCore import pyqtSignal, pyqtSlot, Qt, QPoint, QPointF, QUrl, QObject from PyQt5.QtNetwork import QAuthenticator -from PyQt5.QtWidgets import QApplication, QWidget +from PyQt5.QtWidgets import QWidget from PyQt5.QtWebEngineWidgets import QWebEnginePage, QWebEngineScript, QWebEngineHistory from qutebrowser.config import config -from qutebrowser.browser import (browsertab, eventfilter, shared, webelem, - history, greasemonkey) +from qutebrowser.browser import browsertab, eventfilter, shared, webelem, greasemonkey from qutebrowser.browser.webengine import (webview, webengineelem, tabhistory, - interceptor, webenginequtescheme, - cookies, webenginedownloads, webenginesettings, certificateerror) -from qutebrowser.misc import miscwidgets, objects, quitter +from qutebrowser.misc import miscwidgets, objects from qutebrowser.utils import (usertypes, qtutils, log, javascript, utils, - message, objreg, jinja, debug) + message, jinja, debug) from qutebrowser.qt import sip -_qute_scheme_handler = None - - -def init(): - """Initialize QtWebEngine-specific modules.""" - # For some reason we need to keep a reference, otherwise the scheme handler - # won't work... - # https://www.riverbankcomputing.com/pipermail/pyqt/2016-September/038075.html - global _qute_scheme_handler - - app = QApplication.instance() - log.init.debug("Initializing qute://* handler...") - _qute_scheme_handler = webenginequtescheme.QuteSchemeHandler(parent=app) - _qute_scheme_handler.install(webenginesettings.default_profile) - if webenginesettings.private_profile: - _qute_scheme_handler.install(webenginesettings.private_profile) - - log.init.debug("Initializing request interceptor...") - req_interceptor = interceptor.RequestInterceptor(parent=app) - req_interceptor.install(webenginesettings.default_profile) - if webenginesettings.private_profile: - req_interceptor.install(webenginesettings.private_profile) - - log.init.debug("Initializing QtWebEngine downloads...") - download_manager = webenginedownloads.DownloadManager(parent=app) - download_manager.install(webenginesettings.default_profile) - if webenginesettings.private_profile: - download_manager.install(webenginesettings.private_profile) - objreg.register('webengine-download-manager', download_manager) - quitter.instance.shutting_down.connect(download_manager.shutdown) - - log.init.debug("Initializing cookie filter...") - cookies.install_filter(webenginesettings.default_profile) - if webenginesettings.private_profile: - cookies.install_filter(webenginesettings.private_profile) - - # Clear visited links on web history clear - for p in [webenginesettings.default_profile, - webenginesettings.private_profile]: - if not p: - continue - history.web_history.history_cleared.connect(p.clearAllVisitedLinks) - history.web_history.url_cleared.connect( - lambda url, profile=p: profile.clearVisitedLinks([url])) - - # Mapping worlds from usertypes.JsWorld to QWebEngineScript world IDs. _JS_WORLD_MAP = { usertypes.JsWorld.main: QWebEngineScript.MainWorld, diff --git a/qutebrowser/commands/userscripts.py b/qutebrowser/commands/userscripts.py index ce25d7d28..6d2c2f147 100644 --- a/qutebrowser/commands/userscripts.py +++ b/qutebrowser/commands/userscripts.py @@ -395,6 +395,7 @@ def _lookup_path(cmd): directories = [ os.path.join(standarddir.data(), "userscripts"), os.path.join(standarddir.data(system=True), "userscripts"), + os.path.join(standarddir.config(), "userscripts"), ] for directory in directories: cmd_path = os.path.join(directory, cmd) diff --git a/qutebrowser/completion/models/listcategory.py b/qutebrowser/completion/models/listcategory.py index 79dc0770a..d4193b6d8 100644 --- a/qutebrowser/completion/models/listcategory.py +++ b/qutebrowser/completion/models/listcategory.py @@ -22,7 +22,7 @@ import re from typing import Iterable, Tuple -from PyQt5.QtCore import Qt, QSortFilterProxyModel, QRegExp +from PyQt5.QtCore import QSortFilterProxyModel, QRegularExpression from PyQt5.QtGui import QStandardItem, QStandardItemModel from PyQt5.QtWidgets import QWidget @@ -63,9 +63,9 @@ class ListCategory(QSortFilterProxyModel): val = re.sub(r' +', r' ', val) # See #1919 val = re.escape(val) val = val.replace(r'\ ', '.*') - rx = QRegExp(val, Qt.CaseInsensitive) + rx = QRegularExpression(val, QRegularExpression.CaseInsensitiveOption) qtutils.ensure_valid(rx) - self.setFilterRegExp(rx) + self.setFilterRegularExpression(rx) self.invalidate() sortcol = 0 self.sort(sortcol) diff --git a/qutebrowser/config/qtargs.py b/qutebrowser/config/qtargs.py index 8ab93c904..2136f7e7f 100644 --- a/qutebrowser/config/qtargs.py +++ b/qutebrowser/config/qtargs.py @@ -203,12 +203,18 @@ def _qtwebengine_settings_args() -> Iterator[str]: } } - referrer_setting = settings['content.headers.referer'] - if qtutils.version_check('5.14', compiled=False): + if (qtutils.version_check('5.14', compiled=False) and + not qtutils.version_check('5.15.2', compiled=False)): + # In Qt 5.14 to 5.15.1, `--force-dark-mode` is used to set the + # preferred colorscheme. In Qt 5.15.2, this is handled by a + # blink-setting instead. settings['colors.webpage.prefers_color_scheme_dark'] = { True: '--force-dark-mode', False: None, } + + referrer_setting = settings['content.headers.referer'] + if qtutils.version_check('5.14', compiled=False): # Starting with Qt 5.14, this is handled via --enable-features referrer_setting['same-domain'] = None else: diff --git a/qutebrowser/misc/earlyinit.py b/qutebrowser/misc/earlyinit.py index d1c57760e..d742a6706 100644 --- a/qutebrowser/misc/earlyinit.py +++ b/qutebrowser/misc/earlyinit.py @@ -211,6 +211,10 @@ def _check_modules(modules): ), log.py_warning_filter( category=DeprecationWarning, message=r'the imp module is deprecated', + ), log.py_warning_filter( + # WORKAROUND for https://github.com/pypa/setuptools/issues/2466 + category=DeprecationWarning, + message=r'Creating a LegacyVersion has been deprecated', ): # pylint: enable=bad-continuation importlib.import_module(name) diff --git a/qutebrowser/misc/utilcmds.py b/qutebrowser/misc/utilcmds.py index 56138c798..fa327b772 100644 --- a/qutebrowser/misc/utilcmds.py +++ b/qutebrowser/misc/utilcmds.py @@ -42,15 +42,17 @@ from qutebrowser.qt import sip @cmdutils.register(maxsplit=1, no_cmd_split=True, no_replace_variables=True) @cmdutils.argument('win_id', value=cmdutils.Value.win_id) -def later(ms: int, command: str, win_id: int) -> None: +def later(duration: str, command: str, win_id: int) -> None: """Execute a command after some time. Args: - ms: How many milliseconds to wait. + duration: Duration to wait in format XhYmZs or a number for milliseconds. command: The command to run, with optional args. """ - if ms < 0: - raise cmdutils.CommandError("I can't run something in the past!") + try: + ms = utils.parse_duration(duration) + except ValueError as e: + raise cmdutils.CommandError(e) commandrunner = runners.CommandRunner(win_id) timer = usertypes.Timer(name='later', parent=QApplication.instance()) try: diff --git a/qutebrowser/qutebrowser.py b/qutebrowser/qutebrowser.py index bca1df020..7f36d4807 100644 --- a/qutebrowser/qutebrowser.py +++ b/qutebrowser/qutebrowser.py @@ -85,6 +85,11 @@ def get_argparser(): parser.add_argument('--json-args', help=argparse.SUPPRESS) parser.add_argument('--temp-basedir-restarted', help=argparse.SUPPRESS) + parser.add_argument('--desktop-file-name', + default="org.qutebrowser.qutebrowser", + help="Set the base name of the desktop entry for this " + "application. Used to set the app_id under Wayland. See " + "https://doc.qt.io/qt-5/qguiapplication.html#desktopFileName-prop") debug = parser.add_argument_group('debug arguments') debug.add_argument('-l', '--loglevel', dest='loglevel', diff --git a/qutebrowser/utils/jinja.py b/qutebrowser/utils/jinja.py index f60c46bbc..c4ba703cc 100644 --- a/qutebrowser/utils/jinja.py +++ b/qutebrowser/utils/jinja.py @@ -118,8 +118,7 @@ class Environment(jinja2.Environment): def _data_url(self, path: str) -> str: """Get a data: url for the broken qutebrowser logo.""" data = utils.read_file(path, binary=True) - filename = utils.resource_filename(path) - mimetype = utils.guess_mimetype(filename) + mimetype = utils.guess_mimetype(path) return urlutils.data_url(mimetype, data).toString() def getattr(self, obj: Any, attribute: str) -> Any: diff --git a/qutebrowser/utils/utils.py b/qutebrowser/utils/utils.py index 31ff5bf50..331bf5f96 100644 --- a/qutebrowser/utils/utils.py +++ b/qutebrowser/utils/utils.py @@ -42,6 +42,11 @@ from typing import (Any, Callable, IO, Iterator, Optional, Sequence, Tuple, Type from PyQt5.QtCore import QUrl, QVersionNumber from PyQt5.QtGui import QClipboard, QDesktopServices from PyQt5.QtWidgets import QApplication +# We cannot use the stdlib version on 3.7-3.8 because we need the files() API. +if sys.version_info >= (3, 9): + import importlib.resources as importlib_resources +else: # pragma: no cover + import importlib_resources import pkg_resources import yaml try: @@ -69,7 +74,7 @@ is_posix = os.name == 'posix' try: # Protocol was added in Python 3.8 - from typing import Protocol + from typing import Protocol # pylint: disable=ungrouped-imports except ImportError: # pragma: no cover if not TYPE_CHECKING: class Protocol: @@ -216,13 +221,12 @@ def read_file(filename: str, binary: bool = False) -> Any: with open(fn, 'r', encoding='utf-8') as f: return f.read() else: - data = pkg_resources.resource_string( - qutebrowser.__name__, filename) + p = importlib_resources.files(qutebrowser) / filename if binary: - return data + return p.read_bytes() - return data.decode('UTF-8') + return p.read_text() def resource_filename(filename: str) -> str: @@ -773,3 +777,30 @@ def libgl_workaround() -> None: libgl = ctypes.util.find_library("GL") if libgl is not None: # pragma: no branch ctypes.CDLL(libgl, mode=ctypes.RTLD_GLOBAL) + + +def parse_duration(duration: str) -> int: + """Parse duration in format XhYmZs into milliseconds duration.""" + if duration.isdigit(): + # For backward compatibility return milliseconds + return int(duration) + + match = re.fullmatch( + r'(?P<hours>[0-9]+(\.[0-9])?h)?\s*' + r'(?P<minutes>[0-9]+(\.[0-9])?m)?\s*' + r'(?P<seconds>[0-9]+(\.[0-9])?s)?', + duration + ) + if not match or not match.group(0): + raise ValueError( + f"Invalid duration: {duration} - " + "expected XhYmZs or a number of milliseconds" + ) + seconds_string = match.group('seconds') if match.group('seconds') else '0' + seconds = float(seconds_string.rstrip('s')) + minutes_string = match.group('minutes') if match.group('minutes') else '0' + minutes = float(minutes_string.rstrip('m')) + hours_string = match.group('hours') if match.group('hours') else '0' + hours = float(hours_string.rstrip('h')) + milliseconds = int((seconds + minutes * 60 + hours * 3600) * 1000) + return milliseconds diff --git a/qutebrowser/utils/version.py b/qutebrowser/utils/version.py index 1dc623881..a31b4eae2 100644 --- a/qutebrowser/utils/version.py +++ b/qutebrowser/utils/version.py @@ -362,6 +362,7 @@ MODULE_INFO: Mapping[str, ModuleInfo] = collections.OrderedDict([ ('yaml', ['__version__']), ('adblock', ['__version__'], "0.3.2"), ('attr', ['__version__']), + ('importlib_resources', []), ('PyQt5.QtWebEngineWidgets', []), ('PyQt5.QtWebEngine', ['PYQT_WEBENGINE_VERSION_STR']), ('PyQt5.QtWebKitWidgets', []), diff --git a/requirements.txt b/requirements.txt index fa1c5f31b..0d682f809 100644 --- a/requirements.txt +++ b/requirements.txt @@ -3,8 +3,9 @@ adblock==0.3.2 attrs==20.3.0 colorama==0.4.4 +importlib-resources==3.3.0 ; python_version<"3.9" Jinja2==2.11.2 MarkupSafe==1.1.1 -Pygments==2.7.2 +Pygments==2.7.3 pyPEG2==2.15.2 PyYAML==5.3.1 diff --git a/scripts/dev/build_release.py b/scripts/dev/build_release.py index 6044a1e18..479283a92 100755 --- a/scripts/dev/build_release.py +++ b/scripts/dev/build_release.py @@ -116,7 +116,13 @@ def smoke_test(executable): (r'\[.*:ERROR:mach_port_broker.mm\(48\)\] bootstrap_look_up ' r'org\.chromium\.Chromium\.rohitfork\.1: Permission denied \(1100\)'), (r'\[.*:ERROR:mach_port_broker.mm\(43\)\] bootstrap_look_up: ' - r'Unknown service name \(1102\)') + r'Unknown service name \(1102\)'), + + # Windows N: + # https://github.com/microsoft/playwright/issues/2901 + (r'\[.*:ERROR:dxva_video_decode_accelerator_win.cc\(\d+\)\] ' + r'DXVAVDA fatal error: could not LoadLibrary: .*: The specified ' + r'module could not be found. \(0x7E\)'), ] proc = subprocess.run([executable, '--no-err-windows', '--nowindow', diff --git a/scripts/dev/check_coverage.py b/scripts/dev/check_coverage.py index 728a36873..aa88c97a3 100644 --- a/scripts/dev/check_coverage.py +++ b/scripts/dev/check_coverage.py @@ -339,7 +339,7 @@ def main_check(): print("or check https://codecov.io/github/qutebrowser/qutebrowser") print() - if 'CI' in os.environ: + if scriptutils.ON_CI: print("Keeping coverage.xml on CI.") else: os.remove('coverage.xml') diff --git a/scripts/dev/ci/docker/Dockerfile.j2 b/scripts/dev/ci/docker/Dockerfile.j2 index 412c42cf2..1835f0a2f 100644 --- a/scripts/dev/ci/docker/Dockerfile.j2 +++ b/scripts/dev/ci/docker/Dockerfile.j2 @@ -24,4 +24,4 @@ WORKDIR /home/user CMD git clone /outside qutebrowser.git && \ cd qutebrowser.git && \ - tox -e py38 + tox -e py diff --git a/scripts/dev/ci/problemmatchers.py b/scripts/dev/ci/problemmatchers.py index 3e804af05..cc423f922 100644 --- a/scripts/dev/ci/problemmatchers.py +++ b/scripts/dev/ci/problemmatchers.py @@ -188,7 +188,7 @@ MATCHERS = { "severity": "error", "pattern": [ { - "regexp": r'^([^:]+):(\d+): (Found .*)', + "regexp": r'^([^:]+):(\d+): \033\[34m(Found .*)\033\[0m', "file": 1, "line": 2, "message": 3, diff --git a/scripts/dev/misc_checks.py b/scripts/dev/misc_checks.py index 14373f94f..ad446412c 100644 --- a/scripts/dev/misc_checks.py +++ b/scripts/dev/misc_checks.py @@ -145,8 +145,8 @@ def _check_spelling_file(path, fobj, patterns): for pattern, explanation in patterns: if pattern.search(line): ok = False - print(f'{path}:{num}: Found "{pattern.pattern}" - ', end='') - utils.print_col(explanation, 'blue') + print(f'{path}:{num}: ', end='') + utils.print_col(f'Found "{pattern.pattern}" - {explanation}', 'blue') return ok @@ -185,7 +185,7 @@ def check_spelling(args: argparse.Namespace) -> Optional[bool]: "'type: ignore[error-code]' instead."), ), ( - re.compile(r'# type: (?!ignore\[)'), + re.compile(r'# type: (?!ignore(\[|$))'), "Don't use type comments, use type annotations instead.", ), ( @@ -274,12 +274,35 @@ def check_userscripts_descriptions(_args: argparse.Namespace = None) -> bool: return ok +def check_userscript_shebangs(_args: argparse.Namespace) -> bool: + """Check that we're using /usr/bin/env in shebangs.""" + ok = True + folder = pathlib.Path('misc/userscripts') + + for sub in folder.iterdir(): + if sub.is_dir() or sub.name == 'README.md': + continue + + with sub.open('r', encoding='utf-8') as f: + shebang = f.readline() + assert shebang.startswith('#!'), shebang + binary = shebang.split()[0][2:] + + if binary not in ['/bin/sh', '/usr/bin/env']: + bin_name = pathlib.Path(binary).name + print(f"In {sub}, use #!/usr/bin/env {bin_name} instead of #!{binary}") + ok = False + + return ok + + def main() -> int: checkers = { 'git': check_git, 'vcs': check_vcs_conflict, 'spelling': check_spelling, - 'userscripts': check_userscripts_descriptions, + 'userscript-descriptions': check_userscripts_descriptions, + 'userscript-shebangs': check_userscript_shebangs, 'changelog-urls': check_changelog_urls, } diff --git a/scripts/dev/recompile_requirements.py b/scripts/dev/recompile_requirements.py index 3257b4734..5cda4b3d7 100644 --- a/scripts/dev/recompile_requirements.py +++ b/scripts/dev/recompile_requirements.py @@ -73,7 +73,7 @@ CHANGELOG_URLS = { 'pytest-bdd': 'https://github.com/pytest-dev/pytest-bdd/blob/master/CHANGES.rst', 'snowballstemmer': 'https://github.com/snowballstem/snowball/blob/master/NEWS', 'virtualenv': 'https://virtualenv.pypa.io/en/latest/changelog.html', - 'packaging': 'https://pypi.org/project/packaging/', + 'packaging': 'https://packaging.pypa.io/en/latest/changelog.html', 'build': 'https://github.com/pypa/build/commits/master', 'attrs': 'http://www.attrs.org/en/stable/changelog.html', 'Jinja2': 'https://github.com/pallets/jinja/blob/master/CHANGES.rst', @@ -130,7 +130,7 @@ CHANGELOG_URLS = { 'six': 'https://github.com/benjaminp/six/blob/master/CHANGES', 'altgraph': 'https://github.com/ronaldoussoren/altgraph/blob/master/doc/changelog.rst', 'urllib3': 'https://github.com/urllib3/urllib3/blob/master/CHANGES.rst', - 'lxml': 'https://lxml.de/4.6/changes-4.6.0.html', + 'lxml': 'https://lxml.de/index.html#old-versions', 'jwcrypto': 'https://github.com/latchset/jwcrypto/commits/master', 'wrapt': 'https://github.com/GrahamDumpleton/wrapt/blob/develop/docs/changes.rst', 'pep517': 'https://github.com/pypa/pep517/blob/master/doc/changelog.rst', @@ -175,6 +175,7 @@ CHANGELOG_URLS = { 'pyroma': 'https://github.com/regebro/pyroma/blob/master/HISTORY.txt', 'adblock': 'https://github.com/ArniDagur/python-adblock/blob/master/CHANGELOG.md', 'pyPEG2': None, + 'importlib-resources': 'https://importlib-resources.readthedocs.io/en/latest/changelog%20%28links%29.html', } @@ -262,13 +263,17 @@ def get_all_names(): yield basename[len('requirements-'):-len('.txt-raw')] -def run_pip(venv_dir, *args, **kwargs): +def run_pip(venv_dir, *args, quiet=False, **kwargs): """Run pip inside the virtualenv.""" + args = list(args) + if quiet: + args.insert(1, '-q') + arg_str = ' '.join(str(arg) for arg in args) utils.print_col('venv$ pip {}'.format(arg_str), 'blue') + venv_python = os.path.join(venv_dir, 'bin', 'python') - return subprocess.run([venv_python, '-m', 'pip'] + list(args), - check=True, **kwargs) + return subprocess.run([venv_python, '-m', 'pip'] + args, check=True, **kwargs) def init_venv(host_python, venv_dir, requirements, pre=False): @@ -277,8 +282,8 @@ def init_venv(host_python, venv_dir, requirements, pre=False): utils.print_col('$ python3 -m venv {}'.format(venv_dir), 'blue') subprocess.run([host_python, '-m', 'venv', venv_dir], check=True) - run_pip(venv_dir, 'install', '-U', 'pip') - run_pip(venv_dir, 'install', '-U', 'setuptools', 'wheel') + run_pip(venv_dir, 'install', '-U', 'pip', quiet=not utils.ON_CI) + run_pip(venv_dir, 'install', '-U', 'setuptools', 'wheel', quiet=not utils.ON_CI) install_command = ['install', '-r', requirements] if pre: @@ -292,6 +297,8 @@ def init_venv(host_python, venv_dir, requirements, pre=False): def parse_args(): """Parse commandline arguments via argparse.""" parser = argparse.ArgumentParser() + parser.add_argument('--force-test', help="Force running environment tests", + action='store_true') parser.add_argument('names', nargs='*') return parser.parse_args() @@ -358,6 +365,7 @@ def _get_changed_files(): def parse_versioned_line(line): """Parse a requirements.txt line into name/version.""" if '==' in line: + line = line.rsplit('#', maxsplit=1)[0] # Strip comments name, version = line.split('==') if ';' in version: # pip environment markers version = version.split(';')[0].strip() @@ -412,7 +420,7 @@ def print_changed_files(): utils.print_subtitle('Diff') print(diff_text) - if 'CI' in os.environ: + if utils.ON_CI: print() print('::set-output name=changed::' + files_text.replace('\n', '%0A')) @@ -481,7 +489,6 @@ def build_requirements(name): def test_tox(): """Test requirements via tox.""" - utils.print_title('Testing via tox') host_python = get_host_python('tox') req_path = os.path.join(REQ_DIR, 'requirements-tox.txt') @@ -506,11 +513,15 @@ def test_tox(): check=True) -def test_requirements(name, outfile): +def test_requirements(name, outfile, *, force=False): """Test a resulting requirements file.""" print() utils.print_subtitle("Testing") + if name not in _get_changed_files() and not force: + print(f"Skipping test as there were no changes for {name}.") + return + host_python = get_host_python(name) with tempfile.TemporaryDirectory() as tmpdir: init_venv(host_python, tmpdir, outfile) @@ -528,11 +539,16 @@ def main(): for name in names: utils.print_title(name) outfile = build_requirements(name) - test_requirements(name, outfile) + test_requirements(name, outfile, force=args.force_test) - if not args.names: + utils.print_title('Testing via tox') + if args.names and not args.force_test: # If we selected a subset, let's not go through the trouble of testing # via tox. + print("Skipping: Selected a subset only") + elif not _get_changed_files() and not args.force_test: + print("Skipping: No changes") + else: test_tox() print_changed_files() diff --git a/scripts/dev/update_version.py b/scripts/dev/update_version.py index 5f7df5cae..1f744d392 100644 --- a/scripts/dev/update_version.py +++ b/scripts/dev/update_version.py @@ -82,7 +82,7 @@ if __name__ == "__main__": .format(v=version)) print("* Windows: git fetch; git checkout v{v}; " "py -3.7 -m tox -e build-release -- --asciidoc " - "$env:userprofile\\bin\\asciidoc-9.0.2\\asciidoc.py --upload" + "$env:userprofile\\bin\\asciidoc-9.0.4\\asciidoc.py --upload" .format(v=version)) print("* macOS: git fetch && git checkout v{v} && " "tox -e build-release -- --upload" diff --git a/scripts/mkvenv.py b/scripts/mkvenv.py index ad5f2073e..897088539 100644 --- a/scripts/mkvenv.py +++ b/scripts/mkvenv.py @@ -87,6 +87,9 @@ def parse_args(argv: List[str] = None) -> argparse.Namespace: parser.add_argument('--skip-docs', action='store_true', help="Skip doc generation.") + parser.add_argument('--skip-smoke-test', + action='store_true', + help="Skip Qt smoke test.") parser.add_argument('--tox-error', action='store_true', help=argparse.SUPPRESS) @@ -296,12 +299,19 @@ def apply_xcb_util_workaround( def _find_libs() -> Dict[Tuple[str, str], List[str]]: """Find all system-wide .so libraries.""" all_libs: Dict[Tuple[str, str], List[str]] = {} + + if pathlib.Path("/sbin/ldconfig").exists(): + # /sbin might not be in PATH on e.g. Debian + ldconfig_bin = "/sbin/ldconfig" + else: + ldconfig_bin = "ldconfig" ldconfig_proc = subprocess.run( - ['ldconfig', '-p'], + [ldconfig_bin, '-p'], check=True, stdout=subprocess.PIPE, encoding=sys.getfilesystemencoding(), ) + pattern = re.compile(r'(?P<name>\S+) \((?P<abi_type>[^)]+)\) => (?P<path>.*)') for line in ldconfig_proc.stdout.splitlines(): match = pattern.fullmatch(line.strip()) @@ -421,7 +431,7 @@ def run(args) -> None: raise AssertionError apply_xcb_util_workaround(venv_dir, args.pyqt_type, args.pyqt_version) - if args.pyqt_type != 'skip': + if args.pyqt_type != 'skip' and not args.skip_smoke_test: run_qt_smoke_test(venv_dir) install_requirements(venv_dir) @@ -71,7 +71,8 @@ try: entry_points={'gui_scripts': ['qutebrowser = qutebrowser.qutebrowser:main']}, zip_safe=True, - install_requires=['pypeg2', 'jinja2', 'pygments', 'PyYAML', 'attrs'], + install_requires=['pypeg2', 'jinja2', 'pygments', 'PyYAML', 'attrs', + 'importlib_resources>=1.1.0; python_version < "3.9"'], python_requires='>=3.6', name='qutebrowser', version=_get_constant('version'), diff --git a/tests/conftest.py b/tests/conftest.py index 017c11ba8..fd317d6c4 100644 --- a/tests/conftest.py +++ b/tests/conftest.py @@ -282,7 +282,7 @@ def check_yaml_c_exts(): Not available yet with a nightly Python, see: https://github.com/yaml/pyyaml/issues/416 """ - if 'CI' in os.environ and sys.version_info[:2] != (3, 10): + if testutils.ON_CI and sys.version_info[:2] != (3, 10): from yaml import CLoader diff --git a/tests/end2end/data/adblock/external_logo.html b/tests/end2end/data/adblock/external_logo.html new file mode 100644 index 000000000..7fa7e9ebb --- /dev/null +++ b/tests/end2end/data/adblock/external_logo.html @@ -0,0 +1,15 @@ +<!DOCTYPE html> +<html> + <head> + <meta charset="utf-8"> + <title>External logo</title> + </head> + <body> + <p> + <b>NOTE:</> This should never be used in a test where + qutebrowser.org isn't blocked, as no network requests should be + made while running the testsuite. + </p> + <img src="https://qutebrowser.org/icons/qutebrowser.svg"> + </body> +</html> diff --git a/tests/end2end/data/adblock/qutebrowser b/tests/end2end/data/adblock/qutebrowser new file mode 100644 index 000000000..d104c0104 --- /dev/null +++ b/tests/end2end/data/adblock/qutebrowser @@ -0,0 +1 @@ +qutebrowser.org diff --git a/tests/end2end/features/private.feature b/tests/end2end/features/private.feature index fe870dded..2698555ab 100644 --- a/tests/end2end/features/private.feature +++ b/tests/end2end/features/private.feature @@ -199,3 +199,49 @@ Feature: Using private browsing - history: - active: true url: http://localhost:*/data/numbers/5.txt + + # https://github.com/qutebrowser/qutebrowser/issues/5810 + + Scenario: Using qute:// scheme after reiniting private profile + When I open about:blank in a private window + And I run :close + And I open qute://version in a private window + Then the page should contain the plaintext "Version info" + + Scenario: Downloading after reiniting private profile + When I open about:blank in a private window + And I run :close + And I open data/downloads/downloads.html in a private window + And I run :click-element id download + And I wait for "*PromptMode.download*" in the log + And I run :leave-mode + Then "Removed download *: download.bin *" should be logged + + Scenario: Adblocking after reiniting private profile + When I open about:blank in a private window + And I run :close + And I set content.host_blocking.lists to ["http://localhost:(port)/data/adblock/qutebrowser"] + And I run :adblock-update + And I wait for the message "adblock: Read 1 hosts from 1 sources." + And I open data/adblock/external_logo.html in a private window + Then "Request to qutebrowser.org blocked by host blocker." should be logged + + @pyqt!=5.15.0 # cookie filtering is broken on QtWebEngine 5.15.0 + Scenario: Cookie filtering after reiniting private profile + When I open about:blank in a private window + And I run :close + And I set content.cookies.accept to never + And I open data/title.html in a private window + And I open cookies/set?unsuccessful-cookie=1 without waiting in a new tab + And I wait until cookies is loaded + And I open cookies + Then the cookie unsuccessful-cookie should not be set + + Scenario: Disabling JS after reiniting private profile + When I open about:blank in a new window + And I run :window-only + And I set content.javascript.enabled to false + And I open about:blank in a private window + And I run :close + And I open data/javascript/enabled.html in a private window + Then the page should contain the plaintext "JavaScript is disabled" diff --git a/tests/end2end/fixtures/quteprocess.py b/tests/end2end/fixtures/quteprocess.py index 7ad739997..7d27e1166 100644 --- a/tests/end2end/fixtures/quteprocess.py +++ b/tests/end2end/fixtures/quteprocess.py @@ -691,7 +691,7 @@ class QuteProc(testprocess.Process): is_dl_inconsistency = str(self.captured_log[-1]).endswith( "_dl_allocate_tls_init: Assertion " "`listp->slotinfo[cnt].gen <= GL(dl_tls_generation)' failed!") - if 'CI' in os.environ and is_dl_inconsistency: + if testutils.ON_CI and is_dl_inconsistency: # WORKAROUND for https://sourceware.org/bugzilla/show_bug.cgi?id=19329 self.captured_log = [] self._log("NOTE: Restarted after libc DL inconsistency!") @@ -809,7 +809,7 @@ class QuteProc(testprocess.Process): testprocess.WaitForTimeout)) if timeout is None: - if 'CI' in os.environ: + if testutils.ON_CI: timeout = 15000 else: timeout = 5000 diff --git a/tests/end2end/fixtures/testprocess.py b/tests/end2end/fixtures/testprocess.py index 51352c539..7a70e4de9 100644 --- a/tests/end2end/fixtures/testprocess.py +++ b/tests/end2end/fixtures/testprocess.py @@ -20,7 +20,6 @@ """Base class for a subprocess run for tests.""" import re -import os import time import warnings @@ -234,7 +233,7 @@ class Process(QObject): self._started = True verbose = self.request.config.getoption('--verbose') - timeout = 60 if 'CI' in os.environ else 20 + timeout = 60 if utils.ON_CI else 20 for _ in range(timeout): with self._wait_signal(self.ready, timeout=1000, raising=False) as blocker: @@ -476,7 +475,7 @@ class Process(QObject): if timeout is None: if do_skip: timeout = 2000 - elif 'CI' in os.environ: + elif utils.ON_CI: timeout = 15000 else: timeout = 5000 diff --git a/tests/end2end/test_invocations.py b/tests/end2end/test_invocations.py index e34bd912d..74805cec2 100644 --- a/tests/end2end/test_invocations.py +++ b/tests/end2end/test_invocations.py @@ -417,3 +417,17 @@ def test_referrer(quteproc_new, server, server2, request, value, expected): expected = expected.replace(key, str(val)) assert headers.get('Referer') == expected + + +@pytest.mark.qtwebkit_skip +@utils.qt514 +def test_preferred_colorscheme(request, quteproc_new): + """Make sure the the preferred colorscheme is set.""" + args = _base_args(request.config) + [ + '--temp-basedir', + '-s', 'colors.webpage.prefers_color_scheme_dark', 'true', + ] + quteproc_new.start(args) + + quteproc_new.send_cmd(':jseval matchMedia("(prefers-color-scheme: dark)").matches') + quteproc_new.wait_for(message='True') diff --git a/tests/helpers/fixtures.py b/tests/helpers/fixtures.py index 3c0623dfb..6f80099bb 100644 --- a/tests/helpers/fixtures.py +++ b/tests/helpers/fixtures.py @@ -166,7 +166,7 @@ def fake_web_tab(stubs, tab_registry, mode_manager, qapp): @pytest.fixture -def greasemonkey_manager(monkeypatch, data_tmpdir): +def greasemonkey_manager(monkeypatch, data_tmpdir, config_tmpdir): gm_manager = greasemonkey.GreasemonkeyManager() monkeypatch.setattr(greasemonkey, 'gm_manager', gm_manager) diff --git a/tests/unit/browser/webengine/test_darkmode.py b/tests/unit/browser/webengine/test_darkmode.py index 3e62000d2..cd84526c3 100644 --- a/tests/unit/browser/webengine/test_darkmode.py +++ b/tests/unit/browser/webengine/test_darkmode.py @@ -32,6 +32,26 @@ def patch_backend(monkeypatch): monkeypatch.setattr(objects, 'backend', usertypes.Backend.QtWebEngine) +@pytest.mark.parametrize('qversion, enabled, expected', [ + # Disabled or nothing set + ("5.14", False, []), + ("5.15.0", False, []), + ("5.15.1", False, []), + ("5.15.2", False, []), + + # Enabled in configuration + ("5.14", True, []), + ("5.15.0", True, []), + ("5.15.1", True, []), + ("5.15.2", True, [("preferredColorScheme", "1")]), +]) +@utils.qt514 +def test_colorscheme(config_stub, monkeypatch, qversion, enabled, expected): + monkeypatch.setattr(darkmode.qtutils, 'qVersion', lambda: qversion) + config_stub.val.colors.webpage.prefers_color_scheme_dark = enabled + assert list(darkmode.settings()) == expected + + @pytest.mark.parametrize('settings, expected', [ # Disabled ({}, []), diff --git a/tests/unit/browser/webengine/test_webenginesettings.py b/tests/unit/browser/webengine/test_webenginesettings.py index 53f3e21b2..92ac8aca6 100644 --- a/tests/unit/browser/webengine/test_webenginesettings.py +++ b/tests/unit/browser/webengine/test_webenginesettings.py @@ -21,31 +21,51 @@ import logging import pytest -pytest.importorskip('PyQt5.QtWebEngineWidgets') +QtWebEngineWidgets = pytest.importorskip('PyQt5.QtWebEngineWidgets') from qutebrowser.browser.webengine import webenginesettings from qutebrowser.utils import usertypes -from qutebrowser.misc import objects -@pytest.fixture(autouse=True) -def init(qapp, config_stub, cache_tmpdir, data_tmpdir, monkeypatch): - monkeypatch.setattr(webenginesettings.webenginequtescheme, 'init', - lambda: None) - monkeypatch.setattr(objects, 'backend', usertypes.Backend.QtWebEngine) - webenginesettings.init() - config_stub.changed.disconnect(webenginesettings._update_settings) +@pytest.fixture +def global_settings(monkeypatch, default_profile): + wrapper = webenginesettings._SettingsWrapper() + settings = webenginesettings.WebEngineSettings(wrapper) + settings.init_settings() + monkeypatch.setattr(webenginesettings, '_global_settings', settings) -def test_big_cache_size(config_stub): +@pytest.fixture +def default_profile(monkeypatch): + """A profile to use which is set as default_profile. + + Note we use a "private" profile here to avoid actually storing data during tests. + """ + profile = QtWebEngineWidgets.QWebEngineProfile() + profile.setter = webenginesettings.ProfileSetter(profile) + monkeypatch.setattr(profile, 'isOffTheRecord', lambda: False) + monkeypatch.setattr(webenginesettings, 'default_profile', profile) + return profile + + +@pytest.fixture +def private_profile(monkeypatch): + """A profile to use which is set as private_profile.""" + profile = QtWebEngineWidgets.QWebEngineProfile() + profile.setter = webenginesettings.ProfileSetter(profile) + monkeypatch.setattr(webenginesettings, 'private_profile', profile) + return profile + + +def test_big_cache_size(config_stub, default_profile): """Make sure a too big cache size is handled correctly.""" config_stub.val.content.cache.size = 2 ** 63 - 1 - profile = webenginesettings.default_profile - profile.setter.set_http_cache_size() - assert profile.httpCacheMaximumSize() == 2 ** 31 - 1 + default_profile.setter.set_http_cache_size() + assert default_profile.httpCacheMaximumSize() == 2 ** 31 - 1 -def test_non_existing_dict(config_stub, monkeypatch, message_mock, caplog): +def test_non_existing_dict(config_stub, monkeypatch, message_mock, caplog, + global_settings): monkeypatch.setattr(webenginesettings.spell, 'local_filename', lambda _code: None) config_stub.val.spellcheck.languages = ['af-ZA'] @@ -59,29 +79,25 @@ def test_non_existing_dict(config_stub, monkeypatch, message_mock, caplog): assert msg.text == expected -def test_existing_dict(config_stub, monkeypatch): +def test_existing_dict(config_stub, monkeypatch, global_settings, + default_profile, private_profile): monkeypatch.setattr(webenginesettings.spell, 'local_filename', lambda _code: 'en-US-8-0') config_stub.val.spellcheck.languages = ['en-US'] webenginesettings._update_settings('spellcheck.languages') - for profile in [webenginesettings.default_profile, - webenginesettings.private_profile]: + for profile in [default_profile, private_profile]: assert profile.isSpellCheckEnabled() assert profile.spellCheckLanguages() == ['en-US-8-0'] -def test_spell_check_disabled(config_stub, monkeypatch): +def test_spell_check_disabled(config_stub, monkeypatch, global_settings, + default_profile, private_profile): config_stub.val.spellcheck.languages = [] webenginesettings._update_settings('spellcheck.languages') - for profile in [webenginesettings.default_profile, - webenginesettings.private_profile]: + for profile in [default_profile, private_profile]: assert not profile.isSpellCheckEnabled() -def test_default_user_agent_saved(): - assert webenginesettings.parsed_user_agent is not None - - def test_parsed_user_agent(qapp): webenginesettings.init_user_agent() parsed = webenginesettings.parsed_user_agent diff --git a/tests/unit/config/test_qtargs.py b/tests/unit/config/test_qtargs.py index 051956a00..b050113b4 100644 --- a/tests/unit/config/test_qtargs.py +++ b/tests/unit/config/test_qtargs.py @@ -289,20 +289,25 @@ class TestQtArgs: else: assert arg in args - @pytest.mark.parametrize('dark, new_qt, added', [ - (True, True, True), - (True, False, False), - (False, True, False), - (False, False, False), + @pytest.mark.parametrize('dark, qt_version, added', [ + (True, "5.13", False), # not supported + (True, "5.14", True), + (True, "5.15.0", True), + (True, "5.15.1", True), + (True, "5.15.2", False), # handled via blink setting + + (False, "5.13", False), + (False, "5.14", False), + (False, "5.15.0", False), + (False, "5.15.1", False), + (False, "5.15.2", False), ]) @utils.qt514 def test_prefers_color_scheme_dark(self, config_stub, monkeypatch, parser, - dark, new_qt, added): + dark, qt_version, added): monkeypatch.setattr(qtargs.objects, 'backend', usertypes.Backend.QtWebEngine) - monkeypatch.setattr(qtargs.qtutils, 'version_check', - lambda version, exact=False, compiled=True: - new_qt) + monkeypatch.setattr(qtargs.qtutils, 'qVersion', lambda: qt_version) config_stub.val.colors.webpage.prefers_color_scheme_dark = dark diff --git a/tests/unit/javascript/test_greasemonkey.py b/tests/unit/javascript/test_greasemonkey.py index 5d549ca68..5f2c94b56 100644 --- a/tests/unit/javascript/test_greasemonkey.py +++ b/tests/unit/javascript/test_greasemonkey.py @@ -41,12 +41,15 @@ test_gm_script = r""" console.log("Script is running."); """ -pytestmark = pytest.mark.usefixtures('data_tmpdir') +pytestmark = [ + pytest.mark.usefixtures('data_tmpdir'), + pytest.mark.usefixtures('config_tmpdir') +] def _save_script(script_text, filename): # pylint: disable=no-member - file_path = py.path.local(greasemonkey._scripts_dir()) / filename + file_path = py.path.local(greasemonkey._scripts_dirs()[0]) / filename # pylint: enable=no-member file_path.write_text(script_text, encoding='utf-8', ensure=True) diff --git a/tests/unit/scripts/test_problemmatchers.py b/tests/unit/scripts/test_problemmatchers.py new file mode 100644 index 000000000..98bd9c7a5 --- /dev/null +++ b/tests/unit/scripts/test_problemmatchers.py @@ -0,0 +1,38 @@ +# vim: ft=python fileencoding=utf-8 sts=4 sw=4 et: + +# Copyright 2015-2020 Florian Bruhin (The Compiler) <mail@qutebrowser.org> + +# This file is part of qutebrowser. +# +# qutebrowser is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# qutebrowser is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with qutebrowser. If not, see <http://www.gnu.org/licenses/>. + +import re + +import pytest + +from scripts.dev.ci import problemmatchers + + +@pytest.mark.parametrize('matcher_name', list(problemmatchers.MATCHERS)) +def test_patterns(matcher_name): + """Make sure all regexps are valid. + + They aren't actually Python syntax, but hopefully close enough to it to compile with + Python's re anyways. + """ + for matcher in problemmatchers.MATCHERS[matcher_name]: + for pattern in matcher['pattern']: + regexp = pattern['regexp'] + print(regexp) + re.compile(regexp) diff --git a/tests/unit/utils/test_utils.py b/tests/unit/utils/test_utils.py index ac7ed5ce7..404185548 100644 --- a/tests/unit/utils/test_utils.py +++ b/tests/unit/utils/test_utils.py @@ -818,3 +818,52 @@ def test_libgl_workaround(monkeypatch, skip): if skip: monkeypatch.setenv('QUTE_SKIP_LIBGL_WORKAROUND', '1') utils.libgl_workaround() # Just make sure it doesn't crash. + + +@pytest.mark.parametrize('duration, out', [ + ("0", 0), + ("0s", 0), + ("0.5s", 500), + ("59s", 59000), + ("60", 60), + ("60.4s", 60400), + ("1m1s", 61000), + ("1.5m", 90000), + ("1m", 60000), + ("1h", 3_600_000), + ("0.5h", 1_800_000), + ("1h1s", 3_601_000), + ("1h 1s", 3_601_000), + ("1h1m", 3_660_000), + ("1h1m1s", 3_661_000), + ("1h1m10s", 3_670_000), + ("10h1m10s", 36_070_000), +]) +def test_parse_duration(duration, out): + assert utils.parse_duration(duration) == out + + +@pytest.mark.parametrize('duration', [ + "-1s", # No sense to wait for negative seconds + "-1", + "34ss", + "", + "h", + "1.s", + "1.1.1s", + ".1s", + ".s", + "10e5s", + "5s10m", +]) +def test_parse_duration_invalid(duration): + with pytest.raises(ValueError, match='Invalid duration'): + utils.parse_duration(duration) + + +@hypothesis.given(strategies.text()) +def test_parse_duration_hypothesis(duration): + try: + utils.parse_duration(duration) + except ValueError: + pass diff --git a/tests/unit/utils/test_version.py b/tests/unit/utils/test_version.py index a82514e08..c429effee 100644 --- a/tests/unit/utils/test_version.py +++ b/tests/unit/utils/test_version.py @@ -562,11 +562,13 @@ class ImportFake: ('yaml', True), ('adblock', True), ('attr', True), + ('importlib_resources', True), ('PyQt5.QtWebEngineWidgets', True), ('PyQt5.QtWebEngine', True), ('PyQt5.QtWebKitWidgets', True), ]) self.no_version_attribute = ['sip', + 'importlib_resources', 'PyQt5.QtWebEngineWidgets', 'PyQt5.QtWebKitWidgets', 'PyQt5.QtWebEngine'] @@ -12,11 +12,12 @@ minversion = 3.15 [testenv] setenv = PYTEST_QT_API=pyqt5 - pyqt{,512,513,514,515}: LINK_PYQT_SKIP=true - pyqt{,512,513,514,515}: QUTE_BDD_WEBENGINE=true + pyqt{,512,513,514,515,5150}: LINK_PYQT_SKIP=true + pyqt{,512,513,514,515,5150}: QUTE_BDD_WEBENGINE=true cov: PYTEST_ADDOPTS=--cov --cov-report xml --cov-report=html --cov-report= passenv = PYTHON DISPLAY XAUTHORITY HOME USERNAME USER CI XDG_* QUTE_* DOCKER QT_QUICK_BACKEND PY_COLORS basepython = + py: {env:PYTHON:python3} py3: {env:PYTHON:python3} py36: {env:PYTHON:python3.6} py37: {env:PYTHON:python3.7} @@ -30,6 +31,7 @@ deps = pyqt513: -r{toxinidir}/misc/requirements/requirements-pyqt-5.13.txt pyqt514: -r{toxinidir}/misc/requirements/requirements-pyqt-5.14.txt pyqt515: -r{toxinidir}/misc/requirements/requirements-pyqt-5.15.txt + pyqt5150: -r{toxinidir}/misc/requirements/requirements-pyqt-5.15.0.txt commands = {envpython} scripts/link_pyqt.py --tox {envdir} {envpython} -bb -m pytest {posargs:tests} @@ -44,7 +46,7 @@ basepython = {env:PYTHON:python3} passenv = HOME deps = commands = - {envpython} scripts/dev/misc_checks.py all + {envpython} scripts/dev/misc_checks.py {posargs:all} [testenv:vulture] basepython = {env:PYTHON:python3} |