XFail permission tests that have changed behavior on 6.8

WebEngine now has it's own mechanism to remember permission, and it's
turned on by default. We can't disable it until PyQt picks up the new
`QWebEngineProfile::setPersistentPermissionsPolicy()`. So the first test
that prompts for a permission will persist that setting and later ones
will fail because they don't get the prompt they expect.

For now lets set them to xfail while we figure out what to do with
permission persisting for actual users. That we we can reduce the noise
in the test results!

The WebEngine permissions persistence mechanism doesn't seem to
respect whatever we are doing to separate storage per-basedir. Testing
with a temp basedir like so:

    python3 -m qutebrowser -T https://web-push-book.gauntface.com/demos/notification-examples/

I see this file has been created under my home directory:

    $ cat ~/.local/share/qutebrowser/qutebrowser/QtWebEngine/Default/permissions.json
    {"Notifications":{"https://web-push-book.gauntface.com/":true}}

I've raised an issue upstream about that here: https://bugreports.qt.io/browse/QTBUG-126595

We've got two things to think about regarding how to deal with this new
on-by-default feature:

1. what do we do for the tests? We can Disable the feature (if on new
   enough PyQt) or add a test setup step that ... restarts the browser
   and deletes the permissions.json. That's not great
2. what do we do for real users? See below

By default I would recommend disabling the webengine one since we
already have our own. BUT we can't actually disable it until PyQt
updates with the new APIs, which can take a while, and it's pretty
likely people will be using the new Qt version before PyQt updates. So
it would be best to figure out what we can do before that! Can we make
it respect the basedir data path? Can we make it write to some fake file
we throwaway? chmod +i?

Hopefully Qt makes the permission JSON respect the data path we set and
then at the very least we can remove the JSON file after a permission is
set. It'll still be a change in behavior for users on Qt 6.8 and PyQt
6.7 though as it'll likely remember permissions within a browser
instance by default, which isn't the case for our implementation
currently.

Related to: https://github.com/qutebrowser/qutebrowser/issues/8242#issuecomment-2175949686

3 files changed, 21 insertions, 10 deletions

diff --git a/pytest.ini b/pytest.ini
index fac89d32d..da826c901 100644
--- a/pytest.ini
+++ b/pytest.ini
@@ -41,6 +41,7 @@ markers =
     qt6_only: Tests which should only run with Qt 6
     qt5_xfail: Tests which fail with Qt 5
     qt6_xfail: Tests which fail with Qt 6
+    qt68_beta1_xfail: Fails on Qt 6.8 beta 1
 qt_log_level_fail = WARNING
 qt_log_ignore =
     # GitHub Actions
diff --git a/tests/end2end/conftest.py b/tests/end2end/conftest.py
index ab973175d..ffc5e9b4e 100644
--- a/tests/end2end/conftest.py
+++ b/tests/end2end/conftest.py
@@ -12,6 +12,7 @@ import pstats
 import operator
 
 import pytest
+from qutebrowser.qt import machinery
 from qutebrowser.qt.core import PYQT_VERSION, QCoreApplication
 
 pytest.register_assert_rewrite('end2end.fixtures')
@@ -23,7 +24,7 @@ from end2end.fixtures.quteprocess import (quteproc_process, quteproc,
                                           quteproc_new)
 from end2end.fixtures.testprocess import pytest_runtest_makereport
 # pylint: enable=unused-import
-from qutebrowser.utils import qtutils, utils
+from qutebrowser.utils import qtutils, utils, version
 
 
 def pytest_configure(config):
@@ -186,6 +187,12 @@ def pytest_collection_modifyitems(config, items):
          'Skipped on Windows',
          pytest.mark.skipif,
          utils.is_windows),
+        ('qt68_beta1_xfail',
+         "Fails on Qt 6.8 beta 1",
+         pytest.mark.xfail,
+         machinery.IS_QT6 and version.qtwebengine_versions(
+             avoid_init=True
+         ).webengine == utils.VersionNumber(6, 8) and version.PYQT_WEBENGINE_VERSION_STR < '6.8.0'),
     ]
 
     for item in items:
diff --git a/tests/end2end/features/prompts.feature b/tests/end2end/features/prompts.feature
index 43199fa3b..33e9e3abf 100644
--- a/tests/end2end/features/prompts.feature
+++ b/tests/end2end/features/prompts.feature
@@ -112,7 +112,7 @@ Feature: Prompts
         Then the javascript message "notification permission granted" should be logged
         And "Added quickmark test for *" should be logged
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: Async question interrupted by blocking one
         Given I have a fresh instance
         When I set content.notifications.enabled to ask
@@ -251,6 +251,7 @@ Feature: Prompts
         And I run :click-element id button
         Then the javascript message "geolocation permission denied" should be logged
 
+    @qt68_beta1_xfail
     Scenario: geolocation with ask -> false
         When I set content.geolocation to ask
         And I open data/prompt/geolocation.html in a new tab
@@ -259,6 +260,7 @@ Feature: Prompts
         And I run :prompt-accept no
         Then the javascript message "geolocation permission denied" should be logged
 
+    @qt68_beta1_xfail
     Scenario: geolocation with ask -> false and save
         When I set content.geolocation to ask
         And I open data/prompt/geolocation.html in a new tab
@@ -268,6 +270,7 @@ Feature: Prompts
         Then the javascript message "geolocation permission denied" should be logged
         And the per-domain option content.geolocation should be set to false for http://localhost:(port)
 
+    @qt68_beta1_xfail
     Scenario: geolocation with ask -> abort
         When I set content.geolocation to ask
         And I open data/prompt/geolocation.html in a new tab
@@ -278,7 +281,7 @@ Feature: Prompts
 
     # Notifications
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: Always rejecting notifications
         Given I have a fresh instance
         When I set content.notifications.enabled to false
@@ -286,7 +289,7 @@ Feature: Prompts
         And I run :click-element id button
         Then the javascript message "notification permission denied" should be logged
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: Always accepting notifications
         Given I have a fresh instance
         When I set content.notifications.enabled to true
@@ -294,7 +297,7 @@ Feature: Prompts
         And I run :click-element id button
         Then the javascript message "notification permission granted" should be logged
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: notifications with ask -> false
         Given I have a fresh instance
         When I set content.notifications.enabled to ask
@@ -304,7 +307,7 @@ Feature: Prompts
         And I run :prompt-accept no
         Then the javascript message "notification permission denied" should be logged
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: notifications with ask -> false and save
         Given I have a fresh instance
         When I set content.notifications.enabled to ask
@@ -315,7 +318,7 @@ Feature: Prompts
         Then the javascript message "notification permission denied" should be logged
         And the per-domain option content.notifications.enabled should be set to false for http://localhost:(port)
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: notifications with ask -> true
         Given I have a fresh instance
         When I set content.notifications.enabled to ask
@@ -325,7 +328,7 @@ Feature: Prompts
         And I run :prompt-accept yes
         Then the javascript message "notification permission granted" should be logged
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: notifications with ask -> true and save
         Given I have a fresh instance
         When I set content.notifications.enabled to ask
@@ -347,7 +350,7 @@ Feature: Prompts
         And I run :mode-leave
         Then the javascript message "notification permission aborted" should be logged
 
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: answering notification after closing tab
         Given I have a fresh instance
         When I set content.notifications.enabled to ask
@@ -521,7 +524,7 @@ Feature: Prompts
 
     # https://github.com/qutebrowser/qutebrowser/issues/1249#issuecomment-175205531
     # https://github.com/qutebrowser/qutebrowser/pull/2054#issuecomment-258285544
-    @qtwebkit_skip
+    @qtwebkit_skip @qt68_beta1_xfail
     Scenario: Interrupting SSL prompt during a notification prompt
         Given I have a fresh instance
         When I set content.notifications.enabled to ask