Run just some flaky tests, to allow for iterating faster

This is not all of the tests that are being flaky. Just the few I looked
at the other day. Hopefully some of them keep failing in this much
smaller set than our usual test suite, otherwise I'll go add some more.

Not sure what I'm going to add in next, probably some code to probe the
state of the page. Run a simple event handler we know works in
isolation or something. Make sure things are still registered. Turn on
chrome debug logs?

7 files changed, 12 insertions, 1 deletions

diff --git a/pytest.ini b/pytest.ini
index f2f746284..cda8acfe8 100644
--- a/pytest.ini
+++ b/pytest.ini
@@ -41,6 +41,7 @@ markers =
     qt6_only: Tests which should only run with Qt 6
     qt5_xfail: Tests which fail with Qt 5
     qt6_xfail: Tests which fail with Qt 6
+    e2e_flaky: End-to-end tests which are fairly consistently failing to find JS log message
 qt_log_level_fail = WARNING
 qt_log_ignore =
     # GitHub Actions
diff --git a/tests/end2end/features/editor.feature b/tests/end2end/features/editor.feature
index 9ca855d27..d96730f60 100644
--- a/tests/end2end/features/editor.feature
+++ b/tests/end2end/features/editor.feature
@@ -203,6 +203,7 @@ Feature: Opening external editors
         And I run :click-element id single_file
         Then the javascript message "Files: 1.txt" should be logged
 
+    @e2e_flaky
     Scenario: Select two files with single file command
         When I setup a fake single_file fileselector selecting "tests/end2end/data/numbers/1.txt tests/end2end/data/numbers/2.txt" and writes to a temporary file
 
@@ -220,6 +221,7 @@ Feature: Opening external editors
         And I run :click-element id multiple_files
         Then the javascript message "Files: 1.txt" should be logged
 
+    @e2e_flaky
     Scenario: Select two files with multiple files command
         When I setup a fake multiple_files fileselector selecting "tests/end2end/data/numbers/1.txt tests/end2end/data/numbers/2.txt" and writes to a temporary file
 
@@ -250,6 +252,7 @@ Feature: Opening external editors
 
     ## Select folder when expecting file
 
+    @e2e_flaky
     Scenario: Select folder for file
         When I set fileselect.handler to external
         When I setup a fake single_file fileselector selecting "tests/end2end/data/numbers" and writes to a temporary file
diff --git a/tests/end2end/features/keyinput.feature b/tests/end2end/features/keyinput.feature
index 3ab5d2434..83096486e 100644
--- a/tests/end2end/features/keyinput.feature
+++ b/tests/end2end/features/keyinput.feature
@@ -30,6 +30,7 @@ Feature: Keyboard input
         When I run :fake-key <blub>
         Then the error "Could not parse '<blub>': Got invalid key!" should be shown
 
+    @e2e_flaky
     Scenario: :fake-key sending key to the website
         When I open data/keyinput/log.html
         And I run :fake-key x
@@ -46,6 +47,7 @@ Feature: Keyboard input
         And I wait for "Focus object changed: <qutebrowser.browser.webkit.webview.WebView *>" in the log
         Then the error "No focused webview!" should be shown
 
+    @e2e_flaky
     Scenario: :fake-key sending special key to the website
         When I open data/keyinput/log.html
         And I run :fake-key <Escape>
diff --git a/tests/end2end/features/misc.feature b/tests/end2end/features/misc.feature
index 1872ca5a6..c7a4001ee 100644
--- a/tests/end2end/features/misc.feature
+++ b/tests/end2end/features/misc.feature
@@ -500,6 +500,7 @@ Feature: Various utility commands.
         And I run :click-element position 20,42.001
         Then the error "String 20,42.001 does not match X,Y" should be shown
 
+    @e2e_flaky
     Scenario: Clicking on focused element when there is none
         When I open data/click_element.html
         # Need to loose focus on input element
@@ -508,6 +509,7 @@ Feature: Various utility commands.
         And I run :click-element focused
         Then the error "No element found with focus!" should be shown
 
+    @e2e_flaky
     Scenario: Clicking on focused element
         When I open data/click_element.html
         And I run :click-element focused
diff --git a/tests/end2end/features/prompts.feature b/tests/end2end/features/prompts.feature
index 32bdd29e7..d8ed905ba 100644
--- a/tests/end2end/features/prompts.feature
+++ b/tests/end2end/features/prompts.feature
@@ -45,6 +45,7 @@ Feature: Prompts
         And I run :prompt-accept
         Then the javascript message "Prompt reply: prompt test" should be logged
 
+    @e2e_flaky
     Scenario: Javascript prompt with default
         When I open data/prompt/jsprompt.html
         And I run :click-element id button-default
@@ -245,6 +246,7 @@ Feature: Prompts
 
     # Geolocation
 
+    @e2e_flaky
     Scenario: Always rejecting geolocation
         When I set content.geolocation to false
         And I open data/prompt/geolocation.html in a new tab
diff --git a/tests/end2end/test_insert_mode.py b/tests/end2end/test_insert_mode.py
index abf32dbde..311741162 100644
--- a/tests/end2end/test_insert_mode.py
+++ b/tests/end2end/test_insert_mode.py
@@ -16,6 +16,7 @@ import pytest
                  'cutebrowser', marks=pytest.mark.flaky),
 ])
 @pytest.mark.parametrize('zoom', [100, 125, 250])
+@pytest.mark.e2e_flaky
 def test_insert_mode(file_name, elem_id, source, input_text, zoom,
                      quteproc, request):
     url_path = 'data/insert_mode_settings/html/{}'.format(file_name)
diff --git a/tox.ini b/tox.ini
index 238c532f3..1306e242a 100644
--- a/tox.ini
+++ b/tox.ini
@@ -54,7 +54,7 @@ deps =
     pyqt66: -r{toxinidir}/misc/requirements/requirements-pyqt-6.6.txt
 commands =
     !pyqt-!pyqt515-!pyqt5152-!pyqt62-!pyqt63-!pyqt64-!pyqt65-!pyqt66: {envpython} scripts/link_pyqt.py --tox {envdir}
-    {envpython} -bb -m pytest {posargs:tests}
+    {envpython} -bb -m pytest -m e2e_flaky tests/end2end
     cov: {envpython} scripts/dev/check_coverage.py {posargs}
 
 [testenv:py-qt5]