| Age | Commit message (Collapse) | Author |
|---|
|
My virtualenv I used to run webkit has rotted long ago and I don't remember
how I set it up. There is a PyQtWebKit project on PyPI but I don't know
who that's published by.
So I figured I would write some notes for myself on using the docker container
used for CI instead. I chose to mount the current directory (which is
presumably a qutebrowser checkout!) directly into the container instead of
cloning it so I could have quicker feedback between making code changes and
running tests.
Then there's a couple of things that stem from that. Since the user in the
container is different from the one in the host we have to move some things
that are normally written to the current directory to be written elsewhere.
There are other ways to approach this (eg you can add `-u $(id -u)` to the
docker command line, although that makes things a bit confusing in the
container) but arguably it's good for the container not to be able to write to
the host, hence making that volume read only.
The TOX_WORK_DIR trick is from
[here](https://github.com/tox-dev/tox/issues/20), apart from with
`{toxinidir}` in it too because the pyroma env was failing with just
`.tox`, saying the pyroma binary needed to be in the allowlist, possibly
it was doing full path matching without normalizing.
The hypothesis folks
[here](https://github.com/HypothesisWorks/hypothesis/issues/2367#issuecomment-595524571)
say if you want to override the examples DB location with an env var to
do it yourself. It's actually only a warning from hypothesis, it says it
falls back to an in-memory DB, but I guess the tests run with
warnings-are-errors. You can also pass `database=None` to make
hypothesis skip example storage altogether.
I'm using tox to run commands in a virtualenv with the right stuff in it
because, uh, because I was copying the CI workflow actually. I just found out
about the `exec` subcommand to override the `commands` defined for the env,
neat! One point of awkwardness about that is that since we are using the
PyQt from the OS we need any virtualenv we use to have access to the OS
packages, which isn't the default for virtualenvs created by tox. The
text envs use the link_pyqt script for that but if you are using this
container and the first thing you do is run `tox exec` then that
wouldn't have been run. So I'm setting `VIRTUALENV_SYSTEM_SITE_PACKAGES`
to tell tox to always make the system packages available in the
virtualenvs it manages.
I did try using the mkvenv script instead of tox but it complained when
trying to install the current directory in editable mode because
setup.py tries to write to a git-commit-id file.
|
|
When taking screenshots of the test process running under xvfb it's
offset from the top left corner, the default geometry of qutebrowser is
800x600+50+50. The default size of pytest-xvfb is 800x600, which means
part of the browser window is outside the X11 screen and doesn't get
captured.
This commit duplicates the width and height from the default geometry in
mainwindow.py but sets the x and y offsets to zero so that the browser
window is fully contained within the X11 window.
|
|
This commit takes a screenshot of the active browser window when an
end2end test fails. When running on CI a zip file of screenshots will be
attached to the run summary as an artifact. When run locally screenshots
will be left in /$TMPDIR/pytest-screenshots/.
The screenshot is of the Xvfb screen that the tests are running under.
If there are multiple windows open it will likely only show the active
window because a) we aren't running with a window manager b) the Xvfb
display is, by default, the same size as the browser window.
I'm using pillow ImageGrab, the same as pyvirtualdisplay.smartdisplay. I'm
getting the display number from the pytest-xvfb plugin and formatting it
appropriately (pyvirtualdisplay has an already formatted one which is used by
the smartdisplay, but that's not accessible).
Pillow is now a requirement for running the tests. I though about making
it gracefully not required but I'm not sure how to inform the user with
a warning from pytest, or if they would even want one. Maybe we could
add a config thing to allow not taking screenshots?
I had to bump the colordepth config for pytest-xvfb otherwise pillow
complained that the default 16bit color depth wasn't supported.
I'm saving screenshots to a temp dir because I don't want to put them in
my workdir when running locally. I want to clear the directory for each
run so that you don't get confused by looking at old images. I'm not
100% sure about the lifecycle of the process classes though. Eg if we
have two processes they might both try to create the output directory.
I'm using pytest.session.stash to save the directory so perhaps the
lifecycle of the stash will handle that? Not sure.
Ideally the images would be uploaded somewhere where we could click
through and open them in the browser without having to download a zip
file, but I'm not sure how to achieve that.
It would be nice to print in the test logs that a screenshot was saved
and where to. Just so you could copy paste the filename instead of
having to match the sanitized filename against failing test names. But I
don't know how to log stuff from this stage in the pytest lifecycle.
TODO:
* I'm not sure that the screenshot captures the whole browser window?
Maybe the browser windows is bigger than the X11 display?
Closes: #7625
|
|
Previously (a209c86c55a4) I've added a delay in browser code before sending
events to the page to account with a race condition where events weren't
processed after navigating. Then I had to add extra checks to tests that had
tight timing requirements.
That was for click-element, this commit reverts an attempt at the same
strategy for fake-key and instead adds wait statements in the tests that where
hitting the original race condition (sending events "too soon" after a
navigation).
The reason for the different approach here is that after adding the delay in
fake-key I had to add an extra "wait for log line" message to a few tests with
tight timing requirements to watch for a Tab key press. That worked great for
webengine but it made some tests start failing on webkit. We don't seem to get
that log message on webkit. I've got no-idea why and frankly think I've spent
way too much time on just a handful of tests already.
It's unfortunate we have to add manually delays in the e2e tests. It makes me
think if anyone else adds a test in the future with this combination of steps
(open page, run fake-key) they'll run into the same issue and it'll be hard to
spot. Oh well, we'll deal with that when it comes up.
The tests that where failing on webkit are the ones in caret.feature
touched in this commit.
Reverts included in this commit:
Revert "Delay fake-key events by 10ms"
This reverts commit 9f050c7460c42f317ceaa20b320e97d371a2c0a0.
Revert "Wait for evidence of Tab key press in tests before proceeding"
This reverts commit d47d247941c4b1fe3adac0dae8be301b36aec85b.
|
|
In the previous commit 9f050c7460c4 "Delay fake-key events by 10ms" I delayed
events for fake-key by 10ms to deal with a timing issue with sending events
right after a navigation not being processed.
Just like last time that's caused a few tests with tight timing constraints to
break because they proceed to the next command before the fake-key events have
taken effect.
Maybe it would have been better to just put the waits in the e2e tests in the
first case instead of in the production code? Well, we'll see. Maybe I'll
never have to deal with this again.
|
|
Similar to a209c86c55a4 "Delay QEvent driven mouse clicks by 10ms" it seems
that sending fake-key QEvents immediately after a navigation causes the events
to not be processed.
This is causing e2e tests in keyinput.feature to fail in a flaky, but
frequent, manner. This can also be resolved in some other ways like putting
a wait in the e2e tests like so:
When I open data/keyinput/log.html
And I wait 0.01s
And I run :fake-key <Escape>
But relying on maintainers to remember to do that seems error prone. If this
10ms delay turns out to be something to get rid of we could try keep this
delay to be used in less cases:
1. add some magic to e2e tests where it compares the previous and current line
and if it sees an open and a click-element or fake-key line next to each
other it delays for a bit
2. in the application code in tab.send_event() store the time of last
navigation and queue events for sending till after that
The affected tests in this case where:
tests/end2end/features/test_keyinput_bdd.py::test_fakekey_sending_key_to_the_website
tests/end2end/features/test_keyinput_bdd.py::test_fakekey_sending_special_key_to_the_website
tests/end2end/features/test_keyinput_bdd.py::test_fakekey_sending_keychain_to_the_website
|
|
Exit prompt mode on the last test in downloads.feature because it's
preventing the next tests in editor.feature from running. Screenshots of
the first failing test in editor.feature shows a prompt open and
jsprompt open in the background.
This shouldn't happen as there is a module level fixture in conftest.py
that is supposed to close and re-open the browser process between each
feature file. That bears more examination but for now this change looks
pretty painless.
|
|
I've added a 10ms delay when sending a fake click via QEvents to help with a
timing issue in the prompt e2e tests in Qt6.7. Apparently that throws off the
tight timing of this one test! I guess the `:scroll bottom` comes before the
iframe has been clicked.
I'm a little worried that if we are depending on timing stuff like this we
should be looking at a more thorough solution. Maybe the "and follow" suffix
can trigger it to wait for "Clicked *editable element!"?
Also I can try to reduce the delay in the click, but 10ms is pretty short
already.
|
|
On CI with Qt 6.7 we are seeing several tests failing in a flaky, but
frequent, manner. These tests all seem to be doing :click-element and
sending a "fake" click, as in one driven by QEvents, to a button on the
test page after opening the page in an existing tab. In the logs we see
that the element was identified with JS correctly but we don't see any
evidence of the actual click after that.
I've been testing locally with `python3 -m pytest
tests/end2end/features/test_prompts_bdd.py -k test_javascript_confirm`.
Delaying the click event by as little as 5ms seems to make the tests
consistently pass. I'm setting it to an arbitrary 10ms here for no good
reason. It's unfortunate that we have to change production code to make
tests pass, it's unlikely that a 10ms window will affect real usage, and
the problem is only immediately after a navigation. If we can't find a
better way to resolve this and want to get rid of the 10ms delay when no
needed we could maybe look at when the last navigation time for the tab
was to decide whether to delay or not.
I also tried changing all the "open in" lines in the failing tests to be
"open in new tab" and that seemed to make the flaky tests pass
consistently too. But relying on people writing tests in a specific way
wouldn't really be fixing the problem.
Additionally, running just one test at a time (with `taskset -c 1 ...`)
made all the tests pass consistently. So it seems like the root cause
could have something to do with that, somehow. Or maybe just running the
pytest process and the browser process on the same CPU causes enough of
a delay to not trigger it.
I don't know what the root cause of this change is, or if we can use a
more event based way of knowing when the page (or its focus proxy) is
ready to receive events. I've tried digging through events and debugging
webengine and haven't got anywhere, hopefully this delay is painless
enough that it'll do. To summarize, the page, or focus proxy, doesn't
process events (all events, just mouse events?) immediately after
navigating.
I initially tried delaying all events in `tab.send_event()`, but that
caused some caret tests to fail (the `selectionfollow_with_link_tabbing`
e2e ones), not sure why. So I walked back to just delaying click events.
I've seen some intermittent flakyness in
`test_misc_bdd.py::test_clicking_on_focused_element` and
`tests/end2end/features/test_keyinput_bdd.py::test_fakekey_sending_keychain_to_the_website`
which may indicate that `:fake-key` needs the same treatment. We'll see
how things stand once it's been stable on the main branch for a while
instead of being muddled up in this investigation branch.
I thought the issue might be that the RenderWidgetHostViewQt was being swapped out
on the new page load and the old one being sent the event. So I added a bunch
of debug logging to try to get a better view of that. None of this debug
logging is showing up when the tests fail so it seems that isn't the case.
The tests failing in the last four bleeding edge qt6 tests before these
changes were (test, count):
('tests/end2end/features/test_keyinput_bdd.py::test_fakekey_sending_special_key_to_the_website', 2),
('tests/end2end/features/test_prompts_bdd.py::test_using_contentjavascriptalert', 2),
('tests/end2end/features/test_editor_bdd.py::test_select_two_files_with_multiple_files_command', 2),
('tests/end2end/features/test_misc_bdd.py::test_clicking_first_element_matching_a_selector', 2),
('tests/end2end/features/test_misc_bdd.py::test_clicking_on_focused_element', 2),
('tests/end2end/features/test_prompts_bdd.py::test_javascript_prompt_with_default', 2),
('tests/end2end/features/test_prompts_bdd.py::test_using_contentjavascriptprompt', 2),
('tests/end2end/features/test_prompts_bdd.py::test_javascript_confirm__aborted', 2),
('tests/end2end/features/test_editor_bdd.py::test_file_selector_deleting_temporary_file', 1),
('tests/end2end/features/test_keyinput_bdd.py::test_fakekey_sending_keychain_to_the_website', 1),
('tests/end2end/features/test_prompts_bdd.py::test_javascript_confirm__no', 1),
('tests/end2end/features/test_prompts_bdd.py::test_javascript_confirm_with_default_value', 1),
('tests/end2end/test_insert_mode.py::test_insert_mode[100-input.html-qute-input-keypress-awesomequtebrowser]', 1),
('tests/end2end/features/test_misc_bdd.py::test_clicking_an_element_by_position', 1),
('tests/end2end/features/test_prompts_bdd.py::test_javascript_confirm__yes', 1),
('tests/end2end/features/test_prompts_bdd.py::test_javascript_confirm_with_invalid_value', 1),
('tests/end2end/test_insert_mode.py::test_insert_mode[125-input.html-qute-input-keypress-awesomequtebrowser]', 1),
('tests/end2end/features/test_editor_bdd.py::test_select_two_files_with_single_file_command', 1),
('tests/end2end/features/test_keyinput_bdd.py::test_fakekey_sending_key_to_the_website', 1),
('tests/end2end/features/test_misc_bdd.py::test_clicking_an_element_by_id_with_dot', 1),
('tests/end2end/features/test_prompts_bdd.py::test_javascript_alert_with_value', 1)
For debugging in GBD under test I ran the test process under GDB and
passed a script to GDB to print out a message, and then continue, when
it hit a breakpoint. Unfortunately, the tests always passed when run
under GDB.
For that I changed `_executable_args()` in quteprocess to return `"gdb",
"-q -x handleMouseEvent-breakpoint -args".split() + [executable] + args`
where `handleMouseEvent-breakpoint` is:
set breakpoint pending on
set debuginfod enabled off
break RenderWidgetHostViewQtDelegateClient::handleMouseEvent
commands
call (void) fprintf(stderr, "Entering handleMouseEvent\n")
continue
end
run
The tests were consistently passing, but erroring on the invalid gdb log lines.
|
|
qutebrowser/feat/7187_chromium_security_patch_in_version
Show chromium security patch version in :version
|
|
See https://bugreports.qt.io/browse/QTBUG-124386
Fixes #8143
|
|
Update dependencies
|
|
|
|
|
|
We've already ignored this in the tests, it started showing up in the
nightly builds a few days ago, only on macos for some reason.
|
|
Changed line 35 of tor_identity userscript to stop it throwing an err…
|
|
'--control-port' is passed
|
|
|
|
Default since flake8-pytest-style 2.0.0: https://github.com/m-burst/flake8-pytest-style#change-log
|
|
Default since 0.991: https://mypy-lang.blogspot.com/2022/11/mypy-0990-released.html
|
|
Update dependencies
|
|
This only shows up on the webkit CI job.
Probably something to do with the fact that we are using webkit builds
from the distant archives. I'm sure it'll break for real one of these
days.
|
|
It's a new externalised dependency of jaraco.context, doesn't seem to
have a RTD site yet.
ref: https://github.com/jaraco/jaraco.context/commit/e13fc7f2b379683c326153a3d6f4d2800f812fd0
|
|
flake8 got a new warning about a module name shadowing a builtin module: https://github.com/gforcada/flake8-builtins/pull/121
Probably would have been safe enough to ignore it. But I don't think
moving it is that hard anyway. Hopefully I didn't miss anything!
|
|
|
|
See https://github.com/qutebrowser/qutebrowser/issues/8159
|
|
A couple of tests seem to be failing because there is two windows open and
they don't expect it. I haven't fixed the root cause, I looked through the
logs and couldn't see why a second window was open. But if this makes the
tests pass, I guess we can address that if someone reports it.
Also changed `_get_scroll_values()` to handle multiple tabs/windows being open
too while I was there.
Example logs: https://github.com/qutebrowser/qutebrowser/actions/runs/8548730512/job/23422922448
Sessions file from the test logs:
Current session data:
windows:
- geometry: !!binary |
AdnQywADAAAAAAAyAAAAMgAAA1EAAAKJAAAAMgAAADIAAANRAAACiQAAAAAAAAAAAyAAAAAyAAAA
MgAAA1EAAAKJ
tabs:
- active: true
history:
- active: true
last_visited: '2024-04-04T03:21:00'
pinned: false
scroll-pos:
x: 0
y: 0
title: about:blank
url: about:blank
zoom: 1.0
- active: true
geometry: !!binary |
AdnQywADAAAAAAAyAAAAMgAAA1EAAAKJAAAAMgAAADIAAANRAAACiQAAAAAAAAAAAyAAAAAyAAAA
MgAAA1EAAAKJ
tabs:
- active: true
history:
- active: true
last_visited: '2024-04-04T03:21:03'
pinned: false
scroll-pos:
x: 0
y: 40
title: Scrolling
url: http://localhost:39235/data/scroll/simple.html
zoom: 1.0
|
|
In the bleeding edge CI tests I'm seeing:
IGNORED: Path override failed for key base::DIR_APP_DICTIONARIES and path '/__w/qutebrowser/qutebrowser/.tox/bleeding/lib/python3.11/sitePath override failed for key base::DIR_APP_DICTIONARIES and path '/__w/qutebrowser/qutebrowser/.tox/bleeding/lib/python3.11/site-packages/PyQt6/Qt6/libexec/qtwebengine_dictionaries'
INVALID: -packages/PyQt6/Qt6/libexec/qtwebengine_dictionaries'
Where the second line looks like maybe spill over from the first one
that is for some reason being printed twice and interleaved! Maybe
different processes or threads?
I did a regex for "line ending with this and with no spaces in it" which
is hopefully safe enough to not pick up a wrong line accidentally.
|
|
See https://www.reddit.com/r/qutebrowser/comments/1bomb3h/closing_popups_within_a_webpage_and_toggling/
|
|
|
|
|
|
|
|
|
|
|
|
Similarly to the last commit (982b8bdcecfba6fc1687a6a4942b9e3ab3221eb7),
we run into a race between autofocus triggering and the test trying to
click the focused element.
This also affects the "when there is none" test, which loses the focus
before it has been set, thus ending up with a focused element but expecting
none.
We fix both in one go by manually focusing things with a tab keypress
instead of using autofocus.
See #8145 and #5390.
|
|
Fixes #8145, see #5390.
As long as we don't have a solution to get notified about focus happening
(#2471 possibly?), it looks like there is no better way to get notified
about this, so a delay will need to do for now.
|
|
|
|
Right now, the Riverbank PyPI server has a PyQt5 sdist but no wheels.
|
|
Seems to break in bleeding edge tests
|
|
Fails bleeding edge CI due to warnings
|
|
Starting with PyInstaller 6.5.0, it imports Qt bindings early, due to
this change: https://github.com/pyinstaller/pyinstaller/pull/8315
We warn about this in order to avoid unintentional early Qt imports in
qutebrowser. However, in the case of using PyInstaller, we just suppress
the warning now, as it's not us to blame.
See https://github.com/qutebrowser/qutebrowser/pull/8123
|
|
|
|
|
|
With a Qt 6.7 developer build, the tests fail with:
ASSERT failure in QtGlobalStatic::ApplicationHolder<QAS>::PlainType*
QtGlobalStatic::ApplicationHolder<QAS>::pointer() [with QAS =
{anonymous}::Q_QAS_qtlsbLoader; PlainType = QFactoryLoader]: "The
application static was used without a QCoreApplication instance", file
.../qtbase/src/corelib/kernel/qapplicationstatic.h, line 54
Fatal Python error: Aborted
[...]
Current thread 0x00007c18bb3f3740 (most recent call first):
File ".../tests/unit/browser/webkit/test_certificateerror.py", line 23 in <module>
See https://codereview.qt-project.org/c/qt/qtbase/+/495239
|
|
This is going to be default behavior in mypy 2.0, see:
- #8123
- https://mypy-lang.blogspot.com/2024/03/mypy-19-released.html
- https://mypy.readthedocs.io/en/stable/command_line.html#cmdoption-mypy-local-partial-types
|
|
Update dependencies
|
|
jaraco added some utility libraries they maintain to the keyring package
that they also maintain.
Also update changelog URLs to be consistent since they have a "skeleton"
project which lays everything out and publishes all their projects the
same. Should we be preferring GH or RTD links? idk
|
|
|
|
qutebrowser/dependabot/github_actions/softprops/action-gh-release-2
build(deps): bump softprops/action-gh-release from 1 to 2
|
|
Update dependencies
|
