Getting stacktraces on crashes ============================== :toc: The Compiler When there is a fatal crash in qutebrowser - most of the times a https://en.wikipedia.org/wiki/Segmentation_fault[segfault] - the crash report usually doesn't contain much viable information, as these crashes usually happen inside of the Qt mainloop in C++. To know what the issue is, a https://en.wikipedia.org/wiki/Stack_trace[stack trace] with https://en.wikipedia.org/wiki/Debug_symbol[debugging symbols] is required. The rest of this guide is quite Linux specific, though there is a <> at the end. Crashes which can be reproduced ------------------------------- If a crash can be reproduced, packages with debugging symbols should be installed, and the crash should be reproduced under gdb. Getting debugging symbols ~~~~~~~~~~~~~~~~~~~~~~~~~ Debian/Ubuntu/... ^^^^^^^^^^^^^^^^^ For Debian based systems (Debian, Ubuntu, Linux Mint, ...), debug information is for QtWebEngine is available in a dedicated repository. Enable that repository (https://wiki.debian.org/HowToGetABacktrace#Installing_the_debugging_symbols[Debian], https://wiki.ubuntu.com/Debug%20Symbol%20Packages[Ubuntu], https://www.linuxmint.com/rel_tessa_mate_whatsnew.php[Linux Mint]) and install the debug packages: ---- # apt install python3-dbg python3-pyqt5-dbg python3-pyqt5.qtwebengine-dbg libqt5webengine5-dbgsym libqt5webenginecore5-dbgsym ---- or with the QtWebKit backend: ---- # apt install python3-dbg python3-pyqt5-dbg python3-pyqt5.qtwebkit-dbg libqt5webkit5-dbg ---- Fedora ^^^^^^ For Fedora you first need to install the dnf/yum-utils: ---- # dnf install dnf-utils ---- Or: ---- # yum install yum-utils ---- Then install the needed debuginfo packages: ---- # debuginfo-install python3 qt5-qtwebengine python3-qt5-webengine python3-qt5-base python-qt5 python3-qt5 python3-qt5-webkit ---- Archlinux ^^^^^^^^^ For Archlinux, no debug information is provided. You can either compile Qt yourself (which will take a few hours even on a modern machine) or use debugging symbols compiled/packaged by me (x86_64 only). To install my pre-built packages ++++++++++++++++++++++++++++++++ First download and sign the key: ---- # pacman-key -r 0xD6A1C70FE80A0C82 $ pacman-key -f 0xD6A1C70FE80A0C82 Key fingerprint = 14AF EC28 70C6 4863 C5C7 ACCB D6A1 C70F E80A 0C82 # pacman-key --lsign-key 0xD6A1C70FE80A0C82 ---- Then edit your `/etc/pacman.conf` to add the repository to the bottom: ---- [qt-debug] Server = https://qutebrowser.org/qt-debug/$arch ---- Then install the packages: ---- # pacman -Suy python-pyqt5-debug qt5-base-debug qt5-webkit-debug qt5-webengine-debug ---- The `-debug` packages conflict with the non-debug variants - it's safe to remove them. To compile by yourself ++++++++++++++++++++++ Note that building Qt will likely take multiple hours, even on a recent system. I'd also expect it to take around 6 GB of RAM and 30 GB of disk space for a successful compile run. ---- $ git clone https://github.com/qutebrowser/qt-debug-pkgbuild.git $ cd qt-debug-pkgbuild $ export DEBUG_CFLAGS='-ggdb3 -fvar-tracking-assignments -Og' $ export DEBUG_CXXFLAGS='-ggdb3 -fvar-tracking-assignments -Og' $ cd qt5 $ makepkg -si --pkg qt5-base-debug,qt5-webkit-debug,qt5-webengine-debug $ cd ../pyqt5 $ makepkg -si --pkg python-pyqt5-debug ---- Getting the stack trace ~~~~~~~~~~~~~~~~~~~~~~~ First install `gdb` on your system if it's not installed already. Then run qutebrowser directly inside gdb like this: ---- $ gdb -ex r --args $(readlink -f $(which python3)) -m qutebrowser --debug --temp-basedir ---- Note qutebrowser/gdb will take a long time to start. After you reproduce the crash, you should now see something like: ---- Program received signal SIGSEGV, Segmentation fault. ... (gdb) ---- Now enter these commands at the gdb prompt: ---- (gdb) set pagination off (gdb) set logging overwrite on (gdb) set logging on (gdb) bt (gdb) quit ---- This will create a `gdb.txt` in your current directory. Copy the last few lines of the debug log (before you got the gdb prompt) and the full content of `gdb.txt` into the bug report. Please also add some words about what you were doing (or what pages you visited) before the crash happened. Crashes which can NOT be reproduced ----------------------------------- If you cannot reproduce the problem, you need to check if a coredump got written somewhere. You should not install debug symbols as they won't match the generated coredump. First install `gdb` on your system if it's not installed already. Then check the file `/proc/sys/kernel/core_pattern` on your system. If it does not start with a `|` character (pipe), check if there is a file named `core` or `core.NNNN` in the directory from that file, or in the current directory. If so, execute gdb like this: ---- $ gdb $(readlink -f $(which python3)) /path/to/core ---- If your `/proc/sys/kernel/core_pattern` contains something like `|/usr/lib/systemd/systemd-coredump`, use `coredumpctl` to run gdb: ---- $ coredumpctl gdb $(readlink -f $(which python3)) ---- Getting the stack trace ~~~~~~~~~~~~~~~~~~~~~~~ Now enter these commands at the gdb prompt: ---- (gdb) set pagination off (gdb) set logging overwrite on (gdb) set logging on (gdb) bt (gdb) quit ---- Copy the content of `gdb.txt` into the bug report. Please also add some words about what you were doing (or what pages you visited) before the crash happened. [[windows]] For Windows ----------- First install https://www.microsoft.com/en-us/download/details.aspx?id=58210[DebugDiag] from Microsoft. If you see the _qutebrowser.exe has stopped working_ window, do not click "Close the program". Instead, open your task manager, there right-click on `qutebrowser.exe` and select "Create dump file". Remember the path of the dump file displayed there. If you do not see such a window, instead run *DebugDiag 2 Collection* while qutebrowser is still running. There, use *Add Rule* -> *Crash* -> *A specific process* and select `qutebrowser.exe`. Accept the *Advanced Configuration* as-is and select a location to save dump files. Finally, tell DebugDiag to activate the rule and reproduce the crash. After a while, a log file (`.txt`) and crash dump should appear in that directory. Finally, run the *DebugDiag 2 Analysis* tool. There, check *CrashHangAnalysis* and add your crash dump via *Add Data files*. Then click *Start analysis*. Close the Internet Explorer which opens when it's done and use the folder-button at the top left to get to the reports. There, find the report file (as well as the logfile, if any), zip them (important, as some mail providers like GMail corrupt the file otherwise) and send them to mail@qutebrowser.org.