aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/pprof
AgeCommit message (Collapse)Author
2021-04-27runtime/pprof: fix block profile biasFelix Geisendörfer
Block profiles were biased towards infrequent long events over frequent short events. This fix corrects the bias by aggregating shorter events as longer but less frequent in the profiles. As a result their cumulative duration will be accurately represented in the profile without skewing their sample mean (duration/count). Credit to @dvyukov for suggesting to adjust the count in the saveblockevent function. Fixes #44192. Change-Id: I71a99d7f6ebdb2d484d44890a2517863cceb4004 Reviewed-on: https://go-review.googlesource.com/c/go/+/299991 Trust: Michael Pratt <mpratt@google.com> Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-04-06runtime/pprof: deflake TestMorestackCherry Zhang
In TestMorestack, on macOS, for some reason it got most of the samples in synchronization (e.g. pthread_cond_signal and pthread_cond_wait) and sometimes in other "syscalls" (usleep, nanotime1), and very few samples in stack copying, sometimes 0, which causes the test to fail. Maybe synchronization is slower on macOS? (It doesn't seem so to me.) Or it is the OS's accounting problem, where it is more likely to trigger a profiling signal at a syscall (or certain kinds of syscalls)? As the test is really about whether it can connect stack copying with the function that grows the stack, this CL makes it spend more time in copying stack than synchronization. Now it's getting ~100 samples for stack copying on a 5 second interval on my machine, and the test passes reliably. Fixes #44418. Change-Id: I3a462c8c39766f2d67d697598f8641bbe64f16ad Reviewed-on: https://go-review.googlesource.com/c/go/+/307730 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2021-04-04runtime/pprof: skip tests for AIXClément Chigot
Most of the time, the pprof tests are passing, except for the builder. The reason is still unknown but I'd rather release the builder to avoid missing other more important bugs. Updates #45170 Change-Id: I667543ee1ae309b7319c5b3676a0901b4d0ecf2e Reviewed-on: https://go-review.googlesource.com/c/go/+/306489 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-03-18runtime/pprof: move common code to writeProfileInternal functionElvina Yakubova
This patch provides changes according to TODO. Since writeMutex and writeBlock functions have a lot of code in common, it is better to move this code to one function. Change-Id: I81aaad067b0cb1647824909f3b5f6861add3a7ba Reviewed-on: https://go-review.googlesource.com/c/go/+/280152 Reviewed-by: Cherry Zhang <cherryyz@google.com> Trust: Hyang-Ah Hana Kim <hyangah@gmail.com>
2021-02-20all: go fmt std cmd (but revert vendor)Russ Cox
Make all our package sources use Go 1.17 gofmt format (adding //go:build lines). Part of //go:build change (#41184). See https://golang.org/design/draft-gobuild Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/294430 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-02-20runtime/pprof: disable TestMorestack on darwin/arm64Russ Cox
Something is weird about darwin and TestMorestack, but it is only manifesting on arm64 and race+amd64. Disable for now. Change-Id: I5862372fdd0b5ffae802fdefb65b2aa04e266fcc Reviewed-on: https://go-review.googlesource.com/c/go/+/294409 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-02-19runtime/pprof: disable TestMorestack on macOS under race detectorRuss Cox
This is failing but only under the race detector. It doesn't really seem fair to expect pprof to find specific profile events with the race detector slowing everything down anyway. Change-Id: I4b353d3d63944c87884d117e07d119b2c7bf4684 Reviewed-on: https://go-review.googlesource.com/c/go/+/294071 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime/pprof: expect tests to pass on macOSRuss Cox
macOS tests have been disabled since CL 12429045 (Aug 2013). At the time, macOS required a kernel patch to get a working profiler (https://research.swtch.com/macpprof), which we didn't want to require, of course. macOS has improved - it no longer requires the kernel patch - but we never updated the list of exceptions. As far as I can tell, the builders have no problem passing the pprof test now. (It is possible that the iOS builders have trouble, but that is now a different GOOS.) Remove the exception for macOS. The test should now pass. Fixes #6047. Change-Id: Iab49036cacc1025e56f515bd19d084390c2f5357 Reviewed-on: https://go-review.googlesource.com/c/go/+/292229 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-02-19runtime: fix windows/arm CONTEXT_CONTROLRuss Cox
The constant was wrong, and the “right” constant doesn't work either. But with the actually-right constant (and possibly earlier fixes in this stack as well), profiling now works. Change-Id: If8caff1da556826db40961fb9bcfe2b1f31ea9f9 Reviewed-on: https://go-review.googlesource.com/c/go/+/288808 Trust: Russ Cox <rsc@golang.org> Trust: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
2021-02-19runtime: use FuncInfo SPWRITE flag to identify untraceable profile samplesRuss Cox
The old code was very clever about predicting whether a traceback was safe. That cleverness has not aged well. In particular, the setsSP function is missing a bunch of functions that write to SP and will confuse traceback. And one such function - jmpdefer - was handled as a special case in gentraceback instead of simply listing it in setsSP. Throw away all the clever prediction about whether traceback will crash. Instead, make traceback NOT crash, by checking whether the function being walked writes to SP. This CL is part of a stack adding windows/arm64 support (#36439), intended to land in the Go 1.17 cycle. This CL is, however, not windows/arm64-specific. It is cleanup meant to make the port (and future ports) easier. Change-Id: I3d55fe257a22745e4919ac4dc9a9378c984ba0da Reviewed-on: https://go-review.googlesource.com/c/go/+/288801 Trust: Russ Cox <rsc@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-12-09all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTempRuss Cox
As part of #42026, these helpers from io/ioutil were moved to os. (ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.) Update the Go tree to use the preferred names. As usual, code compiled with the Go 1.4 bootstrap toolchain and code vendored from other sources is excluded. ReadDir changes are in a separate CL, because they are not a simple search and replace. For #42026. Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266365 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-28runtime/pprof: ignore test failures on windows/armJason A. Donenfeld
This is blocking forward progress of the de-bitrotting work, and I don't know off hand how to fix this. Seeing as its disabled on other platforms, I suspect pprof might not be a very reliable feature, so just allow for the tests to fail for now, until somebody more motivated comes along to fix it. Updates #42862. Change-Id: Ibc5cd1d82d97b9c2f887d7f3565f2fa70207c8b0 Reviewed-on: https://go-review.googlesource.com/c/go/+/273826 Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Trust: Jason A. Donenfeld <Jason@zx2c4.com>
2020-10-26runtime: rename mcache fields to match Go styleMichael Anthony Knyszek
This change renames a bunch of malloc statistics stored in the mcache that are all named with the "local_" prefix. It also renames largeAlloc to allocLarge to prevent a naming conflict, and next_sample because it would be the last mcache field with the old C naming style. Change-Id: I29695cb83b397a435ede7e9ad5c3c9be72767ea3 Reviewed-on: https://go-review.googlesource.com/c/go/+/246969 Trust: Michael Knyszek <mknyszek@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com>
2020-09-23all: add GOOS=iosCherry Zhang
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin" build tag, like GOOS=android matches "linux" and GOOS=illumos matches "solaris". Only ios/arm64 is supported (ios/amd64 is not). GOOS=ios and GOOS=darwin remain essentially the same at this point. They will diverge at later time, to differentiate macOS and iOS. Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"), except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"), it remains GOOS=="darwin". Updates #38485. Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c Reviewed-on: https://go-review.googlesource.com/c/go/+/254740 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-04-29runtime/pprof: fix units of MaxRSS on LinuxAustin Clements
Rusage.Maxrss is in bytes on Darwin but in KiB on Linux. Fix this discrepancy so it's always in bytes. Change-Id: Ic714abc3276566b8fe5e30565072092212610854 Reviewed-on: https://go-review.googlesource.com/c/go/+/230979 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-22runtime/pprof: plumb labels for goroutine profilesDavid Finkel
Goroutines are directly associated with labels. It's relatively easy to plumb those through without creating goroutine-locals in the wild. This is accomplished by splitting out most of the code from the public `runtime.GoroutineProfile` into a new unexported `runtime.goroutineProfileWithLabels`, which then has a thin wrapper linked into the `runtime/pprof` package as `runtime_goroutineProfileWithLabels`. (mirroring the way labels get associated with the `g` for a goroutine in the first place) Per-#6104, OS-thread creation profiles are a bit useless, as `M`s tend to be created be created by a background goroutine. As such, I decided not to add support for capturing the labels at `M`-creation-time, since the stack-traces seem to always come out `nil` for my simple test binaries. This change currently provides labels for debug=0 and debug=1, as debug=2 is currently entirely generated by the runtime package and I don't see a clean way of getting the `labelMap` type handled properly within the `runtime` package. Update the comment added in cl/131275 to mention goroutine support for labels. Updates #23458 Change-Id: Ia4b558893d7d10156b77121cd9b70c4ccd9e1889 Reviewed-on: https://go-review.googlesource.com/c/go/+/189318 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2020-04-21Revert "runtime/pprof: speed up CPU profiling shutdown"Josh Bleecher Snyder
This reverts commit 1f0738c1577a55a6b7229b821ddfe762b84771d0. Reason for revert: This May have caused issue 38567. Change-Id: I2afa6a9d42cb29cfad09e706fb465c57e3774abd Reviewed-on: https://go-review.googlesource.com/c/go/+/229301 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2020-04-21test/codegen, runtime/pprof, runtime: apply fmtalex-semenyuk
Change-Id: Ife4e065246729319c39e57a4fbd8e6f7b37724e1 GitHub-Last-Rev: e71803eaeb366c00f6c156de0b0b2c50927a0e82 GitHub-Pull-Request: golang/go#38527 Reviewed-on: https://go-review.googlesource.com/c/go/+/228901 Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-04-20runtime/pprof: speed up CPU profiling shutdownJosh Bleecher Snyder
The core CPU profiling loop contains a 100ms sleep. This is important to reduce overhead. However, it means that it takes 200ms to shutting down a program with CPU profiling enabled. When trying to collect many samples by running a short-lived program many times, this adds up. This change cuts the shutdown penalty in half by skipping the sleep whenever possible. Change-Id: Ic3177f8e1a2d331fe1a1ecd7c8c06f50beb42535 Reviewed-on: https://go-review.googlesource.com/c/go/+/228886 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2020-04-17net/http/pprof: support the "seconds" param for block, mutex profilesHana Kim
When the seconds param is given, the block and mutex profile endpoints report the difference between two measurements collected the given seconds apart. Historically, the block and mutex profiles have reported the cumulative counts since the process start, and it turned out they are more useful when interpreted along with the time duration. Note: cpu profile and trace endpoints already accept the "seconds" parameter. With this CL, the block and mutex profile endpoints will accept the "seconds" parameter. Providing the "seconds" parameter to other types of profiles is an error. This change moves runtime/pprof/internal/profile to internal/profile and adds part of merge logic from github.com/google/pprof/profile/merge.go to internal/profile, in order to allow both net/http/pprof and runtime/pprof to access it. Fixes #23401 Change-Id: Ie2486f1a63eb8ff210d7d3bc2de683e9335fd5cd Reviewed-on: https://go-review.googlesource.com/c/go/+/147598 Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-04-13runtime/pprof: clarify recursive inline heuristicMichael Pratt
Following CL 226818, the compiler will allow inlining a single cycle in an inline chain. Immediately-recursive functions are still disallowed, which is what this heuristic refers to. Add a regression test for this case. Note that in addition to this check, if the compiler were to inline multiple cycles via a loop (i.e., rather than appending duplicate code), much more work would be required here to handle a single address appearing in multiple different inline frames. Updates #29737 Change-Id: I88de15cfbeabb9c04381e1c12cc36778623132a5 Reviewed-on: https://go-review.googlesource.com/c/go/+/227346 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2020-04-13runtime/pprof: try to use real stack in TestTryAddMichael Pratt
TestTryAdd is particularly brittle because it tests some real cases by constructing fake sample stack frames. If those frames don't correctly represent what the runtime would generate then they may fail to catch regressions. Instead, call runtime.Callers at the bottom of real function calls to generate real frames as a base for truncation, etc in tests. Several of these tests still have to fake parts of the frames to test the right thing, but this is a bit less fragile. This change is equivalent to the original 0dfb0513ec6a0e97db166bd91a2dc0a1ceb154f7 (golang.org/cl/227484), except that the test skips if the test functions aren't inline (e.g., noopt builders). Change-Id: Ie9e32b5660cfe28a924f9cfcddcd887ea2effd66 Reviewed-on: https://go-review.googlesource.com/c/go/+/227922 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-08all: remove scattered remnants of darwin/armAustin Clements
This removes all conditions and conditional code (that I could find) that depended on darwin/arm. Fixes #35439 (since that only happened on darwin/arm) Fixes #37611. Change-Id: Ia4c32a5a4368ed75231075832b0b5bfb1ad11986 Reviewed-on: https://go-review.googlesource.com/c/go/+/227198 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-08Revert "runtime/pprof: try to use real stack in TestTryAdd"Bryan C. Mills
This reverts CL 227484. Reason for revert: failing on linux-amd64-noopt builder. Change-Id: Id677de8cfb18fbccb7d9b04b0f8a21d3c1d2b060 Reviewed-on: https://go-review.googlesource.com/c/go/+/227580 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-08runtime/pprof: try to use real stack in TestTryAddMichael Pratt
TestTryAdd is particularly brittle because it tests some real cases by constructing fake sample stack frames. If those frames don't correctly represent what the runtime would generate then they may fail to catch regressions. Instead, call runtime.Callers at the bottom of real function calls to generate real frames as a base for truncation, etc in tests. Several of these tests still have to fake parts of the frames to test the right thing, but this is a bit less fragile. Change-Id: I62522a9ded5544b06d1bf28550af5400f3af667b Reviewed-on: https://go-review.googlesource.com/c/go/+/227484 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2020-03-27runtime/pprof: increment fake overflow record PCMichael Pratt
gentraceback generates PCs which are usually following the CALL instruction. For those that aren't, it fixes up the PCs so that functions processing the output can unconditionally decrement the PC. runtime_expandInlineFrames does this unconditional decrement when looking up the function. However, the fake stack frame generated for overflow records fails to meet the contract, and decrementing the PC results in a PC in the previous function. If that function contains inlined call, runtime_expandInlineFrames will not short-circuit and will panic trying to look up a PC that doesn't exist. Note that the added test does not fail at HEAD. It will only fail (with a panic) if the function preceeding lostProfileEvent contains inlined function calls. At the moment (on linux/amd64), that is runtime/pprof.addMaxRSS, which does not. Fixes #38096 Change-Id: Iad0819f23c566011c920fd9a5b1254719228da0b Reviewed-on: https://go-review.googlesource.com/c/go/+/225661 Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-20runtime: handle empty stack in expandFinalInlineFrameKeith Randall
Fixes #37967 Change-Id: I6fc22bdd65f0263d5672731b73d09249201ab0aa Reviewed-on: https://go-review.googlesource.com/c/go/+/224458 Reviewed-by: Michael Pratt <mpratt@google.com>
2020-03-17runtime/pprof: export max rss when saving memory profiles.Jeremy Faller
NB: Adds syscall to deps on runtime/pprof. Change-Id: I5dd14c2b25eb9c3c446832f5818de45fafd48a27 Reviewed-on: https://go-review.googlesource.com/c/go/+/183844 Run-TryBot: Jeremy Faller <jeremy@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
2020-03-11runtime/pprof: document that debug=0 emits protoTamir Duberstein
Updates #16093. Change-Id: I629b3d44d6b2083f5e62701cc0c23fe2362502d4 Reviewed-on: https://go-review.googlesource.com/c/go/+/222676 Reviewed-by: Michael Matloob <matloob@golang.org>
2020-03-05runtime/pprof: expand final stack frame to avoid truncationMichael Pratt
When generating stacks, the runtime automatically expands inline functions to inline all inline frames in the stack. However, due to the stack size limit, the final frame may be truncated in the middle of several inline frames at the same location. As-is, we assume that the final frame is a normal function, and emit and cache a Location for it. If we later receive a complete stack frame, we will first use the cached Location for the inlined function and then generate a new Location for the "caller" frame, in violation of the pprof requirement to merge inlined functions into the same Location. As a result, we: 1. Nondeterministically may generate a profile with the different stacks combined or split, depending on which is encountered first. This is particularly problematic when performing a diff of profiles. 2. When split stacks are generated, we lose the inlining information. We avoid both of these problems by performing a second expansion of the last stack frame to recover additional inline frames that may have been lost. This expansion is a bit simpler than the one done by the runtime because we don't have to handle skipping, and we know that the last emitted frame is not an elided wrapper, since it by definition is already included in the stack. Fixes #37446 Change-Id: If3ca2af25b21d252cf457cc867dd932f107d4c61 Reviewed-on: https://go-review.googlesource.com/c/go/+/221577 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2020-03-01runtime/pprof/internal/profile: make error message readableIan Lance Taylor
The error message for an unrecognized type in decodeField was using string(i) for an int type i. It was recently changed (by me) to string(rune(i)), but that just avoided a vet warning without fixing the problem. This CL fixes the problem by using fmt.Errorf. We also change the message to "unknown wire type" to match the master copy of this code in github.com/google/pprof/profile/proto.go. Updates #32479 Change-Id: Ia91ea6d5edbd7cd946225d1ee96bb7623b52bb44 Reviewed-on: https://go-review.googlesource.com/c/go/+/221384 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-02-26all: avoid string(i) where i has type intIan Lance Taylor
Instead use string(r) where r has type rune. This is in preparation for a vet warning for string(i). Updates #32479 Change-Id: Ic205269bba1bd41723950219ecfb67ce17a7aa79 Reviewed-on: https://go-review.googlesource.com/c/go/+/220844 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Akhil Indurti <aindurti@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Toshihiro Shiino <shiino.toshihiro@gmail.com>
2019-11-22runtime/pprof: avoid crash due to truncated stack tracesHana Kim
The profile proto message builder maintains a location entry cache that maps a location (possibly involving multiple user frames that represent inlined function calls) to the location id. We have been using the first pc of the inlined call sequence as the key of the cached location entry assuming that, for a given pc, the sequence of frames representing the inlined call stack is deterministic and stable. Then, when analyzing the new stack trace, we expected the exact number of pcs to be present in the captured stack trace upon the cache hit. This assumption does not hold, however, in the presence of the stack trace truncation in the runtime during profiling, and also with the potential bugs in runtime. A better fix is to use all the pcs of the inlined call sequece as the key instead of the first pc. But that is a bigger code change. This CL avoids the crash assuming the trace was truncated. Fixes #35538 Change-Id: I8c6bae98bc8b178ee51523c7316f56b1cce6df16 Reviewed-on: https://go-review.googlesource.com/c/go/+/207609 Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2019-11-15all: fix a bunch of misspellingsVille Skyttä
Change-Id: I5b909df0fd048cd66c5a27fca1b06466d3bcaac7 GitHub-Last-Rev: 778c5d21311abee09a5fbda2e4005a5fd4cc3f9f GitHub-Pull-Request: golang/go#35624 Reviewed-on: https://go-review.googlesource.com/c/go/+/207421 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-14runtime/pprof: fix the inlined frame merge logicHana Kim
tryAdd shouldn't succeed (and accept the new frame) if the last existing frame on the deck is not an inlined frame. For example, when we see the followig stack [300656 300664 300655 300664] with each PC corresponds to [{PC:300656 Func:nil Function:runtime.nanotime File:/workdir/go/src/runtime/time_nofake.go Line:19 Entry:300416 {0x28dac8 0x386c80}}] [{PC:300664 Func:0x28dac8 Function:runtime.checkTimers File:/workdir/go/src/runtime/proc.go Line:2623 Entry:300416 {0x28dac8 0x386c80}}] [{PC:300655 Func:nil Function:runtime.nanotime File:/workdir/go/src/runtime/time_nofake.go Line:19 Entry:300416 {0x28dac8 0x386c80}}] [{PC:300664 Func:0x28dac8 Function:runtime.checkTimers File:/workdir/go/src/runtime/proc.go Line:2623 Entry:300416 {0x28dac8 0x386c80}}] PC:300656 and PC:300664 belong to a single location entry, but the bug in the current tryAdd logic placed the entire stack into one location entry. Also adds tests - this crash is a tricky case to test because I think it should happen with normal go code. The new TestTryAdd simulates it by using fake call sequences. The test crashed without the fix. Update #35538 Change-Id: I6d3483f757abf4c429ab91616e4def90832fc04a Reviewed-on: https://go-review.googlesource.com/c/go/+/206958 Reviewed-by: Keith Randall <khr@golang.org>
2019-11-11runtime/pprof: skip checks for inlined functions when inlining is disabledBryan C. Mills
Fixes #35463 Change-Id: I29af27b77cc651395c20570943847729ff12586c Reviewed-on: https://go-review.googlesource.com/c/go/+/206297 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-08runtime: fix line number for faulting instructionsKeith Randall
Unlike function calls, when processing instructions that directly fault we must not subtract 1 from the pc before looking up the file/line information. Since the file/line lookup unconditionally subtracts 1, add 1 to the faulting instruction PCs to compensate. Fixes #34123 Change-Id: Ie7361e3d2f84a0d4f48d97e5a9e74f6291ba7a8b Reviewed-on: https://go-review.googlesource.com/c/go/+/196962 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-11-08runtime/pprof: delete unused locForPCHana (Hyang-Ah) Kim
Change-Id: Ie4754fefba6057b1cf558d0096fe0e83355f8eff Reviewed-on: https://go-review.googlesource.com/c/go/+/205098 TryBot-Result: Gobot Gobot <gobot@golang.org> Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2019-11-08runtime/pprof: correct inlined function location encoding for non-CPU profilesHana (Hyang-Ah) Kim
Change-Id: Id270a3477bf1a581755c4311eb12f990aa2260b5 Reviewed-on: https://go-review.googlesource.com/c/go/+/205097 Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2019-11-08runtime/pprof: correctly encode inlined functions in CPU profileHana (Hyang-Ah) Kim
The pprof profile proto message expects inlined functions of a PC to be encoded in one Location entry using multiple Line entries. https://github.com/google/pprof/blob/5e96527/proto/profile.proto#L177-L184 runtime/pprof has encoded the symbolization information by creating a Location for each PC found in the stack trace and including info from all the frames expanded from the PC using runtime.CallersFrames. This assumes inlined functions are represented as a single PC in the stack trace. (https://go-review.googlesource.com/41256) In the recent years, behavior around inlining and the traceback changed significantly (e.g. https://golang.org/cl/152537, https://golang.org/issue/29582, and many changes). Now the PCs in the stack trace represent user frames even including inline marks. As a result, the profile proto started to allocate a Location entry for each user frame, lose the inline information (so pprof presented incorrect results when inlined functions are involved), and confuse the pprof tool with those PCs made up for inline marks. This CL attempts to detect inlined call frames from the stack traces of CPU profiles, and organize the Location information as intended. Currently, runtime does not provide a reliable and convenient way to detect inlined call frames and expand user frames from a given externally recognizable PCs. So we use heuristics to recover the groups - inlined call frames have nil Func field - inlined call frames will have the same Entry point - but must be careful with recursive functions that have the same Entry point by definition, and non-Go functions that may lack most of the fields of Frame. The followup CL will address the issue with other profile types. Change-Id: I0c9667ab016a3e898d648f31c3f82d84c15398db Reviewed-on: https://go-review.googlesource.com/c/go/+/204636 Reviewed-by: Keith Randall <khr@golang.org>
2019-11-05runtime/pprof: update example comment to check error on CloseIan Lance Taylor
Programs should always check the error return of Close for a file opened for writing. Update the example code in the comment to mention this. Change-Id: I2ff6866ff1fe23b47c54268ac8e182210cc876c5 Reviewed-on: https://go-review.googlesource.com/c/go/+/202137 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-09all: remove the nacl port (part 1)Brad Fitzpatrick
You were a useful port and you've served your purpose. Thanks for all the play. A subsequent CL will remove amd64p32 (including assembly files and toolchain bits) and remaining bits. The amd64p32 removal will be separated into its own CL in case we want to support the Linux x32 ABI in the future and want our old amd64p32 support as a starting point. Updates #30439 Change-Id: Ia3a0c7d49804adc87bf52a4dea7e3d3007f2b1cd Reviewed-on: https://go-review.googlesource.com/c/go/+/199499 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-08all: fix typosAinar Garipov
Use the following (suboptimal) script to obtain a list of possible typos: #!/usr/bin/env sh set -x git ls-files |\ grep -e '\.\(c\|cc\|go\)$' |\ xargs -n 1\ awk\ '/\/\// { gsub(/.*\/\//, ""); print; } /\/\*/, /\*\// { gsub(/.*\/\*/, ""); gsub(/\*\/.*/, ""); }' |\ hunspell -d en_US -l |\ grep '^[[:upper:]]\{0,1\}[[:lower:]]\{1,\}$' |\ grep -v -e '^.\{1,4\}$' -e '^.\{16,\}$' |\ sort -f |\ uniq -c |\ awk '$1 == 1 { print $2; }' Then, go through the results manually and fix the most obvious typos in the non-vendored code. Change-Id: I3cb5830a176850e1a0584b8a40b47bde7b260eae Reviewed-on: https://go-review.googlesource.com/c/go/+/193848 Reviewed-by: Robert Griesemer <gri@golang.org>
2019-08-28runtime: reduce allocations when building pprof LabelSetMartin Möhrmann
Pre-allocate the slice of labels with enough capacity to avoid growslice calls. Change-Id: I89db59ac722c03b0202e042d1f707bb041e0999f Reviewed-on: https://go-review.googlesource.com/c/go/+/181517 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2019-08-07runtime/pprof: Mention goroutine label heritabilityDavid Finkel
Document goroutine label inheritance. Goroutine labels are copied upon goroutine creation and there is a test enforcing this, but it was not mentioned in the docstrings for `Do` or `SetGoroutineLabels`. Add notes to both of those functions' docstrings so it's clear that one does not need to set labels as soon as a new goroutine is spawned if they want to propagate tags. Updates #32223 Updates #23458 Change-Id: Idfa33031af0104b884b03ca855ac82b98500c8b4 Reviewed-on: https://go-review.googlesource.com/c/go/+/189317 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-22all: shorten some testsRuss Cox
Shorten some of the longest tests that run during all.bash. Removes 7r 50u 21s from all.bash. After this change, all.bash is under 5 minutes again on my laptop. For #26473. Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79 Reviewed-on: https://go-review.googlesource.com/c/go/+/177559 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-04-30all: add new GOOS=illumos, split out of GOOS=solarisJoshua M. Clulow
Like GOOS=android which implies the "linux" build tag, GOOS=illumos implies the "solaris" build tag. This lets the existing ecosystem of packages still work on illumos, but still permits packages to start differentiating between solaris and illumos. Fixes #20603 Change-Id: I8f4eabf1a66060538dca15d7658c1fbc6c826622 Reviewed-on: https://go-review.googlesource.com/c/go/+/174457 Run-TryBot: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Benny Siegert <bsiegert@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-26runtime: add support for openbsd/arm64Joel Sing
Updates #31656 Change-Id: I0b7486f7381fd8bd16a76278c0e9ec9763671fcc Reviewed-on: https://go-review.googlesource.com/c/go/+/174119 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-03-26runtime: improve sigtramp on aix/ppc64 to handle SIGPROFClément Chigot
R14, R15 must be saved in sigtramp because they might be modified by Go code when a SIGPROF occurs. Fixes #28555 Change-Id: I573541f108d7d6aac8e60d33c649e5db943f3ef5 Reviewed-on: https://go-review.googlesource.com/c/go/+/169117 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-19runtime: disable pprof test with cgo on aix/ppc64Clément Chigot
This commit disables new cgo pprof tests and adds an handler in sigtramp to refuse SIGPROF signal. Updates #28555 Change-Id: I152a871f8636e93328d411329104c6f023bd1691 Reviewed-on: https://go-review.googlesource.com/c/go/+/164013 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>