aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-12-03[release-branch.go1.14] go1.14.13go1.14.13Carlos Amedee
Change-Id: Ifdfd9cd7edb8c3afd785cf75e818e3d301cd8dae Reviewed-on: https://go-review.googlesource.com/c/go/+/275133 Run-TryBot: Carlos Amedee <carlos@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2020-12-03[release-branch.go1.14] cmd/compile: sign extend constant folding properlyKeith Randall
MOVLconst must have a properly sign-extended auxint constant. The bit operations in these rules don't enforce that invariant. Fixes #42755 Change-Id: I729afcad18752d9b7739e49709020e3be7b3653e Reviewed-on: https://go-review.googlesource.com/c/go/+/272030 Trust: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: David Chase <drchase@google.com>
2020-11-20[release-branch.go1.14] runtime: block signals in needm before allocating MIan Lance Taylor
Otherwise, if a signal occurs just after we allocated the M, we can deadlock if the signal handler needs to allocate an M itself. For #42207 Fixes #42635 Change-Id: I76f44547f419e8b1c14cbf49bf602c6e645d8c14 Reviewed-on: https://go-review.googlesource.com/c/go/+/265759 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> (cherry picked from commit 368c40116434532dc0b53b72fa04788ca6742898) Reviewed-on: https://go-review.googlesource.com/c/go/+/271848
2020-11-16[release-branch.go1.14] cmd/go: permit CGO_LDFLAGS to appear in //go:ldflagIan Lance Taylor
For #42565 Fixes #42566 Change-Id: If7cf39905d124dbd54dfac6a53ee38270498efed Reviewed-on: https://go-review.googlesource.com/c/go/+/269818 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> (cherry picked from commit 782cf560db4c919790fdb476d1bbe18e5ddf5ffd) Reviewed-on: https://go-review.googlesource.com/c/go/+/270080
2020-11-12[release-branch.go1.14] all: merge release-branch.go1.14-security into ↵Katie Hockman
release-branch.go1.14 Change-Id: I87a2c27ce88913c2867ef355d589debfbb522167
2020-11-12[release-branch.go1.14-security] go1.14.12go1.14.12Carlos Amedee
Change-Id: I8ce7093f7e119216d3a5d8941968788b70b6afaf Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/901408 Reviewed-by: Katie Hockman <katiehockman@google.com>
2020-11-12[release-branch.go1.14-security] math/big: fix shift for recursive divisionKatie Hockman
The previous s value could cause a crash for certain inputs. Will check in tests and documentation improvements later. Thanks to the Go Ethereum team and the OSS-Fuzz project for reporting this. Thanks to Rémy Oudompheng and Robert Griesemer for their help developing and validating the fix. Fixes CVE-2020-28362 Change-Id: Ibbf455c4436bcdb07c84a34fa6551fb3422356d3 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/899974 Reviewed-by: Roland Shoemaker <bracewell@google.com> Reviewed-by: Filippo Valsorda <valsorda@google.com> (cherry picked from commit 28015462c2a83239543dc2bef651e9a5f234b633) Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/901064
2020-11-12[release-branch.go1.14-security] cmd/go: in cgoflags, permit -DX1, prohibit ↵Ian Lance Taylor
-Wp,-D,opt Restrict -D and -U to ASCII C identifiers, but do permit trailing digits. When using -Wp, prohibit commas in -D values. Thanks to Imre Rad (https://www.linkedin.com/in/imre-rad-2358749b) for reporting this. Fixes CVE-2020-28367 Change-Id: Ibfc4dfdd6e6c258e131448e7682610c44eee9492 Reviewed-on: https://go-review.googlesource.com/c/go/+/267277 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/899923 Reviewed-by: Filippo Valsorda <valsorda@google.com>
2020-11-12[release-branch.go1.14-security] cmd/go, cmd/cgo: don't let bogus symbol set ↵Ian Lance Taylor
cgo_ldflag A hand-edited object file can have a symbol name that uses newline and other normally invalid characters. The cgo tool will generate Go files containing symbol names, unquoted. That can permit those symbol names to inject Go code into a cgo-generated file. If that Go code uses the //go:cgo_ldflag pragma, it can cause the C linker to run arbitrary code when building a package. If you build an imported package we permit arbitrary code at run time, but we don't want to permit it at package build time. This CL prevents this in two ways. In cgo, reject invalid symbols that contain non-printable or space characters, or that contain anything that looks like a Go comment. In the go tool, double check all //go:cgo_ldflag directives in generated code, to make sure they follow the existing LDFLAG restrictions. Thanks to Chris Brown and Tempus Ex for reporting this. Fixes CVE-2020-28366 Change-Id: Ia1ad8f3791ea79612690fa7d26ac451d0f6df7c1 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/895832 Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Cherry Zhang <cherryyz@google.com> (cherry picked from commit 6bc814dd2bbfeaafa41d314dd4cc591b575dfbf6) Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/901055 Reviewed-by: Filippo Valsorda <valsorda@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com>
2020-11-05[release-branch.go1.14] go1.14.11go1.14.11Alexander Rakoczy
Change-Id: I1b4231179d0825113f2cbb6e84e92b3453e2ee45 Reviewed-on: https://go-review.googlesource.com/c/go/+/267878 Run-TryBot: Alexander Rakoczy <alex@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Alexander Rakoczy <alex@golang.org>
2020-10-30[release-branch.go1.14] net/http: update bundled x/net/http2Dmitri Shuralyov
Bring in the change in CL 266157 with: go get -d golang.org/x/net@release-branch.go1.14 go mod tidy go mod vendor go generate -run=bundle std Updates #39337. Fixes #42112. Change-Id: Iefd0012369c7f0c58201256e29d21210cb9f2f7a Reviewed-on: https://go-review.googlesource.com/c/go/+/266374 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-30[release-branch.go1.14] cmd/go/internal/modfetch: drop ↵Tobias Klauser
gopkg.in/russross/blackfriday.v2 from TestCodeRepoVersions Follow-up for CL 265819. Given the -pre tag added recently, a new stable version is likely tagged soon. This would break TestCodeRepoVersions on the longtest builders again. Since the other test cases in codeRepoVersionsTests already provide enough coverage, drop gopkg.in/russross/blackfriday.v2 to avoid breaking TestCodeRepoVersions once the release happens. Updates #28856 Change-Id: If86a637b5e47f59faf9048fc1cbbae6e8f1dcc53 Reviewed-on: https://go-review.googlesource.com/c/go/+/265917 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> (cherry picked from commit 421d4e72de802ed65cb38317660654771cfb13e9) Reviewed-on: https://go-review.googlesource.com/c/go/+/266178 Trust: Dmitri Shuralyov <dmitshur@golang.org> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> (cherry picked from commit 8687f6d924ee3a311e08db855c6dc1024c1f9349) Reviewed-on: https://go-review.googlesource.com/c/go/+/266302 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-29[release-branch.go1.14] time: fix LoadLocationFromTZData with slim tzdataChristopher Hlubek
The extend information of a time zone file with last transition < now could result in a wrong cached zone because it used the zone of the last transition. This could lead to wrong zones in systems with slim zoneinfo. Fixes #42155 Change-Id: I7c57c35b5cfa58482ac7925b5d86618c52f5444d Reviewed-on: https://go-review.googlesource.com/c/go/+/264939 Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> (cherry picked from commit 70e022e4a83dc996ac4f108e811fbc399ad5565b) Reviewed-on: https://go-review.googlesource.com/c/go/+/266303 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-10-29[release-branch.go1.14] time: support slim tzdata formatIan Lance Taylor
Backport of part of https://golang.org/cl/261877 to support the slim tzdata format. As of tzdata 2020b, the default is to use the slim format. We need to support that format so that Go installations continue to work when tzdata is updated. Relevant part of the CL description: The reason for the failed tests was that when caching location data, the extended time format past the end of zone transitions was not considered. The respective change was introduced in (*Location).lookup by CL 215539. For #42155 Change-Id: I37f52a0917b2c6e3957e6b4612c8ef104c736e65 Reviewed-on: https://go-review.googlesource.com/c/go/+/264301 Trust: Ian Lance Taylor <iant@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> (cherry picked from commit 414668cfbc41fd8cadf74e981849d1e05cc23b2e) Reviewed-on: https://go-review.googlesource.com/c/go/+/266298 Run-TryBot: Ian Lance Taylor <iant@golang.org>
2020-10-29[release-branch.go1.14] time: use extended time format past end of zone ↵Ian Lance Taylor
transitions This gives us better expected information for daylight savings time transitions in year 2038 and beyond. For #36654 For #42155 Change-Id: I5a39aed3c40b184e1d7bb7d6ce3aff5307c4c146 Reviewed-on: https://go-review.googlesource.com/c/go/+/215539 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> (cherry picked from commit b71eafbcece175db33acfb205e9090ca99a8f984) Reviewed-on: https://go-review.googlesource.com/c/go/+/264302 Trust: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-10-23[release-branch.go1.14] net/http: deep copy Request.TransferEncodingdqu123
The existing implementation in Request.Clone() assigns the wrong pointer to r2.TransferEncoding. Updates #41907. Fixes #41913. Change-Id: I7f220a41b1b46a55d1a1005e47c6dd69478cb025 Reviewed-on: https://go-review.googlesource.com/c/go/+/261377 Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Carlos Amedee <carlos@golang.org> Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-22[release-branch.go1.14] src, net/http: update vendor, regenerate h2_bundle.goEmmanuel T Odeke
Features CL: net/http2: send WINDOW_UPDATE on a body's write failure (fixes #41386) https://golang.org/cl/258497 Created by: go get -d golang.org/x/net@release-branch.go1.14 go mod tidy go mod vendor go generate -run=bundle std Updates #40423 Fixes #41386 Change-Id: I3e75527d381dd4c4262db5f2ff755029d448c48b Reviewed-on: https://go-review.googlesource.com/c/go/+/258538 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Trust: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-10-20[release-branch.go1.14] runtime: wait for preemption signals before syscall.ExecIan Lance Taylor
For #41702 For #41703 For #42023 Change-Id: If07f40b1d73b8f276ee28ffb8b7214175e56c24d Reviewed-on: https://go-review.googlesource.com/c/go/+/262817 Trust: Ian Lance Taylor <iant@golang.org> Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> (cherry picked from commit 05739d6f17c57f09264272621b88725a463234d0) Reviewed-on: https://go-review.googlesource.com/c/go/+/264023
2020-10-20[release-branch.go1.14] syscall: use MustHaveExec in TestExecIan Lance Taylor
For #41702 For #41703 Change-Id: Ib2b15e52aa1fef2f5e644b316c726150252fa9f8 Reviewed-on: https://go-review.googlesource.com/c/go/+/262738 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> (cherry picked from commit 11cfb48df192c14d185c1cfcaad1ba3e7b84c807) Reviewed-on: https://go-review.googlesource.com/c/go/+/264021
2020-10-20[release-branch.go1.14] runtime: stop preemption during syscall.Exec on DarwinIan Lance Taylor
On current macOS versions a program that receives a signal during an execve can fail with a SIGILL signal. This appears to be a macOS kernel bug. It has been reported to Apple. This CL partially works around the problem by using execLock to not send preemption signals during execve. Of course some other stray signal could occur, but at least we can avoid exacerbating the problem. We can't simply disable signals, as that would mean that the exec'ed process would start with all signals blocked, which it likely does not expect. For #41702 Fixes #41703 Change-Id: I91b0add967b315671ddcf73269c4d30136e579b4 Reviewed-on: https://go-review.googlesource.com/c/go/+/262438 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> (cherry picked from commit 64fb6ae95f1c322486cbfb758552bb8439a8e6e8) Reviewed-on: https://go-review.googlesource.com/c/go/+/262737
2020-10-15[release-branch.go1.14] runtime: set g to gsignal before adjustSignalStackCherry Zhang
When a signal is received, the runtime probes whether an alternate signal stack is set, if so, adjust gsignal's stack to point to the alternate signal stack. This is done in adjustSignalStack, which calls sigaltstack "syscall", which is a libc call on darwin through asmcgocall. asmcgocall decides whether to do stack switch based on whether we're running on g0 stack, gsignal stack, or regular g stack. If g is not set to gsignal, asmcgocall may make wrong decision. Set g first. adjustSignalStack is recursively nosplit, so it is okay that temporarily gsignal.stack doesn't match the stack we're running on. Updates #39079. Fixes #41991. Change-Id: I59b2c5dc08c3c951f1098fff038bf2e06d7ca055 Reviewed-on: https://go-review.googlesource.com/c/go/+/238020 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> (cherry picked from commit d286e61b6787fe2b55bf0ec8a814962ebda8d202) Reviewed-on: https://go-review.googlesource.com/c/go/+/262557 Trust: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2020-10-14[release-branch.go1.14] go1.14.10go1.14.10Alexander Rakoczy
Change-Id: Ia983336cdedc9fa835bfc792dd1e819eef31596f Reviewed-on: https://go-review.googlesource.com/c/go/+/262338 Run-TryBot: Alexander Rakoczy <alex@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Alexander Rakoczy <alex@golang.org>
2020-10-14[release-branch.go1.14] runtime: correct signature of call16Austin Clements
The signature of call16 is currently missing the "typ" parameter. This CL fixes this. This wasn't caught by vet because call16 is defined by macro expansion (see #17544), and we didn't notice the mismatch with the other call* functions because call16 is defined only on 32-bit architectures and lives alone in stubs32.go. Unfortunately, this means its GC signature is also wrong: the "arg" parameter is treated as a scalar rather than a pointer, so GC won't trace it and stack copying won't adjust it. This turns out to matter in exactly one case right now: on 32-bit architectures (which are the only architectures where call16 is defined), a stack-allocated defer of a function with a 16-byte or smaller argument frame including a non-empty result area can corrupt memory if the deferred function grows the stack and is invoked during a panic. Whew. All other current uses of reflectcall pass a heap-allocated "arg" frame (which happens to be reachable from other stack roots, so tracing isn't a problem). Curiously, in 2016, the signatures of all call* functions were wrong in exactly this way. CL 31654 fixed all of them in stubs.go, but missed the one in stubs32.go. Updates #41795. Fixes #41796. Change-Id: I31e3c0df201f79ee5707eeb8dc4ff0d13fc10ada Reviewed-on: https://go-review.googlesource.com/c/go/+/259338 Trust: Austin Clements <austin@google.com> Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/259597
2020-10-08[release-branch.go1.14] database/sql: de-flake TestTxCannotCommitAfterRollbackDaniel Theophanes
Do not cancel rows during test. Only cancel the Tx. Correct the referenced issue number on the test. Updates #38597. Fixes #41815. Change-Id: I0e8ba1bf2a8ba638d121c9c6938501fec1d5e961 Reviewed-on: https://go-review.googlesource.com/c/go/+/229478 Run-TryBot: Daniel Theophanes <kardianos@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> (cherry picked from commit ed7888aea6021e25b0ea58bcad3f26da2b139432) Reviewed-on: https://go-review.googlesource.com/c/go/+/259858 Trust: Dmitri Shuralyov <dmitshur@golang.org> Trust: Emmanuel Odeke <emm.odeke@gmail.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-10-07[release-branch.go1.14] runtime: disable stack shrinking in activeStackChans ↵Michael Anthony Knyszek
race window Currently activeStackChans is set before a goroutine blocks on a channel operation in an unlockf passed to gopark. The trouble is that the unlockf is called *after* the G's status is changed, and the G's status is what is used by a concurrent mark worker (calling suspendG) to determine that a G has successfully been suspended. In this window between the status change and unlockf, the mark worker could try to shrink the G's stack, and in particular observe that activeStackChans is false. This observation will cause the mark worker to *not* synchronize with concurrent channel operations when it should, and so updating pointers in the sudog for the blocked goroutine (which may point to the goroutine's stack) races with channel operations which may also manipulate the pointer (read it, dereference it, update it, etc.). Fix the problem by adding a new atomically-updated flag to the g struct called parkingOnChan, which is non-zero in the race window above. Then, in isShrinkStackSafe, check if parkingOnChan is zero. The race is resolved like so: * Blocking G sets parkingOnChan, then changes status in gopark. * Mark worker successfully suspends blocking G. * If the mark worker observes parkingOnChan is non-zero when checking isShrinkStackSafe, then it's not safe to shrink (we're in the race window). * If the mark worker observes parkingOnChan as zero, then because the mark worker observed the G status change, it can be sure that gopark's unlockf completed, and gp.activeStackChans will be correct. The risk of this change is low, since although it reduces the number of places that stack shrinking is allowed, the window here is incredibly small. Essentially, every place that it might crash now is replaced with no shrink. This change adds a test, but the race window is so small that it's hard to trigger without a well-placed sleep in park_m. Also, this change fixes stackGrowRecursive in proc_test.go to actually allocate a 128-byte stack frame. It turns out the compiler was destructuring the "pad" field and only allocating one uint64 on the stack. For #40641. Fixes #40642. Change-Id: I7dfbe7d460f6972b8956116b137bc13bc24464e8 Reviewed-on: https://go-review.googlesource.com/c/go/+/247050 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> Trust: Michael Knyszek <mknyszek@google.com> (cherry picked from commit eb3c6a93c3236bbde5dee6cc5bd4ca9f8ab1647a) Reviewed-on: https://go-review.googlesource.com/c/go/+/256301 Reviewed-by: Austin Clements <austin@google.com>
2020-10-05[release-branch.go1.14] cmd/compile: fix live variable computation for ↵Keith Randall
deferreturn Taking the live variable set from the last return point is problematic. See #40629 for details, but there may not be a return point, or it may be before the final defer. Additionally, keeping track of the last call as a *Value doesn't quite work. If it is dead-code eliminated, the storage for the Value is reused for some other random instruction. Its live variable information, if it is available at all, is wrong. Instead, just mark all the open-defer argument slots as live throughout the function. (They are already zero-initialized.) Fixes #40647 Change-Id: Ie456c7db3082d0de57eaa5234a0f32525a1cce13 Reviewed-on: https://go-review.googlesource.com/c/go/+/247522 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com> (cherry picked from commit 32a84c99e136ed5af0686dbedd31fd7dff40fb38) Reviewed-on: https://go-review.googlesource.com/c/go/+/248622 TryBot-Result: Go Bot <gobot@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-05[release-branch.go1.14] testing: flush test summaries to stdout atomically ↵Bryan C. Mills
when streaming output While debugging #40771, I realized that the chatty printer should only ever print to a single io.Writer (normally os.Stdout). The other Writer implementations in the chain write to local buffers, but if we wrote a test's output to a local buffer, then we did *not* write it to stdout and we should not store it as the most recently logged test. Because the chatty printer should only ever print to one place, it shouldn't receive an io.Writer as an argument — rather, it shouldn't be used at all for destinations other than the main output stream. On the other hand, when we flush the output buffer to stdout in the top-level flushToParent call, it is important that we not allow some other test's output to intrude between the test summary header and the remainder of the test's output. cmd/test2json doesn't know how to parse such an intrusion, and it's confusing to humans too. No test because I couldn't reproduce the user-reported error without modifying the testing package. (This behavior seems to be very sensitive to output size and/or goroutine scheduling.) Fixes #40880 Updates #40771 Updates #38458 Change-Id: Ic19bf1d535672b096ba1c8583a3b74aab6d6d766 Reviewed-on: https://go-review.googlesource.com/c/go/+/249026 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> (cherry picked from commit 51c0bdc6d15dcd7f753c25896039ab41ac787ebb) Reviewed-on: https://go-review.googlesource.com/c/go/+/252638 TryBot-Result: Go Bot <gobot@golang.org> Trust: Bryan C. Mills <bcmills@google.com>
2020-10-01[release-branch.go1.14] cmd/compile: prevent 387+float32+pie from clobbering ↵Keith Randall
registers The 387 port needs to load a floating-point control word from a global location to implement float32 arithmetic. When compiling with -pie, loading that control word clobbers an integer register. If that register had something important in it, boom. Fix by using LEAL to materialize the address of the global location first. LEAL with -pie works because the destination register is used as the scratch register. 387 support is about to go away (#40255), so this will need to be backported to have any effect. No test. I have one, but it requires building with -pie, which requires cgo. Our testing infrastructure doesn't make that easy. Not worth it for a port which is about to vanish. Fixes #41619 Change-Id: I140f9fc8fdce4e74a52c2c046e2bd30ae476d295 Reviewed-on: https://go-review.googlesource.com/c/go/+/257277 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Keith Randall <khr@golang.org> (cherry picked from commit ea106cc07ac73110a8a25fcc5aef07b283159db0) Reviewed-on: https://go-review.googlesource.com/c/go/+/257208
2020-09-10[release-branch.go1.14] runtime: fix ReadMemStatsSlow's and ↵Michael Anthony Knyszek
CheckScavengedBits' chunk iteration Both ReadMemStatsSlow and CheckScavengedBits iterate over the page allocator's chunks but don't actually check if they exist. During the development process the chunks index became sparse, so now this was a possibility. If the runtime tests' heap is sparse we might end up segfaulting in either one of these functions, though this will generally be very rare. The pattern here to return nil for a nonexistent chunk is also useful elsewhere, so this change introduces tryChunkOf which won't throw, but might return nil. It also updates the documentation of chunkOf. For #41296. Fixes #41322. Change-Id: Id5ae0ca3234480de1724fdf2e3677eeedcf76fa0 Reviewed-on: https://go-review.googlesource.com/c/go/+/253777 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> (cherry picked from commit 34835df04891a1d54394888b763af88f9476101d) Reviewed-on: https://go-review.googlesource.com/c/go/+/253922 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-09-10[release-branch.go1.14] cmd/internal/obj: fix inline marker issue on s390xMichael Munday
The optimization that replaces inline markers with pre-existing instructions assumes that 'Prog' values produced by the compiler are still reachable after the assembler has run. This was not true on s390x where the assembler was removing NOP instructions from the linked list of 'Prog' values. This led to broken inlining data which in turn caused an infinite loop in the runtime traceback code. Fix this by stopping the s390x assembler backend removing NOP values. It does not make any difference to the output of the assembler because NOP instructions are 0 bytes long anyway. Note: compiler check omitted from backport to reduce risk of change. Fixes #40694. Change-Id: I9f9bdbe895c3478549b5e7e623f9521f841e926a Reviewed-on: https://go-review.googlesource.com/c/go/+/248477 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2020-09-09[release-branch.go1.14] go1.14.9go1.14.9Dmitri Shuralyov
Change-Id: I556ecd19f81692ddbd3faf1d918e36466833f12e Reviewed-on: https://go-review.googlesource.com/c/go/+/253737 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-03[release-branch.go1.14] cmd/compile, runtime: mark R12 clobbered for write ↵Cherry Zhang
barrier call on PPC64 When external linking, for large binaries, the external linker may insert a trampoline for the write barrier call, which looks 0000000005a98cc8 <__long_branch_runtime.gcWriteBarrier>: 5a98cc8: 86 01 82 3d addis r12,r2,390 5a98ccc: d8 bd 8c e9 ld r12,-16936(r12) 5a98cd0: a6 03 89 7d mtctr r12 5a98cd4: 20 04 80 4e bctr It clobbers R12 (and CTR, which is never live across a call). As at compile time we don't know whether the binary is big and what link mode will be used, I think we need to mark R12 as clobbered for write barrier call. For extra safety (future-proof) we mark caller-saved register that cannot be used for function arguments, which includes R11, as potentially clobbered as well. Updates #40851. Fixes #40938. Change-Id: Iedd901c5072f1127cc59b0a48cfeb4aaec81b519 Reviewed-on: https://go-review.googlesource.com/c/go/+/248917 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> (cherry picked from commit b58d29741650c7bf10b17f455666e2727e1cdd2e) Reviewed-on: https://go-review.googlesource.com/c/go/+/249697
2020-09-03[release-branch.go1.14] cmd/internal/obj: stop removing NOPs from ↵Keith Randall
instruction stream This has already been done for s390x, ppc64. This CL is for all the other architectures. Fixes #40797 Change-Id: Idd1816e057df63022d47e99fa06617811d8c8489 Reviewed-on: https://go-review.googlesource.com/c/go/+/248684 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> (cherry picked from commit 46ca7b5ee2a8582736f1ddac27d8660e1104c345) Reviewed-on: https://go-review.googlesource.com/c/go/+/249443 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-03[release-branch.go1.14] cmd/internal/obj/ppc64: don't remove NOP in assemblerLynn Boger
Previously, the assembler removed NOPs from the Prog list in obj9.go. NOPs shouldn't be removed if they were added as an inline mark, as described in the issue below. Fixes #40766 Once the NOPs were left in the Prog list, some instructions were flagged as invalid because they had an operand which was not represented in optab. In order to preserve the previous assembler behavior, entries were added to optab for those operand cases. They were not flagged as errors before because the NOP instructions were removed before the code to check the valid opcode/operand combinations. Change-Id: Iae5145f94459027cf458e914d7c5d6089807ccf8 Reviewed-on: https://go-review.googlesource.com/c/go/+/247842 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Paul Murphy <murp@ibm.com> Reviewed-by: Michael Munday <mike.munday@ibm.com> Reviewed-by: Keith Randall <khr@golang.org> (cherry picked from commit 7d7bd5abc7f7ac901830b79496f63ce86895e262) Reviewed-on: https://go-review.googlesource.com/c/go/+/248382 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-02[release-branch.go1.14] net/http/fgci: skip flaky testFilippo Valsorda
A test introduced in the security release is flaky due to a pre-existing issue that does not qualify for backport itself. Updates #41167 Fixes #41192 Change-Id: Ie6014e0796c1baee7b077881b5a799f9947fc9c2 Reviewed-on: https://go-review.googlesource.com/c/go/+/252718 Run-TryBot: Filippo Valsorda <filippo@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-02[release-branch.go1.14] doc/go1.14: document json.Umarshal map key support ↵ShihCheng Tu
of TextUnmarshaler Document that json.Unmarshal supports map keys whose underlying types implement encoding.TextUnmarshaler. Updates #38801. Fixes #38904. Change-Id: Icb9414e9067517531ba0da910bd4a2bb3daace65 Reviewed-on: https://go-review.googlesource.com/c/go/+/237857 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> (cherry picked from commit 47b450997778163dfed6f58cae379d928fc37687) Reviewed-on: https://go-review.googlesource.com/c/go/+/252617 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2020-09-01[release-branch.go1.14] testing: treat PAUSE lines as changing the active ↵Bryan C. Mills
test name We could instead fix cmd/test2json to treat PAUSE lines as *not* changing the active test name, but that seems like it would be more confusing to humans, and also wouldn't fix tools that parse output using existing builds of cmd/test2json. Fixes #40848 Updates #40657 Change-Id: I937611778f5b1e7dd1d6e9f44424d7e725a589ed Reviewed-on: https://go-review.googlesource.com/c/go/+/248727 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jean de Klerk <deklerk@google.com> (cherry picked from commit cdc77d34d7770ed02d84b9193380f9646017dce6) Reviewed-on: https://go-review.googlesource.com/c/go/+/249098 TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-01[release-branch.go1.14] all: merge release-branch.go1.14-security into ↵Filippo Valsorda
release-branch.go1.14 Change-Id: I52c14764e354cb9b11be6019cf8fb44930786ab8
2020-09-01[release-branch.go1.14-security] go1.14.8go1.14.8Dmitri Shuralyov
Change-Id: Ie582b6c53c6b120c56fbdd22b0c6946dd87f093b Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/835358 Reviewed-by: Filippo Valsorda <valsorda@google.com>
2020-09-01[release-branch.go1.14-security] net/http/cgi,net/http/fcgi: add ↵Roberto Clapis
Content-Type detection This CL ensures that responses served via CGI and FastCGI have a Content-Type header based on the content of the response if not explicitly set by handlers. If the implementers of the handler did not explicitly specify a Content-Type both CGI implementations would default to "text/html", potentially causing cross-site scripting. Thanks to RedTeam Pentesting GmbH for reporting this. Fixes CVE-2020-24553 Change-Id: I82cfc396309b5ab2e8d6e9a87eda8ea7e3799473 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/823217 Reviewed-by: Russ Cox <rsc@google.com> (cherry picked from commit 23d675d07fdc56aafd67c0a0b63d5b7e14708ff0) Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/835312 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2020-08-27[release-branch.go1.14] net/http: fix cancelation of requests with a ↵Damien Neil
readTrackingBody wrapper Use the original *Request in the reqCanceler map, not the transient wrapper created to handle body rewinding. Change the key of reqCanceler to a struct{*Request}, to make it more difficult to accidentally use the wrong request as the key. Updates #40453. Fixes #41016. Change-Id: I4e61ee9ff2c794fb4c920a3a66c9a0458693d757 Reviewed-on: https://go-review.googlesource.com/c/go/+/245357 Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/250299 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2020-08-27[release-branch.go1.14] net/http: make Transport.RoundTrip preserve RequestsDamien Neil
Ensure that the exact Request passed to Transport.RoundTrip is returned in the Response. Do not replace the Request with a copy when resetting the request body. Updates #39533. Fixes #40973. Change-Id: Ie6fb080c24b0f6625b0761b7aa542af3d2411817 Reviewed-on: https://go-review.googlesource.com/c/go/+/237560 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/249880 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Damien Neil <dneil@google.com>
2020-08-27[release-branch.go1.14] net/http: handle body rewind in HTTP/2 connection ↵Russ Cox
loss better In certain cases the HTTP/2 stack needs to resend a request. It obtains a fresh body to send by calling req.GetBody. This call was missing from the path where the HTTP/2 round tripper returns ErrSkipAltProtocol, meaning fall back to HTTP/1.1. The result was that the HTTP/1.1 fallback request was sent with no body at all. This CL changes that code path to rewind the body before falling back to HTTP/1.1. But rewinding the body is easier said than done. Some requests have no GetBody function, meaning the body can't be rewound. If we need to rewind and can't, that's an error. But if we didn't read anything, we don't need to rewind. So we have to track whether we read anything, with a new ReadCloser wrapper. That in turn requires adding to the couple places that unwrap Body values to look at the underlying implementation. This CL adds the new rewinding code in the main retry loop as well. The new rewindBody function also takes care of closing the old body before abandoning it. That was missing in the old rewind code. Thanks to Aleksandr Razumov for CL 210123 and to Jun Chen for CL 234358, both of which informed this CL. Updates #32441. Fixes #39279. Change-Id: Id183758526c087c6b179ab73cf3b61ed23a2a46a Reviewed-on: https://go-review.googlesource.com/c/go/+/234894 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> (cherry picked from commit e3491c46034cecbaf0f33928b09e1e3c0c6a0d20) Reviewed-on: https://go-review.googlesource.com/c/go/+/242117 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-08-22[release-branch.go1.14] runtime: detect services in signal handlerJason A. Donenfeld
The service handler needs to handle CTRL+C-like events -- including those sent by the service manager itself -- using the default Windows implementation if no signal handler from Go is already listening to those events. Ordinarily, the signal handler would call exit(2), but we actually need to allow this to be passed onward to the service handler. So, we detect if we're in a service and skip calling exit(2) in that case, just like we do for shared libraries. Updates #40167. Updates #40074. Fixes #40411. Change-Id: Ia77871737a80e1e94f85b02d26af1fd2f646af96 Reviewed-on: https://go-review.googlesource.com/c/go/+/244958 Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2020-08-22[release-branch.go1.14] runtime: ensure startm new M is consistently visible ↵Michael Pratt
to checkdead If no M is available, startm first grabs an idle P, then drops sched.lock and calls newm to start a new M to run than P. Unfortunately, that leaves a window in which a G (e.g., returning from a syscall) may find no idle P, add to the global runq, and then in stopm discover that there are no running M's, a condition that should be impossible with runnable G's. To avoid this condition, we pre-allocate the new M ID in startm before dropping sched.lock. This ensures that checkdead will see the M as running, and since that new M must eventually run the scheduler, it will handle any pending work as necessary. Outside of startm, most other calls to newm/allocm don't have a P at all. The only exception is startTheWorldWithSema, which always has an M if there is 1 P (i.e., the currently running M), and if there is >1 P the findrunnable spinning dance ensures the problem never occurs. This has been tested with strategically placed sleeps in the runtime to help induce the correct race ordering, but the timing on this is too narrow for a test that can be checked in. For #40368 Fixes #40398 Change-Id: If5e0293a430cc85154b7ed55bc6dadf9b340abe2 Reviewed-on: https://go-review.googlesource.com/c/go/+/245018 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com> (cherry picked from commit 85afa2eb190d5d1a06584803bde4b4ee9b0e79b0) Reviewed-on: https://go-review.googlesource.com/c/go/+/245297
2020-08-22[release-branch.go.1.14] runtime: validate candidate searchAddr in ↵Michael Anthony Knyszek
pageAlloc.find Currently pageAlloc.find attempts to find a better estimate for the first free page in the heap, even if the space its looking for isn't necessarily going to be the first free page in the heap (e.g. if npages >= 2). However, in doing so it has the potential to return a searchAddr candidate that doesn't actually correspond to mapped memory, but this candidate might still be adopted. As a result, pageAlloc.alloc's fast path may look at unmapped summary memory and segfault. This case is rare on most operating systems since the heap is kept fairly contiguous, so the chance that the candidate searchAddr discovered is unmapped is fairly low. Even so, this is totally possible and outside the user's control when it happens (in fact, it's likely to happen consistently for a given user on a given system). Fix this problem by ensuring that our candidate always points to mapped memory. We do this by looking at mheap's arenas structure first. If it turns out our candidate doesn't correspond to mapped memory, then we look at inUse to round up the searchAddr to the next mapped address. While we're here, clean up some documentation related to searchAddr. For #40191. Fixes #40192. Change-Id: I759efec78987e4a8fde466ae45aabbaa3d9d4214 Reviewed-on: https://go-review.googlesource.com/c/go/+/242680 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Austin Clements <austin@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> (cherry picked from commit b56791cdea5caa87ffcd585d29c294bd3d08a06a) Reviewed-on: https://go-review.googlesource.com/c/go/+/246197 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-21[release-branch.go1.14] cmd/link: fix GC data reading from shared library ↵Cherry Zhang
(attempt 2) This is a backport of CL 240621. This is not a clean cherry-pick, as Go 1.15 switches to the new linker while it is still the old linker here. Backporting is straightforward, though. When linking against a Go shared library, when a global variable in the main module has a type defined in the shared library, the linker needs to pull the GC data from the shared library to build the GC program for the global variable. Currently, this fails silently, as the shared library file is closed too early and the read failed (with no error check), causing a zero GC map emitted for the variable, which in turn causes the runtime to treat the variable as pointerless. For now, fix this by keeping the file open. In the future we may want to use mmap to read from the shared library instead. Also add error checking. And fix a (mostly harmless) mistake in size caluculation. Also remove an erroneous condition for ARM64. ARM64 has a special case to get the addend from the relocation on the gcdata field. But that doesn't actually work. And it's no longer necessary to have any special case, since the addend is now applied directly to the gcdata field on ARM64, like on all the other platforms. Fixes #39955. Updates #39927. Change-Id: I01c82422b9f67e872d833336885935bc509bc91b Reviewed-on: https://go-review.googlesource.com/c/go/+/240621 Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> (cherry picked from commit 7799756a50f0a4070d66c67e9615375f852f2c04) Reviewed-on: https://go-review.googlesource.com/c/go/+/240511 Reviewed-by: Austin Clements <austin@google.com>
2020-08-21[release-branch.go1.14] cmd/compile: mark s390x int <-> float conversions as ↵Michael Munday
clobbering flags These conversion instructions set the condition code and so should be marked as clobbering flags. Updates #39651. Fixes #39690. Change-Id: I1e3f2cf33337128d321b52ac72f46d1b8798ebd9 Reviewed-on: https://go-review.googlesource.com/c/go/+/242237 Run-TryBot: Michael Munday <mike.munday@ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-08-21[release-branch.go1.14] cmd/compile: fix checkptr handling of &^Matthew Dempsky
checkptr has code to recognize &^ expressions, but it didn't take into account that "p &^ x" gets rewritten to "p & ^x" during walk, which resulted in false positive diagnostics. This CL changes walkexpr to mark OANDNOT expressions with Implicit when they're rewritten to OAND, so that walkCheckPtrArithmetic can still recognize them later. It would be slightly more idiomatic to instead mark the OBITNOT expression as Implicit (as it's a compiler-generated Node), but the OBITNOT expression might get constant folded. It's not worth the extra complexity/subtlety of relying on n.Right.Orig, so we set Implicit on the OAND node instead. To atone for this transgression, I add documentation for nodeImplicit. Updates #40917. Fixes #40968. Change-Id: I386304171ad299c530e151e5924f179e9a5fd5b8 Reviewed-on: https://go-review.googlesource.com/c/go/+/249477 Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/249838 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-08-06[release-branch.go1.14] all: merge release-branch.go1.14-security into ↵Katie Hockman
release-branch.go1.14 Change-Id: I9d47ff55ec056567d453e55b215b1e4fc906a407