aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-11-07[release-branch.go1.21] go1.21.4go1.21.4Gopher Robot
Change-Id: I3d607ba9f701a76a46f3ab3223fa83e5c517d285 Reviewed-on: https://go-review.googlesource.com/c/go/+/540517 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Gopher Robot <gobot@golang.org> Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-11-07[release-branch.go1.21] path/filepath: fix various issues in parsing Windows ↵Damien Neil
paths On Windows, A root local device path is a path which begins with \\?\ or \??\. A root local device path accesses the DosDevices object directory, and permits access to any file or device on the system. For example \??\C:\foo is equivalent to common C:\foo. The Clean, IsAbs, IsLocal, and VolumeName functions did not recognize root local device paths beginning with \??\. Clean could convert a rooted path such as \a\..\??\b into the root local device path \??\b. It will now convert this path into .\??\b. IsAbs now correctly reports paths beginning with \??\ as absolute. IsLocal now correctly reports paths beginning with \??\ as non-local. VolumeName now reports the \??\ prefix as a volume name. Join(`\`, `??`, `b`) could convert a seemingly innocent sequence of path elements into the root local device path \??\b. It will now convert this to \.\??\b. In addition, the IsLocal function did not correctly detect reserved names in some cases: - reserved names followed by spaces, such as "COM1 ". - "COM" or "LPT" followed by a superscript 1, 2, or 3. IsLocal now correctly reports these names as non-local. For #63713 Fixes #63715 Fixes CVE-2023-45283 Fixes CVE-2023-45284 Change-Id: I446674a58977adfa54de7267d716ac23ab496c54 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2040691 Reviewed-by: Roland Shoemaker <bracewell@google.com> Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2072596 Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/540276 Auto-Submit: Heschi Kreinick <heschi@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-30[release-branch.go1.21] net/http: pull http2 underflow fix from x/net/http2Mauri de Souza Meneguzzo
After CL 534295 was merged to fix a CVE it introduced an underflow when we try to decrement sc.curHandlers in handlerDone. Pull in a fix from x/net/http2: http2: fix underflow in http2 server push https://go-review.googlesource.com/c/net/+/535595 For #63511 Fixes #63560 Change-Id: I5c678ce7dcc53635f3ad5e4999857cb120dfc1ab GitHub-Last-Rev: 587ffa3cafbb9da6bc82ba8a5b83313f81e5c89b GitHub-Pull-Request: golang/go#63561 Reviewed-on: https://go-review.googlesource.com/c/go/+/535575 Run-TryBot: Mauri de Souza Meneguzzo <mauri870@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: David Chase <drchase@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit 0046c1414c4910dfe54abfcdbe18e565dd5a60f6) Reviewed-on: https://go-review.googlesource.com/c/go/+/537996 Reviewed-by: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-30[release-branch.go1.21] runtime/cgo: avoid taking the address of crosscall2 ↵Cherry Mui
in code Currently, set_crosscall2 takes the address of crosscall2 without using the GOT, which, on some architectures, results in a PC-relative relocation (e.g. R_AARCH64_ADR_PREL_PG_HI21 on ARM64) to the crosscall2 symbol. But crosscall2 is dynamically exported, so the C linker thinks it may bind to a symbol from a different DSO. Some C linker may not like a PC-relative relocation to such a symbol. Using a local trampoline to avoid taking the address of a dynamically exported symbol. It may be possible to not dynamically export crosscall2. But this CL is safer for backport. Later we may remove the trampolines after unexport crosscall2, if they are not needed. Fixes #63509. Updates #62556. Change-Id: Id28457f65ef121d3f87d8189803abc65ed453283 Reviewed-on: https://go-review.googlesource.com/c/go/+/533535 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Ian Lance Taylor <iant@google.com> (cherry picked from commit 872d7181f4084461441787c70ffd1354314987af) Reviewed-on: https://go-review.googlesource.com/c/go/+/534915 Reviewed-by: David Chase <drchase@google.com>
2023-10-13[release-branch.go1.21] cmd/compile/internal/typecheck: fix closure field namingMatthew Dempsky
When creating the struct type to hold variables captured by a function literal, we currently reuse the captured variable names as fields. However, there's no particular reason to do this: these struct types aren't visible to users, and it adds extra complexity in making sure fields belong to the correct packages. Further, it turns out we were getting that subtly wrong. If two function literals from different packages capture variables with identical names starting with an uppercase letter (and in the same order and with corresponding identical types) end up in the same function (e.g., due to inlining), then we could end up creating closure struct types that are "different" (i.e., not types.Identical) yet end up with equal LinkString representations (which violates LinkString's contract). The easy fix is to just always use simple, exported, generated field names in the struct. This should allow further struct reuse across packages too, and shrink binary sizes slightly. For #62498. Fixes #62545. Change-Id: I9c973f5087bf228649a8f74f7dc1522d84a26b51 Reviewed-on: https://go-review.googlesource.com/c/go/+/527135 Auto-Submit: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> (cherry picked from commit e3ce3126212115808bc248bdc9ad92c0a46436fe) Reviewed-on: https://go-review.googlesource.com/c/go/+/534916 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-10-13[release-branch.go1.21] go/types, types2: don't use genericsRobert Griesemer
This fixes cherry-pick CL 531998. For #63339. Change-Id: I6dac0909ca85d68684ce36025284d25db32e0b15 Reviewed-on: https://go-review.googlesource.com/c/go/+/535135 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alan Donovan <adonovan@google.com>
2023-10-12[release-branch.go1.21] runtime: don't eagerly collapse hugepagesMichael Anthony Knyszek
This has caused performance issues in production environments. MADV_COLLAPSE can go into direct reclaim, but we call it with the heap lock held. This means that the process could end up stalled fairly quickly if just one allocating goroutine ends up in the madvise call, at least until the madvise(MADV_COLLAPSE) call returns. A similar issue occurred with madvise(MADV_HUGEPAGE), because that could go into direct reclaim on any page fault for MADV_HUGEPAGE-marked memory. My understanding was that the calls to madvise(MADV_COLLAPSE) were fairly rare, and it's "best-effort" nature prevented it from going into direct reclaim often, but this was wrong. It tends to be fairly heavyweight even when it doesn't end up in direct reclaim, and it's almost certainly not worth it. Disable it until further notice and let the kernel fully dictate hugepage policy. The updated scavenger policy is still more hugepage friendly by delaying scavening until hugepages are no longer densely packed, so we don't lose all that much. The Sweet benchmarks show a minimal difference. A couple less realistic benchmarks seem to slow down a bit; they might just be getting unlucky with what the kernel decides to back with a huge page. Some benchmarks on the other hand improve. Overall, it's a wash. name old time/op new time/op delta BiogoIgor 13.1s ± 1% 13.2s ± 2% ~ (p=0.182 n=9+10) BiogoKrishna 12.0s ± 1% 12.1s ± 1% +1.23% (p=0.002 n=9+10) BleveIndexBatch100 4.51s ± 4% 4.56s ± 3% ~ (p=0.393 n=10+10) EtcdPut 20.2ms ± 4% 19.8ms ± 2% ~ (p=0.079 n=10+9) EtcdSTM 109ms ± 3% 111ms ± 3% +1.63% (p=0.035 n=10+10) GoBuildKubelet 31.2s ± 1% 31.3s ± 1% ~ (p=0.780 n=9+10) GoBuildKubeletLink 7.77s ± 0% 7.81s ± 2% ~ (p=0.237 n=8+10) GoBuildIstioctl 31.8s ± 1% 31.7s ± 0% ~ (p=0.136 n=9+9) GoBuildIstioctlLink 7.88s ± 1% 7.89s ± 1% ~ (p=0.720 n=9+10) GoBuildFrontend 11.7s ± 1% 11.8s ± 1% ~ (p=0.278 n=10+9) GoBuildFrontendLink 1.15s ± 4% 1.15s ± 5% ~ (p=0.387 n=9+9) GopherLuaKNucleotide 19.7s ± 1% 20.6s ± 0% +4.48% (p=0.000 n=10+10) MarkdownRenderXHTML 194ms ± 3% 196ms ± 3% ~ (p=0.356 n=9+10) Tile38QueryLoad 633µs ± 2% 629µs ± 2% ~ (p=0.075 n=10+10) name old average-RSS-bytes new average-RSS-bytes delta BiogoIgor 69.2MB ± 3% 68.4MB ± 1% ~ (p=0.190 n=10+10) BiogoKrishna 4.40GB ± 0% 4.40GB ± 0% ~ (p=0.605 n=9+9) BleveIndexBatch100 195MB ± 3% 195MB ± 2% ~ (p=0.853 n=10+10) EtcdPut 107MB ± 4% 108MB ± 3% ~ (p=0.190 n=10+10) EtcdSTM 91.6MB ± 5% 92.6MB ± 4% ~ (p=0.481 n=10+10) GoBuildKubelet 2.26GB ± 1% 2.28GB ± 1% +1.22% (p=0.000 n=10+10) GoBuildIstioctl 1.53GB ± 0% 1.53GB ± 0% +0.21% (p=0.017 n=9+10) GoBuildFrontend 556MB ± 1% 554MB ± 2% ~ (p=0.497 n=9+10) GopherLuaKNucleotide 39.0MB ± 3% 39.0MB ± 1% ~ (p=1.000 n=10+8) MarkdownRenderXHTML 21.2MB ± 2% 21.4MB ± 3% ~ (p=0.190 n=10+10) Tile38QueryLoad 5.99GB ± 2% 6.02GB ± 0% ~ (p=0.243 n=10+9) name old peak-RSS-bytes new peak-RSS-bytes delta BiogoIgor 90.2MB ± 4% 89.2MB ± 2% ~ (p=0.143 n=10+10) BiogoKrishna 4.49GB ± 0% 4.49GB ± 0% ~ (p=0.190 n=10+10) BleveIndexBatch100 283MB ± 8% 274MB ± 6% ~ (p=0.075 n=10+10) EtcdPut 147MB ± 4% 149MB ± 2% +1.55% (p=0.034 n=10+8) EtcdSTM 117MB ± 5% 117MB ± 4% ~ (p=0.905 n=9+10) GopherLuaKNucleotide 44.9MB ± 1% 44.6MB ± 1% ~ (p=0.083 n=8+8) MarkdownRenderXHTML 22.0MB ± 8% 22.1MB ± 9% ~ (p=0.436 n=10+10) Tile38QueryLoad 6.24GB ± 2% 6.29GB ± 2% ~ (p=0.218 n=10+10) name old peak-VM-bytes new peak-VM-bytes delta BiogoIgor 1.33GB ± 0% 1.33GB ± 0% ~ (p=0.504 n=10+9) BiogoKrishna 5.77GB ± 0% 5.77GB ± 0% ~ (p=1.000 n=10+9) BleveIndexBatch100 3.53GB ± 0% 3.53GB ± 0% ~ (p=0.642 n=10+10) EtcdPut 12.1GB ± 0% 12.1GB ± 0% ~ (p=0.564 n=10+10) EtcdSTM 12.1GB ± 0% 12.1GB ± 0% ~ (p=0.633 n=10+10) GopherLuaKNucleotide 1.26GB ± 0% 1.26GB ± 0% ~ (p=0.297 n=9+10) MarkdownRenderXHTML 1.26GB ± 0% 1.26GB ± 0% ~ (p=0.069 n=10+10) Tile38QueryLoad 7.47GB ± 2% 7.53GB ± 2% ~ (p=0.280 n=10+10) name old p50-latency-ns new p50-latency-ns delta EtcdPut 19.8M ± 5% 19.3M ± 3% -2.74% (p=0.043 n=10+9) EtcdSTM 81.4M ± 4% 83.4M ± 4% +2.46% (p=0.029 n=10+10) Tile38QueryLoad 241k ± 1% 240k ± 1% ~ (p=0.393 n=10+10) name old p90-latency-ns new p90-latency-ns delta EtcdPut 30.4M ± 5% 30.6M ± 5% ~ (p=0.971 n=10+10) EtcdSTM 222M ± 3% 226M ± 4% ~ (p=0.063 n=10+10) Tile38QueryLoad 687k ± 2% 691k ± 1% ~ (p=0.173 n=10+8) name old p99-latency-ns new p99-latency-ns delta EtcdPut 42.3M ±10% 41.4M ± 7% ~ (p=0.353 n=10+10) EtcdSTM 486M ± 7% 487M ± 4% ~ (p=0.579 n=10+10) Tile38QueryLoad 6.43M ± 2% 6.37M ± 3% ~ (p=0.280 n=10+10) name old ops/s new ops/s delta EtcdPut 48.6k ± 3% 49.5k ± 2% ~ (p=0.065 n=10+9) EtcdSTM 9.09k ± 2% 8.95k ± 3% -1.56% (p=0.045 n=10+10) Tile38QueryLoad 28.4k ± 1% 28.6k ± 1% +0.87% (p=0.016 n=9+10) Fixes #63335. For #63334. Related to #61718 and #59960. Change-Id: If84c5a8685825d43c912a71418f2597e44e867e5 Reviewed-on: https://go-review.googlesource.com/c/go/+/531816 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> (cherry picked from commit 595deec3dda8e81d514389efdbb4ee2bc38dcabe) Reviewed-on: https://go-review.googlesource.com/c/go/+/532255 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-10-12[release-branch.go1.21] go/types, types2: don't implicitly modify an ↵Robert Griesemer
argument function's type See the comment in the (very small) fix for a detailed description. Use the opportunity to introduce a generic clone function which may be useful elsewhere. Fixes #63339. Change-Id: Ic63c6b8bc443011b1a201908254f7d062e1aec71 Reviewed-on: https://go-review.googlesource.com/c/go/+/532157 Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Auto-Submit: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-on: https://go-review.googlesource.com/c/go/+/531998 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-10-12[release-branch.go1.21] cmd/link: split text sections for arm 32-bitThan McIntosh
This CL is a roll-forward (tweaked slightly) of CL 467715, which turned on text section splitting for GOARCH=arm. The intent is to avoid recurrent problems with external linking where there is a disagreement between the Go linker and the external linker over whether a given branch will reach. In the past our approach has been to tweak the reachability calculations slightly to try to work around potential linker problems, but this hasn't proven to be very robust; section splitting seems to offer a better long term fix. Updates #58425. Fixes #63317. Change-Id: I7372d41abce84097906a3d0805b6b9c486f345d6 Reviewed-on: https://go-review.googlesource.com/c/go/+/531795 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit 1e690409206ff97330b5a91517d453fc5129bab2) Reviewed-on: https://go-review.googlesource.com/c/go/+/532096 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-10-10[release-branch.go1.21] all: tidy dependency versioning after releaseDmitri Shuralyov
Done with: go get golang.org/x/net@internal-branch.go1.21-vendor go mod tidy go mod vendor go generate net/http # zero diff since CL 534235 already did this For #63417. For #63427. For CVE-2023-39325. Change-Id: Ib258e0d8165760a1082e02c2f4c5ce7d2a3c3c90 Reviewed-on: https://go-review.googlesource.com/c/go/+/534415 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2023-10-10[release-branch.go1.21] go1.21.3go1.21.3Gopher Robot
Change-Id: Icdde32852b305006d7064cfe8bbe143128191842 Reviewed-on: https://go-review.googlesource.com/c/go/+/534337 TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com> Commit-Queue: Gopher Robot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Auto-Submit: Gopher Robot <gobot@golang.org>
2023-10-10[release-branch.go1.21] net/http: regenerate h2_bundle.goDamien Neil
Pull in a security fix from x/net/http2: http2: limit maximum handler goroutines to MaxConcurrentStreamso For #63417 Fixes #63427 Fixes CVE-2023-39325 Change-Id: I70626734e6d56edf508f27a5b055ddf96d806eeb Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2047402 Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Ian Cottrell <iancottrell@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/534235 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-10-05[release-branch.go1.21] go1.21.2go1.21.2Gopher Robot
Change-Id: Ie06bc8d8e8e170f13ed938a33038931513eaf067 Reviewed-on: https://go-review.googlesource.com/c/go/+/533238 Auto-Submit: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-10-05[release-branch.go1.21] cmd/compile: use absolute file name in isCgo checkIan Lance Taylor
For #23672 Updates #63211 Fixes #63214 Fixes CVE-2023-39323 Change-Id: I4586a69e1b2560036afec29d53e53cf25e6c7352 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2032884 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com> (cherry picked from commit 9b19e751918dd218035811b1ef83a8c2693b864a) Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2037958 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Run-TryBot: Roland Shoemaker <bracewell@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/533215 Reviewed-by: Than McIntosh <thanm@google.com> Auto-Submit: Michael Pratt <mpratt@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-09-25[release-branch.go1.21] cmd/compile: absorb InvertFlags into Noov comparisonsKeith Randall
Unfortunately, there isn't a single op that provides the resulting computation. At least, I couldn't find one. Fixes #62506 Change-Id: I236f3965b827aaeb3d70ef9fe89be66b116494f5 Reviewed-on: https://go-review.googlesource.com/c/go/+/526276 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Keith Randall <khr@google.com> (cherry picked from commit fb5bdb4cc94d23209d77a73c4148b3f9fbb10173) Reviewed-on: https://go-review.googlesource.com/c/go/+/526521 Reviewed-by: Than McIntosh <thanm@google.com>
2023-09-25[release-branch.go1.21] cmd/compile: reset memcombine correctly between ↵Keith Randall
basic blocks Not sure why this bug didn't cause a complete failure, but it certainly makes for doing a lot more work than is necessary. Fixes #62668 Change-Id: If0be4acb6eafc3d7eeb42d2f4263c21b4e6e1c7d Reviewed-on: https://go-review.googlesource.com/c/go/+/527699 Reviewed-by: David Chase <drchase@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Heschi Kreinick <heschi@google.com> (cherry picked from commit 08cdfd06ed1ca21f3b8c95361506ab94c429b622) Reviewed-on: https://go-review.googlesource.com/c/go/+/528795 Reviewed-by: Keith Randall <khr@google.com>
2023-09-25[release-branch.go1.21] runtime: always lock OS thread in debugcallMichael Anthony Knyszek
Right now debuggers like Delve rely on the new goroutine created to run a debugcall function to run on the same thread it started on, up until it hits itself with a SIGINT as part of the debugcall protocol. That's all well and good, except debugCallWrap1 isn't particularly careful about not growing the stack. For example, if the new goroutine happens to have a stale preempt flag, then it's possible a stack growth will cause a roundtrip into the scheduler, possibly causing the goroutine to switch to another thread. Previous attempts to just be more careful around debugCallWrap1 were helpful, but insufficient. This change takes everything a step further and always locks the debug call goroutine and the new goroutine it creates to the OS thread. For #61732. Fixes #62509. Change-Id: I038f3a4df30072833e27e6a5a1ec01806a32891f Reviewed-on: https://go-review.googlesource.com/c/go/+/515637 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com> Reviewed-by: Cherry Mui <cherryyz@google.com> (cherry picked from commit d9a4b24a1775c4c5baa4ce3005cb5af61346198e) Reviewed-on: https://go-review.googlesource.com/c/go/+/526576
2023-09-22[release-branch.go1.21] runtime: increase g0 stack size in non-cgo caseCherry Mui
Currently, for non-cgo programs, the g0 stack size is 8 KiB on most platforms. With PGO which could cause aggressive inlining in the runtime, the runtime stack frames are larger and could overflow the 8 KiB g0 stack. Increase it to 16 KiB. This is only one per OS thread, so it shouldn't increase memory use much. Updates #62120. Updates #62489. Fixes #62537. Change-Id: I565b154517021f1fd849424dafc3f0f26a755cac Reviewed-on: https://go-review.googlesource.com/c/go/+/526995 Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> (cherry picked from commit c6d550a6683cebb2a11d7fa91823edf7db1d58a5) Reviewed-on: https://go-review.googlesource.com/c/go/+/527055
2023-09-21[release-branch.go1.21] cmd/link: suppress -bind_at_load deprecation warning ↵Cherry Mui
for ld-prime ld-prime emits a deprecation warning for -bind_at_load. The flag is needed for plugins to not deadlock (#38824) when linking with older darwin linker. It is supposedly not needed with newer linker when chained fixups are used. For now, we always pass it, and suppress the warning. Updates #61229. For #62598. Change-Id: I4b8a6f864a460c40dc38adbb533f664f7fd5343c Reviewed-on: https://go-review.googlesource.com/c/go/+/508696 Reviewed-by: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com> (cherry picked from commit 040dbf9c181a0e3ea9f7bd3ebe3f75acdc878aaf) Reviewed-on: https://go-review.googlesource.com/c/go/+/527817
2023-09-21[release-branch.go1.21] cmd/link: disable DWARF by default in c-shared mode ↵Cherry Mui
on darwin [This is a (manual) backport of CL 527415 to Go 1.21.] Currently, linking a Go c-shared object with C code using Apple's new linker, it fails with % cc a.c go.so ld: segment '__DWARF' filesize exceeds vmsize in 'go.so' Apple's new linker has more checks for unmapped segments. It is very hard to make it accept a Mach-O shared object with an additional DWARF segment. We may want to stop combinding DWARF into the shared object (see also #62577). For now, disable DWARF by default in c-shared mode on darwin. Updates #61229. For #62598. Change-Id: I525987b7fe1a4e64571327cb4696f98cc7b419a1 Reviewed-on: https://go-review.googlesource.com/c/go/+/527816 Reviewed-by: Than McIntosh <thanm@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-21[release-branch.go1.21] cmd/link: force old Apple linker in plugin modeCherry Mui
There are some bugs in Apple's new linker that causes plugins to be built incorrectly. And the bugs probably will not be fixed when Xcode 15 is released (some time soon). Force old Apple linker to work around. Updates #61229. For #62598. Change-Id: I01ba5caadec6dc14f8c85dd02f78c1ed2e8b7d4d Reviewed-on: https://go-review.googlesource.com/c/go/+/527815 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com>
2023-09-20[release-branch.go1.21] cmd/link: avoid deadcode of global map vars for ↵Than McIntosh
programs using plugins If a program imports the plugin package, the mechanisms in place for detecting and deleting unused global map variables are no longer safe, since it's possibly for a given global map var to be unreferenced in the main program but referenced by a plugin. This patch changes the linker to test for plugin use and to avoid removing any unused global map variables if the main program could possibly load up a plugin. Fixes #62505. Updates #62430. Change-Id: Ie00b18b681cb0d259e3c859ac947ade5778cd6c8 (cherry picked from commit 660620dd45dc11f2d889add79bedf2dc771c7d04) Reviewed-on: https://go-review.googlesource.com/c/go/+/526575 Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-19[release-branch.go1.21] cmd/go: in TestScript/gotoolchain_path, remove the ↵Bryan C. Mills
user's PATH This test checks a behavior of GOTOOLCHAIN when an appropriate toolchain is found in PATH. That requires it to exclude any suitable toolchain binaries from the user's $PATH, which may otherwise interfere. Fixes #62711. Updates #62709. Change-Id: Ie9161e52d33a65be0b5265cb49e9f2bc8473e057 Reviewed-on: https://go-review.googlesource.com/c/go/+/529217 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> (cherry picked from commit 30886b1b1ea0ff0282419e4deefdd35a51b2c9fc) Reviewed-on: https://go-review.googlesource.com/c/go/+/529435 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-09-11[release-branch.go1.21] runtime/metrics: fix /gc/scan/* metricsNayef Ghattas
In the existing implementation, all /gc/scan/* metrics are always equal to 0 due to the dependency on gcStatDep not being set. This leads to gcStatAggregate always containing zeros, and always reporting 0 for those metrics. Also, add a test to ensure that /gc/scan/* metrics are not empty. For #62477. Fixes #62478. Change-Id: I67497347d50ed5c3ce1719a18714c062ec938cab Reviewed-on: https://go-review.googlesource.com/c/go/+/526116 Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Heschi Kreinick <heschi@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-09-11[release-branch.go1.21] runtime: ignore SPWrite on innermost traceback frameAustin Clements
Prior to CL 458218, gentraceback ignored the SPWrite function flag on the innermost frame when doing a precise traceback on the assumption that precise tracebacks could only be started from the morestack prologue, and that meant that the innermost function could not have modified SP yet. CL 458218 rearranged this logic a bit and unintentionally lost this particular case. As a result, if traceback starts in an assembly function that modifies SP (either as a result of stack growth or stack scanning during a GC preemption), traceback stop at the SPWrite function and then crash with "traceback did not unwind completely". Fix this by restoring the earlier special case for when the innermost frame is SPWrite. This is a fairly minimal change that should be easy to backport. I think a more robust change would be to encode this per-PC in the spdelta table, so it would be clear that we're unwinding from the morestack prologue and wouldn't rely on a complicated and potentially fragile set of conditions. Fixes #62464. Change-Id: I34f38157631890d33a79d0bd32e32c0fcc2574e4 Reviewed-on: https://go-review.googlesource.com/c/go/+/526100 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Michael Pratt <mpratt@google.com> Auto-Submit: Heschi Kreinick <heschi@google.com>
2023-09-06[release-branch.go1.21] go1.21.1go1.21.1Gopher Robot
Change-Id: I1c208f9cd77af66e98a4d159c2e6c3b6072ea7ec Reviewed-on: https://go-review.googlesource.com/c/go/+/526039 Run-TryBot: Cherry Mui <cherryyz@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Joedian Reid <joedian@golang.org> Auto-Submit: Gopher Robot <gobot@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-09-06[release-branch.go1.21] html/template: properly handle special tags within ↵Roland Shoemaker
the script context The HTML specification has incredibly complex rules for how to handle "<!--", "<script", and "</script" when they appear within literals in the script context. Rather than attempting to apply these restrictions (which require a significantly more complex state machine) we apply the workaround suggested in section 4.12.1.3 of the HTML specification [1]. More precisely, when "<!--", "<script", and "</script" appear within literals (strings and regular expressions, ignoring comments since we already elide their content) we replace the "<" with "\x3C". This avoids the unintuitive behavior that using these tags within literals can cause, by simply preventing the rendered content from triggering it. This may break some correct usages of these tags, but on balance is more likely to prevent XSS attacks where users are unknowingly either closing or not closing the script blocks where they think they are. Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this issue. Fixes #62197 Fixes #62398 Fixes CVE-2023-39319 [1] https://html.spec.whatwg.org/#restrictions-for-contents-of-script-elements Change-Id: Iab57b0532694827e3eddf57a7497ba1fab1746dc Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1976594 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Reviewed-by: Damien Neil <dneil@google.com> Run-TryBot: Roland Shoemaker <bracewell@google.com> Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2014619 Reviewed-on: https://go-review.googlesource.com/c/go/+/526097 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-09-06[release-branch.go1.21] html/template: support HTML-like comments in script ↵Roland Shoemaker
contexts Per Appendix B.1.1 of the ECMAScript specification, support HTML-like comments in script contexts. Also per section 12.5, support hashbang comments. This brings our parsing in-line with how browsers treat these comment types. Thanks to Takeshi Kaneko (GMO Cybersecurity by Ierae, Inc.) for reporting this issue. Fixes #62196 Fixes #62396 Fixes CVE-2023-39318 Change-Id: Id512702c5de3ae46cf648e268cb10e1eb392a181 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1976593 Run-TryBot: Roland Shoemaker <bracewell@google.com> Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2014618 Reviewed-on: https://go-review.googlesource.com/c/go/+/526096 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com>
2023-09-06[release-branch.go1.21] cmd/go: reject toolchain directives containing path ↵Bryan C. Mills
separators If GOTOOLCHAIN="path" or "auto", the go command uses exec.LookPath to search for it in order to allow toolchains to refer to local-only toolchain variants (such as toolchains built from enterprise- or distro-patched source). However, those toolchains should only be resolved from $PATH, not relative to the working directory of the command. Thanks to Juho Nurminen of Mattermost for reporting this issue. Fixes #62198. Fixes #62394. Fixes CVE-2023-39320. Change-Id: I247c7acea95d737362dd0475e9fc8515430d0fcc Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1996318 Run-TryBot: Roland Shoemaker <bracewell@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com> (cherry picked from commit e41c0a55d45e9a9acbc5d7c1143ea4fff8fb9283) Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/2014013 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Tatiana Bradley <tatianabradley@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/526095 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Cherry Mui <cherryyz@google.com>
2023-08-31[release-branch.go1.21] net/http: revert "support streaming POST content in ↵haruyama480
wasm" CL 458395 added support for streaming POST content in Wasm. Unfortunately, this breaks requests to servers that only support HTTP/1.1. Revert the change until a suitable fallback or opt-in strategy can be decided. For #61889. Fixes #62328. Change-Id: If53a77e1890132063b39abde867d34515d4ac2af Reviewed-on: https://go-review.googlesource.com/c/go/+/522955 Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/524855 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Commit-Queue: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-30[release-branch.go1.21] runtime: restore caller's frame pointer when ↵Nick Ripley
recovering from panic When recovering from a panic, restore the caller's frame pointer before returning control to the caller. Otherwise, if the function proceeds to run more deferred calls before returning, the deferred functions will get invalid frame pointers pointing to an address lower in the stack. This can cause frame pointer unwinding to crash, such as if an execution trace event is recorded during the deferred call on architectures which support frame pointer unwinding. Original CL by Nick Ripley, includes fix from CL 523697, and includes a test update from CL 524315. This CL also deviates from the original fix by doing some extra computation to figure out the fp from the sp, since we don't have the fp immediately available to us in `recovery` on the Go 1.21 branch, and it would probably be complicated to plumb that through its caller. For #61766 Fixes #62046 Change-Id: I5a99ca4f909f6b6e209a330d595d1c99987d4359 Reviewed-on: https://go-review.googlesource.com/c/go/+/523698 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-08-30[release-branch.go1.21] runtime: avoid MADV_HUGEPAGE for heap memoryMichael Anthony Knyszek
Currently the runtime marks all new memory as MADV_HUGEPAGE on Linux and manages its hugepage eligibility status. Unfortunately, the default THP behavior on most Linux distros is that MADV_HUGEPAGE blocks while the kernel eagerly reclaims and compacts memory to allocate a hugepage. This direct reclaim and compaction is unbounded, and may result in significant application thread stalls. In really bad cases, this can exceed 100s of ms or even seconds. Really all we want is to undo MADV_NOHUGEPAGE marks and let the default Linux paging behavior take over, but the only way to unmark a region as MADV_NOHUGEPAGE is to also mark it MADV_HUGEPAGE. The overall strategy of trying to keep hugepages for the heap unbroken however is sound. So instead let's use the new shiny MADV_COLLAPSE if it exists. MADV_COLLAPSE makes a best-effort synchronous attempt at collapsing the physical memory backing a memory region into a hugepage. We'll use MADV_COLLAPSE where we would've used MADV_HUGEPAGE, and stop using MADV_NOHUGEPAGE altogether. Because MADV_COLLAPSE is synchronous, it's also important to not re-collapse huge pages if the huge pages are likely part of some large allocation. Although in many cases it's advantageous to back these allocations with hugepages because they're contiguous, eagerly collapsing every hugepage means having to page in at least part of the large allocation. However, because we won't use MADV_NOHUGEPAGE anymore, we'll no longer handle the fact that khugepaged might come in and back some memory we returned to the OS with a hugepage. I've come to the conclusion that this is basically unavoidable without a new madvise flag and that it's just not a good default. If this change lands, advice about Linux huge page settings will be added to the GC guide. Verified that this change doesn't regress Sweet, at least not on my machine with: /sys/kernel/mm/transparent_hugepage/enabled [always or madvise] /sys/kernel/mm/transparent_hugepage/defrag [madvise] /sys/kernel/mm/transparent_hugepage/khugepaged/max_ptes_none [0 or 511] Unfortunately, this workaround means that we only get forced hugepages on Linux 6.1+. For #61718. Fixes #62329. Change-Id: I7f4a7ba397847de29f800a99f9cb66cb2720a533 Reviewed-on: https://go-review.googlesource.com/c/go/+/516795 Reviewed-by: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Knyszek <mknyszek@google.com> Auto-Submit: Michael Knyszek <mknyszek@google.com> (cherry picked from commit 9f9bb26880388c5bead158e9eca3be4b3a9bd2af) Reviewed-on: https://go-review.googlesource.com/c/go/+/523655 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-30[release-branch.go1.21] go/types, types2: remove order dependency in ↵Robert Griesemer
inference involving channels In inexact unification, when a named type matches against an inferred unnamed type, we change the previously inferred type to the named type. This preserves the type name and assignability. We have to do the same thing when encountering a directional channel: a bidirectional channel can always be assigned to a directional channel but not the other way around. Thus, if we see a directional channel, we must choose the directional channel. This CL extends the previously existing logic for named types to directional channels and also makes the code conditional on inexact unification. The latter is an optimization - if unification is exact, type differences don't exist and updating an already inferred type has no effect. Fixes #62205. Change-Id: I807e3b9f9ab363f9ed848bdb18b2577b1d680ea7 Reviewed-on: https://go-review.googlesource.com/c/go/+/524256 Run-TryBot: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
2023-08-30[release-branch.go1.21] cmd/go: retry ETXTBSY errors when running test binariesBryan C. Mills
An ETXTBSY error when starting a test binary is almost certainly caused by the race reported in #22315. That race will resolve quickly on its own, so we should just retry the command instead of reporting a spurious failure. Fixes #62222. Updates #62221. Change-Id: I408f3eaa7ab5d7efbc7a2b1c8bea3dbc459fc794 Reviewed-on: https://go-review.googlesource.com/c/go/+/522015 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> (cherry picked from commit 4dc2564933146efc411efad16b662589306744d1) Reviewed-on: https://go-review.googlesource.com/c/go/+/522176 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-30[release-branch.go1.21] crypto/tls: QUIC: fix panics when processing ↵Damien Neil
post-handshake messages The check for fragmentary post-handshake messages in QUICConn.HandleData was reversed, resulting in a potential panic when HandleData receives a partial message. In addition, HandleData wasn't checking the size of buffered post-handshake messages. Produce an error when a post-handshake message is larger than maxHandshake. TestQUICConnectionState was using an onHandleCryptoData hook in runTestQUICConnection that was never being called. (I think it was inadvertently removed at some point while the CL was in review.) Fix this test while making the hook more general. For #62266 Fixes #62290 Change-Id: I210b70634e50beb456ab3977eb11272b8724c241 Reviewed-on: https://go-review.googlesource.com/c/go/+/522595 Run-TryBot: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Marten Seemann <martenseemann@gmail.com> Reviewed-by: Roland Shoemaker <roland@golang.org> (cherry picked from commit e92c0f846c54d88f479b1c48f0dbc001d2ff53e9) Reviewed-on: https://go-review.googlesource.com/c/go/+/523039 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2023-08-30[release-branch.go1.21] cmd/go: find GOROOT using os.Executable when ↵Bryan C. Mills
installed to GOROOT/bin/GOOS_GOARCH When running make.bash in a cross-compiled configuration (for example, GOARCH different from GOHOSTARCH), cmd/go is installed to GOROOT/bin/GOOS_GOARCH instead of GOROOT/bin. That means that we need to look for GOROOT in both ../.. and ../../.., not just the former. Fixes #62144. Updates #62119. Updates #18678. Change-Id: I283c6a10c46df573ff44da826f870417359226a7 Reviewed-on: https://go-review.googlesource.com/c/go/+/521015 Reviewed-by: Michael Matloob <matloob@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit 9e9556d328c53ed0a4d8b36feee949885d648ba8) Reviewed-on: https://go-review.googlesource.com/c/go/+/521695 Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-30[release-branch.go1.21] encoding/xml: overriding by empty namespace when no ↵xuri
new name declaration The unmarshal and marshal XML text should be consistent if not modified deserialize variable. For #61881 Fixes #62051 Change-Id: I475f7b05211b618685597d3ff20b97e3bbeaf8f8 GitHub-Last-Rev: 6831c770c384831798cb1c6dc4674e5d4caa5e3c GitHub-Pull-Request: golang/go#58401 Reviewed-on: https://go-review.googlesource.com/c/go/+/522316 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Ian Lance Taylor <iant@google.com> Commit-Queue: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-08-25[release-branch.go1.21] encoding/gob: prevent panic from index out of range ↵Andy Pan
in Decoder.typeString I believe this bug is introduced by CL 460543 which optimizes the allocations by changing the type of `idToType` from map to slice, but didn't update the access code in `Decoder.typeString` that is safe for map but not for slice. For #62117 Fixes #62154 Change-Id: I0f2e4cc2f34c54dada1f83458ba512a6fde6dcbe Reviewed-on: https://go-review.googlesource.com/c/go/+/520757 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Run-TryBot: Andy Pan <panjf2000@gmail.com> Auto-Submit: Ian Lance Taylor <iant@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> (cherry picked from commit ba626ac327f45a6d9d211fddd5b48e321fa0702a) Reviewed-on: https://go-review.googlesource.com/c/go/+/521156 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-08-25[release-branch.go1.21] runtime: fix maps.Clone bug when cloning a map mid-growKeith Randall
Fixes #62204 Change-Id: I0459d3f481b0cd20102f6d9fd3ea84335a7739a8 Reviewed-on: https://go-review.googlesource.com/c/go/+/522317 Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Reviewed-by: Keith Randall <khr@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit b303fb48558612e5c2a1f10acbc0b1accdb8f260) Reviewed-on: https://go-review.googlesource.com/c/go/+/522936 Reviewed-by: Carlos Amedee <carlos@golang.org>
2023-08-24[release-branch.go1.21] context: fix synchronization in ExampleAfterFunc_condBryan C. Mills
Condition variables are subtle and error-prone, and this example demonstrates exactly the sorts of problems that they introduce. Unfortunately, we're stuck with them for the foreseeable future. As previously implemented, this example was racy: since the callback passed to context.AfterFunc did not lock the mutex before calling Broadcast, it was possible for the Broadcast to occur before the goroutine was parked in the call to Wait, causing in a missed wakeup resulting in deadlock. The example also had a more insidious problem: it was not safe for multiple goroutines to call waitOnCond concurrently, but the whole point of using a sync.Cond is generally to synchronize concurrent goroutines. waitOnCond must use Broadcast to ensure that it wakes up the target goroutine, but the use of Broadcast in this way would produce spurious wakeups for all of the other goroutines waiting on the same condition variable. Since waitOnCond did not recheck the condition in a loop, those spurious wakeups would cause waitOnCond to spuriously return even if its own ctx was not yet done. Fixing the aforementioned bugs exposes a final problem, inherent to the use of condition variables in this way. This one is a performance problem: for N concurrent calls to waitOnCond, the resulting CPU cost is at least O(N²). This problem cannot be addressed without either reintroducing one of the above bugs or abandoning sync.Cond in the example entirely. Given that this example was already published in Go 1.21, I worry that Go users may think that it is appropriate to use a sync.Cond in conjunction with context.AfterFunc, so I have chosen to retain the Cond-based example and document its pitfalls instead of removing or replacing it entirely. I described this class of bugs and performance issues — and suggested some channel-based alternatives — in my GopherCon 2018 talk, “Rethinking Classical Concurrency Patterns”. The section on condition variables starts on slide 37. (https://youtu.be/5zXAHh5tJqQ?t=679) Fixes #62189. Updates #62180. For #20491. Change-Id: If987cd9d112997c56171a7ef4fccadb360bb79bc Reviewed-on: https://go-review.googlesource.com/c/go/+/521596 Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> (cherry picked from commit 1081f8c05807728738e1e6bf9097a41796fe5a05) Reviewed-on: https://go-review.googlesource.com/c/go/+/521598
2023-08-24[release-branch.go1.21] go/types, types2: disable interface inference for ↵Robert Griesemer
versions before Go 1.21 Change the internal constant enableInterfaceInference to a unifier field that can be controlled dynamically and set it for Go 1.21 or later. This restores Go 1.20 unification behavior for interfaces. Fixes #61930. Change-Id: Iefd6c0899811f8208a8be9cef2650a07787ae177 Reviewed-on: https://go-review.googlesource.com/c/go/+/519855 Reviewed-by: Robert Findley <rfindley@google.com> Auto-Submit: Robert Griesemer <gri@google.com> Reviewed-by: Robert Griesemer <gri@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Robert Griesemer <gri@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/520601
2023-08-24[release-branch.go1.21] cmd/compile: in expandCalls, move all arg ↵David Chase
marshalling into call block For aggregate-typed arguments passed to a call, expandCalls decomposed them into parts in the same block where the value was created. This is not necessarily the call block, and in the case where stores are involved, can change the memory leaving that block, and getting that right is problematic. Instead, do all the expanding in the same block as the call, which avoids the problems of (1) not being able to reorder loads/stores across a block boundary to conform to memory order and (2) (incorrectly, not) exposing the new memory to consumers in other blocks. Putting it all in the same block as the call allows reordering, and the call creates its own new memory (which is already dealt with correctly). Fixes #62057. Updates #61992. Change-Id: Icc7918f0d2dd3c480cc7f496cdcd78edeca7f297 Reviewed-on: https://go-review.googlesource.com/c/go/+/519276 Reviewed-by: Keith Randall <khr@google.com> Run-TryBot: David Chase <drchase@google.com> Reviewed-by: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit e72ecc6a6becab9ca895c0108047db4723394296) Reviewed-on: https://go-review.googlesource.com/c/go/+/520058
2023-08-23[release-branch.go1.21] Revert "os: use handle based APIs to read ↵Bryan Mills
directories on windows" This reverts CL 452995. Reason for revert: caused os.File.ReadDir to fail on filesystems that do not support FILE_ID_BOTH_DIR_INFO. This is an alternative to a fix-forward change in CL 518196. Since the original change was mostly a performance improvement, reverting to the previous implementation seems less risky than backporting a larger fix. Fixes #61910 Fixes #61964 Change-Id: I60f1602b9eb6ea353e7eb23429f19f1ffa16b394 Reviewed-on: https://go-review.googlesource.com/c/go/+/520156 Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Austin Clements <austin@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Quim Muntal <quimmuntal@gmail.com>
2023-08-23[release-branch.go1.21] Revert "cmd/compile: omit redundant sign/unsign ↵Keith Randall
extension on arm64" This reverts CL 427454. Reason for revert: causes incorrect generated code in some rare cases We'll fix-forward at tip, so the revert just needs to be done for 1.21. Fixes #62143 Change-Id: Id242230481ff4d4ba5f58236c6d8237729fc3b80 Reviewed-on: https://go-review.googlesource.com/c/go/+/520976 Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: Ruinan Sun <Ruinan.Sun@arm.com> Reviewed-by: Keith Randall <khr@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-08-23[release-branch.go1.21] path/filepath: don't drop .. elements when cleaning ↵Damien Neil
invalid Windows paths Fix a bug where Clean could improperly drop .. elements from a path on Windows, when the path contains elements containing a ':'. For example, Clean("a/../b:/../../c") now correctly returns "..\c" rather than "c". For #61866. Fixes #61868. Change-Id: I97b0238953c183b2ce19ca89c14f26700008ea72 Reviewed-on: https://go-review.googlesource.com/c/go/+/517216 Run-TryBot: Damien Neil <dneil@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Quim Muntal <quimmuntal@gmail.com> (cherry picked from commit 6e43407931ee4acc204620a9fae59c7903164901) Reviewed-on: https://go-review.googlesource.com/c/go/+/519655 Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2023-08-23[release-branch.go1.21] cmd/link: don't mangle string symbol namesCherry Mui
String symbol names could contain weird characters as we put the string literal into the symbol name. So it may appear to need mangling. However, as string symbols are grouped into a single "go:string.*" symbol, the individual symbol names actually don't matter. So don't mangle them. Also make the mangling code more defensive in case of weird symbol names. Updates #62098. Fixes #62140. Change-Id: I533012567a9fffab69debda934f426421c7abb04 Reviewed-on: https://go-review.googlesource.com/c/go/+/520856 Reviewed-by: Than McIntosh <thanm@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> (cherry picked from commit b65e34f03814889f0edd3ddd9778864762511443) Reviewed-on: https://go-review.googlesource.com/c/go/+/520857
2023-08-18[release-branch.go1.21] runtime/internal/wasitest: skip racy TCP echo testChris O'Hara
The wasip1 TCP echo test introduced in CL 493358 has a race condition with port selection. The test runner probes for a free port and then asks the WASM runtime to listen on the port, which may be taken by another process in the interim. Due to limitations with WASI preview 1, the guest is unable to query the port it's listening on. The test cannot ask the WASM runtime to listen on port 0 (choose a free port) since there's currently no way for the test to query the selected port and connect to it. Given the race condition is unavoidable, this test is now disabled by default and requires opt-in via an environment variable. This commit also eliminates the hard-coded connection timeout. Updates #61820. Fixes #61821. Change-Id: I375145c1a1d03ad45c44f528da3347397e6dcb01 Reviewed-on: https://go-review.googlesource.com/c/go/+/519895 Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> (cherry picked from commit 795e779610cd1ef8ad4c137c067a23ace1190115) Reviewed-on: https://go-review.googlesource.com/c/go/+/520955 Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
2023-08-17[release-branch.go1.21] runtime: mark traceEnabled and traceShuttingDown as ↵Mauri de Souza Meneguzzo
no:split This fixes a regression from CL 494181. The traceEnabled function splits the stack and is being called by reentersyscall that shouldn't call anything that splits the stack. Same with traceShuttingDown. For #61975 Fixes #61987 Change-Id: I5eca0ba74cfa6acb0259e8400b03c2093cd59dd1 GitHub-Last-Rev: 9e55ae9d7cc700de9757d32a7905127a349d973a GitHub-Pull-Request: golang/go#61981 Reviewed-on: https://go-review.googlesource.com/c/go/+/519055 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@google.com> (cherry picked from commit aa5d483f25e89f3bf399362ccc9a5b146cae5c79) Reviewed-on: https://go-review.googlesource.com/c/go/+/519495 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Austin Clements <austin@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-08-17[release-branch.go1.21] cmd/compile: ensure empty blocks in write barriers ↵Keith Randall
are marked unpreemptible Fixes #61958 Change-Id: I242ab77ad2f1ea1dad2d14ef756fa92f9378429f Reviewed-on: https://go-review.googlesource.com/c/go/+/518755 Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Reviewed-by: David Chase <drchase@google.com>
2023-08-17[release-branch.go1.21] runtime: profiling on Darwin cannot use blocking readsDavid Chase
On Darwin (and assume also on iOS but not sure), notetsleepg cannot be called in a signal-handling context. Avoid this by disabling block reads on Darwin. An alternate approach was to add "sigNote" with a pipe-based implementation on Darwin, but that ultimately would have required at least one more linkname between runtime and syscall to avoid racing with fork and opening the pipe, so, not. Fixes #62019. Updates #61768. Change-Id: I0e8dd4abf9a606a3ff73fc37c3bd75f55924e07e Reviewed-on: https://go-review.googlesource.com/c/go/+/518836 Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com> (cherry picked from commit c6ee8e31e3e7b01da54d241ae5a8eed350ab0e9f) Reviewed-on: https://go-review.googlesource.com/c/go/+/519375 Reviewed-by: Austin Clements <austin@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>