aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2023-01-10[dev.boringcrypto] misc/boring: add new releases to RELEASES filedev.boringcryptoCarlos Amedee
Change-Id: Idd94c25072d625c158553db2aecb3466c16d395b Reviewed-on: https://go-review.googlesource.com/c/go/+/461395 Reviewed-by: Heschi Kreinick <heschi@google.com> TryBot-Bypass: Carlos Amedee <carlos@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org> Auto-Submit: Carlos Amedee <carlos@golang.org>
2022-12-07[dev.boringcrypto] misc/boring: add new releases to RELEASES fileHeschi Kreinick
Change-Id: I75ba45bd555ee43dc4bf97593c88a54f518b4c4f Reviewed-on: https://go-review.googlesource.com/c/go/+/455817 TryBot-Bypass: Heschi Kreinick <heschi@google.com> Auto-Submit: Heschi Kreinick <heschi@google.com> Reviewed-by: Michael Pratt <mpratt@google.com>
2022-11-02[dev.boringcrypto] misc/boring: add new releases to RELEASES fileMatthew Dempsky
Change-Id: Ibea3a943a0834a16438a29c83bcd975240a77720 Reviewed-on: https://go-review.googlesource.com/c/go/+/446917 Auto-Submit: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-04[dev.boringcrypto] misc/boring: add new releases to RELEASES fileCarlos Amedee
Change-Id: I8e0e28ce26c43a6c1c55e97fcd28c5944044c400 Reviewed-on: https://go-review.googlesource.com/c/go/+/438738 Reviewed-by: Heschi Kreinick <heschi@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org> Run-TryBot: Carlos Amedee <carlos@golang.org>
2022-09-07[dev.boringcrypto] misc/boring: add new releases to RELEASES fileMichael Anthony Knyszek
Change-Id: I285b202b2688ecece7429838b11e47b95c120406 Reviewed-on: https://go-review.googlesource.com/c/go/+/429137 Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Carlos Amedee <carlos@golang.org>
2022-08-01[dev.boringcrypto] misc/boring: add new releases to RELEASES fileDmitri Shuralyov
Change-Id: I244091c2ea9f09701b83accfdab3180eaef88a8d Reviewed-on: https://go-review.googlesource.com/c/go/+/420596 Reviewed-by: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2022-07-13[dev.boringcrypto] misc/boring: add new releases to RELEASES fileMichael Anthony Knyszek
Change-Id: Ic6041c0928cc4c3c4c864b59e2297966da3b9f03 Reviewed-on: https://go-review.googlesource.com/c/go/+/417474 Reviewed-by: Heschi Kreinick <heschi@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-01[dev.boringcrypto] misc/boring: add new releases to RELEASES fileDmitri Shuralyov
Change-Id: I3569007abc12ca2bfcc3ec9376e112b8174f358f Reviewed-on: https://go-review.googlesource.com/c/go/+/409895 Reviewed-by: Alex Rakoczy <alex@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-05-10[dev.boringcrypto] misc/boring: add new releases to RELEASES fileHeschi Kreinick
Change-Id: Iae528a2e192a346bac74626c302fbaf681ab28d1 Reviewed-on: https://go-review.googlesource.com/c/go/+/405485 Auto-Submit: Heschi Kreinick <heschi@google.com> Reviewed-by: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Heschi Kreinick <heschi@google.com>
2022-05-03[dev.boringcrypto] README.boringcrypto.md: add note about the demise of the ↵Russ Cox
branch CL 403154 merged the branch back into the standard Go branch. Fixes #51940. Change-Id: Ibda89792c01630adce7ee3fbd56edbd780fcf79e Reviewed-on: https://go-review.googlesource.com/c/go/+/403615 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Russ Cox <rsc@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-04-29[dev.boringcrypto] cmd/compile: remove the awful boringcrypto kludgeRuss Cox
CL 60271 introduced this “AwfulBoringCryptoKludge.” iant approved that CL saying “As long as it stays out of master...” Now that the rsa and ecdsa code uses boring.Cache, the “boring unsafe.Pointer” fields are gone from the key structs, and this code is no longer needed. So delete it. With the kludge deleted, we are one step closer to being able to merge dev.boringcrypto into master. For #51940. Change-Id: Ie549db14b0b699c306dded2a2163f18f31d45530 Reviewed-on: https://go-review.googlesource.com/c/go/+/395884 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-29[dev.boringcrypto] crypto/ecdsa, crypto/rsa: use boring.CacheRuss Cox
In the original BoringCrypto port, ecdsa and rsa's public and private keys added a 'boring unsafe.Pointer' field to cache the BoringCrypto form of the key. This led to problems with code that “knew” the layout of those structs and in particular that they had no unexported fields. In response, as an awful kludge, I changed the compiler to pretend that field did not exist when laying out reflect data. Because we want to merge BoringCrypto in the main tree, we need a different solution. Using boring.Cache is that solution. For #51940. Change-Id: Ideb2b40b599a1dc223082eda35a5ea9abcc01e30 Reviewed-on: https://go-review.googlesource.com/c/go/+/395883 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-04-29[dev.boringcrypto] crypto/internal/boring: add GC-aware cacheRuss Cox
In the original BoringCrypto port, ecdsa and rsa's public and private keys added a 'boring unsafe.Pointer' field to cache the BoringCrypto form of the key. This led to problems with code that “knew” the layout of those structs and in particular that they had no unexported fields. In response, as an awful kludge, I changed the compiler to pretend that field did not exist when laying out reflect data. Because we want to merge BoringCrypto in the main tree, we need a different solution. The different solution is this CL's boring.Cache, which is a concurrent, GC-aware map from unsafe.Pointer to unsafe.Pointer (if generics were farther along we could use them nicely here, but I am afraid of breaking tools that aren't ready to see generics in the standard library yet). More complex approaches are possible, but a simple, fixed-size hash table is easy to make concurrent and should be fine. For #51940. Change-Id: I44062a8defbd87b705a787cffc64c6a9d0132785 Reviewed-on: https://go-review.googlesource.com/c/go/+/395882 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-04-29[dev.boringcrypto] crypto/x509: remove VerifyOptions.IsBoringRuss Cox
This API was added only for BoringCrypto, never shipped in standard Go. This API is also not compatible with the expected future evolution of crypto/x509, as we move closer to host verifiers on macOS and Windows. If we want to merge BoringCrypto into the main tree, it is best not to have differing API. So instead of a hook set by crypto/tls, move the actual check directly into crypto/x509, eliminating the need for exposed API. For #51940. Change-Id: Ia2ae98c745de818d39501777014ea8166cab0b03 Reviewed-on: https://go-review.googlesource.com/c/go/+/395878 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-04-29[dev.boringcrypto] crypto/..., go/build: align deps test with standard rulesRuss Cox
One annoying difference between dev.boringcrypto and master is that there is not a clear separation between low-level (math/big-free) crypto and high-level crypto, because crypto/internal/boring imports both encoding/asn1 and math/big. This CL removes both those problematic imports and aligns the dependency rules in the go/build test with the ones in the main branch. To remove encoding/asn1, the crypto/internal/boring APIs change to accepting and returning encoded ASN.1, leaving crypto/ecdsa to do the marshaling and unmarshaling, which it already contains code to do. To remove math/big, the crypto/internal/boring package defines type BigInt []uint, which is the same representation as a big.Int's internal storage. The new package crypto/internal/boring/bbig provides conversions between BigInt and *big.Int. The boring package can then be in the low-level crypto set, and any package needing to use bignum APIs (necessarily in the high-level crypto set) can import bbig to convert. To simplify everything we hide from the test the fact that crypto/internal/boring imports cgo. Better to pretend it doesn't and keep the prohibitions that other packages like crypto/aes must not use cgo (outside of BoringCrypto). $ git diff origin/master src/go/build/deps_test.go diff --git a/src/go/build/deps_test.go b/src/go/build/deps_test.go index 6ce872e297..a63979cc93 100644 --- a/src/go/build/deps_test.go +++ b/src/go/build/deps_test.go @@ -402,9 +402,13 @@ var depsRules = ` NET, log < net/mail; + NONE < crypto/internal/boring/sig; + sync/atomic < crypto/internal/boring/fipstls; + crypto/internal/boring/sig, crypto/internal/boring/fipstls < crypto/tls/fipsonly; + # CRYPTO is core crypto algorithms - no cgo, fmt, net. # Unfortunately, stuck with reflect via encoding/binary. - encoding/binary, golang.org/x/sys/cpu, hash + crypto/internal/boring/sig, encoding/binary, golang.org/x/sys/cpu, hash < crypto < crypto/subtle < crypto/internal/subtle @@ -413,6 +417,8 @@ var depsRules = ` < crypto/ed25519/internal/edwards25519/field, golang.org/x/crypto/curve25519/internal/field < crypto/ed25519/internal/edwards25519 < crypto/cipher + < crypto/internal/boring + < crypto/boring < crypto/aes, crypto/des, crypto/hmac, crypto/md5, crypto/rc4, crypto/sha1, crypto/sha256, crypto/sha512 < CRYPTO; @@ -421,6 +427,7 @@ var depsRules = ` # CRYPTO-MATH is core bignum-based crypto - no cgo, net; fmt now ok. CRYPTO, FMT, math/big, embed + < crypto/internal/boring/bbig < crypto/rand < crypto/internal/randutil < crypto/ed25519 @@ -443,7 +450,8 @@ var depsRules = ` < golang.org/x/crypto/hkdf < crypto/x509/internal/macos < crypto/x509/pkix - < crypto/x509 + < crypto/x509; + crypto/internal/boring/fipstls, crypto/x509 < crypto/tls; # crypto-aware packages @@ -653,6 +661,9 @@ func findImports(pkg string) ([]string, error) { } var imports []string var haveImport = map[string]bool{} + if pkg == "crypto/internal/boring" { + haveImport["C"] = true // kludge: prevent C from appearing in crypto/internal/boring imports + } fset := token.NewFileSet() for _, file := range files { name := file.Name() For #51940. Change-Id: I26fc752484310d77d22adb06495120a361568d04 Reviewed-on: https://go-review.googlesource.com/c/go/+/395877 TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org> Run-TryBot: Russ Cox <rsc@golang.org>
2022-04-29[dev.boringcrypto] crypto/internal/boring: make SHA calls allocation-freeRuss Cox
The standard Go implementations are allocation-free. Making the BoringCrypto ones the same helps avoid surprises, including in some of our own tests. For #51940. Change-Id: Ic9c5dc46f5e29ca85f571244be2b380ec2cf89c9 Reviewed-on: https://go-review.googlesource.com/c/go/+/395876 Reviewed-by: Roland Shoemaker <roland@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29[dev.boringcrypto] crypto/internal/boring: avoid allocation in big.Int ↵Russ Cox
conversion The conversion via byte slices is inefficient; we can convert via word slices and avoid the copy entirely. For #51940. Change-Id: I06f747e0acffffae427d9706d43bdacf146c027d Reviewed-on: https://go-review.googlesource.com/c/go/+/395875 Reviewed-by: Roland Shoemaker <roland@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29[dev.boringcrypto] cmd/dist: default to use of boringcryptoRuss Cox
The dev.boringcrypto branch has historically forced use of boringcrypto with no additional configuration flags. The previous CL undid that. This CL redoes it, so that direct uses of dev.boringcrypto don't lapse unexpectedly into not having boringcrypto enabled. When dev.boringcrypto is merged into master, we will undo this change as part of the merge, so that the only final difference between master and dev.boringcrypto will be this CL. For #51940. Change-Id: I816593a0b30b4e71093a7da9451bae7807d7167e Reviewed-on: https://go-review.googlesource.com/c/go/+/402597 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29[dev.boringcrypto] all: add boringcrypto build tagsRuss Cox
A plain make.bash in this tree will produce a working, standard Go toolchain, not a BoringCrypto-enabled one. The BoringCrypto-enabled one will be created with: GOEXPERIMENT=boringcrypto ./make.bash For #51940. Change-Id: Ia9102ed993242eb1cb7f9b93eca97e81986a27b3 Reviewed-on: https://go-review.googlesource.com/c/go/+/395881 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-04-29[dev.boringcrypto] cmd/go: pass dependency syso to cgo tooRuss Cox
Proposal #42477 asked for a way to apply conditional build tags to syso files (which have no source code to hold //go:build lines). We ended up suggesting that the standard answer should be to put the syso in its own package and then import that package from a source file that is itself conditionally compiled. A followup comment on that issue pointed out a problem that I did not understand until I tried to use this approach myself: the cgo build fails by default, because the link step only uses syso files from the current package. You have to override this explicitly by arranging to pass a “ignore unresolved symbols” flag to the host linker. Many users will not know how to do this. (I don't know how to do this off the top of my head.) If we want users to use this approach, we should make it work better. This CL does that, by including the syso files from dependencies of the current package in the link step. For #51940. Change-Id: I53a0371b2df17e39a000a645b7686daa6a98722d Reviewed-on: https://go-review.googlesource.com/c/go/+/402596 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29[dev.boringcrypto] cmd: use notsha256 instead of md5, sha1, sha256Russ Cox
When we add GOEXPERIMENT=boringcrypto, the bootstrap process will not converge if the compiler itself depends on the boringcrypto cgo-based implementations of sha1 and sha256. Using notsha256 avoids boringcrypto and makes bootstrap converge. Removing md5 is not strictly necessary but it seemed worthwhile to be consistent. For #51940. Change-Id: Iba649507e0964d1a49a1d16e463dd23c4e348f14 Reviewed-on: https://go-review.googlesource.com/c/go/+/402595 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29[dev.boringcrypto] cmd/internal/notsha256: add new packageRuss Cox
Package notsha256 implements the NOTSHA256 hash, defined as bitwise NOT of SHA-256. It will be used from the Go compiler toolchain where an arbitrary hash is needed and the code currently reaches for MD5, SHA1, or SHA256. The problem with all of those is that when we add GOEXPERIMENT=boringcrypto, the bootstrap process will not converge if the compiler itself depends on the boringcrypto cgo code. Using notsha256 avoids boringcrypto. It is possible that I don't fully understand the convergence problem and that there is a way to make the compiler converge when using cgo, but keeping cgo out of the compiler seems safest. It also makes clear that (except for the hack in codesign) the code using this package doesn't care which hash is used. For #51940. Change-Id: Ie7c661183eacf8413a9d2074c96cbb9361e125ef Reviewed-on: https://go-review.googlesource.com/c/go/+/402594 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-27[dev.boringcrypto] all: merge master into dev.boringcryptoChressie Himpel
Change-Id: Ic5f71c04f08c03319c043f35be501875adb0a3b0
2022-04-27crypto/tls: remove tls10default GODEBUG flagFilippo Valsorda
Updates #45428 Change-Id: Ic2ff459e6a3f1e8ded2a770c11d34067c0b39a8a Reviewed-on: https://go-review.googlesource.com/c/go/+/400974 Reviewed-by: Filippo Valsorda <valsorda@google.com> Auto-Submit: Filippo Valsorda <valsorda@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Filippo Valsorda <valsorda@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-04-27crypto/elliptic: split up P-256 field and group opsFilippo Valsorda
This makes Gerrit recognize the rename of the field implementation and facilitates the review. No code changes. For #52182 Change-Id: I827004e175db1ae2fcdf17d0f586ff21503d27e3 Reviewed-on: https://go-review.googlesource.com/c/go/+/390754 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org> Run-TryBot: Filippo Valsorda <filippo@golang.org> Auto-Submit: Filippo Valsorda <filippo@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-27crypto/elliptic: inline marshaling into nistec pointFromAffineFilippo Valsorda
Marshal behavior for invalid points is undefined, so don't use it to check if points are valid. For #52182 Change-Id: If167893bc4b029f71bb2528564f2bd96bee7221c Reviewed-on: https://go-review.googlesource.com/c/go/+/382994 Run-TryBot: Filippo Valsorda <filippo@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-04-27crypto/elliptic: use generics for nistec-based curvesFilippo Valsorda
There was no way to use an interface because the methods on the Point types return concrete Point values, as they should. A couple somewhat minor annoyances: - Allocations went up due to #48849. This is fine here, where math/big causes allocations anyway, but would probably not be fine in nistec itself. - Carrying the newPoint/newGenerator functions around as a field is a little weird, even if type-safe. It also means we have to make what were functions methods so they can access newPoint to return the zero value. This is #35966. For #52182 Change-Id: I050f3a27f15d3f189818da80da9de0cba0548931 Reviewed-on: https://go-review.googlesource.com/c/go/+/360015 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2022-04-27crypto/elliptic: refactor package structureFilippo Valsorda
Not quite golang.org/wiki/TargetSpecific compliant, but almost. The only substantial code change is in randFieldElement: it used to use Params().BitSize instead of Params().N.BitLen(), which is semantically incorrect, even if the two values are the same for all named curves. For #52182 Change-Id: Ibc47450552afe23ea74fcf55d1d799d5d7e5487c Reviewed-on: https://go-review.googlesource.com/c/go/+/315273 Run-TryBot: Filippo Valsorda <filippo@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Reviewed-by: Roland Shoemaker <roland@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2022-04-27time: document hhmmss formatscuiweixie
Fixes #52516 Change-Id: I173fdb09c245563e09be4e1aacfd374c3a764d74 GitHub-Last-Rev: 14a81e50616e0f268fee9323d0621de861885475 GitHub-Pull-Request: golang/go#52538 Reviewed-on: https://go-review.googlesource.com/c/go/+/402058 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-27runtime: disable windowed Smhasher test on 32-bit systemsKeith Randall
This test tends to be flaky on 32-bit systems. There's not enough bits in the hash output, so we expect a nontrivial number of collisions, and it is often quite a bit higher than expected. Fixes #43130 Change-Id: If35413b7c45eed778a08b834dacf98009ceca840 Reviewed-on: https://go-review.googlesource.com/c/go/+/402456 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Keith Randall <khr@google.com>
2022-04-26runtime: refactor the scavenger and make it testableMichael Anthony Knyszek
This change refactors the scavenger into a type whose methods represent the actual function and scheduling of the scavenger. It also stubs out access to global state in order to make it testable. This change thus also adds a test for the scavenger. In writing this test, I discovered the lack of a behavior I expected: if the pageAlloc.scavenge returns < the bytes requested scavenged, that means the heap is exhausted. This has been true this whole time, but was not documented or explicitly relied upon. This change rectifies that. In theory this means the scavenger could spin in run() indefinitely (as happened in the test) if shouldStop never told it to stop. In practice, shouldStop fires long before the heap is exhausted, but for future changes it may be important. At the very least it's good to be intentional about these things. While we're here, I also moved the call to stopTimer out of wake and into sleep. There's no reason to add more operations to a context that's already precarious (running without a P on sysmon). Change-Id: Ib31b86379fd9df84f25ae282734437afc540da5c Reviewed-on: https://go-review.googlesource.com/c/go/+/384734 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26runtime: disable idle mark workers with at least one dedicated workerMichael Anthony Knyszek
This change completes the proposal laid out in #44163. With #44313 resolved, we now ensure that stopped Ms are able to wake up and become dedicated GC workers. As a result, idle GC workers are in theory no longer required to be a proxy for scheduling dedicated mark workers. And, with at least one dedicated mark worker running (which is non-preemptible) we ensure the GC makes progress in all circumstances when at least one is running. Currently we ensure at least one idle mark worker is available at all times because it's possible before #44313 that a dedicated worker doesn't ever get scheduled, leading to a deadlock if user goroutines block on a GC completing. But now that extra idle mark worker should be unnecessary to ensure GC progress when at least one dedicated mark worker is going to be scheduled. Fixes #44163. Change-Id: I62889ef2db4e69d44da883e8e6eebcfe5398c86d Reviewed-on: https://go-review.googlesource.com/c/go/+/395634 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26runtime: move scheduling decisions by schedule into findrunnableMichael Anthony Knyszek
This change moves several scheduling decisions made by schedule into findrunnable. The main motivation behind this change is the fact that stopped Ms can't become dedicated or fractional GC workers. The main reason for this is that when a stopped M wakes up, it stays in findrunnable until it finds work, which means it will never consider GC work. On that note, it'll also never consider becoming the trace reader, either. Another way of looking at it is that this change tries to make findrunnable aware of more sources of work than it was before. With this change, any M in findrunnable should be capable of becoming a GC worker, resolving #44313. While we're here, let's also make more sources of work, such as the trace reader, visible to handoffp, which should really be checking all sources of work. With that, we also now correctly handle the case where StopTrace is called from the last live M that is also locked (#39004). stoplockedm calls handoffp to start a new M and handle the work it cannot, and once we include the trace reader in that, we ensure that the trace reader gets scheduled. This change attempts to preserve the exact same ordering of work checking to reduce its impact. One consequence of this change is that upon entering schedule, some sources of work won't be checked twice (i.e. the local and global runqs, and timers) as they do now, which in some sense gives them a lower priority than they had before. Fixes #39004. Fixes #44313. Change-Id: I5d8b7f63839db8d9a3e47cdda604baac1fe615ce Reviewed-on: https://go-review.googlesource.com/c/go/+/393880 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26runtime: reduce max idle mark workers during periodic GC cyclesMichael Anthony Knyszek
This change reduces the maximum number of idle mark workers during periodic (currently every 2 minutes) GC cycles to 1. Idle mark workers soak up all available and unused Ps, up to GOMAXPROCS. While this provides some throughput and latency benefit in general, it can cause what appear to be massive CPU utilization spikes in otherwise idle applications. This is mostly an issue for *very* idle applications, ones idle enough to trigger periodic GC cycles. This spike also tends to interact poorly with auto-scaling systems, as the system might assume the load average is very low and suddenly see a massive burst in activity. The result of this change is not to bring down this 100% (of GOMAXPROCS) CPU utilization spike to 0%, but rather min(25% + 1/GOMAXPROCS*100%, 100%) Idle mark workers also do incur a small latency penalty as they must be descheduled for other work that might pop up. Luckily the runtime is pretty good about getting idle mark workers off of Ps, so in general the latency benefit from shorter GC cycles outweighs this cost. But, the cost is still non-zero and may be more significant in idle applications that aren't invoking assists and write barriers quite as often. We can't completely eliminate idle mark workers because they're currently necessary for GC progress in some circumstances. Namely, they're critical for progress when all we have is fractional workers. If a fractional worker meets its quota, and all user goroutines are blocked directly or indirectly on a GC cycle (via runtime.GOMAXPROCS, or runtime.GC), the program may deadlock without GC workers, since the fractional worker will go to sleep with nothing to wake it. Fixes #37116. For #44163. Change-Id: Ib74793bb6b88d1765c52d445831310b0d11ef423 Reviewed-on: https://go-review.googlesource.com/c/go/+/393394 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26runtime: yield instead of sleeping in runqgrab on OpenBSDMichael Anthony Knyszek
OpenBSD has a coarse sleep granularity that rounds up to 10 ms increments. This can cause significant STW delays, among other issues. As far as I can tell, there's only 1 tightly timed sleep without an explicit wakeup for which this actually matters. Fixes #52475. Change-Id: Ic69fc11096ddbbafd79b2dcdf3f912fde242db24 Reviewed-on: https://go-review.googlesource.com/c/go/+/401638 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26runtime: make alloc count metrics truly monotonicMichael Anthony Knyszek
Right now we export alloc count metrics via the runtime/metrics package and mark them as monotonic, but that's not actually true. As an optimization, the runtime assumes a span is always fully allocated before being uncached, and updates the accounting as such. In the rare case that it's wrong, the span has enough information to back out what did not get allocated. This change uses 16 bits of padding in the mspan to house another field that represents the amount of mspan slots filled just as the mspan is cached. This is information is enough to get an exact count, allowing us to make the metrics truly monotonic. Change-Id: Iaff3ca43f8745dc1bbb0232372423e014b89b920 Reviewed-on: https://go-review.googlesource.com/c/go/+/377516 Reviewed-by: Michael Pratt <mpratt@google.com> Run-TryBot: Michael Knyszek <mknyszek@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26SECURITY.md: replace golang.org with go.devHana
Change-Id: Ic0e882fc6666c9adcd5f2dffc96e201f3146fa0f Reviewed-on: https://go-review.googlesource.com/c/go/+/402180 Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-26runtime: use ABIInternal for most calls to sigtrampgoMichael Pratt
sigtramp on openbsd-arm64 is teetering on the edge of the nosplit stack limit. Add more headroom by calling sigtrampgo using ABIInternal, which eliminates a 48-byte ABI wrapper frame. openbsd-amd64 has slightly more space, but is also close to the limit, so convert it as well. Other operating systems don't have it as bad, but many have nearly identical implementations of sigtramp, so I have converted them as well. I've omitted darwin-arm64 and solaris, as those are quite different and would benefit from not needing ifdef for both cases. For #51485. Change-Id: I70512645d4208b346a59d5e5d03836a45833b1d7 Reviewed-on: https://go-review.googlesource.com/c/go/+/390814 Run-TryBot: Michael Pratt <mpratt@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-04-26net: skip TestDialCancel on darwin-arm64Heschi Kreinick
We're turning up Macs in a network environment that clashes with this test. I don't think it's critical to get it working, so skip it. For #49149. Change-Id: I925e3ecc5356c4cefd208bdcff3d98021215d0b4 Reviewed-on: https://go-review.googlesource.com/c/go/+/402181 Reviewed-by: Alex Rakoczy <alex@golang.org> Run-TryBot: Heschi Kreinick <heschi@google.com> Auto-Submit: Heschi Kreinick <heschi@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26README.md: update wiki linkHana
Change-Id: I307c3524f2031e2a3e7ada6e86c73e278481de6d Reviewed-on: https://go-review.googlesource.com/c/go/+/402179 Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2022-04-26os/exec: use a TestMain to avoid hijacking stdout for helper commandsBryan C. Mills
The previous implementation of helperCommand relied on running a well-known Test function which implemented all known commands. That not only added Skip noise in the test's output, but also (and more importantly) meant that the commands could not write directly to stdout in the usual way, since the testing package hijacks os.Stdout for its own use. The new implementation addresses the above issues, and also ensures that all registered commands are actually used, reducing the risk of an unused command sticking around after refactoring. It also sets the subprocess environment variable directly in the test process, instead of on each individual helper command's Env field, allowing helper commands to be used without an explicit Env. Updates #50599. (Also for #50436.) Change-Id: I189c7bed9a07cfe47a084b657b88575b1ee370b9 Reviewed-on: https://go-review.googlesource.com/c/go/+/401934 Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-26os/exec: make skipStdinCopyError a function instead of a variableBryan C. Mills
This makes clearer that skipStdinCopyError is always defined and never overridden in tests. Secondarily, it may also help reduce init-time work and allow the linker and/or inliner to better optimize this package. (Noticed while prototyping #50436.) Change-Id: I4f3c1bc146384a98136a4039f82165ed106c14b8 Reviewed-on: https://go-review.googlesource.com/c/go/+/401897 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-26time: fix quickcheck test to avoid wraparoundsRuss Cox
When we call time.Unix(s, ns), the internal representation is s + 62135596800, where 62135596800 is the number of seconds from Jan 1 1 to Jan 1 1970. If quickcheck generates numbers too close to 2^63, the addition can wraparound to make a very negative internal 64-bit value. Wraparounds are not guarded against, since they would not arise in any reasonable program, so just avoid testing near them. Fixes #52409. Change-Id: Id466c8a34a49055ab26f2687a6b2b657cb64bed6 Reviewed-on: https://go-review.googlesource.com/c/go/+/402177 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26cmd/compile/internal/syntax: parser to accept ~x as unary expressionRobert Griesemer
Accept ~x as ordinary unary expression in the parser but recognize such expressions as invalid in the type checker. This change opens the door to recognizing complex type constraint literals such as `*E|~int` in `[P *E|~int]` and parse them correctly instead of reporting a parse error because `P*E|~int` syntactically looks like an incorrect array length expression (binary expression where the RHS of | is an invalid unary expression ~int). As a result, the parser is more forgiving with expressions but the type checker will reject invalid uses as before. We could pass extra information into the binary/unary expression parse functions to prevent the use of ~ in invalid situations but it doesn't seem worth the trouble. In fact it may be advantageous to allow a more liberal expression syntax especially in the presence of errors (better parser synchronization after an error). Preparation for fixing #49482. Change-Id: I119e8bd9445dfa6460fcd7e0658e3554a34b2769 Reviewed-on: https://go-review.googlesource.com/c/go/+/402255 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Robert Griesemer <gri@google.com> Auto-Submit: Robert Griesemer <gri@google.com>
2022-04-25cmd/compile: fix the missing size for FuncInfoSymzhouguangyuan
Change-Id: I46543e188bf25384e529a9d5a3095033ac618bbd Reviewed-on: https://go-review.googlesource.com/c/go/+/402057 Run-TryBot: Cherry Mui <cherryyz@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-04-25net/http: add MaxBytesErrorCarl Johnson
Fixes #30715 Change-Id: Ia3712d248b6dc86abef71ccea6e705a571933d53 GitHub-Last-Rev: 6ae68402a5a7c57f7f18e945d48c69ba2b134078 GitHub-Pull-Request: golang/go#49359 Reviewed-on: https://go-review.googlesource.com/c/go/+/361397 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com> Reviewed-by: Damien Neil <dneil@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-25reflect: support Len and Cap on pointer-to-array Valuecuiweixie
Fixes #52411 Change-Id: I2fd13a453622992c52d49aade7cd058cfc8a77ca GitHub-Last-Rev: d5987c2ec817ebd01d9e1adb3bd2e75274dbbabd GitHub-Pull-Request: golang/go#52423 Reviewed-on: https://go-review.googlesource.com/c/go/+/400954 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Keith Randall <khr@google.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-04-25os: skip TestRemoveAllRace on dragonflyBryan C. Mills
This test occasionally fails on the dragonfly-amd64 builder with "directory not empty". Since that is the only platform on which we observe these failures, and since the test had a different (and also invalid-looking) failure mode prior to this one (in #50716), we suspect that it is due to either a bug in the platform or a platform-specific Go bug. For #52301. Change-Id: Id36c499651b9c48e6b8b0107d01f73d2a7b6bab8 Reviewed-on: https://go-review.googlesource.com/c/go/+/402155 Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-25os/exec: in TestImplicitPWD, explicitly request the logical pathBryan C. Mills
Fixes #52537 Change-Id: I70959881a31f425e940e7adf86b36be2596aafb7 Reviewed-on: https://go-review.googlesource.com/c/go/+/402158 Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-25slices: use !{{Less}} instead of {{GreaterOrEqual}}张云浩
In CL 371574 PatchSet 18, we replaced all !{{Less}} with {{GreaterOrEqual}} to fix a problem(handle NaNs when sorting float64 slice) in exp/slices. We don't actually need this change, because we don't guarantee that the slice will be sorted eventually if there are NaNs(we could have a[i] < a[j] for some i,j with i>j). This CL reverts all the replacements in exp/slices and does not affect any codes in the sort package. Change-Id: Idc225d480de3e2efef2add35c709ed880d1306cb Reviewed-on: https://go-review.googlesource.com/c/go/+/400534 Reviewed-by: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> Auto-Submit: Keith Randall <khr@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Eli Bendersky <eliben@google.com> Reviewed-by: Keith Randall <khr@google.com> Auto-Submit: Keith Randall <khr@google.com>