aboutsummaryrefslogtreecommitdiff
path: root/src/cmd/go/script_test.go
AgeCommit message (Collapse)Author
2024-05-03cmd/go: remove checks that all counters incremented are in counters.txtMichael Matloob
This change removes cmd/go/testdata/counters.txt. It also removes the code that prepares it and checks that it contains all registered counters as well as counters for all flags and subcommands. It removes the counter registration mechanism, and uses telemetry.NewCounter to create new counters instead. It keeps the tests that check that at least one counter is incremented if the go command is invoked in a script test. Change-Id: Ic6bda5c64e90f0dd7e221968fce0e375e84d6e17 Cq-Include-Trybots: luci.golang.try:gotip-windows-amd64-longtest,gotip-linux-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/582715 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
2024-03-26all: fix a large number of commentscui fliter
Partial typo corrections, following https://go.dev/wiki/Spelling Change-Id: I2357906ff2ea04305c6357418e4e9556e20375d1 Reviewed-on: https://go-review.googlesource.com/c/go/+/573776 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Than McIntosh <thanm@google.com> Run-TryBot: shuang cui <imcusg@gmail.com> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-03-14cmd/internal/telemetry: add a shim package around telemetryMichael Matloob
The purpose of this package is to have a build tagged variant so that when we're building the bootstrap go command it does not depend on the net package. (net is a dependency of golang.org/x/telemetry/counter on Windows). The TESTGO_TELEMETRY_DIR environment variable used by the go tests to change the telemetry directory is renamed to TEST_TELEMETRY_DIR to make it more general to other commands that might want to set it for the purpose of tests. The test telemetry directory is now set using telemetry.Start instead of countertest.Open. This also means that the logic that decides whether to upload counter files is now going to run from the cmd/go tests (but that's okay because it's aleady been running when cmd/go has been invoked outside of its tests. Change-Id: Ic4272e5083facde010482d8b8fc3c95c03564bc9 Reviewed-on: https://go-review.googlesource.com/c/go/+/571096 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2024-03-09cmd/go: check checkCounters counter read only on supported platformsHana (Hyang-Ah) Kim
Telemetry counters writing is disabled on certain platforms. See x/telemetry/internal/telemetry.DisabledOnPlatform. For #66205 Change-Id: I833e15ae33fb27e09d67fc77b921498476237176 Reviewed-on: https://go-review.googlesource.com/c/go/+/570196 Reviewed-by: Michael Matloob <matloob@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-07cmd/go: test that each script test increments at least one counterMichael Matloob
Add code that will set a scriptGoInvoked bit for the testing.TB when it invokes the go command. If the go command was invoked, make sure that at least one counter was incremented. Also add the counters cmd/go/gomodcache-entry-relative, cmd/go/gopath-entry-relative, and cmd/go/invalid-toolchain-in-file so we can increment counters when a test errors out before the flag subcommand counters are processed. This enforces the invariant that at least one counter is incremented by every test that invokes the go command. Add the counter cmd/go/exec-go-toolchain for when a toolchain switch happens. Add cmd/go/subcommand:help for invoking help without arguments and cmd/go/help-unknown-topic for when an unknown command is provided to help. Change-Id: Id90f2bbe4c7e89b846da00ec1ed9595ece2b269c Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/568259 Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-03-04cmd/go: add file with list of all counters we collectMichael Matloob
Maintain a list of counters we collect and test that it hasn't changed. If it has, fail a test and have the user update the list. The update process will print a reminder to update the list of collected counters. Also run go mod vendor to pull in golang.org/x/telemetry/counter/countertest. For #58894 Change-Id: I661a9c3d67cb33f42a5519f4639af7aa05c3821d Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/564555 LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Robert Findley <rfindley@google.com>
2024-02-21cmd: remove support for GOROOT_FINALConstantin Konstantinidis
Fixes #62047 Change-Id: If7811c1eb9073fb09b7006076998f8b2e1810bfb Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest,gotip-windows-amd64-longtest Reviewed-on: https://go-review.googlesource.com/c/go/+/539975 Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-06-14cmd/go: use gover.Local for $goversion in TestScriptBryan C. Mills
Previously we used the highest Go build tag found in the build configuration, which matches gover.Local for development toolchains (it is always a bare language version), but is too low for releases. Updates #57001. Change-Id: I74c2f7ab06231858eee99ecd11ed3759853e01ed Reviewed-on: https://go-review.googlesource.com/c/go/+/503537 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2023-06-09cmd/go: fix TestScript/gotoolchain* when go.env doesn't set GOTOOLCHAIN=autoqmuntal
Some of the TestScript/gotoolchain* tests assume that go.env contains GOTOOLCHAIN=auto, but that's not always the case, for example CI environments may set it to `local` to avoid downloading a new toolchain. This commit fixes the tests to work regardless of the value of GOTOOLCHAIN in go.env. Fixes #60685 Change-Id: Ieda22574f8a028893762274cf9db721c9d69bf7e Reviewed-on: https://go-review.googlesource.com/c/go/+/502035 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Quim Muntal <quimmuntal@gmail.com> Reviewed-by: Bryan Mills <bcmills@google.com> Reviewed-by: Eli Bendersky <eliben@google.com> Auto-Submit: Bryan Mills <bcmills@google.com>
2023-06-06cmd/go: disallow package directories containing newlinesBryan C. Mills
Directory or file paths containing newlines may cause tools (such as cmd/cgo) that emit "//line" or "#line" -directives to write part of the path into non-comment lines in generated source code. If those lines contain valid Go code, it may be injected into the resulting binary. (Note that Go import paths and file paths within module zip files already could not contain newlines.) Thanks to Juho Nurminen of Mattermost for reporting this issue. Fixes #60167. Fixes CVE-2023-29402. Change-Id: I64572e9f454bce7b685d00e2e6a1c96cd33d53df Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1882606 Reviewed-by: Roland Shoemaker <bracewell@google.com> Run-TryBot: Roland Shoemaker <bracewell@google.com> Reviewed-by: Russ Cox <rsc@google.com> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/501226 Run-TryBot: David Chase <drchase@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-05-25cmd/go: show full path name (testdata/script/foo.txt) in script errorsRuss Cox
Showing the full path (relative to the current directory) instead of just foo.txt lets editors that understand file:line jump straight to the file without having to edit it to say testdata/script/ first. Change-Id: I44177b687249f3c7c724b45d02f5167607369e1b Reviewed-on: https://go-review.googlesource.com/c/go/+/498119 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@google.com> Auto-Submit: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-25cmd/go: clear HGRCPATH in script testRuss Cox
Invoking 'hg' in a nonexistant HOME can break extensions the user may have installed; clear HGRCPATH in the script test to keep tests working in that environment. Change-Id: I4d21d024c6229ead38e5f24186883863511fd483 Reviewed-on: https://go-review.googlesource.com/c/go/+/497878 Reviewed-by: Bryan Mills <bcmills@google.com> Auto-Submit: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Russ Cox <rsc@golang.org>
2023-05-19cmd/go: add a GODEBUG to limit the number of concurrent network connectionsBryan C. Mills
I implemented this in order to debug connection failures on a new-to-me VM development environment that uses Cloud NAT. It doesn't directly fix the bug, but perhaps folks will find it useful to diagnose port-exhaustion-related flakiness in other environments. For #52545. Change-Id: Icd3f13dcf62e718560c4f4a965a4df7c1bd785ce Reviewed-on: https://go-review.googlesource.com/c/go/+/473277 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com>
2023-04-24cmd/go: declare net hosts in script testsBryan C. Mills
Although we aren't precise about enforcing the hosts just yet, we can eventually use the declared hostnames to selectively skip tests (for example, if an external service has an outage while a Go release is being tested). Also relax the constraint to [short] in tests that require only vcs-test.golang.org, which has redirected to an in-process server since around CL 427914. Also enforce that tests that use the network actually use the [net] constraint, by setting TESTGONETWORK=panic in the test environment until the condition is evaluated. For #52545. For #54503. Updates #27494. Change-Id: I13be6b42a9beee97657eb45424882e787ac164c3 Reviewed-on: https://go-review.googlesource.com/c/go/+/473276 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Bypass: Bryan Mills <bcmills@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com>
2023-01-13Revert "internal/fsys: follow root symlink in fsys.Walk"Bryan Mills
This reverts CL 448360 and adds a regression test for #57754. Reason for revert: broke 'go list' in Debian's distribution of the Go toolchain Fixes #57754. Updates #50807. Change-Id: I3e8b9126294c55f21572774b549fb28f29eded0f Reviewed-on: https://go-review.googlesource.com/c/go/+/461959 Reviewed-by: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com>
2022-10-25cmd/go: reroute vcs-test.golang.org HTTPS requests to the test-local serverBryan C. Mills
After this CL, the only test requests that should still reach vcs-test.golang.org are for Subversion repos, which are not yet handled. The interceptor implementation should also allow us to redirect other servers (such as gopkg.in) fairly easily in a followup change if desired. For #27494. Change-Id: I8cb85f3a7edbbf0492662ff5cfa779fb9b407136 Reviewed-on: https://go-review.googlesource.com/c/go/+/427254 Auto-Submit: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Russ Cox <rsc@golang.org>
2022-10-25cmd/go: redirect vcs-test.golang.org repo URLs to a test-local serverBryan C. Mills
The new server reconstructs the vcs-test repos on the fly using scripts that run the actual version-control binaries. This allows those repos to be code-reviewed using our normal tools — and, crucially, allows contributors to add new vcs-test contents as part of a contributed CL. It also prevents failures due to network errors reaching vcs-test.golang.org (such as when developing offline), and allows us to iterate on the repo contents without dealing with annoying and unpredictable GCS caching behavior. We can't quite turn down vcs-test.golang.org yet — this server doesn't yet handle "go-import" metadata (and related authentication behaviors), and doesn't serve Subversion repos. But we're getting much closer! For #27494. Change-Id: I233fc718617aed287b0f7248bd8cfe1e5cebe96b Reviewed-on: https://go-review.googlesource.com/c/go/+/421455 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2022-10-24cmd/go/internal/script: remove special-case escaping logic for $WORKBryan C. Mills
Previously, the script engine implicitly escaped the path in the $WORK environment variable to be the literal string '$WORK', which produces somewhat better error messages in case of failure. However, for a general-purpose script engine that implicit behavior is surprising, and it isn't really necessary. For #27494. Change-Id: Ic1d5b8801bbd068157315685539e7cc2795b3aa5 Reviewed-on: https://go-review.googlesource.com/c/go/+/426854 Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com>
2022-10-24cmd/go: extract the TestScript engine into a standalone packageBryan C. Mills
This change decouples the script engine from both cmd/go and the testing package; I intend to reuse it in the replacement for the vcs-test.golang.org server. This change also adds a few new script commands: - 'echo' echoes its arguments, useful for verifying argument expansion. - 'cat' prints the contents of files, useful for debugging failing script tests. - 'help' displays information about script commands and conditions, reducing the toil of maintaining lists in the README file. The 'cmp' and 'cmpenv' commands now use internal/diff instead of their own separate diff implementation. The 'env' command now writes to the script log instead of the stdout buffer. (This makes it more consistent with the behavior of other synchronous builtins.) The 'stale' command no longer logs output when a target is unexpectedly non-stale. (However, the ouput of the 'stale' command is not usually very useful anyway.) The 'grep', 'stdout', and 'stderr' commands now display matching lines (like Unix 'grep'), making their negation behavior more consistent with running real commands on a command-line. Likewise, the 'cmp' command now always displays differences. That makes it useful with the '?' prefix to produce diffs for informational purposes while debugging. For #27494. Change-Id: If49fd81d9b922d07c20618a8e2cef908191f9ef6 Reviewed-on: https://go-review.googlesource.com/c/go/+/419875 Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-04cmd/internal/sys: migrate support.go functions to new internal pkgThan McIntosh
Separate out the functions from cmd/internal/sys/support.go and migrate them to a new package internal/platform, so that functions such as "RaceDetectorSupported" can be called from tests in std as well as in cmd. This isn't a complete move of everything in cmd/internal/sys; there are still many functions left. The original version of this CL (patch set 1) called the new package "internal/sys", but for packages that needed both "internal/sys" and "cmd/internal/sys" the import of the former had to be done with a different name, which was confusing and also required a hack in cmd/dist. Updates #56006. Change-Id: I866d62e75adbf3a640a06e2c7386a6e9e2a18d91 Reviewed-on: https://go-review.googlesource.com/c/go/+/438475 Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Cherry Mui <cherryyz@google.com> Run-TryBot: Than McIntosh <thanm@google.com>
2022-09-29cmd/go: using strings.CutPrefix replace strings.HasPrefix and strings.TrimPrefixcuiweixie
Change-Id: I143d05c24a3e897d0f3ee78dd16954c32ceae091 Reviewed-on: https://go-review.googlesource.com/c/go/+/435137 Run-TryBot: xie cui <523516579@qq.com> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com>
2022-09-28cmd/go: support new hybrid coverage instrumentationThan McIntosh
If GOEXPERIMENT=coverageredesign is in effect, introduce a new top-level '-cover' option to "go build" to turn on new-style hybrid code coverage instrumentation. Similarly, use the new instrumentation for "go test -cover". The main effects of "-cover" under the hood are to instrument files at the package level using cmd/cover and to pass additional options to the compiler when building instrumented packages. The previous workflow for "go tool -cover mypkg" would expand to a series of "go tool cover" commands (one per file) followed by a single package compilation command to build the rewritten sources. With the new workflow, the Go command will pass all of the Go files in a package to the cover tool as a chunk (along with a config file containing other parameters), then the cover tool will write instrumented versions of the sources along with another "output" config with info on coverage variable names for the the compiler. The Go command will then kick off the compiler on the modified source files, also passing in the config file generated by cmd/cover. Updates #51430. Change-Id: Id65621ff6a8c70a30168c1412c2d6f805ff3b9e7 Reviewed-on: https://go-review.googlesource.com/c/go/+/355452 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Than McIntosh <thanm@google.com> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-09-28cmd/go: add hook to check for GOEXPERIMENT in script testsThan McIntosh
Add a new hook to allow script tests to check whether a specific GOEXPERIMENT is enabled. Updates #51430. Change-Id: Icdf39f845ff2c8b10c634d49e9c27bc90e7984f8 Reviewed-on: https://go-review.googlesource.com/c/go/+/402174 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-27all: replace [0-9] with \d in regexpsTomCao New Macbook Pro
1. replace [0-9] with \d in regexps 2. replace [a-zA-Z0-9_] with \w in regexps Change-Id: I9e260538252a0c1071e76aeb1c5f885c6843a431 GitHub-Last-Rev: 286e1a4619c4bdda7f461afbd6d30b9f312c0486 GitHub-Pull-Request: golang/go#54874 Reviewed-on: https://go-review.googlesource.com/c/go/+/428435 Reviewed-by: Ian Lance Taylor <iant@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-08-24cmd/go/internal/work: make NewBuilder safe for concurrent and repeated useBryan C. Mills
Ever since 'go build' was added (in CL 5483069), it has used an atexit handler to clean up working directories. At some point (prior to CL 95900044), Init was called multiple times per builder, registering potentially many atexit handlers that execute asynchronously and make debugging more difficult. The use of an AtExit handler also makes the Builder (and anything that uses it) prone to races: the base.AtExit API is not designed for concurrent use, but cmd/go is becoming increasingly concurrent over time. The AtExit handler also makes the Builder inappropriate to use within a unit-test, since the handlers do not run during the test function and accumulate over time. This change makes NewBuilder safe for concurrent use by registering the AtExit handler only once (during BuildInit, which was already not safe for concurrent use), and using a sync.Map to store the set of builders that need cleanup in case of an unclean exit. In addition, it causes the test variant of cmd/go to fail if any Builder instance leaks from a clean exit, helping to ensure that functions that create Builders do not leak them indefinitely, especially in tests. Updates #54423. Change-Id: Ia227b15b8fa53c33177c71271d756ac0858feebe Reviewed-on: https://go-review.googlesource.com/c/go/+/425254 Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Than McIntosh <thanm@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com>
2022-08-24cmd/go: avoid registering AtExit handlers in testsBryan C. Mills
Ever since 'go build' was added (in CL 5483069), it has used an atexit handler to clean up working directories. CL 154109 introduced 'cc' command to the script test framework that called Init on a builder once per invocation. Unfortunately, since base.AtExit is unsynchronized, the Init added there caused any script that invokes that command to be unsafe for concurrent use. This change fixes the race by having the 'cc' command pass in its working directory instead of allowing the Builder to allocate one. Following modern Go best practices, it also replaces the in-place Init method (which is prone to typestate and aliasing bugs) with a NewBuilder constructor function. Fixes #54423. Change-Id: I8fc2127a7d877bb39a1174e398736bb51d03d4d2 Reviewed-on: https://go-review.googlesource.com/c/go/+/425205 TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Than McIntosh <thanm@google.com>
2022-08-18cmd/go: avoid passing testing.T to isCaseSensitive and goVersionBryan C. Mills
The previous implementation of isCaseSensitive called t.Fatalf in the wrong place, causing tests after the first to proceed past an error determining case-sensitivity. That could lead to confusing errors. (Moreover, I would like to try to disentangle the script engine from testing.T so that I can also use it to generate serving contents in the replacement for vcs-test.golang.org.) The implementation of goVersion called ts.fatalf, which is probably fine but prevents the script environment from being computed outside of a test, as we might want to do for debugging and other scripting. For #27494. Change-Id: Ibfee0704523fdcd6174b544ff84267216435025b Reviewed-on: https://go-review.googlesource.com/c/go/+/419874 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com>
2022-08-18cmd/go: run tests when cmd/go is cross-compiledBryan C. Mills
When the GOOS or GOARCH of the cmd/go test binary does not match the GOOS or GOARCH of the installed 'go' binary itself, the test currently attempts to trick 'go test' into thinking that there were no test functions to run. That makes it very difficult to discover how to actually run the tests, which in turn makes it difficult to diagnose and fix regressions in, say, the linux-386-longtest builders. (We have had a few of those lately, and they shouldn't be as much of an ordeal to fix as they currently are.) There are three underlying problems: 1. cmd/go uses its own GOOS and GOARCH to figure out which variant of other tools to use, and the cache keys for all installed tools and libraries include the IDs of the tools used to build them. So when cmd/go's GOARCH changes, all installed tools and binaries appear stale *even if* they were just installed by invoking the native cmd/go with the appropriate GOARCH value set. 2. The "go/build" library used by cmd/go toggles its default CGO_ENABLED behavior depending on whether the GOOS and GOARCH being imported match runtime.GOOS and runtime.GOARCH. 3. A handful of cmd/go tests explicitly use gccgo, but the user's installed gccgo binary cannot necessarily cross-compile to the same platforms as cmd/go. To address the cache-invalidation problem, we modify the test variant of cmd/go to use the host's native toolchain (as indicated by the new TESTGO_GOHOSTOS and TESTGO_GOHOSTARCH environment variables) instead of the toolchain matching the test binary itself. That allows a test cmd/go binary compiled with GOARCH=386 to use libraries and tools cross-compiled by the native toolchain, so that $ GOARCH=386 go install std cmd suffices to make the packages in std and cmd non-stale in the tests. To address the CGO_ENABLED mismatch, we set CGO_ENABLED explicitly in the test's environment whenever it may differ from the default. Since script tests that use cgo are already expected to use a [cgo] condition, setting the environment to match that condition fixes the cgo-specific tests. To address the gccgo-specific cross-compilation failures, we add a new script condition, [cross], which evaluates to true whenever the platform of the test binary differs from that of the native toolchain. We can then use that condition to explicitly skip the handful of gccgo tests that fail under cross-compilation. Fixes #53936. Change-Id: I8633944f674eb5941ccc95df928991660e7e8137 Reviewed-on: https://go-review.googlesource.com/c/go/+/356611 Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Russ Cox <rsc@golang.org>
2022-07-08cmd/go: avoid setting variables for '/' and ':' in TestScript subprocess ↵Bryan C. Mills
environments Also simplify platform-dependent handling of the PATH variable, to make it more like the existing platform-dependent handling for HOME and TMPDIR. Fixes #53671. Change-Id: Ica2665d3f61988c66fb6982b9feb61ca48eced79 Reviewed-on: https://go-review.googlesource.com/c/go/+/416554 Reviewed-by: Ian Lance Taylor <iant@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-30cmd/go: add a 'sleep' command for script testsBryan C. Mills
Due to mtime skew we don't index mutable packages with an mtime younger than 2 seconds. In order to test indexed packages reliably, we want to be able to sleep long enough for the files in the package to be cached. (As an alternative we could instead use os.Chtimes to fake old enough timestamps, but sleeping keeps the tests more realistic.) For #53586. Change-Id: I1873f47c55a72d928451593b8c989f0092a557db Reviewed-on: https://go-review.googlesource.com/c/go/+/415474 Auto-Submit: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-06-16cmd/go: pass GOEXPERIMENT through to subtestsRuss Cox
This fixes: export GOEXPERIMENT=unified go install cmd go install std cmd go install std cmd go test -short cmd/go -run=TestScript/test_relative_import_dash_i That script test checks that runtime is non-stale, but whether it's stale depends on the setting of GOEXPERIMENT. Stop filtering that variable out. Change-Id: I71bdbca495c16981cdcddf4ab4d87a38ca72a389 Reviewed-on: https://go-review.googlesource.com/c/go/+/412874 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com> Auto-Submit: Russ Cox <rsc@golang.org>
2022-06-08cmd/go: enable module index by defaultMichael Matloob
This changes the module index to be enabled by default, rather than disabled by default. The index can still be disabled by setting GODEBUG=index=0. Fixes #53290. Change-Id: Ic3728fc69d96bb6ef56b56e8c9f2dce35f2923cc Reviewed-on: https://go-review.googlesource.com/c/go/+/410821 Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2022-06-03cmd/go: changes to use modindexMichael Matloob
This CL makes the changes to actually use the module index when loading packages and instead of scanning their directories to see if they contain go files or to extract imports. Change-Id: I70106181cf64d6fd5a416644ba518b6b90030e0a Reviewed-on: https://go-review.googlesource.com/c/go/+/403778 Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org>
2022-06-01cmd/go: set GIT_TRACE_CURL for tests on buildersBryan C. Mills
We have noticed a pattern of connection timeouts connecting to github.com on the builders. Adding tracing may shed some light on the underlying cause. For #52545. Change-Id: Ic73b494be9a3ea8ce3c7631ee8f62bd3d0d8e7bd Reviewed-on: https://go-review.googlesource.com/c/go/+/409575 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-11cmd/go: add timestamps to script test outputBryan C. Mills
Go tests don't include timestamps by default, but we would like to have them in order to correlate builder failures with server and network logs. Since many of the Go tests with external network and service dependencies are script tests for the 'go' command, logging timestamps here adds a lot of logging value with one simple and very low-risk change. For #50541. For #52490. For #52545. For #52851. Change-Id: If3fa86deb4a216ec6a1abc4e6f4ee9b05030a729 Reviewed-on: https://go-review.googlesource.com/c/go/+/405714 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Auto-Submit: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Run-TryBot: Bryan Mills <bcmills@google.com>
2022-04-12cmd/go: fix TestScript/build_trimpath_goroot when built with a mismatched ↵Bryan C. Mills
GOROOT_FINAL Fixes #52236. Updates #51461. Change-Id: Ie91e0256afd45e9bbd60fd8cdc696363027ab696 Reviewed-on: https://go-review.googlesource.com/c/go/+/399156 Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Russ Cox <rsc@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01all: remove trailing blank doc comment linesRuss Cox
A future change to gofmt will rewrite // Doc comment. // func f() to // Doc comment. func f() Apply that change preemptively to all doc comments. For #51082. Change-Id: I4023e16cfb0729b64a8590f071cd92f17343081d Reviewed-on: https://go-review.googlesource.com/c/go/+/384259 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-01all: fix various doc comment formatting nitsRuss Cox
A run of lines that are indented with any number of spaces or tabs format as a <pre> block. This commit fixes various doc comments that format badly according to that (standard) rule. For example, consider: // - List item. // Second line. // - Another item. Because the - lines are unindented, this is actually two paragraphs separated by a one-line <pre> block. This CL rewrites it to: // - List item. // Second line. // - Another item. Today, that will format as a single <pre> block. In a future release, we hope to format it as a bulleted list. Various other minor fixes as well, all in preparation for reformatting. For #51082. Change-Id: I95cf06040d4186830e571cd50148be3bf8daf189 Reviewed-on: https://go-review.googlesource.com/c/go/+/384257 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-18cmd/go: diagnose unset GOROOT when built with -trimpathBryan C. Mills
For #51483 Change-Id: I4546c20cf968b595020a1eba888fe1d9a1c6cfc5 Reviewed-on: https://go-review.googlesource.com/c/go/+/391811 Reviewed-by: Russ Cox <rsc@golang.org> Trust: Bryan Mills <bcmills@google.com>
2022-03-16internal/txtar: bring back to main repo, for tests in stdRuss Cox
This used to be cmd/go/internal/txtar, and then it was moved to golang.org/x/tools/txtar and revendored from there into cmd/vendor/golang.org/x/tools/txtar. We have a use for txtar in a new test in the standard library, which cannot access cmd/vendor. But we also don't really want to vendor it into the standard library as is, because that would be the first vendoring of x/tools in std, and it would be better to keep std separate from x/tools, even for testing. Instead, since a little copying is better than a little dependency, just make a copy in internal/txtar. The package does not change. Having done that, replace the uses in cmd/go so that there's only one copy in the main repo. Change-Id: I70b5cc05da3f6ebcc0fd9052ebcb3d369fb57956 Reviewed-on: https://go-review.googlesource.com/c/go/+/384254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-03-04cmd/go: avoid rebuilding itself in TestMainDaniel Martí
An extra "go build" was happening, for the sake of -tags=testgo, which would insert some extra behavior into ./internal/work. Instead, reuse the test binary as cmd/go directly, by calling the main func when a special env var is set. We still duplicate the test binary into testBin, because we need a "go" executable in that directory for $PATH. Finally, the special behavior is instead inserted via TestMain. The numbers below represent how long it takes to run zero tests, measured via: benchcmd GoTestNothing go test -run=- That is, the time it takes to run the first test is reduced by half. Note that these numbers are on a warm build cache, so if the -tags=testgo build were to be done from scratch, the speed-up would be significantly more noticeable. name old time/op new time/op delta GoTestNothing 830ms ± 2% 380ms ± 7% -54.23% (p=0.008 n=5+5) name old user-time/op new user-time/op delta GoTestNothing 1.64s ± 1% 0.82s ± 3% -50.24% (p=0.008 n=5+5) name old sys-time/op new sys-time/op delta GoTestNothing 306ms ± 7% 159ms ±28% -48.15% (p=0.008 n=5+5) name old peak-RSS-bytes new peak-RSS-bytes delta GoTestNothing 173MB ± 1% 147MB ± 1% -14.96% (p=0.008 n=5+5) Change-Id: I1f8fc71269a7b45bc5b82b7228e13f56589d44c3 Reviewed-on: https://go-review.googlesource.com/c/go/+/378294 Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com>
2022-02-04cmd/go: preserve LIBRARY_PATH and C_INCLUDE_PATH for script testsBrandon Bennett
In bespoke build environments default libraries may be specificied with LIBRARY_PATH, C_INCLUDE_PATH enviroment variables to overide the system (i.e glibc). Allow them though to allow cgo testing of these enviroments. Fixes #50985 Change-Id: I7497a7715d9b635a6ae97efaab94a7ff01cdf8e2 Reviewed-on: https://go-review.googlesource.com/c/go/+/383334 Reviewed-by: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Trust: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-01cmd/go: add detail to test failuresBryan C. Mills
For #50892 Change-Id: I14ff1c43b39687a0ba5e668ee962cecfb49e4beb Reviewed-on: https://go-review.googlesource.com/c/go/+/381836 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-01cmd/go: add missing newlines in TestScript '? cmp' outputBryan C. Mills
Change-Id: I0314c2e9073b162119060a23c5eee9a78ca08ebf Reviewed-on: https://go-review.googlesource.com/c/go/+/382239 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2022-01-26cmd/go: add mv and support "! cmp" in script testsBryan C. Mills
For #50183 Change-Id: Ie384333fb7a69d0d2cfaba0cfc4eb7afba2fd745 Reviewed-on: https://go-review.googlesource.com/c/go/+/380916 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2021-12-23cmd/go: remove MallocNanoZone environment variable workaroundCherry Mui
We added a workaround in runtime/race. This should not be necessary now. Updates #49138. Change-Id: Ia2010e4acc95c4ddf5f463ab2919401d893c0bac Reviewed-on: https://go-review.googlesource.com/c/go/+/374314 Trust: Cherry Mui <cherryyz@google.com> Run-TryBot: Cherry Mui <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13all: gofmt -w -r 'interface{} -> any' srcRuss Cox
And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-11-22cmd/go: forward the MallocNanoZone variable to script testsBryan C. Mills
For #49138 Fixes #49723 Change-Id: Ia93130fdc042a1e2107be95cccd7e7eeaa909a87 Reviewed-on: https://go-review.googlesource.com/c/go/+/366254 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-28cmd/go: consolidate fuzz-support checksBryan C. Mills
We had been repeating conditions for specific platforms and architectures to gate fuzzing tests, but the more of those tests we add the more we will have to update if the set of supported platforms and archictures expands over time. We also ought to provide a friendlier error message when 'go test -fuzz' is used on non-supported platforms. This change adds predicates in cmd/internal/sys, which already contains similar predicates for related functionality (such as the race detector), and uses those predicates in 'go test' and TestScript. For #48495 Change-Id: If24c3997aeb4d201258e21e5b6cf4f7c08fbadd7 Reviewed-on: https://go-review.googlesource.com/c/go/+/359481 Trust: Bryan C. Mills <bcmills@google.com> Trust: Katie Hockman <katie@golang.org> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org>
2021-10-26cmd/go: add -asan optionFannie Zhang
The -asan option compiles Go code to use the address sanitizer. This is intended for use when linking with C/C++ code compiled with -fsanitize=address. When memory blocks are passed back and forth between C/C++ and Go, code in both languages will agree as to whether the memory is validly allocated or not, and will report errors for any use of invalid memory. Updates #44853. Change-Id: I0209002ef795cc1c823daae557fb80c906158db3 Reviewed-on: https://go-review.googlesource.com/c/go/+/298612 Trust: fannie zhang <Fannie.Zhang@arm.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>