aboutsummaryrefslogtreecommitdiff
path: root/src/cmd/go/internal/modget/get.go
AgeCommit message (Collapse)Author
2021-08-25[dev.cmdgo] cmd/go: clean up TODOWorkspaces instancesMichael Matloob
Address some of the easier todos to address and remove the todos that have already been done and redundant todos. For #45713 Change-Id: I3fe4393168b10c6e005325258d9701713c92e9e4 Reviewed-on: https://go-review.googlesource.com/c/go/+/344491 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-08-11[dev.cmdgo] cmd/go: remove modload.ModRoot functionMichael Matloob
In some cases, ModRoot was being called in a multi module context. In those cases, pass in the correct main module. In other cases, a mainModule variable was already available, so call MainModules.ModRoot on that mainModule variable to make it more clear. In yet other cases ModRoot is just needed to determine GoMod, so determine modroot from the current directory in those cases. For #45713 Change-Id: I8c8aa633cfae40d0c8740bdbf985f2b60c9daf2c Reviewed-on: https://go-review.googlesource.com/c/go/+/339171 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-07-30[dev.cmdgo] cmd/go: thread through modroots providing replacementsMichael Matloob
modload.Replacement and modload.resolveReplacement now also return the modroot of the module providing a replacement so that we can correctly construct the path of a replaced module (because the path in the module.Version is relative to the modroot). For #45713 Change-Id: I8c69ccbcc1f40201071e35fcf93d6b5d0ed4cdf7 Reviewed-on: https://go-review.googlesource.com/c/go/+/334941 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-07-22[dev.cmdgo] cmd/go: replace Target with MainModules, allowing for multiple ↵Michael Matloob
targets This change replaces the Target variable that represents the main module and the pathPrefix and inGorootSrc which provide other information about the main module with a single MainModules value that represents multiple main modules and holds their path prefixes, module roots, and whether they are in GOROOT/src. In cases where the code checks Target or its previously associated variables, the code now checks or iterates over MainModules. In some cases, the code still assumes a single main module by calling MainModules.MustGetSingleMainModule. Some of those cases are correct: for instance, there is always only one main module for mod=vendor. Other cases are accompanied with TODOs and will have to be fixed in future CLs to properly support multiple main modules. This CL (and other cls on top of it) are planned to be checked into a branch to allow for those evaluating the workspaces proposal to try it hands on. For #45713 Change-Id: I3b699e1d5cad8c76d62dc567b8460de8c73a87ea Reviewed-on: https://go-review.googlesource.com/c/go/+/334932 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-06-21cmd/go: add a -go flag to 'go mod graph'Bryan C. Mills
For #46366 Change-Id: I8417e6e4dbb8cb56ff7afc16893a01b7bb938217 Reviewed-on: https://go-review.googlesource.com/c/go/+/329529 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-06-16cmd/go: in lazy modules, add transitive imports for 'go get' argumentsBryan C. Mills
I needed to also update TestScript/mod_sumdb_golang. It had been relying on 'go list -mod=mod' to add both the go.mod and go.sum entries for the named package, but when 'go get' actually adds all of the needed dependencies, lazy loading kicks in and 'go list' doesn't end up needing the checksums for go.mod files. We didn't detect the skew before because the 'go list' command was (unexpectedly) also adding the missing dependencies, which triggered a deep scan of the complete module graph. For #45979 Change-Id: Ica917dee22c83ffa71c6ad0f2e189f911b73edf4 Reviewed-on: https://go-review.googlesource.com/c/go/+/328231 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-06-01cmd/go: make 'go get' save sums for incidentally updated modulesJay Conrod
When 'go get' updates a module, it may update another module in the build list that provides a package in 'all' that wasn't loaded as part of the 'go get' command. If 'go get' doesn't add a sum for that module, builds may fail later. With this change, 'go get' will fetch a sum for the content of an updated module if we had a sum for the version before the update. 'go get' won't load the complete package graph, so there are still cases where the build may be broken, like when an updated (but not loaded) package imports a package from a new module. Fixes #44129 Change-Id: I62eba3df4137a3e84e2ca8d549c36eec3670f08c Reviewed-on: https://go-review.googlesource.com/c/go/+/322832 Trust: Jay Conrod <jayconrod@google.com> Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-05-21cmd/go: remove warning from module deprecation notice printingTim Heckman
Fixes #46294 Change-Id: I50023006dab83dee455f98a702ca1c72e61764ea Reviewed-on: https://go-review.googlesource.com/c/go/+/321649 Trust: Jay Conrod <jayconrod@google.com> Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-05-17cmd/go: don't print 'go get' deprecation notices in the main moduleJay Conrod
If a user runs 'go get example.com/cmd' for a package in the main module, it's more likely they intend to fill in missing dependencies for that package (especially with -u). If the intent were only to build and install, 'go install example.com/cmd' would be a better choice. For #43684 Resolving a comment on CL 305670. Change-Id: I5c80ffdcdb3425b448f2f49cc20b07a18cb2bbe9 Reviewed-on: https://go-review.googlesource.com/c/go/+/318570 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-04-27cmd/go/internal/modload: clean up error reportingBryan C. Mills
• Consolidate 'if ld.AllowErrors' conditions into an 'ld.errorf' method. • Rename SilenceErrors to SilencePackageErrors and clarify its documentation. (There is currently no way to silence errors in the module graph. Perhaps we should add one, but for now let's at least clarify the existing behavior.) • Move 'tidy -v' verbose logging into LoadPackages, where other logging happens. • Make checkMultiplePaths a loader method (since it only matters during package loading anyway). • Check package and module-graph errors in loadFromRoots instead of LoadPackages. These checks were previously omitted on the ImportFromFiles path, which seems likely to be a bug. (We now suppress package errors explicitly in ImportFromFiles, which at least makes the bug more explicit.) This somewhat simplifies the code structure in preparation for the lazy-mode tidy implementation. For #36460 Change-Id: I3ce3586c6934989d5194f00f99e7cc4423cf767f Reviewed-on: https://go-review.googlesource.com/c/go/+/313229 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-04-16cmd/go/internal/load: convert two global flags to an options structJay Conrod
PackageOpts is a new struct type accepted by package loading functions. It initially has two fields: IgnoreImports, and ModResolveTests. Previously, these were global variables set by clients. We'll add more to this in the future. For #40775 Change-Id: I6956e56502de836d3815ce788bdf16fc5f3e5338 Reviewed-on: https://go-review.googlesource.com/c/go/+/310669 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-04-09cmd/go: support module deprecationJay Conrod
A module is deprecated if its author adds a comment containing a paragraph starting with "Deprecated:" to its go.mod file. The comment must appear immediately before the "module" directive or as a suffix on the same line. The deprecation message runs from just after "Deprecated:" to the end of the paragraph. This is implemented in CL 301089. 'go list -m -u' loads deprecation messages from the latest version of each module, not considering retractions (i.e., deprecations and retractions are loaded from the same version). By default, deprecated modules are printed with a "(deprecated)" suffix. The full deprecation message is available in the -f and -json output. 'go get' prints deprecation warnings for modules named on the command line. It also prints warnings for modules needed to build packages named on the command line if those modules are direct dependencies of the main module. For #40357 Change-Id: Id81fb2b24710681b025becd6cd74f746f4378e78 Reviewed-on: https://go-review.googlesource.com/c/go/+/306334 Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-04-05cmd/go: print deprecation notice for 'go get cmd'Jay Conrod
The notice is shown when 'go get' is invoked with the -d flag, and the arguments match at least one main package. This reverts CL 274552. For #43684 Change-Id: I42e6731455f22988bf72dde1d5a76d197e9e3954 Reviewed-on: https://go-review.googlesource.com/c/go/+/305670 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-25cmd/go/internal/modload: replace the global buildList with structured ↵Bryan C. Mills
requirements This is intended to be a pure-refactoring change, with very little observable change in the behavior of the 'go' command. A few error messages have prefixes changed (by virtue of being attached to packages or modules instead of the build list overall), and 'go list -m' (without arguments) no longer loads the complete module graph in order to provide the name of the (local) main module. The previous modload.buildList variable contained a flattened build list, from which the go.mod file was reconstructed using various heuristics and metadata cobbled together from the original go.mod file, the package loader (which was occasionally constructed without actually loading packages, for the sole purpose of populating otherwise-unrelated metadata!), and the updated build list. This change replaces that variable with a new package-level variable, named "requirements". The new variable is structured to match the structure of the go.mod file: it explicitly specifies the roots of the module graph, from which the complete module graph and complete build list can be reconstructed (and cached) on demand. Similarly, the "direct" markings on the go.mod requirements are now stored alongside the requirements themselves, rather than side-channeled through the loader. The requirements are now plumbed explicitly though the modload package, with accesses to the package-level variable occurring only within top-level exported functions. The structured requirements are logically immutable, so a new copy of the requirements is constructed whenever the requirements are changed, substantially reducing implicit communication-by-sharing in the package. For #36460 Updates #40775 Change-Id: I97bb0381708f9d3e42af385b5c88a7038e1f0556 Reviewed-on: https://go-review.googlesource.com/c/go/+/293689 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-18cmd/go: suppress errors for 'go get' of module paths that are also ↵Bryan C. Mills
constrained-out packages Fixes #33526 Change-Id: Iedd2d6dbe440499bf074ac632513319a22f2d648 Reviewed-on: https://go-review.googlesource.com/c/go/+/297009 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-03-10cmd/go/internal/modload: make EditBuildList report whether the build list ↵Bryan C. Mills
was changed For #36460 Change-Id: I8dd6e6f998a217a4287212815ce61209df6f007f Reviewed-on: https://go-review.googlesource.com/c/go/+/296609 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-02cmd/go: remove -insecure flag on go getwitchard
Resolves #37519 Change-Id: Iba675a180b0e61b12835cdb6ecd4c6dc61e0605c GitHub-Last-Rev: aa018af6f8fc7f0b829820e831ad96734adcb8d0 GitHub-Pull-Request: golang/go#44724 Reviewed-on: https://go-review.googlesource.com/c/go/+/297709 Trust: Jay Conrod <jayconrod@google.com> Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-03-02cmd/go: resolve std-vendored dependencies as std packages except in 'go get' ↵Bryan C. Mills
and 'go mod' In CL 251159, I removed a hard-coded special case changing the rewriting behavior for std dependencies in GOROOT/src/vendor and GOROOT/src/cmd/vendor. Unfortunately, that caused packages in 'std' to be reported as stale when run within GOROOT/src. This change restores the special-case behavior, but plumbs it through the PackageOpts explicitly instead of comparing strings stored in global variables. Fixes #44725 Change-Id: If084fe74972ce1704715ca79b0b7e092dd90c88b Reviewed-on: https://go-review.googlesource.com/c/go/+/297869 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-03-01cmd/go: add missing newline to retraction warning messageJordan Liggitt
Fixes #44674 Change-Id: Icbdb79084bf7bd2f52cc0a53abcc1ec6f0c4a1bf Reviewed-on: https://go-review.googlesource.com/c/go/+/297350 Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Jay Conrod <jayconrod@google.com>
2021-02-19cmd/go/internal/modget: split resolveCandidates into two methodsBryan C. Mills
It turns out that the existing call sites of the resolveCandidates method pass only *either* a slice of queries or a slice of upgrades (never both), and the behaviors triggered by the two parameters don't overlap much at all. To clarify the two different operations, split them into two separate methods. For #36460 Change-Id: I64651637734fd44fea68740a3cdfbacfb73c19b6 Reviewed-on: https://go-review.googlesource.com/c/go/+/289697 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-02-11cmd/go: multiple small 'go help' fixesJay Conrod
* Link to privacy policies for proxy.golang.org and sum.golang.org in 'go help modules'. It's important that both policies are linked from the go command's documentation. * Fix wording and typo in 'go help vcs' following comments in CL 290992, which adds reference documentation for GOVCS. * Fix whitespace on GOVCS in 'go help environment'. For #41730 Change-Id: I86abceacd4962b748361244026f219157c9285e9 Reviewed-on: https://go-review.googlesource.com/c/go/+/291230 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-02-09cmd/go: suppress errors from 'go get -d' for packages that only ↵Bryan C. Mills
conditionally exist Fixes #44106 Fixes #29268 Change-Id: Id113f2ced274d43fbf66cb804581448218996f81 Reviewed-on: https://go-review.googlesource.com/c/go/+/289769 TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com>
2021-01-08cmd/go: revise 'go help' documentation for modulesJay Conrod
Module-related help pages now contain a brief summary and point to the reference documentation at golang.org/ref/mod for details. Help pages for commands like 'go get' still describe the basic usage and summarize flags but don't provide as much background detail. Fixes #41427 Fixes #43419 Change-Id: Icacd38e0f33c352c447cc5a496c99674493abde2 Reviewed-on: https://go-review.googlesource.com/c/go/+/282615 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-08cmd/go: make hints in error messages more consistentJay Conrod
* All commands the user can run to fix the problem now appear alone on a separate line after a tab. * Removed -d from 'go get' commands. * Replaced 'go mod tidy' with 'go mod download $modpath' when a package might be provided by a module missing a sum. * Errors about 'path@version' syntax are more explicit. Fixes #29415 Fixes #42087 Fixes #43430 Fixes #43523 Change-Id: I4427c2c4506a727a2c727d652fd2d506bb134d3b Reviewed-on: https://go-review.googlesource.com/c/go/+/282121 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-12-14cmd/go: refactor error reporting in internal/loadJay Conrod
Replaced load.PackagesForBuild with a new function, load.CheckPackageErrors. Callers should now call PackagesAndErrors, then CheckPackageErrors for the same functionality. Removed load.Packages. Callers should call base.Errorf and filter the package list as needed. This gives callers more flexibility in handling package load errors. For #42638 Change-Id: Id75463ba695adc1ca3f8693ceb2c8978b74a3500 Reviewed-on: https://go-review.googlesource.com/c/go/+/277354 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-12-01cmd/go: don't print deprecation notice for 'go get exe'Jay Conrod
It's difficult for module authors to provide installation instructions that work in both Go 1.15 and 1.16. We'll wait until 1.17 to print a deprecation warning for installing executables with 'go get'. Fixes #42885 Change-Id: I835b447e83e760f48fd664e8a117749e0cb59f83 Reviewed-on: https://go-review.googlesource.com/c/go/+/274552 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-11-20cmd/go: recommend 'go get' command to switch from retracted versionsJay Conrod
This CL restores a message unintentionally removed in CL 270858. For #24031 Change-Id: I957c5c59e624df98e72dfff351298bfc5bf9a9e7 Reviewed-on: https://go-review.googlesource.com/c/go/+/272066 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2020-11-20cmd/go: report changes and resolved versions in 'go get'Jay Conrod
Fixes #33284 Change-Id: I33daa5eb518985bc7308f29655e04c57e244b479 Reviewed-on: https://go-review.googlesource.com/c/go/+/269018 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-20cmd/go: remove ListModules call in runGetJay Conrod
ListModules was used to download .info files so that 'go list -m all' would succeed later when offline. However, 'go list -m all' may already fail when offline after 'go mod tidy', so it doesn't make sense to add complexity to 'go get'. Instead, remove the ListModules call and fix the test that accidentally depended on it. For #42723 Change-Id: I692597cf5ca15c23fa6fc9d2bac4b6e044299482 Reviewed-on: https://go-review.googlesource.com/c/go/+/271577 Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-18cmd/go: in 'go get', only load retractions for resolved versionsJay Conrod
Previously, 'go get' loaded retractions for every module in the build list, which took a long time and usually wasn't helpful. This rolls forward CL 269019, which was reverted in CL 270521. The new revision adds a call to modload.ListModules at the end of 'go get' to ensure .info files are cached for everything in the build list. Fixes #42185 Change-Id: I684f66c5e674384d5a0176fbc8317e5530b8a915 Reviewed-on: https://go-review.googlesource.com/c/go/+/270858 Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-17cmd/go/internal/modget: clarify error for 'go get' without argumentsJay Conrod
If the current directory doesn't contain a package, 'go get' will say that without additional detail. If there were no arguments, errors will start with "go get:" instead of "go get .:". Fixes #39080 Change-Id: I47366f2a27bce17bd8b79344ad15b8b934a888c9 Reviewed-on: https://go-review.googlesource.com/c/go/+/234681 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com>
2020-11-17cmd/go: revert "in 'go get', only load retractions for resolved versions"Dmitri Shuralyov
This reverts CL 269019. Reason for revert: The TestScript/mod_gonoproxy test is failing on linux-386-longtest and linux-amd64-longtest builders. Change-Id: I7e132fb4fb5a9c00add28e5100a0e96a9250282c Reviewed-on: https://go-review.googlesource.com/c/go/+/270521 Trust: Dmitri Shuralyov <dmitshur@golang.org> Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-16cmd/go: in 'go get', only load retractions for resolved versionsJay Conrod
Previously, 'go get' loaded retractions for every module in the build list, which took a long time and usually wasn't helpful. Fixes #42185 Change-Id: I64294585db141106b63ec74aafa0d266b7536ef2 Reviewed-on: https://go-review.googlesource.com/c/go/+/269019 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com>
2020-11-09cmd/go: print deprecation messages for 'go get' installing executablesJay Conrod
For #40276 Change-Id: I5e631a4c9ce07f23640fb56eb455457bc55072c6 Reviewed-on: https://go-review.googlesource.com/c/go/+/266360 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-09cmd/go: add GOVCS setting to control version control usageRuss Cox
The go command runs commands like git and hg to download modules. In the past, we have had problems with security bugs in version control systems becoming security bugs in “go get”. The original modules draft design removed use of these commands entirely, saying: > We want to move away from invoking version control tools such as bzr, > fossil, git, hg, and svn to download source code. These fragment the > ecosystem: packages developed using Bazaar or Fossil, for example, are > effectively unavailable to users who cannot or choose not to install > these tools. The version control tools have also been a source of > exciting security problems. It would be good to move them outside the > security perimeter. The removal of these commands was not possible in the end: being able to fetch directly from Git repos is too important, especially for closed source. But the security exposure has not gone away. We remain vulnerable to problems in VCS systems, especially the less scrutinized ones. This change adds a GOVCS setting to let users control which version control systems are allowed by default. It also changes the default allowed version control systems to git and hg for public code and any version control system for private code (import path or module path matched by the GOPRIVATE setting). See the changes in alldocs.go for detailed documentation. See #41730 for proposal and discussion. Fixes #41730. [Replay of CL 266420. See changes from Patch Set 1 for updates to fix a few long tests.] Change-Id: I4fe93804548956c42aea985368b4571bdb220f48 Reviewed-on: https://go-review.googlesource.com/c/go/+/267888 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-09cmd/go/internal/modget: fix a typo introduced in CL 263267Bryan C. Mills
Updates #37438 Change-Id: I78f377afd73dad75aed219836725a27fbaa5b69c Reviewed-on: https://go-review.googlesource.com/c/go/+/268117 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Dan Peterson <dpiddy@gmail.com> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2020-11-05cmd/go/internal/modget: resolve paths at the requested versionsBryan C. Mills
Previously, we resolved each argument to 'go get' to a package path or module path based on what was in the build list at existing versions, even if the argument specified a different version explicitly. That resulted in bugs like #37438, in which we variously resolved the wrong version or guessed the wrong argument type for what is unambiguously a package argument at the requested version. We were also using a two-step upgrade/downgrade algorithm, which could not only upgrade more that is strictly necessary, but could also unintentionally upgrade *above* the requested versions during the downgrade step. This change instead uses an iterative approach, with an explicit disambiguation step for the (rare) cases where an argument could match the same package path in multiple modules. We use a hook in the package loader to halt package loading as soon as an incorrect version is found — preventing over-resolving — and verify that the result after applying downgrades successfully obtained the requested versions of all modules. Making 'go get' be correct and usable is especially important now that we are defaulting to read-only mode (#40728), for which we are recommending 'go get' more heavily. While I'm in here refactoring, I'm also reworking the API boundary between the modget and modload packages. Previously, the modget package edited the build list directly, and the modload package accepted the edited build list without validation. For lazy loading (#36460), the modload package will need to maintain additional metadata about the requirement graph, so it needs tighter control over the changes to the build list. As of this change, modget no longer invokes MVS directly, but instead goes through the modload package. The resulting API gives clearer reasons in case of updates, which we can use to emit more useful errors. Fixes #37438 Updates #36460 Updates #40728 Change-Id: I596f0020f3795870dec258147e6fc26a3292c93a Reviewed-on: https://go-review.googlesource.com/c/go/+/263267 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2020-11-05cmd/go/internal/modload: return a module-only result from QueryPatternBryan C. Mills
This allows a single QueryPattern call to resolve a path that could be either a package or a module. It is important to be able to make a single QueryPattern call — rather than a QueryPattern followed by a Query for the specific module path — to provide appropriate fallback behavior: if the proxy returns package results but does not contain a module result, we don't want to fall back to the next proxy to look for the (probably-nonexistent) module. For #37438 Change-Id: I419b8bb3ab4565f443bb5cee9a8b206f453b9801 Reviewed-on: https://go-review.googlesource.com/c/go/+/266657 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-05cmd/go: revert "add GOVCS setting to control version control usage"Dmitri Shuralyov
This reverts CL 266420. Reason for revert: tests aren't passing on linux-{386,amd64}-longtest. Change-Id: Icec47cded795a51ef7569dfb2d93d9211b4fb578 Reviewed-on: https://go-review.googlesource.com/c/go/+/267799 Trust: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-05cmd/go: add GOVCS setting to control version control usageRuss Cox
The go command runs commands like git and hg to download modules. In the past, we have had problems with security bugs in version control systems becoming security bugs in “go get”. The original modules draft design removed use of these commands entirely, saying: > We want to move away from invoking version control tools such as bzr, > fossil, git, hg, and svn to download source code. These fragment the > ecosystem: packages developed using Bazaar or Fossil, for example, are > effectively unavailable to users who cannot or choose not to install > these tools. The version control tools have also been a source of > exciting security problems. It would be good to move them outside the > security perimeter. The removal of these commands was not possible in the end: being able to fetch directly from Git repos is too important, especially for closed source. But the security exposure has not gone away. We remain vulnerable to problems in VCS systems, especially the less scrutinized ones. This change adds a GOVCS setting to let users control which version control systems are allowed by default. It also changes the default allowed version control systems to git and hg for public code and any version control system for private code (import path or module path matched by the GOPRIVATE setting). See the changes in alldocs.go for detailed documentation. See #41730 for proposal and discussion. Fixes #41730. Change-Id: I1999ddf7445b36a7572965be5897c7a1ff7f4265 Reviewed-on: https://go-review.googlesource.com/c/go/+/266420 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-16cmd/go/internal/modload: avoid using the global build list in QueryPatternBryan C. Mills
The Query function allows the caller to specify the current version of the requested module, but the QueryPattern function is missing that parameter: instead, it always assumes that the current version is the one selected from the global build list. This change removes that assumption, instead adding a callback function to determine the current version. (The callback is currently invoked once per candidate module, regardless of whether that module exists, but in a future change we can refactor it to invoke the callback only when needed.) For #36460 For #40775 Change-Id: I001a4a8ab24f5b4fcc66a670d9bd305b47e948ba Reviewed-on: https://go-review.googlesource.com/c/go/+/261640 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2020-10-16cmd/go/internal/modload: allow 'go get' to use replaced versionsBryan C. Mills
'go mod tidy' has been able to use replaced versions since CL 152739, but 'go get' failed for many of the same paths. Now that we are recommending 'go get' more aggressively due to #40728, we should make that work too. In the future, we might consider factoring out the new replacementRepo type so that 'go list' can report the new versions as well. For #41577 For #41416 For #37438 Updates #26241 Change-Id: I9140c556424b584fdd9bdd0a747842774664a7d8 Reviewed-on: https://go-review.googlesource.com/c/go/+/258220 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-06cmd/go/internal/get: improve -insecure deprecation docswitchard
Updates #37519 Change-Id: I212607f1839b729d7da24b1258e56997b13ad830 GitHub-Last-Rev: db6d3c835bdf867a0b18f115276210e3a05902ed GitHub-Pull-Request: golang/go#41613 Reviewed-on: https://go-review.googlesource.com/c/go/+/257157 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Jay Conrod <jayconrod@google.com> Trust: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-22cmd/go/internal/modget: if building packages, only update go.mod if the ↵Bryan C. Mills
build succeeds Fixes #41315 Change-Id: I5b18a0c2d1d72ff556a882e862b95133deb3ef98 Reviewed-on: https://go-review.googlesource.com/c/go/+/255970 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-22cmd/go/internal/modget: diagnose missing transitive dependenciesBryan C. Mills
For #41315 Change-Id: I3989bcb051ae57dd2d8f89759d241d4cdce49659 Reviewed-on: https://go-review.googlesource.com/c/go/+/255969 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-22cmd/go: add a '-e' flag to 'mod tidy' and 'mod vendor'Bryan C. Mills
This flag, like the -e flag to 'go list', instructs the command to make a best effort to continue in spite of errors for specific packages. Fixes #26603 Change-Id: I5ee2f50c71870ae8ef3f9b3e5b045474adcca525 Reviewed-on: https://go-review.googlesource.com/c/go/+/255960 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-22cmd/go/internal/modget: consolidate Load entrypointsBryan C. Mills
This change replaces ImportPaths, ImportPathsQuiet, LoadALL, and LoadVendor with a single LoadPackages function, with a LoadOpts struct that more clearly documents the variations in behavior. It also eliminates the cmd/go/internal/load.ImportPaths function, which was undocumented and had only one call site (within its own package). The modload.LoadTests global variable is subsumed by a field in the new LoadOpts struct, and is no longer needed for callers that invoke LoadPackages directly. It has been (temporarily) replaced with a similar global variable, load.ModResolveTests, which can itself be converted to an explicit, local argument. For #37438 For #36460 Updates #40775 Fixes #26977 Change-Id: I4fb6086c01b04de829d98875db19cf0118d40f8c Reviewed-on: https://go-review.googlesource.com/c/go/+/255938 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-21cmd/go/internal/get: warn about -insecure deprecationwitchard
Adds deprecation warning for -insecure flag on go get in both modules and GOPATH mode. Updates #37519. Change-Id: Ie2efeeb4a91e6dda92955295969e9715314ae50e GitHub-Last-Rev: a9ebe21fe067baa12391ad4f9357d8e5b0cf7051 GitHub-Pull-Request: golang/go#41497 Reviewed-on: https://go-review.googlesource.com/c/go/+/255882 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Trust: Michael Matloob <matloob@golang.org> Trust: Jay Conrod <jayconrod@google.com>
2020-09-15cmd/go/internal/modget: factor out functions for argument resolutionBryan C. Mills
For #37438 For #41315 For #36460 Change-Id: I17041c35ec91ff6ffb547e0f32572673d191b1ed Reviewed-on: https://go-review.googlesource.com/c/go/+/254820 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-15cmd/go/internal/modget: warn about unmatched packages exactly onceBryan C. Mills
Due to an inverted condition, we were emitting a "matched no packages" warning twice in some cases and not at all in others. For #41315 Change-Id: I472cd2d4f75811c8734852f2bdd7346f4c612816 Reviewed-on: https://go-review.googlesource.com/c/go/+/254819 Trust: Bryan C. Mills <bcmills@google.com> Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion