aboutsummaryrefslogtreecommitdiff
path: root/src/cmd/go/internal/modload/import.go
AgeCommit message (Collapse)Author
2021-08-11[dev.cmdgo] cmd/go: remove modload.ModRoot functionMichael Matloob
In some cases, ModRoot was being called in a multi module context. In those cases, pass in the correct main module. In other cases, a mainModule variable was already available, so call MainModules.ModRoot on that mainModule variable to make it more clear. In yet other cases ModRoot is just needed to determine GoMod, so determine modroot from the current directory in those cases. For #45713 Change-Id: I8c8aa633cfae40d0c8740bdbf985f2b60c9daf2c Reviewed-on: https://go-review.googlesource.com/c/go/+/339171 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-07-30[dev.cmdgo] cmd/go: thread through modroots providing replacementsMichael Matloob
modload.Replacement and modload.resolveReplacement now also return the modroot of the module providing a replacement so that we can correctly construct the path of a replaced module (because the path in the module.Version is relative to the modroot). For #45713 Change-Id: I8c69ccbcc1f40201071e35fcf93d6b5d0ed4cdf7 Reviewed-on: https://go-review.googlesource.com/c/go/+/334941 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-07-30[dev.cmdgo] cmd/go: provide a more helpful missing required module error in ↵Michael Matloob
workspaces If the user is in a workspace, they might not be in the main module they need to run go get from to add a module that provides a missing dependency. Figure out what that module is from the import stack (there might be multiple but we pick according to the stack computed by the loader for errors) and tell the user to cd to that directory first in the message. Change-Id: I7c919eb61ea3dd122334ff1acd2d7e817cad4b25 Reviewed-on: https://go-review.googlesource.com/c/go/+/334940 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-07-28[dev.cmdgo] cmd/go: maintain a go.work.sum fileMichael Matloob
This change causes the go command to maintain a separate go.work.sum file when in workspace mode rather than using the go.sum files from the individual modules. This isn't quite what the proposal spec specifies, which is that the sums that don't exist in any of the workspace modules are added to go.work.sum rather than the necessary sums. That will be done in a future change. Change-Id: I528b9b153a93a4cd67c5af471ad6d5bd3628578b Reviewed-on: https://go-review.googlesource.com/c/go/+/334939 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-07-27[dev.cmdgo] cmd/go: fold index and modFile into MainModulesMichael Matloob
For #45713 Change-Id: I5e4b0ae16dcc9ba5ac30683370a3a1d3416e24f2 Reviewed-on: https://go-review.googlesource.com/c/go/+/334935 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-07-22[dev.cmdgo] cmd/go: replace Target with MainModules, allowing for multiple ↵Michael Matloob
targets This change replaces the Target variable that represents the main module and the pathPrefix and inGorootSrc which provide other information about the main module with a single MainModules value that represents multiple main modules and holds their path prefixes, module roots, and whether they are in GOROOT/src. In cases where the code checks Target or its previously associated variables, the code now checks or iterates over MainModules. In some cases, the code still assumes a single main module by calling MainModules.MustGetSingleMainModule. Some of those cases are correct: for instance, there is always only one main module for mod=vendor. Other cases are accompanied with TODOs and will have to be fixed in future CLs to properly support multiple main modules. This CL (and other cls on top of it) are planned to be checked into a branch to allow for those evaluating the workspaces proposal to try it hands on. For #45713 Change-Id: I3b699e1d5cad8c76d62dc567b8460de8c73a87ea Reviewed-on: https://go-review.googlesource.com/c/go/+/334932 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-06-24cmd/go: don't try to add replaced versions that won't be selectedBryan C. Mills
In Go 1.12, we added a heuristic to 'go mod tidy' to resolve packages by adding replaced-but-not-required modules before falling back to searching for modules from the network. Unfortunately, that heuristic fails when the replaced version is already lower than the selected version: adding such a module to the build list doesn't change the selected version of that module, and so it doesn't make progress toward resolving the missing package. Fixes #46659 Change-Id: I75e2387d5290e769f6b0fa1231dcc4605db68597 Reviewed-on: https://go-review.googlesource.com/c/go/+/330432 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-06-11cmd/go: remove hint when no module is suggestedConstantin Konstantinidis
Fixes #46528 Change-Id: I2453d321ece878ea7823865758aa4a16b3ed7fe8 Reviewed-on: https://go-review.googlesource.com/c/go/+/325430 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> Trust: Heschi Kreinick <heschi@google.com> Trust: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-30cmd/go: remove the special case for "unsafe" in importFromModulesBryan C. Mills
The comment for this special case claims: > There's no directory for import "C" or import "unsafe". However, there clearly is a directory for "unsafe" in GOROOT/src/unsafe, and all of our integration tests seem to pass without this special case. As far as I can tell, it's just confusing. Also note that the internal/goroot package explicitly considers package "unsafe" to be in the standard library; see CL 137435. For #36460 Change-Id: Ib857d18f731a7f3c911c1bd116a34e3a9b3d74a5 Reviewed-on: https://go-review.googlesource.com/c/go/+/315412 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-04-30cmd/go/internal/modload: avoid loading the full module graph for imports ↵Bryan C. Mills
satisfied by lazy roots For #36460 Change-Id: Ibdbaa893ded772617e22f12db7a0463604db5195 Reviewed-on: https://go-review.googlesource.com/c/go/+/308516 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-04-28cmd/go/internal/modload: in importFromModules, do not wrap module graph ↵Bryan C. Mills
errors in ImportMissingError If an error occurs in loading the module graph (such as a missing checksum for a relevant go.mod file), that error should be terminal and we should not look elsewhere to try to resolve the import. An ImportMissingError instructs the caller to do exactly that, so don't use that error type for this case. (This behavior is tested incidentally in a later CL in this stack.) For #36460 Change-Id: I963e39cc7fbc457c12a626c1402c0be29203d23b Reviewed-on: https://go-review.googlesource.com/c/go/+/314633 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-25cmd/go: attribute direct imports from indirect dependencies to the importing ↵Bryan C. Mills
package For #36460 Updates #40775 Change-Id: I833ee8ee733151aafcff508bf91d0e6e37c50032 Reviewed-on: https://go-review.googlesource.com/c/go/+/303869 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-03-25cmd/go/internal/modload: replace the global buildList with structured ↵Bryan C. Mills
requirements This is intended to be a pure-refactoring change, with very little observable change in the behavior of the 'go' command. A few error messages have prefixes changed (by virtue of being attached to packages or modules instead of the build list overall), and 'go list -m' (without arguments) no longer loads the complete module graph in order to provide the name of the (local) main module. The previous modload.buildList variable contained a flattened build list, from which the go.mod file was reconstructed using various heuristics and metadata cobbled together from the original go.mod file, the package loader (which was occasionally constructed without actually loading packages, for the sole purpose of populating otherwise-unrelated metadata!), and the updated build list. This change replaces that variable with a new package-level variable, named "requirements". The new variable is structured to match the structure of the go.mod file: it explicitly specifies the roots of the module graph, from which the complete module graph and complete build list can be reconstructed (and cached) on demand. Similarly, the "direct" markings on the go.mod requirements are now stored alongside the requirements themselves, rather than side-channeled through the loader. The requirements are now plumbed explicitly though the modload package, with accesses to the package-level variable occurring only within top-level exported functions. The structured requirements are logically immutable, so a new copy of the requirements is constructed whenever the requirements are changed, substantially reducing implicit communication-by-sharing in the package. For #36460 Updates #40775 Change-Id: I97bb0381708f9d3e42af385b5c88a7038e1f0556 Reviewed-on: https://go-review.googlesource.com/c/go/+/293689 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-24cmd/go: move psuedo-version and version sorting to x/modJay Conrod
Fixes #44969 Change-Id: I01e7b1cf73f0f506aa805bbfe4a9ccaed3d44efe Reviewed-on: https://go-review.googlesource.com/c/go/+/304229 Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-03-04cmd/go: clarify errors for commands run outside a moduleJay Conrod
The new error message tells the user what was wrong (no go.mod found) and directs them to 'go help modules', which links to tutorials. Fixes #44745 Change-Id: I98f31fec4a8757eb1792b45491519da4c552cb0f Reviewed-on: https://go-review.googlesource.com/c/go/+/298650 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-13cmd/go: add hints to more missing sum error messagesJay Conrod
When a command fails due to a module zip sum missing from go.sum, if the module is in the build list, the go command will print a 'go mod download' command the user can run to fix it. Previously, a hint was only printed if the module provided a package in 'all'. We don't print a 'go get' hint, since we may not want to add a new requirement to go.mod. Fixes #43572 Change-Id: I88c61b1b42ad56c04e4482f6a1bb97ce758aaeff Reviewed-on: https://go-review.googlesource.com/c/go/+/282712 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com>
2021-01-08cmd/go: make hints in error messages more consistentJay Conrod
* All commands the user can run to fix the problem now appear alone on a separate line after a tab. * Removed -d from 'go get' commands. * Replaced 'go mod tidy' with 'go mod download $modpath' when a package might be provided by a module missing a sum. * Errors about 'path@version' syntax are more explicit. Fixes #29415 Fixes #42087 Fixes #43430 Fixes #43523 Change-Id: I4427c2c4506a727a2c727d652fd2d506bb134d3b Reviewed-on: https://go-review.googlesource.com/c/go/+/282121 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-01-06cmd/go: make module suggestion more friendlyBaokun Lee
We are trying to avoid by not automatically updating go.mod. The suggestion should be that users actually add the dependencies they need, and the command in an easily copy-pastable form now. Fixes: #43430 Change-Id: I2227dab498fcd8d66184c94ebe9e776629ccadfd Reviewed-on: https://go-review.googlesource.com/c/go/+/280713 Run-TryBot: Baokun Lee <bk@golangcn.org> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Trust: Bryan C. Mills <bcmills@google.com>
2021-01-05cmd/go: set cfg.BuildMod to "readonly" by default with no module rootJay Conrod
modload.Init now sets the default value for -mod if it wasn't set explicitly. This happens before go.mod is loaded, so modload.LoadModFile sets the default value again in order to enable automatic vendoring. Previously, cfg.BuildMod wasn't set at all if LoadModFile wasn't called, as is the case for commands that run outside of a module root. This problem only affected 'go install pkg@version' since other commands are either forbidden in module mode or run with -mod=mod (like 'go get' and 'go mod' subcommands). This change also suppresses "missing sum" errors when -mod=readonly is enabled and there is no module root. Fixes #43278 Related #40278 Change-Id: I6071cc42bc5e24d0d7e84556e5bfd8e368e0019d Reviewed-on: https://go-review.googlesource.com/c/go/+/279490 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-20cmd/go: support overlaying go.mod filesMichael Matloob
This change updates the lockedfile package to open files using the new fsys.OpenFile function. The logic of fsys.Open has been moved into fsys.OpenFile, and fsys.Open is now just a light wrapper around it. For #39958 Change-Id: I552f1a45ac00ac06b5812008d17a61e610b4b113 Reviewed-on: https://go-review.googlesource.com/c/go/+/266797 Trust: Michael Matloob <matloob@golang.org> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-05cmd/go/internal/modget: resolve paths at the requested versionsBryan C. Mills
Previously, we resolved each argument to 'go get' to a package path or module path based on what was in the build list at existing versions, even if the argument specified a different version explicitly. That resulted in bugs like #37438, in which we variously resolved the wrong version or guessed the wrong argument type for what is unambiguously a package argument at the requested version. We were also using a two-step upgrade/downgrade algorithm, which could not only upgrade more that is strictly necessary, but could also unintentionally upgrade *above* the requested versions during the downgrade step. This change instead uses an iterative approach, with an explicit disambiguation step for the (rare) cases where an argument could match the same package path in multiple modules. We use a hook in the package loader to halt package loading as soon as an incorrect version is found — preventing over-resolving — and verify that the result after applying downgrades successfully obtained the requested versions of all modules. Making 'go get' be correct and usable is especially important now that we are defaulting to read-only mode (#40728), for which we are recommending 'go get' more heavily. While I'm in here refactoring, I'm also reworking the API boundary between the modget and modload packages. Previously, the modget package edited the build list directly, and the modload package accepted the edited build list without validation. For lazy loading (#36460), the modload package will need to maintain additional metadata about the requirement graph, so it needs tighter control over the changes to the build list. As of this change, modget no longer invokes MVS directly, but instead goes through the modload package. The resulting API gives clearer reasons in case of updates, which we can use to emit more useful errors. Fixes #37438 Updates #36460 Updates #40728 Change-Id: I596f0020f3795870dec258147e6fc26a3292c93a Reviewed-on: https://go-review.googlesource.com/c/go/+/263267 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2020-11-05cmd/go/internal/modload: return a module-only result from QueryPatternBryan C. Mills
This allows a single QueryPattern call to resolve a path that could be either a package or a module. It is important to be able to make a single QueryPattern call — rather than a QueryPattern followed by a Query for the specific module path — to provide appropriate fallback behavior: if the proxy returns package results but does not contain a module result, we don't want to fall back to the next proxy to look for the (probably-nonexistent) module. For #37438 Change-Id: I419b8bb3ab4565f443bb5cee9a8b206f453b9801 Reviewed-on: https://go-review.googlesource.com/c/go/+/266657 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-23cmd/go: change error message for missing import with unused replacementJay Conrod
In readonly mode, if a package is not provided by any module in the build list, and there is an unused replacement that contains the package, we now recommend a 'go get' command to add a requirement on the highest replaced version. Fixes #41416 Change-Id: Iedf3539292c70ea6ba6857433fd184454d9325da Reviewed-on: https://go-review.googlesource.com/c/go/+/263146 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com>
2020-10-23cmd/go: don't fetch files missing sums in readonly modeJay Conrod
If the go command needs a .mod or .zip file in -mod=readonly mode (now the default), and that file doesn't have a hash in the main module's go.sum file, the go command will now report an error before fetching the file, rather than at the end when failing to update go.sum. The error says specifically which entry is missing. If this error is encountered when loading the build list, it will suggest 'go mod tidy'. If this error is encountered when loading a specific package (an import or command line argument), the error will mention that package and will suggest 'go mod tidy' or 'go get -d'. Fixes #41934 Fixes #41935 Change-Id: I96ec2ef9258bd4bade9915c43d47e6243c376a81 Reviewed-on: https://go-review.googlesource.com/c/go/+/262341 Reviewed-by: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com>
2020-10-20all: update references to symbols moved from os to io/fsRuss Cox
The old os references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. For #41190. Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd Reviewed-on: https://go-review.googlesource.com/c/go/+/243907 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-10-16cmd/go/internal/modload: avoid using the global build list in QueryPatternBryan C. Mills
The Query function allows the caller to specify the current version of the requested module, but the QueryPattern function is missing that parameter: instead, it always assumes that the current version is the one selected from the global build list. This change removes that assumption, instead adding a callback function to determine the current version. (The callback is currently invoked once per candidate module, regardless of whether that module exists, but in a future change we can refactor it to invoke the callback only when needed.) For #36460 For #40775 Change-Id: I001a4a8ab24f5b4fcc66a670d9bd305b47e948ba Reviewed-on: https://go-review.googlesource.com/c/go/+/261640 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2020-10-16cmd/go/internal/modload: allow 'go get' to use replaced versionsBryan C. Mills
'go mod tidy' has been able to use replaced versions since CL 152739, but 'go get' failed for many of the same paths. Now that we are recommending 'go get' more aggressively due to #40728, we should make that work too. In the future, we might consider factoring out the new replacementRepo type so that 'go list' can report the new versions as well. For #41577 For #41416 For #37438 Updates #26241 Change-Id: I9140c556424b584fdd9bdd0a747842774664a7d8 Reviewed-on: https://go-review.googlesource.com/c/go/+/258220 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-16cmd/go/internal/modload: move fetch to import.goJay Conrod
From a comment in CL 262341. It makes more sense in import.go than in mvs.go. Change-Id: If4dfa1091077e110c5041bc849d99bc0be2bd8e2 Reviewed-on: https://go-review.googlesource.com/c/go/+/262780 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-10-13cmd/go: adjust ImportMissingError when module lookup is disabledBryan C. Mills
Previously, ImportMissingError said "cannot find module providing package …" even when we didn't even attempt to find such a module. Now, we write "no module requirement provides package …" when we did not attempt to identify a suitable module, and suggest either 'go mod tidy' or 'go get -d' as appropriate. Fixes #41576 Change-Id: I979bb999da4066828c54d99a310ea66bb31032ec Reviewed-on: https://go-review.googlesource.com/c/go/+/258298 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-06cmd/go: add basic support for overlaysMichael Matloob
This CL adds basic support for listing packages with overlays. The new cmd/go/internal/fs package adds an abstraction for communicating with the file system that will open files according to their overlaid paths, and provides functions to override those in the build context to open overlaid files. There is also some support for executing builds on packages with overlays. In cmd/go/internal/work.(*Builder).build, paths are mapped to their overlaid paths before they are given as arguments to tools. For #39958 Change-Id: I5ec0eb9ebbca303e2f1e7dbe22ec32613bc1fd17 Reviewed-on: https://go-review.googlesource.com/c/go/+/253747 Trust: Michael Matloob <matloob@golang.org> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-22cmd/go/internal/modload: eliminate QueryPackageBryan C. Mills
QueryPackage was a wrapper around QueryPattern with extra validation, called only once from within the same package. Most of that validation was already performed much earlier, in (*loader).Load. Inline the remaining validation and remove the needless indirection. For #36460 Change-Id: I108a01d416197db8f886889554e07b29f0c37f3f Reviewed-on: https://go-review.googlesource.com/c/go/+/256057 Trust: Bryan C. Mills <bcmills@google.com> Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-17cmd/go: flip relationship between load and modloadJay Conrod
Previously, modload imported load, but it mainly just did so in order to install callbacks to the modload API. This was important during vgo development, but there's no longer a strong reason to do this. Nothing modload imports strongly depends on load, so there's little danger of a dependency cycle. This change deletes the callbacks in load and instead, makes load call exported functions in modload directly. In the future, these functions may have different signatures than their GOPATH counterparts. Change-Id: Ifde5c3ffebd190b5bd184924ec447d272b936f27 Reviewed-on: https://go-review.googlesource.com/c/go/+/255719 Trust: Jay Conrod <jayconrod@google.com> Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-11cmd/go: refactor modload.Import for better -mod=readonly errorsJay Conrod
When -mod=readonly is set, Import will now allow imports from replacements without explicit requirements. With -mod=mod, this would add a new requirement but does not trigger a module lookup, so it's determinisitic. Before reporting an error for an unknown import with -mod=readonly, check whether the import is valid. If there's a typo in the import, that's more relevant. For #40728 Change-Id: I05e138ff76ba3d0eb2e3010c15589fa363deb8d3 Reviewed-on: https://go-review.googlesource.com/c/go/+/253745 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-09cmd/go/internal/modload: fix spurious import resolution errorBryan C. Mills
Due to a bug in CL 173017, if QueryPackages found multiple candidates for the given package and *at least* one of those candidates was not available to add, we would reject *all* such candidates — even those that were still viable. Now, we return the first viable candidate, and only return an error if *no* candidate is viable given the current build list. Fixes #41113 Change-Id: Idb2e77244be7c0f5dd511efb142c3059925d7336 Reviewed-on: https://go-review.googlesource.com/c/go/+/251446 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09cmd/go/internal/modload: rework import resolutionBryan C. Mills
modload.Import previously performed two otherwise-separable tasks: 1. Identify which module in the build list contains the requested package. 2. If no such module exists, search available modules to try to find the missing package. This change splits those two tasks into two separate unexported functions, and reports import-resolution errors by attaching them to the package rather than emitting them directly to stderr. That allows 'list' to report the errors, but 'list -e' to ignore them. With the two tasks now separate, it will be easier to avoid the overhead of resolving missing packages during lazy loading if we discover that some existing dependency needs to be promoted to the top level (potentially altering the main module's selected versions, and thus suppling packages that were previously missing). For #36460 Updates #26909 Change-Id: I32bd853b266d7cd231d1f45f92b0650d95c4bcbd Reviewed-on: https://go-review.googlesource.com/c/go/+/251445 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com> Reviewed-by: Michael Matloob <matloob@golang.org>
2020-08-26cmd/go/internal/modload: refactor version filtering for excludeJay Conrod
Query and other functions now accept an "allowed" function that returns an error (previously, the function returned a bool). If the error is equivalent to ErrDisallowed, it indicates the version is excluded (or, in a future CL, retracted). This provides predicates a chance to explain why a version is not allowed. When a query refers to a specific revision (by version, branch, tag, or commit name), most callers will not use the Allowed predicate. This allows commands like 'go list -m' and 'go mod download' to handle disallowed versions when explicitly requested. 'go get' will reject excluded versions though. When a query does not refer to a specific revision (for example, "latest"), disallowed versions will not be considered. When an "allowed" predicate returns an error not equivalent to ErrDisallowed, it may be ignored or returned, depending on the case. This never happens for excluded versions, but it may happen for retractions (in a future CL). This indicates a list of retractions could not be loaded. This frequently happens when offline, and it shouldn't cause a fatal or warning in most cases. For #24031 Change-Id: I4df6fb6bd60e3e0259e5b3b4bf71a307b4b32298 Reviewed-on: https://go-review.googlesource.com/c/go/+/228379 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-08-20cmd/go: do context propagation for tracing downloadsMichael Matloob
This change does context propagation (and only context propagation) necessary to add context to modfetch.Download and pkg.LoadImport. This was done by adding context to their callers, and then adding context to all call-sites, and then repeating adding context to callers of those enclosing functions and their callers until none were left. In some cases the call graph expansion was pruned by using context.TODOs. The next CL will add a span to Download. I kept it out of this change to avoid making it any larger (and harder to review) than it needs to be. Updates #38714 Change-Id: I7a03416e04a14ca71636d96f2c1bda2c4c30d348 Reviewed-on: https://go-review.googlesource.com/c/go/+/249021 Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-08-18cmd/go: revert 3 CLs affecting par.Work, context propagation, tracingDmitri Shuralyov
This reverts the following changes: • cmd/go: add tracing for querying and downloading from the proxy CL 242786, commit 1a3558341860357c2400e37773e5076bb3a51628 • cmd/go: do context propagation for tracing downloads CL 248327, commit c0cf190d226cc3defb71d17c01d0b45bf49a8a85 • cmd/go/internal: remove some users of par.Work CL 248326, commit f30044a03bc7cf107dbec03c02fb6d0072878252 Reason for revert: broke linux 386 and amd64 longtest builders. The problem started with CL 248326, but CL 248327 and CL 242786 are reverted as well due to conflicts. Updates #38714. Fixes #40861. Change-Id: I68496b4e5a27e47a42183553c3a645b288edac83 Reviewed-on: https://go-review.googlesource.com/c/go/+/249017 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com> Reviewed-by: Carlos Amedee <carlos@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-17cmd/go: do context propagation for tracing downloadsMichael Matloob
This change does context propagation (and only context propagation) necessary to add context to modfetch.Download and pkg.LoadImport. This was done by adding context to their callers, and then adding context to all call-sites, and then repeating adding context to callers of those enclosing functions and their callers until none were left. In some cases the call graph expansion was pruned by using context.TODOs. The next CL will add a span to Download. I kept it out of this change to avoid making it any larger (and harder to review) than it needs to be. Updates #38714 Change-Id: I5bf2d599aafef67334c384dfccd5e255198c85b4 Reviewed-on: https://go-review.googlesource.com/c/go/+/248327 Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-05-13cmd/go: do not ignore permission errors when matching patternsBryan C. Mills
While reviewing CL 228784, I noticed that various filepath.WalkFunc implementations within cmd/go were dropping non-nil errors. Those errors turn out to be significant, at least in some cases: for example, they can cause packages to appear to be missing when any parent of the directory had the wrong permissions set. (This also turned up a bug in the existing list_dedup_packages test, which was accidentally passing a nonexistent directory instead of the intended duplicate path.) Change-Id: Ia09a0a33aa7a966d9f132d3747d6c674a5370b2d Reviewed-on: https://go-review.googlesource.com/c/go/+/232579 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-02-28cmd/go: rationalize errors in internal/load and internal/modloadBryan C. Mills
This change is a non-minimal fix for #32917, but incidentally fixes several other bugs and makes the error messages much more ergonomic. Updates #32917 Updates #27122 Updates #28459 Updates #29280 Updates #30590 Updates #37214 Updates #36173 Updates #36587 Fixes #36008 Fixes #30992 Change-Id: Iedb26d2e0963697c130df5d0f72e7f83ec2dcf06 Reviewed-on: https://go-review.googlesource.com/c/go/+/185345 Reviewed-by: Michael Matloob <matloob@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-02-28cmd/go/internal/modload: make AmbiguousImportError an ImportPathErrorMichael Matloob
AmbiguousImportErrors will now be formatted like other ImportPathErrors: this means that now the ambiguously imported package won't be printed twice. Whereas the error message looked like the following: can't load package: package example.com/m/importy: ambiguous import: found package example.com/m/importy in multiple directories: $WORK/importy $WORK/vendor/example.com/m/importy It now looks like this: can't load package: ambiguous import: found package example.com/m/importy in multiple directories: $WORK/importy $WORK/vendor/example.com/m/importy Change-Id: I52a2074a6b3f5eb7d78d331d0852b7ea6b3735e6 Reviewed-on: https://go-review.googlesource.com/c/go/+/221457 Run-TryBot: Michael Matloob <matloob@golang.org> Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-02-25cmd/go: emit an error for extraneous files in GOROOT/src in module modeMichael Matloob
If there's a go file immediately in GOROOT/src, it was probably accidentally added by the user. Since that package shouldn't exist, return an error if a user tries to list it. We're only making this change for GOPATH mode because we don't want to break cases where users have been doing this historically, but want to fix this case for the future. This also leaves open the weird cases where files are placed directly in vendor directories. Fixes #36587 Change-Id: I9738e47b1e89fd5048cbb8dd28e44648834b8ea7 Reviewed-on: https://go-review.googlesource.com/c/go/+/216381 Run-TryBot: Michael Matloob <matloob@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-02-12cmd/go: eliminate empty '()' when passing -mod=readonly explicitly to 'go list'Bryan C. Mills
Discovered while investigating #37197. Updates #33326 Updates #34822 Change-Id: I38b136a4ee762a580a554125066b9778491295f2 Reviewed-on: https://go-review.googlesource.com/c/go/+/219237 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Michael Matloob <matloob@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-09cmd/go: make "finding" logging deterministicBryan C. Mills
In CL 204777, I made the "finding" messages in cachingRepo only print after a “longish” delay, on the theory that they would help diagnose slow or stuck fetches. However, as I've been testing Go 1.14 beta 1, I've found that these messages are mostly just noise, and the fact that they are so nondeterministic causes both confusion and test flakes (#35539). Moreover, it currently triggers once for each candidate module, when what we're usually after is actually a specific package within the module. So let's log the package operation unconditionally instead of the module fetches nondeterministically. Fixes #35539 Updates #26152 Change-Id: I41a1c772465b2f0b357d3402bc372b6907773741 Reviewed-on: https://go-review.googlesource.com/c/go/+/213679 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-12-20cmd/go: avoid erroneous canonicalization when trying to resolve imports ↵Bryan C. Mills
using replacements Updates #32700 Fixes #33795 Change-Id: I16897a0a2f3aa2f0b0bf8cf8252f3f39eef2e7ba Reviewed-on: https://go-review.googlesource.com/c/go/+/212200 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-06cmd/go/internal/modload: revert duplicate path suppression in ↵Bryan C. Mills
modload.ImportMissingError CL 210338 suppressed duplication for import paths mentioned in an ImportMissingError. Unfortunately, that broke one of the cases in cmd/go/internal/modload.TestImport, and the new error message is still kind of awkward anyway. Let's revert that part of the change — we can try again with more coverage for that case. Updates #35986 Change-Id: Ib0858aec4f89a7231e32c35ec876da80d80f2098 Reviewed-on: https://go-review.googlesource.com/c/go/+/210342 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-12-06cmd/go: include cfg.BuildModReason in 'import lookup disabled' errorsBryan C. Mills
This location was missed in CL 204521. Updates #33326 Updates #33848 Change-Id: I0ece6d9b37548d8abb54f79c69be5548a0428c76 Reviewed-on: https://go-review.googlesource.com/c/go/+/210341 Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-12-06cmd/go: avoid generating "malformed module path" errors for standard-library ↵Bryan C. Mills
paths If the path looks like it belongs in GOROOT/src and isn't there, we should mention that in the error message — instead of the fact that the path is not a valid module path, which the user likely already knows. Fixes #34769 Fixes #35734 Change-Id: I3589336d102e420a5ad3bf246816e29f3cbe6d71 Reviewed-on: https://go-review.googlesource.com/c/go/+/210339 Run-TryBot: Bryan C. Mills <bcmills@google.com> Reviewed-by: Jay Conrod <jayconrod@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-12-06cmd/go: reduce redundancy in direct mode lookup error messagesJay Conrod
get.RepoRootForImportPath now returns errors that satisfy load.ImportPathError in cases where the import path appears in the messages. (The import path probably should appear in all errors from this function, but this CL does not change these errors). Changed modfetch.notExistError to be a wrapper (with an Unwrap method) instead of a string. This means errors.As works with notFoundError and ImportPathError. ImportMissingError no longer prints the package path if it wraps an ImportPathError. TestMissingImportErrorRepetition no longer counts the package path within a URL (like https://...?go-get=1). Fixes #35986 Change-Id: I38f795191c46d04b542c553e705f23822260c790 Reviewed-on: https://go-review.googlesource.com/c/go/+/210338 Run-TryBot: Jay Conrod <jayconrod@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>