cmd/go/internal/modload: replace the global buildList with structured requirements

This is intended to be a pure-refactoring change, with very little
observable change in the behavior of the 'go' command. A few error
messages have prefixes changed (by virtue of being attached to
packages or modules instead of the build list overall), and
'go list -m' (without arguments) no longer loads the complete module
graph in order to provide the name of the (local) main module.

The previous modload.buildList variable contained a flattened build
list, from which the go.mod file was reconstructed using various
heuristics and metadata cobbled together from the original go.mod
file, the package loader (which was occasionally constructed without
actually loading packages, for the sole purpose of populating
otherwise-unrelated metadata!), and the updated build list.

This change replaces that variable with a new package-level variable,
named "requirements". The new variable is structured to match the
structure of the go.mod file: it explicitly specifies the roots of the
module graph, from which the complete module graph and complete build
list can be reconstructed (and cached) on demand. Similarly, the
"direct" markings on the go.mod requirements are now stored alongside
the requirements themselves, rather than side-channeled through the
loader.

The requirements are now plumbed explicitly though the modload
package, with accesses to the package-level variable occurring only
within top-level exported functions. The structured requirements are
logically immutable, so a new copy of the requirements is constructed
whenever the requirements are changed, substantially reducing implicit
communication-by-sharing in the package.

For #36460
Updates #40775

Change-Id: I97bb0381708f9d3e42af385b5c88a7038e1f0556
Reviewed-on: https://go-review.googlesource.com/c/go/+/293689
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>

1 files changed, 10 insertions, 2 deletions

diff --git a/src/cmd/go/internal/modcmd/download.go b/src/cmd/go/internal/modcmd/download.go
index e7d3d869cb..32c5b7f340 100644
--- a/src/cmd/go/internal/modcmd/download.go
+++ b/src/cmd/go/internal/modcmd/download.go
@@ -137,7 +137,8 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
 	listRetractions := false
 	type token struct{}
 	sem := make(chan token, runtime.GOMAXPROCS(0))
-	for _, info := range modload.ListModules(ctx, args, listU, listVersions, listRetractions) {
+	infos, infosErr := modload.ListModules(ctx, args, listU, listVersions, listRetractions)
+	for _, info := range infos {
 		if info.Replace != nil {
 			info = info.Replace
 		}
@@ -188,5 +189,12 @@ func runDownload(ctx context.Context, cmd *base.Command, args []string) {
 	}
 
 	// Update go.mod and especially go.sum if needed.
-	modload.WriteGoMod()
+	modload.WriteGoMod(ctx)
+
+	// If there was an error matching some of the requested packages, emit it now
+	// (after we've written the checksums for the modules that were downloaded
+	// successfully).
+	if infosErr != nil {
+		base.Errorf("go mod download: %v", infosErr)
+	}
 }