| Age | Commit message (Collapse) | Author |
|---|
|
targets
This change replaces the Target variable that represents the main module
and the pathPrefix and inGorootSrc which provide other information about
the main module with a single MainModules value that represents multiple
main modules and holds their path prefixes, module roots, and whether
they are in GOROOT/src. In cases where the code checks Target or its
previously associated variables, the code now checks or iterates over
MainModules. In some cases, the code still assumes a single main module
by calling MainModules.MustGetSingleMainModule. Some of those cases are
correct: for instance, there is always only one main module for
mod=vendor. Other cases are accompanied with TODOs and will have to be
fixed in future CLs to properly support multiple main modules.
This CL (and other cls on top of it) are planned to be checked into a
branch to allow for those evaluating the workspaces proposal to try it
hands on.
For #45713
Change-Id: I3b699e1d5cad8c76d62dc567b8460de8c73a87ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/334932
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
The new Graph type implements an incremental version of the MVS
algorithm, with requirements pushed in by the caller instead of pulled
by an internal MVS traversal.
To avoid redundancy going forward (and to ensure adequate test
coverage of the incremental implementation), the existing buildList
function is reimplemented in terms of Graph.
For #36460
Change-Id: Idd0b6ab8f17cc41d83a2a4c25a95f82e9ce1eab0
Reviewed-on: https://go-review.googlesource.com/c/go/+/244760
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
|
|
Previously, mvs.Downgrade could introduce spurious dependencies if the
downgrade computed for one module lands on a “hidden” version (such as
a pseudo-version) due to a requirement introduced by the downgrade for
another module.
To eliminate those spurious dependencies, we can add one more call to
BuildList to recompute the “actual” downgraded versions, and then
including only those actual versions in the final call to BuildList.
For #36460
Change-Id: Icc6b54aa004907221b2bcbbae74598b0e4100776
Reviewed-on: https://go-review.googlesource.com/c/go/+/294294
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
|
|
Reqs currently combines requirements with upgrades and downgrades.
However, only Upgrade needs the Upgrade method, and only Downgrade
needs the Previous method.
When we eventually add lazy loading, the lazily-loaded module graph
will not be able to compute upgrades and downgrades, so the
implementation work from here to there will be clearer if we are
explicit about which are still needed.
For #36460
Change-Id: I7bf8c2a84ce6bc4ef493a383e3d26850e9a6a6c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/290771
Trust: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
mvs.Downgrade is pretty clearly intended to match Algorithm 4 from the
MVS blog post (https://research.swtch.com/vgo-mvs#algorithm_4).
Per the blog post:
“Downgrading one module may require downgrading other modules, but we
want to downgrade as few other modules as possible. … To avoid an
unnecessary downgrade to E 1.1, we must also add a new requirement on
E 1.2. We can apply Algorithm R to find the minimal set of new
requirements to write to go.mod.”
mvs.Downgrade does not match that behavior today: it fails to retain
the selected versions of transitive dependencies that are not implied
by downgraded direct dependencies of the target (module E in the
post). This bug is currently masked by the fact that we only call
Downgrade today with a *modload.mvsReqs, for which the Required method
happens to return the complete build list — rather than only the
direct dependencies as documented for the mvs.Reqs interface.
For #36460
Change-Id: If9c8f413b156b5f67c02787d9359394e169951b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/287633
Trust: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Req is supposed to return “a minimal requirement list”
that includes each of the module paths listed in base.
Currently, if base contains duplicates Req emits duplicates,
and a list containing duplicates is certainly not minimal.
That, in turn, requires callers to be careful to deduplicate the base
slice, and there are multiple callers that are already quite
complicated to reason about even without the added complication of
deduplicating slices.
For #36460
Change-Id: I391a1dc0641fe1dd424c16b7a1082da0d00c7292
Reviewed-on: https://go-review.googlesource.com/c/go/+/287632
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
This has no impact on the resulting build list, but provides clearer
diagnostics if reqs.Required returns an error for one of the upgraded
modules.
For #37438
Change-Id: I5cd8f72a9b7b9a0b185e1a728f46fefbd2f09b4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/266897
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
|
|
itself
Also clean up the test assertions, and add a check for assertions
missing function invocations (there was one).
For #37438
Change-Id: Iafbfeae2c25217eac894181e01480b25b7cffbd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/266859
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
|
|
For #37438
Change-Id: Icb28035ae4027aa09d8959d4ac2f4b94a6c843a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/266339
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
|
|
Also factor out BuildListError to a separate file.
For #36460
Change-Id: Ibd1143893b09a2bbef659bea1e8c5dd35184a7ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/247764
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
When we print the stack from a BuildListError, we print the main
module first and the error last. That was the opposite of the order in
which in was stored in memory, leading to (arguably) more complex code
and (definitely) my own inability to reason about the contents of the
slice.
For now, it's still more convenient to construct the stack reversed,
so we do that and then reverse it before packing it into the error.
For #36460
Change-Id: I6312fb67b2ad9bf9b64071fe829854833208bad7
Reviewed-on: https://go-review.googlesource.com/c/go/+/244759
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
The version "" denotes the main module, which has no version. The
mvs.Reqs interface documentation hints this is allowed, but it's not
obvious from the implementation in modload.mvsReqs.Max.
Also, replace a related TODO with a comment in mvs.Downgrade.
Fixes #39042
Change-Id: I11e10908c9b3d8c2283eaa5c04bd8e1b936851fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/234003
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
This change deletes several internal packages, replaces imports to
them with the equivalent golang.org/x/mod packages, updates x/mod, and
re-runs 'go mod vendor'.
Packages are replaced as follows:
cmd/go/internal/modfile → golang.org/x/mod/modfile
cmd/go/internal/module → golang.org/x/mod/module
cmd/go/internal/semver → golang.org/x/mod/semver
cmd/go/internal/sumdb → golang.org/x/mod/sumdb
cmd/go/internal/dirhash → golang.org/x/mod/sumdb/dirhash
cmd/go/internal/note → golang.org/x/mod/sumdb/note
cmd/go/internal/tlog → golang.org/x/mod/sumdb/tlog
Updates #31761
Fixes #34924
Change-Id: Ie3bf677bb0be49af969f654a0214243a6547eb57
Reviewed-on: https://go-review.googlesource.com/c/go/+/202698
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
modload.MinReqs was passing modload.buildList to mvs.Reqs explicitly,
apparently as an optimization. However, we do not always have the
invariant that modload.buildList is complete: in particular, 'go mod
tidy' begins by reducing modload.buildList to only the set of modules
that provide packages to the build, which may be substantially smaller
than the final build list.
Other operations, such as 'go mod graph', do not load the entire
import graph, and therefore call Reqs with the unreduced build list.
Since Reqs retains modules according to a post-order traversal of the
list, an incomplete list may produce a different traversal order — and
therefore a different minimal solution, when multiple minimal
solutions exist. That caused 'go mod tidy' to produce different output
from other 'go' subcommands when certain patterns of dependencies are
present.
Since passing in the build list is only an optimization anyway, remove
the parameter and recompute the actual (complete) list at the
beginning of mvs.Reqs itself. That way, it is guaranteed to be
complete and in canonical order.
Fixes #34086
Change-Id: I3101bb81a1853c4a5e773010da3e44d2d90a570c
Reviewed-on: https://go-review.googlesource.com/c/go/+/193397
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
versions already in the graph
Fixes #31248
Change-Id: Ia54f2098c3b85549681198a487a31e8ce8fc59eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/186557
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Fixes #29773
Updates #31248
Change-Id: Ic1923119c8cf3a60c586df1b270c3af0c9095f29
Reviewed-on: https://go-review.googlesource.com/c/go/+/186537
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Previously, most operations involving pseudo-versions allowed any
arbitrary combination of version string and date, and would resolve to
the underlying revision (typically a Git commit hash) as long as that
revision existed.
There are a number of problems with that approach:
• The pseudo-version participates in minimal version selection. If its
version prefix is inaccurate, the pseudo-version may appear to have
higher precedence that the releases that follow it, effectively
“pinning” the module to that commit. For release tags, module
authors are the ones who make the decision about release tagging;
they should also have control over the pseudo-version precedence
within their module.
• The commit date within the pseudo-version provides a total order
among pseudo-versions. If it is not accurate, the pseudo-version
will sort into the wrong place relative to other commits with the
same version prefix.
To address those problems, this change restricts the pseudo-versions
that the 'go' command accepts, rendering some previously
accepted-but-not-canonical versions invalid. A pseudo-version is now
valid only if all of:
1. The tag from which the pseudo-version derives points to the named
revision or one of its ancestors as reported by the underlying VCS
tool, or the pseudo-version is not derived from any tag (that is,
has a "vX.0.0-" prefix before the date string and uses the lowest
major version appropriate to the module path).
2. The date string within the pseudo-version matches the UTC timestamp
of the revision as reported by the underlying VCS tool.
3. The short name of the revision within the pseudo-version (such as a
Git hash prefix) is the same as the short name reported by the
underlying cmd/go/internal/modfetch/codehost.Repo. Specifically, if
the short name is a SHA-1 prefix, it must use the same number of
hex digits (12) as codehost.ShortenSHA1.
4. The pseudo-version includes a '+incompatible' suffix only if it is
needed for the corresponding major version, and only if the
underlying module does not have a go.mod file.
We believe that all releases of the 'go' tool have generated
pseudo-versions that meet these constraints. However, a few
pseudo-versions edited by hand or generated by third-party tools do
not. If we discover invalid-but-benign pseudo-versions in widely-used
existing dependencies, we may choose to add a whitelist for those
specific path/version combinations.
―
To work around invalid dependencies in leaf modules, users may add a
'replace' directive from the invalid version to its valid equivalent.
Note that the go command's go.mod parser automatically resolves commit
hashes found in 'replace' directives to the appropriate
pseudo-versions, so in most cases one can write something like:
replace github.com/docker/docker v1.14.0-0.20190319215453-e7b5f7dbe98c => github.com/docker/docker e7b5f7dbe98c
and then run any 'go' command (such as 'go list' or 'go mod tidy') to
resolve it to an appropriate pseudo-version. Note that the invalid
version will still be used in minimal version selection, so this use
of 'replace' directives is an incomplete workaround.
―
One of the common use cases for higher-than-tagged pseudo-versions is
for projects that do parallel development on release branches. For
example, if a project cuts a 'v1.2' release branch at v1.2.0, they may
want future commits on the main branch to show up as pre-releases for
v1.3.0 rather than for v1.2.1 — especially if v1.2.1 is already tagged
on the release branch. (On the other hand, a backport of a patch to
the v1.2 branch should not show up as a pre-release for v1.3.0.)
To address this use-case, module authors can make use of our existing
support for pseudo-versions derived from pre-release tags: if the
author adds an explicit pre-release tag (such as 'v1.3.0-devel') to
the first commit after the branch, then the pseudo-versions for that
commit and its descendents will be derived from that tag and will sort
appropriately in version selection.
―
Updates #27171
Fixes #29262
Fixes #27173
Fixes #32662
Fixes #32695
Change-Id: I0d50a538b6fdb0d3080aca9c9c3df1040da1b329
Reviewed-on: https://go-review.googlesource.com/c/go/+/181881
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Currently, 'go get -u' and 'go get -u=patch' avoid accidentally
downgrading modules by preventing upgrades in two cases:
1) If the current version is a prerelease that is semantically later
than the "latest" or "patch" version.
2) If the current version is a pseudoversion that is chronologically
newer than the "latest" or "patch" version.
With this change, 'go get m@latest' and 'go get m@patch' prevent
downgrades using the same checks.
Also: 'go get m@patch' now works if m is a module path but not a
package path (i.e., there is no package in the module root directory).
Fixes #30634
Fixes #32537
Change-Id: I916630c385b5f3ba7c13e0d65ba08f73a1a67829
Reviewed-on: https://go-review.googlesource.com/c/go/+/180337
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
When we downgrade a module (using 'go get m@none' or similar), we
exclude versions of other modules that depend on it. We'll try
previous versions (in the "versions" list returned by the proxy or in
codeRepo.Versions for vcs) until we find a version that doesn't
require an excluded module version.
If older versions of a module are broken for some reason,
mvs.Downgrade currently panics. With this change, we ignore versions
with errors during downgrade.
A frequent cause of this is incompatible v2+ versions. These are
common if a repository tagged v2.0.0 before migrating to modules, then
tagged v2.0.1 with a go.mod file later. v2.0.0 is incorrectly
considered part of the v2 module.
Fixes #31942
Change-Id: Icaa75c5c93f73f18a400c22f18a8cc603aa4011a
Reviewed-on: https://go-review.googlesource.com/c/go/+/177337
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
If a user runs 'go get mod@vers' where the module transitively
requires itself at a newer version, 'go get' attempts to perform a
downgrade, which necessarily excludes the requested version of the
module.
Previously, we called mvs.BuildList with the requested module
version as the target. This panicked because BuildList doesn't allow
the target module (typically the main module) to require a newer
version of itself.
With this change, when we lose an upgrade due to a downgrade, we call
mvs.BuildList through a wrapper that treats the lost module version as
requirement of a synthetic root module, rather than the target
module. This avoids the panic.
This change also starts reporting errors when an upgraded module is
lost entirely (downgrades caused the module to be completely removed
from the build list).
Fixes #31491
Change-Id: I70ca261c20af7553cad2d3b840a1eaf3d18a4191
Reviewed-on: https://go-review.googlesource.com/c/go/+/177602
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
When we construct the build list by loading packages (e.g., in
"go build", "go list", or "go test"), we may load additional modules
not mentioned in the original build list. If we encounter an error
loading one of these modules, mvs.BuildList currently returns a
BuildListError with a chain of requirments. Unfortunately, this is not
helpful, since the graph is structured such that these missing modules
are direct requirements of the main module.
With this change, loader.load keeps track of the package that caused
each "missing" module to be added. If an error occurs in a missing
module, the chain of package imports is printed instead of the module
requirements.
Fixes #31475
Change-Id: Ie484814af42ceea3e85fedc38e705ba3a38cd495
Reviewed-on: https://go-review.googlesource.com/c/go/+/171859
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
mvs.BuildList reports errors with a chain of modules to make it
clear why the module where the error occurred was part of the
build. This is a little confusing with "go get -u" since there are
edges in the module graph for requirements and for updates.
With this change, we now print "requires" or "updates to" between
each module version in the chain.
Updates #30661
Change-Id: Ie689500ea86857e715b250b9e0cae0bc6686dc32
Reviewed-on: https://go-review.googlesource.com/c/go/+/171150
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
mvs.BuildList and functions that invoke it directly (UpgradeAll) now
return an *mvs.BuildListError when there is an error retrieving the
requirements for a module. This new error prints the chain of
requirements from the main module to the module where the error
occurred.
These errors come up most commonly when a go.mod file has an
unexpected module path or can't be parsed for some other reason. It's
currently difficult to debug these errors because it's not clear where
the "bad" module is required from. Tools like "go list -m" and
"go mod why" don't work without the build graph.
Fixes #30661
Change-Id: I3c9d4683dcd9a5d7c259e5e4cc7e1ee209700b10
Reviewed-on: https://go-review.googlesource.com/c/go/+/166984
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
Updates #28102
Change-Id: Iee1ff64c7720108d6d26bfbff60ea51877093960
Reviewed-on: https://go-review.googlesource.com/c/140862
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
|
|
This CL corresponds to CL 123361, the final manual CL in that repo,
making this the final manual sync.
All future commits will happen in this repo (the main Go repo),
and we'll update x/vgo automatically with a fixed patch+script.
Change-Id: I572243309c1809727604fd704705a23c30e85d1a
Reviewed-on: https://go-review.googlesource.com/123576
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
This CL corresponds to golang.org/cl/118096 (7fbc8df48a7)
in the vgo repo.
It copies the bulk of the code from vgo back into the main repo,
but completely disabled - vgo.Init is a no-op and vgo.Enabled
returns false unconditionally.
The point of this CL is to make the two trees easier to diff and
to make future syncs smaller.
Change-Id: Ic34fd5ddd8272a70c5a3b3437b5169e967d0ed03
Reviewed-on: https://go-review.googlesource.com/118095
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
