| Age | Commit message (Collapse) | Author |
|---|
|
Conflicts:
- src/cmd/go/internal/modload/init.go
- src/cmd/go/internal/modload/load.go
Merge List:
+ 2021-07-28 9eee0ed439 cmd/go: fix go.mod file name printed in error messages for replacements
+ 2021-07-28 b39e0f461c runtime: don't crash on nil pointers in checkptrAlignment
+ 2021-07-27 7cd10c1149 cmd/go: use .mod instead of .zip to determine if version has go.mod file
+ 2021-07-27 c8cf0f74e4 cmd/go: add missing flag in UsageLine
+ 2021-07-27 7ba8e796c9 testing: clarify T.Name returns a distinct name of the running test
+ 2021-07-27 33ff155970 go/types: preserve untyped constants on the RHS of a shift expression
+ 2021-07-26 840e583ff3 runtime: correct variable name in comment
+ 2021-07-26 bfbb288574 runtime: remove adjustTimers counter
+ 2021-07-26 9c81fd53b3 cmd/vet: add missing copyright header
+ 2021-07-26 ecaa6816bf doc: clarify non-nil zero length slice to array pointer conversion
+ 2021-07-26 1868f8296e crypto/x509: update iOS bundled roots to version 55188.120.1.0.1
+ 2021-07-25 849b791129 spec: use consistent capitalization for rune literal hex constants
+ 2021-07-23 0914646ab9 doc/1.17: fix two dead rfc links
+ 2021-07-22 052da5717e cmd/compile: do not change field offset in ABI analysis
+ 2021-07-22 798ec73519 runtime: don't clear timerModifiedEarliest if adjustTimers is 0
+ 2021-07-22 fdb45acd1f runtime: move mem profile sampling into m-acquired section
+ 2021-07-21 3e48c0381f reflect: add missing copyright header
+ 2021-07-21 48c88f1b1b reflect: add Value.CanConvert
+ 2021-07-20 9e26569293 cmd/go: don't add C compiler ID to hash for standard library
+ 2021-07-20 d568e6e075 runtime/debug: skip TestPanicOnFault on netbsd/arm
+ 2021-07-19 c8f4e6152d spec: correct example comment in Conversions from slice to array
+ 2021-07-19 1d91551b73 time: correct typo in documentation for UnixMicro
+ 2021-07-19 404127c30f cmd/compile: fix off-by-one error in traceback argument counting
+ 2021-07-19 6298cfe672 cmd/compile: fix typo in fatal message of builtinCall
+ 2021-07-19 49402bee36 cmd/{compile,link}: fix bug in map.zero handling
+ 2021-07-18 a66190ecee test/bench/go1: fix size for RegexpMatchMedium_32
+ 2021-07-18 650fc2117a text/scanner: use Go convention in Position doc comment
+ 2021-07-16 aa4e0f528e net/http: correct capitalization in cancelTimeBody comment
+ 2021-07-15 0941dbca6a testing: clarify in docs that TestMain is advanced
+ 2021-07-15 69728ead87 cmd/go: update error messages in tests to match CL 332573
+ 2021-07-15 c1cc9f9c3d cmd/compile: fix lookup package of redeclared dot import symbol
+ 2021-07-15 21a04e3335 doc/go1.17: mention GOARCH=loong64
+ 2021-07-14 2b00a54baf go/build, runtime/internal/sys: reserve GOARCH=loong64
+ 2021-07-14 60ddf42b46 cmd/go: change link in error message from /wiki to /doc.
+ 2021-07-13 d8f348a589 cmd/go: remove a duplicated word from 'go help mod graph'
+ 2021-07-12 a98589711d crypto/tls: test key type when casting
+ 2021-07-12 cfbd73ba33 doc/go1.17: editing pass over the "Compiler" section
+ 2021-07-09 ab4085ce84 runtime/pprof: call runtime.GC twice in memory profile test
+ 2021-07-08 296ddf2a93 net: filter bad names from Lookup functions instead of hard failing
+ 2021-07-08 ce76298ee7 Update oudated comment
+ 2021-07-08 2ca44fe221 doc/go1.17: linkify time.UnixMilli and time.UnixMicro
+ 2021-07-07 5c59e11f5e cmd/compile: remove special-casing of blank in types.sconv{,2}
+ 2021-07-07 b003a8b1ae cmd/compile: optimize types.sconv
+ 2021-07-07 11f5df2d67 cmd/compile: extract pkgqual from symfmt
+ 2021-07-07 991fd381d5 cmd/go: don't lock .mod and .sum files for read in overlay
+ 2021-07-07 186a3bb4b0 cmd/go/internal/modfetch/codehost: skip hg tests if no hg binary is present
+ 2021-07-07 00c00558e1 cmd/go/internal/modload: remove unused functions
+ 2021-07-07 f264879f74 cmd/go/internal/modload: fix an apparent typo in the AutoRoot comment
+ 2021-07-07 c96833e5ba doc: remove stale comment about arm64 port
Change-Id: Ide5fb43a0fcb5d998386ed60dfd6ced241238a44
|
|
This change causes the go command to maintain a separate go.work.sum
file when in workspace mode rather than using the go.sum files
from the individual modules. This isn't quite what the proposal
spec specifies, which is that the sums that don't exist in any
of the workspace modules are added to go.work.sum rather than
the necessary sums. That will be done in a future change.
Change-Id: I528b9b153a93a4cd67c5af471ad6d5bd3628578b
Reviewed-on: https://go-review.googlesource.com/c/go/+/334939
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
On Plan 9, locking a file requires a chmod call. In general, the go
command should not modify files in the overlay, even metadata. With
this change, we won't lock these files for reading.
The go command already reported errors when attempting to write these
files if they were in the overlay, but this change moves those checks
to the point of access for clearer error
messages. cmd/go/internal/lockedfile no longer imports
cmd/go/internal/fsys.
Fixes #44700
Change-Id: Ib544459dd6cf56ca0f7a27b3285f045f08040d7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/333012
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
The last primary usage of renameio was the WriteFile in
modfetch.WriteDiskCache. Because it's not guaranteed that the fsync in
WriteDiskCache will eliminate file corruption, and it slows down tests
on Macs significantly, inline that last usage, removing the fsync.
Also, remove the uses of renameio.Pattern. The ziphash file is no
longer written to a temporary location before being copied to its
final location, so that usage can just be cut. The remaining use is
for the zipfile . Remove the first because the files are no longer
written using the pattern anyway, so that the pattern variable has no
effect. Replace it with a local pattern variable that is also passed
to os.CreateTemp.
Change-Id: Icf3adabf2a26c37b82afa1d07f821a46b30d69ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/301889
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
|
|
cache.Trim, dowloadZip, rewriteVersionList, writeDiskCache all use
renameio.WriteFile to write their respective files to disk. For the
uses in cache.Trim and downloadZip, instead do of renameio.WriteFile,
do a truncate to the length of the file, then write the relevant bytes
so that a corrupt file (which would contain null bytes because of
the truncate) could be detected. For rewriteVersionList, use
lockedfile.Transform to do the write (which does a truncate as part of
the write too. writeDiskCache stays the same in this CL.
Also desete renameio methods that aren't used and remove the
renameio.WriteFile wrapper and just use renameio.WriteToFile which it
wraps.
There is a possibility of corrupt files in the cache (which was true
even before this CL) so later CLs will add facilities to clear corrupt
files in the cache.
Change-Id: I0d0bda40095e4cb898314315bf313e71650d8d25
Reviewed-on: https://go-review.googlesource.com/c/go/+/277412
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Previously, if an extracted module directory existed in the module
cache, but the corresponding ziphash file did not, if the sum was
missing from go.sum, we would not verify the sum. This caused 'go get'
not to write missing sums. 'go build' in readonly mode (now the
default) checks for missing sums and doesn't attempt to fetch modules
that can't be verified against go.sum.
With this change, when requesting the module directory with
modfetch.DownloadDir, if the ziphash file is missing, the go command
will re-hash the zip without downloading or re-extracting it again.
Note that the go command creates the ziphash file before the module
directory, but another program could remove it separately, and it
might not be present after a crash.
Fixes #44749
Change-Id: I64551e048a3ba17d069de1ec123d5b8b2757543c
Reviewed-on: https://go-review.googlesource.com/c/go/+/298352
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
Go already rejects relative paths in a couple environment variables,
It should reject relative paths in GOMODCACHE.
Fixes #43715
Change-Id: Id1ceff839c7ab21c00cf4ace45ce48324733a526
Reviewed-on: https://go-review.googlesource.com/c/go/+/284432
Run-TryBot: Baokun Lee <bk@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
Trust: Baokun Lee <bk@golangcn.org>
|
|
Module-related help pages now contain a brief summary and point to the
reference documentation at golang.org/ref/mod for details.
Help pages for commands like 'go get' still describe the basic usage
and summarize flags but don't provide as much background detail.
Fixes #41427
Fixes #43419
Change-Id: Icacd38e0f33c352c447cc5a496c99674493abde2
Reviewed-on: https://go-review.googlesource.com/c/go/+/282615
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)
Update the Go tree to use the preferred names.
As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.
ReadDir changes are in a separate CL, because they are not a
simple search and replace.
For #42026.
Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Now that filepath.WalkDir is available, it is more efficient
and should be used in place of filepath.Walk.
Update the tree to reflect best practices.
As usual, the code compiled with Go 1.4 during bootstrap is excluded.
(In this CL, that's only cmd/dist.)
For #42027.
Change-Id: Ib0f7b1e43e50b789052f9835a63ced701d8c411c
Reviewed-on: https://go-review.googlesource.com/c/go/+/267719
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
|
|
The go command runs commands like git and hg to download modules.
In the past, we have had problems with security bugs in version
control systems becoming security bugs in “go get”.
The original modules draft design removed use of these commands
entirely, saying:
> We want to move away from invoking version control tools such as bzr,
> fossil, git, hg, and svn to download source code. These fragment the
> ecosystem: packages developed using Bazaar or Fossil, for example, are
> effectively unavailable to users who cannot or choose not to install
> these tools. The version control tools have also been a source of
> exciting security problems. It would be good to move them outside the
> security perimeter.
The removal of these commands was not possible in the end: being able
to fetch directly from Git repos is too important, especially for
closed source. But the security exposure has not gone away.
We remain vulnerable to problems in VCS systems, especially the less
scrutinized ones.
This change adds a GOVCS setting to let users control which version
control systems are allowed by default.
It also changes the default allowed version control systems to git and hg
for public code and any version control system for private code
(import path or module path matched by the GOPRIVATE setting).
See the changes in alldocs.go for detailed documentation.
See #41730 for proposal and discussion.
Fixes #41730.
[Replay of CL 266420. See changes from Patch Set 1 for updates to fix
a few long tests.]
Change-Id: I4fe93804548956c42aea985368b4571bdb220f48
Reviewed-on: https://go-review.googlesource.com/c/go/+/267888
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
|
This reverts CL 266420.
Reason for revert: tests aren't passing on linux-{386,amd64}-longtest.
Change-Id: Icec47cded795a51ef7569dfb2d93d9211b4fb578
Reviewed-on: https://go-review.googlesource.com/c/go/+/267799
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
The go command runs commands like git and hg to download modules.
In the past, we have had problems with security bugs in version
control systems becoming security bugs in “go get”.
The original modules draft design removed use of these commands
entirely, saying:
> We want to move away from invoking version control tools such as bzr,
> fossil, git, hg, and svn to download source code. These fragment the
> ecosystem: packages developed using Bazaar or Fossil, for example, are
> effectively unavailable to users who cannot or choose not to install
> these tools. The version control tools have also been a source of
> exciting security problems. It would be good to move them outside the
> security perimeter.
The removal of these commands was not possible in the end: being able
to fetch directly from Git repos is too important, especially for
closed source. But the security exposure has not gone away.
We remain vulnerable to problems in VCS systems, especially the less
scrutinized ones.
This change adds a GOVCS setting to let users control which version
control systems are allowed by default.
It also changes the default allowed version control systems to git and hg
for public code and any version control system for private code
(import path or module path matched by the GOPRIVATE setting).
See the changes in alldocs.go for detailed documentation.
See #41730 for proposal and discussion.
Fixes #41730.
Change-Id: I1999ddf7445b36a7572965be5897c7a1ff7f4265
Reviewed-on: https://go-review.googlesource.com/c/go/+/266420
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
If the go command needs a .mod or .zip file in -mod=readonly mode
(now the default), and that file doesn't have a hash in the main
module's go.sum file, the go command will now report an error before
fetching the file, rather than at the end when failing to update
go.sum. The error says specifically which entry is missing.
If this error is encountered when loading the build list, it will
suggest 'go mod tidy'.
If this error is encountered when loading a specific package (an
import or command line argument), the error will mention that package
and will suggest 'go mod tidy' or 'go get -d'.
Fixes #41934
Fixes #41935
Change-Id: I96ec2ef9258bd4bade9915c43d47e6243c376a81
Reviewed-on: https://go-review.googlesource.com/c/go/+/262341
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
|
|
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.
For #41190.
Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
|
|
The old os references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
For #41190.
Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/243907
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
|
|
We generally don't care about errors in resolving a repo if the result
we're looking for is already in the module cache. Moreover, we can
avoid some expense in initializing the repo if all of the methods we
plan to call on it hit in the cache — especially when using
GOPROXY=direct.
This also incidentally fixes a possible (but rare) bug in Download:
we had forgotten to reset the downloaded file in case the Zip method
returned an error after writing a nonzero number of bytes.
For #37438
Change-Id: Ib64f10f763f6d1936536b8e1f7d31ed1b463e955
Reviewed-on: https://go-review.googlesource.com/c/go/+/259158
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Previously by default, we extracted modules to a temporary directory,
then renamed it into place. This failed with ERROR_ACCESS_DENIED on
Windows if another process (usually an anti-virus scanner) opened
files in the temporary directory.
Since Go 1.15, users have been able to set
GODEBUG=modcacheunzipinplace=1 to opt into new behavior: we extract
modules at their final location, and we create and later delete a
.partial file to prevent the directory from being used if we crash.
.partial files are recognized by Go 1.14.2 and later.
With this change, the new behavior is the only behavior.
modcacheunzipinplace is no longer recognized.
Fixes #36568
Change-Id: Iff19fca5cd6eaa3597975a69fa05c4cb1b834bd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/258798
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
module.VersionErrors
Better still would be to avoid passing around module.Version instances
with invalid Version strings in the first place, so that any time we
see a module.Version we know that it is actually a version of a module
(and not a structurally-similar datum with something else tacked on to
one of the fields). But that's a bigger cleanup for which I don't
currently have enough bandwidth.
Fixes #41060
Change-Id: I32fba5619105cbf67dd03691064c82b8ebb3ce18
Reviewed-on: https://go-review.googlesource.com/c/go/+/250951
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
|
|
This CL adds tracing spans for modload.queryPattern, modload.queryProxy,
modload.QueryPattern, modload.QueryPattern.queryModule,
modload.queryPrefixModules and modfetch.Download.
Updates #38714
Change-Id: I91af3f022a6e18ab8d9c1d9b0ccf4928b6c236bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/249022
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
This change does context propagation (and only context propagation)
necessary to add context to modfetch.Download and pkg.LoadImport.
This was done by adding context to their callers, and then
adding context to all call-sites, and then repeating adding
context to callers of those enclosing functions and their
callers until none were left. In some cases the call graph expansion
was pruned by using context.TODOs.
The next CL will add a span to Download. I kept it out of this
change to avoid making it any larger (and harder to review)
than it needs to be.
Updates #38714
Change-Id: I7a03416e04a14ca71636d96f2c1bda2c4c30d348
Reviewed-on: https://go-review.googlesource.com/c/go/+/249021
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
This reverts the following changes:
• cmd/go: add tracing for querying and downloading from the proxy
CL 242786, commit 1a3558341860357c2400e37773e5076bb3a51628
• cmd/go: do context propagation for tracing downloads
CL 248327, commit c0cf190d226cc3defb71d17c01d0b45bf49a8a85
• cmd/go/internal: remove some users of par.Work
CL 248326, commit f30044a03bc7cf107dbec03c02fb6d0072878252
Reason for revert: broke linux 386 and amd64 longtest builders.
The problem started with CL 248326, but CL 248327 and CL 242786
are reverted as well due to conflicts.
Updates #38714.
Fixes #40861.
Change-Id: I68496b4e5a27e47a42183553c3a645b288edac83
Reviewed-on: https://go-review.googlesource.com/c/go/+/249017
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
This CL adds tracing spans for modload.queryPattern, modload.queryProxy,
modload.QueryPattern, modload.QueryPattern.queryModule,
modload.queryPrefixModules and modfetch.Download.
Updates #38714
Change-Id: I537c7fa4f466c691c1b60ec73ef8a2277af49cd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/242786
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
This change does context propagation (and only context propagation)
necessary to add context to modfetch.Download and pkg.LoadImport.
This was done by adding context to their callers, and then
adding context to all call-sites, and then repeating adding
context to callers of those enclosing functions and their
callers until none were left. In some cases the call graph expansion
was pruned by using context.TODOs.
The next CL will add a span to Download. I kept it out of this
change to avoid making it any larger (and harder to review)
than it needs to be.
Updates #38714
Change-Id: I5bf2d599aafef67334c384dfccd5e255198c85b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/248327
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
modfetch.WriteGoSum now accepts a map[module.Version]bool parameter.
This is used to prevent some new sums from being saved to go.sum when
they would be removed by the next 'go mod tidy'. Previusly, sums were
saved for modules looked up during import resolution.
A new function, modload.TrimGoSum, is also introduced, which marks
sums for deletion. 'go mod tidy' now uses this. The new logic
distinguishes between go.mod sums and content sums, which lets 'go mod
tidy' delete sums for modules in the build graph but not the build
list.
Fixes #31580
Fixes #36260
Fixes #33008
Change-Id: I06c4125704a8bbc9969de05265967ec1d2e6d3e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/237017
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
go.modverify was renamed to go.sum before vgo was merged into
cmd/go. It's been long enough that we can safely drop support for it.
For #25525
Change-Id: If8da66280a0fb6a4d4db0b170700775523c18571
Reviewed-on: https://go-review.googlesource.com/c/go/+/240458
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
In cmd/go/internal/modfetch/fetch.go,
`checkModSum()` forgets Unlock before return, which may lead to deadlock.
https://github.com/golang/go/blob/876c1feb7d5e10a6ff831de9db19b9ff0ea92fa8/src/cmd/go/internal/modfetch/fetch.go#L514-L520
The fix is to add `goSum.mu.Unlock()` before return.
Change-Id: I855b1c1bc00aeada2c1e84aabb5328f02823007d
GitHub-Last-Rev: afeb3763dd1d08c4c15c4c4183fd5705da60dc4c
GitHub-Pull-Request: golang/go#38563
Reviewed-on: https://go-review.googlesource.com/c/go/+/229219
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Adds a GOMODCACHE environment variable that's used by cmd/go to determine the
location of the module cache. The default value of GOMODCACHE will be
GOPATH[0]/pkg/mod, the default location of the module cache before this change.
Replace the cmd/go/internal/modfetch.PkgMod variable which previously held the
location of the module cache with the new cmd/go/internal/cfg.GOMODCACHE
variable, for consistency with many of the other environment variables that
affect the behavior of cmd/go. (Most of the changes in this CL are due to
moving/renaming the variable.)
The value of cfg.GOMODCACHE is now set using a variable initializer. It was
previously set in cmd/go/internal/modload.Init.
The location of GOPATH/pkg/sumdb is unchanged by this CL. While it was
previously determined using the value of PkgMod, it now is determined
independently dirctly from the value of GOPATH[0].
Fixes #34527
Change-Id: Id4d31d217b3507d6057c8ef7c52af1a0606603e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/219538
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Previously, we extracted module zip files to temporary directories
with random names, then renamed them to their final locations. This
failed with ERROR_ACCESS_DENIED on Windows if any file in the
temporary was open. Antivirus programs did this occasionally. Retrying
the rename did not work (CL 220978).
With this change, we extract module zip files in place. We create a
.partial file alongside the .lock file to indicate a directory is not
fully populated, and we delete this at the end of the process.
Updates #36568
Change-Id: I75c09df879a602841f3459322c021896292b2fdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/221157
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
|
|
Currently, we extract module zip files to temporary directories, then
atomically rename them into place. On Windows, this can fail with
ERROR_ACCESS_DENIED if another process (antivirus) has files open
before the rename. In CL 220978, we repeated the rename operation in a
loop over 500 ms, but this didn't solve the problem for everyone.
A better solution will extract module zip files to their permanent
locations in the cache and will keep a ".partial" marker file,
indicating when a module hasn't been fully extracted (CL 221157).
This approach is not safe if current versions of Go access the module
cache concurrently, since the module directory is detected with a
single os.Stat.
In the interim, this CL makes two changes:
1. Flaky file system operations are repeated over 2000 ms to reduce
the chance of this error occurring.
2. cmd/go will now check for .partial files created by future
versions. If a .partial file is found, it will lock the lock file,
then remove the .partial file and directory if needed.
After some time has passed and Go versions lacking this CL are no
longer supported, we can start extracting module zip files in place.
Updates #36568
Change-Id: I467ee11aa59a90b63cf0e3e761c4fec89d57d3b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/221820
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
No test because this is difficult to reproduce, and such a test would
always be flaky.
Updates #36568
Change-Id: I8170410a7729ecc6f90baf8005444d6b1241185e
Reviewed-on: https://go-review.googlesource.com/c/go/+/220978
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
|
|
zip.Create is now used to filter and translate zip files from VCS tools.
zip.Unzip is now used instead of Unzip.
Fixes #35290
Change-Id: I4aa41b2e96bf147c09db43d1d189b8393cafb06f
Reviewed-on: https://go-review.googlesource.com/c/go/+/204917
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
This change is based on the previous discussion in CL 202442.
Fixes #34634
Change-Id: I1319aa26d5cfcd034bc576555787b3ca79968c38
Reviewed-on: https://go-review.googlesource.com/c/go/+/205637
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Suppress “finding” messages unless they are unusually slow, and
“extracting” messages always (they almost always occur conjunction
with “downloading”, which is already logged).
Log “found” messages for module dependencies added to satisfy missing
import paths.
Log top-level version changes in 'go get' when the selected version
is not identical to the version requested on the command line.
Updates #26152
Updates #33284
Change-Id: I4d0de60fab58d7cc7df8a2aff05c8b5b2220e626
Reviewed-on: https://go-review.googlesource.com/c/go/+/204777
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
This change deletes several internal packages, replaces imports to
them with the equivalent golang.org/x/mod packages, updates x/mod, and
re-runs 'go mod vendor'.
Packages are replaced as follows:
cmd/go/internal/modfile → golang.org/x/mod/modfile
cmd/go/internal/module → golang.org/x/mod/module
cmd/go/internal/semver → golang.org/x/mod/semver
cmd/go/internal/sumdb → golang.org/x/mod/sumdb
cmd/go/internal/dirhash → golang.org/x/mod/sumdb/dirhash
cmd/go/internal/note → golang.org/x/mod/sumdb/note
cmd/go/internal/tlog → golang.org/x/mod/sumdb/tlog
Updates #31761
Fixes #34924
Change-Id: Ie3bf677bb0be49af969f654a0214243a6547eb57
Reviewed-on: https://go-review.googlesource.com/c/go/+/202698
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
The -modcacherw flag is now registered in work.AddModCommonFlags,
which is called from work.AddBuildFlags, where it was registered
before. 'go mod' subcommands register the flag by calling
work.AddModCommonFlags directly.
Also, build commands now exit with an error if -modcacherw is set
explicitly (not in GOFLAGS) in GOPATH mode.
Updates #31481
Change-Id: I461e59a51ed31b006fff4d5c57c2a866be0bbf38
Reviewed-on: https://go-review.googlesource.com/c/go/+/202563
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
This change adds the '-modcacherw' build flag, which leaves
newly-created directories (but not the files!) in the module cache
read-write instead of making them unwritable.
Fixes #31481
Change-Id: I7c21a53dd145676627c3b51096914ce797991d99
Reviewed-on: https://go-review.googlesource.com/c/go/+/202079
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
When the -json flag is passed to go mod download,
the sumdb error is embedded in the json Error field.
Other errors for the same command behave this way as
well such as module not found. The fix is done by changing
base.Fatalf into proper error returns.
Fixes #34485
Change-Id: I2727a5c70c7ab03988cad8661894d0f8ec71a768
Reviewed-on: https://go-review.googlesource.com/c/go/+/197062
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
Updates #30748
Change-Id: I38a6cdc9c9b488fec579e6362a4284e26e0f526e
Reviewed-on: https://go-review.googlesource.com/c/go/+/189782
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
This CL makes the go command understand that
GOSUMDB=sum.golang.google.cn should connect
to that domain but expect to find a checksum database
signed by sum.golang.org there.
The host sum.golang.google.cn is not yet completely
configured; we hope it will be available in a few weeks.
Change-Id: Ie0fc4323f0c7084dda59bd3b45fc406717fa16d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/191137
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
|
|
Change-Id: Idbdcb78d40cf58d7c878cfeaa4a34ab580fe472f
Reviewed-on: https://go-review.googlesource.com/c/go/+/183626
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
It is too confusing to have to set GONOSUMDB and GONOPROXY
in common use cases, but one cannot be guaranteed to be a
subset of the other.
This CL adds GOPRIVATE, which takes the same kind of pattern list
but is defined as "these patterns are private (non-public) modules".
Today the implication is that GOPRIVATE is the default setting for
GONOSUMDB and GONOPROXY. If there are other accommodations
to make for private packages in the future or in other tools,
having this clear statement of intent will let us do that.
(For example maybe an IDE integration would hyperlink an import
path to godoc.org; consulting GOPRIVATE would be a reasonable
way to decide not to do that for certain imports. In contrast,
consulting GONOPROXY or GONOSUMDB clearly would not.)
Fixes #32184.
Change-Id: If54c12d353c7a0a5c0e0273764140cce3c154a02
Reviewed-on: https://go-review.googlesource.com/c/go/+/181719
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
Windows does not have atomic renames; instead, it produces one of a
handful of errors in case a read races with a rename.
CL 180219 added a utility function that retries those errors in most
cases; this change updates the locations that use renameio for writes
to also use the new renameio.ReadFile function for reads.
It remains possible for a renameio.ReadFile to fail with a spurious
ERROR_FILE_NOT_FOUND, but with retries in place for the other errors
(and practical limits on write concurrency) such failures are unlikely
in practice.
Fixes #32188
Change-Id: I78c81051cc871325c1e3229e696b921b0fcd865a
Reviewed-on: https://go-review.googlesource.com/c/go/+/180517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
|
|
the next proxy
Since we're mucking with error-propagation in modload.Query* anyway,
simplify the classification logic. Ensure that “module not found”
errors are reported as such in various places, since non-“not found”
errors terminate the module search.
Fixes #31785
Change-Id: Ie3ca5f4eec10a5f2a6037ec7e1c2cf47bd37a232
Reviewed-on: https://go-review.googlesource.com/c/go/+/177958
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
|
|
This CL changes the default module download and module verification mechanisms
to use the Go module mirror and Go checksum database run by Google.
See https://proxy.golang.org/privacy for the services' privacy policy.
(Today, that URL is a redirect to Google's standard privacy policy,
which covers these services as well. If we publish a more specific
privacy policy just for these services, that URL will be updated to
display or redirect to it.)
See 'go help modules' and 'go help modules-auth' for details (added in this CL).
To disable the mirror and checksum database for non-public modules:
go env -w GONOPROXY=*.private.net,your.com/*
go env -w GONOSUMDB=*.private.net,your.com/*
(If you are using a private module proxy then you'd only do the second.)
If you run into problems with the behavior of the go command when using
the Go module mirror or the Go checksum database, please file issues at
https://golang.org/issue/new, so that we can address them for the
Go 1.13 release.
For #25530.
This CL also documents GONOPROXY.
Fixes #32056.
Change-Id: I2fde82e071742272b0842efd9580df1a56947fec
Reviewed-on: https://go-review.googlesource.com/c/go/+/178179
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
Now renameio package creates file use ioutil.TempFile, which calls
OpenFile with mode 0600, we should support creates a file with given
permission bits.
Fixes #31871
Change-Id: I0436e9f7081f2fce18bf9f3b14d55b02d4d995fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/175958
Run-TryBot: Baokun Lee <nototon@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
When running go build with the flag -mod=readonly, it fails the build if
go.sum files requires updating. This ensures that CI/CD systems get a
complete go.sum file so that they'd never hit a notary,
assuming the CI/CD system passes the above flag.
I am not familiar with the entire codebase but I assume goSum.dirty
will always be true if go.sum has any missing lines.
Fixes #30667
Change-Id: I767d3b594055d8c10048f4c68e6687c94bb0545c
Reviewed-on: https://go-review.googlesource.com/c/go/+/166237
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
This CL adds support for consulting the Go checksum database
when downloading a module that is not already listed in go.sum.
The overall system is described at golang.org/design/25530-sumdb,
and this CL implements the functionality described specifically in
golang.org/design/25530-sumdb#command-client.
Although the eventual plan is to set GOPROXY and GOSUMDB to
default to a Google-run proxy serving the public Go ecosystem,
this CL leaves them off by default.
Fixes #30601.
Change-Id: Ie46140f93c6cc2d85573fbce0878a258819ff44d
Reviewed-on: https://go-review.googlesource.com/c/go/+/173951
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
As an experiment to better understand the impact of
having an authoritative source of truth for module hashes
before the real notary is available, this CL adds the basic
notary authorization checks using a partial whitelist of
known go.sum values for popular modules.
In addition to the temporary whitelist, this CL adds code
implementing $GONOVERIFY, a new 'go help modules-auth',
and clearer error messages for verification mismatches.
See #25530 for notary proposal.
Filed #30601 to remove whitelist when notary lands.
Change-Id: Ibcb6ac39c5e60455edf003d8c20af6932aeb7e88
Reviewed-on: https://go-review.googlesource.com/c/go/+/165380
Reviewed-by: Bryan C. Mills <bcmills@google.com>
|
|
The call to os.Rename was failing on the darwin builders, despite having passed in the TryBots.
(I tested this change by running 'go test cmd/go' manually on a darwin gomote.)
This fixes the builder failures after CL 146382.
Updates #26794
Fixes #29030
Change-Id: I3644773421789f65e56f183d077b4e4fd17b8325
Reviewed-on: https://go-review.googlesource.com/c/151798
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
