| Age | Commit message (Collapse) | Author |
|---|
|
Previously, opening a zip with (*Reader).Open could result in a panic if
the zip contained a file whose name was exclusively made up of slash
characters or ".." path elements.
Open could also panic if passed the empty string directly as an argument.
Now, any files in the zip whose name could not be made valid for
fs.FS.Open will be skipped, and no longer added to the fs.FS file list,
although they are still accessible through (*Reader).File.
Note that it was already the case that a file could be accessible from
(*Reader).Open with a name different from the one in (*Reader).File, as
the former is the cleaned name, while the latter is the original one.
Finally, made the actual panic site robust as a defense-in-depth measure.
Fixes CVE-2021-41772
Fixes #48251
Updates #48085
Co-authored-by: Filippo Valsorda <filippo@golang.org>
Change-Id: I6271a3f2892e7746f52e213b8eba9a1bba974678
Reviewed-on: https://go-review.googlesource.com/c/go/+/349770
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Julie Qiu <julie@golang.org>
(cherry picked from commit b24687394b55a93449e2be4e6892ead58ea9a10f)
Reviewed-on: https://go-review.googlesource.com/c/go/+/360858
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
|
|
overflowing
If the indicated directory size in the archive header is so large that
subtracting it from the archive size overflows a uint64, the check that
the indicated number of files in the archive can be effectively
bypassed. Prevent this from happening by checking that the indicated
directory size is less than the size of the archive.
Thanks to the OSS-Fuzz project for discovering this issue and to
Emmanuel Odeke for reporting it.
Fixes #47985
Updates #47801
Fixes CVE-2021-39293
Change-Id: Ifade26b98a40f3b37398ca86bd5252d12394dd24
Reviewed-on: https://go-review.googlesource.com/c/go/+/343434
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
(cherry picked from commit bacbc33439b124ffd7392c91a5f5d96eca8c0c0b)
Reviewed-on: https://go-review.googlesource.com/c/go/+/345409
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Cherry Mui <cherryyz@google.com>
|
|
reasonably sized
Since the number of files in the EOCD record isn't validated, it isn't
safe to preallocate Reader.Files using that field. A malformed archive
can indicate it contains up to 1 << 128 - 1 files. We can still safely
preallocate the slice by checking if the specified number of files in
the archive is reasonable, given the size of the archive.
Thanks to the OSS-Fuzz project for discovering this issue and to
Emmanuel Odeke for reporting it.
Updates #46242
Fixes #46397
Fixes CVE-2021-33196
Change-Id: I3c76d8eec178468b380d87fdb4a3f2cb06f0ee76
Reviewed-on: https://go-review.googlesource.com/c/go/+/318909
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
(cherry picked from commit 74242baa4136c7a9132a8ccd9881354442788c8c)
Reviewed-on: https://go-review.googlesource.com/c/go/+/322909
Reviewed-by: Filippo Valsorda <filippo@golang.org>
|
|
While we're here fix the ModTime value for directories.
For #43872
For #45345
Fixes #45347
Change-Id: I155e6517713ef6a9482b9431f1167a44337c6ad2
Reviewed-on: https://go-review.googlesource.com/c/go/+/311530
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
(cherry picked from commit 87e4dcd446df2ab1985ef61ce15da329493248a1)
Reviewed-on: https://go-review.googlesource.com/c/go/+/315249
Trust: Jeremy Faller <jeremy@golang.org>
|
|
When operating on a Zip file that contains a file prefixed with "../",
Open(...) would cause a panic in toValidName when attempting to strip
the prefixed path components.
Fixes CVE-2021-27919
Change-Id: Ic755d8126cb0897e2cbbdacf572439c38dde7b35
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004761
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Katie Hockman <katiehockman@google.com>
(cherry picked from commit ce22003b26eaf8e4a690757f699aae7062d41472)
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1013753
Reviewed-by: Roland Shoemaker <bracewell@google.com>
|
|
padded lengths
Handles the case in which padding of a PAX record's length field
violates invariants about the formatting of record, whereby it no
longer matches the prescribed format:
"%d %s=%s\n", <length>, <keyword>, <value>
as per:
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html#tag_20_92_13_03
0-padding, and paddings of other sorts weren't handled and we assumed
that only non-padded decimal lengths would be passed in.
Added test cases to ensure that the parsing still proceeds as expected.
The prior crashing repro:
0000000000000000000000000000000030 mtime=1432668921.098285006\n30 ctime=2147483649.15163319
exposed the fallacy in the code, that assumed that the length would ALWAYS be a
non-padded decimal length string.
This bug has existed since Go1.1 as per CL 6700047.
Thanks to Josh Bleecher Snyder for fuzzing this package, and thanks to Tom
Thorogood for advocacy, raising parity with GNU Tar, but for providing more test cases.
Fixes #40196
Change-Id: I32e0af4887bc9221481bd9e8a5120a79f177f08c
Reviewed-on: https://go-review.googlesource.com/c/go/+/289629
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
Change-Id: Ifcc565b34b3c3bb7ee62bb0525648a5d2895bf0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/282013
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
|
|
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)
Update the Go tree to use the preferred names.
As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.
ReadDir changes are in a separate CL, because they are not a
simple search and replace.
For #42026.
Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Fixes #42374
Change-Id: I0ed1eb052d79bcc65810b74bff48f1e615e1dc1e
Reviewed-on: https://go-review.googlesource.com/c/go/+/267657
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
|
|
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.
For #41190.
Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
|
|
Now a zip.Reader (an open zip file) can be passed to code
that accepts a file system, such as (soon) template parsing.
For #41190.
Change-Id: If51b12e39db3ccc27f643c2453d3300a38035360
Reviewed-on: https://go-review.googlesource.com/c/go/+/243937
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
|
|
The old os references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
For #41190.
Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/243907
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
|
|
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin"
build tag, like GOOS=android matches "linux" and GOOS=illumos
matches "solaris". Only ios/arm64 is supported (ios/amd64 is
not).
GOOS=ios and GOOS=darwin remain essentially the same at this
point. They will diverge at later time, to differentiate macOS
and iOS.
Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"),
except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"),
it remains GOOS=="darwin".
Updates #38485.
Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c
Reviewed-on: https://go-review.googlesource.com/c/go/+/254740
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
|
|
After golang.org/cl/210124, I wondered if the same error had gone
unnoticed elsewhere. I quickly spotted another dozen mistakes after
reading through the output of:
git grep '\<[Aa]n [bcdfgjklmnpqrtvwyz][a-z]'
Many results are false positives for acronyms like "an mtime", since
it's pronounced "an em-time". However, the total amount of output isn't
that large given how simple the grep pattern is.
Change-Id: Iaa2ca69e42f4587a9e3137d6c5ed758887906ca6
Reviewed-on: https://go-review.googlesource.com/c/go/+/210678
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Zach Jones <zachj1@gmail.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
Starting with macOS 10.15 (Catalina), Apple now requires all software
distributed outside of the App Store to be notarized. Any binaries we
distribute must abide by a strict set of requirements like code-signing
and having a minimum target SDK of 10.9 (amongst others).
Apple’s notarization service will recursively inspect archives looking to
find notarization candidate binaries. If it finds a binary that does not
meet the requirements or is unable to decompress an archive, it will
reject the entire distribution. From cursory testing, it seems that the
service uses content sniffing to determine file types, so changing
the file extension will not work.
There are some binaries and archives included in our distribution that
are being detected by Apple’s service as potential candidates for
notarization or decompression. As these are files used by tests and some
are intentionally invalid, we don’t intend to ever make them compliant.
As a workaround for this, we base64-encode any binaries or archives that
Apple’s notarization service issues a warning for, as these warnings will
become errors in January 2020.
Updates #34986
Change-Id: I106fbb6227b61eb221755568f047ee11103c1680
Reviewed-on: https://go-review.googlesource.com/c/go/+/208118
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
This removes a special case that was added to fix issue #10956, but that
was never actually effective. The code in the test case still fails to
read, so perhaps the zip64 support added in CL 6463050 inadvertently
caught this particular case.
It's possible that the original theorized bug still exists, but I'm not
convinced it was ever fixed.
Update #28700
Change-Id: I4854de616364510f64a6def30b308686563f8dbb
Reviewed-on: https://go-review.googlesource.com/c/go/+/179757
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Shorten some of the longest tests that run during all.bash.
Removes 7r 50u 21s from all.bash.
After this change, all.bash is under 5 minutes again on my laptop.
For #26473.
Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79
Reviewed-on: https://go-review.googlesource.com/c/go/+/177559
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
The Modified field allows representation of extended timestamps, which provide more accuracy than the legacy MS-DOS timestamps.
The FileInfo method provides an implementation of the os.FileInfo interface for files inside archives.
With this change, we make FileInfo use the Modified field, if present, to return more detailed timestamps from its ModTime method.
Fixes #28350
Change-Id: Ia31b5b871a3e61df38a3a1325787ae23ea0b8088
GitHub-Last-Rev: 13e94be3f8ba58717911354146670fc2bc594692
GitHub-Pull-Request: golang/go#28352
Reviewed-on: https://go-review.googlesource.com/c/go/+/144382
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
This commit add statUnix function for aix/ppc64. It also adds Unix
and Nano methods for AIX time structure.
Change-Id: I9fd62d34a47e87cd46f2f936cb736da0bdff7959
Reviewed-on: https://go-review.googlesource.com/c/163957
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
This applies the new gofmt literal normalizations to the library.
Change-Id: I8c1e8ef62eb556fc568872c9f77a31ef236348e7
Reviewed-on: https://go-review.googlesource.com/c/162539
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Fixes #29555
Change-Id: Ia3c0dd65bcf94dea3f6e04c23c1fe5d6d0b2c1e9
Reviewed-on: https://go-review.googlesource.com/c/156399
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
Stat_t on FreeBSD
CL 138595 introduced the new names when the hardcoded stat8 definitions was replaced
with a cgo generated one.
Fixes #29393
Updates #22448
Change-Id: I6309958306329ff301c17344b2e0ead0cc874224
Reviewed-on: https://go-review.googlesource.com/c/155958
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Change-Id: I94cebca86706e072fbe3be782d3edbe0e22b9432
GitHub-Last-Rev: 8e15a40545704fb21b41a8768079f2da19341ef3
GitHub-Pull-Request: golang/go#28067
Reviewed-on: https://go-review.googlesource.com/c/140437
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
This is similar to CL 136816 for x/sys/unix, changing the FreeBSD ABI to use 64-bit inodes in
Stat_t, Statfs_t, and Dirent types.
The changes are forward compatible, that is FreeBSD 10.x, 11.x continue to use their current sysnum numbers.
The affected types are converted to the new layout (with some overhead).
Thus the same statically linked binary should work using the native sysnums (without any conversion) on FreeBSD 12.
Breaking API changes in package syscall are:
Mknod takes a uint64 (C dev_t) instead of int.
Stat_t: Dev, Ino, Nlink, Rdev, Gen became uint64.
Atimespec, Mtimespec, Ctimespec, Birthtimespec renamed to Atim, Mtim, Ctim, Birthtim respectively.
Statfs_t: Mntonname and Mntfromname changed from [88]int8 to [1024]int8 arrays.
Dirent: Fileno became uint64, Namlen uint16 and an additional field Off int64 (currently unused) was added.
The following commands were run to generate ztypes_* and zsyscall_* on FreeBSD-12.0-ALPHA6 systems (GOARCH=386 were run on the same amd64 host):
GOOS=freebsd GOARCH=amd64 ./mksyscall.pl -tags freebsd,amd64 syscall_bsd.go syscall_freebsd.go syscall_freebsd_amd64.go |gofmt >zsyscall_freebsd_amd64.go
GOOS=freebsd GOARCH=amd64 go tool cgo -godefs types_freebsd.go | GOOS=freebsd GOARCH=amd64 go run mkpost.go >ztypes_freebsd_amd64.go
GOOS=freebsd GOARCH=386 ./mksyscall.pl -l32 -tags freebsd,386 syscall_bsd.go syscall_freebsd.go syscall_freebsd_386.go |gofmt >zsyscall_freebsd_386.go
GOOS=freebsd GOARCH=386 go tool cgo -godefs types_freebsd.go | GOOS=freebsd GOARCH=386 go run mkpost.go >ztypes_freebsd_386.go
GOOS=freebsd GOARCH=arm ./mksyscall.pl -l32 -arm -tags freebsd,arm syscall_bsd.go syscall_freebsd.go syscall_freebsd_arm.go |gofmt >zsyscall_freebsd_arm.go
GOOS=freebsd GOARCH=arm go tool cgo -godefs -- -fsigned-char types_freebsd.go | GOOS=freebsd GOARCH=arm go run mkpost.go >ztypes_freebsd_arm.go
The Kevent struct was changed to use the FREEBSD_COMPAT11 version always (requiring the COMPAT_FREEBSD11 kernel option FreeBSD-12, this is the default).
The definitions of ifData were not updated, their functionality in has have been replaced by vendored golang.org/x/net/route.
freebsdVersion initialization was dropped from init() in favor of a sync.Once based wrapper - supportsABI().
Updates #22448.
Change-Id: I359b756e2849c036d7ed7f75dbd6ec836e0b90b4
Reviewed-on: https://go-review.googlesource.com/c/138595
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
Simplify `(T)` expressions to `T` where possible.
Found using https://go-critic.github.io/overview.html#typeUnparen-ref
Change-Id: Ic5ef335e03898f9fea1ff90fd83956376657fe67
Reviewed-on: https://go-review.googlesource.com/123379
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
Fixes #26589
Change-Id: I180883a13cec229093654004b42c48d76ee20272
GitHub-Last-Rev: 2d9879de43fbcfb413116d69accdade6bc042c97
GitHub-Pull-Request: golang/go#26667
Reviewed-on: https://go-review.googlesource.com/126617
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
Change-Id: I4d6f7440747d4f935acddc9a5c5928ed911a2fb0
Reviewed-on: https://go-review.googlesource.com/120515
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
Several adjustments:
1) When encoding the FileHeader for a directory, explicitly set all of the sizes
to zero regardless of their prior values. These values are currently populated
by FileInfoHeader as it calls os.FileInfo.Size regardless of whether the file is
a directory or not. We avoid fixing FileInfoHeader now as it is too late in the
release cycle (see #24082).
We silently adjust slightly wrong FileHeader fields as opposed to returning
an error because the CreateHeader method already does such mutations
(e.g., for UTF-8 detection, data descriptor, etc).
2) Have dirWriter.Write only return an error if some number of bytes are written.
Some code still call Write for both normal files and directories, but just pass
an empty []byte to Write for directories.
Change-Id: I85492a31356107fcf76dc89ceb00a28853754289
Reviewed-on: https://go-review.googlesource.com/124155
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Updates #25849
Change-Id: I09ee928b462ab538a9d38c4e317eaeb8856919f2
Reviewed-on: https://go-review.googlesource.com/118335
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
Each URL was manually verified to ensure it did not serve up incorrect
content.
Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
|
|
Fixes #25599
Change-Id: I19ac3463682f662515feaf4c6132f55c12ba5386
Reviewed-on: https://go-review.googlesource.com/115618
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Java fails to unzip archives created by archive/zip because directories are
written with the "data descriptor" flag (bit 3) set, but emits no such
descriptor. To fix this, we explicitly clear the flag.
Fixes #25215
Change-Id: Id3af4c7f863758197063df879717c1710f86c0e5
Reviewed-on: https://go-review.googlesource.com/110795
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
The general policy for the current state of js/wasm is that it only
has to support tests that are also supported by nacl.
The test nilptr3.go makes assumptions about which nil checks can be
removed. Since WebAssembly does not signal on reading a null pointer,
all nil checks have to be explicit.
Updates #18892
Change-Id: I06a687860b8d22ae26b1c391499c0f5183e4c485
Reviewed-on: https://go-review.googlesource.com/110096
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
When creating a directory, Writer.Create now returns a dummy
io.Writer that always returns an error on Write.
Fixes #24043
Change-Id: I7792f54440d45d22d0aa174cba5015ed5fab1c5c
Reviewed-on: https://go-review.googlesource.com/108615
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
The go/printer (and thus gofmt) uses a heuristic to determine
whether to break alignment between elements of an expression
list which is spread across multiple lines. The heuristic only
kicked in if the entry sizes (character length) was above a
certain threshold (20) and the ratio between the previous and
current entry size was above a certain value (4).
This heuristic worked reasonably most of the time, but also
led to unfortunate breaks in many cases where a single entry
was suddenly much smaller (or larger) then the previous one.
The behavior of gofmt was sufficiently mysterious in some of
these situations that many issues were filed against it.
The simplest solution to address this problem is to remove
the heuristic altogether and have a programmer introduce
empty lines to force different alignments if it improves
readability. The problem with that approach is that the
places where it really matters, very long tables with many
(hundreds, or more) entries, may be machine-generated and
not "post-processed" by a human (e.g., unicode/utf8/tables.go).
If a single one of those entries is overlong, the result
would be that the alignment would force all comments or
values in key:value pairs to be adjusted to that overlong
value, making the table hard to read (e.g., that entry may
not even be visible on screen and all other entries seem
spaced out too wide).
Instead, we opted for a slightly improved heuristic that
behaves much better for "normal", human-written code.
1) The threshold is increased from 20 to 40. This disables
the heuristic for many common cases yet even if the alignment
is not "ideal", 40 is not that many characters per line with
todays screens, making it very likely that the entire line
remains "visible" in an editor.
2) Changed the heuristic to not simply look at the size ratio
between current and previous line, but instead considering the
geometric mean of the sizes of the previous (aligned) lines.
This emphasizes the "overall picture" of the previous lines,
rather than a single one (which might be an outlier).
3) Changed the ratio from 4 to 2.5. Now that we ignore sizes
below 40, a ratio of 4 would mean that a new entry would have
to be 4 times bigger (160) or smaller (10) before alignment
would be broken. A ratio of 2.5 seems more sensible.
Applied updated gofmt to all of src and misc. Also tested
against several former issues that complained about this
and verified that the output for the given examples is
satisfactory (added respective test cases).
Some of the files changed because they were not gofmt-ed
in the first place.
For #644.
For #7335.
For #10392.
(and probably more related issues)
Fixes #22852.
Change-Id: I5e48b3d3b157a5cf2d649833b7297b33f43a6f6e
|
|
I grepped for "bytes.Buffer" and "buf.String" and mostly ignored test
files. I skipped a few on purpose and probably missed a few others,
but otherwise I think this should be most of them.
Updates #18990
Change-Id: I5a6ae4296b87b416d8da02d7bfaf981d8cc14774
Reviewed-on: https://go-review.googlesource.com/102479
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
FileHeader.Name also reflects this fact.
Fixes #24018
Change-Id: Id0860a9b23c264ac4c6ddd65ba20e0f1f36e4865
Reviewed-on: https://go-review.googlesource.com/97057
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
CL 14624 introduced this label. At that time,
the switch-case had a break to label statement which made this necessary.
But now, the code no longer has a break statement and it directly returns.
Hence, it is no longer necessary to have a label.
Change-Id: Idde0fcc4d2db2d76424679f5acfe33ab8573bce4
Reviewed-on: https://go-review.googlesource.com/96935
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
The Info-ZIP Unix1 extra field is specified as such:
>>>
Value Size Description
----- ---- -----------
0x5855 Short tag for this extra block type ("UX")
TSize Short total data size for this block
AcTime Long time of last access (GMT/UTC)
ModTime Long time of last modification (GMT/UTC)
<<<
The previous handling was incorrect in that it read the AcTime field
instead of the ModTime field.
The test-osx.zip test unfortunately locked in the wrong behavior.
Manually parsing that ZIP file shows that the encoded MS-DOS
date and time are 0x4b5f and 0xa97d, which corresponds with a
date of 2017-10-31 21:11:58, which matches the correct mod time
(off by 1 second due to MS-DOS timestamp resolution).
Fixes #23901
Change-Id: I567824c66e8316b9acd103dbecde366874a4b7ef
Reviewed-on: https://go-review.googlesource.com/96895
Run-TryBot: Joe Tsai <joetsai@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Currently zip benchmarks spend 60% in the rleBuffer code,
which is used only to test zip archive/zip itself:
17.48s 37.02% 37.02% 18.12s 38.37% archive/zip.(*rleBuffer).ReadAt
9.51s 20.14% 57.16% 10.43s 22.09% archive/zip.(*rleBuffer).Write
9.15s 19.38% 76.54% 10.85s 22.98% compress/flate.(*compressor).deflate
This means that benchmarks currently test performance of test helper.
Updating ReadAt/Write methods to be more performant makes benchmarks closer to real world.
name old time/op new time/op delta
CompressedZipGarbage-8 2.34ms ± 0% 2.34ms ± 1% ~ (p=0.684 n=10+10)
Zip64Test-8 58.1ms ± 2% 10.7ms ± 1% -81.54% (p=0.000 n=10+10)
Zip64TestSizes/4096-8 4.05µs ± 2% 3.65µs ± 5% -9.96% (p=0.000 n=9+10)
Zip64TestSizes/1048576-8 238µs ± 0% 43µs ± 0% -82.06% (p=0.000 n=10+10)
Zip64TestSizes/67108864-8 15.3ms ± 1% 2.6ms ± 0% -83.12% (p=0.000 n=10+9)
name old alloc/op new alloc/op delta
CompressedZipGarbage-8 17.9kB ±14% 16.0kB ±24% -10.48% (p=0.026 n=9+10)
name old allocs/op new allocs/op delta
CompressedZipGarbage-8 44.0 ± 0% 44.0 ± 0% ~ (all equal)
Change-Id: Idfd920d0e4bed4aec2f5be84dc7e3919d9f1dd2d
Reviewed-on: https://go-review.googlesource.com/83857
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Change-Id: Ia5908e94a6bd362099ca3c63f6ffb7e94457131d
GitHub-Last-Rev: 545a40571a912f433546d8c94a9d63459313515d
GitHub-Pull-Request: golang/go#23942
Reviewed-on: https://go-review.googlesource.com/95435
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Change-Id: I1f25b11fb9b7cd3c09968ed99913dc85db2025ef
Reviewed-on: https://go-review.googlesource.com/94976
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
Change Reader to promote TypeRegA to TypeReg in headers, unless their
name have a trailing slash which is already promoted to TypeDir. This
will allow client code to handle just TypeReg instead both TypeReg and
TypeRegA.
Change Writer to promote TypeRegA to TypeReg or TypeDir in the headers
depending on whether the name has a trailing slash. This normalization
is motivated by the specification (in pax(1)):
0 represents a regular file. For backwards-compatibility, a
typeflag value of binary zero ( '\0' ) should be recognized as
meaning a regular file when extracting files from the
archive. Archives written with this version of the archive file
format create regular files with a typeflag value of the
ISO/IEC 646:1991 standard IRV '0'.
Fixes #22768.
Change-Id: I149ec55824580d446cdde5a0d7a0457ad7b03466
Reviewed-on: https://go-review.googlesource.com/85656
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
Lock in fix for #22738, submitted in CL 78031.
Fixes #22738.
Change-Id: I6896feb158569e3f12fa7055387cbd7caad29ef4
Reviewed-on: https://go-review.googlesource.com/80635
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
Several usages of tar (reasonably) just use the Header.FileInfo
to determine the type of the header. However, the os.FileMode type
is not expressive enough to represent "files" that are not files
at all, but some form of metadata.
Thus, Header{Typeflag: TypeXGlobalHeader}.FileInfo().Mode().IsRegular()
reports true, even though the expected result may have been false.
To reduce (not eliminate) the possibility of failure for such usages,
use the placeholder filename from the global PAX headers.
Thus, in the event the user did not handle special "meta" headers
specifically, they will just be written to disk as a regular file.
As an example use case, the "git archive --format=tgz" command produces
an archive where the first "file" is a global PAX header with the
name "global_pax_header". For users that do not explicitly check
the Header.Typeflag field to ignore such headers, they may end up
extracting a file named "global_pax_header". While it is a bogus file,
it at least does not stop the extraction process.
Updates #22748
Change-Id: I28448b528dcfacb4e92311824c33c71b482f49c9
Reviewed-on: https://go-review.googlesource.com/78355
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
In order to avoid a regression where the date of the ModTime method
changed behavior, simply preserve the old behavior of determining
the date based on the legacy fields.
This ensures that anyone relying on ModTime before Go1.10 will have
the exact same behavior as before.
New users should use FileHeader.Modified instead.
We keep the UTC coersion logic in SetModTime since some users
manually compute timezone offsets in order to have precise control
over the MS-DOS time field.
Fixes #22738
Change-Id: Ib18b6ebd863bcf645748e083357dce9bc788cdba
Reviewed-on: https://go-review.googlesource.com/78031
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
A method is more in keeping with the rest of the Writer API and
incidentally allows the comment error to be reported earlier.
Fixes #22737.
Change-Id: I1eee2103a0720c76d0c394ccd6541e6219996dc0
Reviewed-on: https://go-review.googlesource.com/79415
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
|
This CL removes the following APIs:
type SparseEntry struct{ ... }
type Header struct{ SparseHoles []SparseEntry; ... }
func (*Header) DetectSparseHoles(f *os.File) error
func (*Header) PunchSparseHoles(f *os.File) error
func (*Reader) WriteTo(io.Writer) (int, error)
func (*Writer) ReadFrom(io.Reader) (int, error)
This API was added during the Go1.10 dev cycle, and are safe to remove.
The rationale for reverting is because Header.DetectSparseHoles and
Header.PunchSparseHoles are functionality that probably better belongs in
the os package itself.
The other API like Header.SparseHoles, Reader.WriteTo, and Writer.ReadFrom
perform no OS specific logic and only perform the actual business logic of
reading and writing sparse archives. Since we do know know what the API added to
package os may look like, we preemptively revert these non-OS specific changes
as well by simply commenting them out.
Updates #13548
Updates #22735
Change-Id: I77842acd39a43de63e5c754bfa1c26cc24687b70
Reviewed-on: https://go-review.googlesource.com/78030
Reviewed-by: Russ Cox <rsc@golang.org>
|
|
Change-Id: I491c5ddd1a5d8e55f8e6bb9377bc3811e42773f8
Reviewed-on: https://go-review.googlesource.com/77870
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
The replacement rune is a valid rune and can appear as itself in valid UTF8
(it encodes as three bytes). To check for invalid UTF8 it is necessary to
look for utf8.DecodeRune returning the replacement rune and size==1.
Change-Id: I169be8d1fe61605c921ac13cc2fde94f80f3463c
Reviewed-on: https://go-review.googlesource.com/78126
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
|
