aboutsummaryrefslogtreecommitdiff
path: root/src/archive
AgeCommit message (Collapse)Author
2022-01-13all: add a handful of fuzz targetsRoland Shoemaker
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip, encoding/json, image/jpeg, image/gif, and image/png. Second attempt, this time we don't use the archives in testdata when fuzzing archive/tar, since those are rather memory intensive, and were crashing a number of builders. Change-Id: I4828d64fa4763c0d8c980392a6578e4dfd956e13 Reviewed-on: https://go-review.googlesource.com/c/go/+/378174 Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-12Revert "all: add a handful of fuzz targets"Bryan Mills
This reverts CL 352109. Reason for revert: causing OOM failures on several builders, and may cause OOMs for end users with small machines as well. Change-Id: I58308d09919969d5a6512ee5cee6aa5c4af6769b Reviewed-on: https://go-review.googlesource.com/c/go/+/377934 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Trust: Katie Hockman <katie@golang.org>
2022-01-12all: add a handful of fuzz targetsRoland Shoemaker
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip, encoding/json, image/jpeg, image/gif, and image/png. Change-Id: Ide1a8de88a9421e786eeeaea3bb93f41e0bae347 Reviewed-on: https://go-review.googlesource.com/c/go/+/352109 Trust: Katie Hockman <katie@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13all: gofmt -w -r 'interface{} -> any' srcRuss Cox
And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-11-07archive/zip: don't read data descriptor earlyArran Walker
Go 1.17 introduced an unnecessary change to when a zip's data descriptor is read for file entries, how it is parsed and how the crc32 field is used. Before Go 1.17, the data descriptor was read immediately after a file entry's content. This continuous read is a pattern existing applications have come to rely upon (for example, where reads at specific offsets might be translated to HTTP range requests). In Go 1.17, all data descriptors are immediately read upon opening the file. This results in scattered and non-continuous reads of the archive, and depending on the underlying reader, might have severe performance implications. In addition, an additional object is now initialized for each entry, but is mostly redundant. Previously, the crc32 field in the data descriptor would return an error if it did not match the central directory's entry. This check has seemingly been unintentionally removed. If the central directory crc32 is invalid and a data descriptor is present, no error is returned. This change reverts to the previous handling of data descriptors, before CL 312310. Fixes #48374 Fixes #49089 Change-Id: I5df2878c4fcc9e500064e7175f3ab9727c82f100 Reviewed-on: https://go-review.googlesource.com/c/go/+/357489 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Dmitri Shuralyov <dmitshur@golang.org>
2021-11-02archive/zip: don't panic on (*Reader).OpenJason7602
Previously, opening a zip with (*Reader).Open could result in a panic if the zip contained a file whose name was exclusively made up of slash characters or ".." path elements. Open could also panic if passed the empty string directly as an argument. Now, any files in the zip whose name could not be made valid for fs.FS.Open will be skipped, and no longer added to the fs.FS file list, although they are still accessible through (*Reader).File. Note that it was already the case that a file could be accessible from (*Reader).Open with a name different from the one in (*Reader).File, as the former is the cleaned name, while the latter is the original one. Finally, made the actual panic site robust as a defense-in-depth measure. Fixes CVE-2021-41772 Fixes #48085 Co-authored-by: Filippo Valsorda <filippo@golang.org> Change-Id: I6271a3f2892e7746f52e213b8eba9a1bba974678 Reviewed-on: https://go-review.googlesource.com/c/go/+/349770 Run-TryBot: Filippo Valsorda <filippo@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org> Trust: Katie Hockman <katie@golang.org> Trust: Julie Qiu <julie@golang.org>
2021-10-28all: go fix -fix=buildtag std cmd (except for bootstrap deps, vendor)Russ Cox
When these packages are released as part of Go 1.18, Go 1.16 will no longer be supported, so we can remove the +build tags in these files. Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs as defined in src/cmd/dist/buildtool.go, which need to continue to build with Go 1.4 for now. Also reverted src/vendor and src/cmd/vendor, which will need to be updated in their own repos first. Manual changes in runtime/pprof/mprof_test.go to adjust line numbers. For #41184. Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e Reviewed-on: https://go-review.googlesource.com/c/go/+/344955 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-06all: use bytes.Cut, strings.CutRuss Cox
Many uses of Index/IndexByte/IndexRune/Split/SplitN can be written more clearly using the new Cut functions. Do that. Also rewrite to other functions if that's clearer. For #46336. Change-Id: I68d024716ace41a57a8bf74455c62279bde0f448 Reviewed-on: https://go-review.googlesource.com/c/go/+/351711 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-08-20archive/zip: prevent preallocation check from overflowingRoland Shoemaker
If the indicated directory size in the archive header is so large that subtracting it from the archive size overflows a uint64, the check that the indicated number of files in the archive can be effectively bypassed. Prevent this from happening by checking that the indicated directory size is less than the size of the archive. Thanks to the OSS-Fuzz project for discovering this issue and to Emmanuel Odeke for reporting it. Fixes #47801 Fixes CVE-2021-39293 Change-Id: Ifade26b98a40f3b37398ca86bd5252d12394dd24 Reviewed-on: https://go-review.googlesource.com/c/go/+/343434 Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2021-08-19archive/tar: unexport internal methodsRoger Peppe
Many of the methods inside the archive/tar package don't need to be exported. Doing so sets a bad precedent that it's OK to export methods to indicate an internal public API. That's not a good idea in general, because exported methods increase cognitive load when reading code: the reader needs to consider whether the exported method might be used via some external interface or reflection. This CL should have no externally visible behaviour changes at all. Change-Id: I94a63de5e6a28e9ac8a283325217349ebce4f308 Reviewed-on: https://go-review.googlesource.com/c/go/+/341410 Reviewed-by: Joe Tsai <joetsai@digital-static.net> Trust: Joe Tsai <joetsai@digital-static.net> Trust: Michael Knyszek <mknyszek@google.com>
2021-05-25archive/zip: only preallocate File slice if reasonably sizedRoland Shoemaker
Since the number of files in the EOCD record isn't validated, it isn't safe to preallocate Reader.Files using that field. A malformed archive can indicate it contains up to 1 << 128 - 1 files. We can still safely preallocate the slice by checking if the specified number of files in the archive is reasonable, given the size of the archive. Thanks to the OSS-Fuzz project for discovering this issue and to Emmanuel Odeke for reporting it. Fixes #46242 Fixes CVE-2021-33196 Change-Id: I3c76d8eec178468b380d87fdb4a3f2cb06f0ee76 Reviewed-on: https://go-review.googlesource.com/c/go/+/318909 Trust: Roland Shoemaker <roland@golang.org> Trust: Katie Hockman <katie@golang.org> Trust: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-05-03archive/zip: add File.OpenRaw, Writer.CreateRaw, Writer.CopyEddie Scholtz
These new methods provide support for cases where performance is a primary concern. For example, copying files from an existing zip to a new zip without incurring the decompression and compression overhead. Using an optimized, external compression method and writing the output to a zip archive. And compressing file contents in parallel and then sequentially writing the compressed bytes to a zip archive. TestWriterCopy is copied verbatim from https://github.com/rsc/zipmerge Fixes #34974 Change-Id: Iade5bc245ba34cdbb86364bf59f79f38bb9e2eb6 Reviewed-on: https://go-review.googlesource.com/c/go/+/312310 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Carlos Amedee <carlos@golang.org>
2021-04-29archive/zip: only return directory once via io/fs.FSIan Lance Taylor
While we're here fix the ModTime value for directories. Fixes #43872 Fixes #45345 Change-Id: I155e6517713ef6a9482b9431f1167a44337c6ad2 Reviewed-on: https://go-review.googlesource.com/c/go/+/311530 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jeremy Faller <jeremy@golang.org>
2021-04-18archive/zip: fix imports block of biggestZipBytes generatorPaschalis Tsilias
Fixes #45529 Change-Id: I4d64c40aa6733b783dc4066e222f17abeb7ad413 Reviewed-on: https://go-review.googlesource.com/c/go/+/309357 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Trust: Joe Tsai <thebrokentoaster@gmail.com> Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-13archive/tar: replace os.MkdirTemp with T.TempDirManlio Perillo
Updates #45402 Change-Id: I296f8c676c68ed1e10b6ad1a17b5b23d2c395252 Reviewed-on: https://go-review.googlesource.com/c/go/+/309355 Run-TryBot: Ian Lance Taylor <iant@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-04-03archive/zip: fix character device handling in fileModeToUnixModeTom Thorogood
The switch case for fs.ModeDevice can only be reached for block devices while character devices match fs.ModeDevice | fs.ModeCharDevice. This would cause character devices to wrongly be reported as regular files. This bug has existed since the switch was first introduced in CL 5624048. Change-Id: Icdbedb015e5376b385b3115d2e4574daa052f796 Reviewed-on: https://go-review.googlesource.com/c/go/+/300891 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-10archive/zip: fix panic in Reader.OpenRoland Shoemaker
When operating on a Zip file that contains a file prefixed with "../", Open(...) would cause a panic in toValidName when attempting to strip the prefixed path components. Fixes CVE-2021-27919 Fixes #44916 Change-Id: Ic755d8126cb0897e2cbbdacf572439c38dde7b35 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004761 Reviewed-by: Filippo Valsorda <valsorda@google.com> Reviewed-by: Russ Cox <rsc@google.com> Reviewed-by: Katie Hockman <katiehockman@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/300489 Trust: Katie Hockman <katie@golang.org> Run-TryBot: Katie Hockman <katie@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-02-20all: go fmt std cmd (but revert vendor)Russ Cox
Make all our package sources use Go 1.17 gofmt format (adding //go:build lines). Part of //go:build change (#41184). See https://golang.org/design/draft-gobuild Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/294430 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-02-09archive/tar: detect out of bounds accesses in PAX records resulting from ↵Emmanuel T Odeke
padded lengths Handles the case in which padding of a PAX record's length field violates invariants about the formatting of record, whereby it no longer matches the prescribed format: "%d %s=%s\n", <length>, <keyword>, <value> as per: https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html#tag_20_92_13_03 0-padding, and paddings of other sorts weren't handled and we assumed that only non-padded decimal lengths would be passed in. Added test cases to ensure that the parsing still proceeds as expected. The prior crashing repro: 0000000000000000000000000000000030 mtime=1432668921.098285006\n30 ctime=2147483649.15163319 exposed the fallacy in the code, that assumed that the length would ALWAYS be a non-padded decimal length string. This bug has existed since Go1.1 as per CL 6700047. Thanks to Josh Bleecher Snyder for fuzzing this package, and thanks to Tom Thorogood for advocacy, raising parity with GNU Tar, but for providing more test cases. Fixes #40196 Change-Id: I32e0af4887bc9221481bd9e8a5120a79f177f08c Reviewed-on: https://go-review.googlesource.com/c/go/+/289629 Trust: Emmanuel Odeke <emmanuel@orijtech.com> Trust: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2021-01-08archive/tar: fix typo in commentyangwenmai
Change-Id: Ifcc565b34b3c3bb7ee62bb0525648a5d2895bf0b Reviewed-on: https://go-review.googlesource.com/c/go/+/282013 Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-12-09all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTempRuss Cox
As part of #42026, these helpers from io/ioutil were moved to os. (ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.) Update the Go tree to use the preferred names. As usual, code compiled with the Go 1.4 bootstrap toolchain and code vendored from other sources is excluded. ReadDir changes are in a separate CL, because they are not a simple search and replace. For #42026. Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266365 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-07archive/zip: fix documentation to mention fs.FS interfaceArtyom Pervukhin
Fixes #42374 Change-Id: I0ed1eb052d79bcc65810b74bff48f1e615e1dc1e Reviewed-on: https://go-review.googlesource.com/c/go/+/267657 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-20all: update references to symbols moved from io/ioutil to ioRuss Cox
The old ioutil references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. Also excluded vendored code. For #41190. Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/263142 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20archive/zip: make Reader implement fs.FSRuss Cox
Now a zip.Reader (an open zip file) can be passed to code that accepts a file system, such as (soon) template parsing. For #41190. Change-Id: If51b12e39db3ccc27f643c2453d3300a38035360 Reviewed-on: https://go-review.googlesource.com/c/go/+/243937 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-10-20all: update references to symbols moved from os to io/fsRuss Cox
The old os references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. For #41190. Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd Reviewed-on: https://go-review.googlesource.com/c/go/+/243907 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-09-23all: add GOOS=iosCherry Zhang
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin" build tag, like GOOS=android matches "linux" and GOOS=illumos matches "solaris". Only ios/arm64 is supported (ios/amd64 is not). GOOS=ios and GOOS=darwin remain essentially the same at this point. They will diverge at later time, to differentiate macOS and iOS. Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"), except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"), it remains GOOS=="darwin". Updates #38485. Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c Reviewed-on: https://go-review.googlesource.com/c/go/+/254740 Trust: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2019-12-10all: fix a number of misuses of the word "an"Daniel Martí
After golang.org/cl/210124, I wondered if the same error had gone unnoticed elsewhere. I quickly spotted another dozen mistakes after reading through the output of: git grep '\<[Aa]n [bcdfgjklmnpqrtvwyz][a-z]' Many results are false positives for acronyms like "an mtime", since it's pronounced "an em-time". However, the total amount of output isn't that large given how simple the grep pattern is. Change-Id: Iaa2ca69e42f4587a9e3137d6c5ed758887906ca6 Reviewed-on: https://go-review.googlesource.com/c/go/+/210678 Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: Zach Jones <zachj1@gmail.com> Reviewed-by: Robert Griesemer <gri@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-21all: base64-encode binaries that will cause Apple notarization to failAndrew
Starting with macOS 10.15 (Catalina), Apple now requires all software distributed outside of the App Store to be notarized. Any binaries we distribute must abide by a strict set of requirements like code-signing and having a minimum target SDK of 10.9 (amongst others). Apple’s notarization service will recursively inspect archives looking to find notarization candidate binaries. If it finds a binary that does not meet the requirements or is unable to decompress an archive, it will reject the entire distribution. From cursory testing, it seems that the service uses content sniffing to determine file types, so changing the file extension will not work. There are some binaries and archives included in our distribution that are being detected by Apple’s service as potential candidates for notarization or decompression. As these are files used by tests and some are intentionally invalid, we don’t intend to ever make them compliant. As a workaround for this, we base64-encode any binaries or archives that Apple’s notarization service issues a warning for, as these warnings will become errors in January 2020. Updates #34986 Change-Id: I106fbb6227b61eb221755568f047ee11103c1680 Reviewed-on: https://go-review.googlesource.com/c/go/+/208118 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-08-27archive/zip: remove unused special caseAndrew Gerrand
This removes a special case that was added to fix issue #10956, but that was never actually effective. The code in the test case still fails to read, so perhaps the zip64 support added in CL 6463050 inadvertently caught this particular case. It's possible that the original theorized bug still exists, but I'm not convinced it was ever fixed. Update #28700 Change-Id: I4854de616364510f64a6def30b308686563f8dbb Reviewed-on: https://go-review.googlesource.com/c/go/+/179757 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-22all: shorten some testsRuss Cox
Shorten some of the longest tests that run during all.bash. Removes 7r 50u 21s from all.bash. After this change, all.bash is under 5 minutes again on my laptop. For #26473. Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79 Reviewed-on: https://go-review.googlesource.com/c/go/+/177559 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-04-08archive/zip: use Modified in FileHeader.FileInfoJannis Andrija Schnitzer
The Modified field allows representation of extended timestamps, which provide more accuracy than the legacy MS-DOS timestamps. The FileInfo method provides an implementation of the os.FileInfo interface for files inside archives. With this change, we make FileInfo use the Modified field, if present, to return more detailed timestamps from its ModTime method. Fixes #28350 Change-Id: Ia31b5b871a3e61df38a3a1325787ae23ea0b8088 GitHub-Last-Rev: 13e94be3f8ba58717911354146670fc2bc594692 GitHub-Pull-Request: golang/go#28352 Reviewed-on: https://go-review.googlesource.com/c/go/+/144382 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-02-27archive/tar, syscall: add statUnix for aix/ppc64Clément Chigot
This commit add statUnix function for aix/ppc64. It also adds Unix and Nano methods for AIX time structure. Change-Id: I9fd62d34a47e87cd46f2f936cb736da0bdff7959 Reviewed-on: https://go-review.googlesource.com/c/163957 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-02-19src, misc: apply gofmtRobert Griesemer
This applies the new gofmt literal normalizations to the library. Change-Id: I8c1e8ef62eb556fc568872c9f77a31ef236348e7 Reviewed-on: https://go-review.googlesource.com/c/162539 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-01-07archive/zip: fix casting overflow on 32-bit archLE Manh Cuong
Fixes #29555 Change-Id: Ia3c0dd65bcf94dea3f6e04c23c1fe5d6d0b2c1e9 Reviewed-on: https://go-review.googlesource.com/c/156399 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-12-30syscall: revert to pre-FreeBSD 10 / POSIX-2008 timespec field names in ↵Yuval Pavel Zholkover
Stat_t on FreeBSD CL 138595 introduced the new names when the hardcoded stat8 definitions was replaced with a cgo generated one. Fixes #29393 Updates #22448 Change-Id: I6309958306329ff301c17344b2e0ead0cc874224 Reviewed-on: https://go-review.googlesource.com/c/155958 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-08all: fix a bunch of misspellingsIgor Zhilianin
Change-Id: I94cebca86706e072fbe3be782d3edbe0e22b9432 GitHub-Last-Rev: 8e15a40545704fb21b41a8768079f2da19341ef3 GitHub-Pull-Request: golang/go#28067 Reviewed-on: https://go-review.googlesource.com/c/140437 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-05syscall: FreeBSD 12 ino64 supportYuval Pavel Zholkover
This is similar to CL 136816 for x/sys/unix, changing the FreeBSD ABI to use 64-bit inodes in Stat_t, Statfs_t, and Dirent types. The changes are forward compatible, that is FreeBSD 10.x, 11.x continue to use their current sysnum numbers. The affected types are converted to the new layout (with some overhead). Thus the same statically linked binary should work using the native sysnums (without any conversion) on FreeBSD 12. Breaking API changes in package syscall are: Mknod takes a uint64 (C dev_t) instead of int. Stat_t: Dev, Ino, Nlink, Rdev, Gen became uint64. Atimespec, Mtimespec, Ctimespec, Birthtimespec renamed to Atim, Mtim, Ctim, Birthtim respectively. Statfs_t: Mntonname and Mntfromname changed from [88]int8 to [1024]int8 arrays. Dirent: Fileno became uint64, Namlen uint16 and an additional field Off int64 (currently unused) was added. The following commands were run to generate ztypes_* and zsyscall_* on FreeBSD-12.0-ALPHA6 systems (GOARCH=386 were run on the same amd64 host): GOOS=freebsd GOARCH=amd64 ./mksyscall.pl -tags freebsd,amd64 syscall_bsd.go syscall_freebsd.go syscall_freebsd_amd64.go |gofmt >zsyscall_freebsd_amd64.go GOOS=freebsd GOARCH=amd64 go tool cgo -godefs types_freebsd.go | GOOS=freebsd GOARCH=amd64 go run mkpost.go >ztypes_freebsd_amd64.go GOOS=freebsd GOARCH=386 ./mksyscall.pl -l32 -tags freebsd,386 syscall_bsd.go syscall_freebsd.go syscall_freebsd_386.go |gofmt >zsyscall_freebsd_386.go GOOS=freebsd GOARCH=386 go tool cgo -godefs types_freebsd.go | GOOS=freebsd GOARCH=386 go run mkpost.go >ztypes_freebsd_386.go GOOS=freebsd GOARCH=arm ./mksyscall.pl -l32 -arm -tags freebsd,arm syscall_bsd.go syscall_freebsd.go syscall_freebsd_arm.go |gofmt >zsyscall_freebsd_arm.go GOOS=freebsd GOARCH=arm go tool cgo -godefs -- -fsigned-char types_freebsd.go | GOOS=freebsd GOARCH=arm go run mkpost.go >ztypes_freebsd_arm.go The Kevent struct was changed to use the FREEBSD_COMPAT11 version always (requiring the COMPAT_FREEBSD11 kernel option FreeBSD-12, this is the default). The definitions of ifData were not updated, their functionality in has have been replaced by vendored golang.org/x/net/route. freebsdVersion initialization was dropped from init() in favor of a sync.Once based wrapper - supportsABI(). Updates #22448. Change-Id: I359b756e2849c036d7ed7f75dbd6ec836e0b90b4 Reviewed-on: https://go-review.googlesource.com/c/138595 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-08-21archive/tar: remore redundant parens in type expressionsIskander Sharipov
Simplify `(T)` expressions to `T` where possible. Found using https://go-critic.github.io/overview.html#typeUnparen-ref Change-Id: Ic5ef335e03898f9fea1ff90fd83956376657fe67 Reviewed-on: https://go-review.googlesource.com/123379 Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-08-21archive/zip: return error from NewReader when negative size is passedJeet Parekh
Fixes #26589 Change-Id: I180883a13cec229093654004b42c48d76ee20272 GitHub-Last-Rev: 2d9879de43fbcfb413116d69accdade6bc042c97 GitHub-Pull-Request: golang/go#26667 Reviewed-on: https://go-review.googlesource.com/126617 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-08-20archive/zip: makes receiver name consistentSanthosh Kumar Tekuri
Change-Id: I4d6f7440747d4f935acddc9a5c5928ed911a2fb0 Reviewed-on: https://go-review.googlesource.com/120515 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-07-16archive/zip: fix regression when writing directoriesJoe Tsai
Several adjustments: 1) When encoding the FileHeader for a directory, explicitly set all of the sizes to zero regardless of their prior values. These values are currently populated by FileInfoHeader as it calls os.FileInfo.Size regardless of whether the file is a directory or not. We avoid fixing FileInfoHeader now as it is too late in the release cycle (see #24082). We silently adjust slightly wrong FileHeader fields as opposed to returning an error because the CreateHeader method already does such mutations (e.g., for UTF-8 detection, data descriptor, etc). 2) Have dirWriter.Write only return an error if some number of bytes are written. Some code still call Write for both normal files and directories, but just pass an empty []byte to Write for directories. Change-Id: I85492a31356107fcf76dc89ceb00a28853754289 Reviewed-on: https://go-review.googlesource.com/124155 Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-06-13archive/zip: warn about FileHeader.Name being unvalidated on readBrad Fitzpatrick
Updates #25849 Change-Id: I09ee928b462ab538a9d38c4e317eaeb8856919f2 Reviewed-on: https://go-review.googlesource.com/118335 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-06-01all: update comment URLs from HTTP to HTTPS, where possibleTim Cooper
Each URL was manually verified to ensure it did not serve up incorrect content. Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df Reviewed-on: https://go-review.googlesource.com/115798 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org>
2018-05-31archive/zip: remove unnecessary words in (*Writer).Close docsIan Lance Taylor
Fixes #25599 Change-Id: I19ac3463682f662515feaf4c6132f55c12ba5386 Reviewed-on: https://go-review.googlesource.com/115618 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-05-02archive/zip: avoid data descriptor when writing directoriesAntonin Amand
Java fails to unzip archives created by archive/zip because directories are written with the "data descriptor" flag (bit 3) set, but emits no such descriptor. To fix this, we explicitly clear the flag. Fixes #25215 Change-Id: Id3af4c7f863758197063df879717c1710f86c0e5 Reviewed-on: https://go-review.googlesource.com/110795 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-04-30all: skip unsupported tests for js/wasmRichard Musiol
The general policy for the current state of js/wasm is that it only has to support tests that are also supported by nacl. The test nilptr3.go makes assumptions about which nil checks can be removed. Since WebAssembly does not signal on reading a null pointer, all nil checks have to be explicit. Updates #18892 Change-Id: I06a687860b8d22ae26b1c391499c0f5183e4c485 Reviewed-on: https://go-review.googlesource.com/110096 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-04-26archive/zip: prevent writing data for a directoryDiogo Pinela
When creating a directory, Writer.Create now returns a dummy io.Writer that always returns an error on Write. Fixes #24043 Change-Id: I7792f54440d45d22d0aa174cba5015ed5fab1c5c Reviewed-on: https://go-review.googlesource.com/108615 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-04-04go/printer, gofmt: tuned table alignment for better resultsRobert Griesemer
The go/printer (and thus gofmt) uses a heuristic to determine whether to break alignment between elements of an expression list which is spread across multiple lines. The heuristic only kicked in if the entry sizes (character length) was above a certain threshold (20) and the ratio between the previous and current entry size was above a certain value (4). This heuristic worked reasonably most of the time, but also led to unfortunate breaks in many cases where a single entry was suddenly much smaller (or larger) then the previous one. The behavior of gofmt was sufficiently mysterious in some of these situations that many issues were filed against it. The simplest solution to address this problem is to remove the heuristic altogether and have a programmer introduce empty lines to force different alignments if it improves readability. The problem with that approach is that the places where it really matters, very long tables with many (hundreds, or more) entries, may be machine-generated and not "post-processed" by a human (e.g., unicode/utf8/tables.go). If a single one of those entries is overlong, the result would be that the alignment would force all comments or values in key:value pairs to be adjusted to that overlong value, making the table hard to read (e.g., that entry may not even be visible on screen and all other entries seem spaced out too wide). Instead, we opted for a slightly improved heuristic that behaves much better for "normal", human-written code. 1) The threshold is increased from 20 to 40. This disables the heuristic for many common cases yet even if the alignment is not "ideal", 40 is not that many characters per line with todays screens, making it very likely that the entire line remains "visible" in an editor. 2) Changed the heuristic to not simply look at the size ratio between current and previous line, but instead considering the geometric mean of the sizes of the previous (aligned) lines. This emphasizes the "overall picture" of the previous lines, rather than a single one (which might be an outlier). 3) Changed the ratio from 4 to 2.5. Now that we ignore sizes below 40, a ratio of 4 would mean that a new entry would have to be 4 times bigger (160) or smaller (10) before alignment would be broken. A ratio of 2.5 seems more sensible. Applied updated gofmt to all of src and misc. Also tested against several former issues that complained about this and verified that the output for the given examples is satisfactory (added respective test cases). Some of the files changed because they were not gofmt-ed in the first place. For #644. For #7335. For #10392. (and probably more related issues) Fixes #22852. Change-Id: I5e48b3d3b157a5cf2d649833b7297b33f43a6f6e
2018-03-26all: use strings.Builder instead of bytes.Buffer where appropriateBrad Fitzpatrick
I grepped for "bytes.Buffer" and "buf.String" and mostly ignored test files. I skipped a few on purpose and probably missed a few others, but otherwise I think this should be most of them. Updates #18990 Change-Id: I5a6ae4296b87b416d8da02d7bfaf981d8cc14774 Reviewed-on: https://go-review.googlesource.com/102479 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-02-26archive/zip: improve Writer.Create documentation on how to add directoriesYury Smolsky
FileHeader.Name also reflects this fact. Fixes #24018 Change-Id: Id0860a9b23c264ac4c6ddd65ba20e0f1f36e4865 Reviewed-on: https://go-review.googlesource.com/97057 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion