aboutsummaryrefslogtreecommitdiff
path: root/src/compress
AgeCommit message (Collapse)Author
2022-01-13all: add a handful of fuzz targetsRoland Shoemaker
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip, encoding/json, image/jpeg, image/gif, and image/png. Second attempt, this time we don't use the archives in testdata when fuzzing archive/tar, since those are rather memory intensive, and were crashing a number of builders. Change-Id: I4828d64fa4763c0d8c980392a6578e4dfd956e13 Reviewed-on: https://go-review.googlesource.com/c/go/+/378174 Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-12Revert "all: add a handful of fuzz targets"Bryan Mills
This reverts CL 352109. Reason for revert: causing OOM failures on several builders, and may cause OOMs for end users with small machines as well. Change-Id: I58308d09919969d5a6512ee5cee6aa5c4af6769b Reviewed-on: https://go-review.googlesource.com/c/go/+/377934 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Trust: Katie Hockman <katie@golang.org>
2022-01-12all: add a handful of fuzz targetsRoland Shoemaker
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip, encoding/json, image/jpeg, image/gif, and image/png. Change-Id: Ide1a8de88a9421e786eeeaea3bb93f41e0bae347 Reviewed-on: https://go-review.googlesource.com/c/go/+/352109 Trust: Katie Hockman <katie@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-10-12compress/lzw: output a Clear code first, per GIF specNigel Tao
The TestStartsWithClearCode test is new, but if it existed beforehand, the want strings would be "\x81" and "Hi\x81" without a starting "\x80". Fixes #26108 Fixes #33748 Updates makeworld-the-better-one/didder#7 Updates nothings/stb#1222 Change-Id: I35ac0ed862ba6ee921ba9aee257bc19828abaa82 Reviewed-on: https://go-review.googlesource.com/c/go/+/354710 Trust: Nigel Tao <nigeltao@golang.org> Run-TryBot: Nigel Tao <nigeltao@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Andrew Gerrand <adg@golang.org>
2021-09-28compress/gzip: add missing licensejiahua wang
Change-Id: I80c2c6054f6a5be3555c72056159a47f927f9a44 Reviewed-on: https://go-review.googlesource.com/c/go/+/352409 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Heschi Kreinick <heschi@google.com>
2021-03-17compress/lzw: add Reset method to Reader and WriterAgniva De Sarker
We add a Reset method which clears any internal state of an encoder or a decoder to let it be reused again as a new Writer or Reader respectively. We also export the encoder and decoder structs, renaming them to be Reader and Writer, and we guarantee that the underlying types from the constructors will always be Reader and Writer respectively. Benchmark results by reusing the encoder: on cpu: Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz name time/op Decoder/1e4-8 93.6µs ± 1% Decoder/1e-Reuse4-8 87.7µs ± 1% Decoder/1e5-8 877µs ± 1% Decoder/1e-Reuse5-8 860µs ± 3% Decoder/1e6-8 8.79ms ± 1% Decoder/1e-Reuse6-8 8.82ms ± 4% Encoder/1e4-8 168µs ± 2% Encoder/1e-Reuse4-8 160µs ± 1% Encoder/1e5-8 1.64ms ± 1% Encoder/1e-Reuse5-8 1.61ms ± 2% Encoder/1e6-8 16.2ms ± 6% Encoder/1e-Reuse6-8 15.8ms ± 2% name speed Decoder/1e4-8 107MB/s ± 1% Decoder/1e-Reuse4-8 114MB/s ± 1% Decoder/1e5-8 114MB/s ± 1% Decoder/1e-Reuse5-8 116MB/s ± 3% Decoder/1e6-8 114MB/s ± 1% Decoder/1e-Reuse6-8 113MB/s ± 5% Encoder/1e4-8 59.7MB/s ± 2% Encoder/1e-Reuse4-8 62.4MB/s ± 1% Encoder/1e5-8 61.1MB/s ± 1% Encoder/1e-Reuse5-8 62.0MB/s ± 2% Encoder/1e6-8 61.7MB/s ± 5% Encoder/1e-Reuse6-8 63.4MB/s ± 2% name alloc/op Decoder/1e4-8 21.8kB ± 0% Decoder/1e-Reuse4-8 50.0B ± 0% Decoder/1e5-8 21.8kB ± 0% Decoder/1e-Reuse5-8 70.4B ± 2% Decoder/1e6-8 21.9kB ± 0% Decoder/1e-Reuse6-8 271B ± 3% Encoder/1e4-8 77.9kB ± 0% Encoder/1e-Reuse4-8 4.17kB ± 0% Encoder/1e5-8 77.9kB ± 0% Encoder/1e-Reuse5-8 4.27kB ± 0% Encoder/1e6-8 77.9kB ± 0% Encoder/1e-Reuse6-8 5.22kB ± 0% name allocs/op Decoder/1e4-8 2.00 ± 0% Decoder/1e-Reuse4-8 1.00 ± 0% Decoder/1e5-8 2.00 ± 0% Decoder/1e-Reuse5-8 1.00 ± 0% Decoder/1e6-8 2.00 ± 0% Decoder/1e-Reuse6-8 1.00 ± 0% Encoder/1e4-8 3.00 ± 0% Encoder/1e-Reuse4-8 2.00 ± 0% Encoder/1e5-8 3.00 ± 0% Encoder/1e-Reuse5-8 2.00 ± 0% Encoder/1e6-8 3.00 ± 0% Encoder/1e-Reuse6-8 2.00 ± 0% Fixes #26535 Change-Id: Icde613fea6234a5bdce95f1e49910f5687e30b22 Reviewed-on: https://go-review.googlesource.com/c/go/+/273667 Trust: Agniva De Sarker <agniva.quicksilver@gmail.com> Trust: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-12-09all: update to use os.ReadFile, os.WriteFile, os.CreateTemp, os.MkdirTempRuss Cox
As part of #42026, these helpers from io/ioutil were moved to os. (ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.) Update the Go tree to use the preferred names. As usual, code compiled with the Go 1.4 bootstrap toolchain and code vendored from other sources is excluded. ReadDir changes are in a separate CL, because they are not a simple search and replace. For #42026. Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3 Reviewed-on: https://go-review.googlesource.com/c/go/+/266365 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-12-02all: update to use filepath.WalkDir instead of filepath.WalkRuss Cox
Now that filepath.WalkDir is available, it is more efficient and should be used in place of filepath.Walk. Update the tree to reflect best practices. As usual, the code compiled with Go 1.4 during bootstrap is excluded. (In this CL, that's only cmd/dist.) For #42027. Change-Id: Ib0f7b1e43e50b789052f9835a63ced701d8c411c Reviewed-on: https://go-review.googlesource.com/c/go/+/267719 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-10-23all: fix quoting for compress/bzip2 and time's godocsubham sarkar
The existing usage of grave accent (`) and apostrophe (') at some places made godoc to ignore them and show it as it is. So, use both of the characters twice (consecutively) so that godoc can convert it to {left,right} double quotation mark. Fixes #41958 Change-Id: I64fd9b5fa34f416ad595009d09f5482e10bd8b4f Reviewed-on: https://go-review.googlesource.com/c/go/+/262397 Reviewed-by: Russ Cox <rsc@golang.org> Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-10-20all: update references to symbols moved from io/ioutil to ioRuss Cox
The old ioutil references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. Also excluded vendored code. For #41190. Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/263142 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20all: update references to symbols moved from os to io/fsRuss Cox
The old os references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. For #41190. Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd Reviewed-on: https://go-review.googlesource.com/c/go/+/243907 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2020-10-19compress/flate: fix corrupted outputKlaus Post
The fastest compression mode can pick up a false match for every 2GB of input data resulting in incorrectly decompressed data. Since matches are allowed to be up to and including at maxMatchOffset we must offset the buffer by an additional element to prevent the first 4 bytes to match after an out-of-reach value after shiftOffsets has been called. We offset by `maxMatchOffset + 1` so offset 0 in the table will now fail the `if offset > maxMatchOffset` in all cases. Fixes #41420 Change-Id: If1fbe01728e132b8a207e3f3f439edd832dcc710 GitHub-Last-Rev: 50fabab0da874c37543b139459a810e12e83cee2 GitHub-Pull-Request: golang/go#41477 Reviewed-on: https://go-review.googlesource.com/c/go/+/255879 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Run-TryBot: Matthew Dempsky <mdempsky@google.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Joe Tsai <thebrokentoaster@gmail.com> Trust: Matthew Dempsky <mdempsky@google.com>
2020-10-16compress/flate: revert a goto for-loopDaniel Martí
In https://golang.org/cl/16528, a goto loop was chosen over a regular for loop since that would make the function inlinable. Thanks to the recent https://golang.org/cl/256459, for loops without a label can now be inlined. So we can undo the workaround and simplify the code. Also add the function to TestIntendedInlining, which passes both before and after the change, as expected. For #14768. Change-Id: Ie5df55a6bcb07c538ca331eef2f908807ff0b516 Reviewed-on: https://go-review.googlesource.com/c/go/+/263037 Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Go Bot <gobot@golang.org>
2020-10-06compress/flate: remove unneeded zeroing of bytes array in ↵Dan Scales
(*huffmanBitWriter).reset There is no correctness reason to zero out the w.bytes array in (w *huffmanBitWriter).reset, since w.nbytes is correctly set to zero. The elements of the bytes array are always written sequentially, with nbytes indicating how many elements have been written, and are only read up to the current value of nybytes. We have a pprof profile of a web server that compresses its request/responses, and the zeroing in reset() is taking up 2.6% of the CPU time of the server (and could be causing more slowdowns elsewhere due to its effects on the cache). This overhead may be showing up especially because there are many request/responses that are all fairly small. I'm not sure if the zeroing of the bytes array was intended as extra protection of data across reset uses in the same program, but no protection is needed as long as the huffman_bit_writer code remains correct. Change-Id: I67f2b2f56cff9dcc38d8fc0aea885bb010aeedbf Reviewed-on: https://go-review.googlesource.com/c/go/+/258577 Run-TryBot: Dan Scales <danscales@google.com> Run-TryBot: Joe Tsai <joetsai@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Joe Tsai <joetsai@google.com> Trust: Joe Tsai <joetsai@google.com> Trust: Dan Scales <danscales@google.com>
2020-07-16compress/flate: fix another deflate Reset inconsistencyRuss Cox
While investigating #34121, fixed by CL 193605, I discovered another case where Reset was not quite resetting enough. This specific case is not a problem in Reset itself but rather that the Huffman bit writer in one code path is using uninitialized memory left over from a previous block, making the compression not choose the optimal compression method. Fixes #34121. Change-Id: I29245b28214d924e382f91e2c56b4b8a9b7da13d Reviewed-on: https://go-review.googlesource.com/c/go/+/243140 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2020-07-16compress/flate: fix deflate Reset consistencyKlaus Post
Modify the overflow detection logic to shuffle the contents of the table to a lower offset to avoid leaking the effects of a previous use of compress.Writer past Reset calls. Fixes #34121 Change-Id: I9963eadfa5482881e7b7adbad4c2cae146b669ab GitHub-Last-Rev: 8b35798cdd4d5a901d6422647b12984d7e500ba3 GitHub-Pull-Request: golang/go#34128 Reviewed-on: https://go-review.googlesource.com/c/go/+/193605 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-21all: base64-encode binaries that will cause Apple notarization to failAndrew
Starting with macOS 10.15 (Catalina), Apple now requires all software distributed outside of the App Store to be notarized. Any binaries we distribute must abide by a strict set of requirements like code-signing and having a minimum target SDK of 10.9 (amongst others). Apple’s notarization service will recursively inspect archives looking to find notarization candidate binaries. If it finds a binary that does not meet the requirements or is unable to decompress an archive, it will reject the entire distribution. From cursory testing, it seems that the service uses content sniffing to determine file types, so changing the file extension will not work. There are some binaries and archives included in our distribution that are being detected by Apple’s service as potential candidates for notarization or decompression. As these are files used by tests and some are intentionally invalid, we don’t intend to ever make them compliant. As a workaround for this, we base64-encode any binaries or archives that Apple’s notarization service issues a warning for, as these warnings will become errors in January 2020. Updates #34986 Change-Id: I106fbb6227b61eb221755568f047ee11103c1680 Reviewed-on: https://go-review.googlesource.com/c/go/+/208118 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-14compress/lzw: clarify code invariantsNigel Tao
This follows on from https://go-review.googlesource.com/c/go/+/191358 which was submitted as a comment-only change. Benchmarks don't show any significant change: compress/lzw name old speed new speed delta Decoder/1e4-56 92.8MB/s ± 1% 92.7MB/s ± 1% ~ (p=1.000 n=5+5) Decoder/1e5-56 100MB/s ± 1% 100MB/s ± 1% ~ (p=0.746 n=5+5) Decoder/1e6-56 101MB/s ± 1% 101MB/s ± 1% ~ (p=0.381 n=5+5) image/gif name old speed new speed delta Decode-56 63.2MB/s ± 1% 63.2MB/s ± 1% ~ (p=0.690 n=5+5) Change-Id: Ic36b5410cb06ca258da32e40da1f1ff6c44cff86 Reviewed-on: https://go-review.googlesource.com/c/go/+/194938 Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-12compress/lzw: fix comment re high-code invariantNigel Tao
The listed invariant, while technically true, was misleading, and the invariant can be tightened. We never actually get to (d.hi == d.overflow), due to the "d.hi--" line in the decoder.decode method. This is a comment-only commit, changing the comment to match the code. A follow-up commit could restore the comment, changing the code to match the original intented invariant. But the first step is to have the comment and the code say the same thing. Change-Id: Ifc9f78d5060454fc107af9be298026bf3043d400 Reviewed-on: https://go-review.googlesource.com/c/go/+/191358 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Bryan C. Mills <bcmills@google.com> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-08-28compress/flate: improve deflate performance by register allocating the indexNao YONASHIRO
Use local index variable to help the go compiler use a register to for the hash index instead of continuous memory read and write operations. compress/flate: Encode/Digits/Huffman/1e4-4 35.3µs ± 1% 32.7µs ± 0% -7.48% (p=0.000 n=17+19) Encode/Digits/Huffman/1e5-4 330µs ± 0% 312µs ± 0% -5.55% (p=0.000 n=17+18) Encode/Digits/Huffman/1e6-4 3.30ms ± 0% 3.12ms ± 0% -5.64% (p=0.000 n=18+19) Encode/Digits/Speed/1e4-4 157µs ± 0% 156µs ± 0% -0.41% (p=0.000 n=17+19) Encode/Digits/Speed/1e5-4 1.46ms ± 0% 1.46ms ± 1% ~ (p=0.478 n=20+19) Encode/Digits/Speed/1e6-4 14.4ms ± 0% 14.4ms ± 0% ~ (p=0.835 n=19+20) Encode/Digits/Default/1e4-4 309µs ± 0% 310µs ± 0% +0.23% (p=0.000 n=19+17) Encode/Digits/Default/1e5-4 4.76ms ± 0% 4.76ms ± 0% ~ (p=0.297 n=19+19) Encode/Digits/Default/1e6-4 51.0ms ± 0% 51.0ms ± 1% ~ (p=0.233 n=18+19) Encode/Digits/Compression/1e4-4 309µs ± 0% 310µs ± 0% +0.21% (p=0.000 n=17+20) Encode/Digits/Compression/1e5-4 4.76ms ± 0% 4.76ms ± 0% ~ (p=0.749 n=20+19) Encode/Digits/Compression/1e6-4 50.9ms ± 0% 50.9ms ± 0% ~ (p=0.499 n=18+19) Encode/Newton/Huffman/1e4-4 51.9µs ± 0% 48.0µs ± 0% -7.61% (p=0.000 n=19+19) Encode/Newton/Huffman/1e5-4 396µs ± 0% 377µs ± 0% -4.79% (p=0.000 n=18+19) Encode/Newton/Huffman/1e6-4 3.95ms ± 0% 3.74ms ± 0% -5.21% (p=0.000 n=20+17) Encode/Newton/Speed/1e4-4 155µs ± 0% 154µs ± 0% -0.67% (p=0.000 n=17+18) Encode/Newton/Speed/1e5-4 1.17ms ± 0% 1.16ms ± 0% -0.64% (p=0.000 n=20+16) Encode/Newton/Speed/1e6-4 11.6ms ± 0% 11.5ms ± 0% -0.63% (p=0.000 n=19+20) Encode/Newton/Default/1e4-4 347µs ± 0% 347µs ± 0% ~ (p=0.744 n=20+19) Encode/Newton/Default/1e5-4 5.06ms ± 0% 5.02ms ± 0% -0.77% (p=0.000 n=20+19) Encode/Newton/Default/1e6-4 53.3ms ± 1% 52.8ms ± 0% -0.91% (p=0.000 n=18+16) Encode/Newton/Compression/1e4-4 351µs ± 0% 351µs ± 0% ~ (p=0.277 n=20+20) Encode/Newton/Compression/1e5-4 6.90ms ± 0% 6.85ms ± 0% -0.61% (p=0.000 n=19+18) Encode/Newton/Compression/1e6-4 73.2ms ± 0% 72.8ms ± 0% -0.52% (p=0.000 n=18+18) name old speed new speed delta Encode/Digits/Huffman/1e4-4 283MB/s ± 1% 306MB/s ± 0% +8.09% (p=0.000 n=17+19) Encode/Digits/Huffman/1e5-4 303MB/s ± 0% 321MB/s ± 0% +5.87% (p=0.000 n=18+18) Encode/Digits/Huffman/1e6-4 303MB/s ± 0% 321MB/s ± 0% +5.98% (p=0.000 n=18+19) Encode/Digits/Speed/1e4-4 63.9MB/s ± 0% 64.2MB/s ± 0% +0.41% (p=0.000 n=17+19) Encode/Digits/Speed/1e5-4 68.5MB/s ± 0% 68.4MB/s ± 1% ~ (p=0.481 n=20+19) Encode/Digits/Speed/1e6-4 69.4MB/s ± 0% 69.3MB/s ± 0% ~ (p=0.712 n=19+20) Encode/Digits/Default/1e4-4 32.3MB/s ± 0% 32.3MB/s ± 0% -0.23% (p=0.000 n=19+17) Encode/Digits/Default/1e5-4 21.0MB/s ± 0% 21.0MB/s ± 0% ~ (p=0.460 n=19+19) Encode/Digits/Default/1e6-4 19.6MB/s ± 0% 19.6MB/s ± 1% ~ (p=0.180 n=18+19) Encode/Digits/Compression/1e4-4 32.3MB/s ± 0% 32.3MB/s ± 0% -0.21% (p=0.000 n=17+20) Encode/Digits/Compression/1e5-4 21.0MB/s ± 0% 21.0MB/s ± 0% ~ (p=0.700 n=20+19) Encode/Digits/Compression/1e6-4 19.6MB/s ± 0% 19.6MB/s ± 0% ~ (p=0.486 n=18+19) Encode/Newton/Huffman/1e4-4 193MB/s ± 0% 208MB/s ± 0% +8.23% (p=0.000 n=19+19) Encode/Newton/Huffman/1e5-4 252MB/s ± 0% 265MB/s ± 0% +5.04% (p=0.000 n=18+19) Encode/Newton/Huffman/1e6-4 253MB/s ± 0% 267MB/s ± 0% +5.49% (p=0.000 n=20+17) Encode/Newton/Speed/1e4-4 64.5MB/s ± 0% 65.0MB/s ± 0% +0.67% (p=0.000 n=17+18) Encode/Newton/Speed/1e5-4 85.7MB/s ± 0% 86.3MB/s ± 0% +0.65% (p=0.000 n=20+16) Encode/Newton/Speed/1e6-4 86.2MB/s ± 0% 86.7MB/s ± 0% +0.63% (p=0.000 n=19+20) Encode/Newton/Default/1e4-4 28.9MB/s ± 0% 28.9MB/s ± 0% ~ (p=0.840 n=20+19) Encode/Newton/Default/1e5-4 19.8MB/s ± 0% 19.9MB/s ± 0% +0.78% (p=0.000 n=20+19) Encode/Newton/Default/1e6-4 18.8MB/s ± 1% 18.9MB/s ± 0% +0.93% (p=0.000 n=18+16) Encode/Newton/Compression/1e4-4 28.5MB/s ± 0% 28.5MB/s ± 0% ~ (p=0.244 n=20+20) Encode/Newton/Compression/1e5-4 14.5MB/s ± 0% 14.6MB/s ± 0% +0.61% (p=0.000 n=19+18) Encode/Newton/Compression/1e6-4 13.7MB/s ± 0% 13.7MB/s ± 0% +0.53% (p=0.000 n=18+18) image/png: name old time/op new time/op delta EncodeGray-4 2.16ms ± 1% 1.85ms ± 1% -14.17% (p=0.000 n=86+91) EncodeGrayWithBufferPool-4 1.99ms ± 0% 1.69ms ± 0% -15.09% (p=0.000 n=97+94) EncodeNRGBOpaque-4 6.51ms ± 1% 5.62ms ± 1% -13.66% (p=0.000 n=90+92) EncodeNRGBA-4 7.33ms ± 1% 6.12ms ± 1% -16.49% (p=0.000 n=89+90) EncodePaletted-4 5.10ms ± 1% 4.96ms ± 1% -2.76% (p=0.000 n=90+87) EncodeRGBOpaque-4 6.51ms ± 1% 5.63ms ± 1% -13.49% (p=0.000 n=94+87) EncodeRGBA-4 24.3ms ± 2% 23.0ms ± 0% -5.23% (p=0.000 n=91+89) name old speed new speed delta EncodeGray-4 142MB/s ± 1% 166MB/s ± 1% +16.50% (p=0.000 n=86+91) EncodeGrayWithBufferPool-4 154MB/s ± 0% 182MB/s ± 0% +17.78% (p=0.000 n=97+94) EncodeNRGBOpaque-4 189MB/s ± 1% 219MB/s ± 1% +15.82% (p=0.000 n=90+93) EncodeNRGBA-4 168MB/s ± 1% 201MB/s ± 1% +19.75% (p=0.000 n=89+90) EncodePaletted-4 60.3MB/s ± 1% 62.0MB/s ± 1% +2.84% (p=0.000 n=90+87) EncodeRGBOpaque-4 189MB/s ± 1% 218MB/s ± 1% +15.60% (p=0.000 n=94+87) EncodeRGBA-4 50.6MB/s ± 2% 53.4MB/s ± 0% +5.51% (p=0.000 n=91+89) Change-Id: Ifed4486a7ba19a26abe5cbf2142f15cc7464e84f Reviewed-on: https://go-review.googlesource.com/c/go/+/187837 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2019-05-24compress/gzip: add missing error check in testLeon Klingele
Change-Id: Ib7be503a46f3609ea29464569e57e1f620e97a53 GitHub-Last-Rev: a43cffe45f3b9d208028f7682088cbe537d6deca GitHub-Pull-Request: golang/go#30007 Reviewed-on: https://go-review.googlesource.com/c/go/+/160431 Run-TryBot: Leon Klingele <git@leonklingele.de> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-22all: shorten some testsRuss Cox
Shorten some of the longest tests that run during all.bash. Removes 7r 50u 21s from all.bash. After this change, all.bash is under 5 minutes again on my laptop. For #26473. Change-Id: Ie0460aa935808d65460408feaed210fbaa1d5d79 Reviewed-on: https://go-review.googlesource.com/c/go/+/177559 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-03-31cmd/go: further reduce init workDaniel Martí
The first biggest offender was crypto/des.init at ~1%. It's cryptographically broken and the init function is relatively expensive, which is unfortunate as both crypto/tls and crypto/x509 (and by extension, cmd/go) import it. Hide the work behind sync.Once. The second biggest offender was flag.sortFlags at just under 1%, used by the Visit flagset methods. It allocated two slices, which made a difference as cmd/go iterates over multiple flagsets during init. Use a single slice with a direct sort.Interface implementation. Another big offender is initializing global maps. Reducing this work in cmd/go/internal/imports and net/textproto gives us close to another whole 1% in saved work. The former can use map literals, and the latter can hide the work behind sync.Once. Finally, compress/flate used newHuffmanBitWriter as part of init, which allocates many objects and slices. Yet it only used one of the slice fields. Allocating just that slice saves a surprising ~0.3%, since we generated a lot of unnecessary garbage. All in all, these little pieces amount to just over 3% saved CPU time. name old time/op new time/op delta ExecGoEnv-8 3.61ms ± 1% 3.50ms ± 0% -3.02% (p=0.000 n=10+10) Updates #26775. Updates #29382. Change-Id: I915416e88a874c63235ba512617c8aef35c0ca8b Reviewed-on: https://go-review.googlesource.com/c/go/+/166459 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-02-26compress/gzip: clarify that Multistream gzip requires a ByteReaderJeremy Jay
Change-Id: Ib24778f3172c011e6a39ee65dce8764f3cc911ea GitHub-Last-Rev: 9c617c1e60ac48db67e26e64ce240d3845c0e6ac GitHub-Pull-Request: golang/go#30284 Reviewed-on: https://go-review.googlesource.com/c/162999 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-02-19src, misc: apply gofmtRobert Griesemer
This applies the new gofmt literal normalizations to the library. Change-Id: I8c1e8ef62eb556fc568872c9f77a31ef236348e7 Reviewed-on: https://go-review.googlesource.com/c/162539 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-12-20compress/flate: fix the old url for the flate algorithmOsamu TONOMORI
Change-Id: I84b74bc96516033bbf4a01f9aa81fe60d5a41355 Reviewed-on: https://go-review.googlesource.com/c/155317 Reviewed-by: Matthew Dempsky <mdempsky@google.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-06all: fix a bunch of misspellingsIgor Zhilianin
Change-Id: If2954bdfc551515403706b2cd0dde94e45936e08 GitHub-Last-Rev: d4cfc41a5504cf10befefdb881d4c45986a1d1f8 GitHub-Pull-Request: golang/go#28049 Reviewed-on: https://go-review.googlesource.com/c/140299 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-01Revert "compress: move benchmark text from src/testdata to ↵Katie Hockman
src/compress/testdata" This reverts commit 067bb443af6b44cb026ab182a26d157dbd1b2dd6. Reason for revert: Failing Darwin-arm builds because that testing environment does not access testdata from sibling directories. A future change will likely be made to move this testdata out of src/testdata to create a solution that doesn't require the single-file directory. Updates #27151 Change-Id: I8dbf5dd9512c94a605ee749ff4655cb00b0de686 Reviewed-on: https://go-review.googlesource.com/138737 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-09-28compress: move benchmark text from src/testdata to src/compress/testdataKatie Hockman
This text is used mainly for benchmark compression testing, and in one net test. The text was prevoiusly in a src/testdata directory, but since that directory would only include one file, the text is moved to the existing src/compression/testdata directory. This does not cause any change to the benchmark results. Updates #27151 Change-Id: I38ab5089dfe744189a970947d15be50ef1d48517 Reviewed-on: https://go-review.googlesource.com/138495 Run-TryBot: Katie Hockman <katie@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-09-24compress: reduce copies of new text for compression testingKatie Hockman
The previous book was 387 KiB decompressed and 119 KiB compressed, the new book is 567 KiB decompressed and 132 KiB compressed. Overall, this change will reduce the release binary size by 196 KiB. The new book will allow for slightly more extensive compression testing with a larger text. Command to run the benchmark tests used with benchstat: `../bin/go test -run='^$' -count=4 -bench=. compress/bzip2 compress/flate` When running the benchmarks locally, changed "Newton" to "Twain" and filtered the tests with the -bench flag to include only those which were relevant to these changes. benchstat results below: name old time/op new time/op delta DecodeTwain-8 19.6ms ± 2% 24.1ms ± 1% +23.04% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 140µs ± 3% 139µs ± 5% ~ (p=0.886 n=4+4) Decode/Twain/Huffman/1e5-8 1.27ms ± 3% 1.26ms ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e6-8 12.4ms ± 0% 13.2ms ± 1% +6.42% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 133µs ± 1% 123µs ± 1% -7.35% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 1.20ms ± 0% 1.02ms ± 3% -15.32% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 12.0ms ± 2% 10.1ms ± 3% -15.89% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 131µs ± 6% 108µs ± 5% -17.84% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 1.06ms ± 2% 0.80ms ± 1% -24.97% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 10.0ms ± 3% 8.0ms ± 3% -20.06% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 128µs ± 4% 115µs ± 4% -9.70% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 1.04ms ± 2% 0.83ms ± 4% -20.37% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 10.4ms ± 4% 8.1ms ± 5% -22.25% (p=0.029 n=4+4) Encode/Twain/Huffman/1e4-8 55.7µs ± 2% 55.6µs ± 1% ~ (p=1.000 n=4+4) Encode/Twain/Huffman/1e5-8 441µs ± 0% 435µs ± 2% ~ (p=0.343 n=4+4) Encode/Twain/Huffman/1e6-8 4.31ms ± 4% 4.30ms ± 4% ~ (p=0.886 n=4+4) Encode/Twain/Speed/1e4-8 193µs ± 1% 166µs ± 2% -14.09% (p=0.029 n=4+4) Encode/Twain/Speed/1e5-8 1.54ms ± 1% 1.22ms ± 1% -20.53% (p=0.029 n=4+4) Encode/Twain/Speed/1e6-8 15.3ms ± 1% 12.2ms ± 3% -20.62% (p=0.029 n=4+4) Encode/Twain/Default/1e4-8 393µs ± 1% 390µs ± 1% ~ (p=0.114 n=4+4) Encode/Twain/Default/1e5-8 6.12ms ± 4% 6.02ms ± 5% ~ (p=0.486 n=4+4) Encode/Twain/Default/1e6-8 69.4ms ± 5% 59.0ms ± 4% -15.07% (p=0.029 n=4+4) Encode/Twain/Compression/1e4-8 423µs ± 2% 379µs ± 2% -10.34% (p=0.029 n=4+4) Encode/Twain/Compression/1e5-8 7.00ms ± 1% 7.88ms ± 3% +12.49% (p=0.029 n=4+4) Encode/Twain/Compression/1e6-8 76.6ms ± 5% 80.9ms ± 3% ~ (p=0.114 n=4+4) name old speed new speed delta DecodeTwain-8 19.8MB/s ± 2% 23.6MB/s ± 1% +18.84% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 71.7MB/s ± 3% 72.1MB/s ± 6% ~ (p=0.943 n=4+4) Decode/Twain/Huffman/1e5-8 78.8MB/s ± 3% 79.5MB/s ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e6-8 80.5MB/s ± 0% 75.6MB/s ± 1% -6.03% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 75.2MB/s ± 1% 81.2MB/s ± 1% +7.93% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 83.4MB/s ± 0% 98.6MB/s ± 3% +18.16% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 83.6MB/s ± 2% 99.5MB/s ± 3% +18.91% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 76.3MB/s ± 6% 92.8MB/s ± 4% +21.62% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 94.4MB/s ± 3% 125.7MB/s ± 1% +33.24% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 100MB/s ± 3% 125MB/s ± 3% +25.12% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 78.4MB/s ± 4% 86.8MB/s ± 4% +10.73% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 95.7MB/s ± 2% 120.3MB/s ± 4% +25.65% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 96.4MB/s ± 4% 124.0MB/s ± 5% +28.64% (p=0.029 n=4+4) Encode/Twain/Huffman/1e4-8 179MB/s ± 2% 180MB/s ± 1% ~ (p=1.000 n=4+4) Encode/Twain/Huffman/1e5-8 227MB/s ± 0% 230MB/s ± 2% ~ (p=0.343 n=4+4) Encode/Twain/Huffman/1e6-8 232MB/s ± 4% 233MB/s ± 4% ~ (p=0.886 n=4+4) Encode/Twain/Speed/1e4-8 51.8MB/s ± 1% 60.4MB/s ± 2% +16.43% (p=0.029 n=4+4) Encode/Twain/Speed/1e5-8 65.1MB/s ± 1% 81.9MB/s ± 1% +25.83% (p=0.029 n=4+4) Encode/Twain/Speed/1e6-8 65.2MB/s ± 1% 82.2MB/s ± 3% +26.00% (p=0.029 n=4+4) Encode/Twain/Default/1e4-8 25.4MB/s ± 1% 25.6MB/s ± 1% ~ (p=0.114 n=4+4) Encode/Twain/Default/1e5-8 16.4MB/s ± 4% 16.6MB/s ± 5% ~ (p=0.486 n=4+4) Encode/Twain/Default/1e6-8 14.4MB/s ± 6% 17.0MB/s ± 4% +17.67% (p=0.029 n=4+4) Encode/Twain/Compression/1e4-8 23.6MB/s ± 2% 26.4MB/s ± 2% +11.54% (p=0.029 n=4+4) Encode/Twain/Compression/1e5-8 14.3MB/s ± 1% 12.7MB/s ± 3% -11.08% (p=0.029 n=4+4) Encode/Twain/Compression/1e6-8 13.1MB/s ± 4% 12.4MB/s ± 3% ~ (p=0.114 n=4+4) name old alloc/op new alloc/op delta DecodeTwain-8 3.63MB ± 0% 3.63MB ± 0% +0.15% (p=0.029 n=4+4) Decode/Twain/Huffman/1e4-8 42.0kB ± 0% 41.3kB ± 0% -1.62% (p=0.029 n=4+4) Decode/Twain/Huffman/1e5-8 43.5kB ± 0% 45.1kB ± 0% +3.74% (p=0.029 n=4+4) Decode/Twain/Huffman/1e6-8 71.7kB ± 0% 80.0kB ± 0% +11.55% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 41.2kB ± 0% 41.3kB ± 0% ~ (p=0.286 n=4+4) Decode/Twain/Speed/1e5-8 45.1kB ± 0% 43.9kB ± 0% -2.80% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 72.8kB ± 0% 81.3kB ± 0% +11.72% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 41.2kB ± 0% 41.2kB ± 0% -0.22% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 44.4kB ± 0% 43.0kB ± 0% -3.02% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 71.0kB ± 0% 61.8kB ± 0% -13.00% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 41.3kB ± 0% 41.2kB ± 0% -0.29% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 43.3kB ± 0% 43.0kB ± 0% -0.72% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 69.1kB ± 0% 63.7kB ± 0% -7.90% (p=0.029 n=4+4) name old allocs/op new allocs/op delta DecodeTwain-8 51.0 ± 0% 51.2 ± 1% ~ (p=1.000 n=4+4) Decode/Twain/Huffman/1e4-8 15.0 ± 0% 14.0 ± 0% -6.67% (p=0.029 n=4+4) Decode/Twain/Huffman/1e5-8 20.0 ± 0% 23.0 ± 0% +15.00% (p=0.029 n=4+4) Decode/Twain/Huffman/1e6-8 134 ± 0% 161 ± 0% +20.15% (p=0.029 n=4+4) Decode/Twain/Speed/1e4-8 17.0 ± 0% 18.0 ± 0% +5.88% (p=0.029 n=4+4) Decode/Twain/Speed/1e5-8 30.0 ± 0% 31.0 ± 0% +3.33% (p=0.029 n=4+4) Decode/Twain/Speed/1e6-8 193 ± 0% 228 ± 0% +18.13% (p=0.029 n=4+4) Decode/Twain/Default/1e4-8 17.0 ± 0% 15.0 ± 0% -11.76% (p=0.029 n=4+4) Decode/Twain/Default/1e5-8 28.0 ± 0% 32.0 ± 0% +14.29% (p=0.029 n=4+4) Decode/Twain/Default/1e6-8 199 ± 0% 158 ± 0% -20.60% (p=0.029 n=4+4) Decode/Twain/Compression/1e4-8 17.0 ± 0% 15.0 ± 0% -11.76% (p=0.029 n=4+4) Decode/Twain/Compression/1e5-8 28.0 ± 0% 32.0 ± 0% +14.29% (p=0.029 n=4+4) Decode/Twain/Compression/1e6-8 196 ± 0% 150 ± 0% -23.47% (p=0.029 n=4+4) Updates #27151 Change-Id: I6c439694ed16a33bb4c63fbfb8570c7de46b4f2d Reviewed-on: https://go-review.googlesource.com/135495 Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
2018-09-11cmd/link,compress/zip,image/png: use binary.{Big,Little}Endian methodsLynn Boger
Use the binary.{Big,Little}Endian integer encoding methods rather than variations found in local implementations. The functions in the binary package have been tested to ensure they inline correctly and don't add unnecessary bounds checking. Change-Id: Ie10111ca6edb7c11e8e5e21c58a5748ae99b7f87 Reviewed-on: https://go-review.googlesource.com/134375 Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Munday <mike.munday@ibm.com>
2018-06-01all: update comment URLs from HTTP to HTTPS, where possibleTim Cooper
Each URL was manually verified to ensure it did not serve up incorrect content. Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df Reviewed-on: https://go-review.googlesource.com/115798 Reviewed-by: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org>
2018-05-07compress/flate: do not rename math/bits importRuss Cox
Makes compress/flate work better with cmd/dist bootstrap. Change-Id: Ifc7d74027367008e82c1d14ec77141830583ba82 Reviewed-on: https://go-review.googlesource.com/111815 Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-05-04compress/gzip: skip builder-only test on iOSElias Naur
The new iOS test harness forwards environment variables to the test program, which means that it runs builder-only tests that were previously skipped because GO_BUILDER_NAME was missing. Skip one such unblocked test, TestGZIPFilesHaveZeroMTimes, which assumes a valid GOROOT. Change-Id: I5daf0f4c1897afbeb8b1a380669a1d2aa47e764a Reviewed-on: https://go-review.googlesource.com/111475 Run-TryBot: Elias Naur <elias.naur@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-04-17compress/flate: optimize huffSymIlya Tocar
By using local variables and assigning them back to decompressor at the end of huffSym, we allow compiler to keep them in registers and avoid reloading/storing them repeatedly. To archive this, moreBits was inlined and specialized to work with local variables. Also move EOF error conversion to helper function, to make inlined part of moreBits more readable. Together this results in nice speed-up: name old time/op new time/op delta Decode/Digits/Huffman/1e4-6 278µs ± 1% 240µs ± 2% -13.72% (p=0.000 n=10+10) Decode/Digits/Huffman/1e5-6 2.38ms ± 1% 2.05ms ± 1% -14.12% (p=0.000 n=10+10) Decode/Digits/Huffman/1e6-6 23.4ms ± 1% 19.9ms ± 0% -14.69% (p=0.000 n=9+9) Decode/Digits/Speed/1e4-6 280µs ± 2% 254µs ± 1% -9.28% (p=0.000 n=10+9) Decode/Digits/Speed/1e5-6 2.53ms ± 1% 2.35ms ± 1% -7.17% (p=0.000 n=10+10) Decode/Digits/Speed/1e6-6 24.8ms ± 1% 23.0ms ± 1% -7.22% (p=0.000 n=10+10) Decode/Digits/Default/1e4-6 281µs ± 2% 259µs ± 3% -8.03% (p=0.000 n=10+10) Decode/Digits/Default/1e5-6 2.45ms ± 1% 2.30ms ± 1% -6.15% (p=0.000 n=10+10) Decode/Digits/Default/1e6-6 24.1ms ± 1% 22.6ms ± 0% -6.31% (p=0.000 n=9+9) Decode/Digits/Compression/1e4-6 279µs ± 2% 261µs ± 2% -6.53% (p=0.000 n=8+9) Decode/Digits/Compression/1e5-6 2.44ms ± 1% 2.30ms ± 1% -5.72% (p=0.000 n=10+9) Decode/Digits/Compression/1e6-6 24.0ms ± 1% 22.6ms ± 0% -6.10% (p=0.000 n=10+9) Decode/Twain/Huffman/1e4-6 316µs ± 2% 267µs ± 3% -15.30% (p=0.000 n=9+10) Decode/Twain/Huffman/1e5-6 2.62ms ± 0% 2.22ms ± 0% -15.24% (p=0.000 n=10+10) Decode/Twain/Huffman/1e6-6 25.7ms ± 1% 21.8ms ± 0% -15.19% (p=0.000 n=10+10) Decode/Twain/Speed/1e4-6 290µs ± 1% 264µs ± 2% -9.17% (p=0.000 n=9+10) Decode/Twain/Speed/1e5-6 2.35ms ± 1% 2.13ms ± 1% -9.74% (p=0.000 n=9+10) Decode/Twain/Speed/1e6-6 22.9ms ± 0% 20.7ms ± 0% -9.68% (p=0.000 n=10+9) Decode/Twain/Default/1e4-6 270µs ± 2% 252µs ± 2% -6.67% (p=0.000 n=9+10) Decode/Twain/Default/1e5-6 2.02ms ± 1% 1.84ms ± 1% -8.85% (p=0.000 n=10+10) Decode/Twain/Default/1e6-6 19.1ms ± 0% 17.5ms ± 1% -8.73% (p=0.000 n=9+10) Decode/Twain/Compression/1e4-6 272µs ± 1% 250µs ± 4% -8.20% (p=0.000 n=9+10) Decode/Twain/Compression/1e5-6 2.01ms ± 0% 1.84ms ± 1% -8.57% (p=0.000 n=9+10) Decode/Twain/Compression/1e6-6 19.1ms ± 0% 17.4ms ± 1% -8.75% (p=0.000 n=9+10) name old speed new speed delta Decode/Digits/Huffman/1e4-6 35.9MB/s ± 1% 41.7MB/s ± 2% +15.91% (p=0.000 n=10+10) Decode/Digits/Huffman/1e5-6 41.9MB/s ± 1% 48.8MB/s ± 1% +16.44% (p=0.000 n=10+10) Decode/Digits/Huffman/1e6-6 42.8MB/s ± 1% 50.2MB/s ± 0% +17.22% (p=0.000 n=9+9) Decode/Digits/Speed/1e4-6 35.7MB/s ± 2% 39.4MB/s ± 1% +10.22% (p=0.000 n=10+9) Decode/Digits/Speed/1e5-6 39.6MB/s ± 1% 42.6MB/s ± 1% +7.73% (p=0.000 n=10+10) Decode/Digits/Speed/1e6-6 40.3MB/s ± 1% 43.4MB/s ± 1% +7.78% (p=0.000 n=10+10) Decode/Digits/Default/1e4-6 35.6MB/s ± 2% 38.7MB/s ± 2% +8.74% (p=0.000 n=10+10) Decode/Digits/Default/1e5-6 40.9MB/s ± 1% 43.6MB/s ± 1% +6.55% (p=0.000 n=10+10) Decode/Digits/Default/1e6-6 41.5MB/s ± 1% 44.3MB/s ± 0% +6.73% (p=0.000 n=9+9) Decode/Digits/Compression/1e4-6 35.8MB/s ± 2% 38.3MB/s ± 2% +6.99% (p=0.000 n=8+9) Decode/Digits/Compression/1e5-6 40.9MB/s ± 1% 43.4MB/s ± 1% +6.07% (p=0.000 n=10+9) Decode/Digits/Compression/1e6-6 41.6MB/s ± 1% 44.3MB/s ± 0% +6.49% (p=0.000 n=10+9) Decode/Twain/Huffman/1e4-6 31.7MB/s ± 2% 37.4MB/s ± 3% +18.08% (p=0.000 n=9+10) Decode/Twain/Huffman/1e5-6 38.2MB/s ± 0% 45.0MB/s ± 0% +17.97% (p=0.000 n=10+10) Decode/Twain/Huffman/1e6-6 38.9MB/s ± 1% 45.9MB/s ± 0% +17.90% (p=0.000 n=10+10) Decode/Twain/Speed/1e4-6 34.5MB/s ± 1% 38.0MB/s ± 2% +10.11% (p=0.000 n=9+10) Decode/Twain/Speed/1e5-6 42.5MB/s ± 1% 47.0MB/s ± 1% +10.79% (p=0.000 n=9+10) Decode/Twain/Speed/1e6-6 43.7MB/s ± 0% 48.3MB/s ± 0% +10.72% (p=0.000 n=10+9) Decode/Twain/Default/1e4-6 37.1MB/s ± 2% 39.8MB/s ± 2% +7.15% (p=0.000 n=9+10) Decode/Twain/Default/1e5-6 49.5MB/s ± 1% 54.3MB/s ± 1% +9.71% (p=0.000 n=10+10) Decode/Twain/Default/1e6-6 52.3MB/s ± 0% 57.3MB/s ± 1% +9.57% (p=0.000 n=9+10) Decode/Twain/Compression/1e4-6 36.7MB/s ± 1% 40.0MB/s ± 4% +8.96% (p=0.000 n=9+10) Decode/Twain/Compression/1e5-6 49.8MB/s ± 0% 54.5MB/s ± 1% +9.38% (p=0.000 n=9+10) Decode/Twain/Compression/1e6-6 52.3MB/s ± 0% 57.3MB/s ± 1% +9.58% (p=0.000 n=9+10) Change-Id: Iabfd285535ddb210f7f48f33317c6463b5532400 Reviewed-on: https://go-review.googlesource.com/102235 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-04-02compress/gzip: do not count header bytes written in WriteTravis Bischel
Before, if an underlying writer errored within 10 bytes (plus any gzip header metadata), a gzip.Write would erroneously report up to 10 bytes written that were not actually written of the input slice. This is especially problematic when the input slice is less than 10 bytes. The error came from counting the 10 header byte write. If writing the header is completely successful, the 10 bytes written is overridden by the flate write with the input slice. This removes counting the 10 required header bytes, and also changes the return to use zero until the slice is used. The old Write could return one byte written when it actually was not. This is difficult to verify because the smallest input slice is one byte; a test checking that the input slice was the byte written would be quite involved. Thankfully, gzip's minimum header write is 10 bytes. If we test that two bytes are not falsely written, we indirectly cover the one byte case. Fixes #24625 Change-Id: I1c1f8cd791e0c4cffc22aa8acd95186582c832ba Reviewed-on: https://go-review.googlesource.com/103861 Reviewed-by: Joe Tsai <joetsai@google.com> Run-TryBot: Joe Tsai <joetsai@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-03-21compress/bzip2: remove bit-tricksIlya Tocar
Since compiler is now able to generate conditional moves, we can replace bit-tricks with simple if/else. This even results in slightly better performance: name old time/op new time/op delta DecodeDigits-6 13.4ms ± 4% 13.0ms ± 2% -2.63% (p=0.003 n=10+10) DecodeTwain-6 37.5ms ± 1% 36.3ms ± 1% -3.03% (p=0.000 n=10+9) DecodeRand-6 4.23ms ± 1% 4.07ms ± 1% -3.67% (p=0.000 n=10+9) name old speed new speed delta DecodeDigits-6 7.47MB/s ± 4% 7.67MB/s ± 2% +2.69% (p=0.002 n=10+10) DecodeTwain-6 10.4MB/s ± 1% 10.7MB/s ± 1% +3.25% (p=0.000 n=10+8) DecodeRand-6 3.87MB/s ± 1% 4.03MB/s ± 2% +4.08% (p=0.000 n=10+10) diff --git a/src/compress/bzip2/huffman.go b/src/compress/bzip2/huffman.go Change-Id: Ie96ef1a9e07013b07e78f22cdccd531f3341caca Reviewed-on: https://go-review.googlesource.com/102015 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> Reviewed-by: Joe Tsai <joetsai@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-03-13compress/gzip, compress/zlib: fix Writer documentation inconsistenciesDiogo Pinela
Fixes #24379. Change-Id: Ibdc763a0c2b56e26f4269f8be429880e34a2558f Reviewed-on: https://go-review.googlesource.com/100495 Reviewed-by: Joe Tsai <joetsai@google.com>
2018-02-17compress/bzip2: use sort.Slice in huffman.goJoe Kyo
Change-Id: Ie4d23cdb81473a4c989a977a127479cf825084dc Reviewed-on: https://go-review.googlesource.com/77850 Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-09-25compress/bzip2: fix checksum mismatch on empty readsJoe Tsai
Previously, the read method checked whether the current block was fully consumed or not based on whether the buffer could be filled with a non-zero number of bytes. This check is problematic because zero bytes could be read if the provided buffer is empty. We fix this case by simply checking for whether the input buffer provided by the user was empty or not. If empty, we assume that we could not read any bytes because the buffer was too small, rather than indicating that the current block was fully exhausted. This check causes bzip2.Reader to be unable to make progress on the next block unless a non-empty buffer is provided. However, that is an entirely reasonable expectation since a non-empty buffer needs to be provided eventually anyways to read the actual contents of subsequent blocks. Fixes #22028 Change-Id: I2bb1b2d54e78567baf2bf7b490a272c0853d7bfe Reviewed-on: https://go-review.googlesource.com/66110 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-09-15all: fix article typosKunpei Sakai
a -> an Change-Id: I7362bdc199e83073a712be657f5d9ba16df3077e Reviewed-on: https://go-review.googlesource.com/63850 Reviewed-by: Rob Pike <r@golang.org>
2017-08-31compress/flate: remove non-standard extensions to flateJoe Tsai
Some constants were added to flate that seem to be an experimental attempt at increasing the window size. However, according to RFC1951, the largest window size is 32KiB, so these constants are non-standard. Delete them. Fixes #18458 Change-Id: Ia94989637ca031a56bce2548624fa48044caa7b9 Reviewed-on: https://go-review.googlesource.com/60490 Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2017-06-08compress/lzw: don't follow code == hi if last is invalid.Nigel Tao
Fixes #20597. Change-Id: Iaa059277c02e80a2fbef59cf8bb3c58d2b90fda0 Reviewed-on: https://go-review.googlesource.com/45111 Reviewed-by: David Symonds <dsymonds@golang.org>
2017-06-02compress/gzip: clarify behavior of Writer.CloseJoe Tsai
Fixes #20551 Change-Id: Ia47cae14a26fe5f278ad7209218d083cc50a3ff8 Reviewed-on: https://go-review.googlesource.com/44572 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-05-24compress/bzip2: remove dead code in huffman.goJoe Tsai
The logic performs a series of shifts, which are useless given that they are followed by an assignment that overrides the value of the previous computation. I suspect (but cannot prove) that this is leftover logic from an original approach that attempted to store both the Huffman code and the length within the same variable instead of using two different variables as it currently does now. Fixes #17949 Change-Id: Ibf6c807c6cef3b28bfdaf2b68d9bc13503ac21b2 Reviewed-on: https://go-review.googlesource.com/44091 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-04-28compress/lzw: fix hi code overflow.Nigel Tao
Change-Id: I2d3c3c715d857305944cd96c45554a16cb7967e9 Reviewed-on: https://go-review.googlesource.com/42032 Reviewed-by: David Symonds <dsymonds@golang.org>
2017-03-04Revert "all: test adjustments for the iOS builder"Elias Naur
This reverts commit 467109bf56fb560d1fd8a27c6184dbfe4f64ffef. Replaced by a improved strategy later in the CL relation chain. Change-Id: Ib90813b5a6c4716b563c8496013d2d57f9c022b8 Reviewed-on: https://go-review.googlesource.com/36066 Reviewed-by: David Crawshaw <crawshaw@golang.org> Run-TryBot: Elias Naur <elias.naur@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-01compress/flate: use math/bits.Reverse8/16 instead of local implementationRobert Griesemer
No measurable impact on performance (specifically, no degradation). Reverse is used in Huffman en/de-coding. For completeness, here are all the speed-related benchmark results: name old time/op new time/op delta Decode/Digits/Huffman/1e4-8 181µs ± 0% 178µs ± 1% ~ (p=0.100 n=3+3) Decode/Digits/Huffman/1e5-8 1.60ms ± 3% 1.56ms ± 3% ~ (p=0.400 n=3+3) Decode/Digits/Huffman/1e6-8 15.7ms ± 1% 15.3ms ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e4-8 179µs ± 0% 180µs ± 0% ~ (p=0.200 n=3+3) Decode/Digits/Speed/1e5-8 1.68ms ± 0% 1.66ms ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e6-8 16.6ms ± 2% 16.6ms ± 5% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e4-8 179µs ± 1% 178µs ± 1% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e5-8 1.62ms ± 3% 1.62ms ± 4% ~ (p=1.000 n=3+3) Decode/Digits/Default/1e6-8 16.0ms ± 2% 16.0ms ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e4-8 179µs ± 1% 179µs ± 0% ~ (p=0.200 n=3+3) Decode/Digits/Compression/1e5-8 1.62ms ± 2% 1.62ms ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e6-8 16.1ms ± 3% 16.0ms ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e4-8 205µs ± 2% 207µs ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e5-8 1.77ms ± 2% 1.77ms ± 4% ~ (p=0.700 n=3+3) Decode/Twain/Huffman/1e6-8 17.4ms ± 2% 17.4ms ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Speed/1e4-8 186µs ± 1% 186µs ± 1% ~ (p=0.400 n=3+3) Decode/Twain/Speed/1e5-8 1.53ms ± 2% 1.52ms ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Speed/1e6-8 14.9ms ± 1% 14.8ms ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Default/1e4-8 176µs ± 1% 174µs ± 0% ~ (p=0.200 n=3+3) Decode/Twain/Default/1e5-8 1.30ms ± 2% 1.31ms ± 1% ~ (p=0.700 n=3+3) Decode/Twain/Default/1e6-8 12.6ms ± 3% 12.5ms ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e4-8 177µs ± 0% 174µs ± 1% ~ (p=0.100 n=3+3) Decode/Twain/Compression/1e5-8 1.30ms ± 1% 1.31ms ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e6-8 12.5ms ± 1% 12.5ms ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Huffman/1e4-8 47.4µs ± 1% 46.5µs ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Huffman/1e5-8 453µs ± 2% 446µs ± 1% ~ (p=0.700 n=3+3) Encode/Digits/Huffman/1e6-8 4.44ms ± 3% 4.39ms ± 0% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e4-8 190µs ± 4% 185µs ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Speed/1e5-8 1.78ms ± 5% 1.75ms ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e6-8 17.9ms ± 7% 17.3ms ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e4-8 366µs ± 1% 361µs ± 0% ~ (p=0.200 n=3+3) Encode/Digits/Default/1e5-8 5.58ms ± 5% 5.44ms ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e6-8 59.0ms ± 3% 58.2ms ± 1% ~ (p=0.700 n=3+3) Encode/Digits/Compression/1e4-8 369µs ± 3% 362µs ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Compression/1e5-8 5.50ms ± 2% 5.47ms ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Compression/1e6-8 59.4ms ± 2% 58.5ms ± 1% ~ (p=0.400 n=3+3) Encode/Twain/Huffman/1e4-8 64.4µs ± 3% 64.7µs ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Huffman/1e5-8 526µs ± 1% 526µs ± 2% ~ (p=1.000 n=3+3) Encode/Twain/Huffman/1e6-8 5.18ms ± 2% 5.17ms ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Speed/1e4-8 206µs ± 1% 204µs ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e5-8 1.73ms ± 2% 1.70ms ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e6-8 16.7ms ± 0% 16.7ms ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e4-8 423µs ± 3% 418µs ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e5-8 6.34ms ± 4% 6.23ms ± 0% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e6-8 68.0ms ± 3% 67.5ms ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e4-8 435µs ± 3% 424µs ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e5-8 7.01ms ± 1% 6.92ms ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Compression/1e6-8 77.1ms ± 4% 75.5ms ± 1% ~ (p=0.400 n=3+3) name old speed new speed delta Decode/Digits/Huffman/1e4-8 55.2MB/s ± 0% 56.2MB/s ± 1% ~ (p=0.100 n=3+3) Decode/Digits/Huffman/1e5-8 62.4MB/s ± 3% 64.1MB/s ± 3% ~ (p=0.400 n=3+3) Decode/Digits/Huffman/1e6-8 63.8MB/s ± 1% 65.3MB/s ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e4-8 55.8MB/s ± 0% 55.4MB/s ± 0% ~ (p=0.200 n=3+3) Decode/Digits/Speed/1e5-8 59.6MB/s ± 0% 60.3MB/s ± 3% ~ (p=0.700 n=3+3) Decode/Digits/Speed/1e6-8 60.1MB/s ± 2% 60.3MB/s ± 4% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e4-8 55.8MB/s ± 1% 56.1MB/s ± 1% ~ (p=0.700 n=3+3) Decode/Digits/Default/1e5-8 61.8MB/s ± 3% 61.7MB/s ± 4% ~ (p=1.000 n=3+3) Decode/Digits/Default/1e6-8 62.4MB/s ± 2% 62.4MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e4-8 55.7MB/s ± 1% 56.0MB/s ± 0% ~ (p=0.300 n=3+3) Decode/Digits/Compression/1e5-8 61.7MB/s ± 2% 61.9MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Digits/Compression/1e6-8 62.2MB/s ± 3% 62.6MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e4-8 48.8MB/s ± 2% 48.4MB/s ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Huffman/1e5-8 56.4MB/s ± 2% 56.6MB/s ± 4% ~ (p=0.700 n=3+3) Decode/Twain/Huffman/1e6-8 57.6MB/s ± 2% 57.5MB/s ± 3% ~ (p=1.000 n=3+3) Decode/Twain/Speed/1e4-8 53.7MB/s ± 1% 53.9MB/s ± 1% ~ (p=0.400 n=3+3) Decode/Twain/Speed/1e5-8 65.5MB/s ± 2% 65.6MB/s ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Speed/1e6-8 66.9MB/s ± 1% 67.4MB/s ± 1% ~ (p=1.000 n=3+3) Decode/Twain/Default/1e4-8 56.9MB/s ± 1% 57.3MB/s ± 0% ~ (p=0.200 n=3+3) Decode/Twain/Default/1e5-8 77.2MB/s ± 2% 76.6MB/s ± 1% ~ (p=0.700 n=3+3) Decode/Twain/Default/1e6-8 79.3MB/s ± 3% 80.0MB/s ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e4-8 56.4MB/s ± 0% 57.5MB/s ± 1% ~ (p=0.100 n=3+3) Decode/Twain/Compression/1e5-8 76.8MB/s ± 1% 76.5MB/s ± 0% ~ (p=0.700 n=3+3) Decode/Twain/Compression/1e6-8 80.1MB/s ± 1% 79.8MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Huffman/1e4-8 211MB/s ± 1% 215MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Huffman/1e5-8 221MB/s ± 2% 224MB/s ± 1% ~ (p=0.700 n=3+3) Encode/Digits/Huffman/1e6-8 225MB/s ± 3% 228MB/s ± 0% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e4-8 52.8MB/s ± 4% 54.1MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Speed/1e5-8 56.2MB/s ± 5% 57.0MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Speed/1e6-8 56.0MB/s ± 6% 57.7MB/s ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e4-8 27.3MB/s ± 1% 27.7MB/s ± 0% ~ (p=0.200 n=3+3) Encode/Digits/Default/1e5-8 17.9MB/s ± 4% 18.4MB/s ± 1% ~ (p=0.400 n=3+3) Encode/Digits/Default/1e6-8 17.0MB/s ± 3% 17.2MB/s ± 1% ~ (p=0.500 n=3+3) Encode/Digits/Compression/1e4-8 27.1MB/s ± 3% 27.6MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Digits/Compression/1e5-8 18.2MB/s ± 2% 18.3MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Digits/Compression/1e6-8 16.9MB/s ± 2% 17.1MB/s ± 1% ~ (p=0.400 n=3+3) Encode/Twain/Huffman/1e4-8 155MB/s ± 3% 155MB/s ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Huffman/1e5-8 190MB/s ± 1% 190MB/s ± 2% ~ (p=1.000 n=3+3) Encode/Twain/Huffman/1e6-8 193MB/s ± 2% 193MB/s ± 1% ~ (p=0.700 n=3+3) Encode/Twain/Speed/1e4-8 48.5MB/s ± 1% 49.1MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e5-8 57.7MB/s ± 2% 59.0MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Speed/1e6-8 59.7MB/s ± 0% 59.7MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e4-8 23.6MB/s ± 3% 23.9MB/s ± 1% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e5-8 15.8MB/s ± 4% 16.1MB/s ± 0% ~ (p=1.000 n=3+3) Encode/Twain/Default/1e6-8 14.7MB/s ± 3% 14.8MB/s ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e4-8 23.0MB/s ± 3% 23.6MB/s ± 0% ~ (p=0.700 n=3+3) Encode/Twain/Compression/1e5-8 14.3MB/s ± 1% 14.5MB/s ± 0% ~ (p=0.100 n=3+3) Encode/Twain/Compression/1e6-8 13.0MB/s ± 4% 13.2MB/s ± 1% ~ (p=0.400 n=3+3) Measured on a "quiet" (no browser running) 2.3 GHz Intel Core i7, running macOS 10.12.3. See also #19279. Change-Id: Ice759eb34eb37442b543957447c264e0aadc1fa9 Reviewed-on: https://go-review.googlesource.com/37460 Run-TryBot: Robert Griesemer <gri@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-01-14all: test adjustments for the iOS builderDavid Crawshaw
The working directory is now adjusted to match the typical Go test working directory in main, as the old trick for adjusting earlier stopped working with the latest version of LLDB bugs. That means the small number of places where testdata files are read before main is called no longer work. This CL adjusts those reads to happen after main is called. (This has the bonus effect of not reading some benchmark testdata files in all.bash.) Fixes compress/bzip2, go/doc, go/parser, os, and time package tests on the iOS builder. Change-Id: If60f026aa7848b37511c36ac5e3985469ec25209 Reviewed-on: https://go-review.googlesource.com/35255 Run-TryBot: David Crawshaw <crawshaw@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-01-12compress/flate: avoid large stack growth in fillDeflateJoe Tsai
Ranging over an array causes the array to be copied over to the stack, which cause large re-growths. Instead, we should iterate over slices of the array. Also, assigning a large struct literal uses the stack even though the actual fields being populated are small in comparison to the entirety of the struct (see #18636). Fixing the stack growth does not alter CPU-time performance much since the stack-growth and copying was such a tiny portion of the compression work: name old time/op new time/op delta Encode/Digits/Default/1e4-8 332µs ± 1% 332µs ± 1% ~ (p=0.796 n=10+10) Encode/Digits/Default/1e5-8 5.07ms ± 2% 5.05ms ± 1% ~ (p=0.815 n=9+8) Encode/Digits/Default/1e6-8 53.7ms ± 1% 53.9ms ± 1% ~ (p=0.075 n=10+10) Encode/Twain/Default/1e4-8 380µs ± 1% 380µs ± 1% ~ (p=0.684 n=10+10) Encode/Twain/Default/1e5-8 5.79ms ± 2% 5.79ms ± 1% ~ (p=0.497 n=9+10) Encode/Twain/Default/1e6-8 61.5ms ± 1% 61.8ms ± 1% ~ (p=0.247 n=10+10) name old speed new speed delta Encode/Digits/Default/1e4-8 30.1MB/s ± 1% 30.1MB/s ± 1% ~ (p=0.753 n=10+10) Encode/Digits/Default/1e5-8 19.7MB/s ± 2% 19.8MB/s ± 1% ~ (p=0.795 n=9+8) Encode/Digits/Default/1e6-8 18.6MB/s ± 1% 18.5MB/s ± 1% ~ (p=0.072 n=10+10) Encode/Twain/Default/1e4-8 26.3MB/s ± 1% 26.3MB/s ± 1% ~ (p=0.616 n=10+10) Encode/Twain/Default/1e5-8 17.3MB/s ± 2% 17.3MB/s ± 1% ~ (p=0.484 n=9+10) Encode/Twain/Default/1e6-8 16.3MB/s ± 1% 16.2MB/s ± 1% ~ (p=0.238 n=10+10) Updates #18636 Fixes #18625 Change-Id: I471b20339bf675f63dc56d38b3acdd824fe23328 Reviewed-on: https://go-review.googlesource.com/35122 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>