aboutsummaryrefslogtreecommitdiff
path: root/src/encoding
AgeCommit message (Collapse)Author
2022-01-13all: add a handful of fuzz targetsRoland Shoemaker
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip, encoding/json, image/jpeg, image/gif, and image/png. Second attempt, this time we don't use the archives in testdata when fuzzing archive/tar, since those are rather memory intensive, and were crashing a number of builders. Change-Id: I4828d64fa4763c0d8c980392a6578e4dfd956e13 Reviewed-on: https://go-review.googlesource.com/c/go/+/378174 Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> Reviewed-by: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org>
2022-01-12Revert "all: add a handful of fuzz targets"Bryan Mills
This reverts CL 352109. Reason for revert: causing OOM failures on several builders, and may cause OOMs for end users with small machines as well. Change-Id: I58308d09919969d5a6512ee5cee6aa5c4af6769b Reviewed-on: https://go-review.googlesource.com/c/go/+/377934 Trust: Bryan Mills <bcmills@google.com> Run-TryBot: Bryan Mills <bcmills@google.com> TryBot-Result: Gopher Robot <gobot@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Trust: Katie Hockman <katie@golang.org>
2022-01-12all: add a handful of fuzz targetsRoland Shoemaker
Adds simple fuzz targets to archive/tar, archive/zip, compress/gzip, encoding/json, image/jpeg, image/gif, and image/png. Change-Id: Ide1a8de88a9421e786eeeaea3bb93f41e0bae347 Reviewed-on: https://go-review.googlesource.com/c/go/+/352109 Trust: Katie Hockman <katie@golang.org> Reviewed-by: Katie Hockman <katie@golang.org> Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Roland Shoemaker <roland@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-13all: gofmt -w -r 'interface{} -> any' srcRuss Cox
And then revert the bootstrap cmd directories and certain testdata. And adjust tests as needed. Not reverting the changes in std that are bootstrapped, because some of those changes would appear in API docs, and we want to use any consistently. Instead, rewrite 'any' to 'interface{}' in cmd/dist for those directories when preparing the bootstrap copy. A few files changed as a result of running gofmt -w not because of interface{} -> any but because they hadn't been updated for the new //go:build lines. Fixes #49884. Change-Id: Ie8045cba995f65bd79c694ec77a1b3d1fe01bb09 Reviewed-on: https://go-review.googlesource.com/c/go/+/368254 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> Reviewed-by: Robert Griesemer <gri@golang.org> TryBot-Result: Gopher Robot <gobot@golang.org>
2021-11-09encoding/xml: add generic encoding testKeith Randall
Fixes #48521 Change-Id: Id8402bcff243c0ab19e4ec0b138b9af8c111f88d Reviewed-on: https://go-review.googlesource.com/c/go/+/355492 Trust: Keith Randall <khr@golang.org> Trust: Dan Scales <danscales@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Dan Scales <danscales@google.com>
2021-11-02encoding: simplify some codeJosh Bleecher Snyder
Change-Id: I63aac663da4ce7bdd198d385f87929c74f6a5cf8 Reviewed-on: https://go-review.googlesource.com/c/go/+/360601 Trust: Josh Bleecher Snyder <josharian@gmail.com> Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-10-28all: go fix -fix=buildtag std cmd (except for bootstrap deps, vendor)Russ Cox
When these packages are released as part of Go 1.18, Go 1.16 will no longer be supported, so we can remove the +build tags in these files. Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs as defined in src/cmd/dist/buildtool.go, which need to continue to build with Go 1.4 for now. Also reverted src/vendor and src/cmd/vendor, which will need to be updated in their own repos first. Manual changes in runtime/pprof/mprof_test.go to adjust line numbers. For #41184. Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e Reviewed-on: https://go-review.googlesource.com/c/go/+/344955 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-26all: use reflect.{Pointer,PointerTo}Cuong Manh Le
Updates #47651 Updates #48665 Change-Id: I69a87b45a5cad7a07fbd855040cd9935cf874554 Reviewed-on: https://go-review.googlesource.com/c/go/+/358454 Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com> Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-10-19encoding/base64: add examples for Encode/Decode180909
Fixes #37595 Change-Id: I83e5f6105748a0a9238322a4f7ec4b0bbf61a263 Reviewed-on: https://go-review.googlesource.com/c/go/+/348394 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Daniel Martí <mvdan@mvdan.cc> Trust: Cherry Mui <cherryyz@google.com>
2021-10-11encoding/gob: follow documented io.EOF semanticsDaniel Martí
The docs say: If the input is at EOF, Decode returns io.EOF and does not modify e. However, the added test fails: --- FAIL: TestDecodePartial (0.00s) encoder_test.go:1263: 31/81: expected io.ErrUnexpectedEOF: EOF encoder_test.go:1263: 51/81: expected io.ErrUnexpectedEOF: EOF In particular, the decoder would return io.EOF after reading a valid message for a type specification, and then hit EOF before reading a data item message. Fix that by only allowing a Decode call to return io.EOF if the reader hits EOF immediately, without successfully reading any message. Otherwise, hitting EOF is an ErrUnexpectedEOF, like in other cases. Also fix a net/rpc test that, coincidentally, expected an io.EOF as an error when feeding bad non-zero data to a gob decoder. An io.ErrUnexpectedEOF is clearly better in that scenario. Fixes #48905. Change-Id: Ied6a0d8ac8377f89646319a18c0380c4f2b09b85 Reviewed-on: https://go-review.googlesource.com/c/go/+/354972 Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2021-10-06all: use bytes.Cut, strings.CutRuss Cox
Many uses of Index/IndexByte/IndexRune/Split/SplitN can be written more clearly using the new Cut functions. Do that. Also rewrite to other functions if that's clearer. For #46336. Change-Id: I68d024716ace41a57a8bf74455c62279bde0f448 Reviewed-on: https://go-review.googlesource.com/c/go/+/351711 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-09-27encoding/base32: Add examples for Encode/Decodejiahua wang
Updates golang/go#37595 Change-Id: I7568e7416d5504e9dc67061c79f66e3a0d597dee Reviewed-on: https://go-review.googlesource.com/c/go/+/351470 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: David Chase <drchase@google.com>
2021-09-21encoding/xml: truncate generic type namesKeith Randall
xml names can't have any of '[],' in them, which might appear in generic type names. Truncate at the first '[' so the names are still valid. Fixes #48318 Change-Id: I110ff4269f763089467e7cf84b0f0c5075fb44b7 Reviewed-on: https://go-review.googlesource.com/c/go/+/349349 Trust: Keith Randall <khr@golang.org> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org>
2021-09-07encoding/gob: optimize decoding of slicekorzhao
In CL 345572, we used the reflect.Value.SetLen method to avoid extra memory allocation for reflect.Value.Slice. This also applies to function decodeSlice name old time/op new time/op delta DecodeStringsSlice-12 96.5µs ±12% 63.0µs ± 8% -34.68% (p=0.000 n=9+10) name old alloc/op new alloc/op delta DecodeStringsSlice-12 89.3kB ± 0% 65.3kB ± 0% -26.89% (p=0.000 n=10+10) name old allocs/op new allocs/op delta DecodeStringsSlice-12 3.18k ± 0% 2.18k ± 0% -31.47% (p=0.000 n=10+10) Change-Id: Ifdb43716cc90a265962dec022704a5571f447fd8 Reviewed-on: https://go-review.googlesource.com/c/go/+/347533 Reviewed-by: Joe Tsai <joetsai@digital-static.net> Reviewed-by: Rob Pike <r@golang.org> Trust: Joe Tsai <joetsai@digital-static.net> Run-TryBot: Joe Tsai <joetsai@digital-static.net> TryBot-Result: Go Bot <gobot@golang.org>
2021-09-07encoding/gob: marshal maps using reflect.Value.MapRangekorzhao
golang.org/cl/33572 added a map iterator. use the reflect.Value.MapRange to fix map keys that contain a NaN Fixes #24075 Change-Id: I0214d6f26c2041797703e48eac16404f189d6982 GitHub-Last-Rev: 5c01e117f4451dbaec657d02d006905df1d0055d GitHub-Pull-Request: golang/go#47476 Reviewed-on: https://go-review.googlesource.com/c/go/+/338609 Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2021-08-27encoding/gob: optimize decoding of []byteJoe Tsai
The reflect.Value.Slice method unfortunately allocates every time since it needs to place the slice header on the heap. This is silly since gob immediately stores the result back into slice. Instead, use the reflect.Value.SetLen method. DecodeBytesSlice 75.0µs ± 2% 35.2µs ± 6% -53.02% Change-Id: I3ca0529d01bf978f2b76e215f52d369f458951ef Reviewed-on: https://go-review.googlesource.com/c/go/+/345572 Trust: Joe Tsai <joetsai@digital-static.net> Run-TryBot: Joe Tsai <joetsai@digital-static.net> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2021-04-29encoding/csv: add FieldPos methodRoger Peppe
This enables a consumer of a CSV to find out the position of a CSV field without implementing an intermediate buffer. This is useful to produce good higher level error messages when the CSV syntax is OK but the field values don't match expectations. This also changes the existing semantics of the `ParseError.Column` field to bring it in line with precedent elsewhere in the Go standard library (notably go/token.Position) - the column is now 1-based and indicates a byte count rather than a rune count, and the error position reporting at the end of a last line without a newline is now fixed. This change has some impact on performance: ``` name old time/op new time/op delta Read-8 2.14µs ± 0% 2.15µs ± 0% ~ (p=0.056 n=5+5) ReadWithFieldsPerRecord-8 2.15µs ± 2% 2.15µs ± 1% ~ (p=0.151 n=5+5) ReadWithoutFieldsPerRecord-8 2.15µs ± 0% 2.15µs ± 0% +0.37% (p=0.024 n=5+5) ReadLargeFields-8 3.55µs ± 2% 3.59µs ± 0% ~ (p=0.206 n=5+5) ReadReuseRecord-8 1.18µs ± 1% 1.22µs ± 1% +2.93% (p=0.008 n=5+5) ReadReuseRecordWithFieldsPerRecord-8 1.18µs ± 0% 1.21µs ± 0% +2.54% (p=0.008 n=5+5) ReadReuseRecordWithoutFieldsPerRecord-8 1.18µs ± 0% 1.22µs ± 1% +3.66% (p=0.008 n=5+5) ReadReuseRecordLargeFields-8 2.53µs ± 1% 2.57µs ± 1% +1.70% (p=0.008 n=5+5) Write-8 1.02µs ± 1% 1.01µs ± 0% -1.18% (p=0.016 n=5+4) ``` Fixes #44221. Change-Id: Id37c50fc396024eef406c5bad45380ecd414f5ea Reviewed-on: https://go-review.googlesource.com/c/go/+/291290 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Paul Jolly <paul@myitcv.org.uk>
2021-04-20all: remove redundant spaces before . and ,Yury Smolsky
Change-Id: I6a4bd2544276d0638bddf07ebcf2ee636db30fea Reviewed-on: https://go-review.googlesource.com/c/go/+/311009 Run-TryBot: Yury Smolsky <yury@smolsky.by> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-15encoding/xml: replace comments inside directives with a spaceFilippo Valsorda
A Directive (like <!ENTITY xxx []>) can't have other nodes nested inside it (in our data structure representation), so there is no way to preserve comments. The previous behavior was to just elide them, which however might change the semantic meaning of the surrounding markup. Instead, replace them with a space which hopefully has the same semantic effect of the comment. Directives are not actually a node type in the XML spec, which instead specifies each of them separately (<!ENTITY, <!DOCTYPE, etc.), each with its own grammar. The rules for where and when the comments are allowed are not straightforward, and can't be implemented without implementing custom logic for each of the directives. Simply preserving the comments in the body of the directive would be problematic, as there can be unmatched quotes inside the comment. Whether those quotes are considered meaningful semantically or not, other parsers might disagree and interpret the output differently. This issue was reported by Juho Nurminen of Mattermost as it leads to round-trip mismatches. See #43168. It's not being fixed in a security release because round-trip stability is not a currently supported security property of encoding/xml, and we don't believe these fixes would be sufficient to reliably guarantee it in the future. Fixes CVE-2020-29510 Updates #43168 Change-Id: Icd86c75beff3e1e0689543efebdad10ed5178ce3 Reviewed-on: https://go-review.googlesource.com/c/go/+/277893 Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Filippo Valsorda <filippo@golang.org> Reviewed-by: Katie Hockman <katie@golang.org>
2021-03-15encoding/xml: handle leading, trailing, or double colons in namesFilippo Valsorda
Before this change, <:name> would parse as <name>, which could cause issues in applications that rely on the parse-encode cycle to round-trip. Similarly, <x name:=""> would parse as expected but then have the attribute dropped when serializing because its name was empty. Finally, <a:b:c> would parse and get serialized incorrectly. All these values are invalid XML, but to minimize the impact of this change, we parse them whole into Name.Local. This issue was reported by Juho Nurminen of Mattermost as it leads to round-trip mismatches. See #43168. It's not being fixed in a security release because round-trip stability is not a currently supported security property of encoding/xml, and we don't believe these fixes would be sufficient to reliably guarantee it in the future. Fixes CVE-2020-29509 Fixes CVE-2020-29511 Updates #43168 Change-Id: I68321c4d867305046f664347192948a889af3c7f Reviewed-on: https://go-review.googlesource.com/c/go/+/277892 Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Filippo Valsorda <filippo@golang.org> Reviewed-by: Katie Hockman <katie@golang.org>
2021-03-14encoding/json: fix package shadowing in MarshalIndent exampleAriel Mashraki
Prior to this CL, pasting the example from the website causes a compilation error for some programs because it was shadowing the "json" package. Change-Id: I39b68a66ca99468547f2027a7655cf1387b61e95 Reviewed-on: https://go-review.googlesource.com/c/go/+/301492 Reviewed-by: Joe Tsai <joetsai@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Joe Tsai <joetsai@google.com> Run-TryBot: Joe Tsai <joetsai@google.com> TryBot-Result: Go Bot <gobot@golang.org>
2021-03-13encoding/gob: ensure "duplicate type received" decoder errors surface upAman Karmani
Previously re-using a decoder with a new stream resulted in a confusing "extra data in buffer" error message. Change-Id: Ia4c4c3a2d4b63c59e37e53faa61a500d5ff6e5f1 Reviewed-on: https://go-review.googlesource.com/c/go/+/297949 Reviewed-by: Rob Pike <r@golang.org> Run-TryBot: Rob Pike <r@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-13all: use HTML5 br tagsJohn Bampton
In HTML5 br tags don't need a closing slash Change-Id: Ic53c43faee08c5b1267daa9a02cc186b1c255ca1 GitHub-Last-Rev: 652208116944d01b23b8af8f1af485da5e916d32 GitHub-Pull-Request: golang/go#44283 Reviewed-on: https://go-review.googlesource.com/c/go/+/292370 Reviewed-by: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-10encoding/xml: prevent infinite loop while decodingKatie Hockman
This change properly handles a TokenReader which returns an EOF in the middle of an open XML element. Thanks to Sam Whited for reporting this. Fixes CVE-2021-27918 Fixes #44913 Change-Id: Id02a3f3def4a1b415fa2d9a8e3b373eb6cb0f433 Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004594 Reviewed-by: Russ Cox <rsc@google.com> Reviewed-by: Roland Shoemaker <bracewell@google.com> Reviewed-by: Filippo Valsorda <valsorda@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/300391 Trust: Katie Hockman <katie@golang.org> Run-TryBot: Katie Hockman <katie@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-03-08encoding/binary: limit bytes read by Uvarint to <= 10Emmanuel T Odeke
Limits the number of bytes that can be consumed by Uvarint to MaxVarintLen64 (10) to avoid wasted computations. With this change, if Uvarint reads more than MaxVarintLen64 bytes, it'll return the erroring byte count of n=-(MaxVarintLen64+1) which is -11, as per the function signature. Updated some tests to reflect the new change in expectations of n when the number of bytes to be read exceeds the limits.. Fixes #41185 Change-Id: Ie346457b1ddb0214b60c72e81128e24d604d083d Reviewed-on: https://go-review.googlesource.com/c/go/+/299531 Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org> Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-05encoding/json: marshal maps using reflect.Value.MapRangeMeng Zhuo
Map serialization using reflect.Value.MapIndex cannot retrieve map keys that contain a NaN, resulting in a panic. Switch the implementation to use the reflect.Value.MapRange method instead, which iterates over all map entries regardless of whether they are directly retrievable. Note that according to RFC 8259, section 4, a JSON object should have unique names, but does not forbid the occurrence of duplicate names. Fixes #43207 Change-Id: If4bc55229b1f64b8ca4b0fed37549725efdace39 Reviewed-on: https://go-review.googlesource.com/c/go/+/278632 Trust: Meng Zhuo <mzh@golangcn.org> Trust: Joe Tsai <thebrokentoaster@gmail.com> Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Meng Zhuo <mzh@golangcn.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2021-02-25reflect: add Method.IsExported and StructField.IsExported methodsJoe Tsai
The IsExported method is a more intuitive helper for checking whether the method or field is exported than checking whether PkgPath is empty. In the same CL, modify the standard library to make use of this helper. Fixes #41563 Change-Id: Iaacfb3b74449501f98e2707aa32095a32bd3c3c1 Reviewed-on: https://go-review.googlesource.com/c/go/+/266197 Trust: Joe Tsai <joetsai@digital-static.net> Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-02-24encoding/json: reduce allocated space in UnmarshalJosh Bleecher Snyder
The decodeState type is a large part of the allocated space during Unmarshal. The errorContext field is infrequently used, and only on error. Extract it into a pointer and allocate it separate when necessary. name old time/op new time/op delta UnmarshalString-8 115ns ± 5% 114ns ± 3% ~ (p=0.170 n=15+15) UnmarshalFloat64-8 113ns ± 2% 106ns ± 1% -6.42% (p=0.000 n=15+14) UnmarshalInt64-8 93.3ns ± 1% 86.9ns ± 4% -6.89% (p=0.000 n=14+15) name old alloc/op new alloc/op delta UnmarshalString-8 192B ± 0% 160B ± 0% -16.67% (p=0.000 n=15+15) UnmarshalFloat64-8 180B ± 0% 148B ± 0% -17.78% (p=0.000 n=15+15) UnmarshalInt64-8 176B ± 0% 144B ± 0% -18.18% (p=0.000 n=15+15) name old allocs/op new allocs/op delta UnmarshalString-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) UnmarshalFloat64-8 2.00 ± 0% 2.00 ± 0% ~ (all equal) UnmarshalInt64-8 1.00 ± 0% 1.00 ± 0% ~ (all equal) Change-Id: I53f3f468e6c65f77a12e5138a2626455b197012d Reviewed-on: https://go-review.googlesource.com/c/go/+/271338 Trust: Josh Bleecher Snyder <josharian@gmail.com> Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2021-02-20all: go fmt std cmd (but revert vendor)Russ Cox
Make all our package sources use Go 1.17 gofmt format (adding //go:build lines). Part of //go:build change (#41184). See https://golang.org/design/draft-gobuild Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/294430 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-01-12encoding/asn1: document unmarshaling behavior for IMPLICIT string fieldsEric Chiang
Fixes #42570. Change-Id: I73e339cdebe1720c141861a12e28a94cef13c75b Reviewed-on: https://go-review.googlesource.com/c/go/+/269798 Reviewed-by: Katie Hockman <katie@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org> Run-TryBot: Katie Hockman <katie@golang.org> Trust: Emmanuel Odeke <emmanuel@orijtech.com> Trust: Katie Hockman <katie@golang.org>
2020-12-01encoding/json: revert "add "json: " prefix to SyntaxError messages"Kevin Burke
This reverts commit 6af088bfc66c13143c9ef46b4cf0805df77a8fbe. Reason for revert: Broke many tests inside Google which implies many tests were broken outside of Google as well. The tests may be brittle but still would require work to change and it's not clear it's worth the benefit. Updates #36221 Fixes #42675 Change-Id: Id3a14eb37e7119f5abe50e80dfbf120fdc44db72 Reviewed-on: https://go-review.googlesource.com/c/go/+/273747 Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Russ Cox <rsc@golang.org> Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com> Trust: Joe Tsai <thebrokentoaster@gmail.com>
2020-11-09encoding/asn1: optimize asn1.UnmarshalDmitrii Okunev
Used type-switch instead of switch by reflect.Type and added BenchmarkUnmarshal. name old time/op new time/op delta Marshal-8 28.1µs ± 2% 27.9µs ± 1% ~ (p=0.094 n=9+9) Unmarshal-8 6.45µs ± 1% 5.83µs ± 4% -9.59% (p=0.000 n=10+10) name old alloc/op new alloc/op delta Marshal-8 8.26kB ± 0% 8.26kB ± 0% ~ (all equal) Unmarshal-8 840B ± 0% 488B ± 0% -41.90% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Marshal-8 363 ± 0% 363 ± 0% ~ (all equal) Unmarshal-8 50.0 ± 0% 43.0 ± 0% -14.00% (p=0.000 n=10+10) Change-Id: I6b53833c7a3e2524f025453311841d03c1256a45 GitHub-Pull-Request: golang/go#36341 Reviewed-on: https://go-review.googlesource.com/c/go/+/268557 Trust: Filippo Valsorda <filippo@golang.org> Trust: Roland Shoemaker <roland@golang.org> Run-TryBot: Filippo Valsorda <filippo@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Roland Shoemaker <roland@golang.org>
2020-10-21encoding/xml: fix reserved namespace check to be case-insensitiveIan Lance Taylor
Fixes the check for the reserved namespace prefix "xml" to be case insensitive, so as to match all variants of: (('X'|'x')('M'|'m')('L'|'l')) as mandated by Section 2.3 of https://www.w3.org/TR/REC-xml/ This is a roll forward of CL 203417, which was rolled back by CL 240179. We've decided that the roll back was incorrect, and any broken tests should be fixed. The original CL 203417 was by Tamás Gulácsi. Fixes #35151 For #39876 Change-Id: I2e6daa7aeb252531fba0b8a56086613e13059528 Reviewed-on: https://go-review.googlesource.com/c/go/+/264024 Trust: Ian Lance Taylor <iant@golang.org> Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Damien Neil <dneil@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-21encoding/json: add "json: " prefix to SyntaxError messagesKevin Burke
The other named errors - UnmarshalTypeError, etc - in this package do the same, so we should prepend the package prefix to error messages for consistency. Add a note to the release docs in case this is interpreted as a breaking change. Fixes #36221. Change-Id: Ie24b532bbf9812e108c259fa377e2a6b64319ed4 Reviewed-on: https://go-review.googlesource.com/c/go/+/263619 Run-TryBot: Kevin Burke <kev@inburke.com> TryBot-Result: Go Bot <gobot@golang.org> Trust: Kevin Burke <kev@inburke.com> Trust: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2020-10-20all: update references to symbols moved from io/ioutil to ioRuss Cox
The old ioutil references are still valid, but update our code to reflect best practices and get used to the new locations. Code compiled with the bootstrap toolchain (cmd/asm, cmd/dist, cmd/compile, debug/elf) must remain Go 1.4-compatible and is excluded. Also excluded vendored code. For #41190. Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1 Reviewed-on: https://go-review.googlesource.com/c/go/+/263142 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-05encoding/asn1: clarify use of SET suffixRoland Bracewell Shoemaker
This change clarifies the usage of the SET type name suffix. Previously the documentation was somewhat confusing about where the suffix should be used, and when used what it applied to. For instance the previous language could be interpreted such that []exampleSET would be parsed as a SEQUENCE OF SET, which is incorrect as the SET suffix only applies to slice types, such as type exampleSET []struct{} which is parsed as a SET OF SEQUENCE. Change-Id: I74201d9969f931f69391c236559f66cb460569ec GitHub-Last-Rev: d0d2ddc587df4564a265c800efb9d8e204002624 GitHub-Pull-Request: golang/go#38543 Reviewed-on: https://go-review.googlesource.com/c/go/+/229078 Trust: Roland Shoemaker <roland@golang.org> Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-09-29encoding/asn1: error instead of panic on invalid value to UnmarshalKimMachineGun
Changes Unmarshal to return an error, instead of panicking when its value is nil or not a pointer. This change matches the behavior of other encoding packages like json. Fixes #41509. Change-Id: I92c3af3a960144566e4c2b55d00c3a6fe477c8d5 GitHub-Last-Rev: c668b6e4ad826f84542c2675eb31ccfb010c45bb GitHub-Pull-Request: golang/go#41485 Reviewed-on: https://go-review.googlesource.com/c/go/+/255881 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Filippo Valsorda <filippo@golang.org> Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-25encoding/json: added docs to UnsupportedValueErrorCarlos Alexandro Becker
Added godoc to UnsupportedValueError. Change-Id: I5fc13bac0b6e14b3a6eba27c9d3331ff5c5269aa GitHub-Last-Rev: 516cd7a92903e1048caa4d560abf5d66339e5a8f GitHub-Pull-Request: golang/go#41364 Reviewed-on: https://go-review.googlesource.com/c/go/+/254540 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Trust: Daniel Martí <mvdan@mvdan.cc> Trust: Heschi Kreinick <heschi@google.com> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Go Bot <gobot@golang.org>
2020-09-24encoding/json: detect cyclic maps and sliceslujjjh
Now reports an error if cyclic maps and slices are to be encoded instead of an infinite recursion. This case wasn't handled in CL 187920. Fixes #40745. Change-Id: Ia34b014ecbb71fd2663bb065ba5355a307dbcc15 GitHub-Last-Rev: 6f874944f4065b5237babbb0fdce14c1c74a3c97 GitHub-Pull-Request: golang/go#40756 Reviewed-on: https://go-review.googlesource.com/c/go/+/248358 Reviewed-by: Daniel Martí <mvdan@mvdan.cc> Trust: Daniel Martí <mvdan@mvdan.cc> Trust: Bryan C. Mills <bcmills@google.com> Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Go Bot <gobot@golang.org>
2020-09-24encoding/json: allow semicolon in field key / struct tagSean Liao
Allow ';' as a valid character for json field keys and struct tags. Fixes #39189 Change-Id: I4b602a1b0674ff028db07623682f0d1e8e9fd6c9 Reviewed-on: https://go-review.googlesource.com/c/go/+/234818 Run-TryBot: Daniel Martí <mvdan@mvdan.cc> TryBot-Result: Go Bot <gobot@golang.org> Trust: Giovanni Bajo <rasky@develer.com> Trust: Daniel Martí <mvdan@mvdan.cc> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2020-09-23encoding/binary: remove TODO in Write and add benchmarkszhouzhongyuan
Benchmarks: goos: linux goarch: amd64 BenchmarkReadSlice1000Uint8s-8 4097088 296 ns/op 3381.06 MB/s BenchmarkWriteSlice1000Uint8s-8 4372588 271 ns/op 3694.96 MB/s Change-Id: I5b6ef0da5052e3381ee9c714bbff541c11ed0259 Reviewed-on: https://go-review.googlesource.com/c/go/+/246837 Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-09-14Revert "encoding/json: implement Is on SyntaxError"Damien Neil
This reverts CL 253037. Reason for revert: The recommended way to check for a type of error is errors.As. API changes should also start with a proposal. Change-Id: I62896717aa47ed491c2c4775d2b05d80e5e9cde3 Reviewed-on: https://go-review.googlesource.com/c/go/+/254837 Trust: Damien Neil <dneil@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-14Revert "encoding/json: implement Is on all errors"Damien Neil
This reverts CL 254537. Reason for revert: Reason for revert: The recommended way to check for a type of error is errors.As. API changes should also start with a proposal. Change-Id: I07c37428575e99c80b17525833a61831d10963bb Reviewed-on: https://go-review.googlesource.com/c/go/+/254857 Trust: Damien Neil <dneil@google.com> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-13encoding/json: implement Is on all errorsCarlos Alexandro Becker
Allows users to check: errors.Is(err, &UnmarshalTypeError{}) errors.Is(err, &UnmarshalFieldError{}) errors.Is(err, &InvalidUnmarshalError{}) errors.Is(err, &UnsupportedValueError{}) errors.Is(err, &MarshalerError{}) which is the recommended way of checking for kinds of errors. SyntaxError.Is was implemented in CL 253037. As and Unwrap relevant methods will be added in future CLs. Change-Id: I1f8a503b8fdc0f3afdfe9669a91f3af8d960e028 GitHub-Last-Rev: 930cda5384c987a0b31f277ba3b4ab690ea74ac3 GitHub-Pull-Request: golang/go#41360 Reviewed-on: https://go-review.googlesource.com/c/go/+/254537 Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-12encoding/json: implement Is on SyntaxErrorCarlos Alexandro Becker
Allows users to check: errors.Is(err, &json.SyntaxError{}) which is the recommended way of checking for kinds of errors. Change-Id: I20dc805f20212765e9936a82d9cb7822e73ec4ef GitHub-Last-Rev: e2627ccf8e2a00cc3459bb9fee86c3c8675a33af GitHub-Pull-Request: golang/go#41210 Reviewed-on: https://go-review.googlesource.com/c/go/+/253037 Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-08-25encoding/binary: replace constant literals with named constant (cleanup)Robert Griesemer
Follow-up on https://golang.org/cl/247120. Brought to my attention by Luke McCoy. Change-Id: I4530c96fb164d23b0ce5311f2cecb1964f2dea74 Reviewed-on: https://go-review.googlesource.com/c/go/+/249837 Reviewed-by: Katie Hockman <katie@golang.org>
2020-08-18encoding/asn1: speed up marshal by reducing allocationsMartin Möhrmann
Replace strings.Split by strings.IndexByte and explicit slicing to avoid the allocation of the return slice of strings.Split. name old time/op new time/op delta Marshal 43.3µs ± 1% 36.7µs ± 1% -15.23% (p=0.000 n=9+9) name old alloc/op new alloc/op delta Marshal 10.7kB ± 0% 9.2kB ± 0% -13.96% (p=0.000 n=10+10) name old allocs/op new allocs/op delta Marshal 444 ± 0% 366 ± 0% -17.57% (p=0.000 n=10+10) Change-Id: I9e727defa23f7e5fc684f246de0136fe28cf8d25 Reviewed-on: https://go-review.googlesource.com/c/go/+/231738 TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-08-06encoding/binary: read at most MaxVarintLen64 bytes in ReadUvarintKatie Hockman
This CL ensures that ReadUvarint consumes only a limited amount of input (instead of an unbounded amount). On some inputs, ReadUvarint could read an arbitrary number of bytes before deciding to return an overflow error. After this CL, ReadUvarint returns that same overflow error sooner, after reading at most MaxVarintLen64 bytes. Fix authored by Robert Griesemer and Filippo Valsorda. Thanks to Diederik Loerakker, Jonny Rhea, Raúl Kripalani, and Preston Van Loon for reporting this. Fixes #40618 Fixes CVE-2020-16845 Change-Id: Ie0cb15972f14c38b7cf7af84c45c4ce54909bb8f Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/812099 Reviewed-by: Filippo Valsorda <valsorda@google.com> Reviewed-on: https://go-review.googlesource.com/c/go/+/247120 Run-TryBot: Katie Hockman <katie@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
2020-07-02Revert "encoding/json: don't reuse slice elements when decoding"Daniel Martí
This reverts https://golang.org/cl/191783. Reason for revert: Broke too many programs which depended on the previous behavior, even when it was the opposite of what the documentation said. We can attempt to fix the original issue again for 1.16, while keeping those programs in mind. Fixes #39427. Change-Id: I7a7f24b2a594c597ef625aeff04fff29aaa88fc6 Reviewed-on: https://go-review.googlesource.com/c/go/+/240657 Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-06-29Revert "encoding/xml: fix reserved namespace check to be case-insensitive"Ian Lance Taylor
This reverts CL 203417. Reason for revert: This change changes uses of tags like "XMLSchema-instance" without any recourse. For #35151 Fixes #39876 Change-Id: I4c85c8267a46b3748664b5078794dafffb42aa26 Reviewed-on: https://go-review.googlesource.com/c/go/+/240179 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com> Reviewed-by: Andrew Bonventre <andybons@golang.org>
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion