| Age | Commit message (Collapse) | Author |
|---|
|
Limit nested parsing to 100,000, which prevents stack exhaustion when
parsing deeply nested statements, types, and expressions. Also limit
the scope depth to 1,000 during object resolution.
Thanks to Juho Nurminen of Mattermost for reporting this issue.
Fixes #53707
Updates #53616
Fixes CVE-2022-1962
Change-Id: I4d7b86c1d75d0bf3c7af1fdea91582aa74272c64
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1491025
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
(cherry picked from commit 6a856f08d58e4b6705c0c337d461c540c1235c83)
Reviewed-on: https://go-review.googlesource.com/c/go/+/417070
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
|
|
Parser object resolution is an auxiliary feature in which the parser
attempts to resolve identifiers to their declarations. In functionality,
it significantly overlaps with go/types and in fact cannot be correctly
computed at parse-time without type information (for example, it is
generally not possible to resolve k in the composite lit c{k: v}). Due
to these limitations, it is of limited utility and rarely used.
Now that object resolution is isolated as a post-processing pass, it is
trivial to offer a parser mode that skips it entirely. This CL adds that
mode.
Fixes #45104
Change-Id: I5a2c05437e298964ad2039e1ff98e63d6efbd1af
Reviewed-on: https://go-review.googlesource.com/c/go/+/306149
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
|
|
This CL changes our approach to guarding type parameter functionality
and API. Previously, we guarded type parameter functionality with the
parser.parseTypeParams parser mode, and were in the process of hiding
the type parameter API behind the go1.18 build constraint.
These mechanisms had several limitations:
+ Requiring the parser.parseTypeParams mode to be set meant that
existing tooling would have to opt-in to type parameters in all
places where it parses Go files.
+ The parseTypeParams mode value had to be copied in several places.
+ go1.18 is not specific to typeparams, making it difficult to set up
the builders to run typeparams tests.
This CL addresses the above limitations, and completes the task of
hiding the AST API, by switching to a new 'typeparams' build constraint
and adding a new go/internal/typeparams helper package.
The typeparams build constraint is used to conditionally compile the new
AST changes. The typeparams package provides utilities for accessing and
writing the new AST data, so that we don't have to fragment our parser
or type checker logic across build constraints. The typeparams.Enabled
const is used to guard tests that require type parameter support.
The parseTypeParams parser mode is gone, replaced by a new
typeparams.DisableParsing mode with the opposite sense. Now, type
parameters are only parsed if go/parser is compiled with the typeparams
build constraint set AND typeparams.DisableParsing not set. This new
parser mode allows opting out of type parameter parsing for tests.
How exactly to run tests on builders is left to a follow-up CL.
Updates #44933
Change-Id: I3091e42a2e5e2f23e8b2ae584f415a784b9fbd65
Reviewed-on: https://go-review.googlesource.com/c/go/+/300649
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
|
|
Coupling object resolution to parsing complicates the parsing code, and
is a barrier to improvement. It requires passing around context such as
'lhs' or 'keyOk', and even then sometimes requires guess-work, such as
whether to resolve the key in a composite literal.
In this CL we delay object resolution to a separate pass after the file
parse completes. This makes it easier to see logic of scoping, and
removes state from the parsing code. This can enable subsequent
improvements such as optionally skipping object resolution, aligning the
parser with cmd/compile/internal/syntax, and allowing alternative
parsers to reuse object resolution.
The additional AST traversal appears to slow down parsing by around 4%.
That seems small enough not to worry about, especially since performance
sensitive users may eventually be able to disable object resolution
entirely, saving around 18% off the previous baseline. I'll also mail a
speculative CL showing how we can significantly mitigate the cost of
object resolution by transposing scopes.
For #45104
Change-Id: I98d9143fd77ae29e84ec7c3ae2fdb1139510da37
Reviewed-on: https://go-review.googlesource.com/c/go/+/304455
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
|
|
While the dev.typeparams branch was merged, the type parameter API is
slated for go1.18. Hide these changes to the go/parser and go/types API.
This was done as follows:
+ For APIs that will probably not be needed for go1.18, simply unexport
them.
+ For APIs that we expect to export in go1.18, prefix symbols with '_',
so that the planned (upper-cased) symbol name is apparent.
+ For APIs that must be exported for testing, move both API and tests
to files guarded by the go1.18 build constraint.
+ parser.ParseTypeParams is unexported and copied wherever it is
needed.
+ The -G flag is removed from gofmt, replaced by enabling type
parameters if built with the go1.18 build constraint.
Notably, changes related to type parameters in go/ast are currently left
exported. We're looking at the AST API separately.
The new API diff from 1.16 is:
+pkg go/ast, method (*FuncDecl) IsMethod() bool
+pkg go/ast, method (*ListExpr) End() token.Pos
+pkg go/ast, method (*ListExpr) Pos() token.Pos
+pkg go/ast, type FuncType struct, TParams *FieldList
+pkg go/ast, type ListExpr struct
+pkg go/ast, type ListExpr struct, ElemList []Expr
+pkg go/ast, type TypeSpec struct, TParams *FieldList
+pkg go/types, type Config struct, GoVersion string
Change-Id: I1baf67e26279b49092e774309a836c460979774a
Reviewed-on: https://go-review.googlesource.com/c/go/+/295929
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
|
|
The following files had merge conflicts and were merged manually:
src/cmd/compile/fmtmap_test.go
src/cmd/compile/internal/gc/noder.go
src/go/parser/error_test.go
test/assign.go
test/chan/perm.go
test/fixedbugs/issue22822.go
test/fixedbugs/issue4458.go
test/init.go
test/interface/explicit.go
test/map1.go
test/method2.go
The following files had manual changes to make tests pass:
test/run.go
test/used.go
src/cmd/compile/internal/types2/stdlib_test.go
Change-Id: Ia495aaaa80ce321ee4ec2a9105780fbe913dbd4c
|
|
os.ReadDir is a replacement for ioutil.ReadDir that returns
a slice of fs.DirEntry instead of fs.FileInfo, meaning it is the
more efficient form.
This CL updates call sites throughout the Go source tree
wherever possible. As usual, code built using the Go 1.4
bootstrap toolchain is not included. There is also a use in
go/build that appears in the public API and can't be changed,
at least not without additional changes.
Fixes #42026.
Change-Id: Icfc9dd52c6045020f6830e22c72128499462d561
Reviewed-on: https://go-review.googlesource.com/c/go/+/266366
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
As part of #42026, these helpers from io/ioutil were moved to os.
(ioutil.TempFile and TempDir became os.CreateTemp and MkdirTemp.)
Update the Go tree to use the preferred names.
As usual, code compiled with the Go 1.4 bootstrap toolchain
and code vendored from other sources is excluded.
ReadDir changes are in a separate CL, because they are not a
simple search and replace.
For #42026.
Change-Id: If318df0216d57e95ea0c4093b89f65e5b0ababb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266365
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
Issue and PR on GoReleaser:
- https://github.com/goreleaser/goreleaser/issues/1897
- https://github.com/goreleaser/goreleaser/pull/1899
Fixes #42951.
Change-Id: Ia0d6018e0bad59cd60cd600188c368c431032a4b
GitHub-Last-Rev: be59d85fe2d473f4dfd828a244023c4064d6e31f
GitHub-Pull-Request: golang/go#42581
Reviewed-on: https://go-review.googlesource.com/c/go/+/269897
Trust: Robert Griesemer <gri@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
|
|
This CL imports changes on the go2go branch to support parsing type
params, as well as the unsubmitted changes from CL 269300 to remove
support for parenthesize type parameter syntax.
Change-Id: I27ab942ce69eab62c2a1800f8f9661c4dcb233fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/270857
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
|
|
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.
For #41190.
Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
|
|
The old os references are still valid, but update our code
to reflect best practices and get used to the new locations.
Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
For #41190.
Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/243907
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
|
|
These changes fix go doc -src mode that vomits comments from random files if
filesystem does not sort files by name. The issue was with parse.ParseDir
using the Readdir order of files, which varies between platforms and filesystem
implementations. Another option is to merge comments using token.FileSet.Iterate
order in cmd/doc, but since ParseDir is mostly used in go doc, I’ve opted for
smaller change because it’s unlikely to break other uses or cause any perfomance
issues.
Example (macOS APFS): `go doc -src net.ListenPacket`
Change-Id: I7f9f368c7d9ccd9a2cbc48665f2cb9798c7b3a3f
GitHub-Last-Rev: 654fb450421266a0bb64518016944db22bd681e3
GitHub-Pull-Request: golang/go#36104
Reviewed-on: https://go-review.googlesource.com/c/go/+/210999
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
|
|
Remove redundant code and improve documentation in the process.
Fixes #34211.
Change-Id: I9a6d1467f1a2c98a163f41f9df147fc6500c6fad
Reviewed-on: https://go-review.googlesource.com/c/go/+/196077
Reviewed-by: Ian Lance Taylor <iant@golang.org>
|
|
This CL fixes a bug in ParseExprFrom which makes
error messages ignored when there are 10+ errors
in a single expression.
fixes #34241
fixes #34274
Change-Id: I29a82d3e3e726279005eb6fbcd7ee3aebffaa679
Reviewed-on: https://go-review.googlesource.com/c/go/+/194638
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
|
|
ioutil.ReadAll didn't exist when we wrote that parser code
originally (in 2009). Now it does, so use it. This may also
make that code path slightly more efficient.
Also, now that we are guaranteed to have a fast path for reading
from an io.Reader (and thus an io.ReadCloser), simplify setup
code for parser.ParseFile calls in srcimporter.Importer.ParseFiles.
Remove the associated TODO since we cannot reproduce any significant
performance differences when running go test -run ImportStdLib for
the case where we used to read directly from a file (even before the
change to the parser).
Fixes #19281.
Change-Id: I816459d092bb9e27fdc85089b8f21d57ec3fd79a
Reviewed-on: https://go-review.googlesource.com/85395
Reviewed-by: Alan Donovan <adonovan@google.com>
|
|
The ParseExprFrom docs refer to Parse. It meant ParseFile.
Fixes #18398
Change-Id: I06fb3b5178c6319e86199823fe4769a8eb9dc49c
Reviewed-on: https://go-review.googlesource.com/34671
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
+ panic with explicit error if no file set it provided
(Not providing a file set is invalid use of the API; panic
is the appropriate action rather than returning an error.)
Fixes #16018.
Change-Id: I207f5b2a2e318d65826bdd9522fce46d614c24ee
Reviewed-on: https://go-review.googlesource.com/24010
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
|
|
This is needed for code that relies on having the correct file set
when parsing an expression only. There's currently no other way to
get to the file set otherwise or to invoke the parser correctly to
work on an expression only with a given file set.
Change-Id: I325f174cb34b69284e627f59fe8334efa4eaa45c
Reviewed-on: https://go-review.googlesource.com/10998
Reviewed-by: Alan Donovan <adonovan@google.com>
|
|
Formerly it would return a BadExpr.
This prevents partial syntax from being discarded, and makes the error
recovery logic more consistent with other places where an identifier
was expected but not found.
+ test
Change-Id: I223c0c0589e7ceb7207ae951b8f71b9275a1eb73
Reviewed-on: https://go-review.googlesource.com/10269
Reviewed-by: Robert Griesemer <gri@golang.org>
|
|
Preparation was in CL 134570043.
This CL contains only the effect of 'hg mv src/pkg/* src'.
For more about the move, see golang.org/s/go14nopkg.
|
