| Age | Commit message (Collapse) | Author |
|---|
|
No change to actual runtime, but helps reduce the laundry list
of functions.
mcall, morestack, and asmcgocall are not actually top-of-frame,
so those need more attention in follow-up CLs.
mstart moved to assembly so that it can be marked TOPFRAME.
Since TOPFRAME also tells DWARF consumers not to unwind
this way, this change should also improve debuggers a
marginal amount.
This CL is part of a stack adding windows/arm64
support (#36439), intended to land in the Go 1.17 cycle.
This CL is, however, not windows/arm64-specific.
It is cleanup meant to make the port (and future ports) easier.
Change-Id: If1e0d46ca973de5e46b62948d076f675f285b5d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/288802
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
A g's sched.g is set in newproc1:
newg.sched.g = guintptr(unsafe.Pointer(newg))
After that, it never changes. Yet lots of assembly code does
"g.sched.g = g" unnecessarily. Remove all those lines to avoid
confusion about whether it ever changes.
Also, split gogo into two functions, one that does the nil g check
and a second that does the actual switch. This way, if the nil g check
fails, we get a stack trace showing the call stack that led to the failure.
(The SP write would otherwise cause the stack trace to abort.)
Also restore the proper nil g check in a handful of assembly functions.
(There is little point in checking for nil g *after* installing it as the real g.)
Change-Id: I22866b093f901f765de1d074e36eeec10366abfb
Reviewed-on: https://go-review.googlesource.com/c/go/+/292109
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
This change adds support for the new register ABI on amd64 to
reflect.(Value).Call. If internal/abi's register counts are non-zero,
reflect will try to set up arguments in registers on the Call path.
Note that because the register ABI becomes ABI0 with zero registers
available, this should keep working as it did before.
This change does not add any tests for the register ABI case because
there's no way to do so at the moment.
For #40724.
Change-Id: I8aa089a5aa5a31b72e56b3d9388dd3f82203985b
Reviewed-on: https://go-review.googlesource.com/c/go/+/272568
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
|
|
[This is a roll-forward of CL 262319, with a fix for some Darwin test
failures].
Change the definitions of selected runtime assembly routines
from ABI0 (the default) to ABIInternal. The ABIInternal def is
intended to indicate that these functions don't follow the existing Go
runtime ABI. In addition, convert the assembly reference to
runtime.main (from runtime.mainPC) to ABIInternal. Finally, for
functions such as "runtime.duffzero" that are called directly from
generated code, make sure that the compiler looks up the correct
ABI version.
This is intended to support the register abi work, however these
changes should not have any issues even when GOEXPERIMENT=regabi is
not in effect.
Updates #27539, #40724.
Change-Id: Idf507f1c06176073563845239e1a54dad51a9ea9
Reviewed-on: https://go-review.googlesource.com/c/go/+/266638
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
|
|
This reverts commit 50af50d136551e2009b2b52e829570536271cdaa.
Reason for revert: Causes failures in the runtime package test on Darwin, apparently.
Change-Id: I006bc1b3443fa7207e92fb4a93e3fb438d4d3de3
Reviewed-on: https://go-review.googlesource.com/c/go/+/266257
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
Change the definitions of selected runtime assembly routines
from ABI0 (the default) to ABIInternal. The ABIInternal def is
intended to indicate that these functions don't follow the existing Go
runtime ABI. In addition, convert the assembly reference to
runtime.main (from runtime.mainPC) to ABIInternal. Finally, for
functions such as "runtime.duffzero" that are called directly from
generated code, make sure that the compiler looks up the correct
ABI version.
This is intended to support the register abi work, however these
changes should not have any issues even when GOEXPERIMENT=regabi is
not in effect.
Updates #27539, #40724.
Change-Id: I9846f8dcaccc95718cf2e61a18b7e924a0677e4c
Reviewed-on: https://go-review.googlesource.com/c/go/+/262319
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
This redesigns the way calls work from C to exported Go functions. It
removes several steps from the call path, makes cmd/cgo no longer
sensitive to the Go calling convention, and eliminates the use of
reflectcall from cgo.
In order to avoid generating a large amount of FFI glue between the C
and Go ABIs, the cgo tool has long depended on generating a C function
that marshals the arguments into a struct, and then the actual ABI
switch happens in functions with fixed signatures that simply take a
pointer to this struct. In a way, this CL simply pushes this idea
further.
Currently, the cgo tool generates this argument struct in the exact
layout of the Go stack frame and depends on reflectcall to unpack it
into the appropriate Go call (even though it's actually
reflectcall'ing a function generated by cgo).
In this CL, we decouple this struct from the Go stack layout. Instead,
cgo generates a Go function that takes the struct, unpacks it, and
calls the exported function. Since this generated function has a
generic signature (like the rest of the call path), we don't need
reflectcall and can instead depend on the Go compiler itself to
implement the call to the exported Go function.
One complication is that syscall.NewCallback on Windows, which
converts a Go function into a C function pointer, depends on
cgocallback's current dynamic calling approach since the signatures of
the callbacks aren't known statically. For this specific case, we
continue to depend on reflectcall. Really, the current approach makes
some overly simplistic assumptions about translating the C ABI to the
Go ABI. Now we're at least in a much better position to do a proper
ABI translation.
For comparison, the current cgo call path looks like:
GoF (generated C function) ->
crosscall2 (in cgo/asm_*.s) ->
_cgoexp_GoF (generated Go function) ->
cgocallback (in asm_*.s) ->
cgocallback_gofunc (in asm_*.s) ->
cgocallbackg (in cgocall.go) ->
cgocallbackg1 (in cgocall.go) ->
reflectcall (in asm_*.s) ->
_cgoexpwrap_GoF (generated Go function) ->
p.GoF
Now the call path looks like:
GoF (generated C function) ->
crosscall2 (in cgo/asm_*.s) ->
cgocallback (in asm_*.s) ->
cgocallbackg (in cgocall.go) ->
cgocallbackg1 (in cgocall.go) ->
_cgoexp_GoF (generated Go function) ->
p.GoF
Notably:
1. We combine _cgoexp_GoF and _cgoexpwrap_GoF and move the combined
operation to the end of the sequence. This combined function also
handles reflectcall's previous role.
2. We combined cgocallback and cgocallback_gofunc since the only
purpose of having both was to convert a raw PC into a Go function
value. We instead construct the Go function value in cgocallbackg1.
3. cgocallbackg1 no longer reaches backwards through the stack to get
the arguments to cgocallback_gofunc. Instead, we just pass the
arguments down.
4. Currently, we need an explicit msanwrite to mark the results struct
as written because reflectcall doesn't do this. Now, the results are
written by regular Go assignments, so the Go compiler generates the
necessary MSAN annotations. This also means we no longer need to track
the size of the arguments frame.
Updates #40724, since now we don't need to teach cgo about the
register ABI or change how it uses reflectcall.
Change-Id: I7840489a2597962aeb670e0c1798a16a7359c94f
Reviewed-on: https://go-review.googlesource.com/c/go/+/258938
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
Currently, runtime.call16 is defined and used only on 32-bit
architectures, while 64-bit architectures all start at call32 and go
up from there. This led to unnecessary complexity because call16's
prototype needed to be in a different file, separate from all of the
other call* prototypes, which in turn led to it getting out of sync
with the other call* prototypes. This CL adds call16 on 64-bit
architectures, bringing them all into sync, and moves the call16
prototype to live with the others.
Prior to CL 31655 (in 2016), call16 couldn't be implemented on 64-bit
architectures because it needed at least four words of argument space
to invoke "callwritebarrier" after copying back the results. CL 31655
changed the way call* invoked the write barrier in preparation for the
hybrid barrier; since the hybrid barrier had to be invoked prior to
copying back results, it needed a different solution that didn't reuse
call*'s stack space. At this point, call16 was no longer a problem on
64-bit, but we never added it. Until now.
Change-Id: Id10ade0e4f75c6ea76afa6229ddaee2b994c27dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/259339
Trust: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
Currently the standard hasher is memhash, which checks whether aes
instructions are available, and if so redirects to aeshash.
With this CL, we call aeshash directly, which then redirects to the
fallback hash if aes instructions are not available.
This reduces the overhead for the hash function in the common case,
as it requires just one call instead of two. On architectures which
have no assembly hasher, it's a single jump slower.
Thanks to Martin for this idea.
name old time/op new time/op delta
BigKeyMap-4 22.6ns ± 1% 21.1ns ± 2% -6.55% (p=0.000 n=9+10)
Change-Id: Ib7ca77b63d28222eb0189bc3d7130531949d853c
Reviewed-on: https://go-review.googlesource.com/c/go/+/190998
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
|
|
Following the previous CL, this removes more global variables on
Wasm.
PC_B is used mostly for intra-function jumps, and for a function
telling its callee where to start or resume. This usage can be
served by a parameter. The top level loop (wasm_pc_f_loop) uses
PC_B for resuming a function. This value is either set by gogo,
or loaded from the Go stack at function return. Instead of
loading PC_B at each function return, we could make gogo stores
PC_B at the same stack location, and let the top level loop do
the load. This way, we don't need to use global PC_B to
communicate with the top level loop, and we can replace global
PC_B with a parameter.
PC_F is similar. It is even more so in that the only reader is
the top level loop. Let the top level loop read it from the stack,
and we can get rid of PC_F entirely.
PC_F and PC_B are used less entensively as SP, so this CL has
smaller performance gain.
Running on Chrome 74.0.3729.108 on Linux/AMD64,
name old time/op new time/op delta
BinaryTree17 16.6s ± 0% 16.2s ± 1% -2.59% (p=0.016 n=4+5)
Fannkuch11 11.1s ± 1% 10.8s ± 0% -2.65% (p=0.008 n=5+5)
FmtFprintfEmpty 231ns ± 1% 217ns ± 0% -6.06% (p=0.008 n=5+5)
FmtFprintfString 407ns ± 3% 375ns ± 2% -7.81% (p=0.008 n=5+5)
FmtFprintfInt 466ns ± 2% 430ns ± 0% -7.79% (p=0.016 n=5+4)
FmtFprintfIntInt 719ns ± 2% 673ns ± 2% -6.37% (p=0.008 n=5+5)
FmtFprintfPrefixedInt 706ns ± 1% 676ns ± 3% -4.31% (p=0.008 n=5+5)
FmtFprintfFloat 1.01µs ± 1% 0.97µs ± 1% -4.30% (p=0.008 n=5+5)
FmtManyArgs 2.67µs ± 1% 2.51µs ± 1% -5.95% (p=0.008 n=5+5)
GobDecode 30.7ms ± 9% 31.3ms ±34% ~ (p=0.222 n=5+5)
GobEncode 24.2ms ±23% 20.2ms ± 0% -16.36% (p=0.016 n=5+4)
Gzip 852ms ± 0% 823ms ± 0% -3.38% (p=0.016 n=4+5)
Gunzip 160ms ± 1% 151ms ± 1% -5.37% (p=0.008 n=5+5)
JSONEncode 35.7ms ± 1% 34.3ms ± 1% -3.81% (p=0.008 n=5+5)
JSONDecode 247ms ± 8% 254ms ± 7% ~ (p=0.548 n=5+5)
Mandelbrot200 5.39ms ± 0% 5.41ms ± 0% +0.42% (p=0.008 n=5+5)
GoParse 18.5ms ± 1% 18.3ms ± 2% ~ (p=0.343 n=4+4)
RegexpMatchEasy0_32 424ns ± 2% 397ns ± 0% -6.23% (p=0.008 n=5+5)
RegexpMatchEasy0_1K 2.88µs ± 0% 2.86µs ± 1% ~ (p=0.079 n=5+5)
RegexpMatchEasy1_32 395ns ± 2% 370ns ± 1% -6.23% (p=0.008 n=5+5)
RegexpMatchEasy1_1K 3.26µs ± 0% 3.19µs ± 1% -2.06% (p=0.008 n=5+5)
RegexpMatchMedium_32 564ns ± 1% 532ns ± 0% -5.71% (p=0.008 n=5+5)
RegexpMatchMedium_1K 146µs ± 2% 140µs ± 1% -4.62% (p=0.008 n=5+5)
RegexpMatchHard_32 8.47µs ± 1% 7.91µs ± 1% -6.65% (p=0.008 n=5+5)
RegexpMatchHard_1K 253µs ± 1% 236µs ± 2% -6.66% (p=0.008 n=5+5)
Revcomp 1.78s ± 4% 1.76s ± 5% ~ (p=1.000 n=5+5)
Template 292ms ±29% 269ms ± 5% ~ (p=0.690 n=5+5)
TimeParse 1.61µs ± 4% 1.54µs ± 1% -4.42% (p=0.008 n=5+5)
TimeFormat 1.66µs ± 3% 1.58µs ± 1% -5.22% (p=0.008 n=5+5)
[Geo mean] 232µs 221µs -4.54%
name old speed new speed delta
GobDecode 25.0MB/s ± 8% 25.1MB/s ±27% ~ (p=0.222 n=5+5)
GobEncode 32.8MB/s ±21% 38.0MB/s ± 0% +15.84% (p=0.016 n=5+4)
Gzip 22.8MB/s ± 0% 23.6MB/s ± 0% +3.49% (p=0.016 n=4+5)
Gunzip 121MB/s ± 1% 128MB/s ± 1% +5.68% (p=0.008 n=5+5)
JSONEncode 54.4MB/s ± 1% 56.5MB/s ± 1% +3.97% (p=0.008 n=5+5)
JSONDecode 7.88MB/s ± 8% 7.65MB/s ± 8% ~ (p=0.548 n=5+5)
GoParse 3.07MB/s ± 8% 3.00MB/s ±22% ~ (p=0.579 n=5+5)
RegexpMatchEasy0_32 75.6MB/s ± 2% 80.5MB/s ± 0% +6.58% (p=0.008 n=5+5)
RegexpMatchEasy0_1K 356MB/s ± 0% 358MB/s ± 1% ~ (p=0.095 n=5+5)
RegexpMatchEasy1_32 81.1MB/s ± 2% 86.5MB/s ± 1% +6.69% (p=0.008 n=5+5)
RegexpMatchEasy1_1K 314MB/s ± 0% 320MB/s ± 0% +2.10% (p=0.008 n=5+5)
RegexpMatchMedium_32 1.77MB/s ± 1% 1.88MB/s ± 0% +6.09% (p=0.016 n=5+4)
RegexpMatchMedium_1K 6.99MB/s ± 2% 7.33MB/s ± 1% +4.83% (p=0.008 n=5+5)
RegexpMatchHard_32 3.78MB/s ± 1% 4.04MB/s ± 1% +7.04% (p=0.008 n=5+5)
RegexpMatchHard_1K 4.04MB/s ± 1% 4.33MB/s ± 2% +7.17% (p=0.008 n=5+5)
Revcomp 143MB/s ± 4% 145MB/s ± 5% ~ (p=1.000 n=5+5)
Template 6.77MB/s ±24% 7.22MB/s ± 5% ~ (p=0.690 n=5+5)
[Geo mean] 24.4MB/s 25.4MB/s +4.18%
Change-Id: Ib80716e62992aec28b2c4a96af280c278f83aa49
Reviewed-on: https://go-review.googlesource.com/c/go/+/173980
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Richard Musiol <neelance@gmail.com>
|
|
Change-Id: Ifc8a731a2efd94fdc4fc6f26ca6e16f0c0292211
Reviewed-on: https://go-review.googlesource.com/c/go/+/176178
Reviewed-by: Austin Clements <austin@google.com>
|
|
The current wasm write barrier implementation incorrectly implements
the "deletion" part of the barrier. It correctly greys the new value
of the pointer, but rather than also greying the old value of the
pointer, it greys the object containing the slot (which, since the old
value was just overwritten, is not going to contain the old value).
This can lead to unmarked, reachable objects.
Often, this is masked by other marking activity, but one specific
sequence that can lead to an unmarked object because of this bug is:
1. Initially, GC is off, object A is reachable from just one pointer
in the heap.
2. GC starts and scans the stack of goroutine G.
3. G copies the pointer to A on to its stack and overwrites the
pointer to A in the heap. (Now A is reachable only from G's stack.)
4. GC finishes while A is still reachable from G's stack.
With a functioning deletion barrier, step 3 causes A to be greyed.
Without a functioning deletion barrier, nothing causes A to be greyed,
so A will be freed even though it's still reachable from G's stack.
This CL fixes the wasm write barrier.
Fixes #30871.
Change-Id: I8a74ee517facd3aa9ad606e5424bcf8f0d78e754
Reviewed-on: https://go-review.googlesource.com/c/go/+/167743
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
The names of some instructions have been updated in the WebAssembly
specification to be more consistent, see
https://github.com/WebAssembly/spec/commit/994591e51c9df9e7ef980b04d660709b79982f75.
This change to the spec is possible because it is still in a draft
state.
Go's support for WebAssembly is still experimental and thus excempt from
the compatibility promise. Being consistent with the spec should
warrant this breaking change to the assembly instruction names.
Change-Id: Iafb8b18ee7f55dd0e23c6c7824aa1fad43117ef1
Reviewed-on: https://go-review.googlesource.com/c/153797
Run-TryBot: Richard Musiol <neelance@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
|
|
Currently, package runtime contains the definition of reflect.call,
even though it's just a jump to runtime.reflectcall. This "push"
symbol is confusing, since it's not clear where the definition of
reflect.call comes from when you're in the reflect package.
Replace this with a "pull" symbol: the runtime now defines only
runtime.reflectcall and package reflect uses a go:linkname to access
this symbol directly. This makes it clear where reflect.call is coming
from without any spooky action at a distance and eliminates all of the
definitions of reflect.call in the runtime.
Change-Id: I3ec73cd394efe9df8d3061a57c73aece2e7048dd
Reviewed-on: https://go-review.googlesource.com/c/148657
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
|
|
This commit enables vet/all for the js/wasm architecture. It got
skipped initially because the codebase did not fully compile yet
for js/wasm, which made vet/all fail.
startTimer and stopTimer are not needed in the syscall package.
Removed their assembly code since their Go stubs were already gone.
Change-Id: Icaeb6d903876e51ceb1edff7631f715a98c28696
Reviewed-on: https://go-review.googlesource.com/118657
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
|
|
This commit adds the js/wasm architecture to the runtime package.
Currently WebAssembly has no support for threads yet, see
https://github.com/WebAssembly/design/issues/1073. Because of that,
there is no preemption of goroutines and no sysmon goroutine.
Design doc: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4
About WebAssembly assembly files: https://docs.google.com/document/d/1GRmy3rA4DiYtBlX-I1Jr_iHykbX8EixC3Mq0TCYqbKc
Updates #18892
Change-Id: I7f12d21b5180500d55ae9fd2f7e926a1731db391
Reviewed-on: https://go-review.googlesource.com/103877
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
|
