aboutsummaryrefslogtreecommitdiff
path: root/src/runtime/os_linux_arm64.go
AgeCommit message (Collapse)Author
2021-02-20all: go fmt std cmd (but revert vendor)Russ Cox
Make all our package sources use Go 1.17 gofmt format (adding //go:build lines). Part of //go:build change (#41184). See https://golang.org/design/draft-gobuild Change-Id: Ia0534360e4957e58cd9a18429c39d0e32a6addb4 Reviewed-on: https://go-review.googlesource.com/c/go/+/294430 Trust: Russ Cox <rsc@golang.org> Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com> Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-20internal/cpu: consolidate arm64 feature detectionMartin Möhrmann
Move code to detect and mask arm64 CPU features from runtime to internal/cpu. Change-Id: Ib784e2ff056e8def125d68827b852f07a3eff0db Reviewed-on: https://go-review.googlesource.com/c/go/+/261878 Trust: Martin Möhrmann <moehrmann@google.com> Trust: Tobias Klauser <tobias.klauser@gmail.com> Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com> Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2019-12-05runtime: mlock top of signal stack on Linux 5.2–5.4.1Austin Clements
Linux 5.2 introduced a bug that can corrupt vector registers on return from a signal if the signal stack isn't faulted in: https://bugzilla.kernel.org/show_bug.cgi?id=205663 This CL works around this by mlocking the top page of all Go signal stacks on the affected kernels. Fixes #35326, #35777 Change-Id: I77c80a2baa4780827633f92f464486caa222295d Reviewed-on: https://go-review.googlesource.com/c/go/+/209899 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Cherry Zhang <cherryyz@google.com> Reviewed-by: Ian Lance Taylor <iant@golang.org> Reviewed-by: David Chase <drchase@google.com>
2019-11-12runtime: consistently seed fastrand state across archsCarlo Alberto Ferraris
Some, but not all, architectures mix in OS-provided random seeds when initializing the fastrand state. The others have TODOs saying we need to do the same. Lift that logic up in the architecture-independent part, and use memhash to mix the seed instead of a simple addition. Previously, dumping the fastrand state at initialization would yield something like the following on linux-amd64, where the values in the first column do not change between runs (as thread IDs are sequential and always start at 0), and the values in the second column, while changing every run, are pretty correlated: first run: 0x0 0x44d82f1c 0x5f356495 0x44f339de 0xbe6ac92a 0x44f91cd8 0x1da02dbf 0x44fd91bc 0x7cd59254 0x44fee8a4 0xdc0af6e9 0x4547a1e0 0x3b405b7e 0x474c76fc 0x9a75c013 0x475309dc 0xf9ab24a8 0x4bffd075 second run: 0x0 0xa63fc3eb 0x5f356495 0xa6648dc2 0xbe6ac92a 0xa66c1c59 0x1da02dbf 0xa671bce8 0x7cd59254 0xa70e8287 0xdc0af6e9 0xa7129d2e 0x3b405b7e 0xa7379e2d 0x9a75c013 0xa7e4c64c 0xf9ab24a8 0xa7ecce07 With this change, we get initial states that appear to be much more unpredictable, both within the same run as well as between runs: 0x11bddad7 0x97241c63 0x553dacc6 0x2bcd8523 0x62c01085 0x16413d92 0x6f40e9e6 0x7a138de6 0xa4898053 0x70d816f0 0x5ca5b433 0x188a395b 0x62778ca9 0xd462c3b5 0xd6e160e4 0xac9b4bd 0xb9571d65 0x597a981d Change-Id: Ib22c530157d74200df0083f830e0408fd4aaea58 Reviewed-on: https://go-review.googlesource.com/c/go/+/203439 Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-11-04runtime: avoid arm64 8.1 atomics on AndroidElias Naur
The kernel on some Samsung S9+ models reports support for arm64 8.1 atomics, but in reality only some of the cores support them. Go programs scheduled to cores without support will crash with SIGILL. This change unconditionally disables the optimization on Android. A better fix is to precisely detect the offending chipset. Fixes #28431 Change-Id: I35a1273e5660603824d30ebef2ce7e429241bf1f Reviewed-on: https://go-review.googlesource.com/c/147377 Run-TryBot: Elias Naur <elias.naur@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-08-21runtime: don't use linkname to refer to internal/cpuIan Lance Taylor
The runtime package already imports the internal/cpu package, so there is no reason for it to use go:linkname comments to refer to internal/cpu functions and variables. Since internal/cpu is internal, we can just export those names. Removing the obscurity of go:linkname outweighs the minor additional complexity added to the internal/cpu API. Change-Id: Id89951b7f3fc67cd9bce67ac6d01d44a647a10ad Reviewed-on: https://go-review.googlesource.com/128755 Run-TryBot: Ian Lance Taylor <iant@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Martin Möhrmann <moehrmann@google.com>
2018-05-01internal/cpu: remove platform specific prefix from cpu hwcap variablesMartin Möhrmann
Go runtime currently only populates hwcap for ppc64 and arm64. While the interpretation of hwcap is platform specific the hwcap information is generally available on linux. Changing the runtime variable name to cpu_hwcap for cpu.hwcap makes it consistent with the general naming of runtime variables that are linked to other packages. Change-Id: I1e1f932a73ed624a219b9298faafbb6355e47ada Reviewed-on: https://go-review.googlesource.com/94757 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-04-10runtime: use internal/cpu in alginitMeng Zhuo
After CL 104636 the feature flags in internal/cpu are initialized before alginit and can now be used for aeshash feature detection. Also remove now unused runtime variables: x86: support_ssse3 support_sse42 support_aes arm64: support_aes Change-Id: I2f64198d91750eaf3c6cf2aac6e9e17615811ec8 Reviewed-on: https://go-review.googlesource.com/106015 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-04-04runtime: implement aeshash for arm64 platformMeng Zhuo
Fix #10109 name old time/op new time/op delta Hash5 72.3ns ± 0% 51.5ns ± 0% -28.71% (p=0.000 n=4+5) Hash16 78.8ns ± 0% 48.7ns ± 0% ~ (p=0.079 n=4+5) Hash64 196ns ±25% 73ns ±16% -62.68% (p=0.008 n=5+5) Hash1024 1.57µs ± 0% 0.27µs ± 1% -82.90% (p=0.000 n=5+4) Hash65536 96.5µs ± 0% 14.3µs ± 0% -85.14% (p=0.016 n=5+4) HashStringSpeed 156ns ± 6% 129ns ± 3% -17.56% (p=0.008 n=5+5) HashBytesSpeed 227ns ± 1% 200ns ± 1% -11.98% (p=0.008 n=5+5) HashInt32Speed 116ns ± 2% 102ns ± 0% -11.92% (p=0.016 n=5+4) HashInt64Speed 120ns ± 3% 101ns ± 2% -15.55% (p=0.008 n=5+5) HashStringArraySpeed 342ns ± 0% 306ns ± 2% -10.58% (p=0.008 n=5+5) FastrandHashiter 217ns ± 1% 217ns ± 1% ~ (p=1.000 n=5+5) name old speed new speed delta Hash5 69.1MB/s ± 0% 97.0MB/s ± 0% +40.32% (p=0.008 n=5+5) Hash16 203MB/s ± 0% 329MB/s ± 0% +61.76% (p=0.016 n=4+5) Hash64 332MB/s ±21% 881MB/s ±14% +165.66% (p=0.008 n=5+5) Hash1024 651MB/s ± 0% 3652MB/s ±17% +460.73% (p=0.008 n=5+5) Hash65536 679MB/s ± 0% 4570MB/s ± 0% +572.85% (p=0.016 n=5+4) Change-Id: I573028979f84cf2e0e087951271d5de8865dbf04 Reviewed-on: https://go-review.googlesource.com/89755 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-11-14internal/cpu: detect cpu features in internal/cpu packageFangming.Fang
change hash/crc32 package to use cpu package instead of using runtime internal variables to check crc32 instruction Change-Id: I8f88d2351bde8ed4e256f9adf822a08b9a00f532 Reviewed-on: https://go-review.googlesource.com/76490 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
2017-04-24runtime: gofmt -w -sMikio Hara
Change-Id: I954b0300554786b7026996a21acfec3b6f205e75 Reviewed-on: https://go-review.googlesource.com/41512 Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com> Reviewed-by: Marvin Stenger <marvin.stenger94@gmail.com> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-04-13hash/crc32: optimize arm64 crc32 implementationWei Xiao
ARMv8 defines crc32 instruction. Comparing to the original crc32 calculation, this patch makes use of crc32 instructions to do crc32 calculation instead of the multiple lookup table algorithms. ARMv8 provides IEEE and Castagnoli polynomials for crc32 calculation so that the perfomance of these two types of crc32 get significant improved. name old time/op new time/op delta CRC32/poly=IEEE/size=15/align=0-32 117ns ± 0% 38ns ± 0% -67.44% CRC32/poly=IEEE/size=15/align=1-32 117ns ± 0% 38ns ± 0% -67.52% CRC32/poly=IEEE/size=40/align=0-32 129ns ± 0% 41ns ± 0% -68.37% CRC32/poly=IEEE/size=40/align=1-32 129ns ± 0% 41ns ± 0% -68.29% CRC32/poly=IEEE/size=512/align=0-32 828ns ± 0% 246ns ± 0% -70.29% CRC32/poly=IEEE/size=512/align=1-32 828ns ± 0% 132ns ± 0% -84.06% CRC32/poly=IEEE/size=1kB/align=0-32 1.58µs ± 0% 0.46µs ± 0% -70.98% CRC32/poly=IEEE/size=1kB/align=1-32 1.58µs ± 0% 0.46µs ± 0% -70.92% CRC32/poly=IEEE/size=4kB/align=0-32 6.06µs ± 0% 1.74µs ± 0% -71.27% CRC32/poly=IEEE/size=4kB/align=1-32 6.10µs ± 0% 1.74µs ± 0% -71.44% CRC32/poly=IEEE/size=32kB/align=0-32 48.3µs ± 0% 13.7µs ± 0% -71.61% CRC32/poly=IEEE/size=32kB/align=1-32 48.3µs ± 0% 13.7µs ± 0% -71.60% CRC32/poly=Castagnoli/size=15/align=0-32 116ns ± 0% 38ns ± 0% -67.07% CRC32/poly=Castagnoli/size=15/align=1-32 116ns ± 0% 38ns ± 0% -66.90% CRC32/poly=Castagnoli/size=40/align=0-32 127ns ± 0% 40ns ± 0% -68.11% CRC32/poly=Castagnoli/size=40/align=1-32 127ns ± 0% 40ns ± 0% -68.11% CRC32/poly=Castagnoli/size=512/align=0-32 828ns ± 0% 132ns ± 0% -84.06% CRC32/poly=Castagnoli/size=512/align=1-32 827ns ± 0% 132ns ± 0% -84.04% CRC32/poly=Castagnoli/size=1kB/align=0-32 1.59µs ± 0% 0.22µs ± 0% -85.89% CRC32/poly=Castagnoli/size=1kB/align=1-32 1.58µs ± 0% 0.22µs ± 0% -85.79% CRC32/poly=Castagnoli/size=4kB/align=0-32 6.14µs ± 0% 0.77µs ± 0% -87.40% CRC32/poly=Castagnoli/size=4kB/align=1-32 6.06µs ± 0% 0.77µs ± 0% -87.25% CRC32/poly=Castagnoli/size=32kB/align=0-32 48.3µs ± 0% 5.9µs ± 0% -87.71% CRC32/poly=Castagnoli/size=32kB/align=1-32 48.4µs ± 0% 6.0µs ± 0% -87.69% CRC32/poly=Koopman/size=15/align=0-32 104ns ± 0% 104ns ± 0% +0.00% CRC32/poly=Koopman/size=15/align=1-32 104ns ± 0% 104ns ± 0% +0.00% CRC32/poly=Koopman/size=40/align=0-32 235ns ± 0% 235ns ± 0% +0.00% CRC32/poly=Koopman/size=40/align=1-32 235ns ± 0% 235ns ± 0% +0.00% CRC32/poly=Koopman/size=512/align=0-32 2.71µs ± 0% 2.71µs ± 0% -0.07% CRC32/poly=Koopman/size=512/align=1-32 2.71µs ± 0% 2.71µs ± 0% -0.04% CRC32/poly=Koopman/size=1kB/align=0-32 5.40µs ± 0% 5.39µs ± 0% -0.06% CRC32/poly=Koopman/size=1kB/align=1-32 5.40µs ± 0% 5.40µs ± 0% +0.02% CRC32/poly=Koopman/size=4kB/align=0-32 21.5µs ± 0% 21.5µs ± 0% -0.16% CRC32/poly=Koopman/size=4kB/align=1-32 21.5µs ± 0% 21.5µs ± 0% -0.05% CRC32/poly=Koopman/size=32kB/align=0-32 172µs ± 0% 172µs ± 0% -0.07% CRC32/poly=Koopman/size=32kB/align=1-32 172µs ± 0% 172µs ± 0% -0.01% name old speed new speed delta CRC32/poly=IEEE/size=15/align=0-32 128MB/s ± 0% 394MB/s ± 0% +207.95% CRC32/poly=IEEE/size=15/align=1-32 128MB/s ± 0% 394MB/s ± 0% +208.09% CRC32/poly=IEEE/size=40/align=0-32 310MB/s ± 0% 979MB/s ± 0% +216.07% CRC32/poly=IEEE/size=40/align=1-32 310MB/s ± 0% 979MB/s ± 0% +216.16% CRC32/poly=IEEE/size=512/align=0-32 618MB/s ± 0% 2074MB/s ± 0% +235.72% CRC32/poly=IEEE/size=512/align=1-32 618MB/s ± 0% 3852MB/s ± 0% +523.55% CRC32/poly=IEEE/size=1kB/align=0-32 646MB/s ± 0% 2225MB/s ± 0% +244.57% CRC32/poly=IEEE/size=1kB/align=1-32 647MB/s ± 0% 2225MB/s ± 0% +243.87% CRC32/poly=IEEE/size=4kB/align=0-32 676MB/s ± 0% 2352MB/s ± 0% +248.02% CRC32/poly=IEEE/size=4kB/align=1-32 672MB/s ± 0% 2352MB/s ± 0% +250.15% CRC32/poly=IEEE/size=32kB/align=0-32 678MB/s ± 0% 2387MB/s ± 0% +252.17% CRC32/poly=IEEE/size=32kB/align=1-32 678MB/s ± 0% 2388MB/s ± 0% +252.11% CRC32/poly=Castagnoli/size=15/align=0-32 129MB/s ± 0% 393MB/s ± 0% +205.51% CRC32/poly=Castagnoli/size=15/align=1-32 129MB/s ± 0% 390MB/s ± 0% +203.41% CRC32/poly=Castagnoli/size=40/align=0-32 314MB/s ± 0% 988MB/s ± 0% +215.04% CRC32/poly=Castagnoli/size=40/align=1-32 314MB/s ± 0% 987MB/s ± 0% +214.68% CRC32/poly=Castagnoli/size=512/align=0-32 618MB/s ± 0% 3860MB/s ± 0% +524.32% CRC32/poly=Castagnoli/size=512/align=1-32 619MB/s ± 0% 3859MB/s ± 0% +523.66% CRC32/poly=Castagnoli/size=1kB/align=0-32 645MB/s ± 0% 4568MB/s ± 0% +608.56% CRC32/poly=Castagnoli/size=1kB/align=1-32 650MB/s ± 0% 4567MB/s ± 0% +602.94% CRC32/poly=Castagnoli/size=4kB/align=0-32 667MB/s ± 0% 5297MB/s ± 0% +693.81% CRC32/poly=Castagnoli/size=4kB/align=1-32 676MB/s ± 0% 5297MB/s ± 0% +684.00% CRC32/poly=Castagnoli/size=32kB/align=0-32 678MB/s ± 0% 5519MB/s ± 0% +713.83% CRC32/poly=Castagnoli/size=32kB/align=1-32 677MB/s ± 0% 5497MB/s ± 0% +712.04% CRC32/poly=Koopman/size=15/align=0-32 143MB/s ± 0% 144MB/s ± 0% +0.27% CRC32/poly=Koopman/size=15/align=1-32 143MB/s ± 0% 144MB/s ± 0% +0.33% CRC32/poly=Koopman/size=40/align=0-32 169MB/s ± 0% 170MB/s ± 0% +0.12% CRC32/poly=Koopman/size=40/align=1-32 170MB/s ± 0% 170MB/s ± 0% +0.08% CRC32/poly=Koopman/size=512/align=0-32 189MB/s ± 0% 189MB/s ± 0% +0.07% CRC32/poly=Koopman/size=512/align=1-32 189MB/s ± 0% 189MB/s ± 0% +0.04% CRC32/poly=Koopman/size=1kB/align=0-32 190MB/s ± 0% 190MB/s ± 0% +0.05% CRC32/poly=Koopman/size=1kB/align=1-32 190MB/s ± 0% 190MB/s ± 0% -0.01% CRC32/poly=Koopman/size=4kB/align=0-32 190MB/s ± 0% 190MB/s ± 0% +0.15% CRC32/poly=Koopman/size=4kB/align=1-32 190MB/s ± 0% 191MB/s ± 0% +0.05% CRC32/poly=Koopman/size=32kB/align=0-32 191MB/s ± 0% 191MB/s ± 0% +0.06% CRC32/poly=Koopman/size=32kB/align=1-32 191MB/s ± 0% 191MB/s ± 0% +0.02% Also fix a bug of arm64 assembler The optimization is mainly contributed by Fangming.Fang <fangming.fang@arm.com> Change-Id: I900678c2e445d7e8ad9e2a9ab3305d649230905f Reviewed-on: https://go-review.googlesource.com/40074 Reviewed-by: Cherry Zhang <cherryyz@google.com> Run-TryBot: Cherry Zhang <cherryyz@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2016-08-30runtime: rename fastrand1 to fastrandJosh Bleecher Snyder
Change-Id: I37706ff0a3486827c5b072c95ad890ea87ede847 Reviewed-on: https://go-review.googlesource.com/28210 Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-04-16runtime: common handling of _AT_RANDOM auxvAustin Clements
The Linux kernel provides 16 bytes of random data via the auxv vector at startup. Currently we consume this separately on 386, amd64, arm, and arm64. Now that we have a common auxv parser, handle _AT_RANDOM in the common path. Change-Id: Ib69549a1d37e2d07a351cf0f44007bcd24f0d20d Reviewed-on: https://go-review.googlesource.com/22062 Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2016-04-16runtime: common auxv parserAustin Clements
Currently several different Linux architectures have separate copies of the auxv parser. Bring these all together into a single copy of the parser that calls out to a per-arch handler for each tag/value pair. This is in preparation for handling common auxv tags in one place. For #9993. Change-Id: Iceebc3afad6b4133b70fca7003561ae370445c10 Reviewed-on: https://go-review.googlesource.com/22061 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
2016-04-06runtime: get randomness from AT_RANDOM AUXV on linux/arm64Shenghou Ma
Fixes #15147. Change-Id: Ibfe46c747dea987787a51eb0c95ccd8c5f24f366 Reviewed-on: https://go-review.googlesource.com/21580 Run-TryBot: Minux Ma <minux@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2015-04-15runtime: mark all runtime.cputicks implementations NOSPLITDave Cheney
Fixes #10450 runtime.cputicks is called from runtime.exitsyscall and must not split the stack. cputicks is implemented in several ways and the NOSPLIT annotation was missing from a few of these. Change-Id: I5cbbb4e5888c5d298fe2fef240782d0e49f59af8 Reviewed-on: https://go-review.googlesource.com/8939 Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
2015-03-16runtime: add support for linux/arm64Aram Hăvărneanu
Change-Id: Ibda6a5bedaff57fd161d63fc04ad260931d34413 Reviewed-on: https://go-review.googlesource.com/7142 Reviewed-by: Russ Cox <rsc@golang.org>
7apu2bq3rhoylwmucxizk54afw5jyda7pho4j5zdcqpwkufke7zdzwad.onion