cmd/internal/obj: make aix/ppc64 builds reproducible

sort.Slice was being used to sort some newly added entries by name
to make the ctxt.Data slice reproducible, but some existing entries
have the same name, and the effect was to take the non-determinism
of the tail entries and scatter it into the earlier, deterministic section
when multiple entries had the same name.

The specific entries with the same name are type SDWARFVAR, which
all have an empty name but different relocations. If they are shuffled,
then the relocation symbols are visited in a different order, which
enters them into the string table in a different order, which results in
different object files, different object file hashes, and different build IDs
for the final executables.

Use sort.SliceStable to avoid reordering entries we don't mean to reorder.

Also add a simple test for scheduling-related non-determinism.
I debugged this originally using 'go install -race cmd/compile',
but that was slow and turned out not to be terribly reliable.
Using a few different GOMAXPROCS settings turns out to be a much more
effective (and faster) way to scramble scheduling decisions.

Fixes #60759.

Change-Id: Ia966b02b9fdaefa971f998a09319ca375bdf8604
Reviewed-on: https://go-review.googlesource.com/c/go/+/502755
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>

2 files changed, 31 insertions, 5 deletions

diff --git a/src/cmd/go/testdata/script/repro_build.txt b/src/cmd/go/testdata/script/repro_build.txt
new file mode 100644
index 0000000000..7c6e317cec
--- /dev/null
+++ b/src/cmd/go/testdata/script/repro_build.txt
@@ -0,0 +1,22 @@
+# Check that goroutine scheduling does not affect compiler output.
+# If it does, reproducible builds will not work very well.
+[short] skip
+[GOOS:aix] env CGO_ENABLED=0  # go.dev/issue/56896
+env GOMAXPROCS=16
+go build -a -o http16.o net/http
+env GOMAXPROCS=17
+go build -a -o http17.o net/http
+cmp -q http16.o http17.o
+env GOMAXPROCS=18
+go build -a -o http18.o net/http
+cmp -q http16.o http18.o
+
+# Check that goroutine scheduling does not affect linker output.
+env GOMAXPROCS=16
+go build -a -o gofmt16.exe cmd/gofmt
+env GOMAXPROCS=17
+go build -a -o gofmt17.exe cmd/gofmt
+cmp -q gofmt16.exe gofmt17.exe
+env GOMAXPROCS=18
+go build -a -o gofmt18.exe cmd/gofmt
+cmp -q gofmt16.exe gofmt18.exe
diff --git a/src/cmd/internal/obj/sym.go b/src/cmd/internal/obj/sym.go
index 6a5ab6c349..2b885f6a10 100644
--- a/src/cmd/internal/obj/sym.go
+++ b/src/cmd/internal/obj/sym.go
@@ -189,11 +189,15 @@ func (ctxt *Link) GCLocalsSym(data []byte) *LSym {
 // in which case all the symbols are non-package (for now).
 func (ctxt *Link) NumberSyms() {
 	if ctxt.Headtype == objabi.Haix {
-		// Data must be sorted to keep a constant order in TOC symbols.
-		// As they are created during Progedit, two symbols can be switched between
-		// two different compilations. Therefore, BuildID will be different.
-		// TODO: find a better place and optimize to only sort TOC symbols
-		sort.Slice(ctxt.Data, func(i, j int) bool {
+		// Data must be in a reliable order for reproducible builds.
+		// The original entries are in a reliable order, but the TOC symbols
+		// that are added in Progedit are added by different goroutines
+		// that can be scheduled independently. We need to reorder those
+		// symbols reliably. Sort by name but use a stable sort, so that
+		// any original entries with the same name (all DWARFVAR symbols
+		// have empty names but different relocation sets) are not shuffled.
+		// TODO: Find a better place and optimize to only sort TOC symbols.
+		sort.SliceStable(ctxt.Data, func(i, j int) bool {
 			return ctxt.Data[i].Name < ctxt.Data[j].Name
 		})
 	}