[dev.link] cmd/compile, cmd/link: remove dead methods if type is not used in interface

Currently, a method of a reachable type is live if it matches a
method of a reachable interface. In fact, we only need to retain
the method if the type is actually converted to an interface. If
the type is never converted to an interface, there is no way to
call the method through an interface method call (but the type
descriptor could still be used, e.g. in calling
runtime.newobject).

A type can be used in an interface in two ways:
- directly converted to interface. (Any interface counts, as it
  is possible to convert one interface to another.)
- obtained by reflection from a related type (e.g. obtaining an
  interface of T from []T).

For the former, we let the compiler emit a marker on the type
descriptor symbol when it is converted to an interface. In the
linker, we only need to check methods of marked types.

For the latter, when the linker visits a marked type, it needs to
visit all its "child" types as marked (i.e. potentially could be
converted to interface).

This reduces binary size:
cmd/compile	18792016	18706096 (-0.5%)
cmd/go		14120572	13398948 (-5.1%)

Change-Id: I4465c7eeabf575f4dc84017214c610fa05ae31fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/237298
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>

1 files changed, 22 insertions, 2 deletions

diff --git a/src/cmd/link/internal/ld/deadcode.go b/src/cmd/link/internal/ld/deadcode.go
index d59b1f2c65..1060bbca3b 100644
--- a/src/cmd/link/internal/ld/deadcode.go
+++ b/src/cmd/link/internal/ld/deadcode.go
@@ -102,8 +102,10 @@ func (d *deadcodePass) flood() {
 
 		isgotype := d.ldr.IsGoType(symIdx)
 		relocs := d.ldr.Relocs(symIdx)
+		var usedInIface bool
 
 		if isgotype {
+			usedInIface = d.ldr.AttrUsedInIface(symIdx)
 			p := d.ldr.Data(symIdx)
 			if len(p) != 0 && decodetypeKind(d.ctxt.Arch, p)&kindMask == kindInterface {
 				for _, sig := range d.decodeIfaceMethods(d.ldr, d.ctxt.Arch, symIdx, &relocs) {
@@ -126,7 +128,9 @@ func (d *deadcodePass) flood() {
 				if i+2 >= relocs.Count() {
 					panic("expect three consecutive R_METHODOFF relocs")
 				}
-				methods = append(methods, methodref{src: symIdx, r: i})
+				if usedInIface {
+					methods = append(methods, methodref{src: symIdx, r: i})
+				}
 				i += 2
 				continue
 			}
@@ -136,7 +140,23 @@ func (d *deadcodePass) flood() {
 				// do nothing for now as we still load all type symbols.
 				continue
 			}
-			d.mark(r.Sym(), symIdx)
+			rs := r.Sym()
+			if isgotype && usedInIface && d.ldr.IsGoType(rs) && !d.ldr.AttrUsedInIface(rs) {
+				// If a type is converted to an interface, it is possible to obtain an
+				// interface with a "child" type of it using reflection (e.g. obtain an
+				// interface of T from []chan T). We need to traverse its "child" types
+				// with UsedInIface attribute set.
+				// When visiting the child type (chan T in the example above), it will
+				// have UsedInIface set, so it in turn will mark and (re)visit its children
+				// (e.g. T above).
+				// We unset the reachable bit here, so if the child type is already visited,
+				// it will be visited again.
+				// Note that a type symbol can be visited at most twice, one without
+				// UsedInIface and one with. So termination is still guaranteed.
+				d.ldr.SetAttrUsedInIface(rs, true)
+				d.ldr.SetAttrReachable(rs, false)
+			}
+			d.mark(rs, symIdx)
 		}
 		naux := d.ldr.NAux(symIdx)
 		for i := 0; i < naux; i++ {