crypto/x509: keep smaller root cert representation in memory until needed

Instead of parsing the PEM files and then storing the *Certificate
values forever, still parse them to see if they're valid and pick out
some fields, but then only store the decoded pem.Block.Bytes until
that cert is first needed.

Saves about 500K of memory on my (Debian stable) machine after doing a
tls.Dial or calling x509.SystemCertPool.

A more aggressive version of this is still possible: we can not keep
the pem.Block.Bytes in memory either, and re-read them from disk when
necessary. But dealing with files disappearing and even large
multi-cert PEM files changing (with offsets sliding around) made this
conservative version attractive. It doesn't change the
slurp-roots-on-startup semantics. It just does so with less memory
retained.

Change-Id: I3aea333f4749ae3b0026042ec3ff7ac015c72204
Reviewed-on: https://go-review.googlesource.com/c/go/+/230025
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>

1 files changed, 16 insertions, 4 deletions

diff --git a/src/crypto/x509/cert_pool.go b/src/crypto/x509/cert_pool.go
index 2cfaeb2d9e..c23ccf1b39 100644
--- a/src/crypto/x509/cert_pool.go
+++ b/src/crypto/x509/cert_pool.go
@@ -10,6 +10,7 @@ import (
 	"encoding/pem"
 	"errors"
 	"runtime"
+	"sync"
 )
 
 type sum224 [sha256.Size224]byte
@@ -224,16 +225,27 @@ func (s *CertPool) AppendCertsFromPEM(pemCerts []byte) (ok bool) {
 			continue
 		}
 
-		cert, err := ParseCertificate(block.Bytes)
+		certBytes := block.Bytes
+		cert, err := ParseCertificate(certBytes)
 		if err != nil {
 			continue
 		}
-
-		s.AddCert(cert)
+		var lazyCert struct {
+			sync.Once
+			v *Certificate
+		}
+		s.addCertFunc(sha256.Sum224(cert.Raw), string(cert.RawSubject), func() (*Certificate, error) {
+			lazyCert.Do(func() {
+				// This can't fail, as the same bytes already parsed above.
+				lazyCert.v, _ = ParseCertificate(certBytes)
+				certBytes = nil
+			})
+			return lazyCert.v, nil
+		})
 		ok = true
 	}
 
-	return
+	return ok
 }
 
 // Subjects returns a list of the DER-encoded subjects of