testing, cmd/go: clarify documentation

This CL removes 'go help fuzz' but expands the testing package
documentation with much of the same information. It also removes
documentation for the unimplemented -keepfuzzing flag and makes a
number of other clarifications.

Addressing comments in CL 348469.

Updates #46629

Change-Id: I12ab5971c900c2e43f2c2d83c6705e8cd642388b
Reviewed-on: https://go-review.googlesource.com/c/go/+/351113
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>

1 files changed, 56 insertions, 16 deletions

diff --git a/src/testing/testing.go b/src/testing/testing.go
index 567eb0dfa3..ac1e52af85 100644
--- a/src/testing/testing.go
+++ b/src/testing/testing.go
@@ -134,28 +134,68 @@
 //
 // Fuzzing
 //
-// Functions of the form
+// 'go test' and the testing package support fuzzing, a testing technique where
+// a function is called with randomly generated inputs to find bugs not
+// anticipated by unit tests.
+//
+// A fuzz target is a function that declares a set of "seed" inputs by calling
+// F.Add, then provides a fuzz function by calling F.Fuzz. A fuzz target has
+// the form:
+//
 //     func FuzzXxx(*testing.F)
-// are considered fuzz targets, and are executed by the "go test" command. When
-// the -fuzz flag is provided, the functions will be fuzzed.
 //
-// For a description of the testing flags, see
-// https://golang.org/cmd/go/#hdr-Testing_flags.
+// For example:
+//
+// func FuzzHex(f *testing.F) {
+//   for _, seed := range [][]byte{{}, {0}, {9}, {0xa}, {0xf}, {1, 2, 3, 4}} {
+//     f.Add(seed)
+//   }
+//   f.Fuzz(func(t *testing.T, in []byte) {
+//     enc := hex.EncodeToString(in)
+//     out, err := hex.DecodeString(enc)
+//     if err != nil {
+//       t.Fatalf("%v: decode: %v", in, err)
+//     }
+//     if !bytes.Equal(in, out) {
+//       t.Fatalf("%v: not equal after round trip: %v", in, out)
+//     }
+//   })
+// }
+//
+// Seed inputs may be registered by calling F.Add or by storing files in the
+// directory testdata/fuzz/<Name> (where <Name> is the name of the fuzz target)
+// within the package containing the fuzz target. Seed inputs are optional, but
+// the fuzzing engine may find bugs more efficiently when provided with a set
+// of small seed inputs with good code coverage.
+//
+// The fuzz function provided to F.Fuzz must accept a *testing.T parameter,
+// followed by one or more parameters for random inputs. The types of arguments
+// passed to F.Add must be identical to the types of these parameters. The fuzz
+// function may signal that it's found a problem the same way tests do: by
+// calling T.Fail (or any method that calls it like T.Error or T.Fatal) or by
+// panicking.
+//
+// When fuzzing is enabled (by setting the -fuzz flag to a regular expression
+// that matches a specific fuzz target), the fuzz function is called with
+// arguments generated by repeatedly making random changes to the seed inputs.
+// On supported platforms, 'go test' compiles the test executable with fuzzing
+// coverage instrumentation. The fuzzing engine uses that instrumentation to
+// find and cache inputs that expand coverage, increasing the liklihood of
+// finding bugs. If the fuzz function finds a problem, the fuzzing engine writes
+// the inputs that caused the problem to a file in the directory
+// testdata/fuzz/<Name> within the package directory. This file later serves as
+// a seed input. If the file can't be written at that location (for example,
+// because the directory is read-only), the fuzzing engine writes the file to
+// the fuzz cache directory within the build cache instead.
+//
+// When fuzzing is disabled, the fuzz function is called with the seed inputs
+// registered with F.Add and seed inputs from testdata/fuzz/<Name>. In this
+// mode, the fuzz target acts much like a regular test, with subtests started
+// with F.Fuzz instead of T.Run.
 //
-// For a description of fuzzing, see golang.org/s/draft-fuzzing-design.
 // TODO(#48255): write and link to documentation that will be helpful to users
 // who are unfamiliar with fuzzing.
 //
-// A sample fuzz target looks like this:
-//
-//     func FuzzBytesCmp(f *testing.F) {
-//         f.Fuzz(func(t *testing.T, a, b []byte) {
-//             if bytes.HasPrefix(a, b) && !bytes.Contains(a, b) {
-//                 t.Error("HasPrefix is true, but Contains is false")
-//             }
-//         })
-//     }
-//
 // Skipping
 //
 // Tests or benchmarks may be skipped at run time with a call to