cmd/dist: build bootstrap without GOEXPERIMENT

Currently, dist attempts to build the bootstrap with the GOEXPERIMENT
set in the environment. However, the logic is incomplete and notably
requires a hack to enable the appropriate build tags for
GOEXPERIMENT=regabi. Without this hack, the build becomes skewed
between a compiler that uses regabi and a runtime that doesn't when
building toolchain2.

We could try to improve the GOEXPERIMENT processing in cmd/dist, but
it will always chase cmd/internal/objabi and it's quite difficult to
share the logic with objabi because of the constraints on building
cmd/dist.

Instead, we switch to building go_bootstrap without any GOEXPERIMENT
and only start using GOEXPERIMENT once we have a working, modern
cmd/go (which has all the GOEXPERIMENT logic in it). We also build
toolchain1 without any GOEXPERIMENT set, in case the bootstrap
toolchain is recent enough to understand build-time GOEXPERIMENT
settings.

As part of this, we make GOEXPERIMENT=none mean "no experiments". This
is necessary since, now that we support setting GOEXPERIMENT at build
time, we need an explicit way to say "ignore all baked-in experiments".

For #40724.

Change-Id: I115399579b766a7a8b2f352f7e5efea5305666cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/302050
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>

src/cmd/dist/build.go

@@ -974,11 +974,6 @@ func matchtag(tag string) bool {
 		}
 		return !matchtag(tag[1:])
 	}
-	if os.Getenv("GOEXPERIMENT") == "regabi" && tag == "goexperiment.regabi" {
-		// TODO: maybe we can handle GOEXPERIMENT more generally.
-		// Or remove once we commit to regabi (#40724).
-		return true
-	}
 	return tag == "gc" || tag == goos || tag == goarch || tag == "cmd_go_bootstrap" || tag == "go1.1" ||
 		(goos == "android" && tag == "linux") ||
 		(goos == "illumos" && tag == "solaris") ||
@@ -1268,6 +1263,20 @@ func cmdbootstrap() {
 	// go tool may complain.
 	os.Setenv("GOPATH", pathf("%s/pkg/obj/gopath", goroot))
 
+	// Disable GOEXPERIMENT when building toolchain1 and
+	// go_bootstrap. We don't need any experiments for the
+	// bootstrap toolchain, and this lets us avoid duplicating the
+	// GOEXPERIMENT-related build logic from cmd/go here. If the
+	// bootstrap toolchain is < Go 1.17, it will ignore this
+	// anyway since GOEXPERIMENT is baked in; otherwise it will
+	// pick it up from the environment we set here. Once we're
+	// using toolchain1 with dist as the build system, we need to
+	// override this to keep the experiments assumed by the
+	// toolchain and by dist consistent. Once go_bootstrap takes
+	// over the build process, we'll set this back to the original
+	// GOEXPERIMENT.
+	os.Setenv("GOEXPERIMENT", "none")
+
 	if debug {
 		// cmd/buildid is used in debug mode.
 		toolchain = append(toolchain, "cmd/buildid")
@@ -1345,6 +1354,8 @@ func cmdbootstrap() {
 	}
 	xprintf("Building Go toolchain2 using go_bootstrap and Go toolchain1.\n")
 	os.Setenv("CC", compilerEnvLookup(defaultcc, goos, goarch))
+	// Now that cmd/go is in charge of the build process, enable GOEXPERIMENT.
+	os.Setenv("GOEXPERIMENT", goexperiment)
 	goInstall(goBootstrap, append([]string{"-i"}, toolchain...)...)
 	if debug {
 		run("", ShowOutput|CheckExit, pathf("%s/compile", tooldir), "-V=full")

src/cmd/internal/objabi/util.go

@@ -135,11 +135,14 @@ func init() {
 
 	goexperiment := envOr("GOEXPERIMENT", defaultGOEXPERIMENT)
 
+	// GOEXPERIMENT=none overrides all experiments enabled at dist time.
+	if goexperiment != "none" {
 		for _, f := range strings.Split(goexperiment, ",") {
 			if f != "" {
 				addexp(f)
 			}
 		}
+	}
 
 	// regabi is only supported on amd64.
 	if GOARCH != "amd64" {