gotest: another try at flags.

doc.go contains the details. The short story:
- command line is passed to the binary
- a new flag, -file, is needed to name files
- known flags have the "test." prefix added for convenience.
- gotest-specific flags are trimmed from the command line.

The effect should be that most existing uses are unaffected,
the ability to name files is still present, and it's nicer to use.
The downside is a lot more code in gotest.

Also allow a test to be called just Test.

R=rsc, niemeyer, rog, r2
CC=golang-dev
https://golang.org/cl/4307049

src/cmd/gotest/Makefile

@@ -6,6 +6,7 @@ include ../../Make.inc
 
 TARG=gotest
 GOFILES=\
+	flag.go\
 	gotest.go\
 
 include ../../Make.cmd

src/cmd/gotest/doc.go

@@ -6,22 +6,22 @@
 
 Gotest is an automated testing tool for Go packages.
 
-Normally a Go package is compiled without its test files.  Gotest
+Normally a Go package is compiled without its test files.  Gotest is a
-is a simple script that recompiles the package along with any files
+tool that recompiles the package whose source in the current
-named *_test.go.  Functions in the test sources named TestXXX
+directory, along with any files named *_test.go.  Functions in the
-(where XXX is any alphanumeric string starting with an upper case
+test source named TestXXX (where XXX is any alphanumeric string not
-letter) will be run when the binary is executed.  Gotest requires
+starting with a lower case letter) will be run when the binary is
-that the package have a standard package Makefile, one that
+executed.  Gotest requires that the package have a standard package
-includes go/src/Make.pkg.
+Makefile, one that includes go/src/Make.pkg.
 
 The test functions are run in the order they appear in the source.
-They should have signature
+They should have the signature,
 
 	func TestXXX(t *testing.T) { ... }
 
-Benchmark functions can be written as well; they will be run only
+Benchmark functions can be written as well; they will be run only when
-when the -test.bench flag is provided.  Benchmarks should have
+the -test.bench flag is provided.  Benchmarks should have the
-signature
+signature,
 
 	func BenchmarkXXX(b *testing.B) { ... }
 
@@ -29,39 +29,47 @@ See the documentation of the testing package for more information.
 
 By default, gotest needs no arguments.  It compiles all the .go files
 in the directory, including tests, and runs the tests.  If file names
-are given, only those test files are added to the package.
+are given (with flag -file=test.go, one per extra test source file),
-(The non-test files are always compiled.)
+only those test files are added to the package.  (The non-test files
+are always compiled.)
 
 The package is built in a special subdirectory so it does not
 interfere with the non-test installation.
 
 Usage:
-	gotest [-c] [-x] [testflags...] [pkg_test.go...]
+	gotest [-file a.go -file b.go ...] [-c] [-x] [args for test binary]
 
-The flags specific to gotest include -x, which prints each subcommand
+The flags specific to gotest are:
-gotest executes, and -c, which causes gotest to compile the test
+	-c         Compile the test binary but do not run it.
-binary but not run it.  The testflags are passed to the test binary
+	-file a.go Use the tests in the source file a.go instead of *_test.go.
-and are documented below.
+	-x         Print each subcommand gotest executes.
+
+Everything else on the command line is passed to the test binary.
 
 The resulting test binary, called (for amd64) 6.out, has several flags.
 
 Usage:
 	6.out [-test.v] [-test.run pattern] [-test.bench pattern] \
-		[-test.memprofile=prof.out] [-test.memprofilerate=1]
+		[-test.cpuprofile=cpu.out] \
+		[-test.memprofile=mem.out] [-test.memprofilerate=1]
 
 The -test.v flag causes the tests to be logged as they run.  The
 -test.run flag causes only those tests whose names match the regular
-expression pattern to be run. By default all tests are run silently.
+expression pattern to be run.  By default all tests are run silently.
-If all the specified test pass, 6.out prints PASS and exits with a 0
+
-exit code.  If any tests fail, it prints FAIL and exits with a
+If all specified tests pass, 6.out prints the word PASS and exits with
-non-zero code.  The -test.bench flag is analogous to the -test.run
+a 0 exit code.  If any tests fail, it prints error details, the word
-flag, but applies to benchmarks.  No benchmarks run by default.
+FAIL, and exits with a non-zero code.  The -test.bench flag is
+analogous to the -test.run flag, but applies to benchmarks.  No
+benchmarks run by default.
+
+The -test.cpuprofile flag causes the testing software to write a CPU
+profile to the specified file before exiting.
 
 The -test.memprofile flag causes the testing software to write a
-memory profile to the specified file when all tests are complete.  Use
+memory profile to the specified file when all tests are complete.  The
--test.run or -test.bench to limit the profile to a particular test or
+-test.memprofilerate flag enables more precise (and expensive)
-benchmark.  The -test.memprofilerate flag enables more precise (and
+profiles by setting runtime.MemProfileRate; run
-expensive) profiles by setting runtime.MemProfileRate;
 	godoc runtime MemProfileRate
 for details.  The defaults are no memory profile and the standard
 setting of MemProfileRate.  The memory profile records a sampling of
@@ -71,16 +79,20 @@ the environment variable GOGC=off to disable the garbage collector,
 provided the test can run in the available memory without garbage
 collection.
 
-The -test.short package tells long-running tests to shorten their
+Use -test.run or -test.bench to limit profiling to a particular test
-run time. It is off by default but set by all.bash so installations
+or benchmark.
-of the Go tree can do a sanity check but not spend time running the
-full test suite.
 
-For convenience, each -test.X flag of the test binary is also
+The -test.short package tells long-running tests to shorten their run
-available as the flag -X in gotest itself.  For instance, the command
+time.  It is off by default but set by all.bash so installations of
-	gotest -v -test.cpuprofile=prof.out
+the Go tree can do a sanity check but not spend time running
-will compile the test binary and then run it as
+exhaustive tests.
-	6.out -test.v -cpuprofile=prof.out
+
+For convenience, each of these -test.X flags of the test binary is
+also available as the flag -X in gotest itself.  Flags not listed here
+are unaffected.  For instance, the command
+	gotest -x -v -cpuprofile=prof.out -dir=testdata -update -file x_test.go
+will compile the test binary using x_test.go and then run it as
+	6.out -test.v -test.cpuprofile=prof.out -dir=testdata -update
 
 */
 package documentation

src/cmd/gotest/flag.go

@@ -0,0 +1,153 @@
+// Copyright 2011 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+import (
+	"fmt"
+	"os"
+	"strconv"
+	"strings"
+)
+
+// The flag handling part of gotest is large and distracting.
+// We can't use the flag package because some of the flags from
+// our command line are for us, and some are for 6.out, and
+// some are for both.
+
+var usageMessage = `Usage of %s:
+  -c=false: compile but do not run the test binary
+  -file=file:
+  -x=false: print command lines as they are executed
+
+  // These flags can be passed with or without a "test." prefix: -v or -test.v.
+  -bench="": passes -test.bench to test
+  -cpuprofile="": passes -test.cpuprofile to test
+  -memprofile="": passes -test.memprofile to test
+  -memprofilerate=0: passes -test.memprofilerate to test
+  -run="": passes -test.run to test
+  -short=false: passes -test.short to test
+  -v=false: passes -test.v to test
+`
+
+// usage prints a usage message and exits.
+func usage() {
+	fmt.Fprintf(os.Stdout, usageMessage, os.Args[0])
+	os.Exit(2)
+}
+
+// flagSpec defines a flag we know about.
+type flagSpec struct {
+	name       string
+	isBool     bool
+	passToTest bool // pass to Test
+	multiOK    bool // OK to have multiple instances
+	present    bool // flag has been seen
+}
+
+// flagDefn is the set of flags we process.
+var flagDefn = []*flagSpec{
+	// gotest-local.
+	&flagSpec{name: "c", isBool: true},
+	&flagSpec{name: "file", multiOK: true},
+	&flagSpec{name: "x", isBool: true},
+
+	// passed to 6.out, adding a "test." prefix to the name if necessary: -v becomes -test.v.
+	&flagSpec{name: "bench", passToTest: true},
+	&flagSpec{name: "cpuprofile", passToTest: true},
+	&flagSpec{name: "memprofile", passToTest: true},
+	&flagSpec{name: "memprofilerate", passToTest: true},
+	&flagSpec{name: "run", passToTest: true},
+	&flagSpec{name: "short", isBool: true, passToTest: true},
+	&flagSpec{name: "v", isBool: true, passToTest: true},
+}
+
+// flags processes the command line, grabbing -x and -c, rewriting known flags
+// to have "test" before them, and reading the command line for the 6.out.
+// Unfortunately for us, we need to do our own flag processing because gotest
+// grabs some flags but otherwise its command line is just a holding place for
+// 6.out's arguments.
+func flags() {
+	for i := 1; i < len(os.Args); i++ {
+		arg := os.Args[i]
+		f, value, extraWord := flag(i)
+		if f == nil {
+			args = append(args, arg)
+			continue
+		}
+		switch f.name {
+		case "c":
+			setBoolFlag(&cFlag, value)
+		case "x":
+			setBoolFlag(&xFlag, value)
+		case "file":
+			fileNames = append(fileNames, value)
+		}
+		if extraWord {
+			i++
+		}
+		if f.passToTest {
+			args = append(args, "-test."+f.name+"="+value)
+		}
+	}
+}
+
+// flag sees if argument i is a known flag and returns its definition, value, and whether it consumed an extra word.
+func flag(i int) (f *flagSpec, value string, extra bool) {
+	arg := os.Args[i]
+	if strings.HasPrefix(arg, "--") { // reduce two minuses to one
+		arg = arg[1:]
+	}
+	if arg == "" || arg[0] != '-' {
+		return
+	}
+	name := arg[1:]
+	// If there's already "test.", drop it for now.
+	if strings.HasPrefix(name, "test.") {
+		name = name[5:]
+	}
+	equals := strings.Index(name, "=")
+	if equals >= 0 {
+		value = name[equals+1:]
+		name = name[:equals]
+	}
+	for _, f = range flagDefn {
+		if name == f.name {
+			// Booleans are special because they have modes -x, -x=true, -x=false.
+			if f.isBool {
+				if equals < 0 { // otherwise, it's been set and will be verified in setBoolFlag
+					value = "true"
+				} else {
+					// verify it parses
+					setBoolFlag(new(bool), value)
+				}
+			} else { // Non-booleans must have a value.
+				extra = equals < 0
+				if extra {
+					if i+1 >= len(os.Args) {
+						usage()
+					}
+					value = os.Args[i+1]
+				}
+			}
+			if f.present && !f.multiOK {
+				usage()
+			}
+			f.present = true
+			return
+		}
+	}
+	f = nil
+	return
+}
+
+// setBoolFlag sets the addressed boolean to the value.
+func setBoolFlag(flag *bool, value string) {
+	x, err := strconv.Atob(value)
+	if err != nil {
+		fmt.Fprintf(os.Stderr, "gotest: illegal bool flag value %s\n", value)
+		usage()
+	}
+	*flag = x
+}

src/cmd/gotest/gotest.go

@@ -7,7 +7,6 @@ package main
 import (
 	"bufio"
 	"exec"
-	"flag"
 	"fmt"
 	"go/ast"
 	"go/parser"
@@ -23,14 +22,16 @@ import (
 
 // Environment for commands.
 var (
-	XGC    []string // 6g -I _test -o _xtest_.6
+	XGC       []string // 6g -I _test -o _xtest_.6
-	GC     []string // 6g -I _test _testmain.go
+	GC        []string // 6g -I _test _testmain.go
-	GL     []string // 6l -L _test _testmain.6
+	GL        []string // 6l -L _test _testmain.6
-	GOARCH string
+	GOARCH    string
-	GOROOT string
+	GOROOT    string
-	GORUN  string
+	GORUN     string
-	O      string
+	O         string
-	env    = os.Environ()
+	args      []string // arguments passed to gotest; also passed to the binary
+	fileNames []string
+	env       = os.Environ()
 )
 
 // These strings are created by getTestNames.
@@ -44,21 +45,10 @@ var (
 	importPath string
 )
 
-// Flags from package "testing" we will forward to 6.out.  See documentation there
+// Flags for our own purposes. We do our own flag processing.
-// or by running "godoc gotest" - details are in ./doc.go.
 var (
-	test_short          bool
+	cFlag bool
-	test_v              bool
+	xFlag bool
-	test_run            string
-	test_memprofile     string
-	test_memprofilerate int
-	test_cpuprofile     string
-)
-
-// Flags for our own purposes
-var (
-	xFlag = flag.Bool("x", false, "print command lines as they are executed")
-	cFlag = flag.Bool("c", false, "compile but do not run the test binary")
 )
 
 // File represents a file that contains tests.
@@ -72,7 +62,7 @@ type File struct {
 }
 
 func main() {
-	flag.Parse()
+	flags()
 	needMakefile()
 	setEnvironment()
 	getTestFileNames()
@@ -87,29 +77,11 @@ func main() {
 	writeTestmainGo()
 	run(GC...)
 	run(GL...)
-	if !*cFlag {
+	if !cFlag {
-		runWithTestFlags("./" + O + ".out")
+		runTestWithArgs("./" + O + ".out")
 	}
 }
 
-// init sets up pairs of flags.  Each pair contains a flag defined as in testing, and one with
-// the "test." prefix missing, for ease of use.
-func init() {
-	flag.BoolVar(&test_short, "test.short", false, "run smaller test suite to save time")
-	flag.BoolVar(&test_v, "test.v", false, "verbose: print additional output")
-	flag.StringVar(&test_run, "test.run", "", "regular expression to select tests to run")
-	flag.StringVar(&test_memprofile, "test.memprofile", "", "write a memory profile to the named file after execution")
-	flag.IntVar(&test_memprofilerate, "test.memprofilerate", 0, "if >=0, sets runtime.MemProfileRate")
-	flag.StringVar(&test_cpuprofile, "test.cpuprofile", "", "write a cpu profile to the named file during execution")
-	// Now the same flags again, but with shorter names that are forwarded.
-	flag.BoolVar(&test_short, "short", false, "passes -test.short to test")
-	flag.BoolVar(&test_v, "v", false, "passes -test.v to test")
-	flag.StringVar(&test_run, "run", "", "passes -test.run to test")
-	flag.StringVar(&test_memprofile, "memprofile", "", "passes -test.memprofile to test")
-	flag.IntVar(&test_memprofilerate, "memprofilerate", 0, "passes -test.memprofilerate to test")
-	flag.StringVar(&test_cpuprofile, "cpuprofile", "", "passes -test.cpuprofile to test")
-}
-
 // needMakefile tests that we have a Makefile in this directory.
 func needMakefile() {
 	if _, err := os.Stat("Makefile"); err != nil {
@@ -175,7 +147,7 @@ func setEnvironment() {
 // getTestFileNames gets the set of files we're looking at.
 // If gotest has no arguments, it scans the current directory for _test.go files.
 func getTestFileNames() {
-	names := flag.Args()
+	names := fileNames
 	if len(names) == 0 {
 		names = filepath.Glob("[^.]*_test.go")
 		if len(names) == 0 {
@@ -241,9 +213,12 @@ func getTestNames() {
 // It is a Test (say) if there is a character after Test that is not a lower-case letter.
 // We don't want TesticularCancer.
 func isTest(name, prefix string) bool {
-	if !strings.HasPrefix(name, prefix) || len(name) == len(prefix) {
+	if !strings.HasPrefix(name, prefix) {
 		return false
 	}
+	if len(name) == len(prefix) { // "Test" is ok
+		return true
+	}
 	rune, _ := utf8.DecodeRuneInString(name[len(prefix):])
 	return !unicode.IsLower(rune)
 }
@@ -264,34 +239,15 @@ func runWithStdout(argv ...string) string {
 	return s
 }
 
-// runWithTestFlags appends any flag settings to the command line before running it.
+// runTestWithArgs appends gotest's runs the provided binary with the args passed on the command line.
-func runWithTestFlags(argv ...string) {
+func runTestWithArgs(binary string) {
-	if test_short {
+	doRun(append([]string{binary}, args...), false)
-		argv = append(argv, "-test.short")
-	}
-
-	if test_v {
-		argv = append(argv, "-test.v")
-	}
-	if test_run != "" {
-		argv = append(argv, fmt.Sprintf("-test.run=%s", test_run))
-	}
-	if test_memprofile != "" {
-		argv = append(argv, fmt.Sprintf("-test.memprofile=%s", test_memprofile))
-	}
-	if test_memprofilerate > 0 {
-		argv = append(argv, fmt.Sprintf("-test.memprofilerate=%d", test_memprofilerate))
-	}
-	if test_cpuprofile != "" {
-		argv = append(argv, fmt.Sprintf("-test.cpuprofile=%s", test_cpuprofile))
-	}
-	doRun(argv, false)
 }
 
 // doRun is the general command runner.  The flag says whether we want to
 // retrieve standard output.
 func doRun(argv []string, returnStdout bool) string {
-	if *xFlag {
+	if xFlag {
 		fmt.Printf("gotest: %s\n", strings.Join(argv, " "))
 	}
 	var err os.Error