This should make parallel execution a bit clearer.
With -p=1 it should make the execution completely unambiguous.
Fixes#19280.
Change-Id: Ib48cdfe96896d01b0d8f98ccb2fab614407a7d92
Reviewed-on: https://go-review.googlesource.com/49430
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.LastIndexByte was introduced in go1.5 and it can be used
effectively wherever the second argument to strings.LastIndex is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.LastIndex.
Change-Id: I7b5679d616197b055cffe6882a8675d24a98b574
Reviewed-on: https://go-review.googlesource.com/66372
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Most of these are return values that were part of a receiving parameter,
so they're still accessible.
A few others are not, but those have never had a use.
Found with github.com/mvdan/unparam, after Kevin Burke's suggestion that
the tool should also warn about unused result parameters.
Change-Id: Id8b5ed89912a99db22027703a88bd94d0b292b8b
Reviewed-on: https://go-review.googlesource.com/55910
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Otherwise, if there are any parallel tests, it will hang and panic with
"all goroutines are asleep - deadlock!".
Do not use flag.Uint to handle the error for us because we also want to
error on N==0, and because it would make setting the default to
GOMAXPROCS(0) more difficult, since it's an int.
Check for it right after flag.Parse, and mimic flag errors by printing
the usage and returning exit code 2.
Fixes#20542.
Change-Id: I0c9d4587f83d406a8f5e42ed74e40be46d639ffb
Reviewed-on: https://go-review.googlesource.com/54150
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This addresses the case of a -timeout panic, but not the more
general case of a signal arriving. See CL 48370 and CL 44352
for recent difficulties in that area.
"-timeout" here means flag usage to distinguish from the
default timeout termination which uses signals.
Fixes#19394
Change-Id: I5452d5422c0c080e940cbcc8c6606049975268c6
Reviewed-on: https://go-review.googlesource.com/48491
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently all package tests are executed once
with Parallel tests executed in parallel.
Then this process is repeated count*cpu times.
Tests are not parallelized over count*cpu.
Parallelizing over cpu is not possible as
GOMAXPROCS is a global setting. But it is
possible for count.
Parallelize over count.
Brings down testing of my package with -count=100
form 10s to 0.3s.
Change-Id: I76d8322adeb8c5c6e70b99af690291fd69d6402a
Reviewed-on: https://go-review.googlesource.com/44830
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The code was adding race.Errors to t.raceErrors before checking
Failed, but Failed was using t.raceErrors+race.Errors. We don't want
to change Failed, since that would affect tests themselves, so modify
the harness to not unnecessarily change t.raceErrors.
Updates #19851Fixes#21338
Change-Id: I7bfdf281f90e045146c92444f1370d55c45221d4
Reviewed-on: https://go-review.googlesource.com/54050
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
SkipNow and FailNow must be called from the goroutine running the
test. This is already documented, but it's easy to call them by
mistake when writing subtests. In the following:
func TestPanic(t *testing.T) {
t.Run("", func(t2 *testing.T) {
t.FailNow() // BAD: should be t2.FailNow()
})
}
the FailNow call on the outer t *testing.T correctly triggers a panic
panic: test executed panic(nil) or runtime.Goexit
The error message confuses users (see issues #17421, #21175) because
there is no way to trace back the relevant part of the message ("test
executed ... runtime.Goexit") to a bad FailNow call without checking
the testing package source code and finding out that FailNow calls
runtime.Goexit.
To help users debug the panic message, mention in the SkipNow and
FailNow documentation that they stop execution by calling
runtime.Goexit.
Fixes#21175
Change-Id: I0a3e5f768e72b464474380cfffbf2b67396ac1b5
Reviewed-on: https://go-review.googlesource.com/52770
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fixes#21205
Change-Id: I81b001eb42cbf2a5d5b7b82eb63548b22f501be5
Reviewed-on: https://go-review.googlesource.com/52110
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
CL 44352 changed the behavior of SIGINT, which can break tests that
themselves use SIGINT. I think we can only implement this if the
testing package has a way to know whether the code under test is using
SIGINT, but os/signal does not provide an API for that. Roll back for
1.9 and think about this again for 1.10.
Updates #19397
Change-Id: I021c314db2b9d0a80d0088b120a6ade685459990
Reviewed-on: https://go-review.googlesource.com/48370
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Now that ReadMemStats is fast (CL 34937), CL 36791 is not so
necessary, and causes confusion. See #20863
This was already partially reverted in CL 46612 but missed two of the
spots.
Fixes#20863
Change-Id: I1307a0f7b1f9e86e8b6ceaa6a677f24f13431110
Reviewed-on: https://go-review.googlesource.com/47350
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Apparently, "all such calls must happen" means that the t.Run call
must *return* before the outer test function returns, or the calls
will cause a data race on t.ran.
Clarify the docs.
Fixes#20339
Change-Id: I191a9af2a9095be1e0aaf10b79c30e00a9c495cb
Reviewed-on: https://go-review.googlesource.com/47150
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If the only way the user indicates they want alloc stats shown
is via ReportAllocs, we don't know that until benchFunc is run.
Therefore, StopTimer's ReadMemStats will return incorrect data
for single cycle runs since there's no counterpart ReadMemStats from
StartTimer that initializes alloc stats.
It appears that this bug was introduced by CL 46612,
"testing: only call ReadMemStats if necessary when benchmarking"
Fixes#20590
Change-Id: I3b5ef91677823f4b98011880a3be15423baf7e33
Reviewed-on: https://go-review.googlesource.com/46612
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Also reword the testing/quick.Config field docs to conform to the
normal subject-first style. Without that style, godoc links
/pkg/testing/quick/#Config.Rand to the wrong line, since it doesn't
recognize the preceding comment as necessarily being attached.
Fixes#20809
Change-Id: I9aebbf763eed9b1ab1a153fa11850d88a65571c6
Reviewed-on: https://go-review.googlesource.com/46910
Reviewed-by: Ian Lance Taylor <iant@golang.org>
If you have BenchmarkX1 with sub-benchmark Y
and you have BenchmarkX2 with no sub-benchmarks,
then
go test -bench=X/Y
runs BenchmarkX1 once with b.N=1 (to find out about Y)
and then not again, because it has sub-benchmarks,
but arguably also because we're interested in Y.
In contrast, it runs BenchmarkX2 in full, even though clearly
that is not relevant to the match X/Y. We do have to run X2
once with b.N=1 to probe for having X2/Y, but we should not
run it with larger b.N.
Fixes#20589.
Change-Id: Ib86907e844f34dcaac6cd05757f57db1019201d0
Reviewed-on: https://go-review.googlesource.com/46031
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Because of parallel tests, which have stalled executions, the RUN
output of a test can be much earlier than its completion output resulting
in hard-to-read verbose output.
The tests are displayed in the order in which the output shows
that they began, to make it easy to line up with the "RUN" output.
Similarly, the definitions of when tests begin and complete is
determined by when RUN and FAIL/SKIP/PASS are output since the
focus of this code is on enhancing readability.
Fixes#19397
Change-Id: I4d0ca3fd268b620484e7a190117f79a33b3dc461
Reviewed-on: https://go-review.googlesource.com/44352
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Some large testing/build systems require some form of test discovery before
running tests. This usually allows for analytics, history, and stats on a per
tests basis. Typically these systems are meant used in multi-language
environments and the original source code is not known or available.
This adds a -test.list option which takes a regular expression as an
argument. Any tests, benchmarks, or examples that match that regular
expression will be printed, one per line, to stdout and then the program
will exit.
Since subtests are named/discovered at run time this will only show
top-level tests names and is a known limitation.
Fixes#17209
Change-Id: I7e607f5f4f084d623a1cae88a1f70e7d92b7f13e
Reviewed-on: https://go-review.googlesource.com/41195
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously, helpers were identified by entry PC, but this breaks if the
helper is inlined (as in notHelperCallingHelper). Instead, identify
helpers by function name (with package path). Now TestTBHelper and
TestTBHelperParallel pass with -l=4.
To keep the code unified, this change makes it so that the runner
is also identified by function name instead of entry PC.
Change-Id: I1b1987fc49d114e69d075fab56aeeacd5294982b
Reviewed-on: https://go-review.googlesource.com/41257
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
This CL implements the proposal at
https://github.com/golang/proposal/blob/master/design/4899-testing-helper.md.
It's based on Josh's CL 79890043 from a few years ago:
https://codereview.appspot.com/79890043 but makes several changes,
most notably by using the new CallersFrames API so that it works with
mid-stack inlining.
Another detail came up while I was working on this: I didn't want the
user to be able to call t.Helper from inside their TestXxx function
directly (which would mean we'd print a file:line from inside the
testing package itself), so I explicitly prevented this from working.
Fixes#4899.
Change-Id: I37493edcfb63307f950442bbaf993d1589515310
Reviewed-on: https://go-review.googlesource.com/38796
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When generating a random int8, uint8, int16, uint16, int32, uint32,
quick.Value chooses among all possible values.
But when generating a random int64 or uint64, it only chooses
values in the range [-2⁶², 2⁶²) (even for uint64).
It should, like for all the other integers, use the full range.
If it had, this would have caught #19807 earlier.
Instead it let us discover the presence of #19809.
While we are here, also make the default source of
randomness not completely deterministic.
Fixes#19808.
Change-Id: I070f852531c92b3670bd76523326c9132bfc9416
Reviewed-on: https://go-review.googlesource.com/39152
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
The run1 call removed in golang.org/cl/36990 was necessary to
initialize the duration of the benchmark. With it gone, the math in
launch() starts from 100. This doesn't work out well for second-long
benchmark methods. Put it back.
Updates #18815
Change-Id: I461f3466c805d0c61124a2974662f7ad45335794
Reviewed-on: https://go-review.googlesource.com/37530
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
When running benchmarks with -cpuprofile,
the entire process gets profiled,
and ReadMemStats is surprisingly expensive.
Running the sort benchmarks right now with
-cpuprofile shows almost half of all execution
time in ReadMemStats.
Since ReadMemStats is not required if the benchmark
does not need allocation stats, simply skip it.
This will make cpu profiles nicer to read
and significantly speed up the process of running benchmarks.
It might also make sense to toggle cpu profiling
on/off as we begin/end individual benchmarks,
but that wouldn't get us the time savings of
skipping ReadMemStats, so this CL is useful in itself.
Change-Id: I425197b1ee11be4bc91d22b929e2caf648ebd7c5
Reviewed-on: https://go-review.googlesource.com/36791
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When running benchmarks, print "goos", "goarch", and "pkg"
labels. This makes it easier to refer to benchmark logs and understand
how they were generated. "pkg" is printed only for benchmarks located
in GOPATH.
Change-Id: I397cbdd57b9fe8cbabbb354ec7bfba59f5625c42
Reviewed-on: https://go-review.googlesource.com/36356
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This reverts commit ed8c62b7fb.
Turns out it was needed in later commits.
Change-Id: I07a7bc2429976d8a5a89f915a11625c118b85500
Reviewed-on: https://go-review.googlesource.com/36113
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Prior to this change it was possible to see interleaved messages:
<<<
=== RUN Test/LongLongLongLongName48
=== RUN Test/LongLon=== RUN Test/LongLongLongLongName50
gLongLongName49
=== RUN Test/LongLongLongLongName51
>>>
This change fixes it such that you see:
<<<
=== RUN Test/LongLongLongLongName48
=== RUN Test/LongLongLongLongName49
=== RUN Test/LongLongLongLongName50
=== RUN Test/LongLongLongLongName51
>>>
Fixes#18741
Change-Id: I2529d724065dc65b3e9eb3d7cbeeda82a2d0cfd4
Reviewed-on: https://go-review.googlesource.com/35556
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
Change-Id: I429637ca91f7db4144f17621de851a548dc1ce76
Reviewed-on: https://go-review.googlesource.com/34923
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
No test because in practice these errors never occur.
Change-Id: I11c77893ae931fc621c98920cba656790d18ed93
Reviewed-on: https://go-review.googlesource.com/34811
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The TestMain docs explain that flag.Parse() should be called if TestMain
itself depends on command-line flags.
The issue here is that the example implementation does not use any
flags, and thus the flag.Parse call is unnecessary. This leads to people
who use this example as a starting point for their own implementations
to forget that the call is not necessary in most cases.
Comment it out instead of removing the line to keep it as a reminder, as
suggested by Minux Ma.
Change-Id: I6ffc5413e7036366ae3cf0f069b7065e832a3b45
Reviewed-on: https://go-review.googlesource.com/33273
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
From the doc comment:
Context returns the context for the current test or benchmark.
The context is cancelled when the test or benchmark finishes.
A goroutine started during a test or benchmark can wait for the
context's Done channel to become readable as a signal that the
test or benchmark is over, so that the goroutine can exit.
Fixes#16221.
Fixes#17552.
Change-Id: I657df946be2c90048cc74615436c77c7d9d1226c
Reviewed-on: https://go-review.googlesource.com/31724
Reviewed-by: Rob Pike <r@golang.org>
Currently, we don't have package testing to import package regexp directly,
because then regexp can't have internal tests (or at least they become more
difficult to write), for fear of an import cycle. The solution we've been using
is for the generated test main package (pseudo-import path "testmain", package main)
to import regexp and pass in a matchString function for use by testing when
implementing the -run flags. This lets testing use regexp but without depending
on regexp and creating unnecessary cycles.
We want to add a few dependencies to runtime/pprof, notably regexp
but also compress/gzip, without causing those packages to have to work
hard to write internal tests.
Restructure the (dare I say it) dependency injection of regexp.MatchString
to be more general, and use it for the runtime/pprof functionality in addition
to the regexp functionality. The new package testing/internal/testdeps is
the root for the testing dependencies handled this way.
Code using testing.MainStart will have to change from passing in a matchString
implementation to passing in testdeps.TestDeps{}. Users of 'go test' don't do this,
but other build systems that have recreated 'go test' (for example, Blaze/Bazel)
may need to be updated. The new testdeps setup should make future updates
unnecessary, but even so we keep the comment about MainStart not being
subject to Go 1 compatibility.
Change-Id: Iec821d2afde10c79f95f3b23de5e71b219f47b92
Reviewed-on: https://go-review.googlesource.com/32455
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This makes it possible to avoid tests where coverage affects the test
results by skipping them (or otherwise adjusting them) when coverage
is enabled.
Update #17699
Change-Id: Ifcc36cfcd88ebd677890e82ba80ee3d696ed3d7c
Reviewed-on: https://go-review.googlesource.com/32483
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In a previous change, cmd/go was taught to show a "no tests ran" warning
if test did nothing. But it missed a case - if no tests nor examples ran
but any benchmarks were meant to be run, it would still produce the
warning. This meant that running only benchmarks, which is common, would
be confusing:
$ go test -run='^$' -bench=.
testing: warning: no tests to run
BenchmarkFoo-4 300000 5056 ns/op
[...]
I believe this was because of a copy-paste error in the tests. This was
being tested, but on the wrong file which does contain a test that was
being run. Fix the path and fix the now failing test by never showing
the warning if -bench was given a non-empty string.
The rationale is that if -bench was given but there was no output, it's
obvious that nothing happened as benchmarks always produce output even
without -v. So showing a warning in those cases is redundant.
To make future typos less likely, make sure that no tests are being run
in the cases where we only want to run benchmarks.
Fixes#17603.
Change-Id: I4c626caf39f72260c6a9761c06446663f465f947
Reviewed-on: https://go-review.googlesource.com/32157
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
runtime.SetMutexProfileFraction(n int) will capture 1/n-th of stack
traces of goroutines holding contended mutexes if n > 0. From runtime/pprof,
pprot.Lookup("mutex").WriteTo writes the accumulated
stack traces to w (in essentially the same format that blocking
profiling uses).
Change-Id: Ie0b54fa4226853d99aa42c14cb529ae586a8335a
Reviewed-on: https://go-review.googlesource.com/29650
Reviewed-by: Austin Clements <austin@google.com>
