It's not intuitive, especially in the presence of subtests, so improve the
explanation and extend and explain the examples.
Change-Id: I6c4d3f8944b60b12311d0c0f0a8e952e7c35a9ed
Reviewed-on: https://go-review.googlesource.com/28995
Reviewed-by: Andrew Gerrand <adg@golang.org>
When ns/op dropped below 1, the old code
ignored benchtime and reverted to 1s.
Change-Id: I59752cef88d8d73bfd5b085f5400ae657f78504e
Reviewed-on: https://go-review.googlesource.com/26664
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Since at least 1.0.3, the testing package has said that logs are dumped
to standard error, but has in fact dumped the logs to standard output.
We could change to dump to standard error, but after doing it this way
for so long I think it's better to change the docs.
Fixes#16138.
Change-Id: If39c7ce91f51c7113f33ebabfb8f84fd4611b9e1
Reviewed-on: https://go-review.googlesource.com/24311
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Allows passing regexps per subtest to --test.run and --test.bench
Note that the documentation explicitly states that the split regular
expressions match the correpsonding parts (path components) of
the bench/test identifier. This is intended and slightly different
from the i'th RE matching the subtest/subbench at the respective
level. Picking this semantics allows guaranteeing that a test or
benchmark identifier as printed by go test can be passed verbatim
(possibly quoted) to, respectively, -run or -bench: subtests and
subbenches might have a '/' in their name, causing a misaligment if
their ID is passed to -run or -bench as is.
This semantics has other benefits, but this is the main motivation.
Fixes golang.go#15126
Change-Id: If72e6d3f54db1df6bc2729ac6edc7ab3c740e7c3
Reviewed-on: https://go-review.googlesource.com/19122
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The synchronization in this test is a bit complicated and likely
incorrect, judging from the sporadically hanging trybots.
Most of what this is supposed to test is already tested in
TestTestContext, so I'll just remove it.
Fixes#15170
Change-Id: If54db977503caa109cec4516974eda9191051888
Reviewed-on: https://go-review.googlesource.com/22080
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This introduces a few changes
- Skipped benchmarks now print a SKIP line, also if there was
no output
- The benchmark name is only printed if there the benchmark
was not skipped or did not fail in the probe phase.
It also fixes a bug of doubling a skip message in chatty mode in
absense of a failure.
The chatty flag is now passed in the common struct to allow
for testing of the printed messages.
Fixes#14799
Change-Id: Ia8eb140c2e5bb467e66b8ef20a2f98f5d95415d5
Reviewed-on: https://go-review.googlesource.com/21504
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
by only testing the lower bound of memalloc
Fixes#15063
Change-Id: Iab2fdd75e9ce98c641bfbce57f142fa47176772d
Reviewed-on: https://go-review.googlesource.com/21507
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
Change control flow to probe with N=1. This calls benchFunc
the same number of times as the old implementation in the
absence of subbenchmarks.
To be compatible with existing tools, benchmarking only
prints a line for "leaf" benchmarks. This means, though, that
the name of a benchmark can only be printed after the first
iteration.
Issue #14863
Change-Id: Ic7b9b89b058f8ebb5287755f24f9e47df8c9537c
Reviewed-on: https://go-review.googlesource.com/21043
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
The matcher is responsible for sanitizing and uniquing the
test and benchmark names and thus needs to be included before the
API can be exposed.
Matching currently uses the regexp to only match the top-level
tests/benchmarks.
Support for subtest matching is for another CL.
Change-Id: I7c8464068faef7ebc179b03a7fe3d01122cc4f0b
Reviewed-on: https://go-review.googlesource.com/18897
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixed bug that slipped probably slipped in after rebasing and
explain why it failed on nacl/netbsd/plan9, which set default
maxparallelism to 1.
Change-Id: I4d59682fb2843d138b320334189f53fcdda5b2f6
Reviewed-on: https://go-review.googlesource.com/20980
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
plan9, nacl, and netbsd to be precise.
Only the first test causes a hang, but just to be sure.
Change-Id: I400bb356ee2a0cf12c8666c95af79c924d1629aa
Reviewed-on: https://go-review.googlesource.com/20839
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
API is not exposed yet.
Change-Id: I729360ef2be1d8ea683ca93cdb1763897cc8657c
Reviewed-on: https://go-review.googlesource.com/18895
Reviewed-by: Russ Cox <rsc@golang.org>
testing.go:
- run method will evolve into the Run method.
- added level field in common
benchmark.go:
- benchContext will be central to distinguish handling of benchmarks
between normal Run methods and ones called from within Benchmark
function.
- expandCPU will evolve into the processing hook for Run methods
called within normal processing.
- runBench will evolve into the Run method.
Change-Id: I1816f9985d5ba94deb0ad062302ea9aee0bb5338
Reviewed-on: https://go-review.googlesource.com/18894
Reviewed-by: Russ Cox <rsc@golang.org>
The biggest change is that each test is now responsible for managing
the starting and stopping of its parallel subtests.
The "Main" test could be run as a tRunner as well. This shows that
the introduction of subtests is merely a generalization of and
consistent with the current semantics.
Change-Id: Ibf8388c08f85d4b2c0df69c069326762ed36a72e
Reviewed-on: https://go-review.googlesource.com/18893
Reviewed-by: Russ Cox <rsc@golang.org>
Adds a type of output to Examples that allows tests to have unordered
output. This is intended to help clarify when the output of a command
will produce a fixed return, but that return might not be in an constant
order.
Examples where this is useful would be documenting the rand.Perm()
call, or perhaps the (os.File).Readdir(), both of which can not guarantee
order, but can guarantee the elements of the output.
Fixes#10149
Change-Id: Iaf0cf1580b686afebd79718ed67ea744f5ed9fc5
Reviewed-on: https://go-review.googlesource.com/19280
Reviewed-by: Andrew Gerrand <adg@golang.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a subset of https://golang.org/cl/20022 with only the copyright
header lines, so the next CL will be smaller and more reviewable.
Go policy has been single space after periods in comments for some time.
The copyright header template at:
https://golang.org/doc/contribute.html#copyright
also uses a single space.
Make them all consistent.
Change-Id: Icc26c6b8495c3820da6b171ca96a74701b4a01b0
Reviewed-on: https://go-review.googlesource.com/20111
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Named returned values should only be used on public funcs and methods
when it contributes to the documentation.
Named return values should not be used if they're only saving the
programmer a few lines of code inside the body of the function,
especially if that means there's stutter in the documentation or it
was only there so the programmer could use a naked return
statement. (Naked returns should not be used except in very small
functions)
This change is a manual audit & cleanup of public func signatures.
Signatures were not changed if:
* the func was private (wouldn't be in public godoc)
* the documentation referenced it
* the named return value was an interesting name. (i.e. it wasn't
simply stutter, repeating the name of the type)
There should be no changes in behavior. (At least: none intended)
Change-Id: I3472ef49619678fe786e5e0994bdf2d9de76d109
Reviewed-on: https://go-review.googlesource.com/20024
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Moves the implementation of RunBenchmarks to a non-exported function
that returns whether the execution was OK, and uses that to identify
failure in benchmarks.The exported function is kept for compatibility.
Like before, benchmarks will only be executed if tests and examples
pass. The PASS message will not be printed if there was a failure in
a benchmark.
Example output
BenchmarkThatCallsFatal-8 --- FAIL: BenchmarkThatCallsFatal-8
x_test.go:6: called by benchmark
FAIL
exit status 1
FAIL _/.../src/cmd/go/testdata/src/benchfatal 0.009s
Fixes#14307.
Change-Id: I6f3ddadc7da8a250763168cc099ae8b325a79602
Reviewed-on: https://go-review.googlesource.com/19889
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change adds support in testing/quick to generate maps and slices
in additional states:
(1.) nil maps
(2.) nil slices
(3.) empty slice occupancy: `len(s) == 0 && s != nil`
(4.) partial slice occupancy: `len(s) < cap(s) && s != nil`
(5.) full slice occupancy: `len(s) == cap(s) && s != nil`
Prior to this, only #5 was ever generated, thereby not sufficiently
exercising all of the fuzzable code path outcomes.
This change depends on https://go-review.googlesource.com/#/c/17499/.
Change-Id: I9343c475cefbd72ffc5237281826465c25872206
Reviewed-on: https://go-review.googlesource.com/16470
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This was supposed to be in CL 18204 but I submitted from the web
instead of my computer and lost this final edit.
Change-Id: I41598e936bb088d77f5e44752eda74222a4208c7
Reviewed-on: https://go-review.googlesource.com/18310
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Before, we reset the timer at the end of T.Parallel, which is okay
assuming that T.Parallel is the first thing in the test.
Snapshot the elapsed time at the beginning of Parallel and include it in
the total duration so that any time spent in the test before calling
Parallel is reported in the test duration as well.
Updates #12243.
Change-Id: Ieca553e1f801e16b9b6416463fa8f7fa65425185
Reviewed-on: https://go-review.googlesource.com/16989
Reviewed-by: Russ Cox <rsc@golang.org>
Calling flag.Parse twice can be problematic if other goroutines called
flag.Parsed in between: the race detector complains due to the
write after read from a different goroutine.
This can happen if TestMain calls flag.Parse and launches goroutines
that call flag.Parsed, for example if it initializes a server which
checks flags.
This patch makes testing.M.Run only parse the flags if they have not
been parsed already.
Change-Id: Id9f8c31c5f90614e3f34c63d1a32cf7e9055d68e
Reviewed-on: https://go-review.googlesource.com/16739
Reviewed-by: Russ Cox <rsc@golang.org>
Recursive types R containing slices of R's did not terminate despite the
effort in CL 10821.
For recursive types there was a competition between slice expansion by a
factor 'complexSize', and termination with probability '1/complexSize'
which lead to stack overflow as soon as a recursive struct had slices
pointing to its own type.
Fix this by shrinking the size hint as a function of recursion depth.
This has the dual effect of reducing the number of elements generated
per slice and also increasing the probability for termination.
Fixes#11148.
Change-Id: Ib61155b4f2e2de3873d508d63a1f4be759426d67
Reviewed-on: https://go-review.googlesource.com/13830
Reviewed-by: Adam Langley <agl@golang.org>
Move tracing functions from runtime/pprof to the new runtime/trace package.
Fixes#9710
Change-Id: I718bcb2ae3e5959d9f72cab5e6708289e5c8ebd5
Reviewed-on: https://go-review.googlesource.com/12511
Reviewed-by: Russ Cox <rsc@golang.org>
The one in misc/makerelease/makerelease.go is particularly bad and
probably warrants rotating our keys.
I didn't update old weekly notes, and reverted some changes involving
test code for now, since we're late in the Go 1.5 freeze. Otherwise,
the rest are all auto-generated changes, and all manually reviewed.
Change-Id: Ia2753576ab5d64826a167d259f48a2f50508792d
Reviewed-on: https://go-review.googlesource.com/12048
Reviewed-by: Rob Pike <r@golang.org>
This commit fixes a cosmetic defect whereby quick.Check reports that
the provided function returns too many values when it may, in fact,
return too few:
func f() {}
func TestFoo(t *testing.T) {
if err := quick.Check(f, nil); err != nil {
t.Fatal(err)
}
}
// yields
// $ go test -v foo_test.go
// === RUN TestFoo
// --- FAIL: TestFoo (0.00s)
// foo_test.go:76: function returns more than one value.
Change-Id: Ia209ff5b57375b30f8db425454e80798908e8ff4
Reviewed-on: https://go-review.googlesource.com/11281
Reviewed-by: Russ Cox <rsc@golang.org>
The number of CPUs is of value when benchmarking but mostly
noise when testing. The recent change to default to the number
of CPUs available has made the tests noisier and confusing.
Fixes#11200
Change-Id: Ifc87d9ccb4177d73e304fb7ffcef4367bd163c9e
Reviewed-on: https://go-review.googlesource.com/11121
Reviewed-by: Russ Cox <rsc@golang.org>
The documentation for quick.Value says that it "returns an arbitrary
value of the given type." In spite of this, nil values for pointers were
never generated, which seems more like an oversight than an intentional
choice.
The lack of nil values meant that testing recursive type like
type Node struct {
Next *Node
}
with testing/quick would lead to a stack overflow since the data
structure would never terminate.
This change may break tests that don't check for nil with pointers
returned from quick.Value. Two such instances were found in the standard
library, one of which was in the testing/quick package itself.
Fixes#8818.
Change-Id: Id390dcce649d12fbbaa801ce6f58f5defed77e60
Reviewed-on: https://go-review.googlesource.com/10821
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
The flag is available from the go test command as -count:
% go test -run XXX -bench . -count 3
PASS
BenchmarkSprintfEmpty 30000000 54.0 ns/op
BenchmarkSprintfEmpty 30000000 51.9 ns/op
BenchmarkSprintfEmpty 30000000 53.8 ns/op
BenchmarkSprintfString 10000000 238 ns/op
BenchmarkSprintfString 10000000 239 ns/op
BenchmarkSprintfString 10000000 234 ns/op
BenchmarkSprintfInt 10000000 232 ns/op
BenchmarkSprintfInt 10000000 226 ns/op
BenchmarkSprintfInt 10000000 225 ns/op
...
If -cpu is set, each test is run n times for each cpu value.
Original by r (CL 10663).
Change-Id: If3dfbdf21698952daac9249b5dbca66f5301e91b
Reviewed-on: https://go-review.googlesource.com/10669
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Delete the colon from RUN: for examples, since it's not there for tests.
Add spaces to line up RUN and PASS: lines.
Before:
=== RUN TestCount
--- PASS: TestCount (0.00s)
=== RUN: ExampleFields
--- PASS: ExampleFields (0.00s)
After:
=== RUN TestCount
--- PASS: TestCount (0.00s)
=== RUN ExampleFields
--- PASS: ExampleFields (0.00s)
Fixes#10594.
Change-Id: I189c80a5d99101ee72d8c9c3a4639c07e640cbd8
Reviewed-on: https://go-review.googlesource.com/9846
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit is largely cosmetic in the sense that it is the remnants
of a change proposal I had prepared for testing/quick, until I
discovered that 3e9ed27 already implemented the feature I was looking
for: quick.Value() for reflect.Kind Array. What you see is a merger
and manual cleanup; the cosmetic cleanups are as follows:
(1.) Keeping the TestCheckEqual and its associated input functions
in the same order as type kinds defined in reflect.Kind. Since
3e9ed27 was committed, the test case began to diverge from the
constant's ordering.
(2.) The `Intptr` derivatives existed to exercise quick.Value with
reflect.Kind's `Ptr` constant. All `Intptr` (unrelated to `uintptr`)
in the test have been migrated to ensure the parallelism of the
listings and to convey that `Intptr` is not special.
(3.) Correct a misspelling (transposition) of "alias", whereby it is
named as "Alais".
Change-Id: I441450db16b8bb1272c52b0abcda3794dcd0599d
Reviewed-on: https://go-review.googlesource.com/8804
Reviewed-by: Russ Cox <rsc@golang.org>
There is no top-level Values function.
Change-Id: I3ea2eea0b5f77f3e1a3f75d1a6472507ef2888bb
Reviewed-on: https://go-review.googlesource.com/8196
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
