If you have multiple runs in old.txt and new.txt
the default behavior is to match them up pairwise
and compare successive pairs (and if you have a
different number of runs in each file, benchcmp
refuses to do anything).
The new -best flag changes the behavior to instead
compare the fastest run of each benchmark from
the two files. This makes sense if you believe that
the fastest speed is the 'actual' speed and the slower
results are due to the computer spending time doing
non-benchmark work while the benchmark was
running.
LGTM=josharian
R=golang-codereviews, josharian
CC=golang-codereviews
https://golang.org/cl/102890047
benchcmp now preserves benchmark order. This restores the original
misc/benchcmp behavior. This also makes the output of benchcmp stable,
and groups together multiple -cpu results.
Magnitude-based sorting is still available via the -mag flag.
It is useful for surfacing items of note (particularly changes
in allocs) when making compiler changes and running broad
benchmarks.
Fixesgolang/go#7259.
LGTM=dave
R=dave, mtj
CC=bradfitz, dvyukov, golang-codereviews
https://golang.org/cl/60840045
This is intended to replace the awk-based misc/benchcmp.
It mostly matches the existing misc/benchcmp.
Notable changes:
* Written in Go.
* Minor whitespace changes in the output; the tabular nature of the
output is preserved, as is most number formatting and verbiage.
* Comparisons within each section are sorted from highest change to lowest.
* Proper handling of multiple benchmarks with the same name (issue 7016).
* Does not omit benchmark comparisons for which the new value is zero.
* Adds -changed flag to only show benchmarks whose value have changed.
Useful for memory-oriented, large-scale benchmark comparisons.
* Has tests.
* Formats small ns measurements with extra precision.
Updates golang/go#7016.
LGTM=r
R=golang-codereviews, dave, dvyukov, oleku.konko, bradfitz, gobot, r
CC=golang-codereviews
https://golang.org/cl/47980043
