Check for and reject environment variables containing NULs.
The conventions for passing environment variables to subprocesses
cause most or all systems to interpret a NUL as a separator. The
syscall package rejects environment variables containing a NUL
on most systems, but erroniously did not do so on Windows. This
causes an environment variable such as "FOO=a\x00BAR=b" to be
interpreted as "FOO=a", "BAR=b".
Check for and reject NULs in environment variables passed to
syscall.StartProcess on Windows.
Add a redundant check to os/exec as extra insurance.
Fixes#56284
Fixes CVE-2022-41716
Change-Id: I2950e2b0cb14ebd26e5629be1521858f66a7d4ae
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1609434
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
TryBot-Result: Security TryBots <security-trybots@go-security-trybots.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/446916
Reviewed-by: Tatiana Bradley <tatiana@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Track the running tests and when they started,
so that we can report the running tests on a test timeout.
% go test -timeout=5s
panic: test timed out after 5s
running tests:
TestTCPSpuriousConnSetupCompletion (4s)
... stack traces as usual ...
% go test -run=Script -timeout=10s cmd/go
vcs-test.golang.org rerouted to http://127.0.0.1:65168https://vcs-test.golang.org rerouted to https://127.0.0.1:65169
go test proxy running at GOPROXY=http://127.0.0.1:65170/mod
panic: test timed out after 10s
running tests:
TestScript (10s)
TestScript/mod_get_patchcycle (0s)
TestScript/mod_get_prefer_incompatible (0s)
TestScript/mod_get_promote_implicit (0s)
TestScript/mod_get_pseudo (0s)
TestScript/mod_get_pseudo_other_branch (0s)
TestScript/mod_get_pseudo_prefix (0s)
TestScript/mod_get_test (0s)
TestScript/mod_get_trailing_slash (0s)
TestScript/mod_get_update_unrelated_sum (0s)
TestScript/mod_gobuild_import (0s)
TestScript/mod_gomodcache (0s)
TestScript/mod_gonoproxy (0s)
TestScript/mod_load_badchain (0s)
TestScript/mod_overlay (0s)
TestScript/test_fuzz_minimize (6s)
TestScript/test_fuzz_minimize_dirty_cov (7s)
... stack traces as usual ...
Change-Id: I3a6647c029097becc06664ebd76a2597c7ed7b8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/446176
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
Use a slightly different line number pragma when emitting instrumented
code, so as to ensure that we don't get any changes in the
"-gcflags=-m" output for coverage vs non-coverage.
Fixes#56475.
Change-Id: I3079171fdf83c0434ed6ea0ce3eb2797c2280c55
Reviewed-on: https://go-review.googlesource.com/c/go/+/446259
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Change the 'PrepareForCoverageBuild' helper function to provide more
sensible defaults in the case where Go packages are listed on the
command line (e.g. cases such as "go run -cover mumble.go").
With the old implementation, if module mode was enabled, we would only
instrument packages in the main module, meaning that if you did
something like this:
$ ls go.mod
go.mod
$ GOCOVERDATA=/tmp/cov go run -cover testdata/somefile.go
$
no coverage profiles would be generated at all. This is due to the
fact that the pseudo-package created by the Go command internally when
building "somefile.go" is not considered part of the main module.
This patch moves the default to "packages explicitly mentioned on the
command line, plus packages in the main module", which will make more
sense to users passing specific packages and *.go files on the command
line. Examples:
// Here cmd/compile is part the Go standard library + commands
// (which we exclude from instrumentation by default), but since
// 'cmd/compile' is mentioned on the command line, we will instrument
// just that single package (not any of its deps).
$ cd $GOROOT/src
$ go build -o gc.exe -cover cmd/compile
$ GOCOVERDATA=/tmp/cov ./gc.exe ...
...
$
// Here we're running a Go file named on the command line, where
// the pseudo-package for the command line is not part of the
// main module, but it makes sense to instrument it anyhow.
$ cd ~/go/k8s.io/kubernetes
$ GOCOVERDATA=/tmp/cov go run -cover test/typecheck/testdata/bad/bad.go
...
$
This patch also simplifies the logic and improves flow/comments in
in the helper function PrepareForCoverageBuild.
Change-Id: Id8fc8571157dac8c09e44cc73baa05aeba1640ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/445918
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
This patch fixes a typo/bug introduced in CL 441858 where when pattern
matching a coverage counter access we were looking at an assingment
node instead of the assignment LHS, and fixes a similar problem in
atomic counter update pattern matching introduced in CL 444835. In
both of these cases the bug was not caught because the test intended
to lock down the behavior was written incorrectly (wasn't
instrumenting what the test author thought it was instrumenting,
ouch).
Change-Id: I6e6ac3beacf12ef1a817de5527340b639f0bb044
Reviewed-on: https://go-review.googlesource.com/c/go/+/446258
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
When coverage testing a local package (defined by a relative import
path such as "./foo/bar") the convention when "-coverprofile" is used
has been to capture source files by full pathname, as opposed to
recording the full import path or the invented import path
("command-line-arguments/") created by the go command in the case of
building named Go files. Doing this makes it much easier to use
collected profiles with "go tool -cover -html=<profile>".
The support for this feature/convention wound up being inadvertantly
dropped during the GOEXPERIMENT=coverageredesign implementation; this
patch restores it.
Fixes#56433.
Change-Id: Ib9556fdc86011b00c155caa614ab23e5148f3eb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/445917
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Change the macOS implementation to use libc calls.
Using libc calls directly is what we do for all the runtime and os syscalls.
Doing so here as well improves consistency and also makes it possible
to cross-compile (from non-Mac systems) macOS binaries that use the
native name resolver.
Fixes#12524.
Change-Id: I011f4fcc5c50fbb5396e494889765dcbb9342336
Reviewed-on: https://go-review.googlesource.com/c/go/+/446178
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Also removes no-longer-needed "Any" field from compiler's DebugFlags.
Test/use case for this is the fmahash CL.
Change-Id: I214f02c91f30fc2ce53caf75fa5e2b905dd33429
Reviewed-on: https://go-review.googlesource.com/c/go/+/445495
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
This an end-to-end test that sets GOROOT to a symlink
to the distribution, approximating copying it to a new
location, and checks that packages in the standard library
are not stale, as they would be if paths were embedded
in artifacts.
For #47257
Change-Id: I4ed04df36656ad946a2f6f5ce3194e76e06372e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/445358
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
We kind of have this mechanism already, just normalizing it and
using it in a bunch of places. Previously a bunch of places cached
slices only for the duration of a single function compilation. Now
we can reuse slices across a whole compiler run.
Use a sync.Pool of powers-of-two sizes. This lets us use not
too much memory, and avoid holding onto memory we're no longer
using when a GC happens.
There's a few different types we need, so generate the code for it.
Generics would be useful here, but we can't use generics in the
compiler because of bootstrapping.
Change-Id: I6cf37e7b7b2e802882aaa723a0b29770511ccd82
Reviewed-on: https://go-review.googlesource.com/c/go/+/444820
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Make them a separate type, so the normal sparse maps don't
need the extra storage.
Change-Id: I3a0219487c35ea63723499723b0c742e321d97c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/444819
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
The recently added rule only works before decomposing slices.
Add a rule that works after decomposing slices.
The reason we need the latter is because although the length may
be a constant, it can be hidden inside a slice that is not constant
(its pointer or capacity might be changing). By applying this
optimization after decomposing slices, we can find more cases
where it applies.
Fixes#56440
Change-Id: I0094e59eee3065ab4d210defdda8227a6e897420
Reviewed-on: https://go-review.googlesource.com/c/go/+/446277
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Since pgo is a new package, it is reasonably straightforward to
encapsulate its state into a non-global object that we pass around,
which will help keep it isolated.
There are no functional changes in this CL, just packaging up the
globals into a new object.
There are two major pieces of cleanup remaining:
1. reflectdata and noder have separate InlineCalls calls for method
wrappers. The Profile is not plumbed there yet, but this is not a
regression as the globals were previously set only right around the
main inlining pass in gc.Main.
2. pgo.ListOfHotCallSites is still global, as it will require more work
to clean up. It is effectively a local variable in InlinePackage,
except that it assumes that InlineCalls is immediately preceded by a
CanInline call for the same function. This is not necessarily true
due to the recursive nature of CanInline. This also means that some
InlineCalls calls may be missing the list of hot callsites right now.
For #55022.
Change-Id: Ic1fe41f73df96861c65f8bfeecff89862b367290
Reviewed-on: https://go-review.googlesource.com/c/go/+/446303
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
The test added in CL 446115 was failing on Plan 9, on which the error
returned by a failed open includes the path that failed.
Change-Id: If27222596c3cb0366a030bb49ae41c4c869c3db2
Reviewed-on: https://go-review.googlesource.com/c/go/+/446641
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Parts of package pgo fetch the line number of a node by parsing the
number out of the string returned from ir.Line().
This is indirect and inefficient, so it should be replaced with a more
direct lookup. It is also potentially buggy: ir.Line uses
ctxt.OutermostPos, i.e., the line number where an inlined node in
inlined. We want ctxt.InnermostPos, because that is the line number used
in pprof profiles that we are matching against (See comments on
OutermostPos and InnermostPos).
I'm not sure whether this was an active, as we use ir.Line before and
during inlining. I think we could see CALL nodes with OutermostPos !=
InnermostPos during midstack inlining, but I am not sure. Regardless,
explicitly using the desired position is clearer.
For #55022.
Change-Id: Ic640761c9e1d01cacbf91f3aaeaf284ad7e38dbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/446302
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
For most tests, the test's deadline itself is more appropriate than an
arbitrary timeout layered atop of it (especially once #48157 is
implemented), and testenv.Command already adds cleaner timeout
behavior when a command would run too close to the test's deadline.
That makes RunWithTimeout something of an attractive nuisance. For
now, migrate the two existing uses of it to testenv.CommandContext,
with a shorter timeout implemented using context.WithTimeout.
As a followup, we may want to drop the extra timeouts from these
invocations entirely.
Updates #50436.
Updates #37405.
Change-Id: I16840fd36c0137b6da87ec54012b3e44661f0d08
Reviewed-on: https://go-review.googlesource.com/c/go/+/445597
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This adds a testenv.CommandContext function, with timeout behavior
based on the existing logic in cmd/go.TestScript: namely, the command
is terminated with SIGQUIT (if supported) with an arbitrary grace
period remaining until the test's deadline.
If the test environment does not support executing subprocesses,
CommandContext skips the test.
If the command is terminated due to the timout expiring or the test
fails to wait for the command after starting it, CommandContext marks
the test as failing.
For tests where a shorter timeout is desired (such as for fail-fast
behavior), one may be supplied using context.WithTimeout.
The more concise Command helper is like CommandContext but without
the need to supply an explicit Context.
Updates #50436.
Change-Id: Ifd81fb86c402f034063c9e9c03045b4106eab81a
Reviewed-on: https://go-review.googlesource.com/c/go/+/445596
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Make linkgetlineFromPos and getFileIndexAndLine methods on Link, and
give the former a more descriptive name.
The docs are expanded to make it more clear that these are final
file/line visible in programs.
In getFileSymbolAndLine use ctxt.InnermostPos instead of ctxt.PosTable
direct, which makes it more clear that we want the semantics of
InnermostPos.
Change-Id: I7c3d344dec60407fa54b191be8a09c117cb87dd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/446301
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The most important change here is to log output from the child, making
it easier to diagnose problems when the child 'go test' fails.
We can also eliminate the cmd.Wait goroutine by using an os.Pipe, whose
reader will return io.EOF when the child exits.
For #55022.
Change-Id: I1573ea444407d545bdca8525c9ff7b0a2baebf5e
Reviewed-on: https://go-review.googlesource.com/c/go/+/446300
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The test sleeps for 1 millisecond to give the cancellation a moment
to take effect. This is flaky because the request can finish before
the cancellation of the context is seen. It's easy to verify by adding
time.Sleep(2*time.Millisecond)
after 0a6c4c8740/src/net/http/transport.go (L2619).
With this modification, the test fails about 5 times out of 10 runs.
The fix is easy. We just need to block the handler of the second
request until this request is cancelled. I have verify that the
updated test can uncover the issue fixed by CL 257818.
Fixes#55226.
Change-Id: I81575beef1a920a2ffaa5c6a5ca70a4008bd5f94
GitHub-Last-Rev: 99cb1c2eae
GitHub-Pull-Request: golang/go#56500
Reviewed-on: https://go-review.googlesource.com/c/go/+/446676
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Wasm can't handle the recusion for XML nested to depth 10,000.
Cut it off at 5,000 instead. This fixes TestCVE202228131 on trybots
in certain conditions.
Also disable TestCVE202230633 to fix 'go test -v encoding/xml' on gomotes.
Also rename errExeceededMaxUnmarshalDepth [misspelled and unwieldy]
to errUnmarshalDepth.
For #56498.
Change-Id: I7cc337ccfee251bfd9771497be0e5272737114f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/446639
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This one slipped through a previous cleanup
and showed up in an allocation profile.
Change-Id: Ife447fe32797f1685c20806718193b0985b4a412
Reviewed-on: https://go-review.googlesource.com/c/go/+/446295
Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This may fix the android-.*-emu builders, which are currently broken.
For #47257.
Change-Id: I370dad2cb8031f8f5fdfbeb9c284c4f79f58d929
Reviewed-on: https://go-review.googlesource.com/c/go/+/446638
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
The js/wasm builder started failing as of CL 432535 due to needing
'go build' to import standard-library packages that are no longer
installed to GOROOT/pkg. Since js/wasm can't exec subprocesses,
it cannot run 'go build' to generate the export data needed for
these tests.
For #47257.
Change-Id: I804235fd725faf00b27cbed79ee1f43dea8ab734
Reviewed-on: https://go-review.googlesource.com/c/go/+/446635
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
The index function was not handling certain corner cases where there
were two more bytes to be examined in the tail end of the string to
complete the comparison. Fix code to ensure that when the string has
to be shifted two more times the correct bytes are examined.
Also hoisted vsplat to V10 so that all paths use the correct value.
Some comments had incorrect register names and corrected the same.
Added the strings that were failing to strings test for verification.
Fixes#56457
Change-Id: Idba7cbc802e3d73c8f4fe89309871cc8447792f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/446135
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Archana Ravindar <ravindararchana@gmail.com>
Address folding is disabled in CL42172, the commit message of which
said that "In shared library, load/store of global is rewritten to
using GOT and temp register, which conflicts with the use of temp
register for assembling large offset.". Actually this doesn't happen
because the sequence of instructions when rewritten to use Got looks
like this:
MOVD $sym, Rx becomes
MOVD sym@GOT, Rx
If there is an offset off, there will be one more instruction:
ADD $off, Rx, Rx
And MOVD sym, Rx becomes
MOVD sym@GOT, REGTMP
MOVx (REGTMP), Ry
If there is a small offset off, it becomes:
MOVD sym@GOT, REGTMP
MOVx (REGTMP)off, Ry
If off is very large, it becomes:
MOVD sym@GOT, REGTMP
MOVD $off, Rt
ADD Rt, REGTMP
MOVx (REGTMP), Ry
We can see that the address can be calculated correctly, and testing
on darwin/arm64 confirms this.
Removing this restriction is beneficial to further optimize the sequence
of "ADRP+ADD+LD/ST" to "ADRP+LD/ST(offset), so this CL removes it.
Change-Id: I0e9f7bc1723e0a027f32cf0ae2c41cd6df49defe
Reviewed-on: https://go-review.googlesource.com/c/go/+/445535
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Eric Fang <eric.fang@arm.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Set the value of the variable tailCall to true and prevent
allocating or clobber the linker register.
Change-Id: I4ec19c67056cb99196911aa7c0054be89ab7eb8d
Reviewed-on: https://go-review.googlesource.com/c/go/+/414954
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: WANG Xuerui <git@xen0n.name>
Test whether gccgo/GoLLVM supports cgo.Incomplete. If it doesn't, use a
local definition rather than importing it.
Roll back 426496, which skipped a gccgo test, as it now works.
For #46731Fixes#54761
Change-Id: I8bb2ad84c317094495405e178bf5c9694f82af56
Reviewed-on: https://go-review.googlesource.com/c/go/+/446260
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This change improves the performance of Binomial by implementing an
algorithm that produces smaller intermediate values at each step.
Working with smaller big.Int values has the advantage that fewer allocations
and computations are required for each mathematical operation.
The algorithm used is the Multiplicative Formula, which is a well known
way of calculating the Binomial coefficient and is described at:
https://en.wikipedia.org/wiki/Binomial_coefficient#Multiplicative_formulahttps://en.wikipedia.org/wiki/Binomial_coefficient#In_programming_languages
In addition to that, an optimization has been made to remove a
redundant computation of (i+1) on each loop which has a measurable
impact when using big.Int.
Performance improvement measured on an M1 MacBook Pro
running the existing benchmark for Binomial:
name old time/op new time/op delta
Binomial-8 589ns ± 0% 435ns ± 0% -26.05% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
Binomial-8 1.02kB ± 0% 0.08kB ± 0% -92.19% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
Binomial-8 38.0 ± 0% 5.0 ± 0% -86.84% (p=0.000 n=10+10)
Change-Id: I5a830386dd42f062e17af88411dd74fcb110ded9
GitHub-Last-Rev: 6b2fca07de
GitHub-Pull-Request: golang/go#56339
Reviewed-on: https://go-review.googlesource.com/c/go/+/444315
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Packages in GOROOT that don't use cgo will not be installed in
GOROOT/pkg, and will instead be cached as usual like other Go
packages.
- add a internal/buildinternal package to hold the identities of the
five packages that use cgo
- update dist's test code to do a go build std cmd before checking
staleness on builders. Because most of those packages no longer have
install locations, and have dependencies that don't either, the
packages need to be cached to not be stale.
- fix index_test to import packages with the path "." when preparing
the "want" values to compare the indexed data to. (the module index
matches the behavior of build.ImportDir, which always passes in "."
as the path.
- In both the index and go/build Importers, don't set
PkgObj for GOROOT packages which will no longer have install
targets. PkgTargetRoot will still be set to compute target paths,
which will still be needed in buildmode=shared.
- "downgrade" all install actions that don't have a target to build
actions. (The target should already not be set for packages that
shouldn't be installed).
For #47257
Change-Id: Ia5aee6b3b20b58e028119cf0352a4c4a2f10f6b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/432535
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Fixes#56479
Change-Id: I1d97eb3ea97304e429e178ad05cb9f861b2bce84
Reviewed-on: https://go-review.googlesource.com/c/go/+/446275
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Fixes the compiler to emit DIEs for zero sized variables.
Fixes#54615
Change-Id: I1e0c86a97f1abcc7edae516b6a7fe35bcb65ed0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/433479
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
When writing the set of commands excuted for go build -x, $WORK is
substituted for the work directory in all the commnands. But this
includes the cat <<EOF commands used to create a file with the given
contents. While we can expect the shell to substitute $WORK properly,
commands that read input files, such as importcfgs won't do that
substitution.
This is necessary to fix the build_dash_x script test for CL 432535
because it removes .a files from the traditional stdlib install
locations. The test can pass even with importcfg packagefiles in $WORK
because all transitive imports are in the stdlib, and the compiler can
fall back to finding stdlib .a files in their traditional places, but
once they're gone the packagefile paths in $WORK will have paths that
contain the string $WORK, and os.Open will fail to open them for
reading. And since the fallback is gone the test will fail.
For #47257
Change-Id: I5db0066de6ed3ccf97927a78ce0939e3eb14aebe
Reviewed-on: https://go-review.googlesource.com/c/go/+/446116
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Before this change, the .a files for the intermediate go toolchains
were produced in the same location as the install target. This change
has them produced in a temporary location instead. This change,
combined with go install not producing .a files for most stdlib
packages by default results in a build of the distribution only
producing .a's for the five packages in std that require cgo. (Before
this change, the .a's for the final build were not being produced, but
stale ones from the intermediate builds were left behind.)
For #47257
Change-Id: I91b826cd1ce9aad9c492fb865e36d34dc8bb188e
Reviewed-on: https://go-review.googlesource.com/c/go/+/436135
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Fixes a performance regression due to CL 418554.
Fixes#56440
Change-Id: I6ff152e9b83084756363f49ee6b0844a7a284880
Reviewed-on: https://go-review.googlesource.com/c/go/+/445875
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
In the previous, we will only use the slicing by 8 look up table when
the data size is greater than 16k, but the threshold seems to be too
large. We may lose some performance for the small data size.
In this CL, we change the threshold to 2k, it shows that the performance
is improved greatly when the data size is 2k ~ 16k.
We did some tests for the Random2K ~ Random16K between various
cores(mostly on x86 and arm architecture). Here are the benchmark and
some results:
1. The benchmark for testing:
func BenchmarkRandom(b *testing.B) {
poly := 0x58993511
b.Run("Random256", func(b *testing.B) {
bench(b, uint64(poly), 256)
})
b.Run("Random512", func(b *testing.B) {
bench(b, uint64(poly), 512)
})
b.Run("Random1KB", func(b *testing.B) {
bench(b, uint64(poly), 1<<10)
})
b.Run("Random2KB", func(b *testing.B) {
bench(b, uint64(poly), 2<<10)
})
b.Run("Random4KB", func(b *testing.B) {
bench(b, uint64(poly), 4<<10)
})
b.Run("Random8KB", func(b *testing.B) {
bench(b, uint64(poly), 8<<10)
})
b.Run("Random16KB", func(b *testing.B) {
bench(b, uint64(poly), 16<<10)
})
}
2. Some results:
Apple silicon M1:
Benchmark old new delta
Random/Random2KB-10 362MB/s 801MB/s +121.41%
Random/Random4KB-10 360MB/s 1083MB/s +200.93%
Random/Random8KB-10 359MB/s 1309MB/s +264.88%
Random/Random16KB-10 358MB/s 1466MB/s +309.79%
Neoverse N1:
Benchmark old new delta
Random/Random2KB-160 397MB/s 493MB/s +24.23%
Random/Random4KB-160 397MB/s 742MB/s +86.86%
Random/Random8KB-160 398MB/s 995MB/s +150.12%
Random/Random16KB-160 398MB/s 1196MB/s +200.58%
Silver 4116:
Benchmark old new delta
Random/Random2KB-48 252MB/s 418MB/s +65.79%
Random/Random4KB-48 253MB/s 621MB/s +145.72%
Random/Random8KB-48 254MB/s 796MB/s +213.07%
Random/Random16KB-48 258MB/s 929MB/s +260.46%
EPYC 7251:
Benchmark old new delta
Random/Random2KB-32 255MB/s 380MB/s +48.88%
Random/Random4KB-32 255MB/s 561MB/s +119.73%
Random/Random8KB-32 255MB/s 738MB/s +189.18%
Random/Random16KB-32 255MB/s 877MB/s +243.80%
Change-Id: Ib7b4f6826c3edd6f315cac8057d52b6da252a652
Reviewed-on: https://go-review.googlesource.com/c/go/+/445475
Run-TryBot: Eric Fang <eric.fang@arm.com>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
The existing implementation uses loops to implement bulk memory
operations such as memcpy and memclr. Now that bulk memory operations
have been standardized and are implemented in all major browsers and
engines (see https://webassembly.org/roadmap/), we should use them
to improve performance.
Updates #28360
Change-Id: I28df0e0350287d5e7e1d1c09a4064ea1054e7575
Reviewed-on: https://go-review.googlesource.com/c/go/+/444935
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Richard Musiol <neelance@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Richard Musiol <neelance@gmail.com>
Reviewed-by: Richard Musiol <neelance@gmail.com>
For #20661.
Change-Id: I1e638cb619e643eadc210d71f92bd1af7bafc912
Reviewed-on: https://go-review.googlesource.com/c/go/+/436955
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: hopehook <hopehook@golangcn.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
The instruction format of MOVBU is the same with MOVB, this CL deletes
MOVBU from optab for simplicity.
Change-Id: Ib034d6c29dd9793cf3e6f9fa8abff0ed0d931d0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/445295
Run-TryBot: Eric Fang <eric.fang@arm.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The size of gnuVersym should be multiples of 2. If not, the input is
invalid. No Library and Version information is added to sym in this
case. The current implementation of gnuVersion does not report errors
for invalid input.
While at here, bring back the comment that states that the undef entry
at the beginning is skipped. This is not an off-by-one error.
No test case because the problem can only happen for invalid data. Let
the fuzzer find cases like this.
Fixes#56429.
Change-Id: Ia39ad8bd509088a81cc77f7a76e23185d40a5765
GitHub-Last-Rev: 3be0cc1b15
GitHub-Pull-Request: golang/go#56431
Reviewed-on: https://go-review.googlesource.com/c/go/+/445555
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
(Fixing the most important part of this bug.)
Updates #56426
Change-Id: If657ae47a5fe7dacc31d2c487e53e9f2dd5d03bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/445695
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Auto-Submit: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Test2json is parsing the output stream from the test, which includes
package testing's own framing lines intermingled with other output,
in particular any output printed via fmt.Printf, println, and so on.
We have had recurring problems with unexpected partial output lines
causing a framing line to be missed.
A recent talk at GopherCon gave an example of an integration test
involving Docker that happened to print \r-terminated lines instead
of \n-terminated lines in some configurations, which in turn broke
test2json badly. (https://www.gophercon.com/agenda/session/944259)
There are also a variety of open reported issues with similar problems,
which this CL also addresses. The general approach is to add a new
testing flag -test.v=json that means to print additional output to help
test2json. And then test2json takes advantage of that output.
Among the fixes:
- Identify testing framing more reliably, using ^V
(#23036, #26325, #43683, GopherCon talk)
- Test that output with \r\n endings is handled correctly
(#43683, #34286)
- Use === RUN in fuzz tests (#52636, #48132)
- Add === RUN lines to note benchmark starts (#27764, #49505)
- Print subtest --- PASS/FAIL lines as they happen (#29811)
- Add === NAME lines to emit more test change events,
such as when a subtest stops and the parent continues running.
- Fix event shown in overall test failure (#27568)
- Avoid interleaving of writes to os.Stdout and os.Stderr (#33419)
Fixes#23036.
Fixes#26325.
Fixes#27568.
Fixes#27764.
Fixes#29811.
Fixes#33419.
Fixes#34286.
Fixes#43683.
Fixes#49505.
Fixes#52636.
Change-Id: Id4207b746a20693f92e52d68c6e4a7f8c41cc7c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/443596
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
When using "MOVD $const, Rx", any 32b constant can be generated in
register quickly. Avoid transforming big uint32 values into a load.
And, fix the instance in runtime.usleep where I discovered this.
Change-Id: I46e156d7edf200f85b5b61162f00223c0ad81fe2
Reviewed-on: https://go-review.googlesource.com/c/go/+/444815
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
i.e.
from "function %!s(*Node=0xc0003b48c0) cannot have ABI wrappers", to "function xxFunctionName cannot have ABI wrappers".
Change-Id: I83cfdf1916e82ab1455db8032153d9cdae85250d
GitHub-Last-Rev: 87b077653f
GitHub-Pull-Request: golang/go#56428
Reviewed-on: https://go-review.googlesource.com/c/go/+/445516
Auto-Submit: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
