1
0
mirror of https://github.com/golang/go synced 2024-11-17 16:44:44 -07:00
Commit Graph

54581 Commits

Author SHA1 Message Date
Russ Cox
0aeda5afe5 cmd/link: remove unnecessary use of sync.Once alongside sync.Mutex
There does not seem to be any point to this sync.Once.
I noticed because I was surveying uses of sync.Once to
understand usage patterns. This seems to be a dreg left over
from some earlier instance of the code.

Change-Id: I99dd258d865a41d0e8f6cfa55887855e477fb9c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/445755
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-02 14:38:08 +00:00
Russ Cox
30b240b102 testing: implement -cpu and -count for fuzz tests
Fuzz tests are meant to be run just like ordinary tests,
so copy the same loop cpu and count loops used in testing.go
(and benchmark.go) into fuzz.go.

Change-Id: Ic585df8ccc577869c877b1055e0493803dbeb828
Reviewed-on: https://go-review.googlesource.com/c/go/+/443377
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
2022-11-02 13:14:08 +00:00
Mateusz Poliwczak
81efd7b347 net: support no-reload option for unix go resolver
It adds support for no-reload option, as specified in resolv.conf(5):
 no-reload (since glibc 2.26)
                     Sets RES_NORELOAD in _res.options.  This option
                     disables automatic reloading of a changed
                     configuration file.

Change-Id: I11182c5829434503f719ed162014f2301e3ba8d4
GitHub-Last-Rev: 7ae44be2d5
GitHub-Pull-Request: golang/go#56489
Reviewed-on: https://go-review.googlesource.com/c/go/+/446555
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-11-01 22:44:57 +00:00
Ian Lance Taylor
e23876a383 net: drop unused _C_ai_addrlen function
Fixes AIX build.

Change-Id: Icbb33896017bbcc488a8baff20e10eb0e14ea4b9
Reviewed-on: https://go-review.googlesource.com/c/go/+/447095
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-11-01 22:15:41 +00:00
Michael Pratt
d73885588a cmd/compile/internal/pgo: remove ListOfHotCallSites
The global ListOfHotCallSites set is used to communicate between
CanInline and InlineCalls the set of call sites that InlineCalls may
increase the budget for.

CanInline clears this map on each call, thus assuming that
InlineCalls(x) is called immediately after CanInline(x). This assumption
is false, as CanInline (among other cases) is recursive (CanInline ->
hairyVisitor.doNode -> inlCallee -> CanInline).

When this assumption proves false, we will lose the opportunity to
inline hot calls.

This CL is the least invasive fix for this. ListOfHotCallSites is
actually just a subset of the candHotEdgeMap, with CallSiteInfo.Callee
cleared. candHotEdgeMap doesn't actually need to distinguish based on
Callee, so we can drop callee from candHotEdgeMap as well and just use
that directly [1].

Later CLs should do more work to remove the globals entirely.

For cmd/compile, this inceases the number of PGO inlined functions by
~50% for one set of PGO parameters. I have no evaluated performance
impact.

[1] This is something that we likely want to change in the future.

For #55022.

Change-Id: I57735958d651f6dfa9bd296499841213d20e1706
Reviewed-on: https://go-review.googlesource.com/c/go/+/446755
Auto-Submit: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-01 22:13:50 +00:00
Bryan C. Mills
1ae93e4c20 cmd/go/internal/vcweb: increase script WaitDelay by 30x
This should reduce spurious VCS failures on slow builders, like the
one observed in
https://build.golang.org/log/e773fe404b2009d67fa34f048e023f0a86663a13

Updates #27494.

Change-Id: Ibb094c8ddf79e0ab481c00fcf501dc955b0da787
Reviewed-on: https://go-review.googlesource.com/c/go/+/447116
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-01 21:32:43 +00:00
Bryan C. Mills
1587c36583 runtime: check for ErrWaitDelay in runBuiltTestProg
ErrWaitDelay is not expected to occur in this test, but if it does
it indicates a failure mode very different from the “failed to start”
catchall that we log for other non-ExitError errors.

Updates #50436.

Change-Id: I3f4d87d502f772bf471fc17303d5a6b483446f8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/446876
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
2022-11-01 21:32:26 +00:00
Bryan C. Mills
082afccebf internal/testenv: adjust timeout calculations in CommandContext
I noticed some test failures in the build dashboard after CL 445597
that made me realize the grace period should be based on the test
timeout, not the Context timeout: if the test itself sets a short
timeout for a command, we still want to give the test process enough
time to consume and log its output.

I also put some more thought into how one might debug a test hang, and
realized that in that case we don't want to set a WaitDelay at all:
instead, we want to leave the processes in their stuck state so that
they can be investigated with tools like `ps` and 'lsof'.

Updates #50436.

Change-Id: I65421084f44eeaaaec5dd2741cd836e9e68dd380
Reviewed-on: https://go-review.googlesource.com/c/go/+/446875
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-01 21:32:24 +00:00
Michael Matloob
50c5919475 cmd/dist: fix a variable scope bug:
We reused p so we were deleting the same directory twice instead of two
different directories. Fix that.

For #47257

Change-Id: I315ad87d0a9182e00ae4c11b82986227e2b02e17
Reviewed-on: https://go-review.googlesource.com/c/go/+/447115
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-11-01 20:46:55 +00:00
Ian Lance Taylor
49bbece44c go/build: ignore files by extension before matching on name
Otherwise given a file like defs_nacl_amd64p32.go.~1~ we will add
"nacl" and "amd64p32" to AllTags. This was causing the
cmd/go/internal/modindex tests to fail on my system, since I had
an old editor backup file lying around.

Change-Id: Ib1c5d835e4871addae6dc78cee07c9839bb880e2
Reviewed-on: https://go-review.googlesource.com/c/go/+/446395
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-11-01 19:48:43 +00:00
Austin Clements
6a44a3aa9f test/bench/go1: eliminate start-up time
The go1 benchmark suite does a lot of work at package init time, which
makes it take quite a while to run even if you're not running any of
the benchmarks, or if you're only running a subset of them. This leads
to an awkward workaround in dist test to compile but not run the
package, unlike roughly all other packages. It also reduces isolation
between benchmarks by affecting the starting heap size of all
benchmarks.

Fix this by initializing all data required by a benchmark when that
benchmark runs, and keeping it local so it gets freed by the GC and
doesn't leak between benchmarks. Now, none of the benchmarks depend on
global state.

Re-initializing the data on each benchmark run does add overhead to an
actual benchmark run, as each benchmark function is called several
times with different values of b.N. A full run of all benchmarks at
the default -benchtime=1s now takes ~10% longer; higher -benchtimes
would be less. It would be quite difficult to cache this data between
invocations of the same benchmark function without leaking between
different benchmarks and affecting GC overheads, as the testing
package doesn't provide any mechanism for this.

This reduces the time to run the binary with no benchmarks from 1.5
seconds to 10 ms, and also reduces the memory required to do this from
342 MiB to 17 MiB.

To make sure data was not leaking between different benchmarks, I ran
the benchmarks with -shuffle=on. The variance remained low: mostly
under 3%. A few benchmarks had higher variance, but in all cases it
was similar to the variance between this change.

This CL naturally changes the measured performance of several of the
benchmarks because it dramatically changes the heap size and hence GC
overheads. However, going forward the benchmarks should be much better
isolated.

For #37486.

Change-Id: I252ebea703a9560706cc1990dc5ad22d1927c7a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/443336
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Austin Clements <austin@google.com>
2022-11-01 17:07:14 +00:00
Austin Clements
767df51b4d runtime: fix missing error print in TestCgoSigfwd
The result of the call to fmt.Errorf was unused. It was clearly
intending to print the message, not simply construct an error.

Change-Id: I14856214c521a51fe4b45690e6c35fbb17e66577
Reviewed-on: https://go-review.googlesource.com/c/go/+/443375
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-11-01 17:06:49 +00:00
Austin Clements
e72da1c15d runtime: skip TestArenaCollision on failed reservation
If TestArenaCollision cannot reserve the address range it expects to
reserve, it currently fails somewhat mysteriously. Detect this case
and skip the test. This could lead to test rot if we wind up always
skipping this test, but it's not clear that there's a better answer.
If the test does fail, we now also log what it thinks it reserved so
the failure message is more useful in debugging any issues.

Fixes #49415
Fixes #54597

Change-Id: I05cf27258c1c0a7a3ac8d147f36bf8890820d59b
Reviewed-on: https://go-review.googlesource.com/c/go/+/446877
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2022-11-01 17:06:23 +00:00
Damien Neil
61ae0a37a8 syscall, os/exec: reject environment variables containing NULs
Check for and reject environment variables containing NULs.

The conventions for passing environment variables to subprocesses
cause most or all systems to interpret a NUL as a separator. The
syscall package rejects environment variables containing a NUL
on most systems, but erroniously did not do so on Windows. This
causes an environment variable such as "FOO=a\x00BAR=b" to be
interpreted as "FOO=a", "BAR=b".

Check for and reject NULs in environment variables passed to
syscall.StartProcess on Windows.

Add a redundant check to os/exec as extra insurance.

Fixes #56284
Fixes CVE-2022-41716

Change-Id: I2950e2b0cb14ebd26e5629be1521858f66a7d4ae
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1609434
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Tatiana Bradley <tatianabradley@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
TryBot-Result: Security TryBots <security-trybots@go-security-trybots.iam.gserviceaccount.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/446916
Reviewed-by: Tatiana Bradley <tatiana@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2022-11-01 16:40:37 +00:00
Russ Cox
ad5d2f64fb testing: print names of running tests on test timeout
Track the running tests and when they started,
so that we can report the running tests on a test timeout.

	% go test -timeout=5s
	panic: test timed out after 5s
	running tests:
		TestTCPSpuriousConnSetupCompletion (4s)

	... stack traces as usual ...

	% go test -run=Script -timeout=10s cmd/go
	vcs-test.golang.org rerouted to http://127.0.0.1:65168
	https://vcs-test.golang.org rerouted to https://127.0.0.1:65169
	go test proxy running at GOPROXY=http://127.0.0.1:65170/mod
	panic: test timed out after 10s
	running tests:
		TestScript (10s)
		TestScript/mod_get_patchcycle (0s)
		TestScript/mod_get_prefer_incompatible (0s)
		TestScript/mod_get_promote_implicit (0s)
		TestScript/mod_get_pseudo (0s)
		TestScript/mod_get_pseudo_other_branch (0s)
		TestScript/mod_get_pseudo_prefix (0s)
		TestScript/mod_get_test (0s)
		TestScript/mod_get_trailing_slash (0s)
		TestScript/mod_get_update_unrelated_sum (0s)
		TestScript/mod_gobuild_import (0s)
		TestScript/mod_gomodcache (0s)
		TestScript/mod_gonoproxy (0s)
		TestScript/mod_load_badchain (0s)
		TestScript/mod_overlay (0s)
		TestScript/test_fuzz_minimize (6s)
		TestScript/test_fuzz_minimize_dirty_cov (7s)

	... stack traces as usual ...

Change-Id: I3a6647c029097becc06664ebd76a2597c7ed7b8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/446176
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2022-11-01 14:20:31 +00:00
Than McIntosh
02cffcde17 cmd/cover: fix buglet causing differences in -m output
Use a slightly different line number pragma when emitting instrumented
code, so as to ensure that we don't get any changes in the
"-gcflags=-m" output for coverage vs non-coverage.

Fixes #56475.

Change-Id: I3079171fdf83c0434ed6ea0ce3eb2797c2280c55
Reviewed-on: https://go-review.googlesource.com/c/go/+/446259
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-01 14:12:36 +00:00
Than McIntosh
99dc2a1859 cmd/go: revamp and simplify logic in PrepareForCoverageBuild
Change the 'PrepareForCoverageBuild' helper function to provide more
sensible defaults in the case where Go packages are listed on the
command line (e.g. cases such as "go run -cover mumble.go").

With the old implementation, if module mode was enabled, we would only
instrument packages in the main module, meaning that if you did
something like this:

  $ ls go.mod
  go.mod
  $ GOCOVERDATA=/tmp/cov go run -cover testdata/somefile.go
  $

no coverage profiles would be generated at all. This is due to the
fact that the pseudo-package created by the Go command internally when
building "somefile.go" is not considered part of the main module.

This patch moves the default to "packages explicitly mentioned on the
command line, plus packages in the main module", which will make more
sense to users passing specific packages and *.go files on the command
line. Examples:

  // Here cmd/compile is part the Go standard library + commands
  // (which we exclude from instrumentation by default), but since
  // 'cmd/compile' is mentioned on the command line, we will instrument
  // just that single package (not any of its deps).
  $ cd $GOROOT/src
  $ go build -o gc.exe -cover cmd/compile
  $ GOCOVERDATA=/tmp/cov ./gc.exe ...
  ...
  $

  // Here we're running a Go file named on the command line, where
  // the pseudo-package for the command line is not part of the
  // main module, but it makes sense to instrument it anyhow.
  $ cd ~/go/k8s.io/kubernetes
  $ GOCOVERDATA=/tmp/cov go run -cover test/typecheck/testdata/bad/bad.go
  ...
  $

This patch also simplifies the logic and improves flow/comments in
in the helper function PrepareForCoverageBuild.

Change-Id: Id8fc8571157dac8c09e44cc73baa05aeba1640ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/445918
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
2022-11-01 14:12:26 +00:00
Than McIntosh
317f2a7df6 cmd/compile: revise inliner coverage tweaks (again)
This patch fixes a typo/bug introduced in CL 441858 where when pattern
matching a coverage counter access we were looking at an assingment
node instead of the assignment LHS, and fixes a similar problem in
atomic counter update pattern matching introduced in CL 444835. In
both of these cases the bug was not caught because the test intended
to lock down the behavior was written incorrectly (wasn't
instrumenting what the test author thought it was instrumenting,
ouch).

Change-Id: I6e6ac3beacf12ef1a817de5527340b639f0bb044
Reviewed-on: https://go-review.googlesource.com/c/go/+/446258
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
2022-11-01 14:12:12 +00:00
Than McIntosh
2730170f63 cmd/{go,cover}: fix for -coverprofile path capture with local pkg
When coverage testing a local package (defined by a relative import
path such as "./foo/bar") the convention when "-coverprofile" is used
has been to capture source files by full pathname, as opposed to
recording the full import path or the invented import path
("command-line-arguments/") created by the go command in the case of
building named Go files. Doing this makes it much easier to use
collected profiles with "go tool -cover -html=<profile>".

The support for this feature/convention wound up being inadvertantly
dropped during the GOEXPERIMENT=coverageredesign implementation; this
patch restores it.

Fixes #56433.

Change-Id: Ib9556fdc86011b00c155caa614ab23e5148f3eb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/445917
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-01 14:11:54 +00:00
Russ Cox
a3559f3301 net: use libc (not cgo) for DNS on macOS
Change the macOS implementation to use libc calls.
Using libc calls directly is what we do for all the runtime and os syscalls.
Doing so here as well improves consistency and also makes it possible
to cross-compile (from non-Mac systems) macOS binaries that use the
native name resolver.

Fixes #12524.

Change-Id: I011f4fcc5c50fbb5396e494889765dcbb9342336
Reviewed-on: https://go-review.googlesource.com/c/go/+/446178
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-01 14:05:39 +00:00
Bryan C. Mills
661e931dd1 internal/testenv: move helpers related to os/exec to a separate file
(Suggested by Austin in CL 445596.)

Change-Id: Ic774d551e8085d15435ef2b3c78d0169a365ce41
Reviewed-on: https://go-review.googlesource.com/c/go/+/446642
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-11-01 13:37:40 +00:00
David Chase
c81c027982 cmd/compile: add ability to indicate 'concurrentOk' for debug flags
Also removes no-longer-needed "Any" field from compiler's DebugFlags.
Test/use case for this is the fmahash CL.

Change-Id: I214f02c91f30fc2ce53caf75fa5e2b905dd33429
Reviewed-on: https://go-review.googlesource.com/c/go/+/445495
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
2022-10-31 22:19:34 +00:00
Michael Matloob
5619dd0849 cmd/go: add move test for goroot
This  an end-to-end test that sets GOROOT to a symlink
to the distribution, approximating copying it to a new
location, and checks that packages in the standard library
are not stale, as they would be if paths were embedded
in artifacts.

For #47257

Change-Id: I4ed04df36656ad946a2f6f5ce3194e76e06372e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/445358
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-10-31 21:51:12 +00:00
Keith Randall
68bd383368 cmd/compile: add cache of sizeable objects so they can be reused
We kind of have this mechanism already, just normalizing it and
using it in a bunch of places. Previously a bunch of places cached
slices only for the duration of a single function compilation. Now
we can reuse slices across a whole compiler run.

Use a sync.Pool of powers-of-two sizes. This lets us use not
too much memory, and avoid holding onto memory we're no longer
using when a GC happens.

There's a few different types we need, so generate the code for it.
Generics would be useful here, but we can't use generics in the
compiler because of bootstrapping.

Change-Id: I6cf37e7b7b2e802882aaa723a0b29770511ccd82
Reviewed-on: https://go-review.googlesource.com/c/go/+/444820
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2022-10-31 21:41:20 +00:00
Keith Randall
7ddc45263c cmd/compile: separate out sparsemaps that need position
Make them a separate type, so the normal sparse maps don't
need the extra storage.

Change-Id: I3a0219487c35ea63723499723b0c742e321d97c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/444819
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2022-10-31 21:41:06 +00:00
Keith Randall
9ce27feaeb cmd/compile: add rule for post-decomposed growslice optimization
The recently added rule only works before decomposing slices.
Add a rule that works after decomposing slices.

The reason we need the latter is because although the length may
be a constant, it can be hidden inside a slice that is not constant
(its pointer or capacity might be changing). By applying this
optimization after decomposing slices, we can find more cases
where it applies.

Fixes #56440

Change-Id: I0094e59eee3065ab4d210defdda8227a6e897420
Reviewed-on: https://go-review.googlesource.com/c/go/+/446277
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2022-10-31 21:40:49 +00:00
Michael Pratt
204be97d24 cmd/compile/internal/pgo: remove most global state
Since pgo is a new package, it is reasonably straightforward to
encapsulate its state into a non-global object that we pass around,
which will help keep it isolated.

There are no functional changes in this CL, just packaging up the
globals into a new object.

There are two major pieces of cleanup remaining:

1. reflectdata and noder have separate InlineCalls calls for method
   wrappers. The Profile is not plumbed there yet, but this is not a
   regression as the globals were previously set only right around the
   main inlining pass in gc.Main.

2. pgo.ListOfHotCallSites is still global, as it will require more work
   to clean up. It is effectively a local variable in InlinePackage,
   except that it assumes that InlineCalls is immediately preceded by a
   CanInline call for the same function. This is not necessarily true
   due to the recursive nature of CanInline. This also means that some
   InlineCalls calls may be missing the list of hot callsites right now.

For #55022.

Change-Id: Ic1fe41f73df96861c65f8bfeecff89862b367290
Reviewed-on: https://go-review.googlesource.com/c/go/+/446303
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
2022-10-31 21:18:33 +00:00
Ian Lance Taylor
3848b44c75 os: in TestDirFS only check returned path for unexpected string
The test added in CL 446115 was failing on Plan 9, on which the error
returned by a failed open includes the path that failed.

Change-Id: If27222596c3cb0366a030bb49ae41c4c869c3db2
Reviewed-on: https://go-review.googlesource.com/c/go/+/446641
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-10-31 21:03:17 +00:00
Michael Pratt
ec0b540293 cmd/compile/internal/pgo: remove ConvertLine2Int
Parts of package pgo fetch the line number of a node by parsing the
number out of the string returned from ir.Line().

This is indirect and inefficient, so it should be replaced with a more
direct lookup. It is also potentially buggy: ir.Line uses
ctxt.OutermostPos, i.e., the line number where an inlined node in
inlined. We want ctxt.InnermostPos, because that is the line number used
in pprof profiles that we are matching against (See comments on
OutermostPos and InnermostPos).

I'm not sure whether this was an active, as we use ir.Line before and
during inlining. I think we could see CALL nodes with OutermostPos !=
InnermostPos during midstack inlining, but I am not sure. Regardless,
explicitly using the desired position is clearer.

For #55022.

Change-Id: Ic640761c9e1d01cacbf91f3aaeaf284ad7e38dbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/446302
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
2022-10-31 21:00:25 +00:00
Bryan C. Mills
e8ec68edfa internal/testenv: remove RunWithTimout
For most tests, the test's deadline itself is more appropriate than an
arbitrary timeout layered atop of it (especially once #48157 is
implemented), and testenv.Command already adds cleaner timeout
behavior when a command would run too close to the test's deadline.

That makes RunWithTimeout something of an attractive nuisance. For
now, migrate the two existing uses of it to testenv.CommandContext,
with a shorter timeout implemented using context.WithTimeout.

As a followup, we may want to drop the extra timeouts from these
invocations entirely.

Updates #50436.
Updates #37405.

Change-Id: I16840fd36c0137b6da87ec54012b3e44661f0d08
Reviewed-on: https://go-review.googlesource.com/c/go/+/445597
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-31 20:54:10 +00:00
Bryan C. Mills
84cd7ab3c3 internal/testenv: add CommandContext and Command
This adds a testenv.CommandContext function, with timeout behavior
based on the existing logic in cmd/go.TestScript: namely, the command
is terminated with SIGQUIT (if supported) with an arbitrary grace
period remaining until the test's deadline.

If the test environment does not support executing subprocesses,
CommandContext skips the test.

If the command is terminated due to the timout expiring or the test
fails to wait for the command after starting it, CommandContext marks
the test as failing.

For tests where a shorter timeout is desired (such as for fail-fast
behavior), one may be supplied using context.WithTimeout.

The more concise Command helper is like CommandContext but without
the need to supply an explicit Context.

Updates #50436.

Change-Id: Ifd81fb86c402f034063c9e9c03045b4106eab81a
Reviewed-on: https://go-review.googlesource.com/c/go/+/445596
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-10-31 20:53:50 +00:00
Michael Pratt
76940b3e57 cmd/internal/obj: cleanup linkgetlineFromPos
Make linkgetlineFromPos and getFileIndexAndLine methods on Link, and
give the former a more descriptive name.

The docs are expanded to make it more clear that these are final
file/line visible in programs.

In getFileSymbolAndLine use ctxt.InnermostPos instead of ctxt.PosTable
direct, which makes it more clear that we want the semantics of
InnermostPos.

Change-Id: I7c3d344dec60407fa54b191be8a09c117cb87dd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/446301
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-31 20:45:15 +00:00
Michael Pratt
1999f256e4 cmd/compile/internal/test: clean up TestPGOIntendedInlining
The most important change here is to log output from the child, making
it easier to diagnose problems when the child 'go test' fails.

We can also eliminate the cmd.Wait goroutine by using an os.Pipe, whose
reader will return io.EOF when the child exits.

For #55022.

Change-Id: I1573ea444407d545bdca8525c9ff7b0a2baebf5e
Reviewed-on: https://go-review.googlesource.com/c/go/+/446300
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-31 20:45:00 +00:00
Zeke Lu
e943dc5a8c net/http: deflake TestCancelRequestWhenSharingConnection
The test sleeps for 1 millisecond to give the cancellation a moment
to take effect. This is flaky because the request can finish before
the cancellation of the context is seen. It's easy to verify by adding

    time.Sleep(2*time.Millisecond)

after 0a6c4c8740/src/net/http/transport.go (L2619).
With this modification, the test fails about 5 times out of 10 runs.

The fix is easy. We just need to block the handler of the second
request until this request is cancelled. I have verify that the
updated test can uncover the issue fixed by CL 257818.

Fixes #55226.

Change-Id: I81575beef1a920a2ffaa5c6a5ca70a4008bd5f94
GitHub-Last-Rev: 99cb1c2eae
GitHub-Pull-Request: golang/go#56500
Reviewed-on: https://go-review.googlesource.com/c/go/+/446676
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-10-31 20:44:00 +00:00
Russ Cox
6695cebeec encoding/xml: reduce depth limit on wasm
Wasm can't handle the recusion for XML nested to depth 10,000.
Cut it off at 5,000 instead. This fixes TestCVE202228131 on trybots
in certain conditions.

Also disable TestCVE202230633 to fix 'go test -v encoding/xml' on gomotes.

Also rename errExeceededMaxUnmarshalDepth [misspelled and unwieldy]
to errUnmarshalDepth.

For #56498.

Change-Id: I7cc337ccfee251bfd9771497be0e5272737114f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/446639
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-31 20:35:56 +00:00
Alan Donovan
4b993bffb8 go/types: avoid formatting unneeded causes
This one slipped through a previous cleanup
and showed up in an allocation profile.

Change-Id: Ife447fe32797f1685c20806718193b0985b4a412
Reviewed-on: https://go-review.googlesource.com/c/go/+/446295
Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-31 20:16:31 +00:00
Changkun Ou
7366b199c5 sync: use atomic.Pointer for entry
Change-Id: Ie3b4bfe483d7ef43da29ea1dd73d423dac36cf39
Reviewed-on: https://go-review.googlesource.com/c/go/+/432055
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Changkun Ou <mail@changkun.de>
2022-10-31 20:12:59 +00:00
Bryan C. Mills
3aebf682e4 cmd/api: skip tests when 'os/exec' is supported but 'go build' is not
This may fix the android-.*-emu builders, which are currently broken.

For #47257.

Change-Id: I370dad2cb8031f8f5fdfbeb9c284c4f79f58d929
Reviewed-on: https://go-review.googlesource.com/c/go/+/446638
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
2022-10-31 19:20:21 +00:00
Bryan C. Mills
0a6c4c8740 cmd/compile/internal/types2: fix tests on js/wasm
The js/wasm builder started failing as of CL 432535 due to needing
'go build' to import standard-library packages that are no longer
installed to GOROOT/pkg. Since js/wasm can't exec subprocesses,
it cannot run 'go build' to generate the export data needed for
these tests.

For #47257.

Change-Id: I804235fd725faf00b27cbed79ee1f43dea8ab734
Reviewed-on: https://go-review.googlesource.com/c/go/+/446635
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
2022-10-31 15:44:19 +00:00
Archana R
6774ddfec7 internal/bytealg: fix bug in index function for ppc64le/power9
The index function was not handling certain corner cases where there
were two more bytes to be examined in the tail end of the string to
complete the comparison. Fix code to ensure that when the string has
to be shifted two more times the correct bytes are examined.
Also hoisted vsplat to V10 so that all paths use the correct value.
Some comments had incorrect register names and corrected the same.
Added the strings that were failing to strings test for verification.

Fixes #56457

Change-Id: Idba7cbc802e3d73c8f4fe89309871cc8447792f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/446135
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Archana Ravindar <ravindararchana@gmail.com>
2022-10-31 12:52:07 +00:00
erifan01
e09bbaec69 cmd/compile: enable address folding for global symbols of shared library
Address folding is disabled in CL42172, the commit message of which
said that "In shared library, load/store of global is rewritten to
using GOT and temp register, which conflicts with the use of temp
register for assembling large offset.". Actually this doesn't happen
because the sequence of instructions when rewritten to use Got looks
like this:
  MOVD $sym, Rx becomes
  MOVD sym@GOT, Rx
If there is an offset off, there will be one more instruction:
  ADD $off, Rx, Rx

And MOVD sym, Rx becomes
  MOVD sym@GOT, REGTMP
  MOVx (REGTMP), Ry

If there is a small offset off, it becomes:
  MOVD sym@GOT, REGTMP
  MOVx (REGTMP)off, Ry

If off is very large, it becomes:
  MOVD sym@GOT, REGTMP
  MOVD $off, Rt
  ADD  Rt, REGTMP
  MOVx (REGTMP), Ry

We can see that the address can be calculated correctly, and testing
on darwin/arm64 confirms this.

Removing this restriction is beneficial to further optimize the sequence
of "ADRP+ADD+LD/ST" to "ADRP+LD/ST(offset), so this CL removes it.

Change-Id: I0e9f7bc1723e0a027f32cf0ae2c41cd6df49defe
Reviewed-on: https://go-review.googlesource.com/c/go/+/445535
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Eric Fang <eric.fang@arm.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-29 04:48:07 +00:00
Guoqi Chen
8a9e2d9d49 cmd/compile: add missing tail calls flag for linux/loong64
Set the value of the variable tailCall to true and prevent
allocating or clobber the linker register.

Change-Id: I4ec19c67056cb99196911aa7c0054be89ab7eb8d
Reviewed-on: https://go-review.googlesource.com/c/go/+/414954
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: WANG Xuerui <git@xen0n.name>
2022-10-29 03:14:03 +00:00
Guoqi Chen
c87e69c2b4 runtime/internal/atomic: enable atomic 64bit types for all functions on linux/loong64
ref. CL 356169

Change-Id: I813750d3fe191afe50d4029024ba97800211f688
Reviewed-on: https://go-review.googlesource.com/c/go/+/414414
Reviewed-by: WANG Xuerui <git@xen0n.name>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-29 02:44:29 +00:00
Ian Lance Taylor
3c17053bba cmd/go, cmd/cgo: support older versions of gccgo that lack cgo.Incomplete
Test whether gccgo/GoLLVM supports cgo.Incomplete. If it doesn't, use a
local definition rather than importing it.

Roll back 426496, which skipped a gccgo test, as it now works.

For #46731
Fixes #54761

Change-Id: I8bb2ad84c317094495405e178bf5c9694f82af56
Reviewed-on: https://go-review.googlesource.com/c/go/+/446260
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-29 00:40:18 +00:00
Riccardo Gerosa
91a1f0d918 math/big: improve performance of Binomial
This change improves the performance of Binomial by implementing an
algorithm that produces smaller intermediate values at each step.

Working with smaller big.Int values has the advantage that  fewer allocations
and computations are required for each mathematical operation.

The algorithm used is the Multiplicative Formula, which is a well known
way of calculating the Binomial coefficient and is described at:
https://en.wikipedia.org/wiki/Binomial_coefficient#Multiplicative_formula
https://en.wikipedia.org/wiki/Binomial_coefficient#In_programming_languages

In addition to that, an optimization has been made to remove a
redundant computation of (i+1) on each loop which has a measurable
impact when using big.Int.

Performance improvement measured on an M1 MacBook Pro
running the existing benchmark for Binomial:

name        old time/op    new time/op    delta
Binomial-8     589ns ± 0%     435ns ± 0%  -26.05%  (p=0.000 n=10+10)

name        old alloc/op   new alloc/op   delta
Binomial-8    1.02kB ± 0%    0.08kB ± 0%  -92.19%  (p=0.000 n=10+10)

name        old allocs/op  new allocs/op  delta
Binomial-8      38.0 ± 0%       5.0 ± 0%  -86.84%  (p=0.000 n=10+10)

Change-Id: I5a830386dd42f062e17af88411dd74fcb110ded9
GitHub-Last-Rev: 6b2fca07de
GitHub-Pull-Request: golang/go#56339
Reviewed-on: https://go-review.googlesource.com/c/go/+/444315
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
2022-10-28 23:50:33 +00:00
Michael Matloob
b726b0cadb cmd/go: don't install most GOROOT .a files in pkg
Packages in GOROOT that don't use cgo will not be installed in
GOROOT/pkg, and will instead be cached as usual like other Go
packages.

- add a internal/buildinternal package to hold the identities of the
  five packages that use cgo
- update dist's test code to do a go build std cmd before checking
  staleness on builders. Because most of those packages no longer have
  install locations, and have dependencies that don't either, the
  packages need to be cached to not be stale.
- fix index_test to import packages with the path "." when preparing
  the "want" values to compare the indexed data to. (the module index
  matches the behavior of build.ImportDir, which always passes in "."
  as the path.
- In both the index and go/build Importers, don't set
  PkgObj for GOROOT packages which will no longer have install
  targets. PkgTargetRoot will still be set to compute target paths,
  which will still be needed in buildmode=shared.
- "downgrade" all install actions that don't have a target to build
  actions. (The target should already not be set for packages that
  shouldn't be installed).

For #47257

Change-Id: Ia5aee6b3b20b58e028119cf0352a4c4a2f10f6b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/432535
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-10-28 23:35:08 +00:00
Ian Lance Taylor
4c69d0eeb8 testing: change Error to Errorf in comment
Fixes #56479

Change-Id: I1d97eb3ea97304e429e178ad05cb9f861b2bce84
Reviewed-on: https://go-review.googlesource.com/c/go/+/446275
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-10-28 20:33:17 +00:00
Ian Lance Taylor
42f334d677 os: don't include DirFS argument in DirFS errors
Otherwise we wind up mixing GOOS paths with slash separated paths.

Change-Id: I63dd733cbdb0668effbc030cfd58945008732d9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/446115
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-10-28 19:25:47 +00:00
Alessandro Arzilli
e59d873ff9 cmd/compile: emit DIEs for zero sized variables
Fixes the compiler to emit DIEs for zero sized variables.

Fixes #54615

Change-Id: I1e0c86a97f1abcc7edae516b6a7fe35bcb65ed0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/433479
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2022-10-28 17:49:22 +00:00
Michael Matloob
77296e3645 cmd/go: don't substitute '$WORK' for work directory in -x heredocs
When writing the set of commands excuted for go build -x, $WORK is
substituted for the work directory in all the commnands. But this
includes the cat <<EOF commands used to create a file with the given
contents. While we can expect the shell to substitute $WORK properly,
commands that read input files, such as importcfgs won't do that
substitution.

This is necessary to fix the build_dash_x script test for CL 432535
because it removes .a files from the traditional stdlib install
locations. The test can pass even with importcfg packagefiles in $WORK
because all transitive imports are in the stdlib, and the compiler can
fall back to finding stdlib .a files in their traditional places, but
once they're gone the packagefile paths in $WORK will have paths that
contain the string $WORK, and os.Open will fail to open them for
reading. And since the fallback is gone the test will fail.

For #47257

Change-Id: I5db0066de6ed3ccf97927a78ce0939e3eb14aebe
Reviewed-on: https://go-review.googlesource.com/c/go/+/446116
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2022-10-28 17:33:10 +00:00