1
0
mirror of https://github.com/golang/go synced 2024-11-22 23:50:03 -07:00
Commit Graph

56332 Commits

Author SHA1 Message Date
Paul E. Murphy
3c46d8f511 cmd/link: load external ELF PPC64 objects which set st_other=1
This indicates the symbol does not use or preserve the TOC pointer in
R2. Likewise, it does not have a distinct local entry point. This
happens when gcc compiles an object with -mcpu=power10.

Recycle the SymLocalentry field of a text symbol to pass through this
hint as the bogus value 1 (A valid offset must be a multiple of 4
bytes), and update the usage to check and generate errors further into
the linking process. This matches the behavior of st_other as used by
ELFv2.

Change-Id: Ic89ce17b57f400ab44213b21a3730a98c7cdf842
Reviewed-on: https://go-review.googlesource.com/c/go/+/490295
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-28 21:26:08 +00:00
Nick Ripley
265d19ed52 runtime/trace: avoid frame pointer unwinding for events during cgocallbackg
The current mp.incgocallback() logic allows for trace events to be
recorded using frame pointer unwinding during cgocallbackg when they
shouldn't be. Specifically, mp.incgo will be false during the
reentersyscall call at the end. It's possible to crash with tracing
enabled because of this, if C code which uses the frame pointer register
for other purposes calls into Go. This can be seen, for example, by
forcing testprogcgo/trace_unix.c to write a garbage value to RBP prior
to calling into Go.

We can drop the mp.incgo check, and instead conservatively avoid doing
frame pointer unwinding if there is any C on the stack. This is the case
if mp.ncgo > 0, or if mp.isextra is true (meaning we're coming from a
thread created by C). Rename incgocallback to reflect that we're
checking if there's any C on the stack. We can also move the ncgo
increment in cgocall closer to where the transition to C happens, which
lets us use frame pointer unwinding for the entersyscall event during
the first Go-to-C call on a stack, when there isn't yet any C on the
stack.

Fixes #59830.

Change-Id: If178a705a9d38d0d2fb19589a9e669cd982d32cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/488755
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Nick Ripley <nick.ripley@datadoghq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-28 21:07:22 +00:00
Michael Pratt
0fd6ae548f cmd/compile: escape package path for PGO symbol matching
Symbol names in the final executable apply escaping to the final
component of a package path (main in example.com becomes
example%2ecom.main).

ir.PkgFuncName does not perform this escaping, meaning we'd fail to
match functions that are escaped in the profile.

Add ir.LinkFuncName which does perform escaping and use it for PGO.

Fixes #59887.

Change-Id: I10634d63d99d0a6fd2f72b929ab35ea227e1336f
Reviewed-on: https://go-review.googlesource.com/c/go/+/490555
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-28 20:33:34 +00:00
Joel Sing
8b67cf0bc6 cmd/internal/obj/arm64: pass obj.As to oaddi rather than an instruction
This simplifies callers, as they do not need to call opirr before calling oaddi.
Additionally, use appropriate types (int16) for registers, which avoids the need
to continually cast.

Change-Id: I8ca3807a97867ac49d63792f6922a18f35824448
Reviewed-on: https://go-review.googlesource.com/c/go/+/471520
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-28 18:45:41 +00:00
Cherry Mui
19fd96512c cmd/link: put zero-sized data symbols at same address as runtime.zerobase
Put zero-sized data symbols at same address as runtime.zerobase,
so zero-sized global variables have the same address as zero-sized
allocations.

Change-Id: Ib3145dc1b663a9794dfabc0e6abd2384960f2c49
Reviewed-on: https://go-review.googlesource.com/c/go/+/490435
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2023-04-28 18:35:43 +00:00
Joel Sing
e7af0e0cac cmd/internal/obj/arm64: use appropriate return type for regoff
All of the callers of regoff cast the return value from uint32 to int32.
Instead, simply return int32 in the first place.

Change-Id: I43a672bb3143a71f4a37779ed8ae9adcda623ba4
Reviewed-on: https://go-review.googlesource.com/c/go/+/490355
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-04-28 18:06:23 +00:00
Robert Griesemer
69e66a1626 go/types, types2: remove parse (we only need mustParse for tests)
While at it, also simplify mustTypecheck again as it can just use
typecheck.

Change-Id: I6cb07b1078d9a39e0f22851028fdd4442127f2f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/490015
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-28 17:58:13 +00:00
Robert Griesemer
c9b0d8b61e go/types, types2: extract package name from test sources automatically
This simplifies explicit tests and ensures that the error messages
contain the package name instead of a generic file name like "p.go".

Fixes #59736.

Change-Id: I1b42e30f53ba88456e92f990d80ca68ffc987e20
Reviewed-on: https://go-review.googlesource.com/c/go/+/486617
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
2023-04-28 17:58:07 +00:00
Takeshi Yoneda
0835701224 os: fix TestChdirAndGetwd/ReaddirnamesOneAtATime on wasip1 to run on Windows hosts
TestReaddirnamesOneAtATime and TestChdirAndGetwd assumes the underlying file system
has /usr/bin but it is not the case when running it on WASI runtime hosted on Windows.

This change adds wasip1 in the special cased switch case to make them host OS agonstic.

Change-Id: Idb667021b565f939c814b9cd9e637cd75f9a610d
Reviewed-on: https://go-review.googlesource.com/c/go/+/489575
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2023-04-28 17:26:10 +00:00
Robert Griesemer
972774c444 go/types, types2: call mustParse when using mustTypecheck
Syntactically incorrect source files may produce valid (but
unexpected) syntax trees, leading to difficult to understand
test failures.

Make sure to call mustParse when we call mustTypecheck.

Change-Id: I9f5ba3fe57ad3bbc16caabf285d2e7aeb5b9de0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/489995
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
2023-04-28 16:30:57 +00:00
Cherry Mui
6c8ea391cf cmd/link: write buildid to plugin
Currently, in plugin build mode we don't write the build ID. This
is disabled in CL 29394 since plugin is supported on Darwin. Maybe
it caused some problem with the Darwin dynamic linker. But it
seems no problem currently. Enabled it.

Fixes #59845.

Change-Id: I60589ffc7937e4d30055960d391cac1e7cd0cd42
Reviewed-on: https://go-review.googlesource.com/c/go/+/489457
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
2023-04-28 16:02:17 +00:00
Lucien Coffe
ff059add10 runtime: resolve checkdead panic by refining startm lock handling in caller context
This change addresses a `checkdead` panic caused by a race condition between
`sysmon->startm` and `checkdead` callers, due to prematurely releasing the
scheduler lock. The solution involves allowing a `startm` caller to acquire the
scheduler lock and call `startm` in this context. A new `lockheld` bool
argument is added to `startm`, which manages all lock and unlock calls within
the function. The`startIdle` function variable in `injectglist` is updated to
call `startm` with the lock held, ensuring proper lock handling in this
specific case. This refined lock handling resolves the observed race condition
issue.

Fixes #59600

Change-Id: I11663a15536c10c773fc2fde291d959099aa71be
Reviewed-on: https://go-review.googlesource.com/c/go/+/487316
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
2023-04-28 15:57:36 +00:00
Ian Lance Taylor
752ef81056 net: re check conf.goos even if it equals runtime.GOOS
This field is only for testing purposes, where we can't assume
that the conf value was initialized as expected for that GOOS.

This fixes the net tests on android.

Change-Id: I8432587f219a05adbb4d234a813467f876a764b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/489975
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-04-28 13:22:04 +00:00
Chressie Himpel
f046180890 Revert "cmd/compile: constant-fold loads from constant dictionaries and types"
This reverts CL 486895.

Reason for revert: This breaks internal tests at Google, see b/280035614.

Change-Id: I48772a44f5f6070a7f06b5704e9f9aa272b5f978
Reviewed-on: https://go-review.googlesource.com/c/go/+/490156
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Michael Stapelberg <stapelberg@google.com>
Reviewed-by: David Chase <drchase@google.com>
2023-04-28 11:15:01 +00:00
Bryan C. Mills
71ad46cd2a internal/testenv: allow 'go build' on android when supported
As of CL 472096, it should work on android/arm64 always (because
internal linking is supported on that platform), and on other android
platforms when a C toolchain is present in the test environment.

Updates #58775.

Change-Id: Ifa38dc69b258b38dcc341979dcbf8cd61265c787
Reviewed-on: https://go-review.googlesource.com/c/go/+/475456
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Changkun Ou <mail@changkun.de>
2023-04-28 01:33:31 +00:00
Bryan C. Mills
7bc8aacec1 misc/android: rework GOROOT installation
- Fall back to 'go env GOROOT' to locate GOROOT if runtime.GOROOT() is
  empty (as may be the case if the tool is built with -trimpath).

- Copy all of $GOROOT/android_$GOARCH/bin, not just cmd/go, to
  $GOROOT/bin.

- For consistency with CL 404134, place $GOROOT/bin at the beginning
  of $PATH, not the end.

- Don't use the install target for the "runtime" package to locate pkg/tool.
  As of Go 1.20 "runtime" doesn't have an install directory anyway.
  Since the real reason we need pkg/tool is for commands in "cmd",
  use an arbitrary command (namely "cmd/compile") to locate it.

- Use 'go list' to determine the package import path for the current
  directory, instead of assuming that it is within GOROOT or GOPATH.
  (That assumption does not hold in module mode.)

Updates #58775.

Change-Id: If76ff22bce76d05175c40678230f046a4aff0940
Reviewed-on: https://go-review.googlesource.com/c/go/+/472096
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Changkun Ou <mail@changkun.de>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-04-28 01:32:45 +00:00
Achille Roussel
da3a184b18 syscall: remove wasip1 O_DIRECTORY workaround
Wasmtime used to error when opening a directory without passing the
O_DIRECTORY flag, which required doing a stat to determine whether to
inject the flag prior to opening any file.

The workaround was subject to races since the stat and open calls were
not atomic.

Wasmtime fixed the issue in v8.0.1.

For details see:
- https://github.com/bytecodealliance/wasmtime/pull/4967
- https://github.com/bytecodealliance/wasmtime/pull/6163
- https://github.com/bytecodealliance/wasmtime/releases/tag/v8.0.1

Change-Id: I0f9fe6a696024b70fffe63b83e8f61e48acd0c4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/489955
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-04-28 01:27:53 +00:00
Ian Lance Taylor
afe2d22219 net: rewrite and simplify resolver configuration
The resulting code behaves mostly the same. There are some minor
differences in error cases when the cgo resolver is not available:
instead of just falling back we keep trying to work out the right
nsswitch.conf order.

Change-Id: I17fadc940528fa2397043ac8f8ed7da3bd7a95c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/487196
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Mateusz Poliwczak <mpoliwczak34@gmail.com>
2023-04-27 22:26:09 +00:00
Keith Randall
635839a17a cmd/compile: constant-fold loads from constant dictionaries and types
Update #59591

Change-Id: Id250a7779c5b53776fff73f3e678fec54d92a8e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/486895
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-27 21:12:07 +00:00
Guoqi Chen
3a7806d387 cmd/dist,internal: enable buildmode=c-archive for linux/loong64
Now the shared flag is supported on the linux/loong64 platform and
misc/cgo/testcarchive has been passed, buildmode=c-archive can be used.

Change-Id: Ice450dc11fcb91942fdf2ddd34bb163853267e01
Reviewed-on: https://go-review.googlesource.com/c/go/+/489576
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: WANG Xuerui <git@xen0n.name>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: WANG Xuerui <git@xen0n.name>
2023-04-27 18:35:10 +00:00
Bryan C. Mills
954ff15dbe cmd/go: skip the 'git' part of get_insecure_redirect in short mode
Invoking 'git' adds about 200ms to this test on a fast machine,
probably more on a slow one. (As a general habit we skip the
'git' tests uniformly in short mode.)

For #52545.

Change-Id: Iea6d86a8c9c8b0f1fe51888faf7f5fe7dd8f1eb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/488236
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-27 17:54:44 +00:00
cui fliter
42f558bd56 all: remove repeated definite articles
Change-Id: Idea3e6ca6e62bd5a5ff6e6d5c3f39efb7628f0ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/489635
Run-TryBot: Michael Pratt <mpratt@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
2023-04-27 15:29:54 +00:00
Dmitri Shuralyov
e8c8b79f00 cmd/api: remove unused functionality
We no longer use the optional parameter to compareAPI.
We now always set allowAdd to false.
(Except in tests, making them less useful than they could be.)
Flags and parsing their value are no more.

Remove all the unused functionality and update test cases so they're
closer to what the API checker does when it runs for real. Order the
features, required, exception variables and fields more consistently.

For #43956.

Change-Id: Iaa4656a89a3fca3129742165a448d385e55e4a98
Reviewed-on: https://go-review.googlesource.com/c/go/+/489436
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-27 14:07:10 +00:00
Dmitri Shuralyov
0972096c5c cmd/api: make api/next/* entries required, not optional
We want the API check to catch if some API present in api/next/*
files is no longer implemented in the tree, and report it in the
same CL that is making the change (by failing loudly). Arguably
this should've been the case since CL 315350, but I didn't notice
it at the time. Do it now.

For #43956.

Change-Id: I73330dd5fd3f5706a1fdf13b2bf8e0f24c6b48e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/488135
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
2023-04-27 14:06:36 +00:00
cui fliter
b4eed1c341 crypto: use t.Parallel() to increase speed
It is necessary to invoke the t.Parallel() method in both the top-level test function and its subtest function to maximize parallelism. In doing so, all subtest functions calling the t.Parallel() method in the package will work in parallel.

On my machine, the execution time of this test file was cut in half.

Change-Id: If09147a2a9969bb044932d71e6bfea29492866d6
Reviewed-on: https://go-review.googlesource.com/c/go/+/482755
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2023-04-27 02:00:03 +00:00
Bryan C. Mills
527745c4d5 misc/wasm: default to /tmp if TMPDIR is unset
Change-Id: Ibf460d86ced08687099725bcd8ea8f38d7e8484c
Reviewed-on: https://go-review.googlesource.com/c/go/+/489435
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-04-26 21:21:32 +00:00
Bryan C. Mills
ffa663c9b3 net/http/cgi: propagate LD_LIBRARY_PATH on Android
Android is functionally a variant on linux, and should be
treated as such.

Change-Id: I08056f00bf98c1935c8cc3c859a6c72fe1a48efa
Reviewed-on: https://go-review.googlesource.com/c/go/+/489395
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-04-26 21:07:34 +00:00
Brad Fitzpatrick
fa80fe7b1c cmd/go: simplify code that still assumed the build cache could be nil
cache.Default always returns a non-nil value since Go 1.12; the docs were
updated in https://go.dev/cl/465555.

This updates all the callers of cache.Default that were checking whether
the result was nil so the code isn't misleading/confusing to readers.

Change-Id: Ia63567dd70affef6041c744259f65cea79a2752e
Reviewed-on: https://go-review.googlesource.com/c/go/+/489355
Auto-Submit: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-04-26 20:37:24 +00:00
Michael Pratt
7b874619be runtime/cgo: store M for C-created thread in pthread key
This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and
CL 485316 incorporated.

CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 482975 is a followup fix to a C declaration in testprogcgo.
CL 485315 is a followup fix for x_cgo_getstackbound on Illumos.
CL 485316 is a followup cleanup for ppc64 assembly.

[Original CL 481061 description]

This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.

CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.

[Original CL 392854 description]

In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key,  only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

[CL 479915 description]

Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

[CL 485500 description]

CL 479915 passed the G to _cgo_getstackbound for direct updates to
gp.stack.lo. A G can be reused on a new thread after the previous thread
exited. This could trigger the C TSAN race detector because it couldn't
see the synchronization in Go (lockextra) preventing the same G from
being used on multiple threads at the same time.

We work around this by passing the address of a stack variable to
_cgo_getstackbound rather than the G. The stack is generally unique per
thread, so TSAN won't see the same address from multiple threads. Even
if stacks are reused across threads by pthread, C TSAN should see the
synchonization in the stack allocator.

A regression test is added to misc/cgo/testsanitizer.

Fixes #51676.
Fixes #59294.
Fixes #59678.

Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/485500
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
2023-04-26 19:25:46 +00:00
Paul E. Murphy
d816f85f78 cmd/link/internal/loadelf: set AttrExternal on text section symbols
PPC64 processes external object relocations against the section
symbols. This needs to be set correctly to determine the type of
PLT stub to generate when both Go and External code make PLT calls.

Change-Id: I5abdd5a0473866164083c33e80324dffcc1707f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/488895
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-26 15:25:47 +00:00
Than McIntosh
39957b5d89 coverage: fix count vs emit discrepancy in coverage counter data writing
This patch revises the way coverage counter data writing takes place
to avoid problems where useful counter data (for user-written functions)
is skipped in favor of counter data from stdlib functions that are
executed "late in the game", during the counter writing process itself.

Reading counter values from a running "--coverpkg=all" program is an
inherently racy operation; while the the code that scans the coverage
counter segment is reading values, the program is still executing,
potentially updating those values, and updates can include execution
of previously un-executed functions. The existing counter data writing
code was using a two-pass model (initial sweep over the counter
segment to count live functions, second sweep to actually write data),
and wasn't properly accounting for the fact that the second pass could
see more functions than the first.

In the bug in question, the first pass discovered an initial set of
1240 functions, but by the time the second pass kicked in, several
additional new functions were also live. The second pass scanned the
counter segment again to write out exactly 1240 functions, but since
some of the counters for the newly executed functions were earlier in
the segment (due to linker layout quirks) than the user's selected
function, the sweep terminated before writing out counters for the
function of interest.

The fix rewrites the counter data file encoder to make a single sweep
over the counter segment instead of using a two-pass scheme.

Fixes #59563.

Change-Id: I5e908e226bb224adb90a2fb783013e52deb341da
Reviewed-on: https://go-review.googlesource.com/c/go/+/484535
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
2023-04-26 12:44:34 +00:00
Than McIntosh
7b89531860 internal/coverage/slicewriter: fix off-by-1 error in seek utilities
The slicewriter Seek method was being too restrictive on offsets
accepted, due to an off-by-one problem in the error checking code.
This fixes the problem and touches up the unit tests.

Change-Id: I75d6121551de19ec9275f0e331810db231db6ea9
Reviewed-on: https://go-review.googlesource.com/c/go/+/488116
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-26 12:44:26 +00:00
ioworker0
ada0eec827 runtime: add a alignment check
The Linux implementation requires that the address addr be
page-aligned, and allows length to be zero.

See Linux notes:
https://man7.org/linux/man-pages/man2/madvise.2.html

Change-Id: Ic49960c32991ef12f23de2de76e9689567c82d03
GitHub-Last-Rev: 35e7f8e5cc
GitHub-Pull-Request: golang/go#59793
Reviewed-on: https://go-review.googlesource.com/c/go/+/488015
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2023-04-26 04:06:08 +00:00
Ian Lance Taylor
3c59639b90 crypto/sha512: add WriteString and WriteByte method
This can reduce allocations when hashing a string or byte
rather than []byte.

For #38776

Change-Id: I4926ae2749f6b167edbebb73d8f68763ffb2f0c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/483816
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-04-25 22:06:33 +00:00
Ian Lance Taylor
6c1792d1ff crypto/sha1: add WriteString and WriteByte method
This can reduce allocations when hashing a string or byte
rather than []byte.

For #38776

Change-Id: I7c1fbdf15abf79d2faf360f75adf4bc550a607e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/483815
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
2023-04-25 22:06:06 +00:00
Ian Lance Taylor
bb079efbdc crypto/sha256: add WriteString and WriteByte method
This can reduce allocations when hashing a string or byte
rather than []byte.

For #38776

Change-Id: I1c6dd1bc018220784a05939e92b47558c0562110
Reviewed-on: https://go-review.googlesource.com/c/go/+/481478
Reviewed-by: Joel Sing <joel@sing.id.au>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-04-25 22:05:33 +00:00
Austin Clements
ce0b914312 cmd/dist: actually only compile tests with -compile-only
Currently, "dist test -compile-only" still runs the test binaries,
just with -run=^$ so no tests are run. It does this because, until
recently, "go test -c" would fail if passed multiple test packages.
But this has some unexpected consequences: init code still runs,
TestMain still runs, and we generally can't test cross-compiling of
tests.

Now that #15513 is fixed, we can pass multiple packages to "go test
-c". Hence, this CL make dist just use "go test -c" as one would
expect.

Found in the course of working on #37486, though it doesn't really
affect that.

Change-Id: If7d3c72c9e0f74d4ea0dd422411e5ee93b314be4
Reviewed-on: https://go-review.googlesource.com/c/go/+/488275
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
2023-04-25 19:49:28 +00:00
qmuntal
715d53c090 runtime: fallback to TEB arbitrary pointer when TLS slots are full
The Go runtime allocates the TLS slot in the TEB TLS slots instead of
using the TEB arbitrary pointer. See CL 431775 for more context.

The problem is that the TEB TLS slots array only has capacity for 64
indices, allocating more requires some complex logic that we don't
support yet.

Although the Go runtime only allocates one index, a Go DLL can be
loaded in a process with more than 64 TLS slots allocated,
in which case it abort.

This CL avoids aborting by falling back to the older behavior, that
is to use the TEB arbitrary pointer.

Fixes #59213

Change-Id: I39c73286fe2da95aa9c5ec5657ee0979ecbec533
Reviewed-on: https://go-review.googlesource.com/c/go/+/486816
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-25 15:37:00 +00:00
Nayef Ghattas
14f833f117 runtime/metrics: specify that bucket counts increase monotonically for histogram metrics
Make it explicit in the documentation that the histogram metrics
are cumulative (i.e. each bucket count increases monotonically).

Change-Id: I89119ba816ac46a63f36e607e695fad3695057ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/487315
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-25 14:24:19 +00:00
fanzha02
e4b03f9425 internal/cpu: add a detection for Neoverse(N2, V2) cores
The memmove implementation relies on the variable
runtime.arm64UseAlignedLoads to select fastest code
path. Considering Neoverse N2 and V2 cores prefer aligned
loads, this patch adds code to detect them for
memmove performance.

And this patch uses a new variable ARM64.IsNeoverse to
represent all Neoverse cores, removing the more specific
versions.

Change-Id: I9e06eae01a0325a0b604ac6af1e55711dd6133f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/487815
Reviewed-by: Joel Sing <joel@sing.id.au>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Fannie Zhang <Fannie.Zhang@arm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-25 14:08:20 +00:00
Alan Donovan
a1284d0185 go/ast: add IsGenerated(*File) predicate
See https://go.dev/s/generatedcode for spec.

Fixes #28089

Change-Id: Ic9bb138bdd180f136f9e8e74e187319acca5dbac
Reviewed-on: https://go-review.googlesource.com/c/go/+/487935
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2023-04-25 13:57:33 +00:00
cui fliter
22d94dfdc8 html/template: fix unavailable url
The previous link is no longer accessible. use latest link.

Change-Id: I76411ee00785f3d92014c5012e4efb446924adaf
Reviewed-on: https://go-review.googlesource.com/c/go/+/487835
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Andrew Polukhin <andrewmathematics2003@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: shuang cui <imcusg@gmail.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-04-25 01:14:08 +00:00
Ian Lance Taylor
f00e947cdf runtime: add raceFiniLock to lock ranking
Also preserve the PC/SP in reentersyscall when doing lock ranking.
The test is TestDestructorCallbackRace with the staticlockranking
experiment enabled.

For #59711

Change-Id: I87ac1d121ec0d399de369666834891ab9e7d11b0
Reviewed-on: https://go-review.googlesource.com/c/go/+/487955
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-04-24 21:37:06 +00:00
Bryan C. Mills
3f987ae61d internal/testenv: actually try to exec on ios and wasm
Due to a stray edit in CL 486275, the assignment to tryExecOk
in tryExec on ios would be immediately overwritten back to false.
This change fixes the stray edit.

Change-Id: I4f45fbf130dc912305e5f453b0d1a622ba199ad4
Reviewed-on: https://go-review.googlesource.com/c/go/+/488076
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-04-24 20:39:25 +00:00
Roland Shoemaker
a8af76284d archive/zip: reject overflowing directorySize & directoryOffset
We added a check for incorrect baseOffset in CL 408734, but in doing so
we introduced a panic when directoryOffset overflowed a int64. The zip
spec uses uint64, but since io.SectionReader requires int64 we convert,
and possibly introduce an overflow. If offset < 0 && size-offset < 0,
SectionReader will panic when we attempt to read from it.

Since it's extremely unlikely we're ever going to process a zip file
larger than 1<<63-1 byte, just limit directory size and offset to the
max int64.

Change-Id: I1aaa755cf4da927a6e12ef59f97dfc83a3426d86
Reviewed-on: https://go-review.googlesource.com/c/go/+/488195
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Roland Shoemaker <roland@golang.org>
2023-04-24 20:28:37 +00:00
Sameer Ajmani
1d00dc3985 doc: fix typo in Go 1.21 release notes
Change-Id: Ib32567fdd12079cd171a4e1bc118ce27d8ce2a5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/488035
Run-TryBot: Sameer Ajmani <sameer@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-24 18:13:34 +00:00
Bryan C. Mills
5256f90c98 runtime: fix CgoRaceprof and CgoRaceSignal build failures
TestRaceProf and TestRaceSignal were changed to run on all platforms
that support the race detector as of CL 487575, but the testprogcgo
source files needed to run the test rely on POSIX threads and were
still build-constrained to only linux/amd64 and freebsd/amd64.

Since the C test program appears to require only POSIX APIs, update
the constraint to build the source file on all Unix platforms, and
update the tests to skip on Windows.

This may slightly increase testprogcgo build time on Unix platforms
that do not support the race detector.

Change-Id: I704dd496d475a3cd2e2da2a09c7d2e3bb8e96d02
Reviewed-on: https://go-review.googlesource.com/c/go/+/488115
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
2023-04-24 18:13:14 +00:00
Jonathan Amsterdam
c80cedec93 testing/slogtest: tests for slog handlers
Add a package for testing that a slog.Handler implementation
satisfies that interface's documented requirements.

Code copied from x/exp/slog/slogtest.

Updates #56345.

Change-Id: I89e94d93bfbe58e3c524758f7ac3c3fba2a2ea96
Reviewed-on: https://go-review.googlesource.com/c/go/+/487895
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
2023-04-24 18:07:26 +00:00
Cherry Mui
ddd822e5ca cmd/link: don't sort data symbols by name
For data symbols, we currently sort them by size, then by name if
the size is the same. Sorting by name is not really necessary.
Instead, we sort by symbol index. Like name, the symbol index is
deterministic, and pretty stable if only a small portion of the
input is changed, and also naturally partitioned by packages. This
reduces the CPU time for reading the symbol names and comparing
strings.

Linking cmd/compile (on macOS/amd64),

Dodata    57.2ms ± 6%    54.5ms ± 4%   -4.74%  (p=0.000 n=19+17)

Change-Id: I1c4f2b83dbbb4b984b2c8ab4a7e8543b9f7f22b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/487515
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-24 16:50:28 +00:00
Cherry Mui
d33a5136e1 cmd/link: use uint32 as symbol index
Currently, a symbol's global index, the Sym type, is defined as an
int, which is 64-bit on 64-bit machines. We're unlikely to have
more than 4 billion symbols in the near future. Even if we will,
we will probably hit some other limit (e.g. section size) before
the symbol number limit. Use a 32-bit type to reduce memory usage.

E,g, linking cmd/compile in external linking mode (on macOS/amd64)

Munmap_GC    43.2M ± 0%     35.5M ± 1%   -17.74%  (p=0.000 n=16+20)

This brings the memory usage back before the previous CL, and even
lower.

Change-Id: Ie185f1586638fe70d8121312bfa9410942d518c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/487416
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-04-24 16:49:08 +00:00