1
0
mirror of https://github.com/golang/go synced 2024-11-18 10:44:45 -07:00
Commit Graph

88 Commits

Author SHA1 Message Date
Michael Pratt
7b874619be runtime/cgo: store M for C-created thread in pthread key
This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and
CL 485316 incorporated.

CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 482975 is a followup fix to a C declaration in testprogcgo.
CL 485315 is a followup fix for x_cgo_getstackbound on Illumos.
CL 485316 is a followup cleanup for ppc64 assembly.

[Original CL 481061 description]

This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.

CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.

[Original CL 392854 description]

In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key,  only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

[CL 479915 description]

Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

[CL 485500 description]

CL 479915 passed the G to _cgo_getstackbound for direct updates to
gp.stack.lo. A G can be reused on a new thread after the previous thread
exited. This could trigger the C TSAN race detector because it couldn't
see the synchronization in Go (lockextra) preventing the same G from
being used on multiple threads at the same time.

We work around this by passing the address of a stack variable to
_cgo_getstackbound rather than the G. The stack is generally unique per
thread, so TSAN won't see the same address from multiple threads. Even
if stacks are reused across threads by pthread, C TSAN should see the
synchonization in the stack allocator.

A regression test is added to misc/cgo/testsanitizer.

Fixes #51676.
Fixes #59294.
Fixes #59678.

Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/485500
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
2023-04-26 19:25:46 +00:00
Michael Pratt
94850c6f79 Revert "runtime/cgo: store M for C-created thread in pthread key"
This reverts CL 481061.

Reason for revert: When built with C TSAN, x_cgo_getstackbound triggers
race detection on `g->stacklo` because the synchronization is in Go,
which isn't instrumented.

For #51676.
For #59294.
For #59678.

Change-Id: I38afcda9fcffd6537582a39a5214bc23dc147d47
Reviewed-on: https://go-review.googlesource.com/c/go/+/485275
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-04-17 18:47:08 +00:00
Ian Lance Taylor
7b1e0bb79c misc/cgo: gofmt
Change-Id: I5d02279d0593a8368b2f249a6b53650b89aed7b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/482275
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-10 22:32:33 +00:00
doujiang24
ccad8a9f9c runtime/cgo: store M for C-created thread in pthread key
This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.

CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.

[Original CL 392854 description]

In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key,  only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

[CL 479915 description]

Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

Fixes #51676.
Fixes #59294.

Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494
Reviewed-on: https://go-review.googlesource.com/c/go/+/481061
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-03 18:34:11 +00:00
Cherry Mui
63ef9059a2 Revert "runtime: get a better g0 stack bound in needm"
This reverts CL 479915.

Reason for revert: breaks a lot google internal tests.

Change-Id: I13a9422e810af7ba58cbf4a7e6e55f4d8cc0ca51
Reviewed-on: https://go-review.googlesource.com/c/go/+/481055
Reviewed-by: Chressie Himpel <chressie@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31 18:42:48 +00:00
Cherry Mui
443eb9757c runtime: get a better g0 stack bound in needm
Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

For #59294.

Change-Id: Ie52a8f931e0648d8753e4c1dbe45468b8748b527
Reviewed-on: https://go-review.googlesource.com/c/go/+/479915
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-03-30 23:23:55 +00:00
Bryan C. Mills
ab0f0459da cmd/dist,internal/platform: reenable the c-archive build mode on ios
Also fix misc/cgo/testcarchive to provide a missing CoreFoundation
dependency at link time.

Fixes #58221.
Updates #58172.
Updates #58225.

Change-Id: Ib8b6e52ed2914596615da4c427df2fe984722de6
Reviewed-on: https://go-review.googlesource.com/c/go/+/463752
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-01 19:47:35 +00:00
hopehook
351037d16c misc/cgo: replace os.SEEK_SET with io.SeekStart
Since os.SEEK_SET was deprecated, use io.SeekStart instead.

Change-Id: I11ae496b071ab35412403ff73e52f3da73d5b120
Reviewed-on: https://go-review.googlesource.com/c/go/+/431096
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-19 16:55:26 +00:00
Ian Lance Taylor
d05ce23756 misc/cgo/testcarchive: permit SIGQUIT for TestSignalForwardingExternal
Occasionally the signal will be sent to a Go thread, which will cause
the program to exit with SIGQUIT rather than SIGSEGV.

Add TestSignalForwardingGo to test the case where the signal is
expected to be delivered to a Go thread.

This is a roll forward of CL 419014 which was rolled back in CL 424954.
This CL differs from 419014 in that it skips TestSignalForwardingGo
on darwin-amd64.

Fixes #53907

Change-Id: I5df3fd610c068df3bd48d9b3d7a9379248b97999
Reviewed-on: https://go-review.googlesource.com/c/go/+/425002
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-08-19 23:07:11 +00:00
Cuong Manh Le
bc69ad3a77 Revert "misc/cgo/testcarchive: permit SIGQUIT for TestSignalForwardingExternal"
This reverts CL 419014.

Reason for revert: broke darwin-amd64 builders

Change-Id: I77838e696a55c415d322ebb9846503de25b546d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/424954
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Joedian Reid <joedian@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-08-19 19:35:55 +00:00
Ian Lance Taylor
a409356c54 misc/cgo/testcarchive: permit SIGQUIT for TestSignalForwardingExternal
Occasionally the signal will be sent to a Go thread, which will cause
the program to exit with SIGQUIT rather than SIGSEGV.

Add TestSignalForwardingGo to test the case where the signal is
expected to be delivered to a Go thread.

Fixes #53907

Change-Id: Iaefb964c2be4a815c11c507fa89648f8a7740ba9
Reviewed-on: https://go-review.googlesource.com/c/go/+/419014
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-08-19 03:09:05 +00:00
Russ Cox
1b7e71e8ae all: disable tests that fail on Alpine
These changes are enough to pass all.bash using the
disabled linux-amd64-alpine builder via debugnewvm.

For #19938.
For #39857.

Change-Id: I7d160612259c77764b70d429ad94f0864689cdce
Reviewed-on: https://go-review.googlesource.com/c/go/+/419995
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-08-02 17:23:42 +00:00
Bryan C. Mills
ad641e8521 misc/cgo/testcarchive: don't rely on an erroneous install target in tests
Non-main packages in module mode should not be installed to
GOPATH/pkg, but due to #37015 they were installed there anyway.

This change switches the 'go install' command in TestPIE to instead
use 'go build', and switches TestInstall and TestCachedInstall
(which appear to be explicitly testing 'go install') to explicitly
request GOPATH mode (which does have a well-defined install target).

For #37015.

Change-Id: Ifb24657d2781d1e35cf40078e8e3ebf56aab9cc8
Reviewed-on: https://go-review.googlesource.com/c/go/+/416954
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-07-11 18:57:03 +00:00
Bryan C. Mills
3c4c10ea8c misc/cgo: fix aliasing bugs in parallel tests that append to shared slices
These tests use a slice to represent the base C compiler command (with
flags). Appending to that slice can cause subtle aliasing bugs, such
as commands that silently corrupt the arguments of other concurrent
commands in parallel tests.

In this change, we explicitly reduce the capacity of the command slice
to force appends to it to always allocate unique new slices.

Fixes #49693

Change-Id: Ide466bf65f12cb6cead3dcba69f513cccb60a160
Reviewed-on: https://go-review.googlesource.com/c/go/+/383754
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-02-07 17:51:57 +00:00
Ian Lance Taylor
cfb0cc3552 cmd/link: use SHT_INIT_ARRAY for .init_array section
Fixes #50295

Change-Id: If55ebcd5f2af724da7c9c744458a56d21a7ddde7
Reviewed-on: https://go-review.googlesource.com/c/go/+/373734
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-12-21 18:34:03 +00:00
Bryan C. Mills
d01cd8fa1d misc/cgo/testcarchive: log command output more consistently
Also check that executables exist immediately after building them
in parallel tests.

The parallel tests in this package occasionally fail with
"no such file or directory", implying that either the build
command failed to actually write out the binary or something
concurrently deleted it.

This is purely a shot in the dark, but I'm hoping that perhaps
the stderr output from one of these commands will shed some
light on the underlying failure mode.

For #49693

Change-Id: I2e768190c56053550879b89a3ac88c027d4741dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/369034
Trust: Bryan Mills <bcmills@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2021-12-16 22:15:57 +00:00
Ian Lance Taylor
a4b2c579e9 runtime: install sigPreempt signal handler for c-archive/c-shared
Fixes #49288

Change-Id: I7bfcbecbefa68871a3e556935a73f241fff44c0e
Reviewed-on: https://go-review.googlesource.com/c/go/+/360861
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-11-03 00:49:50 +00:00
Brad Fitzpatrick
13a4e8c41c all: simplify the spelling of Linux
The prefix didn't help clarify anything.

Change-Id: I897fd4022ce9df42a548b15714e4b592618ca547
Reviewed-on: https://go-review.googlesource.com/c/go/+/309573
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-04-13 13:56:21 +00:00
KimMachineGun
a040ebeb98 all: update references to symbols moved from io/ioutil to io
Update references missed in CL 263142.

For #41190

Change-Id: I778760a6a69bd0440fec0848bdef539c9ccb4ee1
GitHub-Last-Rev: dda42b09ff
GitHub-Pull-Request: golang/go#42874
Reviewed-on: https://go-review.googlesource.com/c/go/+/273946
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Cherry Zhang <cherryyz@google.com>
2021-04-05 17:51:15 +00:00
Ian Lance Taylor
73eb27bd3b misc/cgo/testcarchive: don't use == for string equality in C code
For https://gcc.gnu.org/PR99553

Change-Id: I29a7fbfd89963d4139bc19af99330d70567938ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/300993
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
2021-03-13 03:16:55 +00:00
Cherry Zhang
ed301733bb misc/cgo/testcarchive: remove special flags for Darwin/ARM
The original Darwin/ARM port is gone. For ARM64, it works fine
without the flags on macOS/ARM64. Remove the flags.

Change-Id: I9cc00c49dd71376dd9c52abb78c2d8cec656b3db
Reviewed-on: https://go-review.googlesource.com/c/go/+/280157
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-12-30 22:39:59 +00:00
Cherry Zhang
db428ad7b6 all: enable more tests on macOS/ARM64
Updates #38485.

Change-Id: Iac96f5ffe88521fcb11eab306d0df6463bdce046
Reviewed-on: https://go-review.googlesource.com/c/go/+/256920
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-06 21:56:24 +00:00
Cherry Zhang
a413908dd0 all: add GOOS=ios
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin"
build tag, like GOOS=android matches "linux" and GOOS=illumos
matches "solaris". Only ios/arm64 is supported (ios/amd64 is
not).

GOOS=ios and GOOS=darwin remain essentially the same at this
point. They will diverge at later time, to differentiate macOS
and iOS.

Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"),
except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"),
it remains GOOS=="darwin".

Updates #38485.

Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c
Reviewed-on: https://go-review.googlesource.com/c/go/+/254740
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-09-23 18:12:59 +00:00
Austin Clements
f7e6ab44b4 all: remove scattered remnants of darwin/arm
This removes all conditions and conditional code (that I could find)
that depended on darwin/arm.

Fixes #35439 (since that only happened on darwin/arm)
Fixes #37611.

Change-Id: Ia4c32a5a4368ed75231075832b0b5bfb1ad11986
Reviewed-on: https://go-review.googlesource.com/c/go/+/227198
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-08 18:35:49 +00:00
Bryan C. Mills
476395cb3e misc: remove use of relative directories in overlayDir functions
It turns out that the relative-path support never worked in the first
place.

It had been masked by the fact that we ~never invoke overlayDir with
an absolute path, which caused filepath.Rel to always return an error,
and overlayDir to always fall back to absolute paths.

Since the absolute paths seem to be working fine (and are simpler),
let's stick with those. As far as I can recall, the relative paths
were only a space optimization anyway.

Updates #28387
Updates #30316

Change-Id: Ie8cd28f3c41ca6497ace2799f4193d7f5dde7a37
Reviewed-on: https://go-review.googlesource.com/c/go/+/208481
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-11-25 16:26:15 +00:00
Bryan C. Mills
c02f3b86b4 misc/cgo/testcarchive: avoid writing to GOROOT in tests
Also add a -testwork flag to facilitate debugging the test itself.

Three of the tests of this package invoked 'go install -i
-buildmode=c-archive' in order to generate an archive as well as
multiple C header files.

Unfortunately, the behavior of the '-i' flag is inappropriately broad
for this use-case: it not only generates the library and header files
(as desired), but also attempts to install a number of (unnecessary)
archive files for transitive dependencies to
GOROOT/pkg/$GOOS_$GOARCH_shared, which may not be writable — for
example, if GOROOT is owned by the root user but the test is being run
by a non-root user.

Instead, for now we generate the header files for transitive dependencies
separately by running 'go tool cgo -exportheader'.

In the future, we should consider how to improve the ergonomics for
generating transitive header files without coupling that to
unnecessary library installation.

Updates #28387
Updates #30316
Updates #35715

Change-Id: I3d483f84e22058561efe740aa4885fc3f26137b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/208117
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-22 15:32:23 +00:00
Bryan C. Mills
6375fe4b9e misc: ensure that test overlay directories are writable
Otherwise, the test cannot create new files in the directory.

Updates #32407
Updates #30316

Change-Id: Ief0df94a202be92f57d458d4ab4e4daa9ec189b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/206458
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-11 17:59:46 +00:00
Ian Lance Taylor
fb29e2252c misc/cgo/testcarchive: add missing exit(0) in main7.c
Fixes #35327

Change-Id: I3726bfad24851a0bef8891014f7c5a7c48352307
Reviewed-on: https://go-review.googlesource.com/c/go/+/205077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-04 05:27:25 +00:00
Ian Lance Taylor
8de0bb77eb runtime: clear preemptStop in dropm
Updates #10958
Updates #24543
Fixes #35294

Change-Id: I60f024d08451565df6d9751dab9832b50cbf637a
Reviewed-on: https://go-review.googlesource.com/c/go/+/204957
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02 05:52:33 +00:00
Elias Naur
2afe9d4dec runtime: don't forward SIGPIPE on macOS
macOS and iOS deliver SIGPIPE signals to the main thread and not
the thread that raised it by writing to a closed socket or pipe.

SIGPIPE signals can be suppressed for sockets with the SO_NOSIGPIPE
option, but there is no similar option for pipes. We have no other
choice but to never forward SIGPIPE on macOS.

This is a fixup of reverted CL 188297.

Fixes #33384

Change-Id: I09b258b078857ad3b22025bc2902d1b12d2afd92
Reviewed-on: https://go-review.googlesource.com/c/go/+/191785
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-08-31 06:19:40 +00:00
Daniel Martí
647dc1afc5 Revert "runtime: don't forward SIGPIPE on macOS"
This reverts CL 188297.

Reason for revert: broke multiple of the darwin builders.

Fixes #33943.

Change-Id: Iacff98d1450edc70402dc7a220d16fcd73337c9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/191784
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-08-29 15:37:46 +00:00
Elias Naur
d56a86e01f runtime: don't forward SIGPIPE on macOS
macOS and iOS deliver SIGPIPE signals to the main thread and not
the thread that raised it by writing to a closed socket or pipe.

SIGPIPE signals can be suppressed for sockets with the SO_NOSIGPIPE
option, but there is no similar option for pipes. We have no other
choice but to never forward SIGPIPE on macOS.

Fixes #33384

Change-Id: Ice3de75b121f00006ee11c26d560e619536460be
Reviewed-on: https://go-review.googlesource.com/c/go/+/188297
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-08-29 12:51:31 +00:00
Russ Cox
a6a0ed3c47 misc/cgo: disable testcarchive, testplugin during -short test (including all.bash)
These tests take 20 seconds each to rebuild the entire world
in their respective modes. That's too much for short mode,
including all.bash on developer machines.

Keep doing it on builders and if someone runs 'go test' by hand
in that directory.

For #26473.

Change-Id: I3dc6955bc3aa7a20fd170efcde72a7d19b37bdbf
Reviewed-on: https://go-review.googlesource.com/c/go/+/177417
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:25:04 +00:00
Elias Naur
07f689420a misc/cgo/testcarchive: skip TestExtar on self-hosted iOS
iOS cannot (directly) run shell scripts.

Updates #31722

Change-Id: I69473e9339c50a77338d391c73b4e146bce3fa89
Reviewed-on: https://go-review.googlesource.com/c/go/+/174700
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-01 16:55:33 +00:00
Joshua M. Clulow
f686a2890b all: add new GOOS=illumos, split out of GOOS=solaris
Like GOOS=android which implies the "linux" build tag, GOOS=illumos
implies the "solaris" build tag. This lets the existing ecosystem of
packages still work on illumos, but still permits packages to start
differentiating between solaris and illumos.

Fixes #20603

Change-Id: I8f4eabf1a66060538dca15d7658c1fbc6c826622
Reviewed-on: https://go-review.googlesource.com/c/go/+/174457
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-30 16:59:13 +00:00
Ian Lance Taylor
2da9659158 misc/cgo/testcarchive: skip TestSignalForwardingExternal on darwin/amd64
On darwin/amd64 the runtime method sigctxt.fixsigcode changes SIGSEGV
signals so that they are never marked SI_USER. CL 169120 changed the
signal handler to call fixsigcode even when the signal is delivered to
a non-Go thread. This breaks TestSignalForwardingExternal, so skip it.

Change-Id: I6740fb5a8f4f854ca69793537a983a696da3b495
Reviewed-on: https://go-review.googlesource.com/c/go/+/170446
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-04-02 22:58:55 +00:00
Clément Chigot
3aacfce6cf runtime, cmd/dist, misc/cgo: enable c-archive for aix/ppc64
Change-Id: Ib9a40d5596f5735a00483e2d2db965402f05671b
Reviewed-on: https://go-review.googlesource.com/c/go/+/169120
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-04-02 16:03:24 +00:00
alkesh26
c7f69a2897 misc: fix typos in various docs
Change-Id: Ib03d7e5686e510152042e403b00fb2d65572f393
GitHub-Last-Rev: 57aeedf077
GitHub-Pull-Request: golang/go#30156
Reviewed-on: https://go-review.googlesource.com/c/go/+/161723
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-03-07 07:30:06 +00:00
Bryan C. Mills
b5a68a9e41 misc/cgo: skip cgotest.TestCrossPackageTests on iOS and set PWD
I hope that this will fix the tests on iOS, but 'gomote create' isn't
giving me an instance I can test with. (Please patch and test before
approving.)

Updates #15919
Updates #30228

Change-Id: I1b7cd30d5b127a1ad3243b329fa005d229f69a24
Reviewed-on: https://go-review.googlesource.com/c/163726
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-02-26 13:29:07 +00:00
Bryan C. Mills
13d9a29060 misc/cgo/testcarchive: fix tests in module mode
Updates #30228

Change-Id: I830e3c83416b2e5744f30d1a903a74c50462716b
Reviewed-on: https://go-review.googlesource.com/c/163210
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-02-22 21:56:56 +00:00
Ian Lance Taylor
0829e1b757 misc/cgo/testcarchive: make the tests work when using gccgo
Change-Id: I62a7a8ebbbc1f1a266234b53680768da157b2df5
Reviewed-on: https://go-review.googlesource.com/130416
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-08-21 15:03:10 +00:00
Michael Munday
0268a63ffa misc/cgo/testcarchive: increase timeout duration in TestOsSignal
This test is slightly flaky on the s390x builder and I suspect that
the 100ms timeout is a little too optimistic when the VM is starved.
Increase the timeout to 5s to match the other part of the test.

Fixes #26231.

Change-Id: Ia6572035fb3efb98749f2c37527d250a4c779477
Reviewed-on: https://go-review.googlesource.com/122315
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-07-06 13:55:35 +00:00
Ian Lance Taylor
be012e1e2e cmd/cgo: don't use absolute paths in the export header file
We were using absolute paths in the #line directives in the export
header file. This makes the header file change if you move GOPATH.
The absolute paths aren't helpful for the final user, which is some C
program elsewhere.

Fixes #24945

Change-Id: I2da32c9b477df578bd5087435a03fe97abe462e3
Reviewed-on: https://go-review.googlesource.com/108315
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-04-25 00:36:09 +00:00
Tim Wright
88129f0cb2 all: enable c-shared/c-archive support for freebsd/amd64
Fixes #14327
Much of the code is based on the linux/amd64 code that implements these
build modes, and code is shared where possible.

Change-Id: Ia510f2023768c0edbc863aebc585929ec593b332
Reviewed-on: https://go-review.googlesource.com/93875
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-03-21 21:56:20 +00:00
Ian Lance Taylor
1c0d5427de cmd/go: correct buildmode test (from "c-header" to "c-shared")
Change-Id: I8688a47ae83f6719f6134c64fb3d3f2aa275c641
Reviewed-on: https://go-review.googlesource.com/87158
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2018-01-11 19:24:35 +00:00
Russ Cox
8f70e1f8a9 cmd/go: do not install dependencies during "go install"
This CL makes "go install" behave the way many users expect:
install only the things named on the command line.
Future builds still run as fast, thanks to the new build cache (CL 75473).
To install dependencies as well (the old behavior), use "go install -i".

Actual definitions aside, what most users know and expect of "go install"
is that (1) it installs what you asked, and (2) it's fast, unlike "go build".
It was fast because it installed dependencies, but installing dependencies
confused users repeatedly (see for example #5065, #6424, #10998, #12329,
"go build" and "go test" so that they could be "fast" too, but that only
created new opportunities for confusion. We also had to add -installsuffix
and then -pkgdir, to allow "fast" even when dependencies could not be
installed in the usual place.

The recent introduction of precise content-based staleness logic means that
the go command detects the need for rebuilding packages more often than it
used to, with the consequence that "go install" rebuilds and reinstalls
dependencies more than it used to. This will create more new opportunities
for confusion and will certainly lead to more issues filed like the ones
listed above.

CL 75743 introduced a build cache, separate from the install locations.
That cache makes all operations equally incremental and fast, whether or
not the operation is "install" or "build", and whether or not "-i" is used.

Installing dependencies is no longer necessary for speed, it has confused
users in the past, and the more accurate rebuilds mean that it will confuse
users even more often in the future. This CL aims to end all that confusion
by not installing dependencies by default.

By analogy with "go build -i" and "go test -i", which still install
dependencies, this CL introduces "go install -i", which installs
dependencies in addition to the things named on the command line.

Fixes #5065.
Fixes #6424.
Fixes #10998.
Fixes #12329.
Fixes #18981.
Fixes #22469.

Another step toward #4719.

Change-Id: I3d7bc145c3a680e2f26416e182fa0dcf1e2a15e5
Reviewed-on: https://go-review.googlesource.com/75850
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2017-11-03 22:09:46 +00:00
Lynn Boger
c15c44ec48 misc/cgo/testcarchive: use -no-pie where needed
Starting in gcc 6, -pie is passed to the linker by default
on some platforms, including ppc64le. If the objects
being linked are not built for -pie then in some cases the
executable could be in error. To avoid that problem, -no-pie
should be used with gcc to override the default -pie option
and generate a correct executable that can be run without error.

Fixes #22126

Change-Id: I4a052bba8b9b3bd6706f5d27ca9a7cebcb504c95
Reviewed-on: https://go-review.googlesource.com/70072
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-11 21:01:45 +00:00
Russ Cox
4e8be99590 cmd/go: clean up compile vs link vs shared library actions
Everything got a bit tangled together in b.action1.
Try to tease things apart again.

In general this is a refactoring of the existing code, with limited
changes to the effect of the code.

The main additional change is to complete a.Deps for link actions.
That list now directly contains all the inputs the linker will attempt
to read, eliminating the need for a transitive traversal of the entire
action graph to find those. The comepleteness of a.Deps will be
important when we eventually use it to decide whether an cached
action output can be reused.

all.bash passes, but it's possible I've broken some subtety of
buildmode=shared again. Certainly that code took the longest
to get working.

Change-Id: I34e849eda446dca45a9cfce02b07bec6edb6d0d4
Reviewed-on: https://go-review.googlesource.com/69831
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2017-10-11 17:47:09 +00:00
Elias Naur
07c01e3968 misc/cgo/testcarchive: skip flaky SIGPROF test on darwin
Updates #19320.

Change-Id: Id38df033e3f0873986e668c8ff3855b6e08407a9
Reviewed-on: https://go-review.googlesource.com/69114
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2017-10-09 06:02:50 +00:00
Ian Lance Taylor
994b7eebc7 misc/cgo/testcarchive: fix go env error message
Add a missing newline.  Don't panic on an unexpected error type.

Change-Id: I82a4b12c498fbfdf4972737329631c0c02540005
Reviewed-on: https://go-review.googlesource.com/44092
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-05-24 22:27:16 +00:00