The PATH variable is required to run the testenv tests.
Set it for all the runtime invocations where we don't
already set it by inheriting from the environment.
For #59583
For #59907
For #60097
Change-Id: If582dd8f086e3f40bc58d555f6034dcffe6f8e5f
Reviewed-on: https://go-review.googlesource.com/c/go/+/498616
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Implements net.FileListener and net.FileConn for wasip1.
net.FileListener can be used with a pre-opened socket. If the WASM
module knows the file descriptor, a listener can be constructed with:
l, err := net.FileListener(os.NewFile(fd, ""))
If the WASM module does not know the file descriptor, but knows that at
least one of the preopens is a socket, it can find the file descriptor
and construct a listener like so:
func findListener() (net.Listener, error) {
// We start looking for pre-opened sockets at fd=3 because 0, 1,
// and 2 are reserved for stdio. Pre-opened directories also
// start at fd=3, so we skip fds that aren't sockets. Once we
// reach EBADF we know there are no more pre-opens.
for preopenFd := uintptr(3); ; preopenFd++ {
l, err := net.FileListener(os.NewFile(preopenFd, ""))
var se syscall.Errno
switch errors.As(err, &se); se {
case syscall.ENOTSOCK:
continue
case syscall.EBADF:
err = nil
}
return l, err
}
}
A similar strategy can be used with net.FileConn and pre-opened
connection sockets.
The wasmtime runtime supports pre-opening listener sockets:
$ wasmtime --tcplisten 127.0.0.1:8080 module.wasm
Change-Id: Iec6ae4ffa84b3753cce4f56a2817e150445db643
Reviewed-on: https://go-review.googlesource.com/c/go/+/493358
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Implements netpoll using WASI's poll_oneoff system call.
This enables non-blocking I/O support for wasip1.
Change-Id: Ie395fa49d651c8b8262d485e2847dd65b0a10bc6
Reviewed-on: https://go-review.googlesource.com/c/go/+/493357
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Julien Fabre <ju.pryz@gmail.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
The existing implementation clears and recreates Javascript
timeouts when Go is called from js, leading to excessive
load on the js scheduler. Instead, we should remove redundant
calls to clearTimeout and refrain from creating new timeouts
if the previous event's timestamp is within 1 millisecond of
our target (the js scheduler's max precision)
Fixes#56100
Change-Id: I42bbed4c2f1fa6579c1f3aa519b6ed8fc003a20c
Reviewed-on: https://go-review.googlesource.com/c/go/+/442995
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
This appears to be a very old wrapper around adb for testing on
Android before we had the go_android_exec wrapper.
Change-Id: I847bb15c98febbcffc063f00719a084e5c99a18b
Reviewed-on: https://go-review.googlesource.com/c/go/+/493604
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
This is the last test run from misc by dist.
For #37486.
Change-Id: I1a70ded29ba0de548c9a16611ba987a258121e80
Reviewed-on: https://go-review.googlesource.com/c/go/+/493606
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
We just moved all of the cgo tests out of misc, and this comment was
already a little stale. Update it.
Change-Id: Ide711cce53dbe6d9675de587c1d73514b063e952
Reviewed-on: https://go-review.googlesource.com/c/go/+/493600
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This moves the misc/swig test to cmd/cgo/internal.
This lets these tests access facilities in internal/testenv. It's also
now just a normal test that can run as part of the cmd tests.
For #37486.
Change-Id: Ibe5026219999d175aa0a310b9886bef3f6f9ed17
Reviewed-on: https://go-review.googlesource.com/c/go/+/492722
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Currently, the misc/swig tests directly use Swig and C++ and will fail
to build if either Swig or a C++ compiler are not present. Typically,
we hide this fact from users because dist test itself checks for Swig
and a C++ compiler before even attempting to run this test, though
users will see this is they try to go test ./... from misc.
However, we're about to move the misc/swig tests into the cmd module,
where they will be much more visible and much more likely to run
unintentionally. To prevent build errors, this CL restructures these
tests into a single pure Go test plus two test packages hidden in
testdata. This is relatively easy to do for this test because there
are only four test cases total. The pure Go test can check for the
necessary build tools before trying to build and run the tests in
testdata. This also gives us the opportunity to move the LTO variant
of these tests out of dist and into the test itself, simplifying dist.
For #37486.
Change-Id: Ibda089b4069e36866cb31867a7006c790be2d8b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/493599
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
We're about to move this package to cmd/cgo/internal, where it will
get caught up in the "CGO_ENABLED=0 go install cmd" done by make.bash.
Currently, building this package with CGO_ENABLED=0 fails because it
contains several source files that don't themselves import "C", but do
import a subdirectory where that package imports "C" and thus has no
exported API.
Fix the CGO_ENABLED=0 build of this package by adding the necessary
cgo build tags. Not all source files need it, but this CL makes
"CGO_ENABLED=0 go test -c" work in this package.
For #37486.
Change-Id: Id137cdfbdd950eea802413536d990ab642ebcd7e
Reviewed-on: https://go-review.googlesource.com/c/go/+/493215
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Vet's cgocall check fails on misc/cgo/test with "possibly passing Go
type with embedded pointer to C". This error is confusing, but the
cgocall check is looking for passing pointers to Go slices to C, which
is exactly what this test is doing. Normally we don't notice this
because vet doesn't run on misc, but we're about to move this test to
cmd/cgo/internal, where vet will start failing.
I'm not sure why we're passing a pointer to a slice here. It's
important that we call a C function with an unsafe.Pointer to memory
containing a pointer to test #25941 and that the result is this call
is then passed to another C function for #28540. This CL maintains
these two properties without the use of a slice.
For #37486.
Change-Id: I672a3c35931a59f99363050498d6f0c80fb6cd98
Reviewed-on: https://go-review.googlesource.com/c/go/+/493137
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
This moves most misc/cgo tests to cmd/cgo/internal. This is mostly a
trivial rename and updating dist/test.go for the new paths, plus
excluding these packages from regular dist test registration. A few
tests were sensitive to what path they ran in, so we update those.
This will let these tests access facilities in internal/testenv.
For #37486.
Change-Id: I3ed417c7c22d9b667f2767c0cb1f59118fcd4af6
Reviewed-on: https://go-review.googlesource.com/c/go/+/492720
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The wasmedge runtime will be used to test our wasip1
implementation against the WASI runtime from wasmedge.org.
For #60097
Change-Id: Ib0e886de46240b4d43d02ec8a7bc7cea0730c162
Reviewed-on: https://go-review.googlesource.com/c/go/+/494120
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
The wasmer runtime will be used to test our wasip1
implementation against the WASI runtime from wasmer.io.
For #59907
Change-Id: Ie7e48c39e03075815ddca46d996b6ec87009b12a
Reviewed-on: https://go-review.googlesource.com/c/go/+/493775
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
This reverts CL 485500.
Reason for revert: This breaks internal tests at Google, see b/280861579 and b/280820455.
Change-Id: I426278d400f7611170918fc07c524cb059b9cc55
Reviewed-on: https://go-review.googlesource.com/c/go/+/492995
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Chressie Himpel <chressie@google.com>
go_android_exec gets the exit status of the process run inside the
Android emulator by sending a small shell script that runs the desired
command and then prints "exitcode=" followed by the exit code. This is
necessary because adb does not reliably pass through the exit status
of the subprocess.
An old bug about this
(https://code.google.com/p/android/issues/detail?id=3254) was closed
in 2016 as fixed in Android N (7.0), but it seems that the adb on the
Android builder at least still sometimes fails to pass through the
exit code.
Unfortunately, this workaround has the effect of injecting
"exitcode=N" into the output of the subprocess it runs, which messes
up tests that are looking for golden output from a subprocess.
Fix this by inserting a filter Writer that looks for the final
"exitcode=N" and strips it from the exec wrapper's own stdout.
For #15919.
This will help us in cleaning up "host tests" for #37486.
Change-Id: I9859f5b215e0ec4a7e33ada04a1857f3cfaf55ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/488975
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
This makes it reasonable to "go build" from this directory by changing
the name of the directory to a more reasonable name for the binary and
dropping the unnecessary "ignore" build tag. The resulting binary
doesn't *quite* have the necessary name for a Go exec wrapper because
that needs to have the form, go_android_$GOARCH_exec, but it's close.
Change-Id: I036cb1af9c034462a952b176a794526fa3ffd1ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/490495
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Auto-Submit: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Currently, in plugin build mode we don't write the build ID. This
is disabled in CL 29394 since plugin is supported on Darwin. Maybe
it caused some problem with the Darwin dynamic linker. But it
seems no problem currently. Enabled it.
Fixes#59845.
Change-Id: I60589ffc7937e4d30055960d391cac1e7cd0cd42
Reviewed-on: https://go-review.googlesource.com/c/go/+/489457
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
- Fall back to 'go env GOROOT' to locate GOROOT if runtime.GOROOT() is
empty (as may be the case if the tool is built with -trimpath).
- Copy all of $GOROOT/android_$GOARCH/bin, not just cmd/go, to
$GOROOT/bin.
- For consistency with CL 404134, place $GOROOT/bin at the beginning
of $PATH, not the end.
- Don't use the install target for the "runtime" package to locate pkg/tool.
As of Go 1.20 "runtime" doesn't have an install directory anyway.
Since the real reason we need pkg/tool is for commands in "cmd",
use an arbitrary command (namely "cmd/compile") to locate it.
- Use 'go list' to determine the package import path for the current
directory, instead of assuming that it is within GOROOT or GOPATH.
(That assumption does not hold in module mode.)
Updates #58775.
Change-Id: If76ff22bce76d05175c40678230f046a4aff0940
Reviewed-on: https://go-review.googlesource.com/c/go/+/472096
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Changkun Ou <mail@changkun.de>
Auto-Submit: Bryan Mills <bcmills@google.com>
This reapplies CL 481061, with the followup fixes in CL 482975, CL 485315, and
CL 485316 incorporated.
CL 481061, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 482975 is a followup fix to a C declaration in testprogcgo.
CL 485315 is a followup fix for x_cgo_getstackbound on Illumos.
CL 485316 is a followup cleanup for ppc64 assembly.
[Original CL 481061 description]
This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.
CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.
[Original CL 392854 description]
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.
When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.
When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.
This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.
This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.
For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
[CL 479915 description]
Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.
This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.
[CL 485500 description]
CL 479915 passed the G to _cgo_getstackbound for direct updates to
gp.stack.lo. A G can be reused on a new thread after the previous thread
exited. This could trigger the C TSAN race detector because it couldn't
see the synchronization in Go (lockextra) preventing the same G from
being used on multiple threads at the same time.
We work around this by passing the address of a stack variable to
_cgo_getstackbound rather than the G. The stack is generally unique per
thread, so TSAN won't see the same address from multiple threads. Even
if stacks are reused across threads by pthread, C TSAN should see the
synchonization in the stack allocator.
A regression test is added to misc/cgo/testsanitizer.
Fixes#51676.
Fixes#59294.
Fixes#59678.
Change-Id: Ic62be31a06ee83568215e875a891df37084e08ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/485500
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Allow switching to wasmtime through the GOWASIRUNTIME variable. This
will allow builders to run the wasip1 standard library tests against
the wasmtime WASI runtime.
For #59583
Change-Id: I4d5200df7bb27b66e041f00e89d4c2e585f5da7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/485615
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Bypass: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Previously, all.bash and all.bat restored the original $PATH before
calling 'dist banner', so that it would do its job of checking whether
the user still needs to add $GOROOT/bin to their $PATH. That worked for
those scripts, but had no effect on make.bash nor make.bat.
Instead of trying to extend that logic to more scripts, change the
approach to provide dist an unmodified copy of $PATH via an internal
to dist environment variable $DIST_UNMODIFIED_PATH. The make.bash and
make.bat scripts happen to use dist env -p to modify $PATH, making it
viable to add the internal variable there instead of in each script.
It currently works by adding semicolon terminators to dist env output
so that make.bash's 'eval $(dist env -p)' works as before but is able to
export DIST_UNMODIFIED_PATH for following dist invocations to observe.
Nothing needs to be done for Windows since its 'set ENV=val' format
already has that effect.
Plan 9 doesn't use the -p flag of dist env, and checks that GOROOT/bin
is bound before /bin rather than looking at the $PATH env var like other
OSes, so it may not have this bug. I don't have easy access to Plan 9
and haven't tried to confirm.
Fixes#42563.
Change-Id: I74691931167e974a930f7589d22a48bb6b931163
Reviewed-on: https://go-review.googlesource.com/c/go/+/485896
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
A //go:debug line mentioning an unknown or retired setting
should be diagnosed as making the program invalid. Do that.
We agreed on this in the proposal but I forgot to implement it.
Change-Id: Ie69072a1682d4eeb6866c02adbbb426f608567c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/476280
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This reverts CL 481061.
Reason for revert: When built with C TSAN, x_cgo_getstackbound triggers
race detection on `g->stacklo` because the synchronization is in Go,
which isn't instrumented.
For #51676.
For #59294.
For #59678.
Change-Id: I38afcda9fcffd6537582a39a5214bc23dc147d47
Reviewed-on: https://go-review.googlesource.com/c/go/+/485275
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Linux kernel on loong64 has no Dup2 syscall support, so we use Dup3 to replace it like arm64 and riscv64.
Updates #53301Fixes#58784
Change-Id: I4e0be140a71b86f4626ed39d76cf3ac78f842018
Reviewed-on: https://go-review.googlesource.com/c/go/+/425478
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: WANG Xuerui <git@xen0n.name>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: abner chenc <chenguoqi@loongson.cn>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: xiaodong liu <teaofmoli@gmail.com>
Reviewed-by: WANG Xuerui <git@xen0n.name>
Change-Id: Iae6ac32db5c2aacb323793a7e0dc34e09648d035
Reviewed-on: https://go-review.googlesource.com/c/go/+/482295
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Change-Id: I5d02279d0593a8368b2f249a6b53650b89aed7b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/482275
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
This script uses Wazero, the open source, zero dependencies
pure Go Wasm and WASI runtime. This is the runtime that allows
the greatest number of standard library tests to pass.
For #58141
Co-authored-by: Richard Musiol <neelance@gmail.com>
Co-authored-by: Achille Roussel <achille.roussel@gmail.com>
Co-authored-by: Julien Fabre <ju.pryz@gmail.com>
Co-authored-by: Evan Phoenix <evan@phx.io>
Change-Id: I789465ae4daf2b380f3c05a9365b8d449c6af56c
Reviewed-on: https://go-review.googlesource.com/c/go/+/479620
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.
CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.
[Original CL 392854 description]
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.
When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.
When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.
This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.
This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.
For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
[CL 479915 description]
Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.
This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.
Fixes#51676.
Fixes#59294.
Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494
Reviewed-on: https://go-review.googlesource.com/c/go/+/481061
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This reverts CL 392854.
Reason for revert: caused #59294, which was derived from google
internal tests. The attempted fix of #59294 caused more breakage.
Change-Id: I5a061561ac2740856b7ecc09725ac28bd30f8bba
Reviewed-on: https://go-review.googlesource.com/c/go/+/481060
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This reverts CL 479915.
Reason for revert: breaks a lot google internal tests.
Change-Id: I13a9422e810af7ba58cbf4a7e6e55f4d8cc0ca51
Reviewed-on: https://go-review.googlesource.com/c/go/+/481055
Reviewed-by: Chressie Himpel <chressie@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.
This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.
For #59294.
Change-Id: Ie52a8f931e0648d8753e4c1dbe45468b8748b527
Reviewed-on: https://go-review.googlesource.com/c/go/+/479915
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.
When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.
When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.
This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.
This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.
For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
Fixes#51676
Change-Id: I380702fe2f9b6b401b2d6f04b0aba990f4b9ee6c
GitHub-Last-Rev: 93dc64ad98
GitHub-Pull-Request: golang/go#51679
Reviewed-on: https://go-review.googlesource.com/c/go/+/392854
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: thepudds <thepudds1460@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
For G[T] that was seen and compiled in imported package, it is not added
to typecheck.Target.Decls, prevent wasting compile time re-creating
DUPOKS symbols. However, the linker do not support a type symbol
referencing a method symbol across DSO boundary. That causes unreachable
sym error when building under -linkshared mode.
To fix it, always re-compile generic methods in linkshared mode.
Fixes#58966
Change-Id: I894b417cfe8234ae1fe809cc975889345df22cef
Reviewed-on: https://go-review.googlesource.com/c/go/+/477375
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Change-Id: Ifb8d64f18b67c8b712feec29ffb6719c6e9718ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/474198
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Go programs can now use the //go:wasmimport module_name function_name
directive to import functions from the WebAssembly runtime.
For now, the directive is restricted to the runtime and syscall/js
packages.
* Derived from CL 350737
* Original work modified to work with changes to the IR conversion code.
* Modification of CL 350737 changes to fully exist in Unified IR path (emp)
* Original work modified to work with changes to the ABI configuration code.
* Fixes#38248
Co-authored-by: Vedant Roy <vroy101@gmail.com>
Co-authored-by: Richard Musiol <mail@richard-musiol.de>
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Change-Id: I740719735d91c306ac718a435a78e1ee9686bc16
Reviewed-on: https://go-review.googlesource.com/c/go/+/463018
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
The misc/cgo/life and misc/cgo/stdio tests started out as fairly simple
test cases when they were added, but the machinery to execute them has
grown in complexity over the years.
They currently reuse the test/run.go runner and its "run" action without
needing much of the additional flexibility that said runner implements.
Given that runner isn't well documented, it makes it harder to see that
ultimately these tests just do 'go run' on a few test programs and check
that the output matches a golden file.
Maybe these test cases should move out of misc to be near similar tests,
or the machinery to execute them can made available in a package that is
easier and safer to reuse. I'd rather not block the refactor of the test
directory runner on that, so for now rewrite these to be self-contained.
Also delete misc/cgo/stdio/testdata/run.out which has no effect on the
test. It was seemingly accidentally kept behind during the refactor in
CL 6220049.
For #56844.
Change-Id: I5e2f542824925092cdddb03b44b6295a4136ccb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/465755
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Move this knob from a binary-startup thing to a build-time thing.
This will enable followon optmizations to the write barrier.
Change-Id: Ic3323348621c76a7dc390c09ff55016b19c43018
Reviewed-on: https://go-review.googlesource.com/c/go/+/447778
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
The default NodeJS V8 stack size is 984K, which is not enough to run
the regexp or go/parser tests. This commit increases the stack size
to 8192K, which removes the stack size limit error.
Fixes#56498Fixes#57614
Change-Id: I357885d420daf259187403deab25ff587defa0fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/463994
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Julien Fabre <ju.pryz@gmail.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
The move to NodeJS 18 allows us to replace the custom
crypto functions with the expanded crypto primitives of
the NodeJS crypto library.
Fixes#56860
Change-Id: I8726b4003150f31521f246f613b6976641b9fa69
Reviewed-on: https://go-review.googlesource.com/c/go/+/463975
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Evan Phoenix <evan@phx.io>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
The path /bin/bash is not available on all operating systems.
Use /usr/bin/env bash to find the system bash interpreter.
Change-Id: I493e462a8e261b7fbbd3f3c0b1d10e55c5ed783b
Reviewed-on: https://go-review.googlesource.com/c/go/+/463977
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Evan Phoenix <evan@phx.io>
The upgrade to NodeJS 18 introduces various library
updates that mean we can no longer override the global
performance package. Instead, rely on the performance
library provided by the NodeJS runtime.
Fixes#57516
Change-Id: Ic8ed902c696ad154f676e0b74b42efb84f02f8db
Reviewed-on: https://go-review.googlesource.com/c/go/+/463234
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Evan Phoenix <evan@phx.io>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Doing this allows updates to swig so it can access slices
and strings in a somewhat less-unsafe way.
Change-Id: If2af8f8cf58d2287569dcd7c7ce09728e20b5ff0
Reviewed-on: https://go-review.googlesource.com/c/go/+/463396
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This is the second round to look for spelling mistakes. This time the
manual sifting of the result list was made easier by filtering out
capitalized and camelcase words.
grep -r --include '*.go' -E '^// .*$' . | aspell list | grep -E -x '[A-Za-z]{1}[a-z]*' | sort | uniq
This PR will be imported into Gerrit with the title and first
comment (this text) used to generate the subject and body of
the Gerrit change.
Change-Id: Ie8a2092aaa7e1f051aa90f03dbaf2b9aaf5664a9
GitHub-Last-Rev: fc2bd6e0c5
GitHub-Pull-Request: golang/go#57737
Reviewed-on: https://go-review.googlesource.com/c/go/+/461595
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Change-Id: Ie46bc3cbfb2622b5eb70618557ff5398866f5607
GitHub-Last-Rev: a665ef84dd
GitHub-Pull-Request: golang/go#57813
Reviewed-on: https://go-review.googlesource.com/c/go/+/462044
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
This fixes the test after CL 455357, which builds the time/tzdata file
from $GOROOT/lib/time/zoneinfo.zip.
Change-Id: I0c5afa6521b58dd3b57c3b4c3c704a622b846382
Reviewed-on: https://go-review.googlesource.com/c/go/+/462279
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
