1
0
mirror of https://github.com/golang/go synced 2024-11-23 08:30:05 -07:00
Commit Graph

1818 Commits

Author SHA1 Message Date
Johan Brandhorst-Satzkorn
e61ba8e537 misc/wasm: add wasip1/wasm exec script
This script uses Wazero, the open source, zero dependencies
pure Go Wasm and WASI runtime. This is the runtime that allows
the greatest number of standard library tests to pass.

For #58141

Co-authored-by: Richard Musiol <neelance@gmail.com>
Co-authored-by: Achille Roussel <achille.roussel@gmail.com>
Co-authored-by: Julien Fabre <ju.pryz@gmail.com>
Co-authored-by: Evan Phoenix <evan@phx.io>
Change-Id: I789465ae4daf2b380f3c05a9365b8d449c6af56c
Reviewed-on: https://go-review.googlesource.com/c/go/+/479620
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-04-06 20:59:01 +00:00
doujiang24
ccad8a9f9c runtime/cgo: store M for C-created thread in pthread key
This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.

CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.

[Original CL 392854 description]

In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key,  only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

[CL 479915 description]

Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

Fixes #51676.
Fixes #59294.

Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494
Reviewed-on: https://go-review.googlesource.com/c/go/+/481061
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-04-03 18:34:11 +00:00
Cherry Mui
bfe3c678ab Revert "runtime/cgo: store M for C-created thread in pthread key"
This reverts CL 392854.

Reason for revert: caused #59294, which was derived from google
internal tests. The attempted fix of #59294 caused more breakage.

Change-Id: I5a061561ac2740856b7ecc09725ac28bd30f8bba
Reviewed-on: https://go-review.googlesource.com/c/go/+/481060
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31 19:26:35 +00:00
Cherry Mui
63ef9059a2 Revert "runtime: get a better g0 stack bound in needm"
This reverts CL 479915.

Reason for revert: breaks a lot google internal tests.

Change-Id: I13a9422e810af7ba58cbf4a7e6e55f4d8cc0ca51
Reviewed-on: https://go-review.googlesource.com/c/go/+/481055
Reviewed-by: Chressie Himpel <chressie@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-31 18:42:48 +00:00
Cherry Mui
443eb9757c runtime: get a better g0 stack bound in needm
Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.

This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.

For #59294.

Change-Id: Ie52a8f931e0648d8753e4c1dbe45468b8748b527
Reviewed-on: https://go-review.googlesource.com/c/go/+/479915
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-03-30 23:23:55 +00:00
doujiang24
ef0dedce87 runtime/cgo: store M for C-created thread in pthread key
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.

When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key,  only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.

When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.

This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.

This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.

For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz

Fixes #51676

Change-Id: I380702fe2f9b6b401b2d6f04b0aba990f4b9ee6c
GitHub-Last-Rev: 93dc64ad98
GitHub-Pull-Request: golang/go#51679
Reviewed-on: https://go-review.googlesource.com/c/go/+/392854
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: thepudds <thepudds1460@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-03-24 16:00:24 +00:00
Cuong Manh Le
bcd82125f8 cmd/compile: re-compile instantiated generic methods in linkshared mode
For G[T] that was seen and compiled in imported package, it is not added
to typecheck.Target.Decls, prevent wasting compile time re-creating
DUPOKS symbols. However, the linker do not support a type symbol
referencing a method symbol across DSO boundary. That causes unreachable
sym error when building under -linkshared mode.

To fix it, always re-compile generic methods in linkshared mode.

Fixes #58966

Change-Id: I894b417cfe8234ae1fe809cc975889345df22cef
Reviewed-on: https://go-review.googlesource.com/c/go/+/477375
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-03-22 04:04:11 +00:00
Ian Lance Taylor
618fb4ab06 runtime/cgo: add tsan sync for traceback function
Change-Id: Ifb8d64f18b67c8b712feec29ffb6719c6e9718ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/474198
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-03-08 20:11:59 +00:00
Evan Phoenix
02411bcd7c all: implement wasmimport directive
Go programs can now use the //go:wasmimport module_name function_name
directive to import functions from the WebAssembly runtime.

For now, the directive is restricted to the runtime and syscall/js
packages.

* Derived from CL 350737
* Original work modified to work with changes to the IR conversion code.
* Modification of CL 350737 changes to fully exist in Unified IR path (emp)
* Original work modified to work with changes to the ABI configuration code.
* Fixes #38248

Co-authored-by: Vedant Roy <vroy101@gmail.com>
Co-authored-by: Richard Musiol <mail@richard-musiol.de>
Co-authored-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Change-Id: I740719735d91c306ac718a435a78e1ee9686bc16
Reviewed-on: https://go-review.googlesource.com/c/go/+/463018
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
2023-03-02 05:28:55 +00:00
Dmitri Shuralyov
01f5a17aa3 misc/cgo/{life,stdio}: remove reliance on test/run.go
The misc/cgo/life and misc/cgo/stdio tests started out as fairly simple
test cases when they were added, but the machinery to execute them has
grown in complexity over the years.

They currently reuse the test/run.go runner and its "run" action without
needing much of the additional flexibility that said runner implements.
Given that runner isn't well documented, it makes it harder to see that
ultimately these tests just do 'go run' on a few test programs and check
that the output matches a golden file.

Maybe these test cases should move out of misc to be near similar tests,
or the machinery to execute them can made available in a package that is
easier and safer to reuse. I'd rather not block the refactor of the test
directory runner on that, so for now rewrite these to be self-contained.

Also delete misc/cgo/stdio/testdata/run.out which has no effect on the
test. It was seemingly accidentally kept behind during the refactor in
CL 6220049.

For #56844.

Change-Id: I5e2f542824925092cdddb03b44b6295a4136ccb4
Reviewed-on: https://go-review.googlesource.com/c/go/+/465755
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
2023-02-28 01:11:11 +00:00
Keith Randall
55044288ad runtime: reimplement GODEBUG=cgocheck=2 as a GOEXPERIMENT
Move this knob from a binary-startup thing to a build-time thing.
This will enable followon optmizations to the write barrier.

Change-Id: Ic3323348621c76a7dc390c09ff55016b19c43018
Reviewed-on: https://go-review.googlesource.com/c/go/+/447778
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-16 00:16:24 +00:00
Johan Brandhorst-Satzkorn
53372ab279 misc: increase node stack size
The default NodeJS V8 stack size is 984K, which is not enough to run
the regexp or go/parser tests. This commit increases the stack size
to 8192K, which removes the stack size limit error.

Fixes #56498
Fixes #57614

Change-Id: I357885d420daf259187403deab25ff587defa0fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/463994
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Julien Fabre <ju.pryz@gmail.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-02-02 15:35:28 +00:00
Bryan C. Mills
ab0f0459da cmd/dist,internal/platform: reenable the c-archive build mode on ios
Also fix misc/cgo/testcarchive to provide a missing CoreFoundation
dependency at link time.

Fixes #58221.
Updates #58172.
Updates #58225.

Change-Id: Ib8b6e52ed2914596615da4c427df2fe984722de6
Reviewed-on: https://go-review.googlesource.com/c/go/+/463752
Run-TryBot: Bryan Mills <bcmills@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2023-02-01 19:47:35 +00:00
Johan Brandhorst-Satzkorn
3875258f97 misc/wasm: use NodeJS crypto library
The move to NodeJS 18 allows us to replace the custom
crypto functions with the expanded crypto primitives of
the NodeJS crypto library.

Fixes #56860

Change-Id: I8726b4003150f31521f246f613b6976641b9fa69
Reviewed-on: https://go-review.googlesource.com/c/go/+/463975
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Evan Phoenix <evan@phx.io>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2023-01-30 18:49:42 +00:00
Johan Brandhorst-Satzkorn
01b9656fce misc/wasm: extend executable compatibility
The path /bin/bash is not available on all operating systems.
Use /usr/bin/env bash to find the system bash interpreter.

Change-Id: I493e462a8e261b7fbbd3f3c0b1d10e55c5ed783b
Reviewed-on: https://go-review.googlesource.com/c/go/+/463977
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Evan Phoenix <evan@phx.io>
2023-01-30 18:49:25 +00:00
Johan Brandhorst-Satzkorn
6ab721dfd6 misc/wasm: use NodeJS performance library
The upgrade to NodeJS 18 introduces various library
updates that mean we can no longer override the global
performance package. Instead, rely on the performance
library provided by the NodeJS runtime.

Fixes #57516

Change-Id: Ic8ed902c696ad154f676e0b74b42efb84f02f8db
Reviewed-on: https://go-review.googlesource.com/c/go/+/463234
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Evan Phoenix <evan@phx.io>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-01-30 18:49:10 +00:00
David Chase
e9c5e60cfd misc: update go.mod to 1.21
Doing this allows updates to swig so it can access slices
and strings in a somewhat less-unsafe way.

Change-Id: If2af8f8cf58d2287569dcd7c7ce09728e20b5ff0
Reviewed-on: https://go-review.googlesource.com/c/go/+/463396
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-01-26 16:55:59 +00:00
Marcel Meyer
b419db6c15 all: fix typos in go file comments
This is the second round to look for spelling mistakes. This time the
manual sifting of the result list was made easier by filtering out
capitalized and camelcase words.

grep -r --include '*.go' -E '^// .*$' . | aspell list | grep -E -x '[A-Za-z]{1}[a-z]*' | sort | uniq

This PR will be imported into Gerrit with the title and first
comment (this text) used to generate the subject and body of
the Gerrit change.

Change-Id: Ie8a2092aaa7e1f051aa90f03dbaf2b9aaf5664a9
GitHub-Last-Rev: fc2bd6e0c5
GitHub-Pull-Request: golang/go#57737
Reviewed-on: https://go-review.googlesource.com/c/go/+/461595
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
2023-01-20 03:27:26 +00:00
Alice
ae400d003f misc/cgo/testsanitizers: use fmt.Printf instead fmt.Println
Change-Id: Ie46bc3cbfb2622b5eb70618557ff5398866f5607
GitHub-Last-Rev: a665ef84dd
GitHub-Pull-Request: golang/go#57813
Reviewed-on: https://go-review.googlesource.com/c/go/+/462044
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2023-01-20 01:23:37 +00:00
Ian Lance Taylor
d449c01d12 misc/reboot: overlay $GOROOT/lib in temporary goroot
This fixes the test after CL 455357, which builds the time/tzdata file
from $GOROOT/lib/time/zoneinfo.zip.

Change-Id: I0c5afa6521b58dd3b57c3b4c3c704a622b846382
Reviewed-on: https://go-review.googlesource.com/c/go/+/462279
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Bypass: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2023-01-18 01:53:44 +00:00
Cherry Mui
9123221ccf misc/cgo/testsanitizers: run libfuzzer tests in temp directory
The libFuzzer generated binary by default writes failure input
into the current directory. Set cmd.Dir to the temporary directory
so it won't write to GOROOT when running the test.

Change-Id: I3e4ce7e3f845be5c9f09511c36e7a9a396eafad2
Reviewed-on: https://go-review.googlesource.com/c/go/+/459556
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-12-28 15:34:23 +00:00
Cherry Mui
e870de9936 misc/cgo/testsanitizers: add libfuzzer tests
Apparently we don't have tests for libfuzzer mode. Add some tests.

Updates #57449.

Change-Id: I813da3e71c6d6f15db31914b248db220b0b7041e
Reviewed-on: https://go-review.googlesource.com/c/go/+/459555
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
2022-12-27 21:10:04 +00:00
Cherry Mui
6d3139b203 misc/cgo/testshared: test build std in shared mode
Test that "go install -buildmode=shared std" works.

For #57334.

Change-Id: I465a07cf2e9035995916ef9940b4c1eeba998099
Reviewed-on: https://go-review.googlesource.com/c/go/+/459056
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Tim Scharfenort <timscharfenort89@googlemail.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
2022-12-22 15:23:56 +00:00
Matthew Dempsky
8c0256b398 cmd/cgo: walk {FuncType,TypeSpec}.TypeParams fields
This CL updates the cgo tool to walk the TypeParams fields for
function types and type declarations, so that C.xxx identifiers can
appear within type parameter lists.

Fixes #52542.

Change-Id: Id02a88d529d50fe59b0a834f415c2575204ffd1f
Reviewed-on: https://go-review.googlesource.com/c/go/+/453977
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-30 21:45:10 +00:00
Bryan C. Mills
14a41387ac misc/cgo/testcshared: reapply CL 451816
I accidentally reverted its edits with a bad cherry-pick in CL 452457.

This should re-fix the windows-.*-newcc builders that regressed at
that change.

Updates #47257.
Updates #35006.
Updates #53540.

Change-Id: I5818416af7c4c8c1593c36aa0198331b42b6c7d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/452675
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-22 02:27:05 +00:00
Bryan C. Mills
7656cba9cc cmd/go: do not install .a files for packages in std
As of CL 450739, we do not need install targets for cgo files when a C
compiler is not present because cgo is not enabled by default.
(Without a C compiler, builds will proceed with cgo disabled.)

Fixes #47257.
Fixes #56888.

Change-Id: I274c50a60b5b1382e291df86a5464da8ad3695a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/452457
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-21 22:43:41 +00:00
Than McIntosh
771a98d6b1 misc/cgo/testcshared: handle unsuffixed dlltool path
Adapt the testcshared tests to handle the case where the path output
by invoking

  gcc -print-prog-name=dlltool

is a path lacking the final ".exe" suffix (this seems to be what clang
is doing); tack it on before using if this is the case.

Updates #35006.
Updates #53540.

Change-Id: I04fb7b9fc90677880b1ced4a4ad2a8867a3f5f86
Reviewed-on: https://go-review.googlesource.com/c/go/+/451816
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-19 23:10:07 +00:00
Austin Clements
42105eb8a5 misc/cgo/test: skip Test9400 on Alpine
Alpine has a known issue where setgid clobbers the Go stack (#39857).
misc/cgo/test skips other tests that use setgid on Alpine, but not
this one. It's not clear to me why this test *used to* pass, but when
I refactored misc/cgo/test in CL 447355 it started failing.

Disable this test on Alpine, like the other setgid tests.

Change-Id: I2e646ef55e2201a4f0b377319d719a011ec847f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/448355
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-11-07 21:15:29 +00:00
Austin Clements
b079640404 misc/cgo/test: simplify for module mode
When we were first introducing module mode, CL 163418 moved many of
the tests in misc/cgo/test into their own test binary under testdata
so misc/cgo/test continued to work in both GOPATH mode and module
mode. This introduce a somewhat complicated test driver into
misc/cgo/test. Since the misc/cgo/test test had to invoke "go test" as
a subprocess, this required care to thread any build flags down into
the subprocess. The output from any failures of the sub-process was
also less than ideal.

Now that we don't have to worry about running these in GOPATH mode any
more, this CL moves all of the tests back into misc/cgo/test and drops
the test driver.

There are two slight complications:

- Test41761 was added after this split and has a C type "S" that's
  also present in misc/cgo/test itself. We rename that to keep that
  test working.

- TestCgo in go/internal/srcimporter now fails to import misc/cgo/test
  because misc/cgo/test now contains imports of other "misc" module
  packages and the importer it sets up isn't configured to allow that.
  We fix this by setting up a build context that's configured for
  this.

Preparation for #37486.

Change-Id: I3c4f73540e0482bbd493823cca44b0ce7fac01f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/447355
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-11-04 20:47:59 +00:00
Michael Matloob
b726b0cadb cmd/go: don't install most GOROOT .a files in pkg
Packages in GOROOT that don't use cgo will not be installed in
GOROOT/pkg, and will instead be cached as usual like other Go
packages.

- add a internal/buildinternal package to hold the identities of the
  five packages that use cgo
- update dist's test code to do a go build std cmd before checking
  staleness on builders. Because most of those packages no longer have
  install locations, and have dependencies that don't either, the
  packages need to be cached to not be stale.
- fix index_test to import packages with the path "." when preparing
  the "want" values to compare the indexed data to. (the module index
  matches the behavior of build.ImportDir, which always passes in "."
  as the path.
- In both the index and go/build Importers, don't set
  PkgObj for GOROOT packages which will no longer have install
  targets. PkgTargetRoot will still be set to compute target paths,
  which will still be needed in buildmode=shared.
- "downgrade" all install actions that don't have a target to build
  actions. (The target should already not be set for packages that
  shouldn't be installed).

For #47257

Change-Id: Ia5aee6b3b20b58e028119cf0352a4c4a2f10f6b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/432535
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-10-28 23:35:08 +00:00
Austin Clements
1e9a7550a8 misc/cgo/fortran: convert to Go test
Currently, the entry-point to this test is a Bash script that smoke
tests the FORTRAN compiler and then runs a FORTRAN-containing Go test.
This CL rearranges things so a pure Go Go test smoke tests the FORTRAN
compiler and then runs a non-test FORTRAN-containing Go binary.

While we're here, we fix a discrepancy when the host is GOARCH=amd64,
but the target is GOARCH=386. Previously, we would pick the wrong
libgfortran path because we didn't account for the cross-compilation,
causing the link to fail. Except for some reason this was ignored and
the test nevertheless "passed". In the new test we're a little more
strict, so this build failure will cause the test to fail, so we add a
little logic to account for cross-compilation with the host toolchain.

For #37486.

Change-Id: Ie6f70066885d6fbb4e1b5a2b1e13b85dee5b359b
Reviewed-on: https://go-review.googlesource.com/c/go/+/443069
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
2022-10-17 15:15:37 +00:00
Austin Clements
dacf88e40a misc/cgo/testsigfwd: move to runtime/testprog/testprogcgo
This migrates testsigfwd, which uses some one-off build
infrastructure, to be part of the runtime's testprogcgo.

The test is largely unchanged. Because it's part of a larger binary,
this CL renames a few things and gates the constructor-time signal
handler registration on an environment variable. This CL also replaces
an errant fmt.Errorf with fmt.Fprintf.

For #37486, since it eliminates a non-go-test from dist.

Change-Id: I0efd146ea0a0a3f0b361431349a419af0f0ecc61
Reviewed-on: https://go-review.googlesource.com/c/go/+/443068
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-17 15:15:33 +00:00
Austin Clements
abd592b3d7 misc/cgo/testsigfwd: delete unused code
This code was introduced in CL 17903 but has never executed. It's also
fundamentally non-deterministic. Delete it.

Simplification for #37486.

Change-Id: I049564123fb4fba401154e2ea0fc429e552d4749
Reviewed-on: https://go-review.googlesource.com/c/go/+/431258
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-10-17 15:15:31 +00:00
Austin Clements
e612d212c6 misc/cgo/testasan: drop test
The testasan test was added back in 2013 (CL 10126044), many years
before Go added ASAN support in 2021 (CL 298611). So, in fact,
testasan does not test Go ASAN support at all, as you might expect
(misc/cgo/testsanitizers does that). It's intended to test whether the
Go memory allocator works in a mixed C/Go binary where the C code is
compiled with ASAN. The test doesn't actually use ASAN in any way; it
just simulates where ASAN of 2013 put its shadow mappings. This made
sense to test at the time because Go was picky about where its heap
landed and ASAN happened to put its mappings exactly where Go wanted
to put its heap. These days, Go is totally flexible about its heap
placement, and I wouldn't be surprised if ASAN also works differently.

Given all of this, this test adds almost no value today. Drop it.

For #37486, since it eliminates a non-go-test from dist.

Change-Id: I0292f8efbdc0e1e39650715604535c445fbaa87f
Reviewed-on: https://go-review.googlesource.com/c/go/+/443067
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-10-14 20:17:35 +00:00
Dmitri Goutnik
a4b4717f23 cmd/dist, misc/cgo/testsanitizers: enable msan tests on freebsd/amd64
Adjust os/arch checks to enable msan tests on freebsd/amd64.

R=go1.20

For #53298

Change-Id: I3d0f5259db73d526d006a12de5ba6284528cf819
Reviewed-on: https://go-review.googlesource.com/c/go/+/411276
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-10-14 12:48:50 +00:00
Cherry Mui
9fe17a0340 cmd/link: don't reset variable size when handling -X flag
The linker's -X flag allows setting/changing a string variable's
content at link time. Currently it resets its size then write a
new string header pointing to the new content. This mostly works.
But under ASAN build the string variable can have larger size
than the usual 2 words, due to the red zone. Resetting the size
can cause the variable to "overlap" (in ASAN's view) with other
variables. Don't reset the size.

Fixes #56175.

Change-Id: Ib364208201a7a2fd7f44f9b1797834198736a405
Reviewed-on: https://go-review.googlesource.com/c/go/+/442635
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2022-10-13 18:47:30 +00:00
Michael Anthony Knyszek
6a2a11fc28 misc/cgo/test: add asan and msan arena tests
While we're here, replace a couple uses of os.Environ with cmd.Environ.

For #51317.

Change-Id: Ic5cf4a887a7975a8281223eec0f94df230b6f095
Reviewed-on: https://go-review.googlesource.com/c/go/+/431955
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-10-12 20:44:43 +00:00
Marko Kungla
ff34676cdd misc/wasm: update deprecated substr usage
String.prototype.substr is deprecated and usage is no longer
recommended so using String.prototype.substring instead.

Change-Id: I9eb49a8c065890df73301e3a04af59f550bc3ae1
Reviewed-on: https://go-review.googlesource.com/c/go/+/406094
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-09-27 17:15:12 +00:00
Bryan C. Mills
42a46206b9 cmd/go: omit VCS stamping during bootstrap
cmd/dist can't easily hard-code -buildvcs=false because not all
versions of cmd/go supported for bootstrapping recognize that flag.

However, we don't want to stamp the bootstrap binaries: the stamping
is redundant with the VERSION file writted during bootstrapping (which
is why it is normally omitted for standard-library packages and
commands), and it may also interfere with building the Go repo from a
source tarball or zip file.

Fixes #54852.

Change-Id: If223f094af137c4c202d6bf622619bd2da397ec4
Reviewed-on: https://go-review.googlesource.com/c/go/+/432435
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-09-22 18:55:06 +00:00
hopehook
c91b1009d5 misc/cgo: replace ioutil.ReadFile with os.ReadFile
For #45557

Change-Id: I25be5b437fa1c9b0e0c46802a9b37efc2d47bca0
Reviewed-on: https://go-review.googlesource.com/c/go/+/431097
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-09-19 16:55:59 +00:00
hopehook
351037d16c misc/cgo: replace os.SEEK_SET with io.SeekStart
Since os.SEEK_SET was deprecated, use io.SeekStart instead.

Change-Id: I11ae496b071ab35412403ff73e52f3da73d5b120
Reviewed-on: https://go-review.googlesource.com/c/go/+/431096
Run-TryBot: Ian Lance Taylor <iant@google.com>
Run-TryBot: hopehook <hopehook@golangcn.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-09-19 16:55:26 +00:00
cuiweixie
c761409e01 misc: use strings.Builder
Change-Id: Icb53d32f2de13287b1b4f4f67dab90fe5ee7a3df
Reviewed-on: https://go-review.googlesource.com/c/go/+/428254
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-09-06 15:44:25 +00:00
Archana R
9e810997c0 runtime: add address sanitizer support for ppc64le
updates #44853

Change-Id: I71905ee1bcb99ce7300bbed2daad3617d2643c53
Reviewed-on: https://go-review.googlesource.com/c/go/+/408814
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Archana Ravindar <aravind5@in.ibm.com>
2022-08-26 18:13:33 +00:00
Lynn Boger
d4ff25ac69 misc/cgo/testsanitizers: determine compiler version for tsan tests on ppc64le
Some tests in misc/cgo/testsanitizers had been disabled on ppc64le
until recently, due to an intermittent error in the tsan tests,
with the goal of trying to understand the failure.

After further investigation, I found that the code for tsan within
gcc does not work consistently when ASLR is enabled on ppc64le. A
fix for that problem was integrated in gcc 9.

This adds a check to testsanitizers to determine the gcc compiler
version on ppc64le and skip the test if the version is too old.

A similar check is needed for asan too.

Updates #54645

Change-Id: I70717d1aa9e967cf1e871566e72b3862b91fea3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/425355
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Archana Ravindar <aravind5@in.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
2022-08-25 11:40:57 +00:00
Michael Pratt
6bdca82030 misc/cgo/test: disable setgid tests with musl
We don't have a good musl detection mechanism, so we detect Alpine (the
most common user of musl) instead.

For #39857.
For #19938.

Change-Id: I2fa39248682aed75884476374fe2212be4427347
Reviewed-on: https://go-review.googlesource.com/c/go/+/425001
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-08-22 14:48:53 +00:00
Ian Lance Taylor
d05ce23756 misc/cgo/testcarchive: permit SIGQUIT for TestSignalForwardingExternal
Occasionally the signal will be sent to a Go thread, which will cause
the program to exit with SIGQUIT rather than SIGSEGV.

Add TestSignalForwardingGo to test the case where the signal is
expected to be delivered to a Go thread.

This is a roll forward of CL 419014 which was rolled back in CL 424954.
This CL differs from 419014 in that it skips TestSignalForwardingGo
on darwin-amd64.

Fixes #53907

Change-Id: I5df3fd610c068df3bd48d9b3d7a9379248b97999
Reviewed-on: https://go-review.googlesource.com/c/go/+/425002
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-08-19 23:07:11 +00:00
Cuong Manh Le
bc69ad3a77 Revert "misc/cgo/testcarchive: permit SIGQUIT for TestSignalForwardingExternal"
This reverts CL 419014.

Reason for revert: broke darwin-amd64 builders

Change-Id: I77838e696a55c415d322ebb9846503de25b546d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/424954
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Joedian Reid <joedian@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2022-08-19 19:35:55 +00:00
Ian Lance Taylor
a409356c54 misc/cgo/testcarchive: permit SIGQUIT for TestSignalForwardingExternal
Occasionally the signal will be sent to a Go thread, which will cause
the program to exit with SIGQUIT rather than SIGSEGV.

Add TestSignalForwardingGo to test the case where the signal is
expected to be delivered to a Go thread.

Fixes #53907

Change-Id: Iaefb964c2be4a815c11c507fa89648f8a7740ba9
Reviewed-on: https://go-review.googlesource.com/c/go/+/419014
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-08-19 03:09:05 +00:00
Archana R
dfbecc06e7 misc/cgo/testsanitizers: fix code to detect gcc version correctly
The current implementation of compilerVersion incorrectly gives an
error message that the compiler version is too old even though the
system has a recent compiler. This happens for specifically for the
gcc compiler and causes ASAN tests to be skipped.
Replacing -v with gcc dump version options seems to fix it. Running
./testsanitizers.test -test.v now shows the ASAN tests being run.
--- PASS: TestASAN (16.81s)
    --- PASS: TestASAN/asan_useAfterReturn (0.60s)
    --- PASS: TestASAN/asan_global5 (0.61s)
    --- PASS: TestASAN/asan_unsafe_fail1 (0.73s)
    --- PASS: TestASAN/asan_unsafe_fail3 (0.73s)
    --- PASS: TestASAN/asan_unsafe_fail2 (0.74s)
    --- PASS: TestASAN/asan_global4_fail (0.74s)
    --- PASS: TestASAN/asan5_fail (0.74s)
    --- PASS: TestASAN/asan3_fail (0.88s)
    --- PASS: TestASAN/asan4_fail (0.89s)
    --- PASS: TestASAN/asan2_fail (0.99s)
    --- PASS: TestASAN/asan_global3_fail (1.00s)
    --- PASS: TestASAN/asan_global1_fail (1.01s)
    --- PASS: TestASAN/asan1_fail (1.01s)
    --- PASS: TestASAN/asan_global2_fail (1.02s)
PASS

Fixes #54370

Change-Id: Iac13a1cf37de54432a6e49555f61e9ec1d781ab8
Reviewed-on: https://go-review.googlesource.com/c/go/+/422574
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Archana Ravindar <aravind5@in.ibm.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2022-08-11 17:23:33 +00:00
Than McIntosh
0d9ed0638b test: skip -buildmode=pie tests on alpine
Skip a collection of -buildmode=pie tests on alpine, which are
currently failing on the linux-amd64-alpine builder. Once #54354 has
been investigated and resolved we can turn these tests back on.

Updates #54354.

Change-Id: I99d4016a40873ee6bb4eda571a64eddbe719c76a
Reviewed-on: https://go-review.googlesource.com/c/go/+/422295
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-08-09 15:13:01 +00:00