Unified IR already records the correct type for them.
Fixes#59378
Change-Id: I275c45b48f67bde55c8e2079d60b5868d0acde7f
Reviewed-on: https://go-review.googlesource.com/c/go/+/481555
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Tests in package context cannot depend directly on package testing due to an import cycle.
We resolved this by having test functions in package context_test (x_test.go) forward to
test functions in package context (context_test.go). This is fragile, since it's easy
to add a test to context_test.go and forget to add the forwarding function, and tests
written in this way cannot easily use testing package features like t.Run for subtests.
It turns out that only four test functions actually use unexported members of package
context. This CL moves all except those four to x_test.go and makes them regular tests.
It also updates TestCause to use t.Run and t.Parallel to parallelize its test cases.
It also adds documentation indicating when tests should be added to each file.
Change-Id: Ic60bae32a7a44e07831b5388c9af219d53ba9af3
Reviewed-on: https://go-review.googlesource.com/c/go/+/480375
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Sameer Ajmani <sameer@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
This change documents the new GODEBUG settings introduced for
html/template and mime/multipart, released with Go 1.19.8 and Go 1.20.3
as part of a security fix.
Updates #59153.
Updates #59234.
Change-Id: I25f4d8245da3301dccccfb44da8ff1a5985392a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/482238
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
zip.NewReader was recently improved to return ErrInsecurePath when
insecure entries are encountered.
This change adopts the same logic for the OpenReader interface as well.
Fixes#58641
Change-Id: I0d8be94d073cc14cf93a914dc250f85b19cec4ab
GitHub-Last-Rev: 68391dc515
GitHub-Pull-Request: golang/go#58658
Reviewed-on: https://go-review.googlesource.com/c/go/+/470735
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Damien Neil <dneil@google.com>
The posix_fallocate syscall returns the result in r1 rather than in
errno:
> If successful, posix_fallocate() returns zero. It returns an error on failure, without
> setting errno.
Source: https://man.freebsd.org/cgi/man.cgi?query=posix_fallocate&sektion=2&n=1
Adjust the PosixFallocate wrappers on freebsd to account for that.
Also, CL 479715 used the same syscall wrapper for 386 and arm. However,
on arm the syscall argument order is different. The wrapper was
generated using mksyscall.go from the golang.org/x/sys/unix package,
adjusting the r1 check correspondingly.
Fixes#59352
Change-Id: I9a4e8e4546237010bc5e730c4988a2a476264cf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/481621
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Otherwise we get warnings from the C linker.
Fixes#59422
Change-Id: I61843dbe5245da0185b0f23dc4b774767fffed40
Reviewed-on: https://go-review.googlesource.com/c/go/+/482315
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Since the first client connection is explicitly closed before making
the second request, we cannot in general assume that the second
request uses a different port (it is equally valid to open the new
connection on the same port as the old one that was closed).
Fixes#59438.
Change-Id: I52d5fe493bd8b1b49270d3996d2019d38d375ce9
Reviewed-on: https://go-review.googlesource.com/c/go/+/482175
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Instead, do the cycle checking in recompileForTest once the test
variant packages have been poked in the right places in the dependency
tree(graph?).
(Pair programming with bcmills@.)
For #59157.
Change-Id: I0c644cb9f2c0dac3a5b0189e2aa0eef083c669f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/482237
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
saferio.ReadData avoids unnecessary allocations because the buffer can be
preallocated with the right size (up to a limit) instead of having to resize
and copy it step by step.
Change-Id: Id70f6908971d4f126c601a9571ac3c67ea0accdc
Reviewed-on: https://go-review.googlesource.com/c/go/+/481616
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Test added.
Fixes#22860
Change-Id: I08304834a2b7b10b4ac729bf36761692eb4731da
Reviewed-on: https://go-review.googlesource.com/c/go/+/113075
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
To allow using testenv.SyscallIsNotSupported, rewrite the test to use
the exported API only. Given that path is an absolute path,
exec.LookPath is equivalent to exec.findExecutable on linux.
Fixes#59087
Change-Id: Ia01b84d4e9d5a65a88dd995f9e3c8a81c4ccd19f
Reviewed-on: https://go-review.googlesource.com/c/go/+/481620
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Bypass: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Fix a regression caused by CL 463835. Unlike most platforms, solaris and
illumos don't use a libc_read_trampoline, so we need to skip one frame
less when using frame pointer unwinding in traceGoSysCall.
The solution is a bit hacky, so it might make sense to implement
gp.syscallbp if this causes more test failures in the future.
Fixes#59350
Change-Id: I0f0b08f36efe8a492eb4a535e752c03636857057
Reviewed-on: https://go-review.googlesource.com/c/go/+/481336
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Felix Geisendörfer <felix.geisendoerfer@datadoghq.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Fixes#59331
Change-Id: I62156be2f2758c59349c3b02db6cf9140429c9e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/481915
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Bypass: Ian Lance Taylor <iant@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
The former was broken deliberately; see #58110. The latter is just an
in-progress port.
Updates #58110, #56001.
Change-Id: I7f1c5e2ac016fb7c65c081174d19239fc9b1ea32
Reviewed-on: https://go-review.googlesource.com/c/go/+/482115
Auto-Submit: Heschi Kreinick <heschi@google.com>
TryBot-Bypass: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
ECMAScript 6 introduced template literals[0][1] which are delimited with
backticks. These need to be escaped in a similar fashion to the
delimiters for other string literals. Additionally template literals can
contain special syntax for string interpolation.
There is no clear way to allow safe insertion of actions within JS
template literals, as handling (JS) string interpolation inside of these
literals is rather complex. As such we've chosen to simply disallow
template actions within these template literals.
A new error code is added for this parsing failure case, errJsTmplLit,
but it is unexported as it is not backwards compatible with other minor
release versions to introduce an API change in a minor release. We will
export this code in the next major release.
The previous behavior (with the cavet that backticks are now escaped
properly) can be re-enabled with GODEBUG=jstmpllitinterp=1.
This change subsumes CL471455.
Thanks to Sohom Datta, Manipal Institute of Technology, for reporting
this issue.
Fixes CVE-2023-24538
Fixes#59234
[0] https://tc39.es/ecma262/multipage/ecmascript-language-expressions.html#sec-template-literals
[1] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Template_literals
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802457
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Change-Id: Ia221fefdb273bd0f066dffc2abcf2a616801d2f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/482079
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Setting a large line or column number using a //line directive can cause
integer overflow even in small source files.
Limit line and column numbers in //line directives to 2^30-1, which
is small enough to avoid int32 overflow on all reasonbly-sized files.
For #59180
Fixes CVE-2023-24537
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802456
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Run-TryBot: Damien Neil <dneil@google.com>
Change-Id: I149bf34deca532af7994203fa1e6aca3c890ea14
Reviewed-on: https://go-review.googlesource.com/c/go/+/482078
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
The parsed forms of MIME headers and multipart forms can consume
substantially more memory than the size of the input data.
A malicious input containing a very large number of headers or
form parts can cause excessively large memory allocations.
Set limits on the size of MIME data:
Reader.NextPart and Reader.NextRawPart limit the the number
of headers in a part to 10000.
Reader.ReadForm limits the total number of headers in all
FileHeaders to 10000.
Both of these limits may be set with with
GODEBUG=multipartmaxheaders=<values>.
Reader.ReadForm limits the number of parts in a form to 1000.
This limit may be set with GODEBUG=multipartmaxparts=<value>.
Thanks for Jakob Ackermann (@das7pad) for reporting this issue.
For CVE-2023-24536
For #59153
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802455
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Change-Id: I08dd297bd75724aade4b0bd6a7d19aeca5bbf99f
Reviewed-on: https://go-review.googlesource.com/c/go/+/482077
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
For requests containing large numbers of small parts,
memory consumption of a parsed form could be about 250%
over the estimated size.
When considering the size of parsed forms, account for the size of
FileHeader structs and increase the estimate of memory consumed by
map entries.
Thanks to Jakob Ackermann (@das7pad) for reporting this issue.
For CVE-2023-24536
For #59153
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802454
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Change-Id: I9620758495ed77c09ca6dc5db4b723c29f3baad8
Reviewed-on: https://go-review.googlesource.com/c/go/+/482076
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
When copying form data to disk with io.Copy,
allocate only one copy buffer and reuse it rather than
creating two buffers per file (one from io.multiReader.WriteTo,
and a second one from os.File.ReadFrom).
Thanks to Jakob Ackermann (@das7pad) for reporting this issue.
For CVE-2023-24536
For #59153
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802453
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Change-Id: I732bd2e1e7467918cac8ab9d65d089272ba4656f
Reviewed-on: https://go-review.googlesource.com/c/go/+/482075
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Bypass: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
A parsed MIME header is a map[string][]string. In the common case,
a header contains many one-element []string slices. To avoid
allocating a separate slice for each key, ReadMIMEHeader looks
ahead in the input to predict the number of keys that will be
parsed, and allocates a single []string of that length.
The individual slices are then allocated out of the larger one.
The prediction of the number of header keys was done by counting
newlines in the input buffer, which does not take into account
header continuation lines (where a header key/value spans multiple
lines) or the end of the header block and the start of the body.
This could lead to a substantial amount of overallocation, for
example when the body consists of nothing but a large block of
newlines.
Fix header key count prediction to take into account the end of
the headers (indicated by a blank line) and continuation lines
(starting with whitespace).
Thanks to Jakob Ackermann (@das7pad) for reporting this issue.
For #58975
Fixes CVE-2023-24534
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1802452
Run-TryBot: Damien Neil <dneil@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Julie Qiu <julieqiu@google.com>
Change-Id: Iacc1c2b5ea6509529845a972414199f988ede1e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/481994
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
man getaddrinfo:
EAI_NODATA
The specified network host exists, but does not have any
network addresses defined.
In the go resolver we treat this kind of error as nosuchhost.
Change-Id: I69fab6f8da8e3a86907e65104bca9f055968633a
GitHub-Last-Rev: b4891e2add
GitHub-Pull-Request: golang/go#57507
Reviewed-on: https://go-review.googlesource.com/c/go/+/459955
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Mateusz Poliwczak <mpoliwczak34@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Fixes the misuse of "a" vs "an", according to English grammatical
expectations and using https://www.a-or-an.com/
Change-Id: I53ac724070e3ff3d33c304483fe72c023c7cda47
Reviewed-on: https://go-review.googlesource.com/c/go/+/480536
Run-TryBot: shuang cui <imcusg@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Growing by one is a simpler, and often cheaper,
operation compared to appending one (newly created) zero value.
The method was introduced in Go 1.20.
growSlice in dec_helpers.go is left alone,
as it grows using the builtin append instead of reflect.Append.
No noticeable performance difference on any of our benchmarks,
as this code only runs for slices large enough to not fit in
saferio.SliceCap, and none of our benchmarks use data that large.
goos: linux
goarch: amd64
pkg: encoding/gob
cpu: AMD Ryzen 7 PRO 5850U with Radeon Graphics
│ old │ new │
│ sec/op │ sec/op vs base │
DecodeBytesSlice-8 11.37µ ± 1% 11.46µ ± 4% ~ (p=0.315 n=10)
DecodeInterfaceSlice-8 96.49µ ± 1% 95.75µ ± 1% ~ (p=0.436 n=10)
geomean 33.12µ 33.12µ +0.01%
│ old │ new │
│ B/op │ B/op vs base │
DecodeBytesSlice-8 22.39Ki ± 0% 22.39Ki ± 0% ~ (p=1.000 n=10)
DecodeInterfaceSlice-8 80.25Ki ± 0% 80.25Ki ± 0% ~ (p=0.650 n=10)
geomean 42.39Ki 42.39Ki +0.00%
│ old │ new │
│ allocs/op │ allocs/op vs base │
DecodeBytesSlice-8 169.0 ± 0% 169.0 ± 0% ~ (p=1.000 n=10) ¹
DecodeInterfaceSlice-8 3.178k ± 0% 3.178k ± 0% ~ (p=1.000 n=10) ¹
geomean 732.9 732.9 +0.00%
Change-Id: I468aebf4ae6f197a1fd35f6fee809ca591c1788f
Reviewed-on: https://go-review.googlesource.com/c/go/+/481376
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
I almost exclusively use these benchmarks with -benchtime already.
Change-Id: I6539cbba6abbdb6b275502e122f4e16856d8b9e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/481375
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Rob Pike <r@golang.org>
This was intended to be merged together with changes in CL 479616.
For #58141
Change-Id: I76c38d3d4dfee93a1a170e28af28f0c9d6382830
Reviewed-on: https://go-review.googlesource.com/c/go/+/480656
Run-TryBot: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Johan Brandhorst-Satzkorn <johan.brandhorst@gmail.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Add new helper macros to further simplify the transition from
the host's ABI to Go. Fortunately the same one should work for
all PPC64 targets.
Update the other site which uses these wrappers to further
consolidate. Also, update the call to runtime.sigtrampgo to
call the ABIInternal version directly.
Also, update the SAVE/RESTORE_VR macros to accept R0.
Change-Id: I0046176029e1e1b25838688e4b7bf57805b01bd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/476297
Reviewed-by: Archana Ravindar <aravind5@in.ibm.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This reverts CL 481059, which in turn reverts CL 478917.
Reason for revert: reapply the original CL.
Change-Id: Icf6bb6a620313b44fadcc7f69a62fdbb943e34fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/481075
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Paul Murphy <murp@ibm.com>
Skip one of the testpoints that verifies inlining, since it
no longer passes as a result of reverting CL 479095. Once we
roll forward with a new version of CL 479095 we can re-enable
this testpoint.
Change-Id: I41f6fb3fce78f31e60c5f0ed2856be0e66865149
Reviewed-on: https://go-review.googlesource.com/c/go/+/481755
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
This reapplies CL 392854, with the followup fixes in CL 479255,
CL 479915, and CL 481057 incorporated.
CL 392854, by doujiang24 <doujiang24@gmail.com>, speed up C to Go
calls by binding the M to the C thread. See below for its
description.
CL 479255 is a followup fix for a small bug in ARM assembly code.
CL 479915 is another followup fix to address C to Go calls after
the C code uses some stack, but that CL is also buggy.
CL 481057, by Michael Knyszek, is a followup fix for a memory leak
bug of CL 479915.
[Original CL 392854 description]
In a C thread, it's necessary to acquire an extra M by using needm while invoking a Go function from C. But, needm and dropm are heavy costs due to the signal-related syscalls.
So, we change to not dropm while returning back to C, which means binding the extra M to the C thread until it exits, to avoid needm and dropm on each C to Go call.
Instead, we only dropm while the C thread exits, so the extra M won't leak.
When invoking a Go function from C:
Allocate a pthread variable using pthread_key_create, only once per shared object, and register a thread-exit-time destructor.
And store the g0 of the current m into the thread-specified value of the pthread key, only once per C thread, so that the destructor will put the extra M back onto the extra M list while the C thread exits.
When returning back to C:
Skip dropm in cgocallback, when the pthread variable has been created, so that the extra M will be reused the next time invoke a Go function from C.
This is purely a performance optimization. The old version, in which needm & dropm happen on each cgo call, is still correct too, and we have to keep the old version on systems with cgo but without pthreads, like Windows.
This optimization is significant, and the specific value depends on the OS system and CPU, but in general, it can be considered as 10x faster, for a simple Go function call from a C thread.
For the newly added BenchmarkCGoInCThread, some benchmark results:
1. it's 28x faster, from 3395 ns/op to 121 ns/op, in darwin OS & Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
2. it's 6.5x faster, from 1495 ns/op to 230 ns/op, in Linux OS & Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
[CL 479915 description]
Currently, when C calls into Go the first time, we grab an M
using needm, which sets m.g0's stack bounds using the SP. We don't
know how big the stack is, so we simply assume 32K. Previously,
when the Go function returns to C, we drop the M, and the next
time C calls into Go, we put a new stack bound on the g0 based on
the current SP. After CL 392854, we don't drop the M, and the next
time C calls into Go, we reuse the same g0, without recomputing
the stack bounds. If the C code uses quite a bit of stack space
before calling into Go, the SP may be well below the 32K stack
bound we assumed, so the runtime thinks the g0 stack overflows.
This CL makes needm get a more accurate stack bound from
pthread. (In some platforms this may still be a guess as we don't
know exactly where we are in the C stack), but it is probably
better than simply assuming 32K.
Fixes#51676.
Fixes#59294.
Change-Id: I9bf1400106d5c08ce621d2ed1df3a2d9e3f55494
Reviewed-on: https://go-review.googlesource.com/c/go/+/481061
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: DeJiang Zhu (doujiang) <doujiang24@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
These are replaced by unsafe.String etc, which were added in Go 1.20.
Per https://go.dev/wiki/Deprecated, we must wait until Go 1.21
to mark them deprecated.
Fixes#56906.
Change-Id: I4198c3f3456e9e2031f6c7232842e187e6448892
Reviewed-on: https://go-review.googlesource.com/c/go/+/452762
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Russ Cox <rsc@golang.org>
This file is itself template input, so have to hide the template
in the go command example.
Change-Id: Ifc4eaff35ca8dc2fb479f8e28d64c06b2a9c9d3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/480995
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Casting to a *uintptr is not ok if there isn't at least 8 bytes of
data backing that pointer (on 64-bit archs).
So although we end up making a slice of 0 length with that pointer,
the cast itself doesn't know that.
Instead, bail early if the result is going to be 0 length.
Fixes#59334
Change-Id: Id3c0e09d341d838835c0382cccfb0f71dc3dc7e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/480575
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
File.Chmod is supported on Windows since CL 250077, there is no need
to skip the call anymore.
Updates #18026
Change-Id: Ie03cf016e651b93241f73067614fc4cb341504ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/480416
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
There are many tests in internal/gcimporter that are skipped on
Windows because they build a test program that needs the -D flag
when invoking the Go compiler.
This flag is already passed since CL 442303, so there is no need to
skip those tests.
Change-Id: I877e670194048bda9a52ad2568650cf33eacfb5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/480415
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Windows has supported external linking for a while, there is no
need to skip this test.
Change-Id: Ic3d0cc3441ee670767dae085db5e62fce205ff04
Reviewed-on: https://go-review.googlesource.com/c/go/+/480417
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
This adds the three functions from #56102 to the sync package. These
provide a convenient API for the most common uses of sync.Once.
The performance of these is comparable to direct use of sync.Once:
$ go test -run ^$ -bench OnceFunc\|OnceVal -count 20 | benchstat -row .name -col /v
goos: linux
goarch: amd64
pkg: sync
cpu: 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
│ Once │ Global │ Local │
│ sec/op │ sec/op vs base │ sec/op vs base │
OnceFunc 1.3500n ± 6% 2.7030n ± 1% +100.22% (p=0.000 n=20) 0.3935n ± 0% -70.86% (p=0.000 n=20)
OnceValue 1.3155n ± 0% 2.7460n ± 1% +108.74% (p=0.000 n=20) 0.5478n ± 1% -58.35% (p=0.000 n=20)
The "Once" column represents the baseline of how code would typically
express these patterns using sync.Once. "Global" binds the closure
returned by OnceFunc/OnceValue to global, which is how I expect these
to be used most of the time. Currently, this defeats some inlining
opportunities, which roughly doubles the cost over sync.Once; however,
it's still *extremely* fast. Finally, "Local" binds the returned
closure to a local variable. This unlocks several levels of inlining
and represents pretty much the best possible case for these APIs, but
is also unlikely to happen in practice. In principle the compiler
could recognize that the global in the "Global" case is initialized in
place and never mutated and do the same optimizations it does in the
"Local" case, but it currently does not.
Fixes#56102
Change-Id: If7355eccd7c8de7288d89a4282ff15ab1469e420
Reviewed-on: https://go-review.googlesource.com/c/go/+/451356
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Caleb Spare <cespare@gmail.com>
Auto-Submit: Austin Clements <austin@google.com>
Currently, when the inliner is determining if a function is
inlineable, it descends into the bodies of closures constructed by
that function. This has several unfortunate consequences:
- If the closure contains a disallowed operation (e.g., a defer), then
the outer function can't be inlined. It makes sense that the
*closure* can't be inlined in this case, but it doesn't make sense
to punish the function that constructs the closure.
- The hairiness of the closure counts against the inlining budget of
the outer function. Since we currently copy the closure body when
inlining the outer function, this makes sense from the perspective
of export data size and binary size, but ultimately doesn't make
much sense from the perspective of what should be inlineable.
- Since the inliner walks into every closure created by an outer
function in addition to starting a walk at every closure, this adds
an n^2 factor to inlinability analysis.
This CL simply drops this behavior.
In std, this makes 57 more functions inlinable, and disallows inlining
for 10 (due to the basic instability of our bottom-up inlining
approach), for an net increase of 47 inlinable functions (+0.6%).
This will help significantly with the performance of the functions to
be added for #56102, which have a somewhat complicated nesting of
closures with a performance-critical fast path.
The downside of this seems to be a potential increase in export data
and text size, but the practical impact of this seems to be
negligible:
│ before │ after │
│ bytes │ bytes vs base │
Go/binary 15.12Mi ± 0% 15.14Mi ± 0% +0.16% (n=1)
Go/text 5.220Mi ± 0% 5.237Mi ± 0% +0.32% (n=1)
Compile/binary 22.92Mi ± 0% 22.94Mi ± 0% +0.07% (n=1)
Compile/text 8.428Mi ± 0% 8.435Mi ± 0% +0.08% (n=1)
Change-Id: Ie9e38104fed5689a94c368288653fd7cb4b7a35e
Reviewed-on: https://go-review.googlesource.com/c/go/+/479095
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Auto-Submit: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
This reverts CL 392854.
Reason for revert: caused #59294, which was derived from google
internal tests. The attempted fix of #59294 caused more breakage.
Change-Id: I5a061561ac2740856b7ecc09725ac28bd30f8bba
Reviewed-on: https://go-review.googlesource.com/c/go/+/481060
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This reverts CL 479255.
Reason for revert: need to revert CL 392854, and this caused a conflict.
Change-Id: I6cb105c62e51b47de3f652df5f5ee92673a93919
Reviewed-on: https://go-review.googlesource.com/c/go/+/481058
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
This reverts CL 478917.
Reason for revert: need to revert CL 392854, and this caused a conflict.
Change-Id: I02c3285de5635b431a99adc8790c8310d1c4e6a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/481059
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
This reverts CL 479915.
Reason for revert: breaks a lot google internal tests.
Change-Id: I13a9422e810af7ba58cbf4a7e6e55f4d8cc0ca51
Reviewed-on: https://go-review.googlesource.com/c/go/+/481055
Reviewed-by: Chressie Himpel <chressie@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
This was found by running `git grep 'fmt.Sprintf("%d",' | grep -v test | grep -v vendor`
And this was automatically fixed with gotiti https://github.com/catenacyber/gotiti
and using unconvert https://github.com/mdempsky/unconvert
to check if there was (tool which fixed another useless cast)
Change-Id: I023926bc4aa8d51de45f712ac739a0a80145c28c
GitHub-Last-Rev: 1063e32e5b
GitHub-Pull-Request: golang/go#59144
Reviewed-on: https://go-review.googlesource.com/c/go/+/477675
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
For f()() call, the compiler rewrite it roughly to:
autotmp := f()
autotmp()
However, if f() were inlined, escape analysis will confuse about the
lifetime of autotmp, leading to bad escaping decision.
This CL fixes this issue by rewriting f()() to:
var autotmp
autotmp = f()
autotmp()
This problem also happens with Unified IR, until CL 421821 land.
Fixes#57434
Change-Id: I159a7e4c93bbc172f0eae60e7d40fc64ba70b236
Reviewed-on: https://go-review.googlesource.com/c/go/+/459295
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Document the changes to GODEBUG implemented as
part of proposal #56986.
Fixes#56986.
Change-Id: I23153a123e23820c5b22db4767620e037bbdd083
Reviewed-on: https://go-review.googlesource.com/c/go/+/462202
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
