1
0
mirror of https://github.com/golang/go synced 2024-11-16 20:44:52 -07:00
Commit Graph

56675 Commits

Author SHA1 Message Date
Russ Cox
9c447b7cf6 cmd/go: implement GOTOOLCHAIN=auto
The documentation is yet to be written (more work in the go
command remains first). This CL implements the toolchain
selection described in
https://go.dev/design/57001-gotoolchain#the-and-lines-in-in-the-work-module
with these changes based on the issue discussion:

1. GOTOOLCHAIN=auto looks for a go1.19.1 binary in $PATH
and if found uses it instead of downloading Go 1.19.1 as a module.

2. GOTOOLCHAIN=path is like GOTOOLCHAIN=auto, with
downloading disabled.

3. GOTOOLCHAIN=auto+version and GOTOOLCHAIN=path+version
set a different minimum version of Go to use during the version
selection. The default is to use the newer of what's on the go line
or the current toolchain. If you are have Go 1.22 installed locally
and want to switch to a minimum of Go 1.25 with go.mod files
allowed to bump even further, you would set GOTOOLCHAIN=auto+go1.25.
The minimum is also important when there is no go.mod involved,
such as when you write a tiny x.go program and run "go run x.go".
That would get Go 1.25 in this example, instead of falling back to
the local Go 1.22.

Change-Id: I286625a24420424c313d1082b9949a463b2fe14a
Reviewed-on: https://go-review.googlesource.com/c/go/+/494436
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Russ Cox <rsc@golang.org>
2023-05-20 19:17:27 +00:00
Jabar Asadi
aeb0644bd3 io: prevent seeking to position prior to offsetwrite.base
We don't want to permit writing before the start of an OffsetWriter.

One of the goals of OffsetWriter is to restrict where data
can be written.

However, this rule can be violated by WriteAt() method of OffsetWriter
as the following code shows:

f, _ := os.Create("file.txt")
owr := io.NewOffsetWriter(f, 10)
owr.Write([]byte("world"))
owr.WriteAt([]byte("hello"), -10)

Change-Id: I6c7519fea68daefa641f25130cdd9803dc8aae22
GitHub-Last-Rev: a29d890d6f
GitHub-Pull-Request: golang/go#60222
Reviewed-on: https://go-review.googlesource.com/c/go/+/495155
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Jabar Asadi <jasadi@d2iq.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-05-20 00:46:20 +00:00
Sean Liao
a0d53199b7 net/http/pprof: document query params
Fixes #59452

Change-Id: Ia0b5a03565f663190c480ef9e26309fa85ff192c
Reviewed-on: https://go-review.googlesource.com/c/go/+/496144
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2023-05-20 00:40:54 +00:00
Sven Anderson
697644070c runtime: improve Pinner with gcBits
This change replaces the statically sized pinnerBits with gcBits
based ones, that are copied in each GC cycle if they exist.  The
pinnerBits now include a second bit per object, that indicates if a
pinner counter for multi-pins exists, in order to avoid unnecessary
specials iterations.

This is a follow-up to CL 367296.

Change-Id: I82e38cecd535e18c3b3ae54b5cc67d3aeeaafcfd
Reviewed-on: https://go-review.googlesource.com/c/go/+/493275
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-05-19 23:21:57 +00:00
Dmitri Shuralyov
6891405bbe cmd/dist: use "pkg[:variant]" as dist test name
The work to add the -json flag to the 'dist test' command also cleaned
how dist tests are tracked and registered. By now, a pair of (import
path, variant) strings is sufficient to uniquely identify every dist
test that exists. Some of the custom dist test names have been improved
along the way. And since the names are already changing a little anyway,
we use this opportunity to make them more uniform and predictable.

The mapping from the old dist test names to the new is as follows:

- "go_test:pkg"       → "pkg"  (this is the most common case)
- "go_test_bench:pkg" → "pkg:racebench"
- all other custom names are now called "pkg:variant", where variant
  is a description of their test configuration and pkg is the import
  path of the Go package under test

CL 495016 introduced test variants and used variant names for rewriting
the Package field in JSON events, and now that same name starts to also
be used as the dist test name.

Like previously done in CL 494496, registering a test variant involving
multiple Go packages creates a "pkg:variant" dist test name for each.
In the future we may combine their 'go test' invocation purely as an
optimization.

We can do this with the support of CL 496190 that keeps the coordinator
happy and capable of working with both new and old names.

In the end, all dist tests now have a consistent "pkg[:variant]" name.

For #37486.
For #59990.

Change-Id: I7eb02a42792a9831a2f3eeab583ff635d24269e8
Co-authored-by: Austin Clements <austin@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/496181
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-05-19 23:20:04 +00:00
Austin Clements
fe786638bb cmd/dist: remove unused functionality
The moved_goroot test was the last user of the goroot functionality.
Now that it's been deleted, drop this and clean up loose ends.

Change-Id: Ie5e95644022dab76b1c06cf37f7729ee6616311f
Reviewed-on: https://go-review.googlesource.com/c/go/+/496520
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-19 22:45:24 +00:00
Austin Clements
49ad23a6d2 cmd/dist: delete moved_goroot test
This test is largely obviated by the goroot_executable and
list_goroot_symlink cmd/go script tests. It's the last user of several
special-case features in cmd/dist and runs only under a fairly
constrained set of conditions (including only running on builders, not
locally). Delete it.

Change-Id: Icc744e3f9f04813bfd0cad2ef3e88e42617ecf5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/496519
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
2023-05-19 21:43:49 +00:00
Paul E. Murphy
29d5272da8 all: generate NOTOC shared code on power10/PPC64/linux
An explicit TOC pointer is not needed when building with GOPPC64=power10
on linux/PPC64 as all addressing is PC relative.

Apply changes to the compiler, assembler, and linker to remove R2/TOC
maintenance in these build configurations.

This results in noticeably smaller PIC binaries. For example the size
(in bytes) of the k8s binaries before and with this patch:

	GOFLAGS="-buildmode=pie" \
	FORCE_HOST_GO=y \
	GOPPC64=power10 \
	CGO_CFLAGS="-mcpu=power10 -O2 -g" \
	make all

         apiextensions-apiserver   66060288   64487424   -1572864  -2.38%
                   e2e_node.test  163520856  159850760   -3670096  -2.24%
                        e2e.test  178167304  174890432   -3276872  -1.83%
                          ginkgo   11010048   10747904    -262144  -2.38%
                       go-runner    2162688    2162688          0      0%
                       k8s-tests  170182216  166970880   -3211336  -1.88%
                         kubeadm   52625408   51314688   -1310720  -2.49%
                 kube-aggregator   62849024   61341696   -1507328  -2.39%
                  kube-apiserver  147783680  144375808   -3407872  -2.30%
         kube-controller-manager  131137536  127991808   -3145728  -2.39%
                         kubectl   53608448   52363264   -1245184  -2.32%
                 kubectl-convert   52625408   51314688   -1310720  -2.49%
                         kubelet  120913920  118095872   -2818048  -2.33%
                 kube-log-runner    1900544    1835008     -65536  -3.44%
                        kubemark  119078912  116326400   -2752512  -2.31%
                      kube-proxy   58392576   56885248   -1507328  -2.58%
                  kube-scheduler   60751872   59244544   -1507328  -2.48%
                         mounter    1835008    1769472     -65536  -3.57%
               watch-termination   38076416   37158912    -917504  -2.40%

And text size changes:

         apiextensions-apiserver   30243288   28654116   -1589172  -5.25%
                   e2e_node.test   71132064   67520288   -3611776  -5.07%
                        e2e.test   61843984   58635088   -3208896  -5.18%
                          ginkgo    4975916    4769304    -206612  -4.15%
                       go-runner     896532     858400     -38132  -4.25%
                       k8s-tests   60925792   57752032   -3173760  -5.20%
                         kubeadm   24643240   23404100   -1239140  -5.02%
                 kube-aggregator   28688060   27160976   -1527084  -5.32%
                  kube-apiserver   65627332   62259460   -3367872  -5.13%
         kube-controller-manager   56773844   53706532   -3067312  -5.40%
                         kubectl   24344276   23080640   -1263636  -5.19%
                 kubectl-convert   23733764   22521768   -1211996  -5.10%
                         kubelet   52494580   49720340   -2774240  -5.28%
                 kube-log-runner     787128     753232     -33896  -4.30%
                        kubemark   51576580   48837380   -2739200  -5.31%
                      kube-proxy   26541092   25124080   -1417012  -5.33%
                  kube-scheduler   27448512   25976172   -1472340  -5.36%
                         mounter     744100     712628     -31472  -4.22%
               watch-termination   18047276   17139912    -907364  -5.02%

Change-Id: Ib4872823b06f93861e46a00679b5d4e5e30b538a
Reviewed-on: https://go-review.googlesource.com/c/go/+/495416
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-05-19 21:12:48 +00:00
David Chase
1524bd1c78 cmd/compile: tweaks to loopvar logging
This adds the loop type to the json/LSP logging, to help with
studies of how many loops of which kind were modified.

Change-Id: I637a630cd275b413259601c0070b963f3c6d2185
Reviewed-on: https://go-review.googlesource.com/c/go/+/496515
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 20:55:53 +00:00
Michael Anthony Knyszek
94a948d1b1 runtime: make trace.lock not reentrant
Currently trace.lock is reentrant in a few cases. AFAICT, this was
necessary a long time ago when the trace reader would goparkunlock, and
might flush a trace buffer while parking the goroutine. Today, that's no
longer true, since that always happens without the trace.lock held.

However, traceReadCPU does still rely on this behavior, since it could
get called either with trace.lock held, or without it held. The silver
lining here is that it doesn't *need* trace.lock to be held, so the
trace reader can just drop the lock to call traceReadCPU (this is
probably also nice for letting other goroutines flush while the trace
reader is reading from the CPU log).

Stress-tested with

$ stress ./trace.test -test.run="TestTraceCPUProfile|TestTraceStress|TestTraceStressStartStop"
...

42m0s: 24520 runs so far, 0 failures

Change-Id: I2016292c17fe7384050fcc0c446f5797c4e46437
Reviewed-on: https://go-review.googlesource.com/c/go/+/496296
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-19 20:47:57 +00:00
Michael Anthony Knyszek
7c91e1e568 runtime: replace raw traceEv with traceBlockReason in gopark
This change adds traceBlockReason which leaks fewer implementation
details of the tracer to the runtime. Currently, gopark is called with
an explicit trace event, but this leaks details about trace internals
throughout the runtime.

This change will make it easier to change out the trace implementation.

Change-Id: Id633e1704d2c8838c6abd1214d9695537c4ac7db
Reviewed-on: https://go-review.googlesource.com/c/go/+/494185
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-19 20:47:25 +00:00
qmuntal
656a20a52a net: make Dial fail faster on Windows closed loopback devices
On Windows when connecting to an unavailable port, ConnectEx() will
retry for 2s, even on loopback devices.

This CL uses a call to WSAIoctl to make the ConnectEx() call fail
faster on local connections.

Fixes #23366

Change-Id: Iafeca8ea0053f01116b2504c45d88120f84d05e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/495875
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Quim Muntal <quimmuntal@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 20:19:50 +00:00
Austin Clements
6333725d5f cmd/dist: flush incomplete lines in -json mode
Currently, if a test prints an incomplete line and then exits, in JSON
mode, the filter we use to rewrite Package lines will keep the last
incomplete line in an internal buffer and never print it. In theory
this should never happen anyway because the test should only write
JSON to stdout, but we try pretty hard to pass through any non-JSON,
so it seems inconsistent to swallow incomplete lines.

Fix this by adding a testJSONFilter.Flush method and calling it in the
right places. Unfortunately this is a bit tricky because the filter is
constructed pretty far from where we run the exec.Cmd, so we return
the flush function through the various layers in order to route it to
the place where we call Cmd.Run.

Updates #37486.

Change-Id: I38af67e8ad23458598a32fd428779bb0ec21ac3c
Reviewed-on: https://go-review.googlesource.com/c/go/+/496516
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-19 20:13:24 +00:00
Michael Anthony Knyszek
e6fb1905cb runtime: make traceClockNow nosplit
It's called from exitsyscall, which is nosplit.

Change-Id: I3f5f92e044497a88a72c1870beb2bdd15c4263c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/496517
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
2023-05-19 19:56:59 +00:00
cuiweixie
a1a048a8ab maps: enhance the robustness of the tests
Change-Id: I908e11196f55069d6dca6a19cd206629618e9f37
Reviewed-on: https://go-review.googlesource.com/c/go/+/496079
Auto-Submit: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: xie cui <523516579@qq.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2023-05-19 19:26:57 +00:00
cuiweixie
09e0f50d03 maps: delete unused var
Change-Id: I3d0c196e2ec139d224d057a954c631251b80e921
Reviewed-on: https://go-review.googlesource.com/c/go/+/496077
Run-TryBot: xie cui <523516579@qq.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2023-05-19 19:25:41 +00:00
Keith Randall
bd3f44e4ff cmd/compile: constant-fold loads from constant dictionaries and types
Retrying the original CL with a small modification. The original CL
did not handle the case of reading an itab out of a dictionary
correctly.  When we read an itab out of a dictionary, we must treat
the type inside that itab as maybe being put in an interface.

Original CL: 486895
Revert CL: 490156

Change-Id: Id2dc1699d184cd8c63dac83986a70b60b4e6cbd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/491495
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 18:10:11 +00:00
Michael Anthony Knyszek
b60db8f7d9 runtime: formalize the trace clock
Currently the trace clock is cputicks() with comments sprinkled in
different places as to which clock to use. Since the execution tracer
redesign will use a different clock, it seems like a good time to clean
that up.

Also, rename the start/end timestamps to be more readable (i.e.
startTime vs. timeStart).

Change-Id: If43533eddd0e5f68885bb75cdbadb38da42e7584
Reviewed-on: https://go-review.googlesource.com/c/go/+/494775
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 18:01:57 +00:00
Ian Lance Taylor
dc4993e7c6 cmp: new package
The new cmp package provides types and functions related to
comparing ordered values.

For #59488

Change-Id: I43f4b2e6036f63b87c2152672d2b6fa18235cbeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/496356
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Eli Bendersky‎ <eliben@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
2023-05-19 17:41:52 +00:00
miller
86c6b4763e internal/poll: handle SetDeadline to time.Now() in Plan 9
The implementation of SetDeadline in Plan 9 begins by calculating
d = the offset of the requested deadline from time.Now(). If d > 0,
a timer is set to interrupt future I/O. If d < 0, the channel is
flagged to prevent future I/O and any current I/O is cancelled.
But if d = 0, nothing happens and the deadline isn't set.

The d = 0 case should be handled the same as d < 0.

Fixes #60282
Fixes #52896

Change-Id: Id8167db3604db1c129d99376fa78a3da75417d20
Reviewed-on: https://go-review.googlesource.com/c/go/+/496137
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
Reviewed-by: David du Colombier <0intro@gmail.com>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-05-19 17:10:57 +00:00
Michael Anthony Knyszek
b1aadd034c runtime: emit STW events for all pauses, not just those for the GC
Currently STW events are only emitted for GC STWs. There's little reason
why the trace can't contain events for every STW: they're rare so don't
take up much space in the trace, yet being able to see when the world
was stopped is often critical to debugging certain latency issues,
especially when they stem from user-level APIs.

This change adds new "kinds" to the EvGCSTWStart event, renames the
GCSTW events to just "STW," and lets the parser deal with unknown STW
kinds for future backwards compatibility.

But, this change must break trace compatibility, so it bumps the trace
version to Go 1.21.

This change also includes a small cleanup in the trace command, which
previously checked for STW events when deciding whether user tasks
overlapped with a GC. Looking at the source, I don't see a way for STW
events to ever enter the stream that that code looks at, so that
condition has been deleted.

Change-Id: I9a5dc144092c53e92eb6950e9a5504a790ac00cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/494495
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-19 17:06:45 +00:00
cuiweixie
944911af56 html/template: expose ErrJSTemplate
Fixes #59584

Change-Id: Iabe61476c7457dfffbfe5d0b1fe904901a466c73
Reviewed-on: https://go-review.googlesource.com/c/go/+/496395
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: xie cui <523516579@qq.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2023-05-19 16:51:35 +00:00
Michael Anthony Knyszek
24617a068c runtime: flush idle time to sched.idleTime on limiter event consumption
This was an oversight, which might cause accounted-for idle time to be
lost. Noticed this while working on #60276.

Change-Id: Ic743785d6dc82555e660f2c9b6aaa9dedef56ed8
Reviewed-on: https://go-review.googlesource.com/c/go/+/496117
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2023-05-19 16:24:53 +00:00
Michael Anthony Knyszek
78df00dd6f runtime: fix lockrank ordering for pinner implementation
The new Pinner API's implementation imposes some partial-orders that are
safe but previously did not exist between a mspanSpecial, mheapSpecial,
and mheap. Fix that up in the lock ranking.

For #46787.

Change-Id: I51cc8f7f069240caeb44d749bed43515634f4814
Reviewed-on: https://go-review.googlesource.com/c/go/+/496193
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2023-05-19 16:15:06 +00:00
cuiweixie
f283cba396 maps,runtime: improve maps.Keys
name     old time/op    new time/op    delta
Keys-10    8.65ms ± 0%    7.06ms ± 0%  -18.41%  (p=0.000 n=9+9)

name     old alloc/op   new alloc/op   delta
Keys-10    58.2kB ± 1%    47.4kB ± 2%  -18.54%  (p=0.000 n=10+10)

name     old allocs/op  new allocs/op  delta
Keys-10      0.00           0.00          ~     (all equal)

Change-Id: Ia7061c37be89e906e79bc3ef3bb1ef0d7913f9f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/481435
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: xie cui <523516579@qq.com>
2023-05-19 15:54:43 +00:00
Sven Anderson
251daf46fb runtime: implement Pinner API for object pinning
Some C APIs require the use or structures that contain pointers to
buffers (iovec, io_uring, ...).  The pointer passing rules would
require that these buffers are allocated in C memory and to process
this data with Go libraries it would need to be copied.

In order to provide a zero-copy way to use these C APIs, this CL
implements a Pinner API that allows to pin Go objects, which
guarantees that the garbage collector does not move these objects
while pinned.  This allows to relax the pointer passing rules so that
pinned pointers can be stored in C allocated memory or can be
contained in Go memory that is passed to C functions.

The Pin() method accepts pointers to objects of any type and
unsafe.Pointer.  Slices and arrays can be pinned by calling Pin()
with the pointer to the first element.  Pinning of maps is not
supported.

If the GC collects unreachable Pinner holding pinned objects it
panics.  If Pin() is called with the other non-pointer types it
panics as well.

Performance considerations: This change has no impact on execution
time on existing code, because checks are only done in code paths,
that would panic otherwise.  The memory footprint on existing code is
one pointer per memory span.

Fixes: #46787

Signed-off-by: Sven Anderson <sven@anderson.de>
Change-Id: I110031fe789b92277ae45a9455624687bd1c54f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/367296
Auto-Submit: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
2023-05-19 14:59:14 +00:00
Oleksandr Redko
3b9f99ebaa go/types: fix typos in comments
Change-Id: Ifadb11dc8ba8e63c736cb7ac277247bc587ce4b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/496136
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2023-05-19 14:31:23 +00:00
Michael Anthony Knyszek
e97bd776f9 runtime: make the memory limit heap goal headroom proportional
Currently if GOGC=off and GOMEMLIMIT is set, then the synchronous
scavenger is likely to work fairly often to maintain the limit, since
the heap goal goes right up to the edge of the memory limit (minus a
fixed 1 MiB of headroom).

If the application's allocation rate is high, and page-level
fragmentation is high, then most allocations will scavenge.

This change mitigates this problem by adding a proportional component
to constant headroom added to the memory-limit-based heap goal. This
means the runtime will have much more headroom before fragmentation
forces memory to be eagerly scavenged.

The proportional headroom in this case is 3%, or ~30 MiB for a 1 GiB
heap. This technically will increase GC frequency in the GOGC=off case
by a tiny amount, but will likely have a positive impact on both
allocation throughput and latency that outweighs this difference.

I wrote a small program to reproduce this issue and confirmed that the
issue is resolved by this patch:

https://github.com/golang/go/issues/57069#issuecomment-1551746565

This value of 3% is chosen as it seems to be a inflection point in this
particular small program. 2% still resulted in quite a bit of eager
scavenging work. I confirmed this results in a GC frequency increase of
about 3%.

This choice is still somewhat arbitrary because the program is
arbitrary, so perhaps worth revisiting in the future. Still, it should
help a good number of programs.

Fixes #57069.

Change-Id: Icb9829db0dfefb4fe42a0cabc5aa8d35970dd7d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/460375
Reviewed-by: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 13:53:21 +00:00
Tobias Klauser
9ed2b115fb runtime: report correct fcntl syscall error in checkCloseonexec
runtime.fcntl returns the error value as a negative value, so it needs
to be inverted before being converted to syscall.Errno.

Change-Id: I43cd0b035150424ac59e623b17a9396c7d62c186
Reviewed-on: https://go-review.googlesource.com/c/go/+/495675
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2023-05-19 13:50:58 +00:00
Robert Griesemer
08e39a1961 go/types, types2: fix comments in unify.go
No code changes.

Change-Id: I4c17e87673fc7dc1d87807e73beec828cbd4289b
Reviewed-on: https://go-review.googlesource.com/c/go/+/496355
Run-TryBot: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
2023-05-19 13:43:15 +00:00
Robert Griesemer
ade3f3f5ef go/types, types2: type-check built-ins even if there's a version error
There is no harm in continuing type-checking a built-in even if there
is a version error.

Change-Id: I161abd904a26075694c26639e247a17126947fcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/496415
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
2023-05-19 13:42:46 +00:00
Michael Anthony Knyszek
a3e90dc377 runtime: add eager scavenging details to GODEBUG=scavtrace=1
Also, clean up atomics on released-per-cycle while we're here.

For #57069.

Change-Id: I14026e8281f01dea1e8c8de6aa8944712b7b24d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/495916
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 13:38:43 +00:00
Alessandro Arzilli
0bbb54a02d cmd/link: add ReflectMethod flag to -dumpdep output
Adds ReflectMethod flag to the output of -dumpdep.

Fixes #60221

Change-Id: I631d72ee21b819a3a629780317ff4d962cc121e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/495715
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: Heschi Kreinick <heschi@google.com>
2023-05-19 12:07:32 +00:00
Bryan C. Mills
d0c72c23dc cmd/go: add a GODEBUG to limit the number of concurrent network connections
I implemented this in order to debug connection failures on a
new-to-me VM development environment that uses Cloud NAT. It doesn't
directly fix the bug, but perhaps folks will find it useful to
diagnose port-exhaustion-related flakiness in other environments.

For #52545.

Change-Id: Icd3f13dcf62e718560c4f4a965a4df7c1bd785ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/473277
Run-TryBot: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
2023-05-19 02:47:12 +00:00
Austin Clements
8f1031df77 cmd/cgo/internal/testcarchive: fix nocgo and no-c-archive builds
CL 495918 enabled testcarchive much more widely and added many dynamic
test skips. CL 495855 added TestDeepStack before these dynamic skips
were in. Unfortunately, the two CLs don't logically commute, so when
CL 495918 landed, it broke at least nocgo builders and platforms that
don't support c-archive builds. Fix this by adding the necessary skips
to TestDeepStack.

Change-Id: I3d352f731fe67a01c7b96871fde772db8eb21b5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/496376
Auto-Submit: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
2023-05-19 02:15:30 +00:00
Robert Griesemer
4088e97fc2 cmd/compile: use more lenient type inference for untyped arguments for go1.21
This CL permanently enables the new behavior for -lang=go1.21 and
newer, and keeps the existing behavior if -lang=go1.20 or older.

To be submitted once #58671 is accepted.

For #58671.

Change-Id: I83a1d393f0ce7871be8f38ec35742d393946c55f
Reviewed-on: https://go-review.googlesource.com/c/go/+/496035
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Findley <rfindley@google.com>
2023-05-19 01:56:58 +00:00
Austin Clements
ad7d1d995f cmd/cgo/internal/testsovar: merge into testso
The test driver for testso and testsovar are literally identical, and
only the testdata code is different between the two test packages.
Merge them into a single test package with two tests that share a
driver.

Change-Id: I3f107a6aba345c0dd58606c10e3ac8eee33b33c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/496315
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2023-05-19 01:37:41 +00:00
Austin Clements
b42ce3a65f cmd/cgo/internal/testtls: cleanup and support more arches
Currently, this test only enabled on non-Darwin UNIX platforms because
it uses the non-standard _thread attribute for thread-local storage.
C11 introduced a standard way to declare something thread-local, so
this CL takes advantage of that to generalize the test to Darwin and
Windows.

Change-Id: Iba31b6216721df6eb8e978d7487cd3a787cae588
Reviewed-on: https://go-review.googlesource.com/c/go/+/496295
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
2023-05-19 01:37:39 +00:00
Austin Clements
3ebbaf3a4d cmd/dist: drop remaining conditions on default cgo tests
Currently, dist registers cmd/cgo/internal{test,testtls,testnocgo}
specially, so they're opted out of "go test std cmd". It has to
register these test packages to run in various non-default build
configurations, but at this point they can also run with the default
build configuration (and for test and testtls, we intentionally want
to test them in the default configuration; this is pointless but
harmless for testnocgo). Hence, this CL drops the special registration
of their default build configurations from registerCgoTests and lets
them be registered as part of registerStdTests.

Change-Id: Id283f3cdcdb202955a854648c0ed1e3c4aa554d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/496179
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-19 01:37:38 +00:00
Austin Clements
cbd3305f8c cmd/cgo/internal/testnocgo: always run in default configuration
This test is actually intended to test that we can build in -static
mode even without any cgo. That means it's quite harmless to run in
the default build configuration (in addition to running with various
other build configurations).

Change-Id: Ic6cb5c0eaab83f9bd5718aae57d0fdc69afcb8b6
Reviewed-on: https://go-review.googlesource.com/c/go/+/496178
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 01:37:37 +00:00
Austin Clements
ab3c2b6d89 cmd/cgo/internal/testtls: build and run everywhere
This makes testtls build and run on all platforms in the default build
configuration (though it will Skip on some).

Change-Id: I6aba96a82d618c9798a0d4418b40b2644cfceec9
Reviewed-on: https://go-review.googlesource.com/c/go/+/496177
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-19 01:37:35 +00:00
Austin Clements
408f7a4663 cmd/dist: don't pass -linkmode=auto
This is the default value of this flag, so passing it clutters up
debugging output. This also makes it clearer which tests are running
with a default configuration.

Change-Id: If793934829c79f087c7a6e3fa8f64dc33959c213
Reviewed-on: https://go-review.googlesource.com/c/go/+/496176
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 01:37:34 +00:00
Austin Clements
18ffa7185d cmd/dist: let several cgo tests run as regular cmd tests
Several cgo tests no longer have any special conditions, so they can
just be normal cmd tests. This brings dist's "go_test:.*" tests much
closer to what "go test std cmd" runs.

Change-Id: I4d09f60628a41081e97e6b6e7dd0d93df47a65bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/495919
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-19 01:37:32 +00:00
Austin Clements
a674ab1961 cmd/dist: refine test conditions and enable more cgo tests on Android, iOS
This CL moves many cgo test conditions out of dist and into the tests
themselves, now that they can use the testenv.Must* helpers.

This refines a lot of the conditions, which happens to have the effect
of enabling many tests on Android and iOS that are disabled by
too-coarse GOOS checks in dist today.

Fixes #15919.

Change-Id: I2947526b08928d2f7f89f107b5b2403b32092ed8
Reviewed-on: https://go-review.googlesource.com/c/go/+/495918
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
2023-05-19 01:37:31 +00:00
Austin Clements
071770b846 cmd/dist: enable more cgo tests if GO_GCFLAGS != ""
Currently, we have several tests disabled if GO_GCFLAGS is non-empty.

Long ago, this was critical because many of these tests use "go
install" with no -gcflags and would thus overwrite std packages in
GOROOT built with -gcflags=$GO_GCFLAGS. Now these packages all live in
the build cache, so this is no longer a concern.

The other reason for this (the reason given in the code comment), is
that these tests will rebuild significant portions of std without
flags. While this is still theoretically true, there are many tests
that run "go build" with no -gcflags, so these tests don't contribute
much overall.

Empirically, on my linux/amd64 host, running these tests at all grows
the Go build cache by 14%, from 1.899 GB to 2.165 GB. When building
with GO_GCFLAGS="-N -l" (the only use case on the builders), enabling
them grows the Go build cache by 18%, from 1.424 GB to 1.684 GB. This
is only a 4 percentage point difference, and still results in a build
cache that's smaller than the default build

Given all this, there's little reason to carry the complexity of
disabling these tests when GO_GCFLAGS != "". Removing this condition
is a step toward running these as regular cmd tests.

Change-Id: I2c41be721927c40a742e01476cd9a0f7650d38e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/495917
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2023-05-19 01:37:29 +00:00
Russ Cox
e95982fe30 cmd/api: skip TestIssue21181
It is incredibly slow, taking half as long as the regular cmd/api checks
and over 5 minutes on plan9-arm. Leave it for the longtest builders.

Change-Id: Ic8bd420f174268d0b6a13d84e7bd364f6c13cf41
Reviewed-on: https://go-review.googlesource.com/c/go/+/496375
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2023-05-19 01:36:29 +00:00
Alan Donovan
3ca52f4c31 slices: add in-place Reverse function
Fixes #58565

Change-Id: I583f8380c12386178fb18e553322bbb019d9fae0
Reviewed-on: https://go-review.googlesource.com/c/go/+/468855
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Shay Nehmad <dude500@gmail.com>
2023-05-19 00:43:02 +00:00
Oleksandr Redko
58d40d156e os: remove unnecessary return after t.Fatal
Change-Id: Ibddf36431abb799d8f9288d6e17159ce1538d62e
Reviewed-on: https://go-review.googlesource.com/c/go/+/495879
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-19 00:41:52 +00:00
Heschi Kreinick
38ae922b9b doc: add TODOs to go1.21
Generated with relnote. I did my best to put things where they made
sense, but some weren't obvious, like the Unicode upgrade and backward
compatibility stuff.

For #58645.

Change-Id: Ic3cd9ef32cec7591ace4d2df1da40e4afd97d083
Reviewed-on: https://go-review.googlesource.com/c/go/+/496316
Auto-Submit: Heschi Kreinick <heschi@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Heschi Kreinick <heschi@google.com>
2023-05-18 22:21:11 +00:00
Robert Griesemer
edbef1a848 go/types, types2: remove unnecessary assignment (minor cleanup)
Change-Id: I77e5056a159b6041ca49480a3c493a515d4b3a2f
Reviewed-on: https://go-review.googlesource.com/c/go/+/496255
Reviewed-by: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Run-TryBot: Robert Griesemer <gri@google.com>
Auto-Submit: Robert Griesemer <gri@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2023-05-18 22:02:20 +00:00