1
0
mirror of https://github.com/golang/go synced 2024-11-24 06:50:17 -07:00
Commit Graph

52136 Commits

Author SHA1 Message Date
Paul E. Murphy
8375b54d44 internal/bytealg: improve PPC64 equal
Rewrite the vector loop to process 64B per iteration,
this greatly improves performance on POWER9/POWER10 for
large sizes.

Likewise, use a similar tricks for sizes >= 8 and <= 64.

And, rewrite small comparisons, it's a little slower for 1 byte,
but constant time for 1-7 bytes. Thus, it is increasingly faster
for 2-7B.

Benchmarks results below are from P8/P9 ppc64le (in that order),
several additional testcases have been added to test interesting
sizes. Likewise, the old variant was padded to the same code
size of the new variant to minimize layout related noise:

POWER8/ppc64le/linux:
name       old speed      new speed       delta
Equal/1     110MB/s ± 0%    106MB/s ± 0%    -3.26%
Equal/2     202MB/s ± 0%    203MB/s ± 0%    +0.18%
Equal/3     280MB/s ± 0%    319MB/s ± 0%   +13.89%
Equal/4     350MB/s ± 0%    414MB/s ± 0%   +18.27%
Equal/5     412MB/s ± 0%    533MB/s ± 0%   +29.19%
Equal/6     462MB/s ± 0%    620MB/s ± 0%   +34.11%
Equal/7     507MB/s ± 0%    745MB/s ± 0%   +47.02%
Equal/8     913MB/s ± 0%    994MB/s ± 0%    +8.84%
Equal/9     909MB/s ± 0%   1117MB/s ± 0%   +22.85%
Equal/10    937MB/s ± 0%   1242MB/s ± 0%   +32.59%
Equal/11    962MB/s ± 0%   1370MB/s ± 0%   +42.37%
Equal/12    989MB/s ± 0%   1490MB/s ± 0%   +50.60%
Equal/13   1.01GB/s ± 0%   1.61GB/s ± 0%   +60.27%
Equal/14   1.02GB/s ± 0%   1.74GB/s ± 0%   +71.22%
Equal/15   1.03GB/s ± 0%   1.86GB/s ± 0%   +81.45%
Equal/16   1.60GB/s ± 0%   1.99GB/s ± 0%   +24.21%
Equal/17   1.54GB/s ± 0%   2.04GB/s ± 0%   +32.28%
Equal/20   1.48GB/s ± 0%   2.40GB/s ± 0%   +62.64%
Equal/32   3.58GB/s ± 0%   3.84GB/s ± 0%    +7.18%
Equal/63   3.74GB/s ± 0%   7.17GB/s ± 0%   +91.79%
Equal/64   6.35GB/s ± 0%   7.29GB/s ± 0%   +14.75%
Equal/65   5.85GB/s ± 0%   7.00GB/s ± 0%   +19.66%
Equal/127  6.74GB/s ± 0%  13.74GB/s ± 0%  +103.77%
Equal/128  10.6GB/s ± 0%   12.9GB/s ± 0%   +21.98%
Equal/129  9.66GB/s ± 0%  11.96GB/s ± 0%   +23.85%
Equal/191  9.12GB/s ± 0%  17.80GB/s ± 0%   +95.26%
Equal/192  13.4GB/s ± 0%   17.2GB/s ± 0%   +28.66%
Equal/4K   29.5GB/s ± 0%   37.3GB/s ± 0%   +26.39%
Equal/4M   22.6GB/s ± 0%   23.1GB/s ± 0%    +2.40%
Equal/64M  10.6GB/s ± 0%   11.2GB/s ± 0%    +5.83%

POWER9/ppc64le/linux:
name       old speed      new speed       delta
Equal/1     122MB/s ± 0%    121MB/s ± 0%    -0.94%
Equal/2     223MB/s ± 0%    241MB/s ± 0%    +8.29%
Equal/3     289MB/s ± 0%    362MB/s ± 0%   +24.90%
Equal/4     366MB/s ± 0%    483MB/s ± 0%   +31.82%
Equal/5     427MB/s ± 0%    603MB/s ± 0%   +41.28%
Equal/6     462MB/s ± 0%    723MB/s ± 0%   +56.65%
Equal/7     509MB/s ± 0%    843MB/s ± 0%   +65.57%
Equal/8     974MB/s ± 0%   1066MB/s ± 0%    +9.46%
Equal/9    1.00GB/s ± 0%   1.20GB/s ± 0%   +19.53%
Equal/10   1.00GB/s ± 0%   1.33GB/s ± 0%   +32.81%
Equal/11   1.01GB/s ± 0%   1.47GB/s ± 0%   +45.28%
Equal/12   1.04GB/s ± 0%   1.60GB/s ± 0%   +53.46%
Equal/13   1.05GB/s ± 0%   1.73GB/s ± 0%   +64.67%
Equal/14   1.02GB/s ± 0%   1.87GB/s ± 0%   +82.93%
Equal/15   1.04GB/s ± 0%   2.00GB/s ± 0%   +92.07%
Equal/16   1.83GB/s ± 0%   2.13GB/s ± 0%   +16.58%
Equal/17   1.78GB/s ± 0%   2.18GB/s ± 0%   +22.65%
Equal/20   1.72GB/s ± 0%   2.57GB/s ± 0%   +49.24%
Equal/32   3.89GB/s ± 0%   4.10GB/s ± 0%    +5.53%
Equal/63   3.63GB/s ± 0%   7.63GB/s ± 0%  +110.45%
Equal/64   6.69GB/s ± 0%   7.75GB/s ± 0%   +15.84%
Equal/65   6.28GB/s ± 0%   7.07GB/s ± 0%   +12.46%
Equal/127  6.41GB/s ± 0%  13.65GB/s ± 0%  +112.95%
Equal/128  11.1GB/s ± 0%   14.1GB/s ± 0%   +26.56%
Equal/129  10.2GB/s ± 0%   11.2GB/s ± 0%    +9.44%
Equal/191  8.64GB/s ± 0%  16.39GB/s ± 0%   +89.75%
Equal/192  15.3GB/s ± 0%   17.8GB/s ± 0%   +16.31%
Equal/4K   24.6GB/s ± 0%   27.8GB/s ± 0%   +13.12%
Equal/4M   21.1GB/s ± 0%   22.7GB/s ± 0%    +7.66%
Equal/64M  20.8GB/s ± 0%   22.4GB/s ± 0%    +8.06%

Change-Id: Ie3c582133d526cc14e8846ef364c44c93eb7b9a2
Reviewed-on: https://go-review.googlesource.com/c/go/+/399976
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
2022-05-02 20:18:15 +00:00
Austin Clements
3b01a80319 cmd/compile: fix loopreschedchecks for regabi
The loopreschedchecks pass (GOEXPERIMENT=preemptibleloops) had
bit-rotted in two ways because of the regabi experiment:

1. The call to goschedguarded was generating a pre-regabi StaticCall.
   This CL updates it to construct a new-style StaticCall.

2. The mem finder did not account for tuples or results containing a
   mem. This caused it to construct phis that were supposed to thread
   the mem into the added blocks, but they could instead thread a
   tuple or results containing a mem, causing things to go wrong
   later. This CL updates the mem finder to add an op to select out
   the mem if it finds the last live mem in a block is a tuple or
   results. This isn't ideal since we'll deadcode out most of these,
   but it's the easiest thing to do and this is just an experiment.

Tested by running the runtime tests. Ideally we'd have a real test for
this, but I don't think it's worth the effort for code that clearly
hasn't been enabled by anyone for at least a year.

Change-Id: I8ed01207637c454b68a551d38986c947e17d520b
Reviewed-on: https://go-review.googlesource.com/c/go/+/403475
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2022-05-02 19:53:56 +00:00
Matthew Dempsky
d4bfc87218 cmd/compile: simplify code from CL 398474
This CL:

1. extracts typecheck.LookupNum into a method on *types.Pkg, so that
it can be used with any package, not just types.LocalPkg,

2. adds a new helper function closureSym to generate symbols in the
appropriate package as needed within stencil.go, and

3. updates the existing typecheck.LookupNum+Name.SetSym code to call
closureSym instead.

No functional change (so no need to backport to Go 1.18), but a little
cleaner, and avoids polluting types.LocalPkg.Syms with symbols that we
won't end up using.

Updates #52117.

Change-Id: Ifc8a3b76a37c830125e9d494530d1f5b2e3e3e2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/403197
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
2022-05-02 18:50:03 +00:00
Xiaodong Liu
af99c2092a cmd/internal/objabi: define Go relocation types for loong64
Contributors to the loong64 port are:
  Weining Lu <luweining@loongson.cn>
  Lei Wang <wanglei@loongson.cn>
  Lingqin Gong <gonglingqin@loongson.cn>
  Xiaolin Zhao <zhaoxiaolin@loongson.cn>
  Meidan Li <limeidan@loongson.cn>
  Xiaojuan Zhai <zhaixiaojuan@loongson.cn>
  Qiyuan Pu <puqiyuan@loongson.cn>
  Guoqi Chen <chenguoqi@loongson.cn>

This port has been updated to Go 1.15.6:
https://github.com/loongson/go

Updates #46229

Change-Id: I8d31b3cd827325aa0ff748ca8c0c0da6df6ed99f
Reviewed-on: https://go-review.googlesource.com/c/go/+/396734
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-05-02 18:38:26 +00:00
Russ Cox
d922c0a8f5 all: use os/exec instead of internal/execabs
We added internal/execabs back in January 2021 in order to fix
a security problem caused by os/exec's handling of the current
directory. Now that os/exec has that code, internal/execabs is
superfluous and can be deleted.

This commit rewrites all the imports back to os/exec and
deletes internal/execabs.

For #43724.

Change-Id: Ib9736baf978be2afd42a1225e2ab3fd5d33d19df
Reviewed-on: https://go-review.googlesource.com/c/go/+/381375
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-05-02 17:49:12 +00:00
Wayne Zuo
64369c3ea0 A+C: add Wayne Zuo (individual CLA)
Change-Id: I12fe0b7952a41f6d0f78f892d823244793745279
Reviewed-on: https://go-review.googlesource.com/c/go/+/403336
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2022-05-02 17:05:14 +00:00
Russ Cox
06338941ea net/http: fix for recent go.mod update
cmd/internal/moddeps was failing.
Ran the commands it suggested:

% go mod tidy                               # to remove extraneous dependencies
% go mod vendor                             # to vendor dependencies
% go generate -run=bundle std               # to regenerate bundled packages
% go generate syscall internal/syscall/...  # to regenerate syscall packages

cmd/internal/moddeps is happy now.

Change-Id: I4ee212cdc323f62a6cdcfdddb6813397b23d89e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/403454
Run-TryBot: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-02 15:27:00 +00:00
Russ Cox
349cc83389 os/exec: return error when PATH lookup would use current directory
CL 381374 was reverted because x/sys/execabs broke.

This CL reapplies CL 381374, but adding a lookPathErr error
field back, for execabs to manipulate with reflect.
That field will just be a bit of scar tissue in this package forever,
to keep old code working with new toolchains.

CL 403256 fixes x/sys/execabs's test to be ready for the change.
Older versions of x/sys/execabs will keep working
(that is, will keep rejecting what they should reject),
but they will return a slightly different error from LookPath
without that CL, and the test fails because of the different
error text.

For #43724.

This reverts commit f2b674756b.

Change-Id: Iee55f8cd9939e1bd31e5cbdada50681cdc505117
Reviewed-on: https://go-review.googlesource.com/c/go/+/403274
Auto-Submit: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-02 14:54:05 +00:00
Baokun Lee
32acceb359 internal/poll: clear completed Buffers to permit earlier collection
Updates #45163

Change-Id: I73a6f22715550e0e8b83fbd3ebec72ef019f153f
Reviewed-on: https://go-review.googlesource.com/c/go/+/373374
Run-TryBot: Lee Baokun <bk@golangcn.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-05-02 14:47:27 +00:00
Gregory Man
2719b1a9a8 compress/flate: return error on closed stream write
Previously flate.Writer allowed writes after Close, and this behavior
could lead to stream corruption.

Fixes #27741

Change-Id: Iee1ac69f8199232f693dba77b275f7078257b582
Reviewed-on: https://go-review.googlesource.com/c/go/+/136475
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
2022-05-02 14:46:44 +00:00
Ludi Rehak
fdf1d768f2 all: rename type *testing.B variable to 'b'
Reserve 't' for type *testing.T variables.

Change-Id: I037328df59d3af1aa28714f9efe15695b6fd62a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/400826
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-02 14:44:49 +00:00
Xiaodong Liu
7a22c8a07f cmd/cgo: configure cgo tool for loong64
Define pointer and int type size for loong64
Add "-mabi=lp64d" argument to gcc

Contributors to the loong64 port are:
  Weining Lu <luweining@loongson.cn>
  Lei Wang <wanglei@loongson.cn>
  Lingqin Gong <gonglingqin@loongson.cn>
  Xiaolin Zhao <zhaoxiaolin@loongson.cn>
  Meidan Li <limeidan@loongson.cn>
  Xiaojuan Zhai <zhaixiaojuan@loongson.cn>
  Qiyuan Pu <puqiyuan@loongson.cn>
  Guoqi Chen <chenguoqi@loongson.cn>

This port has been updated to Go 1.15.6:
  https://github.com/loongson/go

Updates #46229

Change-Id: I9699fd9af0112e72193ac24b736b85c580887a0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/342305
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
2022-05-02 04:48:54 +00:00
Jorropo
d8723745ba vendor, net/http: update golang.org/x/net to tip
Needed for CL 400236.

Change-Id: Ia0b4a5963724ed92be27f557ad141335b389e97f
GitHub-Last-Rev: b0e72cb26d
GitHub-Pull-Request: golang/go#52621
Reviewed-on: https://go-review.googlesource.com/c/go/+/403136
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-05-02 04:45:15 +00:00
Xiaodong Liu
d09ca2cb8e cmd/go/internal: configure go tool workflow for loong64
Contributors to the loong64 port are:
  Weining Lu <luweining@loongson.cn>
  Lei Wang <wanglei@loongson.cn>
  Lingqin Gong <gonglingqin@loongson.cn>
  Xiaolin Zhao <zhaoxiaolin@loongson.cn>
  Meidan Li <limeidan@loongson.cn>
  Xiaojuan Zhai <zhaixiaojuan@loongson.cn>
  Qiyuan Pu <puqiyuan@loongson.cn>
  Guoqi Chen <chenguoqi@loongson.cn>

This port has been updated to Go 1.15.6:
  https://github.com/loongson/go

Updates #46229

Change-Id: I6b537a7d842b0683586917fe7ea7cd4d70d888de
Reviewed-on: https://go-review.googlesource.com/c/go/+/342308
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Bryan Mills <bcmills@google.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
2022-05-01 00:24:25 +00:00
Xiaodong Liu
edab07d09f cmd/dist: support dist tool for loong64
Contributors to the loong64 port are:
  Weining Lu <luweining@loongson.cn>
  Lei Wang <wanglei@loongson.cn>
  Lingqin Gong <gonglingqin@loongson.cn>
  Xiaolin Zhao <zhaoxiaolin@loongson.cn>
  Meidan Li <limeidan@loongson.cn>
  Xiaojuan Zhai <zhaixiaojuan@loongson.cn>
  Qiyuan Pu <puqiyuan@loongson.cn>
  Guoqi Chen <chenguoqi@loongson.cn>

This port has been updated to Go 1.15.6:
  https://github.com/loongson/go

Updates #46229

Change-Id: I61dca43680d8e5bd3198a38577450a53f405a987
Reviewed-on: https://go-review.googlesource.com/c/go/+/342307
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-05-01 00:05:20 +00:00
Aleksandr Dobkinimg src=404 onerror=alert(document.domain)
fd6c556dc8 net/url: fix regex typo in comment in url.go
The original author almost certainly intended to match the literal dash
character '-' but ended up matching a range of other characters instead.

Change-Id: I7a0f67c5fdccd70b7ad58a882da851dfc22ce2f0
GitHub-Last-Rev: 76a39577e5
GitHub-Pull-Request: golang/go#52627
Reviewed-on: https://go-review.googlesource.com/c/go/+/403076
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
2022-04-30 04:04:40 +00:00
Bryan C. Mills
0ebe224628 os/signal: scale back the solaris-amd64-oraclerel settle time
The settleTime is arbitrary. Ideally we should refactor the test to
avoid it (using subprocesses instead of sleeps to isolate tests from
each others' delayed signals), but as a shorter-term workaround let's
try scaling it back to match linux/ppc64 (the other builder that
empirically requires a longer settleTime).

For #51054.
Updates #33174.

Change-Id: I574fffaadd74c52c13d63974e87f20b6d3cf3c4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/403199
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Bryan Mills <bcmills@google.com>
2022-04-30 02:35:50 +00:00
Dan Kortschak
11a650bb4a reflect: ensure map keys match key type in MapIndex and SetMapIndex
name                          old time/op    new time/op    delta
Map/StringKeys/MapIndex-8           2.36µs ± 5%    2.55µs ±11%  +7.98%  (p=0.006 n=10+9)
Map/StringKeys/SetMapIndex-8        4.86µs ± 7%    4.77µs ± 1%    ~     (p=0.211 n=10+9)
Map/StringKindKeys/MapIndex-8       2.29µs ± 3%    2.28µs ± 4%    ~     (p=0.631 n=10+10)
Map/StringKindKeys/SetMapIndex-8    4.44µs ± 3%    4.61µs ± 1%  +3.78%  (p=0.000 n=10+10)
Map/Uint64Keys/MapIndex-8           3.42µs ± 9%    3.11µs ± 2%  -9.20%  (p=0.000 n=10+9)
Map/Uint64Keys/SetMapIndex-8        5.17µs ± 3%    5.00µs ± 1%  -3.23%  (p=0.000 n=9+10)

Fixes #52379

Change-Id: I545c71ea3145280828ca4186aad036a6c02016ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/400635
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Joseph Tsai <joetsai@digital-static.net>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2022-04-30 00:14:28 +00:00
Bryan Mills
f2b674756b Revert "os/exec: return error when PATH lookup would use current directory"
This reverts CL 381374.

Reason for revert: broke tests for x/sys/execabs.

Updates #43724.
Updates #43947.

Change-Id: I9eb3adb5728dead66dbd20f6afe1e7a77e2a26f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/403058
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Auto-Submit: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Bryan Mills <bcmills@google.com>
2022-04-29 23:04:17 +00:00
Keith Randall
2278a51fa0 sync/atomic: use consistent first-store-in-progress marker
We need to use the same marker everywhere. My CL to rename the
marker (CL 241661) and the CL to add more uses of the marker
under the old name (CL 241678) weren't coordinated with each other.

Fixes #52612

Change-Id: I97023c0769e518491924ef457fe03bf64a2cefa6
Reviewed-on: https://go-review.googlesource.com/c/go/+/403094
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2022-04-29 20:42:52 +00:00
Russ Cox
3ce203db80 os/exec: return error when PATH lookup would use current directory
Following discussion on #43724, change os/exec to take the
approach of golang.org/x/sys/execabs, refusing to respect
path entries mentioning relative paths by default.

Code that insists on being able to find executables in relative
directories in the path will need to add a couple lines to override the error.

See the updated package docs in exec.go for more details.

Fixes #43724.
Fixes #43947.

Change-Id: I73c1214f322b60b4167a23e956e933d50470fe13
Reviewed-on: https://go-review.googlesource.com/c/go/+/381374
Reviewed-by: Bryan Mills <bcmills@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Bryan Mills <bcmills@google.com>
2022-04-29 20:16:31 +00:00
Matthew Dempsky
8c5917cd76 cmd/compile: consistent unified IR handling of package unsafe
Within the unified IR export format, I was treating package unsafe as
a normal package, but expecting importers to correctly handle
deduplicating it against their respective representation of package
unsafe.

However, the surrounding importer logic differs slightly between
cmd/compile/internal/noder (which unified IR was originally
implemented against) and go/importer (which it was more recently
ported to). In particular, noder initializes its packages map as
`map[string]*types2.Package{"unsafe": types2.Unsafe}`, whereas
go/importer initializes it as just `make(map[string]*types.Package)`.

This CL makes them all consistent. In particular, it:

1. changes noder to initialize packages to an empty map to prevent
further latent issues from the discrepency,

2. adds the same special handling of package unsafe already present in
go/internal/gcimporter's unified IR reader to both of cmd/compile's
implementations, and

3. changes the unified IR writer to treat package unsafe as a builtin
package, to force that readers similarly handle it correctly.

Fixes #52623.

Change-Id: Ibbab9b0a1d2a52d4cc91b56c5df49deedf81295a
Reviewed-on: https://go-review.googlesource.com/c/go/+/403196
Auto-Submit: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29 18:35:07 +00:00
Illirgway
e7c56fe994 strings: adding micro-optimization for TrimSpace
replace for string's end trimming TrimFunc -> TrimRightFunc

strings.TrimSpace string's end trimming should use more specific TrimRightFunc instead of common TrimFunc (because start has already trimmed before)

Change-Id: I827f1a25c141e61edfe1f8b11f6e8cd685f8b384
GitHub-Last-Rev: 040607a831
GitHub-Pull-Request: golang/go#46862
Reviewed-on: https://go-review.googlesource.com/c/go/+/329731
Auto-Submit: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-04-29 02:01:27 +00:00
hopehook
a5dd684293 A+C: add Shuai Tan (individual CLA)
Replaced github username with my real name and added to AUTHORS.

Change-Id: Id68c30eeb31b3f0b7cecc434462029843758f9f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/402794
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2022-04-29 01:45:33 +00:00
Ludi Rehak
e7b0559448 regexp/syntax: fix typo in comment
Fix typo in comment describing IsWordChar.

Change-Id: Ia283813cf5662e218ee6d0411fb0c1b1ad1021f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/393435
Auto-Submit: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-29 01:00:55 +00:00
Michael Pratt
cb6fc99b32 runtime: mark sigtramp as TOPFRAME on the rest of unix
This extends CL 402190 from Linux to the rest of the Unix OSes.

Marking sigtramp as TOPFRAME allows gentraceback to stop tracebacks at
the end of a signal handler, since there is not much beyond sigtramp.

Change-Id: I8b7f5d55d41889f59c0a79c65351b9b0b2d77717
Reviewed-on: https://go-review.googlesource.com/c/go/+/402934
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Michael Pratt <mpratt@google.com>
2022-04-28 23:16:44 +00:00
Cuong Manh Le
c90a19a760 cmd/compile: set correct package for vars/params/results from nested instantiation
Fixes #52117

Change-Id: Ib5b2cdbdbce1d516aa10a0df349449b756f2f404
Reviewed-on: https://go-review.googlesource.com/c/go/+/398474
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2022-04-28 20:14:34 +00:00
Cherry Mui
74f0009422 runtime: use saved LR when unwinding through morestack
On LR machine, consider F calling G calling H, which grows stack.
The stack looks like
...
G's frame:
	... locals ...
	saved LR = return PC in F  <- SP points here at morestack
H's frame (to be created)

At morestack, we save
	gp.sched.pc = H's morestack call
	gp.sched.sp = H's entry SP (the arrow above)
	gp.sched.lr = return PC in G

Currently, when unwinding through morestack (if _TraceJumpStack
is set), we switch PC and SP but not LR. We then have
	frame.pc = H's morestack call
	frame.sp = H's entry SP (the arrow above)
As LR is not set, we load it from stack at *sp, so
	frame.lr = return PC in F
As the SP hasn't decremented at the morestack call,
	frame.fp = frame.sp = H's entry SP

Unwinding a frame, we have
	frame.pc = old frame.lr = return PC in F
	frame.sp = old frame.fp = H's entry SP a.k.a. G's SP
The PC and SP don't match. The unwinding will go off if F and G
have different frame sizes.

Fix this by preserving the LR when switching stack.

Also add code to detect infinite loop in unwinding.

TODO: add some test. I can reproduce the infinite loop (or throw
with added check) but the frequency is low.

May fix #52116.

Change-Id: I6e1294f1c6e55f664c962767a1cf6c466a0c0eff
Reviewed-on: https://go-review.googlesource.com/c/go/+/400575
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Eric Fang <eric.fang@arm.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2022-04-28 20:11:37 +00:00
Wayne Zuo
0eb93d6b43 math/big: delete all mulWW assembly code
Now gc can generate the same assembly code.

Change-Id: Iac503003e14045d63e2def66408c13cee516aa37
Reviewed-on: https://go-review.googlesource.com/c/go/+/402575
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Auto-Submit: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2022-04-28 20:09:27 +00:00
Cuong Manh Le
96a1ad298a cmd/compile: remove inTypeCheckInl boolean
After CL 402974, we do not typecheck an inline body in ImportedBody
anymore, thus the inTypeCheckInl check is un-necessary.

Change-Id: Ia6154a57db507c3e00b67333f1e3b910a6de7d05
Reviewed-on: https://go-review.googlesource.com/c/go/+/402975
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Auto-Submit: Keith Randall <khr@golang.org>
2022-04-28 18:41:25 +00:00
Cuong Manh Le
15381040fa cmd/compile: remove go117ExportTypes constant
Now, 1.17 is the least supported version, the compiler always write
type information when exporting function bodies. So we can get rid of
go117ExportTypes constant and all its conditional checking codes.

Change-Id: I9ac616509c30601e94f99426049d814328253395
Reviewed-on: https://go-review.googlesource.com/c/go/+/402974
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Keith Randall <khr@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2022-04-28 18:41:22 +00:00
Austin Clements
123e27170a runtime: clean up escaping in tests
There are several tests in the runtime that need to force various
things to escape to the heap. This CL centralizes this functionality
into runtime.Escape, defined in export_test.

Change-Id: I2de2519661603ad46c372877a9c93efef8e7a857
Reviewed-on: https://go-review.googlesource.com/c/go/+/402178
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Auto-Submit: Austin Clements <austin@google.com>
2022-04-28 18:28:44 +00:00
Cuong Manh Le
c15d0a93c7 cmd/compile: fix missing export/import init nodes of builtins that allow multiple arguments
Fixes #52590

Change-Id: Ibd0852ae2a9ad8e4598e93320daff1b3c196929f
Reviewed-on: https://go-review.googlesource.com/c/go/+/402854
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Robert Griesemer <gri@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
2022-04-28 18:02:40 +00:00
Michael Pratt
4289bd365c runtime: simply user throws, expand runtime throws
This gives explicit names to the possible states of throwing (-1, 0, 1).

m.throwing is now one of:

throwTypeOff: not throwing, previously == 0
throwTypeUser: user throw, previously == -1
throwTypeRuntime: runtime throw, previously == 1

For runtime throws, we now always include frame metadata and system
goroutines regardless of GOTRACEBACK to aid in debugging the runtime.

For user throws, we no longer include frame metadata or runtime frames,
unless GOTRACEBACK=system or higher.

For #51485.

Change-Id: If252e2377a0b6385ce7756b937929be4273a56c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/390421
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2022-04-28 17:14:41 +00:00
Michael Pratt
29bbca5c2c runtime: differentiate "user" and "system" throws
"User" throws are throws due to some invariant broken by the application.
"System" throws are due to some invariant broken by the runtime,
environment, etc (i.e., not the fault of the application).

This CL sends "user" throws through the new fatal. Currently this
function is identical to throw, but with a different name to clearly
differentiate the throw type in the stack trace, and hopefully be a bit
more clear to users what it means.

This CL changes a few categories of throw to fatal:

1. Concurrent map read/write.
2. Deadlock detection.
3. Unlock of unlocked sync.Mutex.
4. Inconsistent results from syscall.AllThreadsSyscall.

"Thread exhaustion" and "out of memory" (usually address space full)
throws are additional throws that are arguably the fault of user code,
but I've left off for now because there is no specific invariant that
they have broken to get into these states.

For #51485

Change-Id: I713276a6c290fd34a6563e6e9ef378669d74ae32
Reviewed-on: https://go-review.googlesource.com/c/go/+/390420
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
2022-04-28 16:50:31 +00:00
Michael Pratt
4bb45f7549 runtime: mark sigtramp as TOPFRAME
Currently throw() in the signal handler results in "fatal error: unknown
return pc from runtime.sigreturn ...".

Marking sigtramp as TOPFRAME allows gentraceback to stop tracebacks at
the end of a signal handler, since there is not much beyond sigtramp.

This is just done on Linux for now, but may apply to other Unix systems
as well.

Change-Id: I96edcb945283f417a5bfe00ce2fb2b1a0d578692
Reviewed-on: https://go-review.googlesource.com/c/go/+/402190
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2022-04-28 16:29:07 +00:00
Michael Pratt
17371eea25 syscall: add //go:norace to RawSyscall
RawSyscall is used in a variety of rather unsafe conditions, such as
after fork in forkAndExecInChild1. Disable race instrumentation to avoid
calling TSAN in unsafe conditions.

For #51087

Change-Id: I47c35e6f0768c77ddab99010ea0404c45ad2f1da
Reviewed-on: https://go-review.googlesource.com/c/go/+/402914
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
2022-04-28 16:00:39 +00:00
Wayne Zuo
4d350710a2 cmd/compile: intrinsify math/big.mulWW on MIPS64(LE),RISCV64
Change-Id: Ib8b18a892b0b1a59229ebc97a38200b53e701a77
Reviewed-on: https://go-review.googlesource.com/c/go/+/402574
Reviewed-by: Keith Randall <khr@google.com>
Run-TryBot: Wayne Zuo <wdvxdr@golangcn.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-28 15:11:42 +00:00
Filippo Valsorda
89044b6d42 math/big: add comprehensive aliasing tests (and minor fixes to Exp, Rand)
TestAlias checks systematically for aliasing issues, where passing the
same value for an argument and the receiver leads to incorrect results.

We had a number of issues like that over the years:

    - #31084: Lsh on arm64
    - #30217: GCD
    - #22830: Exp due to divLarge
    - #22265: ModSqrt
    - #20490: Add and Sub
    - #11284: GCD

This CL also fixes two new minor bugs that the test found. A wrong
result would be returned by

    - Exp when the modulo and the receiver alias
    - Rand when the limit is negative and it aliases the receiver

The test runs in ~0.05s with the default -quickchecks value.

Change-Id: I8354069ec9886e40c60f2642342ee08e604befb7
Reviewed-on: https://go-review.googlesource.com/c/go/+/168257
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
2022-04-27 17:24:48 +00:00
Filippo Valsorda
f0ee7fda63 crypto/tls: remove tls10default GODEBUG flag
Updates #45428

Change-Id: Ic2ff459e6a3f1e8ded2a770c11d34067c0b39a8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/400974
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Auto-Submit: Filippo Valsorda <valsorda@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Run-TryBot: Filippo Valsorda <valsorda@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-04-27 17:20:34 +00:00
Filippo Valsorda
0b5218cf4e crypto/elliptic: split up P-256 field and group ops
This makes Gerrit recognize the rename of the field implementation and
facilitates the review. No code changes.

For #52182

Change-Id: I827004e175db1ae2fcdf17d0f586ff21503d27e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/390754
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-27 15:22:15 +00:00
Filippo Valsorda
f0c0e0f255 crypto/elliptic: inline marshaling into nistec pointFromAffine
Marshal behavior for invalid points is undefined, so don't use it to
check if points are valid.

For #52182

Change-Id: If167893bc4b029f71bb2528564f2bd96bee7221c
Reviewed-on: https://go-review.googlesource.com/c/go/+/382994
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2022-04-27 14:23:28 +00:00
Filippo Valsorda
83ff0b0c61 crypto/elliptic: use generics for nistec-based curves
There was no way to use an interface because the methods on the Point
types return concrete Point values, as they should.

A couple somewhat minor annoyances:

    - Allocations went up due to #48849. This is fine here, where
      math/big causes allocations anyway, but would probably not be fine
      in nistec itself.

    - Carrying the newPoint/newGenerator functions around as a field is
      a little weird, even if type-safe. It also means we have to make
      what were functions methods so they can access newPoint to return
      the zero value. This is #35966.

For #52182

Change-Id: I050f3a27f15d3f189818da80da9de0cba0548931
Reviewed-on: https://go-review.googlesource.com/c/go/+/360015
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2022-04-27 14:22:53 +00:00
Filippo Valsorda
6796a7924c crypto/elliptic: refactor package structure
Not quite golang.org/wiki/TargetSpecific compliant, but almost.

The only substantial code change is in randFieldElement: it used to use
Params().BitSize instead of Params().N.BitLen(), which is semantically
incorrect, even if the two values are the same for all named curves.

For #52182

Change-Id: Ibc47450552afe23ea74fcf55d1d799d5d7e5487c
Reviewed-on: https://go-review.googlesource.com/c/go/+/315273
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2022-04-27 14:21:49 +00:00
cuiweixie
24b570354c time: document hhmmss formats
Fixes #52516

Change-Id: I173fdb09c245563e09be4e1aacfd374c3a764d74
GitHub-Last-Rev: 14a81e5061
GitHub-Pull-Request: golang/go#52538
Reviewed-on: https://go-review.googlesource.com/c/go/+/402058
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Run-TryBot: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-27 00:11:22 +00:00
Keith Randall
68b655f2b9 runtime: disable windowed Smhasher test on 32-bit systems
This test tends to be flaky on 32-bit systems.
There's not enough bits in the hash output, so we
expect a nontrivial number of collisions, and it is
often quite a bit higher than expected.

Fixes #43130

Change-Id: If35413b7c45eed778a08b834dacf98009ceca840
Reviewed-on: https://go-review.googlesource.com/c/go/+/402456
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gopher Robot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Keith Randall <khr@google.com>
2022-04-27 00:09:45 +00:00
Michael Anthony Knyszek
d29f5247b8 runtime: refactor the scavenger and make it testable
This change refactors the scavenger into a type whose methods represent
the actual function and scheduling of the scavenger. It also stubs out
access to global state in order to make it testable.

This change thus also adds a test for the scavenger. In writing this
test, I discovered the lack of a behavior I expected: if the
pageAlloc.scavenge returns < the bytes requested scavenged, that means
the heap is exhausted. This has been true this whole time, but was not
documented or explicitly relied upon. This change rectifies that. In
theory this means the scavenger could spin in run() indefinitely (as
happened in the test) if shouldStop never told it to stop. In practice,
shouldStop fires long before the heap is exhausted, but for future
changes it may be important. At the very least it's good to be
intentional about these things.

While we're here, I also moved the call to stopTimer out of wake and
into sleep. There's no reason to add more operations to a context that's
already precarious (running without a P on sysmon).

Change-Id: Ib31b86379fd9df84f25ae282734437afc540da5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/384734
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26 22:15:21 +00:00
Michael Anthony Knyszek
d8cf2243e0 runtime: disable idle mark workers with at least one dedicated worker
This change completes the proposal laid out in #44163. With #44313
resolved, we now ensure that stopped Ms are able to wake up and become
dedicated GC workers. As a result, idle GC workers are in theory no
longer required to be a proxy for scheduling dedicated mark workers.

And, with at least one dedicated mark worker running (which is
non-preemptible) we ensure the GC makes progress in all circumstances
when at least one is running. Currently we ensure at least one idle mark
worker is available at all times because it's possible before #44313
that a dedicated worker doesn't ever get scheduled, leading to a
deadlock if user goroutines block on a GC completing. But now that extra
idle mark worker should be unnecessary to ensure GC progress when at
least one dedicated mark worker is going to be scheduled.

Fixes #44163.

Change-Id: I62889ef2db4e69d44da883e8e6eebcfe5398c86d
Reviewed-on: https://go-review.googlesource.com/c/go/+/395634
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26 22:09:56 +00:00
Michael Anthony Knyszek
13b18ec947 runtime: move scheduling decisions by schedule into findrunnable
This change moves several scheduling decisions made by schedule into
findrunnable. The main motivation behind this change is the fact that
stopped Ms can't become dedicated or fractional GC workers. The main
reason for this is that when a stopped M wakes up, it stays in
findrunnable until it finds work, which means it will never consider GC
work. On that note, it'll also never consider becoming the trace reader,
either.

Another way of looking at it is that this change tries to make
findrunnable aware of more sources of work than it was before. With this
change, any M in findrunnable should be capable of becoming a GC worker,
resolving #44313. While we're here, let's also make more sources of
work, such as the trace reader, visible to handoffp, which should really
be checking all sources of work. With that, we also now correctly handle
the case where StopTrace is called from the last live M that is also
locked (#39004). stoplockedm calls handoffp to start a new M and handle
the work it cannot, and once we include the trace reader in that, we
ensure that the trace reader gets scheduled.

This change attempts to preserve the exact same ordering of work
checking to reduce its impact.

One consequence of this change is that upon entering schedule, some
sources of work won't be checked twice (i.e. the local and global
runqs, and timers) as they do now, which in some sense gives them a
lower priority than they had before.

Fixes #39004.
Fixes #44313.

Change-Id: I5d8b7f63839db8d9a3e47cdda604baac1fe615ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/393880
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26 22:09:34 +00:00
Michael Anthony Knyszek
e1b5f347e7 runtime: reduce max idle mark workers during periodic GC cycles
This change reduces the maximum number of idle mark workers during
periodic (currently every 2 minutes) GC cycles to 1.

Idle mark workers soak up all available and unused Ps, up to GOMAXPROCS.
While this provides some throughput and latency benefit in general, it
can cause what appear to be massive CPU utilization spikes in otherwise
idle applications. This is mostly an issue for *very* idle applications,
ones idle enough to trigger periodic GC cycles. This spike also tends to
interact poorly with auto-scaling systems, as the system might assume
the load average is very low and suddenly see a massive burst in
activity.

The result of this change is not to bring down this 100% (of GOMAXPROCS)
CPU utilization spike to 0%, but rather

  min(25% + 1/GOMAXPROCS*100%, 100%)

Idle mark workers also do incur a small latency penalty as they must be
descheduled for other work that might pop up. Luckily the runtime is
pretty good about getting idle mark workers off of Ps, so in general
the latency benefit from shorter GC cycles outweighs this cost. But, the
cost is still non-zero and may be more significant in idle applications
that aren't invoking assists and write barriers quite as often.

We can't completely eliminate idle mark workers because they're
currently necessary for GC progress in some circumstances. Namely,
they're critical for progress when all we have is fractional workers. If
a fractional worker meets its quota, and all user goroutines are blocked
directly or indirectly on a GC cycle (via runtime.GOMAXPROCS, or
runtime.GC), the program may deadlock without GC workers, since the
fractional worker will go to sleep with nothing to wake it.

Fixes #37116.
For #44163.

Change-Id: Ib74793bb6b88d1765c52d445831310b0d11ef423
Reviewed-on: https://go-review.googlesource.com/c/go/+/393394
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26 22:08:42 +00:00