1
0
mirror of https://github.com/golang/go synced 2024-11-27 02:11:17 -07:00
Commit Graph

47073 Commits

Author SHA1 Message Date
Josh Bleecher Snyder
43d5f213e2 cmd/compile: optimize multi-register shifts on amd64
amd64 can shift in bits from another register instead of filling with 0/1.
This pattern is helpful when implementing 128 bit shifts or arbitrary length shifts.
In the standard library, it shows up in pure Go math/big.

Benchmarks results on amd64 with -tags=math_big_pure_go.

name                          old time/op  new time/op  delta
NonZeroShifts/1/shrVU-8       4.45ns ± 3%  4.39ns ± 1%   -1.28%  (p=0.000 n=30+27)
NonZeroShifts/1/shlVU-8       4.13ns ± 4%  4.10ns ± 2%     ~     (p=0.254 n=29+28)
NonZeroShifts/2/shrVU-8       5.55ns ± 1%  5.63ns ± 2%   +1.42%  (p=0.000 n=28+29)
NonZeroShifts/2/shlVU-8       5.70ns ± 2%  5.14ns ± 1%   -9.82%  (p=0.000 n=29+28)
NonZeroShifts/3/shrVU-8       6.79ns ± 2%  6.35ns ± 2%   -6.46%  (p=0.000 n=28+29)
NonZeroShifts/3/shlVU-8       6.69ns ± 1%  6.25ns ± 1%   -6.60%  (p=0.000 n=28+27)
NonZeroShifts/4/shrVU-8       7.79ns ± 2%  7.06ns ± 2%   -9.48%  (p=0.000 n=30+30)
NonZeroShifts/4/shlVU-8       7.82ns ± 1%  7.24ns ± 1%   -7.37%  (p=0.000 n=28+29)
NonZeroShifts/5/shrVU-8       8.90ns ± 3%  7.93ns ± 1%  -10.84%  (p=0.000 n=29+26)
NonZeroShifts/5/shlVU-8       8.68ns ± 1%  7.92ns ± 1%   -8.76%  (p=0.000 n=29+29)
NonZeroShifts/10/shrVU-8      14.4ns ± 1%  12.3ns ± 2%  -14.79%  (p=0.000 n=28+29)
NonZeroShifts/10/shlVU-8      14.1ns ± 1%  11.9ns ± 2%  -15.55%  (p=0.000 n=28+27)
NonZeroShifts/100/shrVU-8      118ns ± 1%    96ns ± 3%  -18.82%  (p=0.000 n=30+29)
NonZeroShifts/100/shlVU-8      120ns ± 2%    98ns ± 2%  -18.46%  (p=0.000 n=29+28)
NonZeroShifts/1000/shrVU-8    1.10µs ± 1%  0.88µs ± 2%  -19.63%  (p=0.000 n=29+30)
NonZeroShifts/1000/shlVU-8    1.10µs ± 2%  0.88µs ± 2%  -20.28%  (p=0.000 n=29+28)
NonZeroShifts/10000/shrVU-8   10.9µs ± 1%   8.7µs ± 1%  -19.78%  (p=0.000 n=28+27)
NonZeroShifts/10000/shlVU-8   10.9µs ± 2%   8.7µs ± 1%  -19.64%  (p=0.000 n=29+27)
NonZeroShifts/100000/shrVU-8   111µs ± 2%    90µs ± 2%  -19.39%  (p=0.000 n=28+29)
NonZeroShifts/100000/shlVU-8   113µs ± 2%    90µs ± 2%  -20.43%  (p=0.000 n=30+27)

The assembly version is still faster, unfortunately, but the gap is narrowing.
Speedup from pure Go to assembly:

name                          old time/op  new time/op  delta
NonZeroShifts/1/shrVU-8       4.39ns ± 1%  3.45ns ± 2%  -21.36%  (p=0.000 n=27+29)
NonZeroShifts/1/shlVU-8       4.10ns ± 2%  3.47ns ± 3%  -15.42%  (p=0.000 n=28+30)
NonZeroShifts/2/shrVU-8       5.63ns ± 2%  3.97ns ± 0%  -29.40%  (p=0.000 n=29+25)
NonZeroShifts/2/shlVU-8       5.14ns ± 1%  3.77ns ± 2%  -26.65%  (p=0.000 n=28+26)
NonZeroShifts/3/shrVU-8       6.35ns ± 2%  4.79ns ± 2%  -24.52%  (p=0.000 n=29+29)
NonZeroShifts/3/shlVU-8       6.25ns ± 1%  4.42ns ± 1%  -29.29%  (p=0.000 n=27+26)
NonZeroShifts/4/shrVU-8       7.06ns ± 2%  5.64ns ± 1%  -20.05%  (p=0.000 n=30+29)
NonZeroShifts/4/shlVU-8       7.24ns ± 1%  5.34ns ± 2%  -26.23%  (p=0.000 n=29+29)
NonZeroShifts/5/shrVU-8       7.93ns ± 1%  6.56ns ± 2%  -17.26%  (p=0.000 n=26+30)
NonZeroShifts/5/shlVU-8       7.92ns ± 1%  6.27ns ± 1%  -20.79%  (p=0.000 n=29+25)
NonZeroShifts/10/shrVU-8      12.3ns ± 2%  10.2ns ± 2%  -17.21%  (p=0.000 n=29+29)
NonZeroShifts/10/shlVU-8      11.9ns ± 2%  10.5ns ± 2%  -12.45%  (p=0.000 n=27+29)
NonZeroShifts/100/shrVU-8     95.9ns ± 3%  77.7ns ± 1%  -19.00%  (p=0.000 n=29+30)
NonZeroShifts/100/shlVU-8     97.5ns ± 2%  66.8ns ± 2%  -31.47%  (p=0.000 n=28+30)
NonZeroShifts/1000/shrVU-8     884ns ± 2%   705ns ± 1%  -20.17%  (p=0.000 n=30+28)
NonZeroShifts/1000/shlVU-8     880ns ± 2%   590ns ± 1%  -32.96%  (p=0.000 n=28+25)
NonZeroShifts/10000/shrVU-8   8.74µs ± 1%  7.34µs ± 3%  -15.94%  (p=0.000 n=27+30)
NonZeroShifts/10000/shlVU-8   8.73µs ± 1%  6.00µs ± 1%  -31.25%  (p=0.000 n=27+28)
NonZeroShifts/100000/shrVU-8  89.6µs ± 2%  75.5µs ± 2%  -15.80%  (p=0.000 n=29+29)
NonZeroShifts/100000/shlVU-8  89.6µs ± 2%  68.0µs ± 3%  -24.09%  (p=0.000 n=27+30)

Change-Id: I18f58d8f5513d737d9cdf09b8f9d14011ffe3958
Reviewed-on: https://go-review.googlesource.com/c/go/+/297050
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2021-03-11 19:11:46 +00:00
Josh Bleecher Snyder
b0733ba12d hash/maphash: increase the buffer size
This helps a lot for larger writes.

name            old time/op    new time/op    delta
Hash8Bytes-8      16.5ns ± 4%    16.8ns ± 2%   +1.76%  (p=0.000 n=29+30)
Hash320Bytes-8    55.0ns ± 2%    41.4ns ± 2%  -24.64%  (p=0.000 n=28+28)
Hash1K-8           190ns ± 2%     130ns ± 3%  -31.65%  (p=0.000 n=30+30)
Hash8K-8          1.57µs ± 2%    1.05µs ± 3%  -33.01%  (p=0.000 n=30+29)

name            old speed      new speed      delta
Hash8Bytes-8     485MB/s ± 4%   476MB/s ± 2%   -1.73%  (p=0.000 n=29+30)
Hash320Bytes-8  5.82GB/s ± 2%  7.72GB/s ± 3%  +32.55%  (p=0.000 n=28+29)
Hash1K-8        5.39GB/s ± 2%  7.88GB/s ± 3%  +46.32%  (p=0.000 n=30+30)
Hash8K-8        5.21GB/s ± 2%  7.77GB/s ± 3%  +49.28%  (p=0.000 n=30+29)

Updates #42710

Change-Id: Idaf4b2a8a41fc62fc16b54c9358cf2cc7009cf29
Reviewed-on: https://go-review.googlesource.com/c/go/+/278760
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2021-03-11 19:04:42 +00:00
Josh Bleecher Snyder
64d323f45a hash/maphash: optimize Write and WriteString
The existing code makes copies of every byte it hashes.
When passed a large chunk of memory, Write and WriteString
can skip the copying and initSeed for most of it.

To ensure that Write, WriteByte, and WriteString continue to
generate output that depends only on the sequence of bytes,
expand the grouping test to include WriteString and interleaved calls.
Also, make the test process a lot more data, to ensure that
Write* handled full buffers correctly.

name            old time/op    new time/op    delta
Hash8Bytes-8      17.1ns ± 3%    16.5ns ± 2%   -3.26%  (p=0.000 n=29+27)
Hash320Bytes-8    74.9ns ± 2%    58.5ns ± 2%  -21.86%  (p=0.000 n=30+29)
Hash1K-8           246ns ± 3%     195ns ± 1%  -20.82%  (p=0.000 n=29+30)
Hash8K-8          1.87µs ± 2%    1.59µs ± 2%  -15.04%  (p=0.000 n=26+30)

name            old speed      new speed      delta
Hash8Bytes-8     468MB/s ± 3%   484MB/s ± 2%   +3.36%  (p=0.000 n=29+27)
Hash320Bytes-8  4.28GB/s ± 2%  5.47GB/s ± 2%  +27.97%  (p=0.000 n=30+29)
Hash1K-8        4.17GB/s ± 3%  5.26GB/s ± 1%  +26.28%  (p=0.000 n=29+30)
Hash8K-8        4.38GB/s ± 2%  5.16GB/s ± 2%  +17.70%  (p=0.000 n=26+30)

Updates #42710

Change-Id: If3cdec1580ffb3e36fab9865e5a9d089c0a34bec
Reviewed-on: https://go-review.googlesource.com/c/go/+/278758
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2021-03-11 19:00:06 +00:00
Josh Bleecher Snyder
ae9cd1299c hash/maphash: manually inline setSeed
Provides a minor performance win.

name            old time/op    new time/op    delta
Hash8Bytes-8      16.5ns ± 2%    16.5ns ± 4%    ~     (p=0.407 n=27+29)
Hash320Bytes-8    58.5ns ± 2%    55.0ns ± 2%  -6.01%  (p=0.000 n=29+28)
Hash1K-8           195ns ± 1%     190ns ± 2%  -2.23%  (p=0.000 n=30+30)
Hash8K-8          1.59µs ± 2%    1.57µs ± 2%  -0.88%  (p=0.002 n=30+30)

name            old speed      new speed      delta
Hash8Bytes-8     484MB/s ± 2%   485MB/s ± 4%    ~     (p=0.417 n=27+29)
Hash320Bytes-8  5.47GB/s ± 2%  5.82GB/s ± 2%  +6.39%  (p=0.000 n=29+28)
Hash1K-8        5.26GB/s ± 1%  5.39GB/s ± 2%  +2.29%  (p=0.000 n=30+30)
Hash8K-8        5.16GB/s ± 2%  5.21GB/s ± 2%  +0.89%  (p=0.002 n=30+30)

Updates #42710

Change-Id: Ia0d7264b648f96099202de21c6b69a9c1776f6c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/278759
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-11 18:50:12 +00:00
Josh Bleecher Snyder
86d6678429 testing/fstest: clarify TestFS docs
The sentence starts "fsys must only contain",
which leads the reader to believe that fsys must not contain others.
The rapid reversal leads to confusion.
I had to read it several times to be sure I'd parsed it correctly.

Remove "only"; rely on the rest of the sentence to clarify.

Change-Id: I9fb7935aed4f9839344d3a00b761d20981fba864
Reviewed-on: https://go-review.googlesource.com/c/go/+/296529
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-11 18:25:50 +00:00
Josh Bleecher Snyder
1853411d83 testing/fstest: test that ReadDirFile on a non-dir fails
ReadDirFile implementations should return an error for non-directories.

Change-Id: I99888562cb6cf829017904ae8c1e8887a416c4cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/296391
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-11 18:25:41 +00:00
Josh Bleecher Snyder
bbf79793bd io/fs: clarify additional File interface docs
Emphasize ReadDirFile. It isn't really optional,
and all filesystems have at least one directory (".").

The remaining two additional interfaces are optimizations.
Call them that.

Fully qualify package package io identifiers.

Change-Id: Ibc425a5dfd27e08c2c10c353f780e4a6304cfd87
Reviewed-on: https://go-review.googlesource.com/c/go/+/296390
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-11 18:25:30 +00:00
Paul E. Murphy
0a655598e1 cmd/link: fix glink resolver generation on ppc64le
AddSymRef grows the section to fit the relocation. This was not
being accounted for. Instead, add a relocation and explicitly
populate it so we patch the desired instructions.

Change-Id: I583147e2545aea34c854f9d35ca920c57be60b90
Reviewed-on: https://go-review.googlesource.com/c/go/+/300949
Run-TryBot: Paul Murphy <murp@ibm.com>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2021-03-11 17:36:20 +00:00
Michael Anthony Knyszek
f009b5b226 runtime: support register ABI for finalizers
This change modifies runfinq to properly pass arguments to finalizers in
registers via reflectcall.

For #40724.

Change-Id: I414c0eff466ef315a0eb10507994e598dd29ccb2
Reviewed-on: https://go-review.googlesource.com/c/go/+/300112
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-11 17:26:22 +00:00
Ian Lance Taylor
415ca3f1f0 test: add test that caused a gofrontend internal error
For #44383

Change-Id: I3610105dad3574e210e226d3ba80a4ba5a7eeaa6
Reviewed-on: https://go-review.googlesource.com/c/go/+/300789
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-11 17:20:58 +00:00
Matt T. Proud
0fc370c5d2 docs: clarify when APIs use context.Background.
The Go standard library retrofitted context support onto existing APIs
using context.Background and later offered variants that directly
supported user-defined context value specification. This commit makes
that behavior clear in documentation and suggests context-aware
alternatives if the user is looking for one.

An example motivation is supporting code for use in systems that expect
APIs to be cancelable for lifecycle correctness or load
shedding/management reasons, as alluded to in
https://blog.golang.org/context-and-structs.

Updates #44143

Change-Id: I2d7f954ddf9b48264d5ebc8d0007058ff9bddf14
Reviewed-on: https://go-review.googlesource.com/c/go/+/296152
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Jean de Klerk <deklerk@google.com>
Trust: Jean de Klerk <deklerk@google.com>
Run-TryBot: Jean de Klerk <deklerk@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-03-11 16:37:38 +00:00
Jason A. Donenfeld
b8e9ec856c syscall: use runtime.KeepAlive for ProcThreadAttributeList arguments
It turns out that if you write Go pointers to Go memory, the Go compiler
must be involved so that it generates various calls to the GC in the
process. Letting Windows write Go pointers to Go memory violated this.
So, we replace that with just a boring call to runtime.KeepAlive. That's
not a great API, but this is all internal code anyway. We fix it up
more elegantly for external consumption in x/sys/windows with CL 300369.

Fixes #44900.

Change-Id: Id6599a793af9c4815f6c9387b00796923f32cb97
Reviewed-on: https://go-review.googlesource.com/c/go/+/300349
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2021-03-11 13:49:01 +00:00
Tobias Klauser
9ece63f064 crypto/rand, internal/syscall/unix: add support for getrandom syscall on solaris
The getrandom syscall is available on Solaris and Illumos, see
https://docs.oracle.com/cd/E88353_01/html/E37841/getrandom-2.html and
https://illumos.org/man/2/getrandom

Change-Id: Id1c65d6a5b2fbc80d20b43d8b32dab137ca950ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/299134
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-11 08:19:39 +00:00
Tobias Klauser
79e3ee52f4 internal/syscall/unix: unify GetRandom implementation
The implementation of GetRandom for Linux, FreeBSD and DragonflyBSD can
be shared.

Also remove GRND_INSECURE on DragonflyBSD as pointed out by Ian in the
review of CL 269999.

Change-Id: I5bf4c1bd51ddb2ad600652a57e0bc1bafa1cf40d
Reviewed-on: https://go-review.googlesource.com/c/go/+/299133
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-11 08:19:31 +00:00
fanzha02
3a3b8164fd cmd/dist: refactor test constraints for misc/cgo/testsantizers
Currently, the cmd/dist runs test cases in misc/cgo/testsantizers only
when memeory sanitizer is supported, but the tsan tests in
misc/cgo/testsanitizers do not require support for -msan option, which
makes tsan tests can not be run on some unsupported -msan option platforms.

Therefore, this patch moves the test constraints from cmd/dist to
msan_test.go, so that the tsan tests in misc/cgo/testsanitizers
can be run on any system where the C compiler supports -fsanitize=thread
option.

Change-Id: I779c92eedd0270050f1a0b1a69ecce50c3712bc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/297774
Trust: fannie zhang <Fannie.Zhang@arm.com>
Run-TryBot: fannie zhang <Fannie.Zhang@arm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-11 05:38:13 +00:00
eric fang
68f3344fe9 cmd/compile: remove 8-byte alignment requirement of stack slot on ppc64
This CL applies CL 267999 to ppc64.

Fixes #42385

Change-Id: I6462d647d1abdf7cec99607c40ef4d1fed1941e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/297770
Reviewed-by: eric fang <eric.fang@arm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: eric fang <eric.fang@arm.com>
2021-03-11 02:32:05 +00:00
Dan Scales
fdded79e6e cmd/compile: fix handling of partially inferred type arguments
In the case of partially inferred type arguments, we need to use the
IndexExpr as the key in g.info.Inferred[] rather than the CallExpr.

Added an extra fromStrings1 call in the settable.go test that tests
partially inferred type arguments. This new call uses a new concrete
type SettableString as well.

I also added another implementation fromStrings3 (derived from a go2go
tests) that typechecks but intentionally causes a panic.

Change-Id: I74d35c5a741f72f37160a96fbec939451157f392
Reviewed-on: https://go-review.googlesource.com/c/go/+/300309
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dan Scales <danscales@google.com>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-11 00:31:14 +00:00
Paul E. Murphy
1bad3831a0 cmd/internal/obj: remove param element from ppc64 optab
This is rarely used, and is implied based on the
memory type of the operand.  This is a step towards
simplifying the MOV* pseudo opcodes on ppc64.

Similarly, remove the bogus param value from AVMULESB.

Change-Id: Ibad4d045ec6d8c5163a468b2db1dfb762ef674ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/300177
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Trust: Ian Lance Taylor <iant@golang.org>
2021-03-10 21:55:05 +00:00
Michael Pratt
aa26687e45 runtime, time: disable preemption in addtimer
The timerpMask optimization updates a mask of Ps (potentially)
containing timers in pidleget / pidleput. For correctness, it depends on
the assumption that new timers can only be added to a P's own heap.

addtimer violates this assumption if it is preempted after computing pp.
That G may then run on a different P, but adding a timer to the original
P's heap.

Avoid this by disabling preemption while pp is in use.

Other uses of doaddtimer should be OK:

* moveTimers: always moves to the current P's heap
* modtimer, cleantimers, addAdjustedTimers, runtimer: does not add net
  new timers to the heap while locked

Fixes #44868

Change-Id: I4a5d080865e854931d0a3a09a51ca36879101d72
Reviewed-on: https://go-review.googlesource.com/c/go/+/300610
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-03-10 21:49:24 +00:00
Bryan C. Mills
f9ed8b3f1e cmd/go/internal/mvs: factor out an incremental implementation
The new Graph type implements an incremental version of the MVS
algorithm, with requirements pushed in by the caller instead of pulled
by an internal MVS traversal.

To avoid redundancy going forward (and to ensure adequate test
coverage of the incremental implementation), the existing buildList
function is reimplemented in terms of Graph.

For #36460

Change-Id: Idd0b6ab8f17cc41d83a2a4c25a95f82e9ce1eab0
Reviewed-on: https://go-review.googlesource.com/c/go/+/244760
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-10 21:01:25 +00:00
Bryan C. Mills
2ceb79db52 cmd/go/internal/modload: make EditBuildList report whether the build list was changed
For #36460

Change-Id: I8dd6e6f998a217a4287212815ce61209df6f007f
Reviewed-on: https://go-review.googlesource.com/c/go/+/296609
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-10 21:00:52 +00:00
Bryan C. Mills
b7f0fb6d9e cmd/go/internal/modload: fuse upgrading with downgrading in EditBuildList
Previosly, EditBuildList performed an mvs.Upgrade followed by an
mvs.Downgrade, with the Downgrade building on the result of the
Upgrade. Unfortunately, that approach potentially folds in irrelevant
dependencies from the first Upgrade, which are then preserved
unnecessarily by the Downgrade (see mod_get_downup_artifact.txt).

Now, we use the initial Upgrade only to compute the maximum allowed
versions of transitive dependencies, and apply the module upgrades and
downgrades together in a single operation.

For #36460

Change-Id: I7590c137111fed4a3b06531c88d90efd49e6943a
Reviewed-on: https://go-review.googlesource.com/c/go/+/290770
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2021-03-10 21:00:16 +00:00
Jay Conrod
a1a3d33b0d cmd/go: test remote lookup of packages with leading dots in path elements
Follow-up to CL 297530.

For #43985
For #34992

Change-Id: I2cfa6c41c013e627c3464c383ca42f5c9ebe521a
Reviewed-on: https://go-review.googlesource.com/c/go/+/297634
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-03-10 20:52:20 +00:00
Lynn Boger
ccf9acefa8 cmd/compile/internal: improve handling of DS form offsets on ppc64x
In the ppc64 ISA DS form loads and stores are restricted to offset
fields that are a multiple of 4. This is currently handled with checks
in the rules that generate MOVDload, MOVWload, MOVDstore and
MOVDstorezero to prevent invalid instructions from getting to the
assembler.

An unhandled case was discovered which led to the search for a better
solution to this problem. Now, instead of checking the offset in the
rules, this will be detected when processing these Ops in
ssaGenValues in ppc64/ssa.go. If the offset is not valid, the address
of the symbol to be loaded or stored will be computed using the base
register + offset, and that value used in the new base register.
With the full address in the base register, the offset field can be
zero in the instruction.

Updates #44739

Change-Id: I4f3c0c469ae70a63e3add295c9b55ea0e30ef9b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/299789
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-10 19:33:23 +00:00
Michael Pratt
c41bf9ee81 runtime: check partial lock ranking order
To ease readability we typically keep the partial order lists sorted by
rank. This isn't required for correctness, it just makes it (slightly)
easier to read the lists.

Currently we must notice out-of-order entries during code review, which
is an error-prone process.

Add a test to enforce ordering, and fix the errors that have crept in.
Most of the existing errors were misordered lockRankHchan or
lockRankPollDesc.

While we're here, I've moved the correctness check that the partial
ordering satisfies the total ordering from init to a test case. This
will allow us to catch these errors without even running
staticlockranking.

Change-Id: I9c11abe49ea26c556439822bb6a3183129600c3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/300171
Trust: Michael Pratt <mpratt@google.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2021-03-10 19:07:29 +00:00
KimMachineGun
5ce51ea741 flag: panic if flag name begins with - or contains =
Fixes #41792

Change-Id: I9b4aae8a899e3c3ac9532d27932d275cfb1fab48
GitHub-Last-Rev: f06b1e1767
GitHub-Pull-Request: golang/go#42737
Reviewed-on: https://go-review.googlesource.com/c/go/+/271788
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Rob Pike <r@golang.org>
2021-03-10 18:33:52 +00:00
Katie Hockman
d0b79e3513 encoding/xml: prevent infinite loop while decoding
This change properly handles a TokenReader which
returns an EOF in the middle of an open XML
element.

Thanks to Sam Whited for reporting this.

Fixes CVE-2021-27918
Fixes #44913

Change-Id: Id02a3f3def4a1b415fa2d9a8e3b373eb6cb0f433
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004594
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/300391
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-03-10 18:19:03 +00:00
Roland Shoemaker
cd3b4ca9f2 archive/zip: fix panic in Reader.Open
When operating on a Zip file that contains a file prefixed with "../",
Open(...) would cause a panic in toValidName when attempting to strip
the prefixed path components.

Fixes CVE-2021-27919
Fixes #44916

Change-Id: Ic755d8126cb0897e2cbbdacf572439c38dde7b35
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/1004761
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-by: Katie Hockman <katiehockman@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/300489
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alexander Rakoczy <alex@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2021-03-10 18:18:28 +00:00
Dan Scales
1811aeae66 cmd/compile: deal with helper generic types that add methods to T
Deal with cases like: 'type P[T any] T' (used to add methods to an
arbitrary type T), In this case, P[T] has kind types.TTYPEPARAM (as does
T itself), but requires more code to substitute than a simple TTYPEPARAM
T. See the comment near the beginning of subster.typ() in stencil.go.

Add new test absdiff.go. This test has a case for complex types (which
I've commented out) that will only work when we deal better with Go
builtins in generic functions (like real and imag).

Remove change in fmt.go for TTYPEPARAMS that is no longer needed (since
all TTYPEPARAMS have a sym) and was sometimes causing an extra prefix
when formatting method names.

Separate out the setting of a TTYPEPARAM bound, since it can reference
the TTYPEPARAM being defined, so must be done separately. Also, we don't
currently (and may not ever) need bounds after types2 typechecking.

Change-Id: Id173057e0c4563b309b95e665e9c1151ead4ba77
Reviewed-on: https://go-review.googlesource.com/c/go/+/300049
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dan Scales <danscales@google.com>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-10 17:36:55 +00:00
Rob Findley
5edab39f49 cmd/gofmt: fix const association to avoid inaccurate comment
The const parseTypeParams was grouped with printer-related consts in
gofmt.go, implicitly suggesting that it must be kept in sync with
go/format/format.go.

Change-Id: Ia65dc15c27fef2c389f963071252adee32ec6bd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/300451
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-10 17:10:00 +00:00
Rob Findley
489231111f go/types: add missing build tag to api_go1.18_test.go
This file has a go:build comment without a corresponding +build comment.

Change-Id: Id01604242a14d8ead16ffb9aa1b45eef7706956a
Reviewed-on: https://go-review.googlesource.com/c/go/+/300450
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-10 17:09:54 +00:00
Michael Pratt
7457462303 runtime/race: update dead link
LLVM changed their main branch name, so this link didn't work anymore.

Change-Id: I4c3a67b26e2bda012071281e29ea3c932c185130
Reviewed-on: https://go-review.googlesource.com/c/go/+/300469
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-03-10 17:07:49 +00:00
Cherry Zhang
818f6b14b4 cmd/compile: remove ".fp" fake arg
No longer needed with previous CLs.

Change-Id: I9a1c11092a2736c190fa8e8ddfbb913b708957eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/300155
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2021-03-10 15:24:31 +00:00
Cherry Zhang
bc489dd6d5 runtime: update signature of reflectcall functions
reflectcall tail calls runtime.call{16,32,...} functions, so they
have the same signature as reflectcall. It is important for them
to have the correct arg map, because those functions, as well as
the function being reflectcall'd, could move the stack. When that
happens, its pointer arguments, in particular regArgs, need to be
adjusted. Otherwise it will still point to the old stack, causing
memory corruption.

This only caused failures on the regabi builder because it is the
only place where internal/abi.RegArgs is not a zero-sized type.

May fix #44821.

Change-Id: Iab400ea6b60c52360d0b43a793f6bfe50ca9989b
Reviewed-on: https://go-review.googlesource.com/c/go/+/300154
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-03-10 15:23:39 +00:00
Bryan C. Mills
30c28bbf05 cmd/go: avoid password prompts in TestScript/mod_get_private_vcs
In some cases, this test would prompt for interactive SSH passwords in
order to authenticate to github.com over SSH. Setting GIT_SSH_COMMAND
to /bin/false prevents that, while still provoking the desired Git
failure mode.

Updates #44904.

Change-Id: Idc9fe9f47d2ccb6c8a4ea988b73d9c8c774e4079
Reviewed-on: https://go-review.googlesource.com/c/go/+/300156
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Kevin Burke <kev@inburke.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Kevin Burke <kev@inburke.com>
2021-03-10 14:45:43 +00:00
David Carlier
cf59850466 crypto/rand: supports for getrandom syscall in DragonFlyBSD
Since the 5.7 release, DragonFlyBSD supports as well
the getrandom function, the actual stable is 5.8.

Change-Id: I2b8fc468771b10ac12b38ea7e8e5314342de6375
GitHub-Last-Rev: c5c496f418
GitHub-Pull-Request: golang/go#42617
Reviewed-on: https://go-review.googlesource.com/c/go/+/269999
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2021-03-10 09:01:05 +00:00
fanzha02
4d608eb224 testing: fix typo in a comment
Change-Id: I781808327be84113cd55c52bc214b821cd166114
Reviewed-on: https://go-review.googlesource.com/c/go/+/300269
Trust: fannie zhang <Fannie.Zhang@arm.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-03-10 04:06:13 +00:00
Andy Pan
643d240a11 internal/poll: implement a pipe pool for splice() call
In scenarios where splice() is called, splice() is usually called not just once, but many times,
which means that a lot of pipes will be created and destroyed frequently, costing an amount of system resources
and slowing down performance, thus I suggest that we add a pipe pool for reusing pipes.

Benchmark tests:

goos: linux
goarch: amd64
pkg: internal/poll
cpu: AMD EPYC 7K62 48-Core Processor

name                  old time/op    new time/op    delta
SplicePipe-8            1.36µs ± 1%    0.02µs ± 0%   -98.57%  (p=0.001 n=7+7)
SplicePipeParallel-8     747ns ± 4%       4ns ± 0%   -99.41%  (p=0.001 n=7+7)

name                  old alloc/op   new alloc/op   delta
SplicePipe-8             24.0B ± 0%      0.0B       -100.00%  (p=0.001 n=7+7)
SplicePipeParallel-8     24.0B ± 0%      0.0B       -100.00%  (p=0.001 n=7+7)

name                  old allocs/op  new allocs/op  delta
SplicePipe-8              1.00 ± 0%      0.00       -100.00%  (p=0.001 n=7+7)
SplicePipeParallel-8      1.00 ± 0%      0.00       -100.00%  (p=0.001 n=7+7)

Fixes #42740

Change-Id: Idff654b7264342084e089b5ba796c87c380c471b
Reviewed-on: https://go-review.googlesource.com/c/go/+/271537
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
2021-03-10 03:52:48 +00:00
Bryan C. Mills
d33e2192a7 cmd/go: allow '+' in package import paths in module mode
This change upgrades x/mod to pull in the fix from CL 300149.

Fixes #44776.

Change-Id: I273f41df2abfff76d91315b7f19fce851c8770d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/300176
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-03-10 03:08:37 +00:00
Robert Griesemer
41245ab283 cmd/compile/internal/types2: remove concept of finals
This is a 1:1 port of the respective change in go/types
in https://golang.org/cl/299590.

Change-Id: I65ad723f2e21e3d95fc0b94665e0121e31871a48
Reviewed-on: https://go-review.googlesource.com/c/go/+/300250
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-03-10 01:46:25 +00:00
Robert Griesemer
acd7cb5887 cmd/compile/internal/types2: better error reporting framework (starting point)
Until now, errors which came with additional details (e.g., a declaration
cycle error followed by the list of objects involved in the cycle, one per
line) were reported as an ordinary error followed by "secondary" errors,
with the secondary errors marked as such by having a tab-indented error
message.

This approach often required clients to filter these secondary errors
(as they are not new errors, they are just clarifying a previously
reported error).

This CL introduces a new internal error_ type which permits accumulating
various error information that may then be reported as a single error.

Change-Id: I25b2f094facd37e12737e517f7ef8853d465ff77
Reviewed-on: https://go-review.googlesource.com/c/go/+/296689
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-03-10 00:50:17 +00:00
Rob Findley
142a76530c go/types: improve the positioning of broken import errors
The heuristic gopls uses to guess error spans can get tripped-up on
certain valid characters in an import path (for example '-').

Update the error for broken imports to capture the full import path
span, so that gopls doesn't need to rely on heuristics.

Change-Id: Ieb8e0dce11933643f701b32271ff5f3477fecaaa
Reviewed-on: https://go-review.googlesource.com/c/go/+/300169
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-03-09 22:00:50 +00:00
Paul E. Murphy
48ddf70128 cmd/asm,cmd/compile: support 5 operand RLWNM/RLWMI on ppc64
These instructions are actually 5 argument opcodes as specified
by the ISA.  Prior to this patch, the MB and ME arguments were
merged into a single bitmask operand to workaround the limitations
of the ppc64 assembler backend.

This limitation no longer exists. Thus, we can pass operands for
these opcodes without having to merge the MB and ME arguments in
the assembler frontend or compiler backend.

Likewise, support for 4 operand variants is unchanged.

Change-Id: Ib086774f3581edeaadfd2190d652aaaa8a90daeb
Reviewed-on: https://go-review.googlesource.com/c/go/+/298750
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Carlos Eduardo Seo <carlos.seo@linaro.org>
Trust: Carlos Eduardo Seo <carlos.seo@linaro.org>
2021-03-09 20:35:41 +00:00
Brad Fitzpatrick
18510ae88f runtime, cmd/link/internal/ld: disable memory profiling when data unreachable
If runtime.MemProfile is unreachable, default to not collecting any
memory profiling samples, to save memory on the hash table.

Fixes #42347

Change-Id: I9a4894a5fc77035fe59b1842e1ec77a1182e70c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/299671
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Keith Randall <khr@golang.org>
2021-03-09 20:09:52 +00:00
Fazlul Shahriar
e4f3cfadf6 net: don't append a dot to TXT records on Plan 9
TXT records are not domain names, so no need to call absDomainName.
The output now matches the pure Go resolver.

Fixes #44619

Change-Id: I1ebf09152ff5c0446d2e2b4c26671358892d9dc9
Reviewed-on: https://go-review.googlesource.com/c/go/+/296589
Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: David du Colombier <0intro@gmail.com>
Trust: Ian Lance Taylor <iant@golang.org>
2021-03-09 19:25:34 +00:00
David Chase
382851c1fd cmd/compile: fix failure to communicate between ABIinfo producer&consumer
ABI info producer and consumer had different ideas for register
order for parameters.

Includes a test, includes improvements to debugging output.

Updates #44816.

Change-Id: I4812976f7a6c08d6fc02aac1ec0544b1f141cca6
Reviewed-on: https://go-review.googlesource.com/c/go/+/299570
Trust: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2021-03-09 18:45:05 +00:00
David Chase
9f5298ca6e cmd/compile: fix confusion in generating SelectN index
Old: return the ABI register index of the result (wrong!)
New: return the index w/in sequence of result registers (right!)

Fixed bug:
genCaller0/genCaller0.go:43:9: internal compiler error: 'Caller0':
   panic during schedule while compiling Caller0:
runtime error: index out of range [10] with length 9

Updates #44816.

Change-Id: I1111e283658a2d6422986ae3d61bd95d1b9bde5e
Reviewed-on: https://go-review.googlesource.com/c/go/+/299549
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-09 18:44:55 +00:00
David Chase
98dfdc82c8 cmd/compile: fix broken type+offset calc for register args
Includes more enhancements to debugging output.

Updates #44816.

Change-Id: I5b21815cf37ed21e7dec6c06f538090f32260203
Reviewed-on: https://go-review.googlesource.com/c/go/+/299409
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-09 18:44:23 +00:00
David Chase
5eb9912084 cmd/compile: fix OpArg decomposer for registers in expandCalls
Includes test taken from
https://github.com/golang/go/issues/44816#issuecomment-791618179
and improved debugging output.

Updates #44816

Change-Id: I94aeb9c5255f175fe80727be29d218bad54bf7ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/299389
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2021-03-09 18:43:58 +00:00
Keith Randall
48895d021b cmd/compile: remove skipping of implicit operations during export
We'll need to attach types to these operations, so we need to
represent them in the import/export data.

Some of the operations use a selector indicating a different package,
so we need to provide an option to encode the package of a selector.
The default selector() function can't encode that extra information,
as selector's exact encoding is used by go/types.

Change-Id: I4c110fe347b3d915f88a722834bc4058baea7854
Reviewed-on: https://go-review.googlesource.com/c/go/+/299771
Trust: Keith Randall <khr@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2021-03-09 18:35:29 +00:00