1
0
mirror of https://github.com/golang/go synced 2024-11-23 16:00:06 -07:00
Commit Graph

43015 Commits

Author SHA1 Message Date
Dan Scales
5113f57766 cmd/go: define a build tag for any GOEXPERIMENT which is enabled.
For each experiment that has been enabled in the toolchain, define a build tag
with the same name (but prefixed by "goexperiment.") which can be used for
compiling alternative files for the experiment. This allows changes for the
experiment, like extra struct fields in the runtime, without affecting the base
non-experiment code at all.

I use this capability in my CL for static lock ranking
(https://go-review.googlesource.com/c/go/+/207619), so that static lock ranking
can be fully enabled as a GOEXPERIMENT, but there is no overhead in the runtime
when the experiment is not enabled.

I added a test in cmd/go/testdata/scripts to make sure the build tags are being
defined properly. In order to implement the test, I needed to provide environment
variable GOEXPSTRING to the test scripts (with its value set from
objabi.Expstring(), so that it can determine the experiments baked into the
toolchain.

I filed https://github.com/golang/go/issues/37937 to make a builder with
GOEXPERIMENT set to 'staticlockranking'. This builder will ensure another variant
of GOEXPERIMENT is being tested regularly for this change, as well as checking
static lock ranking in the runtime.

Change-Id: Ieb4b86107238febd105558c1e639d30cfe57ab5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/222925
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-03-23 19:24:26 +00:00
Andy Pan
6aded2524c strings: narrow the search range of IndexByte in Index
Same as https://golang.org/cl/224589.

Change-Id: I6a9e2ea60bf6e1888a95bad0331c92079a7eff99
GitHub-Last-Rev: 81c13c0f5b
GitHub-Pull-Request: golang/go#38016
Reviewed-on: https://go-review.googlesource.com/c/go/+/224593
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-03-23 19:14:29 +00:00
Keith Randall
3adbdb6d99 cmd/compile: disable mem+op operations on 386
Rolling back portions of CL 222782 to see if that helps
issue #37881 any.

Update #37881

Change-Id: I9cc3ff8c469fa5e4b22daec715d04148033f46f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/224837
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-03-23 18:27:37 +00:00
Filippo Valsorda
5c9bd499e1 crypto/rsa,crypto/ecdsa,crypto/ed25519: implement PublicKey.Equal
This makes all modern public keys in the standard library implement a
common interface (below) that can be used by applications for better
type safety and allows for checking that public (and private keys via
Public()) are equivalent.

interface {
    Equal(crypto.PublicKey) bool
}

Equality for ECDSA keys is complicated, we take a strict interpretation
that works for all secure applications (the ones not using the
unfortunate non-constant time CurveParams implementation) and fails
closed otherwise.

Tests in separate files to make them x_tests and avoid an import loop
with crypto/x509.

Fixes #21704

Change-Id: Id5379c96384a11c5afde0614955360e7470bb1c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/223754
Reviewed-by: Katie Hockman <katie@golang.org>
2020-03-23 17:56:24 +00:00
Giovanni Bajo
24925c7ed9 cmd/compile: fold LEAQ with constant scale into LEA
Discovered this after rebasing CL196679 (use poset bounds in prove).
Some tests fail with that CL applied:

codegen/smallintiface.go:11: linux/amd64/: opcode not found: "^LEAQ\\truntime.staticuint64s\\+8\\(SB\\)"
codegen/smallintiface.go:16: linux/amd64/: opcode not found: "^LEAQ\\truntime.staticuint64s\\+2024\\(SB\\)"
codegen/smallintiface.go:21: linux/amd64/: opcode not found: "^LEAQ\\truntime.staticuint64s\\+24\\(SB\\)"

The only difference in prove SSA dumps is that a single Lsh64x64
op with constant shift (<< 3) is marked as bounded. This triggers
a different rule matching sequence in lower, which manages to generate
worse code for the above testcases.

This CL fixes the above test after CL196679 is applied. Right now,
these rules never trigger (this CL passes toolstash -cmp), so I can't
write a test.

Change-Id: I353f1c79c1875cac1da82cd8afa1e05e42684f1c
Reviewed-on: https://go-review.googlesource.com/c/go/+/224877
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-03-23 17:47:29 +00:00
Roland Shoemaker
5d47f870a6 crypto/x509: add RevocationList and CreateRevocationList
The existing Certificate.CreateCRL method generates non-conformant CRLs and
as such cannot be used for implementations that require standards
compliance. This change implements a new top level method, CreateCRL, which
generates compliant CRLs, and offers an extensible API if any
extensions/fields need to be supported in the future.

Here is an example Issuer/CRL generated using this change:
-----BEGIN CERTIFICATE-----
MIIBNjCB3aADAgECAgEWMAoGCCqGSM49BAMCMBIxEDAOBgNVBAMTB3Rlc3Rpbmcw
IhgPMDAwMTAxMDEwMDAwMDBaGA8wMDAxMDEwMTAwMDAwMFowEjEQMA4GA1UEAxMH
dGVzdGluZzBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABLHrudbSM36sn1VBrmm/
OfQTyEsI4tIUV1VmneOKHL9ENBGCiec4GhQm2SGnDT/sZy2bB3c3yozh/roS6cZJ
UZqjIDAeMA4GA1UdDwEB/wQEAwIBAjAMBgNVHQ4EBQQDAQIDMAoGCCqGSM49BAMC
A0gAMEUCIQCoAYN6CGZPgd5Sw5a1rd5VexciT5MCxTfXj+ZfJNfoiAIgQVCTB8AE
Nm2xset7+HOgtQYlKNw/rGd8cFcv5Y9aUzo=
-----END CERTIFICATE-----
-----BEGIN X509 CRL-----
MIHWMH0CAQEwCgYIKoZIzj0EAwIwEjEQMA4GA1UEAxMHdGVzdGluZxgPMDAwMTAx
MDIwMDAwMDBaGA8wMDAxMDEwMzAwMDAwMFowFjAUAgECGA8wMDAxMDEwMTAxMDAw
MFqgHjAcMA4GA1UdIwQHMAWAAwECAzAKBgNVHRQEAwIBBTAKBggqhkjOPQQDAgNJ
ADBGAiEAjqfj/IG4ys5WkjrbTNpDbr+saHGO/NujLJotlLL9KzgCIQDm8VZPzj0f
NYEQgAW4nsiUzlvEUCoHMw0141VCZXv67A==
-----END X509 CRL-----

Fixes #35428

Change-Id: Id96b6f47698d0bed39d586b46bd12374ee6ff88f
GitHub-Last-Rev: c83a601716
GitHub-Pull-Request: golang/go#36945
Reviewed-on: https://go-review.googlesource.com/c/go/+/217298
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-23 17:23:03 +00:00
Michael Anthony Knyszek
67c2dcbc59 runtime: use OnesCount64 to count allocated objects in a span
This change modifies the implementation of (*mspan).countAlloc by
using OnesCount64 (which on many systems is intrinsified). It does so by
using an unsafe pointer cast, but in this case we don't care about
endianness because we're just counting bits set.

This change means we no longer need the popcnt table which was redundant
in the runtime anyway. We can also simplify the logic here significantly
by observing that mark bits allocations are always 8-byte aligned, so we
don't need to handle any edge-cases due to the fact that OnesCount64
operates on 64 bits at a time: all irrelevant bits will be zero.

Overall, this implementation is significantly faster than the old one on
amd64, and should be similarly faster (or better!) on other systems
which support the intrinsic. On systems which do not, it should be
roughly the same performance because OnesCount64 is implemented using a
table in the general case.

Results on linux/amd64:

name                         old time/op  new time/op  delta
MSpanCountAlloc/bits=64-4    16.8ns ± 0%  12.7ns ± 0%  -24.40%  (p=0.000 n=5+4)
MSpanCountAlloc/bits=128-4   23.5ns ± 0%  12.8ns ± 0%  -45.70%  (p=0.000 n=4+5)
MSpanCountAlloc/bits=256-4   43.5ns ± 0%  12.8ns ± 0%  -70.67%  (p=0.000 n=4+5)
MSpanCountAlloc/bits=512-4   59.5ns ± 0%  15.4ns ± 0%  -74.12%  (p=0.008 n=5+5)
MSpanCountAlloc/bits=1024-4   116ns ± 1%    23ns ± 0%  -79.84%  (p=0.000 n=5+4)

Change-Id: Id4c994be22224653af5333683a69b0937130ed04
Reviewed-on: https://go-review.googlesource.com/c/go/+/216558
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-03-23 17:07:22 +00:00
Michael Anthony Knyszek
7af6a31b48 runtime: add countAlloc benchmark
This change adds a small microbenchmark for (*mspan).countAlloc, which
we're about to replace. Admittedly this isn't a critical piece of code,
but the benchmark was useful in understanding the performance change.

Change-Id: Iea93c00f571ee95534a42f2ef2ab026b382242b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/224438
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-23 16:55:37 +00:00
Michael Anthony Knyszek
8c30971da6 cmd/compile: panic if trying to alias an intrinsic with no definitions
Currently if we try to alias an intrinsic which hasn't been defined for
any architecture (such as by accidentally creating the alias before the
intrinsic is created with addF), then we'll just silently not apply any
intrinsics to those aliases.

Catch this particular case by panicking in alias if we try to apply the
alias and it did nothing.

Change-Id: I98e75fc3f7206b08fc9267cedb8db3e109ec4f5d
Reviewed-on: https://go-review.googlesource.com/c/go/+/224637
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-23 16:01:20 +00:00
Michael Anthony Knyszek
830ee4792e cmd/compile: declare runtime bit func aliases after math/bits intrinsics
Currently runtime/internal/sys bit-manipulation functions are aliased to
math/bits functions, which are intrinsified. Unfortunately these aliases
are declared before the intrinsified versions are generated, resulting
in the generic version of the code being copied over.

This change moves the aliases for bit operations in runtime/internal/sys
after the addF calls to generate those intrinsics in SSA, so that the
intrinsified SSA representation of those functions actually get copied
over.

This should improve the overall performance of the runtime (especially
the page allocator) since these bit operations will actually be
intrinsified now.

Change-Id: I4377da13f9a7bb6aee608e50df0297148bf8f806
Reviewed-on: https://go-review.googlesource.com/c/go/+/224437
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-23 15:52:57 +00:00
Matthew Dempsky
bb929b7452 cmd/compile: skip generating args_stackmap for "pulled" funcs
In golang.org/cl/171464, we cleaned up generation of .stkobj linker
symbols, but we couldn't figure out why a similar cleanup to
.args_stackmap linker symbols caused problems.

The issue is that we only need/want to generate .args_stackmap for
functions that are implemented in assembly in the same package. When
"pulling" a function from another package via //go:linkname, we can
safely skip emitting .args_stackmap, because compiling that package
will have generated it, if necessary.

Fixes #31615.

Change-Id: If8680aa7dd5b4e8f268b6b032d746f1b8536c867
Reviewed-on: https://go-review.googlesource.com/c/go/+/223238
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-03-23 03:56:18 +00:00
Andy Pan
0c30245996 runtime: correct the system-call name of kevent in comment
Change-Id: Ib1f4a6f7e36d28eff39f597df5c4703bf62654a4
GitHub-Last-Rev: 15ea1b9fa8
GitHub-Pull-Request: golang/go#37994
Reviewed-on: https://go-review.googlesource.com/c/go/+/224590
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-03-23 03:51:02 +00:00
Andy Pan
1250a324b4 bytes: narrow the search of IndexByte in Index
Change-Id: I5a47b18b64e7f781dcc77440b06de36966e3d01d
GitHub-Last-Rev: 8576f1931d
GitHub-Pull-Request: golang/go#37993
Reviewed-on: https://go-review.googlesource.com/c/go/+/224589
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-03-23 03:44:45 +00:00
Allen Li
ab2cc45cc9 go/ast: fix inflection in comments to match plurality
Change-Id: I3a725c5691a1090952acdc2ae0bed96aaa8d7067
GitHub-Last-Rev: 339a0dda36
GitHub-Pull-Request: golang/go#37256
Reviewed-on: https://go-review.googlesource.com/c/go/+/219737
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-03-23 03:31:16 +00:00
Ben Schwartz
02e492f1a3 runtime: speed up receive on empty closed channel
Currently, nonblocking receive on an open channel is about
700 times faster than nonblocking receive on a closed channel.
This change makes closed channels equally fast.

Fixes #32529.  Includes a correction based on #36714.

relevant benchstat output:
name                       old time/op    new time/op    delta
MakeChan/Byte-40            140ns ± 4%     137ns ± 7%   -2.38%  (p=0.023 n=17+19)
MakeChan/Int-40             174ns ± 5%     173ns ± 6%     ~     (p=0.437 n=18+19)
MakeChan/Ptr-40             315ns ±15%     301ns ±15%     ~     (p=0.051 n=20+20)
MakeChan/Struct/0-40        123ns ± 8%      99ns ±11%  -19.18%  (p=0.000 n=20+17)
MakeChan/Struct/32-40       297ns ± 8%     241ns ±18%  -19.13%  (p=0.000 n=20+20)
MakeChan/Struct/40-40       344ns ± 5%     273ns ±23%  -20.49%  (p=0.000 n=20+20)
ChanNonblocking-40         0.32ns ± 2%    0.32ns ± 2%   -1.25%  (p=0.000 n=19+18)
SelectUncontended-40       5.72ns ± 1%    5.71ns ± 2%     ~     (p=0.326 n=19+19)
SelectSyncContended-40     10.9µs ±10%    10.6µs ± 3%   -2.77%  (p=0.009 n=20+16)
SelectAsyncContended-40    1.00µs ± 0%    1.10µs ± 0%  +10.75%  (p=0.000 n=18+19)
SelectNonblock-40          1.22ns ± 2%    1.21ns ± 4%     ~     (p=0.141 n=18+19)
ChanUncontended-40          240ns ± 4%     233ns ± 4%   -2.82%  (p=0.000 n=20+20)
ChanContended-40           86.7µs ± 0%    82.7µs ± 0%   -4.64%  (p=0.000 n=20+19)
ChanSync-40                 294ns ± 7%     284ns ± 9%   -3.44%  (p=0.006 n=20+20)
ChanSyncWork-40            38.4µs ±19%    34.0µs ± 4%  -11.33%  (p=0.000 n=20+18)
ChanProdCons0-40           1.50µs ± 1%    1.63µs ± 0%   +8.53%  (p=0.000 n=19+19)
ChanProdCons10-40          1.17µs ± 0%    1.18µs ± 1%   +0.44%  (p=0.000 n=19+20)
ChanProdCons100-40          985ns ± 0%     959ns ± 1%   -2.64%  (p=0.000 n=20+20)
ChanProdConsWork0-40       1.50µs ± 0%    1.60µs ± 2%   +6.54%  (p=0.000 n=18+20)
ChanProdConsWork10-40      1.26µs ± 0%    1.26µs ± 2%   +0.40%  (p=0.015 n=20+19)
ChanProdConsWork100-40     1.27µs ± 0%    1.22µs ± 0%   -4.15%  (p=0.000 n=20+19)
SelectProdCons-40          1.50µs ± 1%    1.53µs ± 1%   +1.95%  (p=0.000 n=20+20)
ChanCreation-40            82.1ns ± 5%    81.6ns ± 7%     ~     (p=0.483 n=19+19)
ChanSem-40                  877ns ± 0%     719ns ± 0%  -17.98%  (p=0.000 n=18+19)
ChanPopular-40             1.75ms ± 2%    1.78ms ± 3%   +1.76%  (p=0.002 n=20+19)
ChanClosed-40               215ns ± 1%       0ns ± 6%  -99.82%  (p=0.000 n=20+18)

Previously committed in CL 181543 and reverted in CL 216158.

Change-Id: Ib767b08d724cfad03598d77271dbc1087485feb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/216818
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-03-22 20:37:22 +00:00
kadern0
9a2db7c41b cmd/compile: remove unused isIntOrdering method
Fixes #37989

Change-Id: Iabf86529fde3be9a98222b7e8d09ff8301cf8830
Reviewed-on: https://go-review.googlesource.com/c/go/+/224777
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-22 11:16:34 +00:00
Giovanni Bajo
683c266f95 build: add default GOROOT_BOOTSTRAP in Windows
CL 57753 added support to make.bash and make.rc to
default GOROOT_BOOTSTRAP to 'go env GOROOT'. This
patch does the same in make.bat for Windows.

Updates #18545
Fixes #28641

Change-Id: I9152cc5080ed219b4de5bad0bd12d7725422ee1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/96455
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2020-03-22 08:44:07 +00:00
Giovanni Bajo
787e7b048c build: force all Windows batch files to CRLF
Batch files should use CRLF endings. LF endings mostly
work but in some situations they cause random errors like
goto commands failing for mysterious reasons. See
golang.org/issue/37791 for more information.

Next CL triggered one of such bug (a label was not being
recognized), so prepare for it by converting to CRLF.

This CL also touches all existing batch files to force git
to update the line endings (unfortunately, changing
.gitattributes only has effect next time the file is checked
out or modified).

Fixes #37791
Updates #9281

Change-Id: I6f9a114351cb7ac9881914400aa210c930eb8cc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/96495
Run-TryBot: Giovanni Bajo <rasky@develer.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2020-03-22 08:42:38 +00:00
Ian Lance Taylor
2910c5b4a0 cmd/go: fix function name in comment
Fixes #37991

Change-Id: Ica58223f8564ec5d501d5b90b4258ffb78c42af1
Reviewed-on: https://go-review.googlesource.com/c/go/+/224587
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-22 01:21:05 +00:00
d-tsuji
36b815edd6 net/http: remove period at end of error message
Change-Id: I4ff5411543c200344babb754fc089e10e29e0fe4
Reviewed-on: https://go-review.googlesource.com/c/go/+/224697
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-22 00:10:27 +00:00
Alex Brainman
d467f3bbc9 runtime: ignore error returned by PowerRegisterSuspendResumeNotification
It appears that PowerRegisterSuspendResumeNotification is not supported
when running inside Docker - see issues #35447, #36557 and #37149.

Our current code relies on error number to determine Docker environment.
But we already saw PowerRegisterSuspendResumeNotification return
ERROR_FILE_NOT_FOUND, ERROR_INVALID_PARAMETERS and ERROR_ACCESS_DENIED
(see issues above). So this approach is not sustainable.

Just ignore PowerRegisterSuspendResumeNotification returned error.

Fixes #37149

Change-Id: I2beba9d45cdb8c1efac5e974e747827a6261915a
Reviewed-on: https://go-review.googlesource.com/c/go/+/219657
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
2020-03-21 06:00:19 +00:00
Keith Randall
287d67e3dd cmd/compile: indexed loads/stores can't be faultOnNilArg0
Because of the index, these ops can't guarantee faulting if arg0 is nil.

Clean up the PPC64 index ops - they can't take a sym or an offset.

Noticed while debugging #37881. I don't think it is the cause, but I guess
there is a chance.
Update #37881

Change-Id: Ic22925250bf7b1ba64e3cea1a65638bc4bab390c
Reviewed-on: https://go-review.googlesource.com/c/go/+/224457
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-21 02:46:16 +00:00
Keith Randall
5bc75a3097 runtime: handle empty stack in expandFinalInlineFrame
Fixes #37967

Change-Id: I6fc22bdd65f0263d5672731b73d09249201ab0aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/224458
Reviewed-by: Michael Pratt <mpratt@google.com>
2020-03-20 20:01:06 +00:00
Alberto Donizetti
c5058652fd math/big: document that Sqrt doesn't set Accuracy
Document that the Float.Sqrt method does not set the receiver's
Accuracy field.

Updates #37915

Change-Id: Ief1dcac07eacc0ef02f86bfac9044501477bca1c
Reviewed-on: https://go-review.googlesource.com/c/go/+/224497
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-03-20 19:24:37 +00:00
Austin Clements
d965bb6130 runtime: use divRoundUp
There are a handful of places where the runtime wants to round up the
result of a division. We just introduced a helper to do this. This CL
replaces all of the hand-coded round-ups (that I could find) with this
helper.

Change-Id: I465d152157ff0f3cad40c0aa57491e4f2de510ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/224385
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-20 16:05:35 +00:00
Austin Clements
ab5a40c5e3 runtime: fix rounding in materializeGCProg
materializeGCProg allocates a temporary buffer for unrolling a GC
program. Unfortunately, when computing the size of the buffer, it
rounds *down* the number of bytes needed to store bitmap before
rounding up the number of pages needed to store those bytes. The fact
that it rounds up to pages usually mitigates the rounding down, but
the type from #37470 exists right on the boundary where this doesn't
work:

type Sequencer struct {
	htable [1 << 17]uint32
	buf    []byte
}

On 64-bit, this GC bitmap is exactly 8 KiB of zeros, followed by three
one bits. Hence, this needs 8193 bytes of storage, but the current
math in materializeGCProg rounds *down* the three one bits to 8192
bytes. Since this is exactly pageSize, the next step of rounding up to
the page size doesn't mitigate this error, and materializeGCProg
allocates a buffer that is one byte too small. runGCProg then writes
one byte past the end of this buffer, causing either a segfault (if
you're lucky!) or memory corruption.

Fixes #37470.

Change-Id: Iad24c463c501cd9b1dc1924bc2ad007991a094a0
Reviewed-on: https://go-review.googlesource.com/c/go/+/221197
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-20 16:05:33 +00:00
Than McIntosh
776a9d0958 test: fix -test.v trace output for cgo/testshared
Trace output showing how dummy GOROOT was being set up was incorrect
(sense of the "cp -r" trace messages was inverted). This patch fixes
the problem.

Change-Id: Ib0ee649e305bfa1bc0c49e0d5ba2ea31e0a4f67e
Reviewed-on: https://go-review.googlesource.com/c/go/+/224377
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-20 14:28:59 +00:00
HowJMay
9d468f482f doc/articles/wiki: use correct variable name in closures guide
Fixes non-existent variable TitleValidator to be validPath in
the closures, functions literal section.

Fixes #36779

Change-Id: I59762c358c3e00d1cc03d9d1e2aace03f145321d
GitHub-Last-Rev: a5e9b17a37
GitHub-Pull-Request: golang/go#36783
Reviewed-on: https://go-review.googlesource.com/c/go/+/216479
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-20 08:42:30 +00:00
Ian Lance Taylor
635a83047b runtime/race: test that close synchronizes with read
Add a test to ensure that the race detector sees that closing a
channel synchronizes with a read from that channel.
This test case failed when CL 181543 was in the tree.
CL 181543 was reverted in CL 216158; this adds a test to make
sure that we don't re-introduce the problem at a later date.

For #32529
For #36714

Change-Id: I5a40f744c67c3f8191d6ad822710c180880a7375
Reviewed-on: https://go-review.googlesource.com/c/go/+/216099
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-20 04:00:36 +00:00
Ian Lance Taylor
65367df477 internal/poll: update Windows Server Performance URL in comment
Change-Id: Ida8048720611ecf50d7e7626a42715921bbc753b
Reviewed-on: https://go-review.googlesource.com/c/go/+/224237
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-20 01:53:12 +00:00
Ian Lance Taylor
d55e0c10ad net: merge common Unix/Windows methods
When we added the internal/poll package, the Unix and Windows implementations
of several netFD methods became exactly the same, except for using a
different name for the string passed to wrapSyscallError.

One case is not an exact duplicate: we slightly tweak the implementation
of (*netFD).shutdown on Windows to wrap the error.

Change-Id: I3d87a317d5468ff8f1958d86f6189ea1ba697e9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/224140
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-20 00:49:34 +00:00
Emmanuel T Odeke
a3a9901c1e net/http: update bundled x/net/http2
Updates bundled http2 to x/net git rev 63522dbf7

    http2: reduce allocations of (*clientConnReadLoop).handleReponse
    https://golang.org/cl/223783 (#37853)

    http2: remove unused errors
    https://golang.org/cl/220458

    http2: remove unused stream struct fields
    https://golang.org/cl/219857

    http2: fix typo in comment
    https://golang.org/cl/214602

    http2: workaround TCPConn CloseWrite not being supported on Plan 9
    https://golang.org/cl/209417 (#17906, #35904)

Change-Id: I0e48f32247938c3858170bf419624367d4faef4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/224217
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-03-20 00:33:16 +00:00
Ian Lance Taylor
a0917eb959 internal/poll: merge a couple of common definitions
(*FD).Shutdown and (*FD).RawControl were exactly identical in fd_unix.go
and fd_windows.go, so merge them into fd_posix.go.

Change-Id: Ie751edb06d293e08f1493682d6bc5f22e8f3e628
Reviewed-on: https://go-review.googlesource.com/c/go/+/224137
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-20 00:27:02 +00:00
Ian Lance Taylor
7f3ca5dfeb os: merge common Unix/Windows methods
Several method implementations were identical in file_unix.go and
file_windows.go. Merge them into file_posix.go.

Change-Id: I8bcfad468829530f81f52fe426b3a8c042e7bbd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/224138
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-20 00:26:13 +00:00
Tobias Klauser
965f4566e9 crypto/x509: bump minimum macOS version to 10.11
The minimum macOS supported version is 10.11 as of Go 1.14, see #23011.
Thus, bump macosx-version-min to 10.11

While at it, drop __MAC_OS_X_VERSION_MAX_ALLOWED as suggested by
Filippo:

  In general, I can see why we'd want to tell the libraries which
  minimum version we target so they drop compatibility with older
  versions. No idea why we'd specify a max version, unless it's to make
  sure we don't use APIs added after that version, but then it would
  have to be 1011 not 1015.

  Let's try dropping that define and see if anything blows up? ¯\_(ツ)_/¯

Change-Id: I6b76623a9404724ccda40311ff95b3475ae8a60c
Reviewed-on: https://go-review.googlesource.com/c/go/+/214059
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-03-19 22:33:15 +00:00
Jay Conrod
b816cb2cd5 cmd/go: update 'go help modules' for automatic vendoring
* Mention vendor/modules.txt verification with -mod=vendor.
* Update "Modules and vendoring" section.

Fixes #37930

Change-Id: Ia3ee72457daabaa2fc15b7ea7427eb324c088b8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/223926
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-03-19 21:31:54 +00:00
Chris Le Roy
16822a2bc4 crypto/tls: update the MITM reference to "machine-in-the-middle"
Changing "man-in-the-middle" references to "machine-in-the-middle",
it's a more inclusive term and still aligns with the MITM acronym.

Change-Id: I81f954cff3d252433443f159ff9edaf59a28ab9d
GitHub-Last-Rev: 3e8f91424a
GitHub-Pull-Request: golang/go#37918
Reviewed-on: https://go-review.googlesource.com/c/go/+/223897
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-03-19 21:31:51 +00:00
Changkun Ou
93a9561b23 testing: fix data race between parallel subtests
This CL fixes a race condition if there are two subtests, and
one finishing but the other is panicking.

Fixes #37551

Change-Id: Ic33963eb338aec228964b95f7c34a0d207b91e00
Reviewed-on: https://go-review.googlesource.com/c/go/+/221322
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-03-19 21:07:59 +00:00
Daniel Martí
a4c48d61f5 sync/atomic: remove panic64
The func has been unused since https://golang.org/cl/93637 in 2018.

Change-Id: I1cab6f265aa5058ac080fd7c7cbf0fe85370f073
Reviewed-on: https://go-review.googlesource.com/c/go/+/224077
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matt Layher <mdlayher@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-03-19 19:46:54 +00:00
Lynn Boger
cbd421f75b runtime: improve memmove for short moves on ppc64
This improves the performance of memmove for almost all moves <= 16 bytes
for the ppc64 assembler, improving linux/ppc64le, linux/ppc64, aix/ppc64.

Only the forward moves were changed, the backward moves were left as is.
Additional macro defines were added to improve the readability of the asm.

Results from power8:
name                      old time/op    new time/op    delta
Memmove/0                   5.70ns ± 0%    5.69ns ± 0%    -0.18%  (p=0.029 n=4+4)
Memmove/1                   5.54ns ± 0%    5.39ns ± 0%    -2.71%  (p=0.029 n=4+4)
Memmove/2                   6.31ns ± 0%    5.55ns ± 0%   -12.08%  (p=0.029 n=4+4)
Memmove/3                   7.41ns ± 0%    5.54ns ± 0%   -25.24%  (p=0.029 n=4+4)
Memmove/4                   8.41ns ± 0%    5.56ns ± 0%   -33.87%  (p=0.029 n=4+4)
Memmove/5                   10.1ns ± 5%     5.5ns ± 0%   -45.30%  (p=0.029 n=4+4)
Memmove/6                   10.3ns ± 0%     5.6ns ± 0%   -45.92%  (p=0.029 n=4+4)
Memmove/7                   11.4ns ± 0%     5.7ns ± 0%   -50.33%  (p=0.029 n=4+4)
Memmove/8                   5.66ns ± 0%    5.54ns ± 0%    -2.12%  (p=0.029 n=4+4)
Memmove/9                   5.66ns ± 0%    6.47ns ± 0%   +14.31%  (p=0.029 n=4+4)
Memmove/10                  6.67ns ± 0%    6.22ns ± 0%    -6.82%  (p=0.029 n=4+4)
Memmove/11                  7.83ns ± 0%    6.45ns ± 0%   -17.60%  (p=0.029 n=4+4)
Memmove/12                  8.91ns ± 0%    6.25ns ± 0%   -29.85%  (p=0.029 n=4+4)
Memmove/13                  9.81ns ± 0%    6.48ns ± 0%   -33.94%  (p=0.029 n=4+4)
Memmove/14                  10.7ns ± 1%     6.4ns ± 0%   -40.00%  (p=0.029 n=4+4)
Memmove/15                  11.8ns ± 0%     6.7ns ± 0%   -42.84%  (p=0.029 n=4+4)
Memmove/16                  5.63ns ± 0%    5.56ns ± 0%    -1.20%  (p=0.029 n=4+4)

Change-Id: I2de434f543c5a017395e0850fb9b9f7219583bbb
Reviewed-on: https://go-review.googlesource.com/c/go/+/223317
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
2020-03-19 17:52:50 +00:00
Tim Möhlmann
f9f57c4443 database/sql: add method Err on sql.Row
The Row.Err method is intended to assist wrapping sql.DB.
Because sql.Row is a struct with private fields,
a wrapper in an existing code base cannot easily provide users
with a different implementation without large rewrites.
Adding this method allows query level errors to be handled
centrally.

Fixes #35804

Change-Id: I94e6329de89a7ee1284ce9ef76af4363d2d081f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/214317
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-19 16:17:59 +00:00
Ian Lance Taylor
b3b174ffcf runtime: minor updates to netpoll comments
In Go 1.4 we renamed READY to pdReady and WAIT to pdWait as part of
rewriting netpoll from C to Go. Finish updating the comments to use
the new names.

Change-Id: I6cefc698b46c58211fd6be1489bdd70419454962
Reviewed-on: https://go-review.googlesource.com/c/go/+/223998
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-03-19 00:08:40 +00:00
Harmen
019421d17f database/sql: fix variable name in example
It's a very minor error, but it's a bad copy/paste example.

Change-Id: Ia6a723c31f2205c933857ce2cf715bddf773ebb6
GitHub-Last-Rev: 7f14b1a5c1
GitHub-Pull-Request: golang/go#37932
Reviewed-on: https://go-review.googlesource.com/c/go/+/223960
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
2020-03-18 20:24:57 +00:00
Kyle Nusbaum
02057906f7 context: prevent creation of invalid contexts
This commit makes it impossible to create derived contexts with nil parents.
Previously it was possible to create derived contexts with nil parents, and
invalid contexts could propogate through the program. Eventually this can
cause a panic downstream, which is difficult to trace back to the source
of the error.

Although `WithCancel` and `WithDeadline` already panic if `parent` is `nil`, this adds explicit checks to give a useful message in the panic.

Fixes #37908

Change-Id: I70fd01f6539c1b0da0e775fc5457e32e7075e52c
GitHub-Last-Rev: 1b7dadd7db
GitHub-Pull-Request: golang/go#37898
Reviewed-on: https://go-review.googlesource.com/c/go/+/223777
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-03-18 19:44:13 +00:00
Bryan C. Mills
61ce82a360 cmd/go: skip the cmd/cgo staleness check on darwin builders
Some of the darwin-amd64 builders are providing a stale environment.
Let's un-break them while we investigate.

Updates #37573
Updates #33598

Change-Id: I8b79778fe4d5aa916557c1ba89fa9c776d130b01
Reviewed-on: https://go-review.googlesource.com/c/go/+/223925
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-03-18 19:30:29 +00:00
Michael Anthony Knyszek
f1f947af28 runtime: don't hold worldsema across mark phase
This change makes it so that worldsema isn't held across the mark phase.
This means that various operations like ReadMemStats may now stop the
world during the mark phase, reducing latency on such operations.

Only three such operations are still no longer allowed to occur during
marking: GOMAXPROCS, StartTrace, and StopTrace.

For the former it's because any change to GOMAXPROCS impacts GC mark
background worker scheduling and the details there are tricky.

For the latter two it's because tracing needs to observe consistent GC
start and GC end events, and if StartTrace or StopTrace may stop the
world during marking, then it's possible for it to see a GC end event
without a start or GC start event without an end, respectively.

To ensure that GOMAXPROCS and StartTrace/StopTrace cannot proceed until
marking is complete, the runtime now holds a new semaphore, gcsema,
across the mark phase just like it used to with worldsema.

This change is being landed once more after being reverted in the Go
1.14 release cycle, since CL 215157 allows it to have a positive
effect on system performance.

For the benchmark BenchmarkReadMemStatsLatency in the runtime, which
measures ReadMemStats latencies while the GC is exercised, the tail of
these latencies reduced dramatically on an 8-core machine:

name                   old 50%tile-ns  new 50%tile-ns  delta
ReadMemStatsLatency-8      4.40M ±74%      0.12M ± 2%  -97.35%  (p=0.008 n=5+5)

name                   old 90%tile-ns  new 90%tile-ns  delta
ReadMemStatsLatency-8       102M ± 6%         0M ±14%  -99.79%  (p=0.008 n=5+5)

name                   old 99%tile-ns  new 99%tile-ns  delta
ReadMemStatsLatency-8       147M ±18%         4M ±57%  -97.43%  (p=0.008 n=5+5)

Fixes #19812.

Change-Id: If66c3c97d171524ae29f0e7af4bd33509d9fd0bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/216557
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-03-18 19:13:50 +00:00
Ian Lance Taylor
e39de05186 doc/go1.14: mention Windows change for Open permissions
For #35033

Change-Id: Ie15353322d5cfe7320199103ad9543fb89a842ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/223957
Reviewed-by: Brendan Jackman <jackmanb@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-03-18 18:59:32 +00:00
Bryan C. Mills
3b76bed3cd cmd/internal/moddeps: skip GOROOT/pkg when locating modules
Fixes #37929

Change-Id: I1435411de43aed98f40e0d98e450310b0fdf804c
Reviewed-on: https://go-review.googlesource.com/c/go/+/223924
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-03-18 18:12:40 +00:00
Michael Anthony Knyszek
7cfb814b0a runtime: add ReadMemStats latency benchmark
This change adds a benchmark to the runtime which measures ReadMemStats
latencies. It generates allocations with lots of pointers to keep the GC
busy while hitting ReadMemStats and measuring the time it takes to
complete.

Updates #19812.

Change-Id: I7a76aaf497ba5324d3c7a7b3df32461b3e6c3ac8
Reviewed-on: https://go-review.googlesource.com/c/go/+/220177
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-03-18 17:59:19 +00:00
Michael Anthony Knyszek
79b43fa819 runtime: preempt dedicated background mark workers for STW
Currently, dedicated background mark workers are essentially always
non-preemptible.

This change makes it so that dedicated background mark workers park if
their preemption flag is set and someone is trying to STW, allowing them
to do so.

This change prepares us for allowing a STW to happen (and happen
promptly) during GC marking in a follow-up change.

Updates #19812.

Change-Id: I67fb6085bf0f0aebd18ca500172767818a1f15e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/215157
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-03-18 17:59:09 +00:00