1
0
mirror of https://github.com/golang/go synced 2024-11-18 08:44:43 -07:00
Commit Graph

40963 Commits

Author SHA1 Message Date
adarsh ravichandran
776e1709e5 math/bits: add example for OnesCount function
Change-Id: Id87db9bed5e8715d554c1bf95c063d7d0a03c3e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/178117
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-20 18:16:09 +00:00
Keith Randall
c77a9e0aa5 runtime: In Frames.Next, delay file/line lookup until just before return
That way we will never have to look up the file/line for the frame
that's next to be returned when the user stops calling Next.

For the benchmark from #32093:

name      old time/op  new time/op  delta
Helper-4   948ns ± 1%   836ns ± 3%  -11.89%  (p=0.000 n=9+9)

(#32093 was fixed with a more specific, and better, fix, but this
fix is much more general.)

Change-Id: I89e796f80c9706706d8d8b30eb14be3a8a442846
Reviewed-on: https://go-review.googlesource.com/c/go/+/178077
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-20 17:41:49 +00:00
Michael Anthony Knyszek
5a90306344 runtime: overhaul TestPhysicalMemoryUtilization
Currently, this test allocates many objects and relies on heap-growth
scavenging to happen unconditionally on heap-growth. However with the
new pacing system for the scavenging, this is no longer true and the
test is flaky.

So, this change overhauls TestPhysicalMemoryUtilization to check the
same aspect of the runtime, but in a much more robust way.

Firstly, it sets up a much more constrained scenario: only 5 objects are
allocated total with a maximum worst-case (i.e. the test fails) memory
footprint of about 16 MiB. The test is now aware that scavenging will
only happen if the heap growth causes us to push way past our scavenge
goal, which is based on the heap goal. So, it makes the holes in the
test much bigger and the actual retained allocations much smaller to
keep the heap goal at the heap's minimum size. It does this twice to
create exactly two unscavenged holes. Because the ratio between the size
of the "saved" objects and the "condemned" object is so small, two holes
are sufficient to create a consistent test.

Then, the test allocates one enormous object (the size of the 4 other
objects allocated, combined) with the intent that heap-growth scavenging
should kick in and scavenge the holes. The heap goal will rise after
this object is allocated, so it's very important we do all the
scavenging in a single allocation that exceeds the heap goal because
otherwise the rising heap goal could foil our test.

Finally, we check memory use relative to HeapAlloc as before. Since the
runtime should scavenge the entirety of the remaining holes,
theoretically there should be no more free and unscavenged memory.
However due to other allocations that may happen during the test we may
still see unscavenged memory, so we need to have some threshold. We keep
the current 10% threshold which, while arbitrary, is very conservative
and should easily account for any other allocations the test makes.

Before, we also had to ensure the allocations we were making looked
large relative to the size of a heap arena since newly-mapped memory was
considered unscavenged, and so that could significantly skew the test.
However, thanks to the fix for #32012 we were able to reduce memory use
to 16 MiB in the worst case.

Fixes #32010.

Change-Id: Ia38130481e292f581da7fa3289c98c99dc5394ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/177237
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-20 16:26:57 +00:00
Alex Myasoedov
82ee4e7f78 context: document CancelFunc to be safe for simultaneous use by multiple goroutines
Fixes #32145

Change-Id: If4c9dd3a2af748974141ad6e571f80efcbaad772
Reviewed-on: https://go-review.googlesource.com/c/go/+/177899
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-20 16:08:56 +00:00
Shulhan
295c56627a internal/envcmd: print GO111MODULE when executing "go env"
If we look at the issues in the past releases that are related
to go command that involved modules, its usually mention or ask about
the value of GO111MODULE, either in separate line or in separate
comment.

There are quite long time range before GO111MODULE will be removed
(unused).  The next release is still default to auto [1], and until Go
1.13 unsupported (two releases after that) there is about one and half
years after that.

Since the change is not that big (one line) [2], maybe temporary adding
it to "go env" give more clarity and benefit in issue reporting rather
than not.

[1] https://github.com/golang/go/issues/31857

Fixes #29656

Change-Id: I609ad6664774018e4f4147ec6158485172968e16
Reviewed-on: https://go-review.googlesource.com/c/go/+/176837
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-20 15:38:52 +00:00
LE Manh Cuong
4ee4607c97 cmd/compile: use internal/race
CL 14870 added internal/race to factor out duplicated race thunks,
we should use it.

No signification changes in compile time and compile binary size.

Change-Id: I786af44dd5bb0f4ab6709432eeb603f27a5b6c63
Reviewed-on: https://go-review.googlesource.com/c/go/+/178118
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-20 15:09:05 +00:00
Brad Fitzpatrick
c6f9321b5a net/http/httptest: update docs, remove old inaccurate sentence
The "After it is called, changing rw.Header will not affect
rw.HeaderMap" claim predates the Result method which changed how the
Recorder should be used.

Fixes #32144
Fixes #32136

Change-Id: I95bdfa5ac489ce7b0202824bb5663f4da188e8a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/178058
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-20 04:22:40 +00:00
Brad Fitzpatrick
be9f10b2b8 cmd/go/internal/work: fix a couple typos
Change-Id: I357669d8c9bc004031b17f057803c9b152edefee
Reviewed-on: https://go-review.googlesource.com/c/go/+/178057
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-20 02:21:25 +00:00
taoyuanyuan
5eeb372418 internal/poll: avoid memory leak in Writev
The chunks that were referenced by fd.iovecs would not be GC.

Change-Id: I7bfcb91a3fef57a4a1861168e9cd3ab55ce1334e
GitHub-Last-Rev: e0b7f68447
GitHub-Pull-Request: golang/go#32138
Reviewed-on: https://go-review.googlesource.com/c/go/+/178037
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-20 01:53:24 +00:00
smasher164
5ca44dc403 math/bits: make Add and Sub fallbacks constant time
Make the extended precision add-with-carry and sub-with-carry operations
take a constant amount of time to execute, regardless of input.

name             old time/op  new time/op  delta
Add-4            1.16ns ±11%  1.51ns ± 5%  +30.52%  (p=0.008 n=5+5)
Add32-4          1.08ns ± 0%  1.03ns ± 1%   -4.86%  (p=0.029 n=4+4)
Add64-4          1.09ns ± 1%  1.95ns ± 3%  +79.23%  (p=0.008 n=5+5)
Add64multiple-4  4.03ns ± 1%  4.55ns ±11%  +13.07%  (p=0.008 n=5+5)
Sub-4            1.08ns ± 1%  1.50ns ± 0%  +38.17%  (p=0.016 n=5+4)
Sub32-4          1.09ns ± 2%  1.53ns ±10%  +40.26%  (p=0.008 n=5+5)
Sub64-4          1.10ns ± 1%  1.47ns ± 1%  +33.39%  (p=0.008 n=5+5)
Sub64multiple-4  4.30ns ± 2%  4.08ns ± 4%   -5.07%  (p=0.032 n=5+5)

Fixes #31267

Change-Id: I1824b1b3ab8f09902ce8b5fef84ce2fdb8847ed9
Reviewed-on: https://go-review.googlesource.com/c/go/+/170758
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-20 01:13:27 +00:00
Keith Randall
1ab063ce53 testing: callerName only needs one PC in the traceback
callerName requested 2 PCs from Callers, and that causes
both to be looked up in the file/line mapping.
We really only need to do the work for one PC.
(And in fact the caller doesn't need file/line at all, but
the Callers API can't express that.)

We used to request 2 PCs because in 1.11 and earlier we
stored an inline skip count in the second entry.
That's not necessary any more (as of 1.12).

Fixes #32093

Change-Id: I7b272626ef6496e848ee8af388cdaafd2556857b
Reviewed-on: https://go-review.googlesource.com/c/go/+/177858
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Caleb Spare <cespare@gmail.com>
2019-05-17 22:32:30 +00:00
Josh Bleecher Snyder
5c3f3fbd0f cmd/compile: optimize postorder
name        old alloc/op      new alloc/op      delta
Template         37.1MB ± 0%       36.8MB ± 0%  -0.57%  (p=0.008 n=5+5)
Unicode          28.1MB ± 0%       28.1MB ± 0%  -0.07%  (p=0.008 n=5+5)
GoTypes           125MB ± 0%        124MB ± 0%  -0.61%  (p=0.008 n=5+5)
Compiler          571MB ± 0%        568MB ± 0%  -0.60%  (p=0.008 n=5+5)
SSA              1.88GB ± 0%       1.86GB ± 0%  -0.82%  (p=0.008 n=5+5)
Flate            22.9MB ± 0%       22.8MB ± 0%  -0.59%  (p=0.008 n=5+5)
GoParser         27.5MB ± 0%       27.3MB ± 0%  -0.53%  (p=0.008 n=5+5)
Reflect          79.8MB ± 0%       79.5MB ± 0%  -0.40%  (p=0.008 n=5+5)
Tar              34.9MB ± 0%       34.7MB ± 0%  -0.44%  (p=0.008 n=5+5)
XML              45.7MB ± 0%       45.4MB ± 0%  -0.58%  (p=0.008 n=5+5)
[Geo mean]       80.3MB            79.9MB       -0.52%

name        old allocs/op     new allocs/op     delta
Template           380k ± 0%         378k ± 0%  -0.57%  (p=0.008 n=5+5)
Unicode            340k ± 0%         340k ± 0%  -0.08%  (p=0.008 n=5+5)
GoTypes           1.36M ± 0%        1.36M ± 0%  -0.44%  (p=0.008 n=5+5)
Compiler          5.52M ± 0%        5.49M ± 0%  -0.45%  (p=0.008 n=5+5)
SSA               17.6M ± 0%        17.5M ± 0%  -0.42%  (p=0.008 n=5+5)
Flate              235k ± 0%         234k ± 0%  -0.65%  (p=0.008 n=5+5)
GoParser           302k ± 0%         300k ± 0%  -0.70%  (p=0.008 n=5+5)
Reflect            982k ± 0%         978k ± 0%  -0.40%  (p=0.008 n=5+5)
Tar                353k ± 0%         351k ± 0%  -0.53%  (p=0.008 n=5+5)
XML                437k ± 0%         435k ± 0%  -0.48%  (p=0.008 n=5+5)
[Geo mean]         844k              840k       -0.47%

Updates #27739

Change-Id: I5d533013270cbbd7c0bad1b43da96c8499be76f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/177917
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-17 19:17:29 +00:00
Filippo Valsorda
e22e2b371d crypto/tls: fix TestVerifyHostnameResumed
In TLS 1.3 session tickets are delivered after the handshake, and it
looks like now the Google servers wait until the first flight of data to
send them (or our timeout is too low). Cause some data to be sent so we
can avoid the guessing game.

Fixes #32090

Change-Id: I54af4acb3a89cc70c9e14a5dfe18a44c29a841a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/177877
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-17 18:28:37 +00:00
Dmitri Shuralyov
3f7c560829 net/http/httputil: remove os.Stderr claim in ReverseProxy.ErrorLog docs
The motivation for doing so is to avoid making inaccurate claims.
Logging may not go to os.Stderr if anyone overrides the log package's
default output via https://godoc.org/log#SetOutput. Saying that
the standard logger is used should be sufficient to explain the
behavior, and users can infer that os.Stderr is used by default,
unless it's changed.

This change is the same as what was applied to http.Server.ErrorLog
documentation in CL 53950.

Change-Id: I32873fc548ceee573f8616b4d49b8a8b98881803
Reviewed-on: https://go-review.googlesource.com/c/go/+/176817
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-17 17:33:16 +00:00
Jay Conrod
06642d8e77 cmd/go: don't attempt to downgrade to incompatible versions
When we downgrade a module (using 'go get m@none' or similar), we
exclude versions of other modules that depend on it. We'll try
previous versions (in the "versions" list returned by the proxy or in
codeRepo.Versions for vcs) until we find a version that doesn't
require an excluded module version.

If older versions of a module are broken for some reason,
mvs.Downgrade currently panics. With this change, we ignore versions
with errors during downgrade.

A frequent cause of this is incompatible v2+ versions. These are
common if a repository tagged v2.0.0 before migrating to modules, then
tagged v2.0.1 with a go.mod file later. v2.0.0 is incorrectly
considered part of the v2 module.

Fixes #31942

Change-Id: Icaa75c5c93f73f18a400c22f18a8cc603aa4011a
Reviewed-on: https://go-review.googlesource.com/c/go/+/177337
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-17 16:39:37 +00:00
Jean de Klerk
703fb665d6 errors: update As example to include else case
The current example illustrates using As when the error is able to be
interpreted as an os.PathError, but elides the "else" case. This CL adds the
small extra else case to make it clear that it's not safe to assume As will
return true.

This CL also squash the err instantiation and the err nil check into one line
for brevity.

Change-Id: I3d3ab483ffb38fb2788d0498b3f03229a87dd7c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/177717
Reviewed-by: Jonathan Amsterdam <jba@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-17 16:21:05 +00:00
Filippo Valsorda
f35338582d crypto/tls: add support for Ed25519 certificates in TLS 1.2 and 1.3
Support for Ed25519 certificates was added in CL 175478, this wires them
up into the TLS stack according to RFC 8422 (TLS 1.2) and RFC 8446 (TLS 1.3).

RFC 8422 also specifies support for TLS 1.0 and 1.1, and I initially
implemented that, but even OpenSSL doesn't take the complexity, so I
just dropped it. It would have required keeping a buffer of the
handshake transcript in order to do the direct Ed25519 signatures. We
effectively need to support TLS 1.2 because it shares ClientHello
signature algorithms with TLS 1.3.

While at it, reordered the advertised signature algorithms in the rough
order we would want to use them, also based on what curves have fast
constant-time implementations.

Client and client auth tests changed because of the change in advertised
signature algorithms in ClientHello and CertificateRequest.

Fixes #25355

Change-Id: I9fdd839afde4fd6b13fcbc5cc7017fd8c35085ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/177698
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
2019-05-17 16:13:45 +00:00
Joel Sing
ee551846fa runtime: correct netbsd/amd64 assembly for timespec handling
A timespec on netbsd/amd64 is int64/int64, not int64/int32.
This bug appears to have been introduced in 7777bac6e4.

Spotted by Cherry Zhang while reviewing https://go-review.googlesource.com/c/go/+/177120.

Change-Id: I163c55d926965defd981bdbfd2511de7d9d4c542
Reviewed-on: https://go-review.googlesource.com/c/go/+/177637
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
2019-05-17 14:32:34 +00:00
Joel Sing
37133b5d63 runtime: fix netbsd/arm64 assembly
Fix various bugs in the netbsd/arm64 runtime assembly.

Updates #30824.

Change-Id: I5ca10926ab663a8ff4df9973530e645e2469c1aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/177120
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-17 14:30:55 +00:00
Ben Shi
9be2d46422 misc/android: fix a typo in README
A cross C compiler should be specified when cross building go
for android on x86 host. This CL fixes the typo in README.

Change-Id: Ideda915364d9f7f0fcd53146abc1e74506a8a8e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/177777
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-05-17 06:01:17 +00:00
Joey
29f14f058d runtime: remove redundant "//"
Change-Id: I44af6cd8baa8fcd2a666429467fae4bed5f21fa0
GitHub-Last-Rev: d0c9722f08
GitHub-Pull-Request: golang/go#32052
Reviewed-on: https://go-review.googlesource.com/c/go/+/177277
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-05-17 04:27:18 +00:00
Keith Randall
6c842cc610 cmd/compile: mention issue 28603 when discussing where inline marks go
Update #28603

Change-Id: I8bdbea57b3661e15dcc760d27252b794daa476db
Reviewed-on: https://go-review.googlesource.com/c/go/+/177697
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-05-17 03:46:38 +00:00
Jay Conrod
97ecc4321e cmd/go: don't panic when explaining lost upgrades due to downgrades
If a user runs 'go get mod@vers' where the module transitively
requires itself at a newer version, 'go get' attempts to perform a
downgrade, which necessarily excludes the requested version of the
module.

Previously, we called mvs.BuildList with the requested module
version as the target. This panicked because BuildList doesn't allow
the target module (typically the main module) to require a newer
version of itself.

With this change, when we lose an upgrade due to a downgrade, we call
mvs.BuildList through a wrapper that treats the lost module version as
requirement of a synthetic root module, rather than the target
module. This avoids the panic.

This change also starts reporting errors when an upgraded module is
lost entirely (downgrades caused the module to be completely removed
from the build list).

Fixes #31491

Change-Id: I70ca261c20af7553cad2d3b840a1eaf3d18a4191
Reviewed-on: https://go-review.googlesource.com/c/go/+/177602
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-16 22:39:38 +00:00
Michael Anthony Knyszek
4e7bef84c1 runtime: mark newly-mapped memory as scavenged
On most platforms newly-mapped memory is untouched, meaning the pages
backing the region haven't been faulted in yet. However, we mark this
memory as unscavenged which means the background scavenger
aggressively "returns" this memory to the OS if the heap is small.

The only platform where newly-mapped memory is actually unscavenged (and
counts toward the application's RSS) is on Windows, since
(*mheap).sysAlloc commits the reservation. Instead of making a special
case for Windows, I change the requirements a bit for a sysReserve'd
region. It must now be both sysMap'd and sysUsed'd, with sysMap being a
no-op on Windows. Comments about memory allocation have been updated to
include a more up-to-date mental model of which states a region of memory
may be in (at a very low level) and how to transition between these
states.

Now this means we can correctly mark newly-mapped heap memory as
scavenged on every platform, reducing the load on the background
scavenger early on in the application for small heaps. As a result,
heap-growth scavenging is no longer necessary, since any actual RSS
growth will be accounted for on the allocation codepath.

Finally, this change also cleans up grow a little bit to avoid
pretending that it's freeing an in-use span and just does the necessary
operations directly.

Fixes #32012.
Fixes #31966.
Updates #26473.

Change-Id: Ie06061eb638162e0560cdeb0b8993d94cfb4d290
Reviewed-on: https://go-review.googlesource.com/c/go/+/177097
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-16 22:00:47 +00:00
Brad Fitzpatrick
cb5c82bc3d net/http: update vendored, bundled x/net/http2
For:

    http2: track reused connections
    https://golang.org/cl/176720 (updates golang/go#31982)

Some x/sys/unix updates come along for the ride too.

I filed #32031 for making the bundling process less difficult and
error-prone in the future.

Change-Id: Ic822080991ffa2d50352c5f613e45648a327cf16
Reviewed-on: https://go-review.googlesource.com/c/go/+/177037
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-16 19:56:27 +00:00
Bryan C. Mills
e1fb8e4291 cmd/go: include "gccgo stdlib" in lost build action check
Fixes #32060

Change-Id: I49ab48a9bdd64a7ed777a045e2d92d28a37b5c0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/177600
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 18:48:26 +00:00
Bryan C. Mills
48b7bdd0d7 cmd/go: always add 'go' directive to the go.mod file if missing
Updates #30790
Fixes #31960

Change-Id: Ib3ac074cf1f98fe69f53af82d2a0441582116570
Reviewed-on: https://go-review.googlesource.com/c/go/+/176925
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-05-16 17:47:34 +00:00
Josh Bleecher Snyder
e7b7f87432 cmd/compile: invalidate CFG when fuseIf triggers
The compiler appears to have a latent bug:
fusePlain calls invalidateCFG when it changes block structure,
but fuseIf does not.

Fix this by hoisting the call to invalidateCFG to the top level.

Change-Id: Ic960fb3ac963b15b4a225aad84863d58efa954e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/177198
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-05-16 17:17:29 +00:00
fanzha02
822a9f537f cmd/compile: fix the error of absorbing boolean tests into block(FGE, FGT)
The CL 164718 mistyped the comparison flags. The rules for floating
point comparison should be GreaterThanF and GreaterEqualF. Fortunately,
the wrong optimizations were overwritten by other integer rules, so the
issue won't cause failure but just some performance impact.

The fixed CL optimizes the floating point test as follows.

source code: func foo(f float64) bool { return f > 4 || f < -4}
previous version: "FCMPD", "CSET\tGT", "CBZ"
fixed version: "FCMPD", BLE"

Add the test case.

Change-Id: Iea954fdbb8272b2d642dae0f816dc77286e6e1fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/177121
Reviewed-by: Ben Shi <powerman1st@163.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-16 13:46:25 +00:00
Jason A. Donenfeld
12279faa72 os: pass correct environment when creating Windows processes
This is CVE-2019-11888.

Previously, passing a nil environment but a non-nil token would result
in the new potentially unprivileged process inheriting the parent
potentially privileged environment, or would result in the new
potentially privileged process inheriting the parent potentially
unprivileged environment. Either way, it's bad. In the former case, it's
an infoleak. In the latter case, it's a possible EoP, since things like
PATH could be overwritten.

Not specifying an environment currently means, "use the existing
environment". This commit amends the behavior to be, "use the existing
environment of the token the process is being created for." The behavior
therefore stays the same when creating processes without specifying a
token. And it does the correct thing when creating processes when
specifying a token.

Fixes #32000

Change-Id: Ia57f6e89b97bdbaf7274d6a89c1d9948b6d40ef5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176619
Run-TryBot: Jason Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2019-05-16 10:24:10 +00:00
David Chase
fba6066975 cmd/compile: add debugging and stats output to numberlines
This is useful for debugging line number assignment and
also for making sense of pathological line number inputs.

Activated with
-gcflags=-d=ssa/number_lines/stats=1 (the bit matters)
-gcflags=-d=ssa/number_lines/debug

Stats:
"SUM_LINE_RANGE",
   SUM for f in files {MAX line in f {line}-MIN line in f {line}}
"MAXMIN_LINE_RANGE",
   MAX for f in files {MAX line in f {line}} -
   MIN for f in files {MIN line in f {line}}
"MAXFILE", maxfile,
   MAX for f in files {f}
"NFILES", len(entries)
   | files |

Change-Id: I8a7336e6370452fe2e3a62de17606db9bd6a6fd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174947
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-05-16 03:26:42 +00:00
Russ Cox
f2694534cf testing: shorten go test -short testing
This cuts the time for 'go test -short testing' from 0.9s to < 0.1s.

Change-Id: Ib8402f80239e1e96ea5221dfd5cd0db08170d85b
Reviewed-on: https://go-review.googlesource.com/c/go/+/177420
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:25:08 +00:00
Russ Cox
6ab049b965 testing: panic on calls to Short/Verbose before Parse
CL 121936 added this diagnostic to avoid a panic accessing *short.
(Hence the "This shouldn't really be a panic" comment.)

That CL was right to produce a clearer error than a plain memory fault,
but I think wrong to print+exit instead of panicking. I just ran into
one of these in a real program, and there is no indication anywhere
of how the program reached this point. The panic will show that.
So change print+exit to a panic with a helpful message, in contrast
to the original panic with an unhelpful message and the current
helpful message without stack trace.

Change-Id: Ib2bae1dead4ccde92f00fa3a34c05241ff7690c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/177419
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-16 03:25:07 +00:00
Russ Cox
80b393e8b2 cmd/dist: say 'go test .' instead of 'go test' in a few places for cleaner output
This just makes all.bash a bit less chatty.

Change-Id: I7d2ecabf0c7d8df2065d7052718f611bb2907801
Reviewed-on: https://go-review.googlesource.com/c/go/+/177418
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-16 03:25:05 +00:00
Russ Cox
a6a0ed3c47 misc/cgo: disable testcarchive, testplugin during -short test (including all.bash)
These tests take 20 seconds each to rebuild the entire world
in their respective modes. That's too much for short mode,
including all.bash on developer machines.

Keep doing it on builders and if someone runs 'go test' by hand
in that directory.

For #26473.

Change-Id: I3dc6955bc3aa7a20fd170efcde72a7d19b37bdbf
Reviewed-on: https://go-review.googlesource.com/c/go/+/177417
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:25:04 +00:00
Russ Cox
018d9b42a7 runtime: disable some tests in -quick mode
Speeds up the "go test runtime -cpu=1,2,4 -short -quick" phase of all.bash.

For #26473.

Change-Id: I090f5a5aa754462b3253a2156dc31fa67ce7af2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/177399
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-16 03:25:03 +00:00
Russ Cox
dccd5da08e cmd/go: cut 'go test -short cmd/go' time by about half
Was 50 seconds on unloaded Mac laptop; now 27.
Still longer than I would like, but every little bit helps.

For #26473.

Change-Id: Id4be016ee1555cbc3512eca0ae10236d7f06bd02
Reviewed-on: https://go-review.googlesource.com/c/go/+/177398
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:25:01 +00:00
Russ Cox
0c47e24df8 cmd/go: rename renamed testing import
_go_testing is equally unlikely and much less concerning
than testing_xxxxxxxxxxxx if it appears in an error message
(as it does, for example, in https://storage.googleapis.com/go-build-log/0d543f89/linux-amd64_3467a10e.log).

Change-Id: I45dc3ebe2d3b6c9e53273cd21782ee11a53f5edb
Reviewed-on: https://go-review.googlesource.com/c/go/+/177197
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:25:00 +00:00
Russ Cox
54260c570f cmd/go: add more information to -debug-actiongraph
Add information about action and build IDs,
commands run, time spent in commands,
and overall time to -debug-actiongraph output.

Also avoid a terrible outcome in:

	go build -debug-actiongraph x.go

Change-Id: I7a3a638f4848d75f2bdc997517f4ab23656da4f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/177138
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-16 03:24:58 +00:00
Russ Cox
86463c157b cmd/vet/all: delete
The work of running full vet on std and cmd during local development
has moved to go test, which of course runs during all.bash.

For errors in other GOOS/GOARCH combinations, the misc-compile
builders (running buildall.bash) also now run go vet std cmd.

The vetall builder need not do anything anymore.
Make it a no-op until it can be retired, and remove
cmd/vet/all and its special case in the go command.

Fixes #31916.

Change-Id: I8f30d184c382ea7c2c8f520e5618f680db633968
Reviewed-on: https://go-review.googlesource.com/c/go/+/176440
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:24:57 +00:00
Russ Cox
4f76871bd7 cmd/go: run full 'go vet' during 'go test' for packages in GOROOT
Now that the main tree complies with 'go vet', enable all vet checks
during 'go test' in the main tree. This helps surface helpful errors
while developing, instead of having to wait for the misc-vet-vetall builder.

During 'go test', the additional vet checks are essentially free:
the vet invocations themselves take only 8 seconds total for the entire tree.

Also update buildall.bash (used by the misc-compile builders)
to run 'go vet std cmd' for each GOOS/GOARCH pair.
This is not as free, since in general it can require recompiling
packages with their tests included before invoking vet.
(That compilation was going on anyway in the 'go test' case.)

On my Mac laptop, ./buildall.bash freebsd used to take
68+16+17+18 = 119 seconds for make.bash and then
the builds of the three freebsd architectures.
Now it takes 68+16+23+17+23+18+24 = 189 seconds, 60% longer.
Some of this is spent doing unnecessary cgo work.
Still, this lets us shard the vet checks and match all.bash.

Fixes #20119.
For #31916.

Change-Id: I6b0c40bac47708a688463c7fca12c0fc23ab2751
Reviewed-on: https://go-review.googlesource.com/c/go/+/176439
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-16 03:24:56 +00:00
Russ Cox
639ac76d45 runtime: fix 'go vet -race runtime'
This updates the Go function declarations to match race_amd64.s.

Change-Id: I2b541a6b335ce732f4c31652aa615240ce7bb1c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/177397
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-16 03:24:54 +00:00
Damien Neil
14491a2ec4 fmt: support %w
When fmt.Errorf is provided with a %w verb with an error operand,
return an error implementing an Unwrap method returning that operand.

It is invalid to use %w with other formatting functions, to use %w
multiple times in a format string, or to use %w with a non-error
operand. When the Errorf format string contains an invalid use of %w,
the returned error does not implement Unwrap.

Change-Id: I534e20d3b163ab22c2b137b1c9095906dc243221
Reviewed-on: https://go-review.googlesource.com/c/go/+/176998
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2019-05-15 19:53:28 +00:00
Damien Neil
3e2c522d5c errors, fmt: revert rejected changes for Go 1.13
Reverts the following changes:

  https://go.googlesource.com/go/+/1f90d081391d4f5911960fd28d81d7ea5e554a8f
  https://go.googlesource.com/go/+/8bf18b56a47a98b9dd2fa03beb358312237a8c76
  https://go.googlesource.com/go/+/5402854c3557f87fa2741a52ffc15dfb1ef333cc
  https://go.googlesource.com/go/+/37f84817247d3b8e687a701ccb0d6bc7ffe3cb78
  https://go.googlesource.com/go/+/6be6f114e0d483a233101a67c9644cd72bd3ae7a

Partially reverts the followinng change, removing the errors.Opaque
function and the errors.Wrapper type definition:

  https://go.googlesource.com/go/+/62f5e8156ef56fa61e6af56f4ccc633bde1a9120

Updates documentation referencing the Wrapper type.

Change-Id: Ia622883e39cafb06809853e3fd90b21441124534
Reviewed-on: https://go-review.googlesource.com/c/go/+/176997
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
2019-05-15 19:53:15 +00:00
Filippo Valsorda
599ec7720f crypto/x509: add support for Ed25519 certificates and keys
Based on RFC 8410.

Updates #25355

Change-Id: If7abb7eeb0ede10a9bb3d2004f2116e587c6207a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175478
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
2019-05-15 18:51:12 +00:00
Joel Sing
ab242dcbc9 syscall: implement rawVforkSyscall for linux/ppc64x and linux/s390x
This allows the use of CLONE_VFORK and CLONE_VM for fork/exec, preventing
"fork/exec ...: cannot allocate memory" failures from occuring when attempting
to execute commands from a Go process that has a large memory footprint.
Additionally, this should reduce the latency of fork/exec on these platforms.

The same problem was addressed on linux/amd64 via issue #5838.

Updates #31936

Change-Id: I7ae0fbbeaa29cab944a49a11272a380d497eb2d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/175697
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-15 18:12:10 +00:00
Russ Cox
38431f1044 cmd/go: do not build test packages unnecessarily during go vet
Vet needs export data for the imports of the package it is analyzing.
Vet does not need export data for the package itself, since vet will
do its own type checking. Assuming that vet is just as good as the compiler
at detecting invalid programs, don't run the compiler unnecessarily.

This especially matters for tests without external test files or for
which the external test files do not import the test-augmented original
package. In that case, the test-augmented original package need not
be compiled at all.

Cuts time for 'go clean -cache && go vet -x cmd/compile/internal/ssa'
from 7.6r 24.3u 2.8s to 3.5r 8.5u 1.9s, by not running the compiler
on the augmented test package.

There is still more to be done here - if we do need to build a
test-augmented package, we rerun cgo unnecessarily.
But this is a big help.

Cuts time for 'go vet std cmd' by about 30%.

For #31916.

Change-Id: If6136b4d384f1da77aed90b43f1a6b95f09b5d86
Reviewed-on: https://go-review.googlesource.com/c/go/+/176438
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-15 17:02:43 +00:00
Russ Cox
ba23fa4833 cmd/vendor: go get -u golang.org/x/tools && go mod vendor
Picks up vet fix from CL 176357.

Change-Id: Ia77cd4a582c4edfbe59bbc311e6ce14046df0e83
Reviewed-on: https://go-review.googlesource.com/c/go/+/177137
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-15 17:02:39 +00:00
Sylvain Zimmer
8116599f01 regexp: optimize for provably too short inputs
For many patterns we can compute the minimum input length at compile time.
If the input is shorter, we can return early and get a huge speedup.

As pointed out by Damian Gryski, Perl's regex engine contains a number of
these kinds of fail-fast optimizations:
https://perldoc.perl.org/perlreguts.html#Peep-hole-Optimisation-and-Analysis

Benchmarks: (including new ones for compile time)

name               old time/op    new time/op    delta
Compile/Onepass-8    4.39µs ± 1%    4.40µs ± 0%  +0.34%  (p=0.029 n=9+8)
Compile/Medium-8     9.80µs ± 0%    9.91µs ± 0%  +1.17%  (p=0.000 n=10+10)
Compile/Hard-8       72.7µs ± 0%    73.5µs ± 0%  +1.10%  (p=0.000 n=9+10)

name                       old time/op    new time/op      delta
Match/Easy0/16-8             52.6ns ± 5%       4.9ns ± 0%     -90.68%  (p=0.000 n=10+9)
Match/Easy0/32-8             64.1ns ±10%      61.4ns ± 1%        ~     (p=0.188 n=10+9)
Match/Easy0/1K-8              280ns ± 1%       277ns ± 2%      -0.97%  (p=0.004 n=10+10)
Match/Easy0/32K-8            4.61µs ± 1%      4.55µs ± 1%      -1.49%  (p=0.000 n=9+10)
Match/Easy0/1M-8              229µs ± 0%       226µs ± 1%      -1.29%  (p=0.000 n=8+10)
Match/Easy0/32M-8            7.50ms ± 1%      7.47ms ± 1%        ~     (p=0.165 n=10+10)
Match/Easy0i/16-8             533ns ± 1%         5ns ± 2%     -99.07%  (p=0.000 n=10+10)
Match/Easy0i/32-8             950ns ± 0%       950ns ± 1%        ~     (p=0.920 n=10+9)
Match/Easy0i/1K-8            27.5µs ± 1%      27.5µs ± 0%        ~     (p=0.739 n=10+10)
Match/Easy0i/32K-8           1.13ms ± 0%      1.13ms ± 1%        ~     (p=0.079 n=9+10)
Match/Easy0i/1M-8            36.7ms ± 2%      36.1ms ± 0%      -1.64%  (p=0.000 n=10+9)
Match/Easy0i/32M-8            1.17s ± 0%       1.16s ± 1%      -0.80%  (p=0.004 n=8+9)
Match/Easy1/16-8             55.5ns ± 6%       4.9ns ± 1%     -91.19%  (p=0.000 n=10+9)
Match/Easy1/32-8             58.3ns ± 8%      56.6ns ± 1%        ~     (p=0.449 n=10+8)
Match/Easy1/1K-8              750ns ± 0%       748ns ± 1%        ~     (p=0.072 n=8+10)
Match/Easy1/32K-8            31.8µs ± 0%      31.6µs ± 1%      -0.50%  (p=0.035 n=10+9)
Match/Easy1/1M-8             1.10ms ± 1%      1.09ms ± 0%      -0.95%  (p=0.000 n=10+9)
Match/Easy1/32M-8            35.5ms ± 0%      35.2ms ± 1%      -1.05%  (p=0.000 n=9+10)
Match/Medium/16-8             442ns ± 2%         5ns ± 1%     -98.89%  (p=0.000 n=10+10)
Match/Medium/32-8             875ns ± 0%       878ns ± 1%        ~     (p=0.071 n=9+10)
Match/Medium/1K-8            26.1µs ± 0%      25.9µs ± 0%      -0.64%  (p=0.000 n=10+10)
Match/Medium/32K-8           1.09ms ± 1%      1.08ms ± 0%      -0.84%  (p=0.000 n=10+9)
Match/Medium/1M-8            34.9ms ± 0%      34.6ms ± 1%      -0.98%  (p=0.000 n=9+10)
Match/Medium/32M-8            1.12s ± 1%       1.11s ± 1%      -0.98%  (p=0.000 n=10+9)
Match/Hard/16-8               721ns ± 1%         5ns ± 0%     -99.32%  (p=0.000 n=10+9)
Match/Hard/32-8              1.32µs ± 1%      1.31µs ± 0%      -0.71%  (p=0.000 n=9+9)
Match/Hard/1K-8              39.8µs ± 1%      39.7µs ± 1%        ~     (p=0.165 n=10+10)
Match/Hard/32K-8             1.57ms ± 0%      1.56ms ± 0%      -0.70%  (p=0.000 n=10+9)
Match/Hard/1M-8              50.4ms ± 1%      50.1ms ± 1%      -0.57%  (p=0.007 n=10+10)
Match/Hard/32M-8              1.62s ± 1%       1.60s ± 0%      -0.98%  (p=0.000 n=10+10)
Match/Hard1/16-8             3.88µs ± 1%      3.86µs ± 0%        ~     (p=0.118 n=10+10)
Match/Hard1/32-8             7.44µs ± 1%      7.46µs ± 1%        ~     (p=0.109 n=10+10)
Match/Hard1/1K-8              232µs ± 1%       229µs ± 1%      -1.31%  (p=0.000 n=10+9)
Match/Hard1/32K-8            7.41ms ± 2%      7.41ms ± 0%        ~     (p=0.237 n=10+8)
Match/Hard1/1M-8              238ms ± 1%       238ms ± 0%        ~     (p=0.481 n=10+10)
Match/Hard1/32M-8             7.69s ± 1%       7.61s ± 0%      -1.00%  (p=0.000 n=10+10)

Fixes #31329

Change-Id: I04640e8c59178ec8b3106e13ace9b109b6bdbc25
Reviewed-on: https://go-review.googlesource.com/c/go/+/171023
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-15 15:28:22 +00:00
Baokun Lee
cefc0bbc90 cmd/go/internal/renameio: allow write file with the specified permissions
Now renameio package creates file use ioutil.TempFile, which calls
OpenFile with mode 0600, we should support creates a file with given
permission bits.

Fixes #31871

Change-Id: I0436e9f7081f2fce18bf9f3b14d55b02d4d995fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/175958
Run-TryBot: Baokun Lee <nototon@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-15 14:32:33 +00:00