1
0
mirror of https://github.com/golang/go synced 2024-11-14 15:10:54 -07:00
Commit Graph

42400 Commits

Author SHA1 Message Date
Ariel Mashraki
8d7be1e3c9 text/template/parse: remove redundant return
Change the `itemChar` clause to be like all other clauses
that don't return a different state function than the default.

Change-Id: I56c863a7d699c1264b24b42ef23138ec47eaacd8
Reviewed-on: https://go-review.googlesource.com/c/go/+/216117
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-23 23:07:10 +00:00
Filippo Valsorda
953bc8f391 crypto/x509: mitigate CVE-2020-0601 verification bypass on Windows
An attacker can trick the Windows system verifier to use a poisoned set
of elliptic curve parameters for a trusted root, allowing it to generate
spoofed signatures. When this happens, the returned chain will present
the unmodified original root, so the actual signatures won't verify (as
they are invalid for the correct parameters). Simply double check them
as a safety measure and mitigation.

Windows users should still install the system security patch ASAP.

This is the same mitigation adopted by Chromium:

https://chromium-review.googlesource.com/c/chromium/src/+/1994434

Change-Id: I2c734f6fb2cb51d906c7fd77034318ffeeb3e146
Reviewed-on: https://go-review.googlesource.com/c/go/+/215905
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ryan Sleevi <sleevi@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
2020-01-23 22:31:25 +00:00
Carlos Amedee
ace25f82df cmd/link: ensure cgo cflags do not leak into tvOS test
Running the 'TestBuildForTvOS' test with CGO_CFLAGS set
with certain values would cause the test to fail. all.bash
would fail when CGO_CFLAGS was set to '-mmacosx-version-min=10.10'
because the --macosx-version-min flag is incompatible with tvOS.
The change guards against using an unintended flag in the unit test.

Updates #35459

Change-Id: Ifc43f3ebfb23d37aabeaac2ea9efae5b877991bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/215957
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-23 21:01:12 +00:00
Kirill Tatchihin
8453844ed7 time: document how Parse handles two-digit years
Fixes #36549

Change-Id: Ia803330fc046d5807bbefd67acb419cb81640a13
GitHub-Last-Rev: bd35431908
GitHub-Pull-Request: golang/go#36584
Reviewed-on: https://go-review.googlesource.com/c/go/+/214980
Reviewed-by: Rob Pike <r@golang.org>
2020-01-23 20:44:44 +00:00
Bryan C. Mills
39ea3d0a23 cmd/go: add a control case to the mod_vendor_trimpath test
In reviewing CL 215940, it took me a while to find the control
condition for the test, which was located in build_cache_trimpath.txt.

We could consolidate the two tests, but since they check for
regressions of separate issues (with separate root-causes), I think it
makes sense to keep them separate.

However, I would like the control condition to be present in the same
source file, so that we'll be more likely to update both cases if the
behavior of one of them is changed.

Updates #36566

Change-Id: Ic588f1dfb7977dd78d1d5ef61b9841e22bad82e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/216018
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-01-23 19:11:41 +00:00
Daniel Martí
8689ea6a4a cmd/go: make Script/test_regexps less flaky under load
With the command below, I was able to reproduce failures within the
first 50 or so runs:

	go test -c -o test && stress -p 32 ./test -test.run Script/test_regexp

When printing the full failure output, we'd see:

	BenchmarkX
	    BenchmarkX: x_test.go:13: LOG: X running N=1
	BenchmarkX/Y
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=100
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=10000
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=100000000
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000000
	BenchmarkX/Y         	1000000000	         0.000050 ns/op
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=30
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1207
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=120700
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=12070000
	    BenchmarkX/Y: x_test.go:15: LOG: Y running N=1000000000
	BenchmarkX/Y         	1000000000	         0.000715 ns/op

In other words, the N values aren't required to be exact. It seems like
they are cut short if the machine is under stress. That's the exact
scenario we reproduce above, since I used -p=32 on my laptop with only 4
real CPU cores.

First, don't require each line to be present. Instead, use patterns
that span multiple lines, so that we can just match the first and last
N= lines.

Second, don't require the last N= lines to be exact; simply require
them to have a reasonably large number of digits.

Fixes #36664.

Change-Id: I7a9818f1a07099fa6482a26da2ac5cbea0f8ab30
Reviewed-on: https://go-review.googlesource.com/c/go/+/215578
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-23 18:03:09 +00:00
Jay Conrod
64378c233b cmd/go: unify trimpath logic for -mod=vendor, -mod=mod
If a package has a module with a version, the package's directory is
replaced with the module path and version, followed by the package's
path within the module.

This is a follow up to CL 214945. We no longer check whether the
module has a directory (with -mod=vendor, it does not).

Updates #36566

Change-Id: I5bc952b13bc7b4659f58ee555bd6c6a087eb7792
Reviewed-on: https://go-review.googlesource.com/c/go/+/215940
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-01-23 16:52:00 +00:00
Joel Sing
82a2f825b7 runtime: add missing code for linux/riscv64
Makes linux/riscv64 runtime buildable.

Updates #27532

Change-Id: I91bcadaaecb8ff3ffd70fcb437b2b6e4bbe11eda
Reviewed-on: https://go-review.googlesource.com/c/go/+/215839
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-23 14:36:57 +00:00
Joel Sing
9f07cdd99e cmd/internal/obj/riscv: restore the ADDI instruction in jalrToSym
While this instruction is not needed for the relocation (the lower immediate
can be patched directly into the JALR instruction), other code currently
depends on the jump sequence being 12 bytes (or three instructions) long.
Put the ADDI instruction back until these can be found and fixed.

Updates #27532

Change-Id: Idb73d716be8eb2eb796591b30f1ec4dc104f2bf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/215840
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-23 07:39:53 +00:00
Joel Sing
fa610c776e cmd/internal/obj/riscv: add missing instructions to the ternary expansion list
Updates #27532

Change-Id: I5beb7941c204755948350b181c713b046bc4f1f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/215841
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-23 07:33:33 +00:00
Jay Conrod
9f1d55c4e7 cmd/dist: print error if GOROOT is inside a module
Fixes #36701

Change-Id: I22738235e7a7ee06bc5d748213aab523aad8cf12
Reviewed-on: https://go-review.googlesource.com/c/go/+/215939
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Marwan Sulaiman <marwan.sameer@gmail.com>
2020-01-23 00:22:41 +00:00
Filippo Valsorda
f9fe783c6c Revert "net/http: support gzip, x-gzip Transfer-Encodings"
This reverts commit e6c12c3d02.

Reason for revert: the assumption that a T-E of "gzip" implies
"chunked" seems incorrect. The RFC does state that one "MUST apply
chunked as the final transfer coding" but that should be interpreted to
mean that a "chunked" encoding must be listed as the last one, not that
one should be assumed to be there if not. This is confirmed by the
alternative option to chunking on the server side being to "terminate
the message by closing the connection".

The issue seems confirmed by the fact that the code in the body of
#29162 fails with the following error:

    net/http: HTTP/1.x transport connection broken: http: failed to gunzip body: unexpected EOF

This late in the cycle, revert rather than fix, also because we don't
apparently have tests for the correct behavior.

Change-Id: I920ec928754cd8e96a06fb7ff8a53316c0f959e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/215757
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-01-22 22:06:30 +00:00
Austin Clements
9b5bd30716 runtime: document special memmove requirements
Unlike C's memmove, Go's memmove must be careful to do indivisible
writes of pointer values because it may be racing with the garbage
collector reading the heap.

We've had various bugs related to this over the years (#36101, #13160,
 #12552). Indeed, memmove is a great target for optimization and it's
easy to forget the special requirements of Go's memmove.

The CL documents these (currently unwritten!) requirements. We're also
adding a test that should hopefully keep everyone honest going
forward, though it's hard to be sure we're hitting all cases of
memmove.

Change-Id: I2f59f8d8d6fb42d2f10006b55d605b5efd8ddc24
Reviewed-on: https://go-review.googlesource.com/c/go/+/213418
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-22 18:54:48 +00:00
Ian Lance Taylor
895b7c85ad runtime: don't skip checkTimers if we would clear deleted timers
The timers code used to have a problem: if code started and stopped a
lot of timers, as would happen with, for example, lots of calls to
context.WithTimeout, then it would steadily use memory holding timers
that had stopped but not been removed from the timer heap.
That problem was fixed by CL 214299, which would remove all deleted
timers whenever they got to be more than 1/4 of the total number of
timers on the heap.

The timers code had a different problem: if there were some idle P's,
the running P's would have lock contention trying to steal their timers.
That problem was fixed by CL 214185, which only acquired the timer lock
if the next timer was ready to run or there were some timers to adjust.

Unfortunately, CL 214185 partially undid 214299, in that we could now
accumulate an increasing number of deleted timers while there were no
timers ready to run. This CL restores the 214299 behavior, by checking
whether there are lots of deleted timers without acquiring the lock.

This is a performance issue to consider for the 1.14 release.

Change-Id: I13c980efdcc2a46eb84882750c39e3f7c5b2e7c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/215722
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-22 18:10:42 +00:00
Joel Sing
fb2e3b82f4 cmd/dist: mark cgo as disabled on linux/riscv64
cgo is not currently supported on this platform.

Updates #27532 and #36641

Change-Id: I4b35f887e869ebc5c156dd754b1c79897a8c5800
Reviewed-on: https://go-review.googlesource.com/c/go/+/215838
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-22 16:57:26 +00:00
Joel Sing
b4f15e79af cmd/dist: detect gohostarch on riscv64 hosts
Updates #27532

Change-Id: I66c194499bb7b831b569c66d0736fa7205eedd80
Reviewed-on: https://go-review.googlesource.com/c/go/+/215837
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-22 16:56:57 +00:00
Joel Sing
6718cb617f cmd/internal/obj/riscv: implement prologue and epilogue
Based on riscv-go port.

Updates #27532

Change-Id: If552225552bf8d27c29b08de31146dd34986a3a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/204630
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-22 15:24:05 +00:00
Jay Conrod
71bbffbc48 cmd/go: add -d flag to mod_get_test
'go get all' was run in this test without -d. This caused some std
packages to be reinstalled if the test is run in a slightly different
configuration than make.bash was run. run.bash would fail in some
situations because of this. Nothing in the cmd/go tests should modify
installed std or cmd packages.

Updates #35459

Change-Id: Idd259a27d55502923b7fc54f361a77f0ac11eea2
Reviewed-on: https://go-review.googlesource.com/c/go/+/215721
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-01-22 00:45:41 +00:00
Joel Sing
fb8cdb7e4b cmd/internal/obj/riscv: populate DWARF register mapping for riscv64
Updates #27532

Change-Id: If147242c45d5c2d5cdc5b0428db32eeec13a958a
Reviewed-on: https://go-review.googlesource.com/c/go/+/215377
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-21 23:58:54 +00:00
Joel Sing
1424889921 cmd/internal/obj/riscv: correctly split immediates for FLW/FLD/FSW/FSD
The FLW/FLD/FSW/FSD instructions can have immediates that exceed 12-bits and
therefore cannot be encoded in the RISCV instruction. Handle these as we do
for other load/store instructions. Also add test coverage for all load/store
instructions with large immediates.

Fixes compilation issue reported by Carlos Eduardo de Paula.

Updates #27532

Change-Id: Ifa62f19493b3acaba5a90ac31d2df209a3afea81
Reviewed-on: https://go-review.googlesource.com/c/go/+/215037
Reviewed-by: Carlos Eduardo de Paula <me@carlosedp.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-21 23:58:09 +00:00
Kévin Dunglas
88ed322afc net/http/httputil: add docs about X-Forwarded-For in ReverseProxy
ReverseProxy automatically sets the X-Forwarded-For header, if the request
already contains a X-Forwarded-For header, the value of the client IP is
appended to the existing header value.
This behavior isn't documented anywhere, and can lead to IP spoofing
security issues is the client is untrusted (the most common situation).
This PR documents this behavior.

For future versions, I proposed #36678 that implements a more secure
default behavior and adds support for other forwarded headers.

Change-Id: Ief14f5063caebfccb87714f54cffa927c714e5fd
GitHub-Last-Rev: fd0bd29a18
GitHub-Pull-Request: golang/go#36672
Reviewed-on: https://go-review.googlesource.com/c/go/+/215617
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-21 21:51:46 +00:00
Carlos Eduardo Seo
71239b4f49 runtime: fix wrong offset when calling ppc64x nanotime syscall
There is a wrong offset when getting the results of a clock_gettime
syscall. Although the syscall will never be called in native ppc64x,
QEMU doesn't implement VDSO, so it will return wrong values.

Fixes #36592

Change-Id: Icf838075228dcdd62cf2c1279aa983e5993d66ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/215397
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-01-20 15:06:42 +00:00
Tobias Klauser
df2999ef43 test: disable test for #36516 when cgo is not enabled
CL 214679 added a -race test which shouldn't be run when cgo is not
enabled.

Fixes the nocgo builder.

Change-Id: Iceddf802c4ef6c0de2c3a968e86342303d2d27d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/215477
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-19 20:36:10 +00:00
Joel Sing
02816a2cad reflect: correct riscv64 assembly
R0 is not a thing in riscv64 assembly - use ZERO (rather than X0) since
the rest of this currently uses ABI names.

Updates #27532

Change-Id: I28fb68e9f80d05231a07c5921e7062777234e2c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/215437
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-19 15:28:32 +00:00
Joel Sing
8e0be05ec7 runtime: add support for linux/riscv64
Based on riscv-go port.

Updates #27532

Change-Id: If522807a382130be3c8d40f4b4c1131d1de7c9e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/204632
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-01-19 14:04:09 +00:00
Yasser Abdolmaleki
cbaa666682 net: document *OpError.Err must not be nil
The point of *net.OpError is to add details to an underlying lower
level error. It makes no sense to have an OpError without an Err and
a nil *OpError.Err will cause *OpError.Error() method to panic.

Fixes #33007

Change-Id: If4fb2501e02dad110a095b73e18c47312ffa6015
Reviewed-on: https://go-review.googlesource.com/c/go/+/187677
Reviewed-by: Rob Pike <r@golang.org>
2020-01-19 02:25:54 +00:00
Joel Sing
98d2717499 cmd/compile: implement compiler for riscv64
Based on riscv-go port.

Updates #27532

Change-Id: Ia329daa243db63ff334053b8807ea96b97ce3acf
Reviewed-on: https://go-review.googlesource.com/c/go/+/204631
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-01-18 14:41:40 +00:00
Joel Sing
91d75f4e4c reflect: add support for riscv64
Based on riscv-go port.

Update #27532

Change-Id: I791924f71078fd8dfe9c2fc03a3f21a3bc673721
Reviewed-on: https://go-review.googlesource.com/c/go/+/204634
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-01-18 13:45:50 +00:00
Roger Peppe
03eeaa9656 testing: clarify that Cleanup is run after subtests complete.
It's good to be explicit, as it's not necessarily obvious (and indeed
the behavior has changed recently with https://go-review.googlesource.com/c/go/+/214822)
without an associated doc comment change).

Change-Id: I99d6398bf15b404b1b1b196e712e926e363251e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/215217
Reviewed-by: Paul Jolly <paul@myitcv.org.uk>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-18 08:45:35 +00:00
David Chase
c112289ee4 cmd/compile: change the "bogus line" to be 1
The previous value was "too bogus" and caused objdump to crash.
Updated infinite loop test results (only run if -args -f) in ssa/debug_test.go
Probably also fixes #36621 but that bug needs more info to tell for certain.

Fixes #36570

Change-Id: I51144641d25d559308a98d726d87806bd340cc5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/215297
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-01-17 23:14:48 +00:00
Keith Randall
316fd8cc4a cmd/compile: mark ... argument to checkptrArithmetic as not escaping
Fixes #36516

Change-Id: Ibf4f86fb3a25fa30e0cd54e2dd2e12c60ee75ddb
Reviewed-on: https://go-review.googlesource.com/c/go/+/214679
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-01-17 17:38:40 +00:00
Jay Conrod
574c286607 cmd/go: trim paths from vendored packages with -trimpath
In CL 199821, we stopped setting the module directory for vendored
packages when -mod=vendor is used. This broke -trimpath, since we
replace the module directory with a string derived from the module
path and version. A comment in CL 202977 makes it clear that the
module directory should not be set.

With this change, -trimpath falls back to replacing the package
directory with the package path if the module directory is not set. We
also fall back to replacing the package directory if the module
version is not set to avoid adding a meaningless @ only for the main
module.

As a consequence of this change, file names in vendored packages will
not have module versions, so file names will be a little different
between -mod=mod and -mod=vendor.

Fixes #36566

Change-Id: I0e9cd76d36a2028a49d0b6697ea9a9b3140d7ff3
Reviewed-on: https://go-review.googlesource.com/c/go/+/214945
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-01-16 22:25:09 +00:00
Ian Lance Taylor
998cbe2983 testing: don't run Cleanup functions until parallel subtests complete
Fixes #31651

Change-Id: Idbab0c4355fcc58520e210126795223435cf0078
Reviewed-on: https://go-review.googlesource.com/c/go/+/214822
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-16 21:32:12 +00:00
Ian Lance Taylor
d2de9bd59c runtime: ignore power notification error seen on Windows Docker
Fixes #36557

Change-Id: Ia8125f382d5e14e5612da811268a58971cc9ac08
Reviewed-on: https://go-review.googlesource.com/c/go/+/214917
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-01-16 04:02:37 +00:00
Josh Bleecher Snyder
f77e7ed7e3 cmd/compile: use a sync.Pool and string interning when printing types
CL 214239 improved type printing, but introduced performance regressions:
3-5% memory increase, 1-2% CPU time increase.

There were two primary sources of the memory regression:

* allocating a bytes.Buffer for every type to print
* always printing to that buffer, even when we could return a constant string

This change addresses both of those regressions.
The sync.Pool allows buffer re-use.
String interning prevents allocation for re-used strings.

It addresses some, but not all, of the CPU time regression.


Memory performance impact vs master:

name        old alloc/op      new alloc/op      delta
Template         37.6MB ± 0%       36.3MB ± 0%  -3.30%  (p=0.008 n=5+5)
Unicode          28.7MB ± 0%       28.3MB ± 0%  -1.55%  (p=0.008 n=5+5)
GoTypes           127MB ± 0%        122MB ± 0%  -4.38%  (p=0.008 n=5+5)
Compiler          584MB ± 0%        568MB ± 0%  -2.72%  (p=0.008 n=5+5)
SSA              1.99GB ± 0%       1.95GB ± 0%  -1.97%  (p=0.008 n=5+5)
Flate            23.5MB ± 0%       22.8MB ± 0%  -2.84%  (p=0.008 n=5+5)
GoParser         29.2MB ± 0%       28.0MB ± 0%  -4.17%  (p=0.008 n=5+5)
Reflect          81.9MB ± 0%       78.6MB ± 0%  -4.09%  (p=0.008 n=5+5)
Tar              35.3MB ± 0%       34.1MB ± 0%  -3.29%  (p=0.008 n=5+5)
XML              45.5MB ± 0%       44.3MB ± 0%  -2.61%  (p=0.008 n=5+5)
[Geo mean]       82.4MB            79.9MB       -3.09%

name        old allocs/op     new allocs/op     delta
Template           394k ± 0%         363k ± 0%  -7.73%  (p=0.008 n=5+5)
Unicode            340k ± 0%         329k ± 0%  -3.25%  (p=0.008 n=5+5)
GoTypes           1.41M ± 0%        1.28M ± 0%  -9.54%  (p=0.008 n=5+5)
Compiler          5.77M ± 0%        5.39M ± 0%  -6.58%  (p=0.008 n=5+5)
SSA               19.1M ± 0%        18.1M ± 0%  -5.13%  (p=0.008 n=5+5)
Flate              247k ± 0%         228k ± 0%  -7.50%  (p=0.008 n=5+5)
GoParser           325k ± 0%         295k ± 0%  -9.24%  (p=0.008 n=5+5)
Reflect           1.04M ± 0%        0.95M ± 0%  -8.48%  (p=0.008 n=5+5)
Tar                365k ± 0%         336k ± 0%  -7.93%  (p=0.008 n=5+5)
XML                449k ± 0%         417k ± 0%  -7.10%  (p=0.008 n=5+5)
[Geo mean]         882k              818k       -7.26%


Memory performance going from 52c4488471,
which is the commit preceding CL 214239, to this change:

name        old alloc/op      new alloc/op      delta
Template         36.5MB ± 0%       36.3MB ± 0%  -0.37%  (p=0.008 n=5+5)
Unicode          28.3MB ± 0%       28.3MB ± 0%  -0.06%  (p=0.008 n=5+5)
GoTypes           123MB ± 0%        122MB ± 0%  -0.64%  (p=0.008 n=5+5)
Compiler          571MB ± 0%        568MB ± 0%  -0.51%  (p=0.008 n=5+5)
SSA              1.96GB ± 0%       1.95GB ± 0%  -0.13%  (p=0.008 n=5+5)
Flate            22.8MB ± 0%       22.8MB ± 0%    ~     (p=0.421 n=5+5)
GoParser         28.1MB ± 0%       28.0MB ± 0%  -0.37%  (p=0.008 n=5+5)
Reflect          78.8MB ± 0%       78.6MB ± 0%  -0.32%  (p=0.008 n=5+5)
Tar              34.3MB ± 0%       34.1MB ± 0%  -0.35%  (p=0.008 n=5+5)
XML              44.3MB ± 0%       44.3MB ± 0%  +0.05%  (p=0.032 n=5+5)
[Geo mean]       80.1MB            79.9MB       -0.27%

name        old allocs/op     new allocs/op     delta
Template           372k ± 0%         363k ± 0%  -2.46%  (p=0.008 n=5+5)
Unicode            333k ± 0%         329k ± 0%  -0.97%  (p=0.008 n=5+5)
GoTypes           1.33M ± 0%        1.28M ± 0%  -3.71%  (p=0.008 n=5+5)
Compiler          5.53M ± 0%        5.39M ± 0%  -2.50%  (p=0.008 n=5+5)
SSA               18.3M ± 0%        18.1M ± 0%  -1.22%  (p=0.008 n=5+5)
Flate              234k ± 0%         228k ± 0%  -2.44%  (p=0.008 n=5+5)
GoParser           305k ± 0%         295k ± 0%  -3.23%  (p=0.008 n=5+5)
Reflect            980k ± 0%         949k ± 0%  -3.12%  (p=0.008 n=5+5)
Tar                345k ± 0%         336k ± 0%  -2.69%  (p=0.008 n=5+5)
XML                425k ± 0%         417k ± 0%  -1.72%  (p=0.008 n=5+5)
[Geo mean]         838k              818k       -2.41%


Remaining CPU time regression, that is,
the change from before CL 214239 to this change:

name        old time/op       new time/op       delta
Template          208ms ± 2%        209ms ± 1%    ~     (p=0.181 n=47+46)
Unicode          82.9ms ± 2%       81.9ms ± 2%  -1.25%  (p=0.000 n=50+48)
GoTypes           709ms ± 3%        714ms ± 3%  +0.77%  (p=0.003 n=48+49)
Compiler          3.31s ± 2%        3.32s ± 2%    ~     (p=0.271 n=48+48)
SSA               10.8s ± 1%        10.9s ± 1%  +0.61%  (p=0.000 n=46+47)
Flate             134ms ± 2%        134ms ± 1%  +0.41%  (p=0.002 n=48+46)
GoParser          166ms ± 2%        167ms ± 2%  +0.41%  (p=0.010 n=46+48)
Reflect           440ms ± 4%        444ms ± 4%  +1.05%  (p=0.002 n=50+49)
Tar               183ms ± 2%        184ms ± 2%    ~     (p=0.074 n=45+45)
XML               247ms ± 2%        248ms ± 2%  +0.67%  (p=0.001 n=49+48)
[Geo mean]        425ms             427ms       +0.34%

name        old user-time/op  new user-time/op  delta
Template          271ms ± 2%        271ms ± 2%    ~     (p=0.654 n=48+48)
Unicode           117ms ± 2%        116ms ± 3%    ~     (p=0.458 n=47+45)
GoTypes           952ms ± 3%        963ms ± 2%  +1.11%  (p=0.000 n=48+49)
Compiler          4.50s ± 5%        4.49s ± 7%    ~     (p=0.894 n=50+50)
SSA               15.0s ± 2%        15.1s ± 2%  +0.46%  (p=0.015 n=50+49)
Flate             166ms ± 2%        167ms ± 2%  +0.40%  (p=0.005 n=49+48)
GoParser          202ms ± 2%        203ms ± 2%  +0.60%  (p=0.002 n=49+47)
Reflect           583ms ± 3%        588ms ± 3%  +0.82%  (p=0.001 n=49+46)
Tar               223ms ± 2%        224ms ± 2%  +0.37%  (p=0.046 n=48+46)
XML               310ms ± 2%        311ms ± 2%  +0.46%  (p=0.009 n=50+49)
[Geo mean]        554ms             556ms       +0.36%


Change-Id: I85951a6538373ef4309a2cc366cc1ebaf1f4582d
Reviewed-on: https://go-review.googlesource.com/c/go/+/214818
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-01-15 22:02:44 +00:00
Brad Fitzpatrick
5d9f1c847f crypto/tls: stop a timeout timer
I noticed this leak while writing CL 214977.

Change-Id: I7566952b8e4bc58939d23435aea86576fc58ddca
Reviewed-on: https://go-review.googlesource.com/c/go/+/214978
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-15 20:29:46 +00:00
Joel Sing
5a3a5d3525 math, math/big: add support for riscv64
Based on riscv-go port.

Updates #27532

Change-Id: Id8ae7d851c393ec3702e4176c363accb0a42587f
Reviewed-on: https://go-review.googlesource.com/c/go/+/204633
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-15 18:49:52 +00:00
Tobias Klauser
3743d21270 runtime: re-enable TestArenaCollision on darwin in race mode
Go 1.14 will drop support for macOS 10.10, see #23011

This reverts CL 155097

Updates #26475
Updates #29340

Change-Id: I64d0275141407313b73068436ee81d13eacc4c76
Reviewed-on: https://go-review.googlesource.com/c/go/+/214058
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-15 17:53:43 +00:00
yah01
ee55dd6b64 spec: add missing space in EBNF
Fixes #36520

Change-Id: I698ab235f82f7c81caa09318c954847cf3833153
GitHub-Last-Rev: 368a1dc788
GitHub-Pull-Request: golang/go#36559
Reviewed-on: https://go-review.googlesource.com/c/go/+/214821
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-01-15 05:32:51 +00:00
Thomas Symborski
a52db6403c strings: update Join parameter name for clarity
Change-Id: I83f806e76ef4d268b187bd273d78ceb41b7e8fa5
GitHub-Last-Rev: ee82eaae64
GitHub-Pull-Request: golang/go#36194
Reviewed-on: https://go-review.googlesource.com/c/go/+/211799
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-01-15 04:21:28 +00:00
Rob Pike
cae9a9fd65 doc: fix up some HTML issues in go_spec.html
The HTML linter 'tidy' reports:

	go_spec.html:2556: Warning: unescaped & which should be written as &amp;
	go_spec.html:3293: Warning: unescaped & or unknown entity "&s1"
	go_spec.html:3293: Warning: unescaped & or unknown entity "&a"
	go_spec.html:3294: Warning: unescaped & or unknown entity "&s2"
	go_spec.html:3294: Warning: unescaped & or unknown entity "&a"
	go_spec.html:2045: Warning: trimming empty <p>
	go_spec.html:4526: Warning: trimming empty <ul>
	go_spec.html:4533: Warning: trimming empty <ul>
	go_spec.html:4539: Warning: trimming empty <ul>

This CL fixes all but the <ul> ones, which I think should be fixed
but are defended by a comment.

Change-Id: I0ca88f5e80755024801877ab1298025ecf8f10c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/214457
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-01-14 22:24:11 +00:00
Ian Lance Taylor
cfe3cd903f runtime: keep P's first timer when in new atomically accessed field
This reduces lock contention when only a few P's are running and
checking for whether they need to run timers on the sleeping P's.
Without this change the running P's would get lock contention
while looking at the sleeping P's timers. With this change a single
atomic load suffices to determine whether there are any ready timers.

Change-Id: Ie843782bd56df49867a01ecf19c47498ec827452
Reviewed-on: https://go-review.googlesource.com/c/go/+/214185
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2020-01-14 19:54:20 +00:00
Michael Anthony Knyszek
71154e061f runtime: better approximate total cost of scavenging
Currently, the scavenger is paced according to how long it takes to
scavenge one runtime page's worth of memory. However, this pacing
doesn't take into account the additional cost of actually using a
scavenged page. This operation, "sysUsed," is a counterpart to the
scavenging operation "sysUnused." On most systems this operation is a
no-op, but on some systems like Darwin and Windows we actually make a
syscall. Even on systems where it's a no-op, the cost is implicit: a
more expensive page fault when re-using the page.

On Darwin in particular the cost of "sysUnused" is fairly close to the
cost of "sysUsed", which skews the pacing to be too fast. A lot of
soon-to-be-allocated memory ends up scavenged, resulting in many more
expensive "sysUsed" operations, ultimately slowing down the application.

The way to fix this problem is to include the future cost of "sysUsed"
on a page in the scavenging cost. However, measuring the "sysUsed" cost
directly (like we do with "sysUnused") on most systems is infeasible
because we would have to measure the cost of the first access.

Instead, this change applies a multiplicative constant to the measured
scavenging time which is based on a per-system ratio of "sysUnused" to
"sysUsed" costs in the worst case (on systems where it's a no-op, we
measure the cost of the first access). This ultimately slows down the
scavenger to a more reasonable pace, limiting its impact on performance
but still retaining the memory footprint improvements from the previous
release.

Fixes #36507.

Change-Id: I050659cd8cdfa5a32f5cc0b56622716ea0fa5407
Reviewed-on: https://go-review.googlesource.com/c/go/+/214517
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-01-14 17:13:34 +00:00
Koya IWAMURA
0c65531ef1 C: add missing name
Even though I contributed to golang/go[^1], the rule[^2] did not reflect my name in CONTRIBUTORS, so I added it.

[^1]: https://go-review.googlesource.com/c/go/+/164199
[^2]: https://github.com/golang/build/blob/e21a90b/cmd/updatecontrib/updatecontrib.go#L118

Change-Id: I5d6b5684d61ea5da679519a9e703d977470de175
Reviewed-on: https://go-review.googlesource.com/c/go/+/214617
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-01-14 15:51:35 +00:00
Ian Lance Taylor
563287ae06 Revert "os: handle long path in RemoveAll for windows"
This reverts CL 214437.

Does not fix the issue, and the test was wrong so it did not detect that it did not fix the issue.

Updates #36375

Change-Id: I6a4112035a1e90f4fdafed6fdf4ec9dfc718b571
Reviewed-on: https://go-review.googlesource.com/c/go/+/214601
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-13 23:35:18 +00:00
Rob Pike
4ead7e8cc4 time: add a sentence about distant times to time.Time.Unix
Since Durations only span 290 years, they are not good for
manipulating very remote times. I bounced off this problem recently
while doing some astronomical calculations and it took me a while to
realize I could get a 64-bit seconds value from time.Time.Unix and
subtract two of them to evaluate the interval.

I thought it worth adding a sentence to make this clear. It didn't
occur to me for quite a while that "Unix time" spans a huge range in
the Go library.

Change-Id: I76c75dc951dfd6bcf86e8b0be3cfec518a3ecdee
Reviewed-on: https://go-review.googlesource.com/c/go/+/213977
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
2020-01-13 21:56:50 +00:00
Robert Griesemer
9ead772a92 spec: uniformly format empty interfaces as "interface{}"
Fixes #36526.

Change-Id: Ic51a287579f139422cc1a7b2fb82d6732114b031
Reviewed-on: https://go-review.googlesource.com/c/go/+/214597
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-13 21:49:46 +00:00
Constantin Konstantinidis
5c44cc47c6 os: handle long path in RemoveAll for windows
Fixes #36375

Change-Id: I407a1db23868880b83e73bc136d274659483fb69
Reviewed-on: https://go-review.googlesource.com/c/go/+/214437
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-13 20:42:57 +00:00
Keith Randall
5d8a61a43e cmd/compile: print recursive types correctly
Change the type printer to take a map of types that we're currently
printing. When we happen upon a type that we're already in the middle
of printing, print a reference to it instead.

A reference to another type is built using the offset of the first
byte of that type's string representation in the result. To facilitate
that computation (and it's probably more efficient, regardless), we
print the type to a buffer as we go, and build the string at the end.

It would be nice to use string.Builder instead of bytes.Buffer, but
string.Builder wasn't around in Go 1.4, and we'd like to bootstrap
from that version.

Fixes #29312

Change-Id: I49d788c1fa20f770df7b2bae3b9979d990d54803
Reviewed-on: https://go-review.googlesource.com/c/go/+/214239
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-01-13 18:52:18 +00:00
fujimoto kyosuke
52c4488471 html: update URL in comment
The comment contained a link that had a file name and ID that no longer existed, so change to the URL of the corresponding part of the latest page.

Change-Id: I74e0885aabf470facc39b84035f7a83fef9c6a8e
GitHub-Last-Rev: 5681c84d9f
GitHub-Pull-Request: golang/go#36514
Reviewed-on: https://go-review.googlesource.com/c/go/+/214181
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-01-13 07:00:18 +00:00