1
0
mirror of https://github.com/golang/go synced 2024-09-29 20:34:36 -06:00
Commit Graph

40280 Commits

Author SHA1 Message Date
Tobias Klauser
afd79150d9 os: fix typo in Chmod godoc
Change-Id: I3e5c20d2ffbbe604e6c8b21e2afa50dd6c9f2b7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/176626
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-13 13:53:07 +00:00
Iskander Sharipov
2e4edf4697 cmd/internal/obj/x86: add oclass function tests
To make refactoring/optimizations easier in future.

Change-Id: I158f52af7c72849df1d6bd9334b3ce9dd199318f
Reviewed-on: https://go-review.googlesource.com/c/go/+/175357
Run-TryBot: Iskander Sharipov <quasilyte@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-05-12 07:14:09 +00:00
Joel Sing
83f205fa88 cmd/link: add support for R_AARCH64_LDST128_ABS_LO12_NC relocations
These are encountered when compiling with -linkmode=internal on openbsd/arm64.

Fixes #31940

Change-Id: I851e6a7da0a3fb3e23b4fa2ed8dce3051c680f11
Reviewed-on: https://go-review.googlesource.com/c/go/+/176697
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-12 02:55:19 +00:00
Tamir Duberstein
09267010ef net: comment duplicate constant
Change-Id: If5a4d8eff4e51d72fb9dc1d5db2bfe674ec5753b
Reviewed-on: https://go-review.googlesource.com/c/go/+/176717
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-11 23:19:40 +00:00
Cherry Zhang
ce5ae2f4bc test: use a real use function in nilptr2.go
Adjust the dummy use function to a real use. As suggested by the
println calls in the test, nilptr2.go supposes to check that a
used nil pointer dereference panics. This use function is not
real enough so an optimized compiler such as gccgo could
eliminate the call.

The spec requires that even a dummy use would cause a panic.
Unfortunately, due to #31151 this is not true for gccgo at -O1 or
above.

Change-Id: Ie07c8a5969ab94dad82d4f7cfec30597c25b7c46
Reviewed-on: https://go-review.googlesource.com/c/go/+/176579
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-11 03:02:33 +00:00
Josh Bleecher Snyder
f61cf6f313 cmd/compile: reduce rulelog noise
When the SSA rules are re-generated to log rules,
they write output like:

rewrite AMD64.rules:527
rewrite AMD64.rules:427
rewrite AMD64.rules:494

This is silly; there are no non-rewrite lines in the file.
Furthermore, the rulelog file tends to be gigantic
for any non-trivial compilation (measured in gigabytes).

Remove the "rewrite " prefix.

No impact to normal builds.

Change-Id: I955995c1cc5f27a4a6a3849e19082ecb3e40bd4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/176677
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-10 23:47:49 +00:00
Jay Conrod
309ab1ecc8 cmd/go: add generated code comment to _testmain.go template
Fixes #31971

Change-Id: I127659be145e348fae20930615666d67dc7971ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/176468
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-10 21:07:01 +00:00
Caleb Spare
49a1a01bb1 cmd/go: move automatic testing.Init call into generated test code
In CL 173722, we moved the flag registration in the testing package into
an Init function. In order to avoid needing changes to user code, we
called Init automatically as part of testing.MainStart.

However, that isn't early enough if flag.Parse is called before the
tests run, as part of package initialization.

Fix this by injecting a bit of code to call testing.Init into test
packages. This runs before any other initialization code in the user's
test package, so testing.Init will be called before any user code can
call flag.Parse.

Fixes #31859

Change-Id: Ib42cd8d3819150c49a3cecf7eef2472319d0c7e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/176098
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-10 18:10:48 +00:00
Yury Smolsky
5833aa507b cmd/vendor: import vet fixes from x/tools
Vet help prints only to stdout.

Fixes #31885

Change-Id: If6089a371fa8e21828eba2e23cddd2d19fb69e8a
Reviewed-on: https://go-review.googlesource.com/c/go/+/176617
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-10 18:00:52 +00:00
Jay Conrod
3b8c804164 cmd/go: force -coverpkg main packages to be built as libraries
This fixes TestScript/cover_pkgall_multiple_mains, which started
failing after CL 174657.

When compiling main packages with coverage instrumentation
(e.g., for -coverpkg all), we now pass -p with the full import path
instead of '-p main'. This avoids link errors
'duplicate symbol main.main (types 1 and 1)'.

Fixes #31946

Change-Id: Id147527b1dbdc14bb33ac133c30d50c250b4365c
Reviewed-on: https://go-review.googlesource.com/c/go/+/176558
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-10 16:28:38 +00:00
Russ Cox
ef4666ecce runtime: fix windows-amd64-2012 build
I forgot that in Go assembly, x+16(SP) is not the same as 16(SP).
The former is the virtual stack pointer (one word below FP on x86)
while the latter is the actual stack pointer.

Change-Id: Ibb7012bb97261949f5e1a0dc70869d9a6f50aa99
Reviewed-on: https://go-review.googlesource.com/c/go/+/176557
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-10 16:16:30 +00:00
Bryan C. Mills
1d1ff46966 cmd/go/testdata/script: remove skips and clarify comments in mod_*_upgrade_pseudo tests
These tests were added in CL 174206.

They required a 'git' binary and network access in an earlier draft,
but now use the test-local module proxy instead, so no longer need to
be skipped when those resources are not present.

Updates #30634

Change-Id: I5f36c6c776209a89bc45d133847df5052b55da59
Reviewed-on: https://go-review.googlesource.com/c/go/+/176537
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-10 15:38:01 +00:00
Jay Conrod
2aa8971910 cmd/go: move two vcs test repos to vcs-test.golang.org
Follow-up to CL 174061. This also fixes a break after GOSUMDB
was introduced.

Updates #31946
Updates #31673
Updates #31287
Updates #27171

Change-Id: I8e91e857f301b6b73cc90f2f2c68523412e22b46
Reviewed-on: https://go-review.googlesource.com/c/go/+/176417
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-10 14:24:43 +00:00
Josh Bleecher Snyder
4ae31dc8c5 cmd/compile: re-use regalloc's []valState
Updates #27739: reduces package ssa's allocated space by 3.77%.

maxrss is harder to measure, but using best-of-three-runs
as reported by /usr/bin/time -l, I see ~2% reduction in maxrss.

We still have a long way to go, though; the new maxrss is still 1.1gb.

name        old alloc/op      new alloc/op      delta
Template         38.8MB ± 0%       37.7MB ± 0%  -2.77%  (p=0.008 n=5+5)
Unicode          28.2MB ± 0%       28.1MB ± 0%  -0.20%  (p=0.008 n=5+5)
GoTypes           131MB ± 0%        127MB ± 0%  -2.94%  (p=0.008 n=5+5)
Compiler          606MB ± 0%        587MB ± 0%  -3.21%  (p=0.008 n=5+5)
SSA              2.14GB ± 0%       2.06GB ± 0%  -3.77%  (p=0.008 n=5+5)
Flate            24.0MB ± 0%       23.3MB ± 0%  -3.00%  (p=0.008 n=5+5)
GoParser         28.8MB ± 0%       28.1MB ± 0%  -2.61%  (p=0.008 n=5+5)
Reflect          83.8MB ± 0%       81.5MB ± 0%  -2.71%  (p=0.008 n=5+5)
Tar              36.4MB ± 0%       35.4MB ± 0%  -2.73%  (p=0.008 n=5+5)
XML              47.9MB ± 0%       46.7MB ± 0%  -2.49%  (p=0.008 n=5+5)
[Geo mean]       84.6MB            82.4MB       -2.65%

name        old allocs/op     new allocs/op     delta
Template           379k ± 0%         379k ± 0%  -0.05%  (p=0.008 n=5+5)
Unicode            340k ± 0%         340k ± 0%    ~     (p=0.151 n=5+5)
GoTypes           1.36M ± 0%        1.36M ± 0%  -0.06%  (p=0.008 n=5+5)
Compiler          5.49M ± 0%        5.48M ± 0%  -0.03%  (p=0.008 n=5+5)
SSA               17.5M ± 0%        17.5M ± 0%  -0.03%  (p=0.008 n=5+5)
Flate              235k ± 0%         235k ± 0%  -0.04%  (p=0.008 n=5+5)
GoParser           302k ± 0%         302k ± 0%  -0.04%  (p=0.008 n=5+5)
Reflect            976k ± 0%         975k ± 0%  -0.10%  (p=0.008 n=5+5)
Tar                352k ± 0%         352k ± 0%  -0.06%  (p=0.008 n=5+5)
XML                436k ± 0%         436k ± 0%  -0.03%  (p=0.008 n=5+5)
[Geo mean]         842k              841k       -0.04%

Change-Id: I0ab6631b5a0bb6303c291dcb0367b586a4e584fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/176221
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-10 00:14:40 +00:00
Russ Cox
40657c282d runtime: fix s390x build
The new prototypes of duffzero and duffcopy must be
accompanied by functions. Otherwise buildmode=shared
(in particular, misc/cgo/testshared) has missing symbols.

The right fix, of course, is to implement these on s390x.

For #31916.

Change-Id: I3efff5e3011956341e1b26223a4847a8a91a0453
Reviewed-on: https://go-review.googlesource.com/c/go/+/176397
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 23:41:02 +00:00
Brad Fitzpatrick
b8d5150f4a os/user: make Current return better error w/o cgo & complete environment
Fixes #31949

Change-Id: Ib96a43e4c56a00c5ba04e4d213255a063058ae08
Reviewed-on: https://go-review.googlesource.com/c/go/+/176337
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-05-09 21:47:43 +00:00
Russ Cox
a4f5c9c40d runtime: fix vet complaints for js/wasm
Change-Id: Ifc8a731a2efd94fdc4fc6f26ca6e16f0c0292211
Reviewed-on: https://go-review.googlesource.com/c/go/+/176178
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 21:13:56 +00:00
Russ Cox
56c1cf315d runtime: fix vet complaints for all nacl
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "go vet -unsafeptr=false runtime" happy for nacl/*,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: I6adb4a7b0c2b03d901fba37f9c05e74e5b7b6691
Reviewed-on: https://go-review.googlesource.com/c/go/+/176107
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 21:13:46 +00:00
Russ Cox
13723d44a0 runtime, crypto/x509: fix vet complaints for all windows
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "go vet -unsafeptr=false runtime" happy for windows/*,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: If37ab2b3f6fca4696b8a6afb2ef11ba6c4fb42e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/176106
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 21:13:39 +00:00
Russ Cox
924c161de4 runtime: fix vet complaints for all freebsd, netbsd, openbsd
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "go vet -unsafeptr=false runtime" happy for these GOOSes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: I63c4805bdd44b301072da66c77086940e2a2765e
Reviewed-on: https://go-review.googlesource.com/c/go/+/176105
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 21:13:26 +00:00
Russ Cox
b4a472b1c3 syscall: fix vet complaints for all dragonfly, freebsd, netbsd, openbsd
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "go vet -unsafeptr=false runtime" happy for these GOOSes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: Id2e1223497bd0cd6e880cd81f3ece6363e58219f
Reviewed-on: https://go-review.googlesource.com/c/go/+/176104
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 21:13:25 +00:00
Russ Cox
60f67631a6 runtime: fix vet complaints for solaris/amd64, illumos/amd64
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: Ic64f7f4034695dd4c32c9b7f258960faf3742a83
Reviewed-on: https://go-review.googlesource.com/c/go/+/176103
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 21:13:23 +00:00
Russ Cox
49f62af790 runtime: fix vet complaints for linux/arm64, linux/mips*, linux/ppc64*, linux/s390x
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "go vet -unsafeptr=false runtime" happy for these GOOS/GOARCHes,
except for an unresolved complaint on mips/mipsle that is a bug in vet,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: I6ef7e982a2fdbbfbc22cee876ca37ac54d8109e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176102
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 21:13:22 +00:00
Russ Cox
1ea76443ba runtime: fix vet complaints for linux/arm
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "GOOS=linux GOARCH=arm go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: Ifae75b832320b5356ac8773cf85055bfb2bd7214
Reviewed-on: https://go-review.googlesource.com/c/go/+/176101
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 21:13:21 +00:00
Russ Cox
fe67ce2efb runtime: fix vet complaints for linux/amd64
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "GOOS=linux GOARCH=amd64 go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: I4ca1acb02f4666b102d25fcc55fac96b8f80379a
Reviewed-on: https://go-review.googlesource.com/c/go/+/176100
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 21:13:19 +00:00
Russ Cox
6ed2ec4aa5 runtime: fix vet complaints for linux/386
Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

This CL makes "GOOS=linux GOARCH=386 go vet -unsafeptr=false runtime" happy,
while keeping "GO_BUILDER_NAME=misc-vetall go tool dist test" happy too.

For #31916.

Change-Id: I3e5586a7ff6e359357350d0602c2259493280ded
Reviewed-on: https://go-review.googlesource.com/c/go/+/176099
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 21:13:18 +00:00
Michael Anthony Knyszek
cd03664f82 runtime: fix js/wasm lock implementation
Trybots started failing on js/wasm after golang.org/cl/175797 landed,
but it seemed completely unrelated. It would fail very consistently on
the heapsampling.go test.

Digging deeper it was very difficult to ascertain what was going wrong,
but clearly m.locks for some m was non-zero when calling into the
scheduler.

The failure comes from the fact that lock calls into gosched, but it's
unclear how exactly we got there in the first place; there should be no
preemption in this single-threaded context.

Furthermore, lock shouldn't be calling gosched_m at all because in a
single-threaded context, the thread shouldn't be preempted until it
actually unlocks.

But, digging further it turns out the implementation in lock_js.go never
incremented or decremented m.locks. This is definitely wrong because
many parts of the runtime depend on that value being set correctly.

So, this change removes the loop which calls into gosched_m (which
should be unnecessary) and increments and decrements m.locks
appropriately. This appears to fix the aforementioned failure.

Change-Id: Id214c0762c3fb2b405ff55543d7e2a78c17443c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/176297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-09 20:22:27 +00:00
Russ Cox
0a2f72b889 cmd/link: use standard-syntax struct tags in large-tag test
These ridiculous tags are testing what happens with very long tags.
There is no need for them to be malformed as well.
Fix them to make vet happier.

Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

For #31916.

Change-Id: I100aed5230fcde41676c79c7074c69c16ea4b96d
Reviewed-on: https://go-review.googlesource.com/c/go/+/176111
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 18:03:17 +00:00
Russ Cox
c514071988 encoding/gob: rename encBuffer.WriteByte to writeByte
Renaming the method makes clear, both to readers and to vet,
that this method is not the implementation of io.ByteWriter.

Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

For #31916.

Change-Id: I5b509eb7f0118d5f2d3c6e352ff2849cd5a3071e
Reviewed-on: https://go-review.googlesource.com/c/go/+/176110
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 18:02:19 +00:00
Russ Cox
50a1d89ab2 fmt: rename buffer.WriteByte to writeByte
Renaming the method makes clear, both to readers and to vet,
that this method is not the implementation of io.ByteWriter.

Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

For #31916.

Change-Id: I79da062ca6469b62a6b9e284c6cf2413c7425249
Reviewed-on: https://go-review.googlesource.com/c/go/+/176109
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-09 17:49:12 +00:00
Jay Conrod
a44c3edbe0 all: document vendoring in the standard library
Added documentation that explains special cases for vendored packages
in the standard library and provides instructions for updating vendor
directories.

Fixes #31806

Change-Id: Ib697ed18eae28023ab0bfb9f4d250992c393571d
Reviewed-on: https://go-review.googlesource.com/c/go/+/174999
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-09 17:11:16 +00:00
Russ Cox
ffd7eba20a cmd/internal/bio: rename Reader.Seek to MustSeek
Renaming the method makes clear, both to readers and to vet,
that this method is not the implementation of io.Seeker:
it cannot fail.

Working toward making the tree vet-safe instead of having
so many exceptions in cmd/vet/all/whitelist.

For #31916.

Change-Id: I3e6ad7264cb0121b4b76935450cccb71d533e96b
Reviewed-on: https://go-review.googlesource.com/c/go/+/176108
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 16:56:31 +00:00
Michael Anthony Knyszek
ff70494b75 runtime: split spans when scavenging if it's more than we need
This change makes it so that during scavenging we split spans when the
span we have next for scavenging is larger than the amount of work we
have left to do.

The purpose of this change is to improve the worst-case behavior of the
scavenger: currently, if the scavenger only has a little bit of work to
do but sees a very large free span, it will scavenge the whole thing,
spending a lot of time to get way ahead of the scavenge pacing for no
reason.

With this change the scavenger should follow the pacing more closely,
but may still over-scavenge by up to a physical huge page since the
splitting behavior avoids breaking up huge pages in free spans.

This change is also the culmination of the scavenging improvements, so
we also include benchmark results for this series (starting from
"runtime: merge all treaps into one implementation" until this patch).

This patch stack results in average and peak RSS reductions (up to 11%
and 7% respectively) for some benchmarks, with mostly minimal
performance degredation (3-4% for some benchmarks, ~0% geomean). Each of
these benchmarks was executed with GODEBUG=madvdontneed=1 on Linux; the
performance degredation is even smaller when MADV_FREE may be used, but
the impact on RSS is much harder to measure. Applications that generally
maintain a steady heap size for the most part show no change in
application performance.

These benchmarks are taken from an experimental benchmarking suite
representing a variety of open-source Go packages, the raw results may
be found here:

https://perf.golang.org/search?q=upload:20190509.1

For #30333.

Change-Id: I618a48534d2d6ce5f656bb66825e3c383ab1ffba
Reviewed-on: https://go-review.googlesource.com/c/go/+/175797
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 16:23:09 +00:00
Michael Anthony Knyszek
fe67ea32bf runtime: add background scavenger
This change adds a background scavenging goroutine whose pacing is
determined when the heap goal changes. The scavenger is paced to use
at most 1% of the mutator's time for most systems. Furthermore, the
scavenger's pacing is computed based on the estimated number of
scavengable huge pages to take advantage of optimizations provided by
the OS.

The purpose of this scavenger is to deal with a shrinking heap: if the
heap goal is falling over time, the scavenger should kick in and start
returning free pages from the heap to the OS.

Also, now that we have a pacing system, the credit system used by
scavengeLocked has become redundant. Replace it with a mechanism which
only scavenges on the allocation path if it makes sense to do so with
respect to the new pacing system.

Fixes #30333.

Change-Id: I6203f8dc84affb26c3ab04528889dd9663530edc
Reviewed-on: https://go-review.googlesource.com/c/go/+/142960
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 16:21:43 +00:00
Michael Anthony Knyszek
eaa1c87b00 runtime: remove periodic scavenging
This change removes the periodic scavenger which goes over every span
in the heap and scavenges it if it hasn't been used for 5 minutes. It
should no longer be necessary if we have background scavenging
(follow-up).

For #30333.

Change-Id: Ic3a1a4e85409dc25719ba4593a3b60273a4c71e0
Reviewed-on: https://go-review.googlesource.com/c/go/+/143157
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 16:20:52 +00:00
Russ Cox
d56199d68e cmd/vendor: import vet fixes from x/tools
Fixes build - I did not understand that vetall was
effectively pinned to a vet version by cmd/go.mod.

Change-Id: I56bfd8f62eadacc97cad0ed48e41a178bbc18b8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/176179
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 16:10:22 +00:00
Than McIntosh
b56d1bad29 cmd/compile: emit DWARF call_line attrs with data4 form on iOS
When targeting iOS, change the format (DWARF "form") of the call line
attribute for inlined subroutine DIEs, to work around an apparent bug
in /usr/bin/symbols on Darwin.

[Just for posterity: there is nothing wrong with using DW_FORM_udata
for the call_line attribute form; this is perfectly legal DWARF (and
is in fact recommended by the standard relative to data4, which is
less descriptive and often takes more space). The form switch is there
just to make the Apple tools happy.]

Updates #31459.

Change-Id: Iaf362788a8c6684eea4cde8956c0661b694cecc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/174538
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2019-05-09 15:55:36 +00:00
Russ Cox
b6f59cbc80 cmd/vet/all: update whitelist for vet fixes
The vetall builder runs vet straight out of golang.org/x/tools,
so submiting CL 176097 in that repo will break the builder
by making all these whitelist entries stale.
Submiting this CL will fix it, by removing them.

The addition of the gcWriteBarrier declaration in runtime/stubs.go
is necessary because the diagnostic is no longer emitted on arm,
so it must be removed from all.txt. Adding it to runtime is better
than adding it to every-other-goarch.txt.

For #31916.

Change-Id: I432f6049cd3ee5a467add5066c440be8616d9d54
Reviewed-on: https://go-review.googlesource.com/c/go/+/176177
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 15:32:36 +00:00
David Chase
5286b2ad42 cmd/compile: test delve instead of gdb in ssa/debug_test.go
This seems to deflake the test, and also allows testing
on macOS.

Fixes #31786.

Change-Id: I10bfba46dd4b8e64cb09fdd4dd9d175c1ce1f022
Reviewed-on: https://go-review.googlesource.com/c/go/+/176058
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-05-09 14:58:03 +00:00
Than McIntosh
0793c81009 cmd/link: fix link time regression in object file reading
In CL 173938, the linker's object file reader was switched over to
selectively create strings backed with read-only mmap'd memory.
In the process a call to r.rd.Offset() was added to readSymName(),
which greatly increased the number of system calls (Offset does a
seek system call).

This patch changes the object file reader so that all reads are done
directly from the mmap'd data if it is present, and adds logic to keep
track of the offset within the rodata consumed so far. Doing this gets
rid of the calls to r.rd.Offset() and the corresponding seek system
calls.

Also as part of this change, hoist the calls to objabi.PathToPrefix
up into the initial setup code for object reading, and store the
result in the reader (since objabi.PathToPrefix was also coming up
as hot in the profile).

Numbers for this change from compilebench:

benchmark                 old ns/op       new ns/op       delta
BenchmarkTemplate         172053975       170357597       -0.99%
BenchmarkUnicode          64564850        64333653        -0.36%
BenchmarkGoTypes          627931042       628043673       +0.02%
BenchmarkCompiler         2982468893      2924575043      -1.94%
BenchmarkSSA              9701681721      9799342557      +1.01%
BenchmarkFlate            106847240       107509414       +0.62%
BenchmarkGoParser         132082319       130734905       -1.02%
BenchmarkReflect          386810586       383036621       -0.98%
BenchmarkTar              154360072       152670594       -1.09%
BenchmarkXML              217725693       216858727       -0.40%
BenchmarkLinkCompiler     908813802       734363234       -19.20%
BenchmarkStdCmd           32378532486     31222542974     -3.57%

Fixes #31898.

Change-Id: Ibf253a52ce9213325f42b1c2b20d0410f5c88c3b
Reviewed-on: https://go-review.googlesource.com/c/go/+/176039
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-05-09 14:51:29 +00:00
Elias Naur
fb63ed2bab misc/android: silence adb output unless an error occurs
Fixes #31917

Change-Id: I794e457b2245d355e2df5077078c67aa09e00ff9
Reviewed-on: https://go-review.googlesource.com/c/go/+/175920
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-09 04:58:53 +00:00
Russ Cox
f766b68023 cmd/asm: accept TEXT f+0(SB) in -gensymabis mode
f+0(SB) is a non-standard but acceptable alias for f(SB).

Fixes #30968.

Change-Id: I499ccee4d3ff3ab4e47f75d99407aace858e59aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/174537
Reviewed-by: Austin Clements <austin@google.com>
2019-05-09 00:02:34 +00:00
Russ Cox
856b57e0d4 cmd/go: diagnose go.mod and vendor out of sync in std and cmd
The most common failure mode of the current std/cmd setup is
going to be people running "go get m@latest" and then not running
"go mod vendor" and being confused about getting the old m.
Diagnose and report what to do.

Also, having done the check, when in the standard library,
switch the go command to -mod=vendor mode.
This avoids some network accesses I saw when running
'go clean -modcache' before doing some work in cmd.

Change-Id: I0ba4a66637b67225a9b97a1c89f26f9015b41673
Reviewed-on: https://go-review.googlesource.com/c/go/+/174528
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-05-09 00:01:25 +00:00
Brad Fitzpatrick
41df5aeb7f cmd/go: set the "generate" build tag in go generate, per design doc
And use it in two internal windows packages, so they don't show up in
"go list std" or binary releases on non-Windows platforms.

Fixes #31920

Change-Id: Iaa292b6015c9d7310dd677c9e296006440ba5e27
Reviewed-on: https://go-review.googlesource.com/c/go/+/175983
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-08 23:13:44 +00:00
Josh Bleecher Snyder
0047353c53 cmd/compile: add countRule rewrite rule helper
noteRule is useful when you're trying to debug
a particular rule, or get a general sense for
how often a rule fires overall.

It is less useful if you're trying to figure
out which functions might be useful to benchmark
to ascertain the impact of a newly added rule.

Enter countRule. You use it like noteRule,
except that you get per-function summaries.

Sample output:

 # runtime
(*mspan).sweep: idx1=1
evacuate_faststr: idx1=1
evacuate_fast32: idx1=1
evacuate: idx1=2
evacuate_fast64: idx1=1
sweepone: idx1=1
purgecachedstats: idx1=1
mProf_Free: idx1=1

This suggests that the map benchmarks
might be good to run for this added rule.

Change-Id: Id471c3231f1736165f2020f6979ff01c29677808
Reviewed-on: https://go-review.googlesource.com/c/go/+/167088
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-05-08 22:58:10 +00:00
Brad Fitzpatrick
591454c44c internal/syscall/windows/sysdll: mark package as Windows-only
Updates #31920

Change-Id: Ie24ed5bab249e2f90d1740f42a8b8d94fd0983f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/176019
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-05-08 21:21:39 +00:00
Austin Clements
7498e8f5a2 cmd/link: write memprofile in legacy format for compilebench
compilebench depends on the legacy heap profile format to read the
allocation stats of build tools. We're adding a benchmark for the
linker to compilebench, so we need the linker to emit the heap profile
in the legacy format.

This is the linker equivalent of CL 35484, which did this for the
compiler.

Change-Id: I16ad60c4f0fd80b4b6d608a5677ebe04e1fb5e5a
Reviewed-on: https://go-review.googlesource.com/c/go/+/176057
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-05-08 21:06:26 +00:00
Marwan Sulaiman
d21c7b7282 cmd/go: disallow go.sum updates in -mod=readonly
When running go build with the flag -mod=readonly, it fails the build if
go.sum files requires updating. This ensures that CI/CD systems get a
complete go.sum file so that they'd never hit a notary,
assuming the CI/CD system passes the above flag.
I am not familiar with the entire codebase but I assume goSum.dirty
will always be true if go.sum has any missing lines.

Fixes #30667

Change-Id: I767d3b594055d8c10048f4c68e6687c94bb0545c
Reviewed-on: https://go-review.googlesource.com/c/go/+/166237
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-05-08 19:40:38 +00:00
Alan Donovan
19966e9b53 go/types: add CheckExpr function to type-check an expression
In IDE-like applications (and also in tests), there is often a need to
type-check an expression as if it appeared at a particular position in
the source code of an already typed-checked package.

Eval was added to address this need, but did so only partially,
stopping short of exposing a type-annotated expression tree. This
makes it impossible to resolve an expression such as new(T).x and
discover what Object x refers to.  CheckExpr exposes that generality.
Eval is now implemented in terms of CheckExpr.

This change includes a test that demonstrates the object resolution
functionality just described.

Historical context:
- https://go-review.googlesource.com/c/tools/+/10800
- https://codereview.appspot.com/10748044/

Change-Id: I715ba934b9fc0c9ceb61270e20c5f91f4eff20c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/144677
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-05-08 18:24:30 +00:00
JT Olio
5a2da5624a math/big: stack allocate scaleDenom return value
benchmark             old ns/op     new ns/op     delta
BenchmarkRatCmp-4     154           77.9          -49.42%

Change-Id: I932710ad8b6905879e232168b1777927f86ba22a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175460
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-05-08 17:11:57 +00:00