1
0
mirror of https://github.com/golang/go synced 2024-11-14 14:10:29 -07:00
Commit Graph

43791 Commits

Author SHA1 Message Date
Than McIntosh
f875f8fe76 [dev.link] cmd/link: tweaks to data alignment processing
Now that the loader's internal storage mechanism for symbol alignment
is array-based and not map-based, we can go back to computing symbol
alignment in the parallel-by-section section of dodata.

With this patch plus the previous one, this produces a small
kubelet speedup:

$ benchstat out.devlink.txt out.align.txt
name                        old time/op  new time/op  delta
RelinkKubelet                13.3s ± 2%   13.1s ± 2%  -1.27%  (p=0.000 n=20+20)
RelinkKubelet-WithoutDebug   7.36s ± 5%   7.14s ± 3%  -3.00%  (p=0.000 n=20+20)

Change-Id: I9eb0e8fea6aeb12f188f499e9031d5a3a23232c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/231221
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-05-01 01:51:18 +00:00
Than McIntosh
2f83d68926 [dev.link] cmd/link/internal/loader: change storage mechanism for sym alignment
Switch the storage mechanism for symbol alignment away from a map and
to a slice of uint8 values per symbol, where value K indicates
alignment 2^K. Intended to help speed up alignment get/set in dodata.

Change-Id: I26416e455c808f697dd0d7f6d2582247ee5c5b40
Reviewed-on: https://go-review.googlesource.com/c/go/+/231220
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-05-01 01:50:58 +00:00
Cherry Zhang
cdfff4d25a [dev.link] cmd/link: use more compact representation for external relocations
Currently, for external relocations, the ExtReloc structure
contains all the fields of the relocation. In fact, many of the
fields are the same with the original relocation. So, instead, we
can just use an index to reference the original relocation and
not expand the fields.

There is one place where we modify relocation type: changing
R_DWARFSECTREF to R_ADDR. Get away with it by changing
downstreams.

It also makes it easier to retrieve the reloc variant.

This reduces some allocation. Linking cmd/compile with external
linking,

name           old alloc/op   new alloc/op   delta
Reloc_GC         34.1MB ± 0%    22.7MB ± 0%  -33.30%  (p=0.000 n=5+4)

Change-Id: Id08a89ed2aee705296886d3b95014b806a0d55cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/231217
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 21:30:02 +00:00
Than McIntosh
ca290169ab [dev.link] cmd/link: performance changes for relocsym
Revise the signature for "relocsym" to reflect the fact that many of
its arguments are invariant: push the invariant args into a struct and
pass the struct by reference.

Add a facility for doing batch allocation of external relocations in
relocsym, so that we don't wind up with wasted space due to the
default "append" behavior.

This produces a small speedup in linking kubelet:

$ benchstat out.devlink.txt out.dodata.txt
name                        old time/op  new time/op  delta
RelinkKubelet                14.2s ± 2%   13.8s ± 2%  -3.11%  (p=0.000 n=19+19)
RelinkKubelet-WithoutDebug   8.02s ± 3%   7.73s ± 3%  -3.67%  (p=0.000 n=20+20)

Change-Id: I8bc94c366ae792a5b0f23697b8e0108443a7a748
Reviewed-on: https://go-review.googlesource.com/c/go/+/231138
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-30 21:04:11 +00:00
Cherry Zhang
1419445926 [dev.link] all: merge branch 'master' into dev.link
Clean merge.

Change-Id: I9a30645ca0ceb52e45bc6b301f9f15f2f42998e8
2020-04-30 12:32:09 -04:00
Than McIntosh
404f626ee5 [dev.link] cmd/link: minor performance tweaks in dodata
Tweak doDataSect to reduce symbol sorting overhead, and calculate size
ahead of allocating the ctxt.datap slice. Yields a small speedup
(2-3%) linking kubelet.

Change-Id: I82869f5276caa4bee9f6e6f41da2b240e601ce50
Reviewed-on: https://go-review.googlesource.com/c/go/+/231047
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-30 16:16:30 +00:00
Cherry Zhang
2a00c137b1 [dev.link] cmd/link: fold zero symbol check into ResolveABIAlias
We call (or will call) ResolveABIAlias in many places. Doing zero
symbol check everytime is annoying. Fold the condition into
ResolveABIAlias.

Change-Id: I10485fe83b9cce2d19b6bd17dc42176f72dae48b
Reviewed-on: https://go-review.googlesource.com/c/go/+/231046
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 15:53:49 +00:00
Cherry Zhang
516c29a79f [dev.link] cmd/link: pass reloc by value to Adddynrel2
Adddynrel2 is a function pointer. In dynrelocsym we pass &r to
it, which will cause r to escape. Pass it by value instead.

Linking cmd/compile,

name           old alloc/op   new alloc/op   delta
Dodata_GC        15.8MB ± 0%     5.9MB ± 0%  -62.55%  (p=0.008 n=5+5)

Change-Id: Ib86005d1026ebaca57777b27ead037e613585f44
Reviewed-on: https://go-review.googlesource.com/c/go/+/231045
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 15:53:43 +00:00
Cherry Zhang
5aa59c6a99 [dev.link] cmd/link: unescape relocs passed to Archreloc2
Archreloc2 is a function pointer. It will escape its pointer
arguments. In relocsym, as we pass &r and &rr to Archreloc2, it
causes them to escape, even if Archreloc2 is not actually called.

Instead, pass r by value. loader.Reloc2 is a small structure
which is intended to be passed by value.

For rr, as Archreloc2 will likely return true, we speculatively
add it to extRelocs slice and use that space to pass to
Archreloc2.

Linking cmd/compile,

name              old alloc/op   new alloc/op   delta
Dwarfcompress_GC     110MB ± 0%      24MB ± 0%   -78.34%  (p=0.008 n=5+5)
Reloc_GC            24.6MB ± 0%     0.0MB ± 0%  -100.00%  (p=0.029 n=4+4)

Linking cmd/compile using external linking

name              old alloc/op   new alloc/op   delta
Reloc_GC             152MB ± 0%      36MB ± 0%   -76.07%  (p=0.008 n=5+5)

Change-Id: I1415479e0c17ea9787f9a62453dce00ad9ea792f
Reviewed-on: https://go-review.googlesource.com/c/go/+/231077
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 15:53:32 +00:00
Brad Fitzpatrick
ecdbffd4ec net/http/httputil: don't append to X-Forwarded-For in ReverseProxy when nil
Fixes #38079

Change-Id: Iac02d7f9574061bb26d1d9a41bb6ee6cc38934e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/230937
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-04-30 14:41:10 +00:00
Ruixin Bao
1d9801223e cmd/compile: adopt strong aux typing for some s390x rules
Convert the remaining lowering rules to strongly-typed versions.

Passes toolstash-check.
Change-Id: I583786806d55376f5463addab8fec32cb59fa7a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/230939
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-30 13:46:20 +00:00
Alex Brainman
c76befe0f4 cmd/go: use -buildmode=pie as default on window
This change adjusts go command to pass -buildmode=pie to cmd/link,
if -buildmode is not explicitly provided.

Fixes #35192

Change-Id: Iec020131e676eb3e9a2df9eea1929b2af2b6df04
Reviewed-on: https://go-review.googlesource.com/c/go/+/230217
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-04-30 08:07:47 +00:00
Tobias Klauser
5c9a8c0761 lib/time, time/tzdata: update tz data to 2020a
Updates #22487

Change-Id: I691b4c8ced7bfb368f46ade0be46b2ab3f1820dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/230360
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-04-30 08:07:39 +00:00
Daniel Martí
b7e54d8d07 cmd/go: make 'mod verify' use multiple CPUs
'go mod verify' checksums one module zip at a time, which is
CPU-intensive on most modern machines with fast disks. As a result, one
can see a CPU bottleneck when running the command on, for example, a
module where 'go list -m all' lists ~440 modules:

	$ /usr/bin/time go mod verify
	all modules verified
	11.47user 0.77system 0:09.41elapsed 130%CPU (0avgtext+0avgdata 24284maxresident)k
	0inputs+0outputs (0major+4156minor)pagefaults 0swaps

Instead, verify up to GOMAXPROCS zips at once, which should line up
pretty well with the amount of processors we can use on a machine. The
results below are obtained via 'benchcmd -n 5 GoModVerify go mod verify'
on the same large module.

	name         old time/op         new time/op         delta
	GoModVerify          9.35s ± 1%          3.03s ± 2%  -67.60%  (p=0.008 n=5+5)

	name         old user-time/op    new user-time/op    delta
	GoModVerify          11.2s ± 1%          16.3s ± 3%  +45.38%  (p=0.008 n=5+5)

	name         old sys-time/op     new sys-time/op     delta
	GoModVerify          841ms ± 9%          865ms ± 8%     ~     (p=0.548 n=5+5)

	name         old peak-RSS-bytes  new peak-RSS-bytes  delta
	GoModVerify         27.8MB ±13%         50.7MB ±27%  +82.01%  (p=0.008 n=5+5)

The peak memory usage nearly doubles, and there is some extra overhead,
but it seems clearly worth the tradeoff given that we see a ~3x speedup
on my laptop with 4 physical cores. The vast majority of developer
machines nowadays should have 2-4 cores at least.

No test or benchmark is included; one can benchmark 'go mod verify'
directly, as I did above. The existing tests also cover correctness,
including any data races via -race.

Fixes #38623.

Change-Id: I45d8154687a6f3a6a9fb0e2b13da4190f321246c
Reviewed-on: https://go-review.googlesource.com/c/go/+/229817
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-30 07:05:55 +00:00
Keyan Pishdadian
e87b0644db cmd/go: add error for cross-compiled -race builds
Race builds require C dependencies, but cross-compiled cgo builds are
not always possible, so don't suggest enabling CGO in those cases.

Fixes #37021

Change-Id: I1fd675efc9cef958a926bd63eac8e6858bc59d0a
GitHub-Last-Rev: cbf43c1bbb
GitHub-Pull-Request: golang/go#38670
Reviewed-on: https://go-review.googlesource.com/c/go/+/230202
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-30 04:02:34 +00:00
Rob Pike
41f6388e70 cmd/cover: include a package name in the HTML title
A recent change added a title to the HTML coverage report but
neglected to include the package name. Add the package name here.
It's a little trickier than you'd think because there may be multiple
packages and we don't want to parse the files, so we just extract
a directory name from the path of the first file.  This will almost
always be right, and has the advantage that it gives a better result
for package main. There are rare cases it will get wrong, but that
will be no hardship.

If this turns out not to be good enough, we can refine it.

Fixes #38609

Change-Id: I2201f6caef906e0b0258b90d7de518879041fe72
Reviewed-on: https://go-review.googlesource.com/c/go/+/230517
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-30 03:58:01 +00:00
Robert Griesemer
1d31f9b1e0 strconv: implement parseFloatPrefix returning no. of bytes consumed
parseFloatPrefix will make it easier to implement ParseComplex.

Verified that there's no relevant performance impact:
Benchmarks run on a "quiet" MacBook Pro, 3.3GHz Dual-Core Intel Core i7,
with 16GB 2133MHz LPDDR3 RAM running macOS 10.15.4.

name                  old time/op  new time/op  delta
Atof64Decimal-4       38.2ns ± 4%  38.4ns ± 3%    ~     (p=0.802 n=5+5)
Atof64Float-4         41.1ns ± 3%  43.0ns ± 1%  +4.77%  (p=0.008 n=5+5)
Atof64FloatExp-4      71.9ns ± 3%  70.1ns ± 1%    ~     (p=0.063 n=5+5)
Atof64Big-4            124ns ± 5%   119ns ± 0%    ~     (p=0.143 n=5+4)
Atof64RandomBits-4    57.2ns ± 1%  55.7ns ± 2%  -2.66%  (p=0.016 n=4+5)
Atof64RandomFloats-4  56.8ns ± 1%  56.9ns ± 4%    ~     (p=0.556 n=4+5)
Atof32Decimal-4       35.4ns ± 5%  35.9ns ± 0%    ~     (p=0.127 n=5+5)
Atof32Float-4         39.6ns ± 7%  40.3ns ± 1%    ~     (p=0.135 n=5+5)
Atof32FloatExp-4      73.7ns ± 7%  71.9ns ± 0%    ~     (p=0.175 n=5+4)
Atof32Random-4         103ns ± 6%    98ns ± 2%  -5.03%  (p=0.008 n=5+5)

Updates #36771.

Change-Id: I8ff66b582ae8b468d89c9ffc35c569c735cf0341
Reviewed-on: https://go-review.googlesource.com/c/go/+/230737
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-04-30 03:50:03 +00:00
Ian Lance Taylor
b255c12337 syscall: on linux-arm64, prefer prlimit to {g,s}etrlimit
Reportedly some Docker images accept the prlimit64 system call,
used by syscall.prlimit, but prohibit the getrlimit and setrlimit
system calls.

Fixes #38604

Change-Id: I91ff9370450b4869098cc8e335bbb7b863060508
Reviewed-on: https://go-review.googlesource.com/c/go/+/230339
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-04-30 03:32:25 +00:00
Cherry Zhang
d23cd597aa [dev.link] cmd/link: remove sym.Symbols.Newsym
No longer needed.

Change-Id: If259a956bc8edb2eb94583b06840b52344cb84be
Reviewed-on: https://go-review.googlesource.com/c/go/+/231037
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 03:11:13 +00:00
Cherry Zhang
4048fb8780 [dev.link] cmd/link: combine decodesym.go and decodesym2.go
And remove "2" from some function names.

Change-Id: Ibf1089970d849a42f53976064ceb9ade20bf6eba
Reviewed-on: https://go-review.googlesource.com/c/go/+/231017
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 03:11:04 +00:00
Ian Lance Taylor
769a1cf7b6 debug/gosym: correct comments for Table.{Files,Objs}
The fields aren't too useful for Go 1.2 and later, but they aren't
actually nil.

Fixes #38754

Change-Id: Ia13a224f623697a00dea8ba0225633e1b9308c9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/230940
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-04-30 03:07:49 +00:00
Austin Clements
eda6fe3572 Revert "cmd/compile: omit file:pos for non-existent, permission errors"
This reverts commit 4f7053c87f.

Reason for revert: Newly added test is failing on several builders.

Change-Id: I22dcbfebf2f57735b2f479886bbeb623f95b132f
Reviewed-on: https://go-review.googlesource.com/c/go/+/231043
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-30 02:29:55 +00:00
Emmanuel T Odeke
4f7053c87f cmd/compile: omit file:pos for non-existent, permission errors
Omits printing the file:line:column when trying to open either
* non-existent files
* files without permission

Given:
    go tool compile x.go

For either of x.go not existing, or if no read permissions:

* Before:
    x.go:0: open x.go: no such file or directory
    x.go:0: open x.go: permission denied

* After:
    open x.go: no such file or directory
    open x.go: permission denied

While here, noticed an oddity with the Linux builders, that appear
to always be running under root, hence the test for permission errors
with 0222 -W-*-W-*-W- can't pass on linux-amd64 builders.
The filed bug is #38608.

Fixes #36437

Change-Id: I9645ef73177c286c99547e3a0f3719fa07b35cb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/229357
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-30 01:17:47 +00:00
Austin Clements
4e00b4c366 runtime: move condition into wakep
All five calls to wakep are protected by the same check of nmidle and
nmspinning. Move this check into wakep.

Change-Id: I2094eec211ce551e462e87614578f37f1896ba38
Reviewed-on: https://go-review.googlesource.com/c/go/+/230757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-04-30 00:42:35 +00:00
kakulisen
df2862cf54 math: Add a function example
When I browsed the source code, I saw that there is no corresponding example of this function. I am not sure if there is a need for an increase, this is my first time to submit CL.

Change-Id: Idbf4e1e1ed2995176a76959d561e152263a2fd26
Reviewed-on: https://go-review.googlesource.com/c/go/+/230741
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-30 00:33:38 +00:00
Matthew Dempsky
844b410922 net/http/cgi: replace constant map with switch statement
The switch statement can be statically optimized by the compiler,
whereas similarly optimizing the map index expression would require
additional compiler analysis to detect the map is never mutated.

Updates #10848.

Change-Id: I2fc70d4a34dc545677b99f218b51023c7891bbbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/231041
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-30 00:13:38 +00:00
Keith Randall
a7e539619e cmd/compile: move last of the generic rules to typed aux
Change-Id: I1193fff570ff2917979e5cb610fb14c3b347aa92
Reviewed-on: https://go-review.googlesource.com/c/go/+/230938
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 22:40:20 +00:00
Austin Clements
9d812cfa5c cmd/compile,runtime: stack maps only at calls, remove register maps
Currently, we emit stack maps and register maps at almost every
instruction. This was originally intended to support non-cooperative
preemption, but was only ever used for debug call injection. Now debug
call injection also uses conservative frame scanning. As a result,
stack maps are only needed at call sites and register maps aren't
needed at all except that we happen to also encode unsafe-point
information in the register map PCDATA stream.

This CL reduces stack maps to only appear at calls, and replace full
register maps with just safe/unsafe-point information.

This is all protected by the go115ReduceLiveness feature flag, which
is defined in both runtime and cmd/compile.

This CL significantly reduces binary sizes and also speeds up compiles
and links:

name                      old exe-bytes     new exe-bytes     delta
BinGoSize                      15.0MB ± 0%       14.1MB ± 0%   -5.72%

name                      old pcln-bytes    new pcln-bytes    delta
BinGoSize                      3.14MB ± 0%       2.48MB ± 0%  -21.08%

name                      old time/op       new time/op       delta
Template                        178ms ± 7%        172ms ±14%  -3.59%  (p=0.005 n=19+19)
Unicode                        71.0ms ±12%       69.8ms ±10%    ~     (p=0.126 n=18+18)
GoTypes                         655ms ± 8%        615ms ± 8%  -6.11%  (p=0.000 n=19+19)
Compiler                        3.27s ± 6%        3.15s ± 7%  -3.69%  (p=0.001 n=20+20)
SSA                             7.10s ± 5%        6.85s ± 8%  -3.53%  (p=0.001 n=19+20)
Flate                           124ms ±15%        116ms ±22%  -6.57%  (p=0.024 n=18+19)
GoParser                        156ms ±26%        147ms ±34%    ~     (p=0.070 n=19+19)
Reflect                         406ms ± 9%        387ms ±21%  -4.69%  (p=0.028 n=19+20)
Tar                             163ms ±15%        162ms ±27%    ~     (p=0.370 n=19+19)
XML                             223ms ±13%        218ms ±14%    ~     (p=0.157 n=20+20)
LinkCompiler                    503ms ±21%        484ms ±23%    ~     (p=0.072 n=20+20)
ExternalLinkCompiler            1.27s ± 7%        1.22s ± 8%  -3.85%  (p=0.005 n=20+19)
LinkWithoutDebugCompiler        294ms ±17%        273ms ±11%  -7.16%  (p=0.001 n=19+18)

(https://perf.golang.org/search?q=upload:20200428.8)

The binary size improvement is even slightly better when you include
the CLs leading up to this. Relative to the parent of "cmd/compile:
mark PanicBounds/Extend as calls":

name                      old exe-bytes     new exe-bytes     delta
BinGoSize                      15.0MB ± 0%       14.1MB ± 0%   -6.18%

name                      old pcln-bytes    new pcln-bytes    delta
BinGoSize                      3.22MB ± 0%       2.48MB ± 0%  -22.92%

(https://perf.golang.org/search?q=upload:20200428.9)

For #36365.

Change-Id: I69448e714f2a44430067ca97f6b78e08c0abed27
Reviewed-on: https://go-review.googlesource.com/c/go/+/230544
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:21 +00:00
Austin Clements
ee7d9f1c37 cmd/compile: make LivenessMap sparse
We're about to switch to having significantly fewer maps in the
liveness map, so switch from a dense representation to a sparse
representation.

Passes toolstash-check.

For #36365.

Change-Id: Icb17bd6ace17667a280bc5fba4039cae3020a8d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/230543
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-04-29 21:29:20 +00:00
Austin Clements
601bc41da2 cmd/compile: don't emit stack maps for write barrier calls
These are necessarily deeply non-preemptible, so there's no point in
emitting stack maps for them. We already mark them as unsafe points,
so this only affects the runtime, since user code does not emit stack
maps at unsafe points. SSAGenState.PrepareCall also excludes them when
it's sanity checking call stack maps.

Right now this only drops a handful of unnecessary stack maps from the
runtime, but we're about to start emitting stack maps only at calls
for user code, too. At that point, this will matter much more.

For #36365.

Change-Id: Ib3abfedfddc8e724d933a064fa4d573500627990
Reviewed-on: https://go-review.googlesource.com/c/go/+/230542
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:18 +00:00
Austin Clements
faafdf5115 cmd/compile: fix unsafe-points with stack maps
The compiler currently conflates whether a Value has a stack map with
whether it's an unsafe point. For the most part, unsafe-points don't
have stack maps, so this is mostly fine, but call instructions can be
both an unsafe-point *and* have a stack map. For example, none of the
instructions in a nosplit function should be preemptible, but calls
must still have stack maps in case the called function grows the stack
or get preempted.

Currently, the compiler can't distinguish this case, so calls in
nosplit functions are marked as safe-points just because they have
stack maps. This is particularly problematic if a nosplit function
calls another nosplit function, since this can introduce a preemption
point where there should be none.

We realized this was a problem for split-stack prologues a while back,
and CL 207349 changed the encoding of unsafe-points to use the
register map index instead of the stack map index so we could record
both a stack map and an unsafe-point at the same instruction. But this
was never extended into the compiler.

This CL fixes this problem in the compiler. We make LivenessIndex
slightly more abstract by separating unsafe-point marks from stack and
register map indexes. We map this to the PCDATA encoding later when
producing Progs. This isn't enough to fix the whole problem for
nosplit functions, because obj still adds prologues and marks those as
preemptible, but it's a step in the right direction.

I checked this CL by comparing maps before and after this change in
the runtime and net/http. In net/http, unsafe-points match exactly; at
anything that isn't an unsafe-point, both the stack and register maps
are unchanged by this CL. In the runtime, at every point that was a
safe-point before this change, the stack maps agree (and mostly the
runtime doesn't have register maps at all now). In both, all CALLs
(except write barrier calls) have stack maps.

For #36365.

Change-Id: I066628938b02e78be5c81a6614295bcf7cc566c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/230541
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:17 +00:00
Austin Clements
a6deafaf9e cmd/compile: rename issafepoint -> hasStackMap
Currently, this function conflates two (easily conflated!) concepts:
whether a Value is a safe-point and whether it has a stack map. In
particular, call Values may not be a safe-point, but may need a stack
map anyway in case the called function grows the stack.

Hence, rename this function to "hasStackMap", since that's really what
it represents.

For #36365.

Change-Id: I89839de0be8db3be3f0d3a7fb5fcf0b0b6ebc98a
Reviewed-on: https://go-review.googlesource.com/c/go/+/230540
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:16 +00:00
Austin Clements
2bad2f7eba cmd/compile: mark PanicBounds/Extend as calls
PanicBounds and PanicExtend are lowered to runtime calls (with a
non-Go ABI), but are not currently marked as calls. Since liveness
analysis only emits stack maps at calls in the runtime, this means
these panic call sites in the runtime won't get a stack map. These
almost immediately turn into throws in the runtime, but there's still
a chance they'll try to grow the stack first, which would lead to a
different panic.

To fix this, mark these operations as calls.

Outside the runtime, we currently emit stack maps for everything that
isn't an unsafe-point, so these panic calls get stack maps by default.
However, we're about to move to emitting stack maps only at call
sites, at which point this will start to matter outside the runtime as
well.

I confirmed that this has no effect on anything but PCDATA/FUNCDATA in
runtime and net/http.

For #36365.

Change-Id: Ic5bb463fd152cc320c815dc04cf62005261ae169
Reviewed-on: https://go-review.googlesource.com/c/go/+/230539
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:14 +00:00
Austin Clements
57d751370c runtime: use conservative scanning for debug calls
A debugger can inject a call at almost any PC, which causes
significant complications with stack scanning and growth. Currently,
the runtime solves this using precise stack maps and register maps at
nearly all PCs, but these extra maps require roughly 5% of the binary.
These extra maps were originally considered worth this space because
they were intended to be used for non-cooperative preemption, but are
now used only for debug call injection.

This CL switches from using precise maps to instead using conservative
frame scanning, much like how non-cooperative preemption works. When a
call is injected, the runtime flushes all potential pointer registers
to the stack, and then treats that frame as well as the interrupted
frame conservatively.

The limitation of conservative frame scanning is that we cannot grow
the goroutine stack. That's doable because the previous CL switched to
performing debug calls on a new goroutine, where they are free to grow
the stack.

With this CL, there are no remaining uses of precise register maps
(though we still use the unsafe-point information that's encoded in
the register map PCDATA stream), and stack maps are only used at call
sites.

For #36365.

Change-Id: Ie217b6711f3741ccc437552d8ff88f961a73cee0
Reviewed-on: https://go-review.googlesource.com/c/go/+/229300
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:13 +00:00
Austin Clements
3633d2c545 runtime: perform debug call injection on a new goroutine
Currently, when a debugger injects a call, that call happens on the
goroutine where the debugger injected it. However, this requires
significant runtime complexity that we're about to remove.

To prepare for this, this CL switches to a different approach that
leaves the interrupted goroutine parked and runs the debug call on a
new goroutine. When the debug call returns, it resumes the original
goroutine.

This should be essentially transparent to debuggers. It follows the
exact same call injection protocol and ensures the whole protocol
executes indivisibly on a single OS thread. The only difference is
that the current G and stack now change part way through the protocol.

For #36365.

Change-Id: I68463bfd73cbee06cfc49999606410a59dd8f653
Reviewed-on: https://go-review.googlesource.com/c/go/+/229299
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:11 +00:00
Austin Clements
b3863fbbc2 runtime: make newproc1 not start the goroutine
Currently, newproc1 allocates, initializes, and schedules a new
goroutine. We're about to change debug call injection in a way that
will need to create a new goroutine without immediately scheduling it.
To prepare for that, make scheduling the responsibility of newproc1's
caller. Currently, there's exactly one caller (newproc), so this
simply shifts that responsibility.

For #36365.

Change-Id: Idacd06b63e738982e840fe995d891bfd377ce23b
Reviewed-on: https://go-review.googlesource.com/c/go/+/229298
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:08 +00:00
Austin Clements
197a2a3799 runtime/pprof: fix units of MaxRSS on Linux
Rusage.Maxrss is in bytes on Darwin but in KiB on Linux. Fix this
discrepancy so it's always in bytes.

Change-Id: Ic714abc3276566b8fe5e30565072092212610854
Reviewed-on: https://go-review.googlesource.com/c/go/+/230979
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-29 20:33:31 +00:00
Austin Clements
45cd312394 runtime: fix debuglog traceback printing off-by-one
The debuglog traceback printer wasn't adjusting for call/return PCs.

Change-Id: I98dda1c0f22cd78651d88124ea51dc166dc91c7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/227646
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-04-29 20:33:30 +00:00
Alberto Donizetti
c8b42f9aa5 cmd/compile: convert CCop arm64 rules to typed aux
Passes

  GOARCH=arm64 gotip build -toolexec 'toolstash -cmp' -a std

Change-Id: Id88141dfc0dfb02c3e958e48ab4adfbac7ba1380
Reviewed-on: https://go-review.googlesource.com/c/go/+/230837
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-04-29 20:32:48 +00:00
Martin Möhrmann
f2163c4d45 bytes, strings: align requirements for functions passed to FieldFuncs
golang.org/cl/229763 removed the documentation of requirements of
the function passed to FieldsFunc. The current implementation does
not require functions to return consistent results but this had not
been the case for previous implementations.

Add the requirement for consistent results back to the documentation
to allow for future implementations to be more allocation efficient
for an output with more than 32 fields. This is possible with a two
pass algorithm first determining the number of fields used to allocate
the output slice and then splitting the input into fields.

While at it align the documentation of bytes.FieldsFunc with
strings.FieldFunc.

Fixes #38630

Change-Id: Iabbf9ca3dff0daa41f4ec930a21a3dd98e19f122
Reviewed-on: https://go-review.googlesource.com/c/go/+/230797
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-29 19:58:47 +00:00
Jay Conrod
35dce1d67c cmd/go: trim source paths when compiling C with -trimpath
When then go command is run with -trimpath, it will now use
-fdebug-prefix-map when invoking the C compiler (if supported) to
replace the source root directory with a dummy root directory.

This should prevent source directories from appearing either literally
or in compressed DWARF in linked binaries.

Updates #36072

Change-Id: Iedd08d5e886f81e981f11248a1be4ed4f58bdd29
Reviewed-on: https://go-review.googlesource.com/c/go/+/212101
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-29 19:03:10 +00:00
Russ Cox
5a550b6951 go/ast: drop //directive comments from doc.Text
This allows writing

	// F does a thing.
	//go:noinline
	func F()

without the //go:noinline or other directive (such as //line)
ending up looking like extra words in the doc comment.

Fixes #37974.

Change-Id: Ic738d72802cc2fa448f7633915e7126d2f76d8ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/224737
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-29 19:02:32 +00:00
Richard Miller
b13ce66d1b cmd/go/internal/modload: use lockedfile to read path-replacement go.mod files
When parsing go.mod files found via file-path replacements, it's safer to
use lockedfile.Read instead of ioutil.ReadFile, in case of overwriting by
other concurrent go commands.

Change-Id: I7dcac3bb5ada84bee1eb634b39f813c461ef103a
Reviewed-on: https://go-review.googlesource.com/c/go/+/230838
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
2020-04-29 18:56:19 +00:00
Katie Hockman
202c43b2ad crypto/x509/pkix: improve docs and Name.String()
Previously, non-standard attributes in Name.Names were being
omitted when printed using Name.String(). Now, any non-standard
attributes that would not already be printed in Name.String()
are being added temporarily to Name.ExtraNames to be printed.

Fixes #33094
Fixes #23069

Change-Id: Id9829c20968e16db7194549f69c0eb5985044944
Reviewed-on: https://go-review.googlesource.com/c/go/+/229864
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-04-29 18:50:32 +00:00
Cherry Zhang
2a33f5368a [dev.link] cmd/link: use new reloc on Wasm
Change-Id: Icf4d075b64340964068ed038911a14194d241960
Reviewed-on: https://go-review.googlesource.com/c/go/+/230977
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 18:47:15 +00:00
Ruixin Bao
3dc4d37387 crypto/ed25519: drop the purego build tag
Per suggestion in CL 202578, this CL drops the purego build tag used
within this package.

Change-Id: I33626c73d6602e321528544ee601741f7e634c1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/230677
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
2020-04-29 18:33:42 +00:00
Lynn Boger
f512041eca cmd/asm: update ppc64enc.s testcase
Adds a few instructions to ppc64enc.s that were missing from the
previous update.

Change-Id: Ieafce39e905cdf4da3bfb00fdd5a39ab28089cb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/230437
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 18:15:08 +00:00
Cherry Zhang
0bbc19760b [dev.link] cmd/link: free compressed symbols after dwarfcompress
Once the symbol is compressed, we will not need the uncompressed
symbol content. Free its memory.

Linking cmd/compile,

name               old live-B     new live-B     delta
Dwarfcompress_GC      42.7M ± 0%     37.9M ± 0%   -11.31%  (p=0.008 n=5+5)

Change-Id: Ib6cc73832946d158ff4f5b4f31be9c35ba7cf103
Reviewed-on: https://go-review.googlesource.com/c/go/+/230859
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-29 18:10:53 +00:00
Jeremy Faller
65c1805e77 [dev.link] cmd/link: add loader.ExtReloc to Archreloc2
Non functional, but required by the majority of the architectures.

Change-Id: I57601016c28ce665a9d434e283a1db8bded9b133
Reviewed-on: https://go-review.googlesource.com/c/go/+/230858
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 17:41:33 +00:00
Keith Randall
3d34c77829 cmd/compile: convert constant divide strength reduction rules to typed aux
Passes toolstash-check.

Change-Id: Ia5d11c099b8c6c0ed670960b2af808200e3b1ca1
Reviewed-on: https://go-review.googlesource.com/c/go/+/230739
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 16:45:32 +00:00