1
0
mirror of https://github.com/golang/go synced 2024-11-18 07:14:44 -07:00
Commit Graph

43978 Commits

Author SHA1 Message Date
Daniel Martí
b7e54d8d07 cmd/go: make 'mod verify' use multiple CPUs
'go mod verify' checksums one module zip at a time, which is
CPU-intensive on most modern machines with fast disks. As a result, one
can see a CPU bottleneck when running the command on, for example, a
module where 'go list -m all' lists ~440 modules:

	$ /usr/bin/time go mod verify
	all modules verified
	11.47user 0.77system 0:09.41elapsed 130%CPU (0avgtext+0avgdata 24284maxresident)k
	0inputs+0outputs (0major+4156minor)pagefaults 0swaps

Instead, verify up to GOMAXPROCS zips at once, which should line up
pretty well with the amount of processors we can use on a machine. The
results below are obtained via 'benchcmd -n 5 GoModVerify go mod verify'
on the same large module.

	name         old time/op         new time/op         delta
	GoModVerify          9.35s ± 1%          3.03s ± 2%  -67.60%  (p=0.008 n=5+5)

	name         old user-time/op    new user-time/op    delta
	GoModVerify          11.2s ± 1%          16.3s ± 3%  +45.38%  (p=0.008 n=5+5)

	name         old sys-time/op     new sys-time/op     delta
	GoModVerify          841ms ± 9%          865ms ± 8%     ~     (p=0.548 n=5+5)

	name         old peak-RSS-bytes  new peak-RSS-bytes  delta
	GoModVerify         27.8MB ±13%         50.7MB ±27%  +82.01%  (p=0.008 n=5+5)

The peak memory usage nearly doubles, and there is some extra overhead,
but it seems clearly worth the tradeoff given that we see a ~3x speedup
on my laptop with 4 physical cores. The vast majority of developer
machines nowadays should have 2-4 cores at least.

No test or benchmark is included; one can benchmark 'go mod verify'
directly, as I did above. The existing tests also cover correctness,
including any data races via -race.

Fixes #38623.

Change-Id: I45d8154687a6f3a6a9fb0e2b13da4190f321246c
Reviewed-on: https://go-review.googlesource.com/c/go/+/229817
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-30 07:05:55 +00:00
Keyan Pishdadian
e87b0644db cmd/go: add error for cross-compiled -race builds
Race builds require C dependencies, but cross-compiled cgo builds are
not always possible, so don't suggest enabling CGO in those cases.

Fixes #37021

Change-Id: I1fd675efc9cef958a926bd63eac8e6858bc59d0a
GitHub-Last-Rev: cbf43c1bbb
GitHub-Pull-Request: golang/go#38670
Reviewed-on: https://go-review.googlesource.com/c/go/+/230202
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-30 04:02:34 +00:00
Rob Pike
41f6388e70 cmd/cover: include a package name in the HTML title
A recent change added a title to the HTML coverage report but
neglected to include the package name. Add the package name here.
It's a little trickier than you'd think because there may be multiple
packages and we don't want to parse the files, so we just extract
a directory name from the path of the first file.  This will almost
always be right, and has the advantage that it gives a better result
for package main. There are rare cases it will get wrong, but that
will be no hardship.

If this turns out not to be good enough, we can refine it.

Fixes #38609

Change-Id: I2201f6caef906e0b0258b90d7de518879041fe72
Reviewed-on: https://go-review.googlesource.com/c/go/+/230517
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-30 03:58:01 +00:00
Robert Griesemer
1d31f9b1e0 strconv: implement parseFloatPrefix returning no. of bytes consumed
parseFloatPrefix will make it easier to implement ParseComplex.

Verified that there's no relevant performance impact:
Benchmarks run on a "quiet" MacBook Pro, 3.3GHz Dual-Core Intel Core i7,
with 16GB 2133MHz LPDDR3 RAM running macOS 10.15.4.

name                  old time/op  new time/op  delta
Atof64Decimal-4       38.2ns ± 4%  38.4ns ± 3%    ~     (p=0.802 n=5+5)
Atof64Float-4         41.1ns ± 3%  43.0ns ± 1%  +4.77%  (p=0.008 n=5+5)
Atof64FloatExp-4      71.9ns ± 3%  70.1ns ± 1%    ~     (p=0.063 n=5+5)
Atof64Big-4            124ns ± 5%   119ns ± 0%    ~     (p=0.143 n=5+4)
Atof64RandomBits-4    57.2ns ± 1%  55.7ns ± 2%  -2.66%  (p=0.016 n=4+5)
Atof64RandomFloats-4  56.8ns ± 1%  56.9ns ± 4%    ~     (p=0.556 n=4+5)
Atof32Decimal-4       35.4ns ± 5%  35.9ns ± 0%    ~     (p=0.127 n=5+5)
Atof32Float-4         39.6ns ± 7%  40.3ns ± 1%    ~     (p=0.135 n=5+5)
Atof32FloatExp-4      73.7ns ± 7%  71.9ns ± 0%    ~     (p=0.175 n=5+4)
Atof32Random-4         103ns ± 6%    98ns ± 2%  -5.03%  (p=0.008 n=5+5)

Updates #36771.

Change-Id: I8ff66b582ae8b468d89c9ffc35c569c735cf0341
Reviewed-on: https://go-review.googlesource.com/c/go/+/230737
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-04-30 03:50:03 +00:00
Ian Lance Taylor
b255c12337 syscall: on linux-arm64, prefer prlimit to {g,s}etrlimit
Reportedly some Docker images accept the prlimit64 system call,
used by syscall.prlimit, but prohibit the getrlimit and setrlimit
system calls.

Fixes #38604

Change-Id: I91ff9370450b4869098cc8e335bbb7b863060508
Reviewed-on: https://go-review.googlesource.com/c/go/+/230339
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-04-30 03:32:25 +00:00
Cherry Zhang
d23cd597aa [dev.link] cmd/link: remove sym.Symbols.Newsym
No longer needed.

Change-Id: If259a956bc8edb2eb94583b06840b52344cb84be
Reviewed-on: https://go-review.googlesource.com/c/go/+/231037
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 03:11:13 +00:00
Cherry Zhang
4048fb8780 [dev.link] cmd/link: combine decodesym.go and decodesym2.go
And remove "2" from some function names.

Change-Id: Ibf1089970d849a42f53976064ceb9ade20bf6eba
Reviewed-on: https://go-review.googlesource.com/c/go/+/231017
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-30 03:11:04 +00:00
Ian Lance Taylor
769a1cf7b6 debug/gosym: correct comments for Table.{Files,Objs}
The fields aren't too useful for Go 1.2 and later, but they aren't
actually nil.

Fixes #38754

Change-Id: Ia13a224f623697a00dea8ba0225633e1b9308c9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/230940
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-04-30 03:07:49 +00:00
Austin Clements
eda6fe3572 Revert "cmd/compile: omit file:pos for non-existent, permission errors"
This reverts commit 4f7053c87f.

Reason for revert: Newly added test is failing on several builders.

Change-Id: I22dcbfebf2f57735b2f479886bbeb623f95b132f
Reviewed-on: https://go-review.googlesource.com/c/go/+/231043
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-30 02:29:55 +00:00
Emmanuel T Odeke
4f7053c87f cmd/compile: omit file:pos for non-existent, permission errors
Omits printing the file:line:column when trying to open either
* non-existent files
* files without permission

Given:
    go tool compile x.go

For either of x.go not existing, or if no read permissions:

* Before:
    x.go:0: open x.go: no such file or directory
    x.go:0: open x.go: permission denied

* After:
    open x.go: no such file or directory
    open x.go: permission denied

While here, noticed an oddity with the Linux builders, that appear
to always be running under root, hence the test for permission errors
with 0222 -W-*-W-*-W- can't pass on linux-amd64 builders.
The filed bug is #38608.

Fixes #36437

Change-Id: I9645ef73177c286c99547e3a0f3719fa07b35cb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/229357
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-30 01:17:47 +00:00
Austin Clements
4e00b4c366 runtime: move condition into wakep
All five calls to wakep are protected by the same check of nmidle and
nmspinning. Move this check into wakep.

Change-Id: I2094eec211ce551e462e87614578f37f1896ba38
Reviewed-on: https://go-review.googlesource.com/c/go/+/230757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-04-30 00:42:35 +00:00
kakulisen
df2862cf54 math: Add a function example
When I browsed the source code, I saw that there is no corresponding example of this function. I am not sure if there is a need for an increase, this is my first time to submit CL.

Change-Id: Idbf4e1e1ed2995176a76959d561e152263a2fd26
Reviewed-on: https://go-review.googlesource.com/c/go/+/230741
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-30 00:33:38 +00:00
Matthew Dempsky
844b410922 net/http/cgi: replace constant map with switch statement
The switch statement can be statically optimized by the compiler,
whereas similarly optimizing the map index expression would require
additional compiler analysis to detect the map is never mutated.

Updates #10848.

Change-Id: I2fc70d4a34dc545677b99f218b51023c7891bbbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/231041
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-30 00:13:38 +00:00
Keith Randall
a7e539619e cmd/compile: move last of the generic rules to typed aux
Change-Id: I1193fff570ff2917979e5cb610fb14c3b347aa92
Reviewed-on: https://go-review.googlesource.com/c/go/+/230938
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 22:40:20 +00:00
Austin Clements
9d812cfa5c cmd/compile,runtime: stack maps only at calls, remove register maps
Currently, we emit stack maps and register maps at almost every
instruction. This was originally intended to support non-cooperative
preemption, but was only ever used for debug call injection. Now debug
call injection also uses conservative frame scanning. As a result,
stack maps are only needed at call sites and register maps aren't
needed at all except that we happen to also encode unsafe-point
information in the register map PCDATA stream.

This CL reduces stack maps to only appear at calls, and replace full
register maps with just safe/unsafe-point information.

This is all protected by the go115ReduceLiveness feature flag, which
is defined in both runtime and cmd/compile.

This CL significantly reduces binary sizes and also speeds up compiles
and links:

name                      old exe-bytes     new exe-bytes     delta
BinGoSize                      15.0MB ± 0%       14.1MB ± 0%   -5.72%

name                      old pcln-bytes    new pcln-bytes    delta
BinGoSize                      3.14MB ± 0%       2.48MB ± 0%  -21.08%

name                      old time/op       new time/op       delta
Template                        178ms ± 7%        172ms ±14%  -3.59%  (p=0.005 n=19+19)
Unicode                        71.0ms ±12%       69.8ms ±10%    ~     (p=0.126 n=18+18)
GoTypes                         655ms ± 8%        615ms ± 8%  -6.11%  (p=0.000 n=19+19)
Compiler                        3.27s ± 6%        3.15s ± 7%  -3.69%  (p=0.001 n=20+20)
SSA                             7.10s ± 5%        6.85s ± 8%  -3.53%  (p=0.001 n=19+20)
Flate                           124ms ±15%        116ms ±22%  -6.57%  (p=0.024 n=18+19)
GoParser                        156ms ±26%        147ms ±34%    ~     (p=0.070 n=19+19)
Reflect                         406ms ± 9%        387ms ±21%  -4.69%  (p=0.028 n=19+20)
Tar                             163ms ±15%        162ms ±27%    ~     (p=0.370 n=19+19)
XML                             223ms ±13%        218ms ±14%    ~     (p=0.157 n=20+20)
LinkCompiler                    503ms ±21%        484ms ±23%    ~     (p=0.072 n=20+20)
ExternalLinkCompiler            1.27s ± 7%        1.22s ± 8%  -3.85%  (p=0.005 n=20+19)
LinkWithoutDebugCompiler        294ms ±17%        273ms ±11%  -7.16%  (p=0.001 n=19+18)

(https://perf.golang.org/search?q=upload:20200428.8)

The binary size improvement is even slightly better when you include
the CLs leading up to this. Relative to the parent of "cmd/compile:
mark PanicBounds/Extend as calls":

name                      old exe-bytes     new exe-bytes     delta
BinGoSize                      15.0MB ± 0%       14.1MB ± 0%   -6.18%

name                      old pcln-bytes    new pcln-bytes    delta
BinGoSize                      3.22MB ± 0%       2.48MB ± 0%  -22.92%

(https://perf.golang.org/search?q=upload:20200428.9)

For #36365.

Change-Id: I69448e714f2a44430067ca97f6b78e08c0abed27
Reviewed-on: https://go-review.googlesource.com/c/go/+/230544
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:21 +00:00
Austin Clements
ee7d9f1c37 cmd/compile: make LivenessMap sparse
We're about to switch to having significantly fewer maps in the
liveness map, so switch from a dense representation to a sparse
representation.

Passes toolstash-check.

For #36365.

Change-Id: Icb17bd6ace17667a280bc5fba4039cae3020a8d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/230543
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-04-29 21:29:20 +00:00
Austin Clements
601bc41da2 cmd/compile: don't emit stack maps for write barrier calls
These are necessarily deeply non-preemptible, so there's no point in
emitting stack maps for them. We already mark them as unsafe points,
so this only affects the runtime, since user code does not emit stack
maps at unsafe points. SSAGenState.PrepareCall also excludes them when
it's sanity checking call stack maps.

Right now this only drops a handful of unnecessary stack maps from the
runtime, but we're about to start emitting stack maps only at calls
for user code, too. At that point, this will matter much more.

For #36365.

Change-Id: Ib3abfedfddc8e724d933a064fa4d573500627990
Reviewed-on: https://go-review.googlesource.com/c/go/+/230542
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:18 +00:00
Austin Clements
faafdf5115 cmd/compile: fix unsafe-points with stack maps
The compiler currently conflates whether a Value has a stack map with
whether it's an unsafe point. For the most part, unsafe-points don't
have stack maps, so this is mostly fine, but call instructions can be
both an unsafe-point *and* have a stack map. For example, none of the
instructions in a nosplit function should be preemptible, but calls
must still have stack maps in case the called function grows the stack
or get preempted.

Currently, the compiler can't distinguish this case, so calls in
nosplit functions are marked as safe-points just because they have
stack maps. This is particularly problematic if a nosplit function
calls another nosplit function, since this can introduce a preemption
point where there should be none.

We realized this was a problem for split-stack prologues a while back,
and CL 207349 changed the encoding of unsafe-points to use the
register map index instead of the stack map index so we could record
both a stack map and an unsafe-point at the same instruction. But this
was never extended into the compiler.

This CL fixes this problem in the compiler. We make LivenessIndex
slightly more abstract by separating unsafe-point marks from stack and
register map indexes. We map this to the PCDATA encoding later when
producing Progs. This isn't enough to fix the whole problem for
nosplit functions, because obj still adds prologues and marks those as
preemptible, but it's a step in the right direction.

I checked this CL by comparing maps before and after this change in
the runtime and net/http. In net/http, unsafe-points match exactly; at
anything that isn't an unsafe-point, both the stack and register maps
are unchanged by this CL. In the runtime, at every point that was a
safe-point before this change, the stack maps agree (and mostly the
runtime doesn't have register maps at all now). In both, all CALLs
(except write barrier calls) have stack maps.

For #36365.

Change-Id: I066628938b02e78be5c81a6614295bcf7cc566c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/230541
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:17 +00:00
Austin Clements
a6deafaf9e cmd/compile: rename issafepoint -> hasStackMap
Currently, this function conflates two (easily conflated!) concepts:
whether a Value is a safe-point and whether it has a stack map. In
particular, call Values may not be a safe-point, but may need a stack
map anyway in case the called function grows the stack.

Hence, rename this function to "hasStackMap", since that's really what
it represents.

For #36365.

Change-Id: I89839de0be8db3be3f0d3a7fb5fcf0b0b6ebc98a
Reviewed-on: https://go-review.googlesource.com/c/go/+/230540
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:16 +00:00
Austin Clements
2bad2f7eba cmd/compile: mark PanicBounds/Extend as calls
PanicBounds and PanicExtend are lowered to runtime calls (with a
non-Go ABI), but are not currently marked as calls. Since liveness
analysis only emits stack maps at calls in the runtime, this means
these panic call sites in the runtime won't get a stack map. These
almost immediately turn into throws in the runtime, but there's still
a chance they'll try to grow the stack first, which would lead to a
different panic.

To fix this, mark these operations as calls.

Outside the runtime, we currently emit stack maps for everything that
isn't an unsafe-point, so these panic calls get stack maps by default.
However, we're about to move to emitting stack maps only at call
sites, at which point this will start to matter outside the runtime as
well.

I confirmed that this has no effect on anything but PCDATA/FUNCDATA in
runtime and net/http.

For #36365.

Change-Id: Ic5bb463fd152cc320c815dc04cf62005261ae169
Reviewed-on: https://go-review.googlesource.com/c/go/+/230539
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:14 +00:00
Austin Clements
57d751370c runtime: use conservative scanning for debug calls
A debugger can inject a call at almost any PC, which causes
significant complications with stack scanning and growth. Currently,
the runtime solves this using precise stack maps and register maps at
nearly all PCs, but these extra maps require roughly 5% of the binary.
These extra maps were originally considered worth this space because
they were intended to be used for non-cooperative preemption, but are
now used only for debug call injection.

This CL switches from using precise maps to instead using conservative
frame scanning, much like how non-cooperative preemption works. When a
call is injected, the runtime flushes all potential pointer registers
to the stack, and then treats that frame as well as the interrupted
frame conservatively.

The limitation of conservative frame scanning is that we cannot grow
the goroutine stack. That's doable because the previous CL switched to
performing debug calls on a new goroutine, where they are free to grow
the stack.

With this CL, there are no remaining uses of precise register maps
(though we still use the unsafe-point information that's encoded in
the register map PCDATA stream), and stack maps are only used at call
sites.

For #36365.

Change-Id: Ie217b6711f3741ccc437552d8ff88f961a73cee0
Reviewed-on: https://go-review.googlesource.com/c/go/+/229300
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:13 +00:00
Austin Clements
3633d2c545 runtime: perform debug call injection on a new goroutine
Currently, when a debugger injects a call, that call happens on the
goroutine where the debugger injected it. However, this requires
significant runtime complexity that we're about to remove.

To prepare for this, this CL switches to a different approach that
leaves the interrupted goroutine parked and runs the debug call on a
new goroutine. When the debug call returns, it resumes the original
goroutine.

This should be essentially transparent to debuggers. It follows the
exact same call injection protocol and ensures the whole protocol
executes indivisibly on a single OS thread. The only difference is
that the current G and stack now change part way through the protocol.

For #36365.

Change-Id: I68463bfd73cbee06cfc49999606410a59dd8f653
Reviewed-on: https://go-review.googlesource.com/c/go/+/229299
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:11 +00:00
Austin Clements
b3863fbbc2 runtime: make newproc1 not start the goroutine
Currently, newproc1 allocates, initializes, and schedules a new
goroutine. We're about to change debug call injection in a way that
will need to create a new goroutine without immediately scheduling it.
To prepare for that, make scheduling the responsibility of newproc1's
caller. Currently, there's exactly one caller (newproc), so this
simply shifts that responsibility.

For #36365.

Change-Id: Idacd06b63e738982e840fe995d891bfd377ce23b
Reviewed-on: https://go-review.googlesource.com/c/go/+/229298
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 21:29:08 +00:00
Austin Clements
197a2a3799 runtime/pprof: fix units of MaxRSS on Linux
Rusage.Maxrss is in bytes on Darwin but in KiB on Linux. Fix this
discrepancy so it's always in bytes.

Change-Id: Ic714abc3276566b8fe5e30565072092212610854
Reviewed-on: https://go-review.googlesource.com/c/go/+/230979
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-29 20:33:31 +00:00
Austin Clements
45cd312394 runtime: fix debuglog traceback printing off-by-one
The debuglog traceback printer wasn't adjusting for call/return PCs.

Change-Id: I98dda1c0f22cd78651d88124ea51dc166dc91c7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/227646
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-04-29 20:33:30 +00:00
Alberto Donizetti
c8b42f9aa5 cmd/compile: convert CCop arm64 rules to typed aux
Passes

  GOARCH=arm64 gotip build -toolexec 'toolstash -cmp' -a std

Change-Id: Id88141dfc0dfb02c3e958e48ab4adfbac7ba1380
Reviewed-on: https://go-review.googlesource.com/c/go/+/230837
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-04-29 20:32:48 +00:00
Martin Möhrmann
f2163c4d45 bytes, strings: align requirements for functions passed to FieldFuncs
golang.org/cl/229763 removed the documentation of requirements of
the function passed to FieldsFunc. The current implementation does
not require functions to return consistent results but this had not
been the case for previous implementations.

Add the requirement for consistent results back to the documentation
to allow for future implementations to be more allocation efficient
for an output with more than 32 fields. This is possible with a two
pass algorithm first determining the number of fields used to allocate
the output slice and then splitting the input into fields.

While at it align the documentation of bytes.FieldsFunc with
strings.FieldFunc.

Fixes #38630

Change-Id: Iabbf9ca3dff0daa41f4ec930a21a3dd98e19f122
Reviewed-on: https://go-review.googlesource.com/c/go/+/230797
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-29 19:58:47 +00:00
Jay Conrod
35dce1d67c cmd/go: trim source paths when compiling C with -trimpath
When then go command is run with -trimpath, it will now use
-fdebug-prefix-map when invoking the C compiler (if supported) to
replace the source root directory with a dummy root directory.

This should prevent source directories from appearing either literally
or in compressed DWARF in linked binaries.

Updates #36072

Change-Id: Iedd08d5e886f81e981f11248a1be4ed4f58bdd29
Reviewed-on: https://go-review.googlesource.com/c/go/+/212101
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-29 19:03:10 +00:00
Russ Cox
5a550b6951 go/ast: drop //directive comments from doc.Text
This allows writing

	// F does a thing.
	//go:noinline
	func F()

without the //go:noinline or other directive (such as //line)
ending up looking like extra words in the doc comment.

Fixes #37974.

Change-Id: Ic738d72802cc2fa448f7633915e7126d2f76d8ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/224737
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-04-29 19:02:32 +00:00
Richard Miller
b13ce66d1b cmd/go/internal/modload: use lockedfile to read path-replacement go.mod files
When parsing go.mod files found via file-path replacements, it's safer to
use lockedfile.Read instead of ioutil.ReadFile, in case of overwriting by
other concurrent go commands.

Change-Id: I7dcac3bb5ada84bee1eb634b39f813c461ef103a
Reviewed-on: https://go-review.googlesource.com/c/go/+/230838
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
2020-04-29 18:56:19 +00:00
Katie Hockman
202c43b2ad crypto/x509/pkix: improve docs and Name.String()
Previously, non-standard attributes in Name.Names were being
omitted when printed using Name.String(). Now, any non-standard
attributes that would not already be printed in Name.String()
are being added temporarily to Name.ExtraNames to be printed.

Fixes #33094
Fixes #23069

Change-Id: Id9829c20968e16db7194549f69c0eb5985044944
Reviewed-on: https://go-review.googlesource.com/c/go/+/229864
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-04-29 18:50:32 +00:00
Cherry Zhang
2a33f5368a [dev.link] cmd/link: use new reloc on Wasm
Change-Id: Icf4d075b64340964068ed038911a14194d241960
Reviewed-on: https://go-review.googlesource.com/c/go/+/230977
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 18:47:15 +00:00
Ruixin Bao
3dc4d37387 crypto/ed25519: drop the purego build tag
Per suggestion in CL 202578, this CL drops the purego build tag used
within this package.

Change-Id: I33626c73d6602e321528544ee601741f7e634c1b
Reviewed-on: https://go-review.googlesource.com/c/go/+/230677
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
2020-04-29 18:33:42 +00:00
Lynn Boger
f512041eca cmd/asm: update ppc64enc.s testcase
Adds a few instructions to ppc64enc.s that were missing from the
previous update.

Change-Id: Ieafce39e905cdf4da3bfb00fdd5a39ab28089cb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/230437
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 18:15:08 +00:00
Cherry Zhang
0bbc19760b [dev.link] cmd/link: free compressed symbols after dwarfcompress
Once the symbol is compressed, we will not need the uncompressed
symbol content. Free its memory.

Linking cmd/compile,

name               old live-B     new live-B     delta
Dwarfcompress_GC      42.7M ± 0%     37.9M ± 0%   -11.31%  (p=0.008 n=5+5)

Change-Id: Ib6cc73832946d158ff4f5b4f31be9c35ba7cf103
Reviewed-on: https://go-review.googlesource.com/c/go/+/230859
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-29 18:10:53 +00:00
Jeremy Faller
65c1805e77 [dev.link] cmd/link: add loader.ExtReloc to Archreloc2
Non functional, but required by the majority of the architectures.

Change-Id: I57601016c28ce665a9d434e283a1db8bded9b133
Reviewed-on: https://go-review.googlesource.com/c/go/+/230858
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 17:41:33 +00:00
Keith Randall
3d34c77829 cmd/compile: convert constant divide strength reduction rules to typed aux
Passes toolstash-check.

Change-Id: Ia5d11c099b8c6c0ed670960b2af808200e3b1ca1
Reviewed-on: https://go-review.googlesource.com/c/go/+/230739
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 16:45:32 +00:00
Cherry Zhang
f886ff7b41 [dev.link] cmd/link: free loader memory after LoadFull
After LoadFull, we'll be using sym.Symbols mostly. We still need
the loader information for symbol index mappings and name
lookups, but not much else. Free some memory.

Linking cmd/compile,

name           old time/op    new time/op    delta
Loadlibfull_GC    44.5M ± 0%     35.8M ± 0%    -19.66%  (p=0.008 n=5+5)
Archive_GC        46.4M ± 0%     37.6M ± 0%    -18.89%  (p=0.008 n=5+5)

Linking cmd/compile with external linking,

name           old time/op    new time/op    delta
Loadlibfull_GC    82.5M ± 0%     57.4M ± 0%    -30.41%  (p=0.008 n=5+5)
Archive_GC        86.8M ± 0%     61.7M ± 0%    -28.90%  (p=0.008 n=5+5)

Change-Id: I6859d488172ef8968918b86de527fbfed6832ebf
Reviewed-on: https://go-review.googlesource.com/c/go/+/230300
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-29 16:21:37 +00:00
Cherry Zhang
df2a46f85a [dev.link] cmd/link: fix R_GOTOFF handling
When applying relocations, we need to resolve ABI aliases.
relocsym does that. Architecture-specific archreloc also needs to
do that. The old code doesn't do that since ABI aliases are
resolved in loadlibfull, or, in the old linker, in a much earlier
stage. We don't do this in the new linker, as we want to avoid
mutating relocations.

While here, move R_CONST and R_GOTOFF handling to generic code.
They appear on several architectures and the handling are same.

Should fix 386-clang and *bsd-386 builds.

Change-Id: I6681c94f0327555d6cf329d0a518c88848773671
Reviewed-on: https://go-review.googlesource.com/c/go/+/230857
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-04-29 15:32:58 +00:00
Cherry Zhang
095d2a4532 [dev.link] cmd/link: support external linking in new reloc implementation
Support external linking for the new reloc pass as well, and
enable it on AMD64 and 386.

Change-Id: Ia71aec3d7c14e9d661e0748d2e988f29f220d1e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/230308
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-04-29 15:31:14 +00:00
Than McIntosh
9a3bf7d542 [dev.link] cmd/link: remove Sub field from sym.Symbol
Remove 'Sub' field from sym.Symbol, replacing uses (those downstream of
loadlibfull) with loader method calls.

NB: removing the Outer field will have to wait for now; it is accessed
in archreloc methods that don't have access to link ctxt or loader
currently.

Change-Id: I2abe5906fc169c64b2ab7d5ad213619bea5a17c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/230617
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 15:05:14 +00:00
Lynn Boger
56933fb838 cmd/compile,cmd/internal/obj/ppc64: use mod instructions on power9
This updates the PPC64.rules file to use the MOD instructions
that are available in power9. Prior to power9 this is done
using a longer sequence with multiply and divide.

Included in this change is removal of the REM* opcode variations
that set the CC or OV bits since their settings are based
on the DIV and are not appropriate for the REM.

Change-Id: Iceed9ce33e128e1911c15592ee674276ce8ba3fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/229761
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-29 14:45:56 +00:00
Nigel Tao
07d9ea64ab image: guard against NewXxx integer overflow
Prior to this commit, NewXxx could panic when passed an image.Rectangle
with one of width or height being negative. But it might not panic if
both were negative, because (bpp * w * h) could still be positive. After
this commit, it will panic if both are negative.

With overflow, NewXxx might not have panicked if (bpp * w * h), the
length passed to "make([]uint8, length)", was still non-negative (after
truncation), but even if w and h were valid (non-negative), the overall
byte slice wasn't long enough. Iterating over the pixels would possibly
panic later with index out of bounds. This change moves the panic
earlier, closer to where the mistake is.

Change-Id: I011feb2d53515fc3f0fe72bb6c23b3953772c577
Reviewed-on: https://go-review.googlesource.com/c/go/+/230220
Reviewed-by: Rob Pike <r@golang.org>
2020-04-29 11:57:50 +00:00
Than McIntosh
8493b64527 [dev.link] all: merge branch 'master' into dev.link
Change-Id: Ied39f4f701a2e64b87262f7cc34108a60b15e08c
2020-04-29 07:49:35 -04:00
Nigel Tao
7250dd2540 image/jpeg: accept "\xff\x00" before a RST marker
Fixes #28717

Change-Id: I0a1e4ef1583fff89b6f46ef647fb6e4499bdf999
Reviewed-on: https://go-review.googlesource.com/c/go/+/230122
Run-TryBot: Nigel Tao <nigeltao@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-04-29 11:48:24 +00:00
Nigel Tao
03efd42631 image: add a NewUniform doc comment
Fixes #38739

Change-Id: I42b9b601e63ab8df69a0e5ce9bcabf75bb98d83e
Reviewed-on: https://go-review.googlesource.com/c/go/+/230777
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 11:47:27 +00:00
Ruixin Bao
e1a96b82df cmd/compile: adopt strong aux typing for some s390x rules
Convert some optimizations rules to strongly-typed versions. Similar to
CL 230338, this CL only converts rules that need no additional changes
(i.e: only need to change '->' to '=>').

This CL covers the rules from line 800 - 1219.

Passes toolstash-check

Change-Id: I94181a809fa38918b78301f1c0c680b7a8ab552f
Reviewed-on: https://go-review.googlesource.com/c/go/+/230738
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-04-29 08:57:33 +00:00
Meng Zhuo
67d40873ad cmd/compile: adjust MIPS64x rewrite rules to use typed aux fields
Pass toolstash-check

Change-Id: I673c9a24bf69c09573be5aeddbd6072ef35d2d83
Reviewed-on: https://go-review.googlesource.com/c/go/+/228937
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-04-29 04:16:08 +00:00
Jeremy Faller
09cd065875 [dev.link] cmd/link: add archrelocsym x86 support for loader
Change-Id: I34822e5610caf537d62203fb6e0023c382a1e60a
Reviewed-on: https://go-review.googlesource.com/c/go/+/230678
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-04-28 23:11:16 +00:00
Cherry Zhang
76c6cce116 [dev.link] cmd/link: convert reloc pass to using the loader when internal linking
Only enabled for AMD64 when internal linking for now.

Change-Id: I2aa9ee47c0f7413ea7bbcdd31b8317c14220bba3
Reviewed-on: https://go-review.googlesource.com/c/go/+/230302
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-04-28 22:44:41 +00:00