1
0
mirror of https://github.com/golang/go synced 2024-11-23 13:00:07 -07:00
Commit Graph

41208 Commits

Author SHA1 Message Date
Giovanni Bajo
1658263bbf cmd/compile: detect indvars that are bound by other indvars
prove wasn't able to detect induction variables that was bound
by another inducation variable. This happened because an indvar
is a Phi, and thus in case of a dependency, the loop bounding
condition looked as Phi < Phi. This triggered an existing
codepath that checked whether the upper bound was a Phi to
detect loop conditions written in reversed order respect to the
idiomatic way (eg: for i:=0; len(n)>i; i++).

To fix this, we call the indvar pattern matching on both operands
of the loop condition, so that the first operand that matches
will be treated as the indvar.

Updates #24660 (removes a boundcheck from Fannkuch)

Change-Id: Iade83d8deb54f14277ed3f2e37b190e1ed173d11
Reviewed-on: https://go-review.googlesource.com/c/go/+/195220
Reviewed-by: David Chase <drchase@google.com>
2019-09-26 18:47:12 +00:00
Giovanni Bajo
9740b60e14 cmd/compile: refactor some code in loopbce.go
This CL extracts the logic for pattern-matching an induction
variable into a separate function, in preparation for next CL
where we would need to call it multiple times.

No functional changes, passes toolstash -cmp.

Change-Id: Ic52391e6c1b2e72bae32a0f3f65dfea321caaf4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/195737
Reviewed-by: David Chase <drchase@google.com>
2019-09-26 18:46:30 +00:00
Matthew Dempsky
fd5ba54f7c cmd/compile: stop exporting OPTRLIT
Removes TODO left by previous commit to appease toolstash.

Change-Id: I5c0bf25f21ba5c0abe3b1b2ed0b17c604717f39b
Reviewed-on: https://go-review.googlesource.com/c/go/+/197121
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-09-26 18:46:09 +00:00
Matthew Dempsky
00b773a4a9 cmd/compile: simplify OPTRLIT handling
Previously, we would recognize &(T{...}) expressions during type
checking, rewrite them into (*T){...}, and then do a lot of extra work
to make sure the user doesn't write (*T){...} themselves and
resynthesizing the OPTRLIT later on.

This CL simply handles &T{...} directly in the straight forward
manner, by changing OADDR directly to OPTRLIT when appropriate.

While here, match go/types's invalid composite literal type error
message.

Passes toolstash-check.

Change-Id: I902b14c7e2cd9fa93e6915dd58272d2352ba38f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/197120
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-09-26 18:45:53 +00:00
Giovanni Bajo
87e2b34f7b cmd/compile: in prove, learn facts from OpSliceMake
Now that OpSliceMake is called by runtime.makeslice callers,
prove can see and record the actual length and cap of each
slice being constructed.

This small patch is enough to remove 260 additional bound checks
from cmd+std.

Thanks to Martin Möhrmann for pointing me to CL141822 that
I had missed.

Updates #24660

Change-Id: I14556850f285392051f3f07d13b456b608b64eb9
Reviewed-on: https://go-review.googlesource.com/c/go/+/196784
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2019-09-26 18:27:38 +00:00
Joel Sing
08165932f7 cmd/internal/obj/riscv: require memory targets for load and store instructions
This allows for `LD 4(X5), X6' rather than `LD $4, X5, X6'. Similar for other
load and store instructions. It is worth noting that none of these are likely
to be used directly once the MOV pseudo-instructions are implemented.

Updates #27532

Change-Id: Ie043c2dedd2cdaceb258b27976cfb3f74aa1cc1d
Reviewed-on: https://go-review.googlesource.com/c/go/+/196842
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 17:56:30 +00:00
Brad Fitzpatrick
cb418dd0d1 test: make -all_codegen default to true on linux-amd64 builder
Fixes #34297

Change-Id: I4584a97d4562d7af0412d683ba1c206e3c1d9edb
Reviewed-on: https://go-review.googlesource.com/c/go/+/197539
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 17:42:40 +00:00
Joel Sing
a37f2b4f60 cmd/internal/obj/riscv: implement control transfer instructions
Add support for assembling control transfer instructions.

Based on the riscv-go port.

Updates #27532

Change-Id: I205d3ccd0a48deeaace0f20fca8516f382a83fae
Reviewed-on: https://go-review.googlesource.com/c/go/+/196841
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 17:37:02 +00:00
Joel Sing
430d2aa349 cmd/internal/obj/riscv: implement AUIPC and LUI instructions
Add support for assembling AUIPC and LUI instructions.

Based on the riscv-go port.

Updates #27532

Change-Id: I178868b6dcc6fdc6b8527454569a3538ed50723e
Reviewed-on: https://go-review.googlesource.com/c/go/+/196840
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 17:34:54 +00:00
Filippo Valsorda
05c8d8d365 doc: document Go 1.13.1 and Go 1.12.10
Change-Id: If694ce529393b8ae9c6c55270665efc3a108a3b2
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/558778
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/197504
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-09-26 16:44:02 +00:00
Filippo Valsorda
41b1f88efa net/textproto: don't normalize headers with spaces before the colon
RFC 7230 is clear about headers with a space before the colon, like

X-Answer : 42

being invalid, but we've been accepting and normalizing them for compatibility
purposes since CL 5690059 in 2012.

On the client side, this is harmless and indeed most browsers behave the same
to this day. On the server side, this becomes a security issue when the
behavior doesn't match that of a reverse proxy sitting in front of the server.

For example, if a WAF accepts them without normalizing them, it might be
possible to bypass its filters, because the Go server would interpret the
header differently. Worse, if the reverse proxy coalesces requests onto a
single HTTP/1.1 connection to a Go server, the understanding of the request
boundaries can get out of sync between them, allowing an attacker to tack an
arbitrary method and path onto a request by other clients, including
authentication headers unknown to the attacker.

This was recently presented at multiple security conferences:
https://portswigger.net/blog/http-desync-attacks-request-smuggling-reborn

net/http servers already reject header keys with invalid characters.
Simply stop normalizing extra spaces in net/textproto, let it return them
unchanged like it does for other invalid headers, and let net/http enforce
RFC 7230, which is HTTP specific. This loses us normalization on the client
side, but there's no right answer on the client side anyway, and hiding the
issue sounds worse than letting the application decide.

Fixes CVE-2019-16276
Fixes #34540

Change-Id: I6d272de827e0870da85d93df770d6a0e161bbcf1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/549719
Reviewed-by: Brad Fitzpatrick <bradfitz@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/197503
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-09-26 16:43:06 +00:00
Russ Cox
0ad368675b context: use fewer goroutines in WithCancel/WithTimeout
If the parent context passed to WithCancel or WithTimeout
is a known context implementation (one created by this package),
we attach the child to the parent by editing data structures directly;
otherwise, for unknown parent implementations, we make a
goroutine that watches for the parent to finish and propagates
the cancellation.

A common problem with this scheme, before this CL, is that
users who write custom context implementations to manage
their value sets cause WithCancel/WithTimeout to start
goroutines that would have not been started before.

This CL changes the way we map a parent context back to the
underlying data structure. Instead of walking up through
known context implementations to reach the *cancelCtx,
we look up parent.Value(&cancelCtxKey) to return the
innermost *cancelCtx, which we use if it matches parent.Done().

This way, a custom context implementation wrapping a
*cancelCtx but not changing Done-ness (and not refusing
to return wrapped keys) will not require a goroutine anymore
in WithCancel/WithTimeout.

For #28728.

Change-Id: Idba2f435c81b19fe38d0dbf308458ca87c7381e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/196521
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-26 16:25:30 +00:00
Brad Fitzpatrick
ae024d9daf Revert "internal/bytealg: add assembly implementation of Compare/CompareString on mips64x"
This reverts CL 196837 (commit 78baea836d).

Reason for revert: broke the mips64le build.

Change-Id: I531da60d098cb227659c9977a3d229474325b0a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/197538
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 15:54:41 +00:00
Michael Anthony Knyszek
62e4156552 runtime: fix lock acquire cycles related to scavenge.lock
There are currently two edges in the lock cycle graph caused by
scavenge.lock: with sched.lock and mheap_.lock. These edges appear
because of the call to ready() and stack growths respectively.
Furthermore, there's already an invariant in the code wherein
mheap_.lock must be acquired before scavenge.lock, hence the cycle.

The fix to this is to bring scavenge.lock higher in the lock cycle
graph, such that sched.lock and mheap_.lock are only acquired once
scavenge.lock is already held.

To faciliate this change, we move scavenger waking outside of
gcSetTriggerRatio such that it doesn't have to happen with the heap
locked. Furthermore, we check scavenge generation numbers with the heap
locked by using gopark instead of goparkunlock, and specify a function
which aborts the park should there be any skew in generation count.

Fixes #34047.

Change-Id: I3519119214bac66375e2b1262b36ce376c820d12
Reviewed-on: https://go-review.googlesource.com/c/go/+/191977
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-09-26 15:44:49 +00:00
Davor Kapsa
f7a00c9906 SECURITY.md: update go versions
Change-Id: I59446965b198767af60d617da29b5c2e457829ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/197477
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-26 15:34:57 +00:00
Bryan C. Mills
81c6bac06f cmd/go/internal/robustio: extend filesystem workarounds to darwin platforms
The macOS filesystem seems to have gotten significantly flakier as of
macOS 10.14, so this causes frequently flakes in the 10.14 builders.

We have no reason to believe that it will be fixed any time soon, so
rather than trying to detect the specific macOS version, we'll apply
the same workarounds that we use on Windows: classifying (and
retrying) the errors known to indicate flakiness and relaxing the
success criteria for renameio.TestConcurrentReadsAndWrites.

Fixes #33041

Change-Id: I74d8c15677951d7a0df0d4ebf6ea03e43eebddf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/197517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-09-26 15:28:04 +00:00
Jeremy Faller
7defbffcda cmd/compile: remove isStmt symbol from FuncInfo
As promised in CL 188238, removing the obsolete symbol.

Here are the latest stats. This is baselined at "e53edafb66" with only
these changes applied, run on magna.cam. The linker looks straight
better (in memory and speed).

There is still a change I'm working on walking the progs to generate the
debug_lines data in the compiler. That will likely result in a compiler
speedup.

name                      old time/op       new time/op       delta
Template                        324ms ± 3%        317ms ± 3%   -2.07%  (p=0.043 n=10+10)
Unicode                         142ms ± 4%        144ms ± 3%     ~     (p=0.393 n=10+10)
GoTypes                         1.05s ± 2%        1.07s ± 2%   +1.59%  (p=0.019 n=9+9)
Compiler                        4.09s ± 2%        4.11s ± 1%     ~     (p=0.218 n=10+10)
SSA                             12.5s ± 1%        12.7s ± 1%   +1.00%  (p=0.035 n=10+10)
Flate                           199ms ± 7%        203ms ± 5%     ~     (p=0.481 n=10+10)
GoParser                        245ms ± 3%        246ms ± 5%     ~     (p=0.780 n=9+10)
Reflect                         672ms ± 4%        688ms ± 3%   +2.42%  (p=0.015 n=10+10)
Tar                             280ms ± 4%        284ms ± 4%     ~     (p=0.123 n=10+10)
XML                             379ms ± 4%        381ms ± 2%     ~     (p=0.529 n=10+10)
LinkCompiler                    1.16s ± 4%        1.12s ± 2%   -3.03%  (p=0.001 n=10+9)
ExternalLinkCompiler            2.28s ± 3%        2.23s ± 3%   -2.51%  (p=0.011 n=8+9)
LinkWithoutDebugCompiler        686ms ± 9%        667ms ± 2%     ~     (p=0.277 n=9+8)
StdCmd                          14.1s ± 1%        14.0s ± 1%     ~     (p=0.739 n=10+10)

name                      old user-time/op  new user-time/op  delta
Template                        604ms ±23%        564ms ± 7%     ~     (p=0.661 n=10+9)
Unicode                         429ms ±40%        418ms ±37%     ~     (p=0.579 n=10+10)
GoTypes                         2.43s ±12%        2.51s ± 7%     ~     (p=0.393 n=10+10)
Compiler                        9.22s ± 3%        9.27s ± 3%     ~     (p=0.720 n=9+10)
SSA                             26.3s ± 3%        26.6s ± 2%     ~     (p=0.579 n=10+10)
Flate                           328ms ±19%        333ms ±12%     ~     (p=0.842 n=10+9)
GoParser                        387ms ± 5%        378ms ± 9%     ~     (p=0.356 n=9+10)
Reflect                         1.36s ±20%        1.43s ±21%     ~     (p=0.631 n=10+10)
Tar                             469ms ±12%        471ms ±21%     ~     (p=0.497 n=9+10)
XML                             685ms ±18%        698ms ±19%     ~     (p=0.739 n=10+10)
LinkCompiler                    1.86s ±10%        1.87s ±11%     ~     (p=0.968 n=10+9)
ExternalLinkCompiler            3.20s ±13%        3.01s ± 8%   -5.70%  (p=0.046 n=8+9)
LinkWithoutDebugCompiler        1.08s ±15%        1.09s ±20%     ~     (p=0.579 n=10+10)

name                      old alloc/op      new alloc/op      delta
Template                       36.3MB ± 0%       36.4MB ± 0%   +0.26%  (p=0.000 n=10+10)
Unicode                        28.5MB ± 0%       28.5MB ± 0%     ~     (p=0.165 n=10+10)
GoTypes                         120MB ± 0%        121MB ± 0%   +0.29%  (p=0.000 n=9+10)
Compiler                        546MB ± 0%        548MB ± 0%   +0.32%  (p=0.000 n=10+10)
SSA                            1.84GB ± 0%       1.85GB ± 0%   +0.49%  (p=0.000 n=10+10)
Flate                          22.9MB ± 0%       23.0MB ± 0%   +0.25%  (p=0.000 n=10+10)
GoParser                       27.8MB ± 0%       27.9MB ± 0%   +0.25%  (p=0.000 n=10+8)
Reflect                        77.5MB ± 0%       77.7MB ± 0%   +0.27%  (p=0.000 n=9+9)
Tar                            34.5MB ± 0%       34.6MB ± 0%   +0.23%  (p=0.000 n=10+10)
XML                            44.2MB ± 0%       44.4MB ± 0%   +0.32%  (p=0.000 n=10+10)
LinkCompiler                    239MB ± 0%        230MB ± 0%   -3.86%  (p=0.000 n=10+10)
ExternalLinkCompiler            243MB ± 0%        243MB ± 0%   +0.22%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler        164MB ± 0%        155MB ± 0%   -5.45%  (p=0.000 n=10+10)

name                      old allocs/op     new allocs/op     delta
Template                         371k ± 0%         372k ± 0%   +0.44%  (p=0.000 n=10+10)
Unicode                          340k ± 0%         340k ± 0%   +0.05%  (p=0.000 n=10+10)
GoTypes                         1.32M ± 0%        1.32M ± 0%   +0.46%  (p=0.000 n=10+10)
Compiler                        5.34M ± 0%        5.37M ± 0%   +0.59%  (p=0.000 n=10+10)
SSA                             17.6M ± 0%        17.7M ± 0%   +0.63%  (p=0.000 n=10+10)
Flate                            233k ± 0%         234k ± 0%   +0.48%  (p=0.000 n=10+10)
GoParser                         309k ± 0%         310k ± 0%   +0.40%  (p=0.000 n=10+10)
Reflect                          964k ± 0%         969k ± 0%   +0.54%  (p=0.000 n=10+10)
Tar                              346k ± 0%         348k ± 0%   +0.48%  (p=0.000 n=10+9)
XML                              424k ± 0%         426k ± 0%   +0.51%  (p=0.000 n=10+10)
LinkCompiler                     751k ± 0%         645k ± 0%  -14.13%  (p=0.000 n=10+10)
ExternalLinkCompiler            1.79M ± 0%        1.69M ± 0%   -5.30%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler         217k ± 0%         222k ± 0%   +2.02%  (p=0.000 n=10+10)

name                      old object-bytes  new object-bytes  delta
Template                        547kB ± 0%        559kB ± 0%   +2.17%  (p=0.000 n=10+10)
Unicode                         215kB ± 0%        216kB ± 0%   +0.60%  (p=0.000 n=10+10)
GoTypes                        1.99MB ± 0%       2.03MB ± 0%   +2.02%  (p=0.000 n=10+10)
Compiler                       7.86MB ± 0%       8.07MB ± 0%   +2.73%  (p=0.000 n=10+10)
SSA                            26.4MB ± 0%       27.2MB ± 0%   +3.27%  (p=0.000 n=10+10)
Flate                           337kB ± 0%        343kB ± 0%   +2.02%  (p=0.000 n=10+10)
GoParser                        432kB ± 0%        441kB ± 0%   +2.11%  (p=0.000 n=10+10)
Reflect                        1.33MB ± 0%       1.36MB ± 0%   +1.87%  (p=0.000 n=10+10)
Tar                             477kB ± 0%        487kB ± 0%   +2.24%  (p=0.000 n=10+10)
XML                             617kB ± 0%        632kB ± 0%   +2.33%  (p=0.000 n=10+10)

name                      old export-bytes  new export-bytes  delta
Template                       18.5kB ± 0%       18.5kB ± 0%     ~     (all equal)
Unicode                        7.92kB ± 0%       7.92kB ± 0%     ~     (all equal)
GoTypes                        35.0kB ± 0%       35.0kB ± 0%     ~     (all equal)
Compiler                        109kB ± 0%        109kB ± 0%   +0.09%  (p=0.000 n=10+10)
SSA                             137kB ± 0%        137kB ± 0%   +0.03%  (p=0.000 n=10+10)
Flate                          4.89kB ± 0%       4.89kB ± 0%     ~     (all equal)
GoParser                       8.49kB ± 0%       8.49kB ± 0%     ~     (all equal)
Reflect                        11.4kB ± 0%       11.4kB ± 0%     ~     (all equal)
Tar                            10.5kB ± 0%       10.5kB ± 0%     ~     (all equal)
XML                            16.7kB ± 0%       16.7kB ± 0%     ~     (all equal)

name                      old text-bytes    new text-bytes    delta
HelloSize                       760kB ± 0%        760kB ± 0%     ~     (all equal)
CmdGoSize                      10.8MB ± 0%       10.8MB ± 0%     ~     (all equal)

name                      old data-bytes    new data-bytes    delta
HelloSize                      10.7kB ± 0%       10.7kB ± 0%     ~     (all equal)
CmdGoSize                       312kB ± 0%        312kB ± 0%     ~     (all equal)

name                      old bss-bytes     new bss-bytes     delta
HelloSize                       122kB ± 0%        122kB ± 0%     ~     (all equal)
CmdGoSize                       146kB ± 0%        146kB ± 0%     ~     (all equal)

name                      old exe-bytes     new exe-bytes     delta
HelloSize                      1.11MB ± 0%       1.13MB ± 0%   +1.10%  (p=0.000 n=10+10)
CmdGoSize                      14.9MB ± 0%       15.0MB ± 0%   +0.77%  (p=0.000 n=10+10)

Change-Id: I42e6087cd6231dbdcfff5464e46d373474e455e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/192417
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
2019-09-26 15:22:09 +00:00
Michael Munday
cf03238020 cmd/compile: use numeric condition code masks on s390x
Prior to this CL conditional branches on s390x always used an
extended mnemonic such as BNE, BLT and so on to represent branch
instructions with different condition code masks. This CL adds
support for numeric condition code masks to the s390x SSA backend
so that we can encode the condition under which a Block's
successor is chosen as a field in that Block rather than in its
type.

This change will be useful as we come to add support for combined
compare-and-branch instructions. Rather than trying to add extended
mnemonics for every possible combination of mask and compare-and-
branch instruction we can instead use a single mnemonic for each
instruction.

Change-Id: Idb7458f187b50906877d683695c291dff5279553
Reviewed-on: https://go-review.googlesource.com/c/go/+/197178
Reviewed-by: Keith Randall <khr@golang.org>
2019-09-26 14:47:12 +00:00
Jeremy Faller
8506b7d42f cmd/link: switch linker over to new debug lines from compiler
This switches the linker over to using the new debug_lines data
generated in the compiler.

Change-Id: If8362d6fcea7db60aaebab670ed6f702ab1c4908
Reviewed-on: https://go-review.googlesource.com/c/go/+/191968
Run-TryBot: Jeremy Faller <jeremy@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 14:16:54 +00:00
Tobias Klauser
770a13546e cmd/compile: use yyerrorl in some typechecking functions
Replace usage of yyerror with yyerrorl in checkdefergo and copytype in
typecheck.go.

All covered error messages already appear in the tests and the yyerror
replacement did not lead to any tests failing.

Passes toolstash-check

Updates #19683

Change-Id: I735e83bcda7ddc6a14afb22e50200bcbb9192fc4
Reviewed-on: https://go-review.googlesource.com/c/go/+/69910
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-26 06:48:55 +00:00
Emmanuel T Odeke
b2f40019da internal/poll: make SendFile work with large files on Windows
CL 192518 was a minimal simplification to get sendfile
on Windows to work with chunked files, but as I had mentioned,
I would add even more improvements.

This CL improves it by:
* If the reader is not an *io.LimitedReader, since the underlying
reader is anyways an *os.File, we fallback and stat that
file to determine the file size and then also invoke the chunked
sendFile on the underlying reader. This issue existed even
before the prior CL.
* Extracting the chunked TransmitFile logic and moving it directly
into internal/poll.SendFile.

Thus if the callers of net.sendFile don't use *io.LimitedReader,
but have a huge file (>2GiB), we can still invoke the chunked
internal/poll.SendFile on it directly.

The test case is not included in this patch as it requires
creating a 3GiB file, but that if anyone wants to view it, they
can find it at
    https://go-review.googlesource.com/c/go/+/194218/13/src/net/sendfile_windows_test.go

Updates #33193.

Change-Id: I97a67c712d558c84ced716d8df98b040cd7ed7f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/194218
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2019-09-26 06:16:42 +00:00
Tardis Xu
f73d80809b runtime: detail the method comment
Change the comment to make more conformable to the function implementation.

Change-Id: I8461e2f09824c50e16223a27d0f61070f04bd21b
GitHub-Last-Rev: c25a8493d3
GitHub-Pull-Request: golang/go#27404
Reviewed-on: https://go-review.googlesource.com/c/go/+/132477
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-09-26 05:45:42 +00:00
Dan Scales
225f484c88 misc, runtime, test: extra tests and benchmarks for defer
Add a bunch of extra tests and benchmarks for defer, in preparation for new
low-cost (open-coded) implementation of defers (see #34481),

 - New file defer_test.go that tests a bunch more unusual defer scenarios,
   including things that might have problems for open-coded defers.
 - Additions to callers_test.go actually verifying what the stack trace looks like
   for various panic or panic-recover scenarios.
 - Additions to crash_test.go testing several more crash scenarios involving
   recursive panics.
 - New benchmark in runtime_test.go measuring speed of panic-recover
 - New CGo benchmark in cgo_test.go calling from Go to C back to Go that
   shows defer overhead

Updates #34481

Change-Id: I423523f3e05fc0229d4277dd00073289a5526188
Reviewed-on: https://go-review.googlesource.com/c/go/+/197017
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-09-25 23:27:16 +00:00
Michael Munday
8c99e45ef9 cmd/asm: add masked branch and conditional load instructions to s390x
The branch-relative-on-condition (BRC) instruction allows us to use
an immediate to specify under what conditions the branch is taken.
For example, `BRC $7, L1` is equivalent to `BNE L1`. It is sometimes
useful to specify branches in this way when either we don't have
an extended mnemonic for a particular mask value or we want to
generate the condition code mask programmatically.

The new load-on-condition (LOCR and LOCGR) and compare-and-branch
(CRJ, CGRJ, CLRJ, CLGRJ, CIJ, CGIJ, CLIJ and CLGIJ) instructions
provide the same flexibility for conditional loads and combined
compare and branch instructions.

Change-Id: Ic6f5d399b0157e278b39bd3645f4ee0f4df8e5fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/196558
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-25 22:24:41 +00:00
Michael Anthony Knyszek
eb96f8a574 runtime: scavenge on growth instead of inline with allocation
Inline scavenging causes significant performance regressions in tail
latency for k8s and has relatively little benefit for RSS footprint.

We disabled inline scavenging in Go 1.12.5 (CL 174102) as well, but
we thought other changes in Go 1.13 had mitigated the issues with
inline scavenging. Apparently we were wrong.

This CL switches back to only doing foreground scavenging on heap
growth, rather than doing it when allocation tries to allocate from
scavenged space.

Fixes #32828.

Change-Id: I1f5df44046091f0b4f89fec73c2cde98bf9448cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/183857
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-09-25 22:20:36 +00:00
Austin Clements
f18109d7e3 runtime: grow the heap incrementally
Currently, we map and grow the heap a whole arena (64MB) at a time.
Unfortunately, in order to fix #32828, we need to switch from
scavenging inline with allocation back to scavenging on heap growth,
but heap-growth scavenging happens in large jumps because we grow the
heap in large jumps.

In order to prepare for better heap-growth scavenging, this CL
separates mapping more space for the heap from actually "growing" it
(tracking the new space with spans). Instead, growing the heap keeps
track of the "current arena" it's growing into. It track that with new
spans as needed, and only maps more arena space when the current arena
is inadequate. The effect to the user is the same, but this will let
us scavenge on much smaller increments of heap growth.

There are two slightly subtleties to this change:

1. If an allocation requires mapping a new arena and that new arena
   isn't contiguous with the current arena, we don't want to lose the
   unused space in the current arena, so we have to immediately track
   that with a span.

2. The mapped space must be accounted as released and idle, even
   though it isn't actually tracked in a span.

For #32828, since this makes heap-growth scavenging far more
effective, especially at small heap sizes. For example, this change is
necessary for TestPhysicalMemoryUtilization to pass once we remove
inline scavenging.

Change-Id: I300e74a0534062467e4ce91cdc3508e5ef9aa73a
Reviewed-on: https://go-review.googlesource.com/c/go/+/189957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-09-25 22:17:21 +00:00
Michael Anthony Knyszek
9b30811280 runtime: redefine scavenge goal in terms of heap_inuse
This change makes it so that the scavenge goal is defined primarily in
terms of heap_inuse at the end of the last GC rather than next_gc. The
reason behind this change is that next_gc doesn't take into account
fragmentation, and we can fall into situation where the scavenger thinks
it should have work to do but there's no free and unscavenged memory
available.

In order to ensure the scavenge goal still tracks next_gc, we multiply
heap_inuse by the ratio between the current heap goal and the last heap
goal, which describes whether the heap is growing or shrinking, and by
how much.

Finally, this change updates the documentation for scavenging and
elaborates on why the scavenge goal is defined the way it is.

Fixes #34048.
Updates #32828.

Change-Id: I8deaf87620b5dc12a40ab8a90bf27932868610da
Reviewed-on: https://go-review.googlesource.com/c/go/+/193040
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2019-09-25 22:15:39 +00:00
Meng Zhuo
78baea836d internal/bytealg: add assembly implementation of Compare/CompareString on mips64x
name                         old time/op    new time/op    delta
BytesCompare/1                 28.9ns ± 4%    22.8ns ± 0%    -21.23%  (p=0.000 n=9+10)
BytesCompare/2                 34.6ns ± 0%    23.5ns ± 0%    -32.01%  (p=0.000 n=8+10)
BytesCompare/4                 54.6ns ± 0%    41.4ns ± 0%    -24.18%  (p=0.001 n=8+9)
BytesCompare/8                 73.9ns ± 0%    49.7ns ± 0%    -32.75%  (p=0.002 n=8+10)
BytesCompare/16                 113ns ± 0%      23ns ± 0%    -79.56%  (p=0.000 n=9+10)
BytesCompare/32                 190ns ± 0%      26ns ± 0%    -86.53%  (p=0.000 n=10+10)
BytesCompare/64                 345ns ± 0%      44ns ± 0%    -87.19%  (p=0.000 n=10+10)
BytesCompare/128                654ns ± 0%      52ns ± 0%    -91.97%  (p=0.000 n=9+8)
BytesCompare/256               1.27µs ± 0%    0.08µs ± 1%    -94.10%  (p=0.000 n=8+10)
BytesCompare/512               2.51µs ± 0%    0.12µs ± 0%    -95.26%  (p=0.000 n=9+9)
BytesCompare/1024              4.99µs ± 0%    0.21µs ± 1%    -95.84%  (p=0.000 n=8+10)
BytesCompare/2048              9.94µs ± 0%    0.38µs ± 0%    -96.13%  (p=0.000 n=8+10)
CompareBytesEqual               105ns ± 0%      64ns ± 0%    -39.05%  (p=0.000 n=10+9)
CompareBytesToNil              34.8ns ± 1%    39.5ns ± 3%    +13.48%  (p=0.000 n=10+10)
CompareBytesEmpty              33.6ns ± 3%    36.6ns ± 0%     +8.77%  (p=0.000 n=10+10)
CompareBytesIdentical          29.7ns ± 0%    36.6ns ± 0%    +23.23%  (p=0.000 n=10+10)
CompareBytesSameLength         69.1ns ± 0%    51.1ns ± 0%    -26.05%  (p=0.000 n=10+10)
CompareBytesDifferentLength    69.8ns ± 0%    51.1ns ± 0%    -26.79%  (p=0.000 n=10+10)
CompareBytesBigUnaligned       5.15ms ± 0%    2.18ms ± 0%    -57.62%  (p=0.000 n=9+9)
CompareBytesBig                5.28ms ± 0%    0.28ms ± 0%    -94.64%  (p=0.000 n=8+10)
CompareBytesBigIdentical       29.7ns ± 0%    36.8ns ± 0%    +23.91%  (p=0.000 n=8+8)

name                         old speed      new speed      delta
CompareBytesBigUnaligned      204MB/s ± 0%   480MB/s ± 0%   +135.94%  (p=0.000 n=9+9)
CompareBytesBig               198MB/s ± 0%  3703MB/s ± 0%  +1765.85%  (p=0.000 n=8+10)
CompareBytesBigIdentical     35.3TB/s ± 0%  28.5TB/s ± 0%    -19.31%  (p=0.000 n=8+8)

Change-Id: I112d9de2324986fd65ed237a86b11856a1c0f4a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/196837
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-25 22:03:38 +00:00
Keith Randall
44e752c38a runtime: fix ppc64le race code
This code is not currently compiling, the asm vet checks fail. When running race.bash on ppc64le, I get:

runtime/race_ppc64le.s:104:1: [ppc64le] RaceReadRange: wrong argument size 24; expected $...-16
runtime/race_ppc64le.s:514:1: [ppc64le] racecallbackthunk: unknown variable cmd; offset 0 is arg+0(FP)
runtime/race_ppc64le.s:515:1: [ppc64le] racecallbackthunk: unknown variable ctx

I'm also not sure why it ever worked; it looks like it is writing
the arguments to racecallback in the wrong place (the race detector
itself probably still works, it would just have trouble symbolizing
any resulting race report).

At a meta-level, we should really add a ppc64le/race builder.
Otherwise this code will rot, as evidenced by the rot this CL fixes :)

Update #33309

Change-Id: I3b49c2442aa78538fbb631a143a757389a1368fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/197337
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-25 21:39:33 +00:00
Marwan Sulaiman
6232dadc63 cmd/go: consistent output for -json failures
When the -json flag is passed to go mod download,
the sumdb error is embedded in the json Error field.
Other errors for the same command behave this way as
well such as module not found. The fix is done by changing
base.Fatalf into proper error returns.

Fixes #34485

Change-Id: I2727a5c70c7ab03988cad8661894d0f8ec71a768
Reviewed-on: https://go-review.googlesource.com/c/go/+/197062
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-09-25 20:48:49 +00:00
Matthew Dempsky
262046716c cmd/compile: optimize escape graph construction and walking
This CL implements several optimizations for the escape analysis flow
graph:

1. Instead of recognizing heapLoc specially within Escape.outlives,
set heapLoc.escapes = true and recognize any location with escapes
set. This allows us to skip adding edges from the heap to escaped
variables in two cases:

1a. In newLoc, if the location is for a variable or allocation too
large to fit on the stack.

1b. During walkOne, if we discover that an object's address flows
somewhere that naturally outlives it.

2. When recording edges in Escape.flow, if x escapes and we're adding
an edge like "x = &y", we can simply mark that y escapes too.

3. During walkOne, if we reach a location that's marked as escaping,
we can skip visiting it again: we've either already walked from it, or
it's in queue to be walked from again.

On average, reduces the number of visited locations by 15%. Reduces
time spent in escape analysis for particularly hairy packages like
runtime and gc by about 8%. Reduces escape.go's TODO count by 22%.

Passes toolstash-check.

Change-Id: Iaf86a29d76044e4b4c8ab581b916ef5bb5df4437
Reviewed-on: https://go-review.googlesource.com/c/go/+/196811
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-25 18:40:25 +00:00
Matthew Dempsky
867ea9c17f cmd/compile: use proper work queue for escape graph walking
The old escape analysis code used to repeatedly walk the entire flow
graph until it reached a fixed point. With escape.go, I wanted to
avoid this if possible, so I structured the walking code with two
constraints:

1. Always walk from the heap location last.

2. If an object escapes, ensure it has flow edge to the heap location.

This works, but it precludes some graph construction
optimizations. E.g., if there's an assignment "heap = &x", then we can
immediately tell that 'x' escapes without needing to visit it during
the graph walk. Similarly, if there's a later assignment "x = &y", we
could immediately tell that 'y' escapes too. However, the natural way
to implement this optimization ends up violating the constraints
above.

Further, the constraints above don't guarantee that the 'transient'
flag is handled correctly. Today I think that's handled correctly
because of the order that locations happen to be constructed and
visited based on the AST, but I've felt uneasy about it for a little
while.

This CL changes walkAll to use a proper work queue (technically a work
stack) to track locations that need to be visited, and allows walkOne
to request that a location be re-visited.

Passes toolstash-check.

Change-Id: Iaa6f4d3fe4719c04d67009fb9a2a3e4930b3d7c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/196958
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-25 18:39:48 +00:00
Than McIntosh
fad0a14d92 test: add testcase for gccgo compiler buglet
New test containing code that caused a gccgo compiler failure.

Updates #34503.

Change-Id: Id895a1e1249062b7fb147e54bcaa657e774ed0d9
Reviewed-on: https://go-review.googlesource.com/c/go/+/197217
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-25 18:10:57 +00:00
Cuong Manh Le
e0dde990de cmd/compile: remove n.SetLikely(false) usage
n.SetLikely(false) is probably mean to indicate that the branch is
"unlikely", but it has the real effect of not marking branch as likely.

So invert the test condition, we can use more meaningful n.SetLikely(true).

Before:

	if l2 < 0 {
		panicmakeslicelen()
	}

After:

	if l2 >= 0 {
	} else {
		panicmakeslicelen
	}

Fixes #32486

Change-Id: I156fdba1f9a5d554a178c8903f1a391ed304199d
Reviewed-on: https://go-review.googlesource.com/c/go/+/195197
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-09-25 18:10:51 +00:00
Jeremy Faller
21bf37b5a2 cmd/compile: update object file format for DWARF file table
In CL 188317, we generate the debug_lines in the compiler, and created a
new symbol to hold the line table. Here we modify the object file format
to output the file table.

Change-Id: Ibee192e80b86ff6af36467a0b1c26ee747dfee37
Reviewed-on: https://go-review.googlesource.com/c/go/+/191167
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-25 17:35:25 +00:00
Matthew Dempsky
efb9739203 cmd/compile: use underlying OCOMPLIT's position for OPTRLIT
Currently, when we create an OPTRLIT node, it defaults to the
OCOMPLIT's final element's position. But it improves error messages to
use the OCOMPLIT's own position instead.

Change-Id: Ibb031f543c7248d88d99fd0737685e01d86e2500
Reviewed-on: https://go-review.googlesource.com/c/go/+/197119
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-25 17:07:09 +00:00
Matthew Dempsky
3e428363c4 cmd/compile: remove -s flag
This is better handled by tools like cmd/gofmt, which can
automatically rewrite the source code and already supports a syntactic
version of this simplification. (go/types can be used if
type-sensitive simplification is actually necessary.)

Change-Id: I51332a8f3ff4ab3087bc6b43a491c6d92b717228
Reviewed-on: https://go-review.googlesource.com/c/go/+/197118
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-09-25 17:07:00 +00:00
Matthew Dempsky
f346a4c44c test: add regress test for #27557
This commit just adds a regress test for a few of the important corner
cases that I identified in #27557, which turn out to not be tested
anywhere.

While here, annotate a few of the existing test cases where we could
improve escape analysis.

Updates #27557.

Change-Id: Ie57792a538f7899bb17915485fabc86100f469a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/197137
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-25 17:06:15 +00:00
Bryan C. Mills
8189a06190 cmd/go/internal/modload: annotate replacements in PackageNotInModuleError
Fixes #34085

Change-Id: I3111f5997466ad33f51e80c71f5fb2ccebdcc6e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/193617
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-09-25 16:33:28 +00:00
Daniel Martí
0f7b4e72a0 text/template: don't evaluate '.' as a float64
When using a '.' constant literal as a reflect.Value variadic argument,
idealConstant would incorrectly result in a float64. This is because
rune literals can be represented as a float64, and contain a period,
which tricked the logic into thinking the literal must have been a
floating point number.

This also happened with other characters that can be part of a floating
point number, such as 'e' or 'P'.

To fix these edge cases, exit the case sooner if the literal was a rune,
since that should always go to the int case instead.

Finally, add test cases that verify that they behave properly. These
would error before, since eq would receive a mix of int and float64,
which aren't comparable.

Fixes #34483.

Change-Id: Icfcb7803bfa0cf317a1d1adacacad3d69a57eb42
Reviewed-on: https://go-review.googlesource.com/c/go/+/196808
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tom Payne <tom@airmap.com>
Reviewed-by: Rob Pike <r@golang.org>
2019-09-25 09:22:38 +00:00
Ariel Mashraki
95cbcc5c1c text/template: support all comparable types in eq
Extends the built-in eq function to support all Go
comparable types.

Fixes #33740

Change-Id: I522310e313e251c4dc6a013d33d7c2034fe2ec8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/193837
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2019-09-25 09:13:11 +00:00
Joel Sing
434e4caf1b cmd/internal/obj/riscv: implement floating point instructions
Add support for assembling various single-precision and double-precision
floating point instructions.

Based on the riscv-go port.

Updates #27532

Change-Id: Iac1aec9b03bb6cbf116b229daeef944d4df550fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/196839
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-25 08:25:43 +00:00
Tobias Klauser
8f5755e76e runtime: gofmt after CL 192937
CL 192937 introduced some changes which weren't properly gofmt'ed. Do so
now.

Change-Id: I2d2d57ea8a79fb41bc4ca59fa23f12198d615fd8
Reviewed-on: https://go-review.googlesource.com/c/go/+/196812
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-25 07:05:18 +00:00
Brad Fitzpatrick
7fc2625ef1 net/http: propagate Client.Timeout down into Request's context deadline
Fixes #31657

Change-Id: I85e9595d3ea30d410f1f4b787925a6879a72bdf2
Reviewed-on: https://go-review.googlesource.com/c/go/+/175857
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-25 04:18:18 +00:00
Cuong Manh Le
caf45cde18 cmd/compile: add documentation for isfat
In CL 192980, I tend to think that canSSAType can be used as replacement
for isfat. It is not the truth as @khr points me out that isfat has very
different purpose.

So this CL adds documentation for isfat, also remove outdated TODO.

Change-Id: I15954d638759bd9f6b28a6aa04c1a51129d9ae7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/196499
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-09-25 03:53:20 +00:00
Alan Donovan
5477623031 log: mention names of flag constants in {Set,}Flags doc comments
Change-Id: I1217f07530dc7586fd7b933bc6a65bad163782db
Reviewed-on: https://go-review.googlesource.com/c/go/+/47232
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-09-24 23:29:19 +00:00
Jeremy Faller
ee3dded36d cmd/compile: generate debug_lines in compiler
This is mostly a copy-paste jobs from the linker to generate the debug
information in the compiler instead of the linker. The new data is
inserted into the debug line numbers symbol defined in CL 188238.

Generating the debug information BEFORE deadcode results in one subtle
difference, and that is that the state machine needs to be reset at the
end of every function's debug line table. The reasoning is that
generating the table AFTER dead code allows the producer and consumer of
the table to agree on the state of the state machine, and since these
blocks will (eventually) be concatenated in the linker, we don't KNOW
the state of the state machine unless we reset it. So,
generateDebugLinesSymbol resets the state machine at the end of every
function.

Right now, we don't do anything with this line information, or the file
table -- we just populate the symbols.

Change-Id: If9103eda6cc5f1f7a11e7e1a97184a060a4ad7fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/188317
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-09-24 22:14:05 +00:00
Jeremy Faller
78a3734714 cmd/link: add notion of multiple compilation units per package
As we move the debug_line generation into the compiler, we need to
upgrade the notion of compilationUnit to not just be on a per package
basis.  That won't be the case as it will be impossible for all
compilationUnits to have the same set of files names used to build the
debug_lines table. (For example, assembled files in a package don't know
about any files but themselves, so the debug_lines table could only
reference themseves. As such, we need to break the 1:1 relationship
between compUnit and package.)

Change-Id: I2e517bb6c01de0115bbf777af828a2fe59c09ce8
Reviewed-on: https://go-review.googlesource.com/c/go/+/189618
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-09-24 21:33:10 +00:00
Lynn Boger
d979ac33a2 crypto/cipher: improve xorBytesVSX asm for ppc64x
This improves the performance of xorBytesVSX in crypto/cipher by
unrolling the loop that does the stores. Improvement on power9:

name                 old time/op    new time/op    delta
XORBytes/8Bytes        17.9ns ± 0%    18.2ns ± 0%   +1.53%  (p=0.029 n=4+4)
XORBytes/128Bytes      24.4ns ± 0%    22.5ns ± 0%   -7.79%  (p=0.029 n=4+4)
XORBytes/2048Bytes      131ns ± 0%     109ns ± 0%  -16.79%  (p=0.029 n=4+4)
XORBytes/32768Bytes    1.74µs ± 0%    1.43µs ± 8%  -18.04%  (p=0.029 n=4+4)

Change-Id: I75bd625d3ae9daa7bda54c523028671ab036b13d
Reviewed-on: https://go-review.googlesource.com/c/go/+/197058
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
2019-09-24 21:30:45 +00:00
Bryan C. Mills
a3426f2571 cmd/go: suppress errors in package-to-module queries if the package is already found
In CL 173017, I changed the package-to-module query logic to query all
possible module paths in parallel in order to reduce latency. (For
long package paths, most such paths will not exist and will fail with
little overhead.)

The module resolution algorithm treats various kinds of non-existence
as “soft errors”, to be reported only if package resolution fails, but
treats any remaining errors as hard errors that should fail the query.

Unfortunately, that interacted badly with the +incompatible version
validation added in CL 181881, causing a regression in the 'direct'
fetch path for modules using the “major branch” layout¹ with a post-v1
version on the repository's default branch. Because we did not
interpret a mismatched module path as “no such module”, a go.mod file
specifying the path 'example.com/foo/v2' would cause the search for
module 'example.com/foo' to error out. (That regression was not caught
ahead of time due to a lack of test coverage for 'go get' on a package
within a /vN module.)

The promotion of hard errors during parallel search also made the 'go'
command less tolerant of servers that advertise 'go-import' tags for
nonexistent repositories. CL 194561 mitigated that problem for HTTP
servers that return code 404 or 410 for a nonexistent repository, but
unfortunately a few servers in common use (notably GitLab and
pre-1.9.3 releases of Gitea) do not.

This change mitigates both of those failure modes by ignoring
“miscellaneous” errors from shorter module paths if the requested
package pattern was successfully matched against a module with a
longer path.

¹https://research.swtch.com/vgo-module#from_repository_to_modules

Updates #34383
Updates #34094

Change-Id: If37dc422e973eba13f3a3aeb68bc7b96e2d7f73d
Reviewed-on: https://go-review.googlesource.com/c/go/+/197059
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-09-24 20:59:21 +00:00