This fixes handling of cgo flags and makes sure packages that are only
implicitly included in the shared library are passed to the link.
Fixes#15885
Change-Id: I1e8a72b5314261973ca903c78834700fb113dde9
Reviewed-on: https://go-review.googlesource.com/23537
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
golang.org/issue/15443 complained that a race-enabled PIE binary crashed at
startup, but other ways of linking in tsan (or other sanitizers) such as
#cgo CFLAGS: -fsanitize=thread
#cgo LDFLAGS: -fsanitize=thread
have the same problem. Pass -no-pie to the host linker (if supported) if any
-fsanitizer=foo cgo LDFLAG is seen when linking.
Fixes#15887
Change-Id: Id799770f8d045f6f40fa8c463563937a5748d1a8
Reviewed-on: https://go-review.googlesource.com/23535
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Add TSAN acquire/release calls to runtime/cgo to match the ones
generated by cgo. This avoids a false positive race around the malloc
memory used in runtime/cgo when other goroutines are simultaneously
calling malloc and free from cgo.
These new calls will only be used when building with CGO_CFLAGS and
CGO_LDFLAGS set to -fsanitize=thread, which becomes a requirement to
avoid all false positives when using TSAN. These are needed not just
for runtime/cgo, but also for any runtime package that uses cgo (such as
net and os/user).
Add an unused attribute to the _cgo_tsan_acquire and _cgo_tsan_release
functions, in case there are no actual cgo function calls.
Add a test that checks that setting CGO_CFLAGS/CGO_LDFLAGS avoids a
false positive report when using os/user.
Change-Id: I0905c644ff7f003b6718aac782393fa219514c48
Reviewed-on: https://go-review.googlesource.com/23492
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
In order to support pprof for position independent executables, pprof
needs to adjust the PC addresses stored in the profile by the address at
which the program is loaded. The legacy profiling support which we use
already supports recording the GNU/Linux /proc/self/maps data
immediately after the CPU samples, so do that. Also change the pprof
symbolizer to use the information, if available, when looking up
addresses in the Go pcline data.
Fixes#15714.
Change-Id: I4bf679210ef7c51d85cf873c968ce82db8898e3e
Reviewed-on: https://go-review.googlesource.com/23525
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
When we do *p = f(), we might need to copy the return value from
f to p with a write barrier. The write barrier itself is a call,
so we need to copy the return value of f to a temporary location
before we call the write barrier function. Otherwise, the call
itself (specifically, marshalling the args to typedmemmove) will
clobber the value we're trying to write.
Fixes#15854
Change-Id: I5703da87634d91a9884e3ec098d7b3af713462e7
Reviewed-on: https://go-review.googlesource.com/23522
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This was just storage for a linked list.
Change-Id: I850e8db1e1f5e72410f5c904be9409179b56a94a
Reviewed-on: https://go-review.googlesource.com/23484
Reviewed-by: David Chase <drchase@google.com>
The main check here is that liveness now crashes if it finds an instruction
using a variable that should be tracked but is not.
Comments and adjustments in nodarg to explain what's going on and
to remove the "-1" argument added a few months ago, plus a sketch
of a future simplification.
The need for n.Orig in the earlier CL seems to have been an intermediate
problem rather than fundamental: the new explanations in nodarg make
clear that nodarg is not causing the problem I thought, and in fact now
using n instead of n.Orig works fine in plive.go.
Change-Id: I3f5cf9f6e4438a6d27abac7d490e7521545cd552
Reviewed-on: https://go-review.googlesource.com/23450
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This is a fixup change for commit 5cd2944803
that added parsing of SCP-like addresses. To get the expected output
from (*url.URL).String(), Path needs to be set, not RawPath.
Add a test for this, since it has already regressed multiple times.
Updates #11457.
Change-Id: I806f5abbd3cf65e5bdcef01aab872caa8a5b8891
Reviewed-on: https://go-review.googlesource.com/23447
Run-TryBot: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
As in the elimination of PHEAP|PPARAM in CL 23393,
this is something the front end can trivially take care of
and then not bother the back ends with.
It also eliminates some suspect (and only lightly exercised)
code paths in the back ends.
I don't have a smoking gun for this one but it seems
more clearly correct.
Change-Id: I3b3f5e669b3b81d091ff1e2fb13226a6f14c69d5
Reviewed-on: https://go-review.googlesource.com/23431
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
The liveness computation of parameters generally was never
correct, but forcing all parameters to be live throughout the
function covered up that problem. The new SSA back end is
too clever: even though it currently keeps the parameter values live
throughout the function, it may find optimizations that mean
the current values are not written back to the original parameter
stack slots immediately or ever (for example if a parameter is set
to nil, SSA constant propagation may replace all later uses of the
parameter with a constant nil, eliminating the need to write the nil
value back to the stack slot), so the liveness code must now
track the actual operations on the stack slots, exposing these
problems.
One small problem in the handling of arguments is that nodarg
can return ONAME PPARAM nodes with adjusted offsets, so that
there are actually multiple *Node pointers for the same parameter
in the instruction stream. This might be possible to correct, but
not in this CL. For now, we fix this by using n.Orig instead of n
when considering PPARAM and PPARAMOUT nodes.
The major problem in the handling of arguments is general
confusion in the liveness code about the meaning of PPARAM|PHEAP
and PPARAMOUT|PHEAP nodes, especially as contrasted with PAUTO|PHEAP.
The difference between these two is that when a local variable "moves"
to the heap, it's really just allocated there to start with; in contrast,
when an argument moves to the heap, the actual data has to be copied
there from the stack at the beginning of the function, and when a
result "moves" to the heap the value in the heap has to be copied
back to the stack when the function returns
This general confusion is also present in the SSA back end.
The PHEAP bit worked decently when I first introduced it 7 years ago (!)
in 391425ae. The back end did nothing sophisticated, and in particular
there was no analysis at all: no escape analysis, no liveness analysis,
and certainly no SSA back end. But the complications caused in the
various downstream consumers suggest that this should be a detail
kept mainly in the front end.
This CL therefore eliminates both the PHEAP bit and even the idea of
"heap variables" from the back ends.
First, it replaces the PPARAM|PHEAP, PPARAMOUT|PHEAP, and PAUTO|PHEAP
variable classes with the single PAUTOHEAP, a pseudo-class indicating
a variable maintained on the heap and available by indirecting a
local variable kept on the stack (a plain PAUTO).
Second, walkexpr replaces all references to PAUTOHEAP variables
with indirections of the corresponding PAUTO variable.
The back ends and the liveness code now just see plain indirected
variables. This may actually produce better code, but the real goal
here is to eliminate these little-used and somewhat suspect code
paths in the back end analyses.
The OPARAM node type goes away too.
A followup CL will do the same to PPARAMREF. I'm not sure that
the back ends (SSA in particular) are handling those right either,
and with the framework established in this CL that change is trivial
and the result clearly more correct.
Fixes#15747.
Change-Id: I2770b1ce3cbc93981bfc7166be66a9da12013d74
Reviewed-on: https://go-review.googlesource.com/23393
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The cgo tool generates compiler errors to find out what kind of name it
is using. Turning on optimization can confuse that process by producing
new unexpected messages.
Fixes#14669.
Change-Id: Idc8e35fd259711ecc9638566b691c11d17140325
Reviewed-on: https://go-review.googlesource.com/23231
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Add a test which compiles a function and checks the
generated assembly to make sure certain patterns are present.
This test allows us to do white box tests of the compiler
to make sure optimizations don't regress.
Added a few simple tests for now. More to come.
Change-Id: I4ab5ce5d95b9e04e7d0d9328ffae47b8d1f95e74
Reviewed-on: https://go-review.googlesource.com/23403
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When rules are generated with -log, log rule application to a file.
The file is opened in append mode so multiple calls to the compiler
union their logs.
Change-Id: Ib35c7c85bf58e5909ea9231043f8cbaa6bf278b7
Reviewed-on: https://go-review.googlesource.com/23406
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
domorder has some non-obvious useful properties
that we’re relying on in cse.
Document them and provide an argument that they hold.
While we’re here, do some minor renaming.
The argument is a re-working of a private email
exchange with Todd Neal and David Chase.
Change-Id: Ie154e0521bde642f5f11e67fc542c5eb938258be
Reviewed-on: https://go-review.googlesource.com/23449
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
I'm glad my CL fixed the library use case inside Google.
It fixes neither of the two tests here.
Change-Id: Ica91722dced8955a0a8ba3aad3d288816b46564e
Reviewed-on: https://go-review.googlesource.com/23482
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This has a minor performance cost, but far less than is being gained by SSA.
As an experiment, enable it during the Go 1.7 beta.
Having frame pointers on by default makes Linux's perf, Intel VTune,
and other profilers much more useful, because it lets them gather a
stack trace efficiently on profiling events.
(It doesn't help us that much, since when we walk the stack we usually
need to look up PC-specific information as well.)
Fixes#15840.
Change-Id: I4efd38412a0de4a9c87b1b6e5d11c301e63f1a2a
Reviewed-on: https://go-review.googlesource.com/23451
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It is timing out on the dashboard.
(We enabled it as an experiment to see if it was still broken. Looks that way.)
Change-Id: I425b7e54a2ab95b623ab7a15554b4173078f75e2
Reviewed-on: https://go-review.googlesource.com/23480
Reviewed-by: Russ Cox <rsc@golang.org>
The offsets computed by the DWARF expressions for local variables
currently don't account for the extra stack slot used by the frame
pointer when GOEXPERIMENT=framepointer is enabled.
Fix this by adding the extra stack slot to the offset.
This fixes TestGdbPython with GOEXPERIMENT=framepointer.
Updates #15840.
Change-Id: I1b2ebb2750cd22266f4a89ec8d9e8bfa05fabd19
Reviewed-on: https://go-review.googlesource.com/23458
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A few other architectures have already defined a NOFRAME flag.
Use it to disable frame pointer code on a few very low-level functions
that must behave like Windows code.
Makes the failing os/signal test pass on a Windows gomote.
Change-Id: I982365f2c59a0aa302b4428c970846c61027cf3e
Reviewed-on: https://go-review.googlesource.com/23456
Reviewed-by: Austin Clements <austin@google.com>
I have been running this patch inside Google against Go 1.6 for the last month.
The new tests will probably break the builders but let's see
exactly how they break.
Change-Id: Ia65cf7d3faecffeeb4b06e9b80875c0e57d86d9e
Reviewed-on: https://go-review.googlesource.com/23452
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The importer had several bugs with respect to labels and gotos:
- it didn't create a new ONAME node for label names (label dcl,
goto, continue, and break)
- it overwrote the symbol for gotos with the dclstack
- it didn't set the dclstack for labels
In the process changed export format slightly to always assume
a label name for labels and gotos, and never assume a label for
fallthroughs.
For fallthroughs and switch cases, now also set Xoffset like in
the parser. (Not setting it, i.e., using 0 was ok since this is
only used for verifying correct use of fallthroughs, which was
checked already. But it's an extra level of verification of the
import.)
Fixes#15838.
Change-Id: I3637f6314b8651c918df0c8cd70cd858c92bd483
Reviewed-on: https://go-review.googlesource.com/23445
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Acquire and release the TSAN synchronization point when calling malloc,
just as we do when calling any other C function. If we don't do this,
TSAN will report false positive errors about races calling malloc and
free.
We used to have a special code path for malloc and free, going through
the runtime functions cmalloc and cfree. The special code path for cfree
was no longer used even before this CL. This CL stops using the special
code path for malloc, because there is no place along that path where we
could conditionally insert the TSAN synchronization. This CL removes
the support for the special code path for both functions.
Instead, cgo now automatically generates the malloc function as though
it were referenced as C.malloc. We need to automatically generate it
even if C.malloc is not called, even if malloc and size_t are not
declared, to support cgo-provided functions like C.CString.
Change-Id: I829854ec0787a80f33fa0a8a0dc2ee1d617830e2
Reviewed-on: https://go-review.googlesource.com/23260
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Covers a bunch of constant-folding rules in generic.rules that aren't
being covered currently.
Increases coverage in generic.rules from 65% to 72%.
Change-Id: I7bf58809faf22e97070183b42e6dd7d3f35bf5f9
Reviewed-on: https://go-review.googlesource.com/23407
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
CL 23400 introduced a check to make sure the gold linker is used
on ARM host links. The check itself works, but the error checking
logic was reversed; fix it.
I manually verified that the check now correctly rejects host links
on my RPi2 running an ancient rasbian without the gold linker
installed.
Updates #15696
Change-Id: I927832620f0a60e91a71fdedf8cbd2550247b666
Reviewed-on: https://go-review.googlesource.com/23421
Run-TryBot: Elias Naur <elias.naur@gmail.com>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Document the correct use of the testdata directory
where test writers might be expecting to find it.
It seems that alldocs.go was out of date, so it
has picked up some other changes with this commit.
Fixes#14715.
Change-Id: I0a22676bb7a64b2a61b56495f7ea38db889d8b37
Reviewed-on: https://go-review.googlesource.com/23353
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 21462 and CL 21463 made this message say explicitly that the problem
was a struct field in a map, but the word "directly" is unnecessary,
sounds wrong, and makes the error long.
Change-Id: I2fb68cdaeb8bd94776b8022cf3eae751919ccf6f
Reviewed-on: https://go-review.googlesource.com/23373
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Chase <drchase@google.com>
The use of a prefix check was too liberal.
Noted in review after submit.
Change-Id: I4fe1df660997efd225609e818040b8392fab79f0
Reviewed-on: https://go-review.googlesource.com/23375
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Now that CSE uses dom tree to order partitions, we need the
dom tree computed before benchmarking CSE.
Fixes#15801
Change-Id: Ifa4702c7b75250f34de185e69a880b3f3cc46a12
Reviewed-on: https://go-review.googlesource.com/23361
Reviewed-by: David Chase <drchase@google.com>
Ignore respective bit in export data, but leave the info to
minimize format changes for 1.7. Scheduled to remove by 1.8.
For #15772.
Change-Id: Ifb3beea655367308a4e2d5dc8cb625915f904287
Reviewed-on: https://go-review.googlesource.com/23285
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Non-syntax errors are always counted to determine if to exit
early, but then deduplication eliminates them. This can lead
to situations which report "too many errors" and only one
error is shown.
De-duplicate non-syntax errors early, at least the ones that
appear consecutively, and only count the ones actually being
shown. This doesn't work perfectly as they may not appear in
sequence, but it's cheap and good enough.
Fixes#14136.
Change-Id: I7b11ebb2e1e082f0d604b88e544fe5ba967af1d7
Reviewed-on: https://go-review.googlesource.com/23259
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
When the generated stub functions write back the results to the stack,
they can in some cases be writing to the same memory on the g0 stack.
There is no race here (assuming there is no race in the Go code), but
the thread sanitizer does not know that. Turn off the thread sanitizer
for the stub functions to prevent false positive warnings.
Current clang suggests the no_sanitize("thread") attribute, but that
does not work with clang 3.6 or GCC. clang 3.6, GCC, and current clang
all support the no_sanitize_thread attribute, so use that
unconditionally.
The test case and first version of the patch are from Dmitriy Vyukov.
Change-Id: I80ce92824c6c8cf88ea0fe44f21cf50cf62474c9
Reviewed-on: https://go-review.googlesource.com/23252
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
They get rewritten to NEWs, and they must be marked as escaping
so walk doesn't try to allocate them back onto the stack.
Fixes#15733
Change-Id: I433033e737c3de51a9e83a5a273168dbc9110b74
Reviewed-on: https://go-review.googlesource.com/23223
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run live vars test only on ssa builds.
We can't just drop KeepAlive ops during regalloc. We need
to replace them with copies.
Change-Id: Ib4b3b1381415db88fdc2165fc0a9541b73ad9759
Reviewed-on: https://go-review.googlesource.com/23225
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Introduce a KeepAlive op which makes sure that its argument is kept
live until the KeepAlive. Use KeepAlive to mark pointer input
arguments as live after each function call and at each return.
We do this change only for pointer arguments. Those are the
critical ones to handle because they might have finalizers.
Doing compound arguments (slices, structs, ...) is more complicated
because we would need to track field liveness individually (we do
that for auto variables now, but inputs requires extra trickery).
Turn off the automatic marking of args as live. That way, when args
are explicitly nulled, plive will know that the original argument is
dead.
The KeepAlive op will be the eventual implementation of
runtime.KeepAlive.
Fixes#15277
Change-Id: I5f223e65d99c9f8342c03fbb1512c4d363e903e5
Reviewed-on: https://go-review.googlesource.com/22365
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Go is being proposed as an officially supported language for elements of
OpenStack:
https://review.openstack.org/#/c/312267/
As such, repos that exist in OpenStack's git infrastructure
are likely to become places from which people might want to go get
things. Allow optional .git suffixes to allow writing code that depends
on git.openstack.org repos that will work with older go versions while
we wait for this support to roll out.
Change-Id: Ia64bdb1dafea33b1c3770803230d30ec1059df22
Reviewed-on: https://go-review.googlesource.com/23135
Reviewed-by: Russ Cox <rsc@golang.org>
sparseSet and sparseMap only need 32 bit integers in their
arrays, since a sparseEntry key is also limited to 32 bits.
This appears to reduce the space allocated for at least
one pathological compilation by 1%, perhaps more.
Not necessarily for 1.7, but it saves a little and is very
low-risk.
Change-Id: Icf1185859e9f5fe1261a206b441e02c34f7d02fd
Reviewed-on: https://go-review.googlesource.com/22972
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Previously statements like
f(unsafe.Pointer(g()), int(h()))
would be reordered into a sequence of statements like
autotmp_g := g()
autotmp_h := h()
f(unsafe.Pointer(autotmp_g), int(autotmp_h))
which can leave g's temporary value on the stack as a uintptr, rather
than an unsafe.Pointer. Instead, recognize uintptr-to-unsafe.Pointer
conversions when reordering function calls to instead produce:
autotmp_g := unsafe.Pointer(g())
autotmp_h := h()
f(autotmp_g, int(autotmp_h))
Fixes#15329.
Change-Id: I2cdbd89d233d0d5c94791513a9fd5fd958d11ed5
Reviewed-on: https://go-review.googlesource.com/22273
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This adds a sparse method for locating nearest ancestors
in a dominator tree, and checks blocks with more than one
predecessor for differences and inserts phi functions where
there are.
Uses reversed post order to cut number of passes, running
it from first def to last use ("last use" for paramout and
mem is end-of-program; last use for a phi input from a
backedge is the source of the back edge)
Includes a cutover from old algorithm to new to avoid paying
large constant factor for small programs. This keeps normal
builds running at about the same time, while not running
over-long on large machine-generated inputs.
Add "phase" flags for ssa/build -- ssa/build/stats prints
number of blocks, values (before and after linking references
and inserting phis, so expansion can be measured), and their
product; the product governs the cutover, where a good value
seems to be somewhere between 1 and 5 million.
Among the files compiled by make.bash, this is the shape of
the tail of the distribution for #blocks, #vars, and their
product:
#blocks #vars product
max 6171 28180 173,898,780
99.9% 1641 6548 10,401,878
99% 463 1909 873,721
95% 152 639 95,235
90% 84 359 30,021
The old algorithm is indeed usually fastest, for 99%ile
values of usually.
The fix to LookupVarOutgoing
( https://go-review.googlesource.com/#/c/22790/ )
deals with some of the same problems addressed by this CL,
but on at least one bug ( #15537 ) this change is still
a significant help.
With this CL:
/tmp/gopath$ rm -rf pkg bin
/tmp/gopath$ time go get -v -gcflags -memprofile=y.mprof \
github.com/gogo/protobuf/test/theproto3/combos/...
...
real 4m35.200s
user 13m16.644s
sys 0m36.712s
and pprof reports 3.4GB allocated in one of the larger profiles
With tip:
/tmp/gopath$ rm -rf pkg bin
/tmp/gopath$ time go get -v -gcflags -memprofile=y.mprof \
github.com/gogo/protobuf/test/theproto3/combos/...
...
real 10m36.569s
user 25m52.286s
sys 4m3.696s
and pprof reports 8.3GB allocated in the same larger profile
With this CL, most of the compilation time on the benchmarked
input is spent in register/stack allocation (cumulative 53%)
and in the sparse lookup algorithm itself (cumulative 20%).
Fixes#15537.
Change-Id: Ia0299dda6a291534d8b08e5f9883216ded677a00
Reviewed-on: https://go-review.googlesource.com/22342
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Trace viewer cannot handle traces larger than 256MB (limit on js string size):
https://github.com/catapult-project/catapult/issues/627
And even that is problematic (chrome hangs and crashes).
Split large traces into 100MB parts. Somewhat clumsy, but I don't see any other
solution (other than rewriting trace viewer). At least it works reliably now.
Fixes#15482
Change-Id: I993b5f43d22072c6f5bd041ab5888ce176f272b2
Reviewed-on: https://go-review.googlesource.com/22731
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Step 5 was deleted in f3575a9 however the numbering of the other
steps wasn't adjusted accordingly.
While we're here: clean up the whitespace, add curly braces where
appropriate and delete semicolons.
Change-Id: I4e77b2d3ee8460abe4bfb993674f83e35be8ff17
Reviewed-on: https://go-review.googlesource.com/23066
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Minor cleanup. Each of these cases appears both during export and
import when running all.bash and thus is tested by all.bash.
Change-Id: Iaa4a5a5b163cefe33e43d08d396e02a02e5c22a5
Reviewed-on: https://go-review.googlesource.com/23060
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is addressing feedback given on golang.org/cl/23052;
we do it in a separate CL to separate the functional from
the rename change.
ONAME was not used in the export data, but it's the natural node op
where we used OPACK instead. Renamed.
Furthermore, OPACK and ONONAME nodes are replaced by the type checker
with ONAME nodes, so OPACK nodes cannot occur when exporting type-checked
code. Removed a special-case for OPACK nodes since they don't appear.
Change-Id: I78b01a1badbf60e9283eaadeca2578a65d28cbd2
Reviewed-on: https://go-review.googlesource.com/23053
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
The pprof tools properly cleans up all files it creates, but forgets
to clean up the temporary directory itself. This CL fixes that.
Fixes#13863
Change-Id: I1151c36cdad5ace7cc97e7e04001cf0149ef0f63
Reviewed-on: https://go-review.googlesource.com/23019
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Include integration test. Confirmed that without the fix, the test case
TestDeleteReadOnly fails.
This permits to revert "cmd/go: reset read-only flag during TestIssue10952"
This reverts commit 3b7841b3af.
Fixes#9606
Change-Id: Ib55c151a8cf1a1da02ab18c34a9b58f615c34254
Reviewed-on: https://go-review.googlesource.com/18235
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The //extern comments are incorrect and cause undefined symbol
errorswhen building cgo code with -compiler=gccgo. The code is already
designed to use weak references, and that support relies on the cgo
check functions being treated as local functions.
Change-Id: Ib38a640cc4ce6eba74cfbf41ba7147ec88769ec0
Reviewed-on: https://go-review.googlesource.com/23014
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In regalloc, a sparse map is preallocated for later use by
spill-in-loop sinking. However, variables (spills) are added
during register allocation before spill sinking, and a map
query involving any of these new variables will index out of
bounds in the map.
To fix:
1) fix the queries to use s.orig[v.ID].ID instead, to ensure
proper indexing. Note that s.orig will be nil for values
that are not eligible for spilling (like memory and flags).
2) add a test.
Fixes#15585.
Change-Id: I8f2caa93b132a0f2a9161d2178320d5550583075
Reviewed-on: https://go-review.googlesource.com/22911
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This flag is experimental and the semantics may change
even after Go 1.7 is released. There are no changes to code
not using the flag.
The first part is for reading by future compiles.
The second part is for reading by the final link step.
Splitting the file this way allows distributed build systems
to ship the compile-input part only to compile steps and
the linker-input part only to linker steps.
The first part is basically just the export data,
and the second part is basically everything else.
The overall files still have the same broad structure,
so that existing tools will work with both halves.
It's just that various pieces are empty in the two halves.
This also copies the two bits of data the linker needed from
export data into the object header proper, so that the linker
doesn't need any export data at all. That eliminates a TODO
that was left for switching to the binary export data.
(Now the linker doesn't need to know about the switch.)
The default is still to write out a combined output file.
Nothing changes unless you pass -linkobj to the compiler.
There is no support in the go command for -linkobj,
since the go command doesn't copy objects around.
The expectation is that other build systems (like bazel, say)
might take advantage of this.
The header adjustment and the option for the split output
was intended as part of the zip archives, but the zip archives
have been cut from Go 1.7. Doing this to the current archives
both unblocks one step in the switch to binary export data
and enables alternate build systems to experiment with the
new flag using the Go 1.7 release.
Change-Id: I8b6eab25b8a22b0a266ba0ac6d31e594f3d117f3
Reviewed-on: https://go-review.googlesource.com/22500
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Currently 386 ELF binaries are generated with dynamic symbols that have
a size of zero bytes, even though the symbol in the symbol table has
the correct size. Fix this by specifying the correct size when creating
dynamic symbols.
Issue found on OpenBSD -current, where ld.so is now producing link
warnings due to mismatched symbol sizes.
Fixes#15593.
Change-Id: Ib1a12b23ff9159c61ac980bf48a983b86f3df256
Reviewed-on: https://go-review.googlesource.com/22912
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The new export format keeps track of all types that are exported.
If a type is seen that was exported before, only a reference to
that type is emitted. The importer maintains a list of all the
seen types and uses that list to resolve type references.
The existing compiler infrastructure's invariants assumes that
only named types are referred to before they are fully set up.
Referring to unnamed incomplete types causes problems. One of
the issues was #15548.
Added a new internal flag 'trackAllTypes' to enable/disable
this type tracking. With this change only named types are
tracked.
Verified that this fix also addresses #15548, even w/o the
prior fix for that issue (in fact that prior fix is turned
off if trackAllTypes is disabled because it's not needed).
The test for #15548 covers also this change.
For #15548.
Change-Id: Id0b3ff983629703d025a442823f99649fd728a56
Reviewed-on: https://go-review.googlesource.com/22839
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Buildmode c-archive now supports position independent code for
darwin/arm (in addition to darwin/arm64). Make PIC (-shared) the
default for both platforms in the default buildmode.
Without this change, gomobile will go install the standard library
into its separate package directory without PIC support.
Also add -shared to darwin/arm64 in buildmode c-archive, for
symmetry (darwin/arm64 always generates position independent code).
Fixes#15519
Change-Id: If27d2cbea8f40982e14df25da2703cbba572b5c6
Reviewed-on: https://go-review.googlesource.com/22920
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The boolean destination in an OAS2DOTTYPE expression craps out during
compilation when trying to assign to a map entry because, unlike slice entries,
map entries are not directly addressable in memory. The solution is to
properly order the boolean destination node so that map entries are set
via autotmp variables.
Fixes#14678
Change-Id: If344e8f232b5bdac1b53c0f0d21eeb43ab17d3de
Reviewed-on: https://go-review.googlesource.com/22833
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes#12272
Change-Id: I2115ec62ed4061084c482eb385a583a1c1909888
Reviewed-on: https://go-review.googlesource.com/22838
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
CL/19862 (f79b50b8d5) recently introduced the constants
SeekStart, SeekCurrent, and SeekEnd to the io package. We should use these constants
consistently throughout the code base.
Updates #15269
Change-Id: If7fcaca7676e4a51f588528f5ced28220d9639a2
Reviewed-on: https://go-review.googlesource.com/22097
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <joetsai@digital-static.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The following performance improvements have been made to the
low-level atomic functions for ppc64le & ppc64:
- For those cases containing a lwarx and stwcx (or other sizes):
sync, lwarx, maybe something, stwcx, loop to sync, sync, isync
The sync is moved before (outside) the lwarx/stwcx loop, and the
sync after is removed, so it becomes:
sync, lwarx, maybe something, stwcx, loop to lwarx, isync
- For the Or8 and And8, the shifting and manipulation of the
address to the word aligned version were removed and the
instructions were changed to use lbarx, stbcx instead of
register shifting, xor, then lwarx, stwcx.
- New instructions LWSYNC, LBAR, STBCC were tested and added.
runtime/atomic_ppc64x.s was changed to use the LWSYNC opcode
instead of the WORD encoding.
Fixes#15469
Ran some of the benchmarks in the runtime and sync directories.
Some results varied from run to run but the trend was improvement
based on best times for base and new:
runtime.test:
BenchmarkChanNonblocking-128 0.88 0.89 +1.14%
BenchmarkChanUncontended-128 569 511 -10.19%
BenchmarkChanContended-128 63110 53231 -15.65%
BenchmarkChanSync-128 691 598 -13.46%
BenchmarkChanSyncWork-128 11355 11649 +2.59%
BenchmarkChanProdCons0-128 2402 2090 -12.99%
BenchmarkChanProdCons10-128 1348 1363 +1.11%
BenchmarkChanProdCons100-128 1002 746 -25.55%
BenchmarkChanProdConsWork0-128 2554 2720 +6.50%
BenchmarkChanProdConsWork10-128 1909 1804 -5.50%
BenchmarkChanProdConsWork100-128 1624 1580 -2.71%
BenchmarkChanCreation-128 237 212 -10.55%
BenchmarkChanSem-128 705 667 -5.39%
BenchmarkChanPopular-128 5081190 4497566 -11.49%
BenchmarkCreateGoroutines-128 532 473 -11.09%
BenchmarkCreateGoroutinesParallel-128 35.0 34.7 -0.86%
BenchmarkCreateGoroutinesCapture-128 4923 4200 -14.69%
sync.test:
BenchmarkUncontendedSemaphore-128 112 94.2 -15.89%
BenchmarkContendedSemaphore-128 133 128 -3.76%
BenchmarkMutexUncontended-128 1.90 1.67 -12.11%
BenchmarkMutex-128 353 310 -12.18%
BenchmarkMutexSlack-128 304 283 -6.91%
BenchmarkMutexWork-128 554 541 -2.35%
BenchmarkMutexWorkSlack-128 567 556 -1.94%
BenchmarkMutexNoSpin-128 275 242 -12.00%
BenchmarkMutexSpin-128 1129 1030 -8.77%
BenchmarkOnce-128 1.08 0.96 -11.11%
BenchmarkPool-128 29.8 27.4 -8.05%
BenchmarkPoolOverflow-128 40564 36583 -9.81%
BenchmarkSemaUncontended-128 3.14 2.63 -16.24%
BenchmarkSemaSyntNonblock-128 1087 1069 -1.66%
BenchmarkSemaSyntBlock-128 897 893 -0.45%
BenchmarkSemaWorkNonblock-128 1034 1028 -0.58%
BenchmarkSemaWorkBlock-128 949 886 -6.64%
Change-Id: I4403fb29d3cd5254b7b1ce87a216bd11b391079e
Reviewed-on: https://go-review.googlesource.com/22549
Reviewed-by: Michael Munday <munday@ca.ibm.com>
Reviewed-by: Minux Ma <minux@golang.org>
If b has exactly one predecessor, as happens
frequently with static calls, we can make
lookupVarOutgoing generate less garbage.
Instead of generating a value that is just
going to be an OpCopy and then get eliminated,
loop. This can lead to lots of looping.
However, this loop is way cheaper than generating
lots of ssa.Values and then eliminating them.
For a subset of the code in #15537:
Before:
28.31 real 36.17 user 1.68 sys
2282450944 maximum resident set size
After:
9.63 real 11.66 user 0.51 sys
638144512 maximum resident set size
Updates #15537.
Excitingly, it appears that this also helps
regular code:
name old time/op new time/op delta
Template 288ms ± 6% 276ms ± 7% -4.13% (p=0.000 n=21+24)
Unicode 143ms ± 8% 141ms ±10% ~ (p=0.287 n=24+25)
GoTypes 932ms ± 4% 874ms ± 4% -6.20% (p=0.000 n=23+22)
Compiler 4.89s ± 4% 4.58s ± 4% -6.46% (p=0.000 n=22+23)
MakeBash 40.2s ±13% 39.8s ± 9% ~ (p=0.648 n=23+23)
name old user-ns/op new user-ns/op delta
Template 388user-ms ±10% 373user-ms ± 5% -3.80% (p=0.000 n=24+25)
Unicode 203user-ms ± 6% 202user-ms ± 7% ~ (p=0.492 n=22+24)
GoTypes 1.29user-s ± 4% 1.17user-s ± 4% -9.67% (p=0.000 n=25+23)
Compiler 6.86user-s ± 5% 6.28user-s ± 4% -8.49% (p=0.000 n=25+25)
name old alloc/op new alloc/op delta
Template 51.5MB ± 0% 47.6MB ± 0% -7.47% (p=0.000 n=22+25)
Unicode 37.2MB ± 0% 37.1MB ± 0% -0.21% (p=0.000 n=25+25)
GoTypes 166MB ± 0% 138MB ± 0% -16.83% (p=0.000 n=25+25)
Compiler 756MB ± 0% 628MB ± 0% -16.96% (p=0.000 n=25+23)
name old allocs/op new allocs/op delta
Template 450k ± 0% 445k ± 0% -1.02% (p=0.000 n=25+25)
Unicode 356k ± 0% 356k ± 0% ~ (p=0.374 n=24+25)
GoTypes 1.31M ± 0% 1.25M ± 0% -4.18% (p=0.000 n=25+25)
Compiler 5.29M ± 0% 5.02M ± 0% -5.15% (p=0.000 n=25+23)
It also seems to help in other cases in which
phi insertion is a pain point (#14774, #14934).
Change-Id: Ibd05ed7b99d262117ece7bb250dfa8c3d1cc5dd2
Reviewed-on: https://go-review.googlesource.com/22790
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
The legacy mips64 backend doesn't handle large uint->float conversion
correctly. See #15552.
Change-Id: I84ceeaa95cc4e85f09cc46dfb30ab5d151f6b205
Reviewed-on: https://go-review.googlesource.com/22800
Reviewed-by: Minux Ma <minux@golang.org>
I overlooked it when rebasing CL 19803.
Change-Id: Ife9d6bcc6a772715d137af903c64bafac0cdb216
Reviewed-on: https://go-review.googlesource.com/22797
Reviewed-by: Minux Ma <minux@golang.org>
Adds the FCFIDU instruction and uses it instead of the FCFID
instruction for unsigned integer to float casts. This change means
that unsigned integers do not have to be cast to signed integers
before being cast to a floating point value. Therefore it is no
longer necessary to insert instructions to detect and fix
values that overflow int64.
The previous code generating the uint64 to int64 cast handled
overflow by truncating the uint64 value. This truncation can
change the result of the rounding performed by the integer to
float cast.
The FCFIDU instruction was added in Power ISA 2.06B.
Fixes#15539.
Change-Id: Ia37a9631293eff91032d4cd9a9bec759d2142437
Reviewed-on: https://go-review.googlesource.com/22772
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
debug/elf is only needed to determine the endianness of the host
machine, which is easy to do without debug/elf.
Fixes#15180.
Change-Id: I21035ed3884871270765a1ca3b812a5d4890a7ee
Reviewed-on: https://go-review.googlesource.com/21662
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
To speed up the ssacheck check builder and make it on by default as a
trybot.
Change-Id: I91a3347491507c84f4878dff744ca426ba3e2e9f
Reviewed-on: https://go-review.googlesource.com/22755
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
external linking is now supported.
Change-Id: I3f552f5f09391205fced509fe8a5a38297ea8153
Reviewed-on: https://go-review.googlesource.com/19810
Reviewed-by: Minux Ma <minux@golang.org>
Fixes#14544
Change-Id: I58b0b164ebbfeafe4ab32039a063df53e3018a6d
Reviewed-on: https://go-review.googlesource.com/22730
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Sean Lake <odysseus9672@gmail.com>
Consider three shared libraries:
libBase.so -- defines a type T
lib2.so -- references type T
lib3.so -- also references type T, and something from lib2
lib2.so will contain a type symbol for T in its symbol table, but no
definition. If, when linking lib3.so the linker reads the symbols from lib2.so
before libBase.so, the linker didn't read the type data and later crashed.
The fix is trivial but the test change is a bit messy because the order the
linker reads the shared libraries in ends up depending on the order of the
import statements in the file so I had to rename one of the test packages so
that gofmt doesn't fix the test by accident...
Fixes#15516
Change-Id: I124b058f782c900a3a54c15ed66a0d91d0cde5ce
Reviewed-on: https://go-review.googlesource.com/22744
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Why not? Because the 386 backend can't handle one of them.
But other than that, it should work.
Change-Id: Iaeb9735f8c3c281136a0734376dec5ddba21be3b
Reviewed-on: https://go-review.googlesource.com/22748
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
It never makes sense to CSE two ops that generate memory.
We might as well start those ops off in their own partition.
Fixes#15520
Change-Id: I0091ed51640f2c10cd0117f290b034dde7a86721
Reviewed-on: https://go-review.googlesource.com/22741
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
1) Blank parameters cannot be accessed so the package doesn't matter.
Do not export it, and consistently use localpkg when importing a
blank parameter.
2) More accurately replicate fmt.go and parser.go logic when importing
a blank struct field. Blank struct fields get exported without
package qualification.
(This is actually incorrect, even with the old textual export format,
but we will fix that in a separate change. See also issue 15514.)
Fixes#15491.
Change-Id: I7978e8de163eb9965964942aee27f13bf94a7c3c
Reviewed-on: https://go-review.googlesource.com/22714
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
1.7 traces embed symbol info and we now generate symbolized pprof profiles,
so we don't need the binary. Make binary argument optional as 1.5 traces
still need it.
Change-Id: I65eb13e3d20ec765acf85c42d42a8d7aae09854c
Reviewed-on: https://go-review.googlesource.com/22410
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
GCC, unlike clang, does not provide any way for code being compiled to tell if
-fsanitize-thread was passed. But cgo can look to see if that flag is being
passed and generate different code in that case.
Fixes#14602
Change-Id: I86cb5318c2e35501ae399618c05af461d1252d2d
Reviewed-on: https://go-review.googlesource.com/22688
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The format has been tweaked several times in the latest cycle, so
replace go13ld with go17ld.
Change-Id: I343c49b02b7516fd781bc96ad46640579da68c59
Reviewed-on: https://go-review.googlesource.com/22708
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I got a complaint that cgo output triggers warnings with
-Wdeclaration-after-statement. I don't think it's worth testing for
this--C has permitted declarations after statements since C99--but it is
easy enough to fix. It may break again; so it goes.
This CL also fixes errno handling to avoid getting confused if the tsan
functions happen to change the global errno variable.
Change-Id: I0ec7c63a6be5653ef44799d134c8d27cb5efa441
Reviewed-on: https://go-review.googlesource.com/22686
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
