Cgo internal linking was disabled when switching to new object
files. Reeanble the tests, as they get supported.
Change-Id: I11d2ac8785cce73f3a42f6935c10d9f067bc90a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/204520
Reviewed-by: Than McIntosh <thanm@google.com>
Even though bitwise operations may be slightly more
performant, the readability improvement of a mod
operation is worth the tradeoff.
Change-Id: I352c92ad355c6eb6ef99e3da00e1eff2d2ea5812
Reviewed-on: https://go-review.googlesource.com/c/go/+/204739
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
On illumos systems, and at least historically on Solaris systems, it is
possible for port_getn(3C) calls to return some number of events and
then fail with error ETIME.
Generally we expect this to happen if the caller passes an nget value
larger than 1 and calls with a timeout; if less than the requested
number of events accumulate the system will still return them after
timeout failure so the caller must check the updated nget value in the
ETIME case. Note that although less likely this can still happen even
when requesting just 1 event, especially with a short timeout value or
on a busy system.
Fixes#35261
Change-Id: I0d83251b69a2fadc64c4e8e280aa596e2e1548ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/204801
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In newobj mode, cgo attributes are typically set later, as we
create sym.Symbols later. But when internal cgo linking, the
host object loaders still work with sym.Symbols, and the cgo
attributes need to be set for them to work properly. Therefore,
set them early. This will cause creating some Symbols eagerly,
but they are mostly host object symbols and will need to be
created anyway.
Now all cgo internal linking tests pass on ELF systems.
Change-Id: I023a4df4429acc8ebf5e185f62e6809198497a78
Reviewed-on: https://go-review.googlesource.com/c/go/+/204857
Reviewed-by: Than McIntosh <thanm@google.com>
If multiple goroutines call time.(*Timer).Reset then the timer will go
from timerWaiting to timerDeleted to timerModifying to timerModifiedLater.
The timer can be on a different P, meaning that simultaneously cleantimers
could change it from timerDeleted to timerRemoving to timerRemoved.
If Reset sees timerRemoved, it was doing an atomic.Store of timerWaiting,
meaning that it did not necessarily see the other values set in the timer,
so the timer could appear to be in an inconsistent state. Use atomic.Cas
to avoid that possibility.
Updates #6239
Updates #27707Fixes#35272
Change-Id: I1d59a13dc4f2ff4af110fc6e032c8c9d59cfc270
Reviewed-on: https://go-review.googlesource.com/c/go/+/204717
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
In CL 204617, I intend to make "bound" parameter to have special meaning
in typecheckarraylit, so we can distinguish between type-checks array
literal and slice literal. But we end up with other solution. The CL was
submitted without reverting the "bound" parameter in case of slice
literal.
Technically, it's not harmful, but causes the code harder to read and maintain.
Change-Id: Ia522ccc9a6b8e25d7eaad4aa4957cb4fa18edc60
Reviewed-on: https://go-review.googlesource.com/c/go/+/204618
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
User's program was mutating time.Local variable and crashing
itself as a consequence. Instead of documenting that time.Local
variable should not be mutated, recommended way of setting the
system's time zone has been documented.
Fixes#34814
Change-Id: I7781189855c3bf2ea979dfa07f86c283eed27091
Reviewed-on: https://go-review.googlesource.com/c/go/+/200457
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I was doing some testing with GODEBUG=schedtrace=1,scheddetail=1 and I
noticed that the program hung after a throw with "all goroutines are
asleep". This is because when doing a throw or fatal panic with schedtrace
the panic code does a final schedtrace, which needs to acquire the
scheduler lock. The checkdead function is always called with the scheduler
lock held. So checkdead would throw with the scheduler lock held, then
the panic code would call schedtrace, which would block trying to acquire
the scheduler lock.
This problem will only happen for people debugging the runtime, but
it's easy to avoid by having checkdead unlock the scheduler lock before
it throws. I only did this for the throws that can happen for a normal
program, not for throws that indicate some corruption in the scheduler data.
Change-Id: Ic62277b3ca6bee6f0fca8d5eb516c59cb67855cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/204778
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change turns off the scavenger if there's less than one physical
page of work to do. If there's less than one phyiscal page of work
today, then the computed time for the work to be done will be zero,
resulting in a floating point division by zero.
This is bad on two accounts. On the one hand it could cause a fault on
some systems. On the other hand, it could cause the pacing computations
done by the scavenger to be nonsense. While this is generally harmless
in the case where there's a very small amount of work to do anyway (the
scavenger might just back off expontentially forever, or do some work
and immediately sleep, because there's not much of it to do), it causes
problems for the deadlock checker. On platforms with a larger physical
page size, such as 64 KiB, we might hit this path in a deadlock
scenario, in which case the deadlock checker will never fire and we'll
just hang.
Specifically, this happens on ppc64 trybot tests, which is where the
issue was discovered.
Fixes#34575.
Change-Id: I8677db539447b2f0e75b8cfcbe33932244e1508c
Reviewed-on: https://go-review.googlesource.com/c/go/+/203517
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
For debugging. (The "go1.4" can be misleading since it might actually
be go1.4.3 or go1.11 or go1.12 or master)
Change-Id: I27520b931a2be018de577a299592d082260aa467
Reviewed-on: https://go-review.googlesource.com/c/go/+/204757
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Suppress “finding” messages unless they are unusually slow, and
“extracting” messages always (they almost always occur conjunction
with “downloading”, which is already logged).
Log “found” messages for module dependencies added to satisfy missing
import paths.
Log top-level version changes in 'go get' when the selected version
is not identical to the version requested on the command line.
Updates #26152
Updates #33284
Change-Id: I4d0de60fab58d7cc7df8a2aff05c8b5b2220e626
Reviewed-on: https://go-review.googlesource.com/c/go/+/204777
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
A majority of work is spent in dataSize when en/decoding the same
struct over and over again. This wastes a lot of work, since
the result doesn't change for a given reflect.Value.
Cache the result of the function for structs, so that subsequent
calls to dataSize can avoid doing work.
name old time/op new time/op delta
ReadStruct 1.00µs ± 1% 0.37µs ± 1% -62.99% (p=0.029 n=4+4)
WriteStruct 1.00µs ± 3% 0.37µs ± 1% -62.69% (p=0.008 n=5+5)
name old speed new speed delta
ReadStruct 75.1MB/s ± 1% 202.9MB/s ± 1% +170.16% (p=0.029 n=4+4)
WriteStruct 74.8MB/s ± 3% 200.4MB/s ± 1% +167.96% (p=0.008 n=5+5)
Fixes#34471
Change-Id: Ic5d987ca95f1197415ef93643a0af6fc1224fdf0
Reviewed-on: https://go-review.googlesource.com/c/go/+/199539
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change the loader to do more bulk allocation when making slices of
small objects (sym.Reloc, etc) as part of creating and populating
sym.Symbols in loader.LoadFull(). This replaces a large number of
small allocations with a smaller number of large allocations,
improving performace. Compilebench numbers (linker portion) for this
change:
name old time/op new time/op delta
LinkCompiler 1.71s ±11% 1.57s ± 9% -8.35% (p=0.000 n=19+20)
LinkWithoutDebugCompiler 1.19s ±14% 1.10s ±13% -7.93% (p=0.000 n=20+19)
name old user-time/op new user-time/op delta
LinkCompiler 1.86s ±15% 1.34s ±10% -28.02% (p=0.000 n=20+20)
LinkWithoutDebugCompiler 1.05s ±14% 0.95s ± 9% -9.17% (p=0.000 n=19+20)
Hyperkube from kubernetes doesn't show any significant benefit (which
seems a little surprising).
Change-Id: Ide97f78532fb60b08bb6e4cfa097e9058f7ea8ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/203457
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Based on the riscv-go port and the linux/riscv64 files in x/sys/unix.
Updates #27532
Change-Id: Ib33a59a61f6b2721b12292c18f1fc9f9d0509cd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/204659
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
config.go needs to be removed from this CL.
Change-Id: I04a267feeae1551bb18f6a03a725adc9db593fdb
Reviewed-on: https://go-review.googlesource.com/c/go/+/204099
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The current implementation performs a plain map lookup,
but other header methods canonicalize header keys before
using them.
Fixes#34918
Change-Id: Id4120488b8b39ecee97fa7a6ad8a34158687ffcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/201357
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Based on the riscv-go port.
Updates #27532
Change-Id: I3a4d86783fbd625e3ade16d08f87d66e4502f3f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/204660
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fix up the new dead code pass to include support for populating the
ctxt "Reachparent" map, which is needed to support field tracking.
Since we don't have sym.Symbols created at the point where new dead
code runs, keep track of reachability using global symbol indices, and
then once loader.LoadFull is complete we can translate the index
mappings into symbol mappings.
The fieldtracking output is unfortunately different relative to
master, due to differences in the order in which symbols are
encountered in deadcode, but I have eyeballed the results to make sure
they look reasonable.
Change-Id: I48c7a4597f05c00f15af3bfd37fc15ab4d0017c2
Reviewed-on: https://go-review.googlesource.com/c/go/+/204342
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
ABI alias references in Go symbols are resolved during
loadObjFull. But for external symbols they are not resolved. If
there is a reference from an external symbol to a Go ABIInternal
symbol, this reference will be invalid as it is not resolved.
The old code resolve ABI aliases in the deadcode pass. But the
new deadcode pass doesn't do it, as it works with indices instead
of Symbols. We do this in LoadFull.
This makes all internal cgo linking tests pass on Mach-O.
Change-Id: Iac6c084c03f5ddbcc9455527800ce7ed7313f9a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/204698
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
In internal linking mode, we need to process cgo_import_static
directives before loading host objects, because the directive is
to tell the host object loader how to deal with imported symbols.
This should fix linking with old object files. I think there
needs some similar logic for new object files, but I'll leave
that for later.
Change-Id: Icaa286de626ea1876086dbdd015047084c92caf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/204697
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
If a previous Write returned an error, any subsequent Write or ReadFrom
must return that error before any operations.
However, only Write behaved correctly and this change fixes that problem
by making sure that ReadFrom firstly checks for the underlying error.
Fixes#35194
Change-Id: I31356a9e8bd945bc0168b2e3be470f3ae69d4813
Reviewed-on: https://go-review.googlesource.com/c/go/+/204000
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When everything is working correctly, any pointer the garbage
collector encounters can only point into a fully initialized heap
span, since the span must have been initialized before that pointer
could escape the heap allocator and become visible to the GC.
However, in various cases, we try to be defensive against bad
pointers. In findObject, this is just a sanity check: we never expect
to find a bad pointer, but programming errors can lead to them. In
spanOfHeap, we don't necessarily trust the pointer and we're trying to
check if it really does point to the heap, though it should always
point to something. Conservative scanning takes this to a new level,
since it can only guess that a word may be a pointer and verify this.
In all of these cases, we have a problem that the span lookup and
check can race with span initialization, since the span becomes
visible to lookups before it's fully initialized.
Furthermore, we're about to start initializing the span without the
heap lock held, which is going to introduce races where accesses were
previously protected by the heap lock.
To address this, this CL makes accesses to mspan.state atomic, and
ensures that the span is fully initialized before setting the state to
mSpanInUse. All loads are now atomic, and in any case where we don't
trust the pointer, it first atomically loads the span state and checks
that it's mSpanInUse, after which it will have synchronized with span
initialization and can safely check the other span fields.
For #10958, #24543, but a good fix in general.
Change-Id: I518b7c63555b02064b98aa5f802c92b758fef853
Reviewed-on: https://go-review.googlesource.com/c/go/+/203286
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Currently, several important fields of a heap span are set by
heapBits.initSpan, which happens after the span has already been
published and returned from the locked region of alloc_m. In
particular, allocBits is set very late, which makes mspan.isFree
unsafe even if you were to lock the heap because it tries to access
allocBits.
This CL fixes this by populating these fields in alloc_m. The next CL
builds on this to only publish the span once it is fully initialized.
Together, they'll make it safe to check allocBits even if there is a
race with alloc_m.
For #10958, #24543, but a good fix in general.
Change-Id: I7fde90023af0f497e826b637efa4d19c32840c08
Reviewed-on: https://go-review.googlesource.com/c/go/+/203285
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Small updates to macho loader, fixing some misunderstandings I had about
using the new object file format.
Change-Id: I9224b01ca327e3a087ebfa36800bd6eef6abcc80
Reviewed-on: https://go-review.googlesource.com/c/go/+/204097
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Follow the recommandation from RFC 8422, section 5.1.2 of sending back the
ec_points_format extension when requested by the client. This is to fix
some clients declining the handshake if omitted.
Fixes#31943
Change-Id: I7b04dbac6f9af75cda094073defe081e1e9a295d
Reviewed-on: https://go-review.googlesource.com/c/go/+/176418
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Olivier Poitrey <rs@rhapsodyk.net>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add support for elf host objects with new object file format.
Change-Id: Ic5be1953359b9b6b78d9a0b715af69763aefd227
Reviewed-on: https://go-review.googlesource.com/c/go/+/201728
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
This is a re-submission of CL 151157, since it was reverted in CL 190909
due to an introduced crash found by a fuzzer. The revert CL included
regression tests, while this CL includes a fixed version of the original
change.
In particular, what we forgot in the original optimization was that we
still need the length and trailing quote checks at the beginning of
unquoteBytes. Without those, we could end up in a crash later on.
We can work out how many bytes can be unquoted trivially in
rescanLiteral, which already iterates over a string's bytes.
Removing the extra loop in unquoteBytes simplifies the function and
speeds it up, especially when decoding simple strings, which are common.
While at it, we can remove the check that s[0]=='"', since all call
sites already meet that condition.
name old time/op new time/op delta
CodeDecoder-8 10.6ms ± 2% 10.5ms ± 1% -1.01% (p=0.004 n=20+10)
name old speed new speed delta
CodeDecoder-8 183MB/s ± 2% 185MB/s ± 1% +1.02% (p=0.003 n=20+10)
Updates #28923.
Change-Id: I8c6b13302bcd86a364bc998d72451332c0809cde
Reviewed-on: https://go-review.googlesource.com/c/go/+/190659
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Peter Weinberger <pjw@google.com>
Add some simple unit tests for these atomic operations. These can't
catch all the bugs that are possible with these operations but at
least they provide some coverage.
Change-Id: I94b9f451fcc9fecdb2a1448c5357b019563ad275
Reviewed-on: https://go-review.googlesource.com/c/go/+/204317
Run-TryBot: Michael Munday <mike.munday@ibm.com>
Reviewed-by: Austin Clements <austin@google.com>
DWARF constant DIE symbols are not referenced by any other symbol,
but are needed by the DWARF pass, where they get linked to the
compilation unit.
Reenable gdb constant test.
Change-Id: If77a0d379d9a6f1591939345bc31b027c2567f22
Reviewed-on: https://go-review.googlesource.com/c/go/+/204397
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
dynexp is used for generating the dynamic symbol table. It is
created from a map. Sort it to ensure deterministic order.
Should fix solaris build.
Change-Id: I561b9da3a4136a7ea41139073f76c98fb069d4fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/204378
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Add a new loader.Loader.ReadAuxSyms method that returns a slice
containing the ids of the aux symbols for a specified global symbol.
This is similar to the new interface recently added that allows you to
get back a slice of relocations (as opposed to making calls into the
loader for each one). This was idea suggested by Cherry. Compilebench
numbers:
name old time/op new time/op delta
LinkCompiler 1.63s ± 9% 1.57s ± 7% -3.84% (p=0.006 n=20+20)
LinkWithoutDebugCompiler 1.15s ±11% 1.11s ±11% ~ (p=0.108 n=20+20)
name old user-time/op new user-time/op delta
LinkCompiler 1.99s ± 8% 2.00s ±12% ~ (p=0.751 n=19+19)
LinkWithoutDebugCompiler 1.14s ±11% 1.19s ±21% ~ (p=0.183 n=20+20)
Change-Id: Iab6cbe18419aaa61d9cadb3f626a4515c71f2686
Reviewed-on: https://go-review.googlesource.com/c/go/+/203501
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
To speed up the loader.Loader.toLocal() method, cache the index of the
most recently accessed object file and check that object's sym range
in toLocal() before doing a full binary search over all object symbol
ranges. This speeds up relink of kubernetes/hyperkube by about 2%, and
improves compilebench (relative to the dev.link branch) by about 5%:
name old time/op new time/op delta
LinkCompiler 1.62s ± 8% 1.50s ± 9% -7.21% (p=0.000 n=20+19)
LinkWithoutDebugCompiler 1.13s ± 8% 1.09s ±12% ~ (p=0.052 n=20+20)
name old user-time/op new user-time/op delta
LinkCompiler 1.94s ±18% 1.97s ±16% ~ (p=0.813 n=19+20)
LinkWithoutDebugCompiler 1.15s ±16% 1.13s ±12% ~ (p=0.547 n=20+20)
Change-Id: Id5a8a847b533858373c0462f03972d436eda6748
Reviewed-on: https://go-review.googlesource.com/c/go/+/204337
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
