This benchmark demonstrates a current problem with the scheduler where
a set of frequently communicating goroutines get very little CPU time
in the presence of another goroutine that hogs that CPU, even if one
of those communicating goroutines is always runnable.
Currently it takes about 0.5 milliseconds to switch between
ping-ponging goroutines in the presence of a CPU hog:
BenchmarkPingPongHog 2000 684287 ns/op
Change-Id: I278848c84f778de32344921ae8a4a8056e4898b0
Reviewed-on: https://go-review.googlesource.com/9288
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
There are a variety of places where we check if a P's run queue is
empty. This test is about to get slightly more complicated, so factor
it out into a new function, runqempty. This function is inlinable, so
this has no effect on performance.
Change-Id: If4a0b01ffbd004937de90d8d686f6ded4aad2c6b
Reviewed-on: https://go-review.googlesource.com/9287
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
We can expand the test cases as we discover problems.
This is some basic tests plus all the things I got wrong
in some recent work.
Change-Id: Id875fcfaf74eb087ae42b441fe47a34c5b8ccb39
Reviewed-on: https://go-review.googlesource.com/9158
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Explicitly specify that we represent polynomial in reversed notation
Fixes#8229
Change-Id: Idf094c01fd82f133cd0c1b50fa967d12c577bdb5
Reviewed-on: https://go-review.googlesource.com/9237
Reviewed-by: David Chase <drchase@google.com>
Some machines are so slow that even with the default timeoutScale,
they still timeout some tests. For example, currently some linux/arm
builders and the openbsd/arm builder are timing out the runtime
test and CL 8397 was proposed to skip some tests on openbsd/arm
to fix the build.
Instead of increasing timeoutScale or skipping tests, this CL
introduces an environment variable $GO_TEST_TIMEOUT_SCALE that
could be set to manually set a larger timeoutScale for those
machines/builders.
Fixes#10314.
Change-Id: I16c9a9eb980d6a63309e4cacd79eee2fe05769ee
Reviewed-on: https://go-review.googlesource.com/9223
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Forward signals to signal handlers installed before Go installs its own,
under certain circumstances. In particular, as iant@ suggests, signals are
forwarded iff:
(1) a non-SIG_DFL signal handler existed before Go, and
(2) signal is synchronous (i.e., one of SIGSEGV, SIGBUS, SIGFPE), and
(3a) signal occured on a non-Go thread, or
(3b) signal occurred on a Go thread but in CGo code.
Supported only on Linux, for now.
Change-Id: I403219ee47b26cf65da819fb86cf1ec04d3e25f5
Reviewed-on: https://go-review.googlesource.com/8712
Reviewed-by: Ian Lance Taylor <iant@golang.org>
They are vestiges of the c2go transition.
Change-Id: I22672e40373ef77d7a0bf69cfff8017e46353055
Reviewed-on: https://go-review.googlesource.com/9265
Reviewed-by: Minux Ma <minux@golang.org>
This addresses iant's comments from CL 9164.
Change-Id: I7b5b282f61b11aab587402c2d302697e76666376
Reviewed-on: https://go-review.googlesource.com/9222
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, it's possible for the next_gc calculation to underflow.
Since next_gc is unsigned, this wraps around and effectively disables
GC for the rest of the program's execution. Besides being obviously
wrong, this is causing test failures on 32-bit because some tests are
running out of heap.
This underflow happens for two reasons, both having to do with how we
estimate the reachable heap size at the end of the GC cycle.
One reason is that this calculation depends on the value of heap_live
at the beginning of the GC cycle, but we currently only record that
value during a concurrent GC and not during a forced STW GC. Fix this
by moving the recorded value from gcController to work and recording
it on a common code path.
The other reason is that we use the amount of allocation during the GC
cycle as an approximation of the amount of floating garbage and
subtract it from the marked heap to estimate the reachable heap.
However, since this is only an approximation, it's possible for the
amount of allocation during the cycle to be *larger* than the marked
heap size (since the runtime allocates white and it's possible for
these allocations to never be made reachable from the heap). Currently
this causes wrap-around in our estimate of the reachable heap size,
which in turn causes wrap-around in next_gc. Fix this by bottoming out
the reachable heap estimate at 0, in which case we just fall back to
triggering GC at heapminimum (which is okay since this only happens on
small heaps).
Fixes#10555, fixes#10556, and fixes#10559.
Change-Id: Iad07b529c03772356fede2ae557732f13ebfdb63
Reviewed-on: https://go-review.googlesource.com/9286
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
To achieve a 2% improvement in the garbage benchmark this CL removes
an unneeded assert and avoids one hbits.next() call per object
being scanned.
Change-Id: Ibd542d01e9c23eace42228886f9edc488354df0d
Reviewed-on: https://go-review.googlesource.com/9244
Reviewed-by: Austin Clements <austin@google.com>
newosproc0 does not work on android/arm.
See issue #10548.
Change-Id: Ieaf6f5d0b77cddf5bf0b6c89fd12b1c1b8723f9b
Reviewed-on: https://go-review.googlesource.com/9293
Reviewed-by: David Crawshaw <crawshaw@golang.org>
github.com/kr/goven says it's deprecated and anyway
it would be preferable to point users to a standard Go tool.
Change-Id: Iac4a0d13233604a36538748d498f5770b2afce19
Reviewed-on: https://go-review.googlesource.com/8969
Reviewed-by: Minux Ma <minux@golang.org>
If the machine's network configuration files (resolv.conf,
nsswitch.conf) don't have any unsupported options, prefer Go's DNS
resolver, which doesn't have the cgo & thread over.
It means users can have more than 500 DNS requests outstanding (our
current limit for cgo lookups) and not have one blocked thread per
outstanding request.
Discussed in thread https://groups.google.com/d/msg/golang-dev/2ZUi792oztM/Q0rg_DkF5HMJ
Change-Id: I3f685d70aff6b47bec30b63e9fba674b20507f95
Reviewed-on: https://go-review.googlesource.com/8945
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
They are vestiges of the c2go translation.
Change-Id: I9a10536f5986b751a35cc7d84b5ba69ae0c2ede7
Reviewed-on: https://go-review.googlesource.com/9262
Reviewed-by: Minux Ma <minux@golang.org>
There is an assumption that the function executed in child thread
created by runtime.close should not return. And different systems
enforce that differently: some exit that thread, some exit the
whole process.
The test TestNewOSProc0 introduced in CL 9161 breaks that assumption,
so we need to adjust the code to only exit the thread should the
called function return.
Change-Id: Id631cb2f02ec6fbd765508377a79f3f96c6a2ed6
Reviewed-on: https://go-review.googlesource.com/9246
Reviewed-by: Dave Cheney <dave@cheney.net>
It was only visible when you run godoc with explicit GOOS=windows,
which is less useful for people developing portable application on
non-windows platforms.
Also added a note that log/syslog is not supported on NaCl.
Change-Id: I81650445fb2a5ee161da7e0608c3d3547d5ac2a6
Reviewed-on: https://go-review.googlesource.com/9245
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The constants in cmd/internal/goobj had gone stale (we had three copies of
these constants, working on reducing that was what got me to noticing this).
Some of the changes to link.hello.darwin.amd64 are the change from absolute
to %rip-relative addressing, a change which happened quite a while ago...
Depends on http://golang.org/cl/9113.
Fixes#10501.
Change-Id: Iaa1511f458a32228c2df2ccd0076bb9ae212a035
Reviewed-on: https://go-review.googlesource.com/9105
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, we set the heap goal for the next GC cycle using the size
of the marked heap at the end of the current cycle. This can lead to a
bad feedback loop if the mutator is rapidly allocating and releasing
pointers that can significantly bloat heap size.
If the GC were STW, the marked heap size would be exactly the
reachable heap size (call it stwLive). However, in concurrent GC,
marked=stwLive+floatLive, where floatLive is the amount of "floating
garbage": objects that were reachable at some point during the cycle
and were marked, but which are no longer reachable by the end of the
cycle. If the GC cycle is short, then the mutator doesn't have much
time to create floating garbage, so marked≈stwLive. However, if the GC
cycle is long and the mutator is allocating and creating floating
garbage very rapidly, then it's possible that marked≫stwLive. Since
the runtime currently sets the heap goal based on marked, this will
cause it to set a high heap goal. This means that 1) the next GC cycle
will take longer because of the larger heap and 2) the assist ratio
will be low because of the large distance between the trigger and the
goal. The combination of these lets the mutator produce even more
floating garbage in the next cycle, which further exacerbates the
problem.
For example, on the garbage benchmark with GOMAXPROCS=1, this causes
the heap to grow to ~500MB and the garbage collector to retain upwards
of ~300MB of heap, while the true reachable heap size is ~32MB. This,
in turn, causes the GC cycle to take upwards of ~3 seconds.
Fix this bad feedback loop by estimating the true reachable heap size
(stwLive) and using this rather than the marked heap size
(stwLive+floatLive) as the basis for the GC trigger and heap goal.
This breaks the bad feedback loop and causes the mutator to assist
more, which decreases the rate at which it can create floating
garbage. On the same garbage benchmark, this reduces the maximum heap
size to ~73MB, the retained heap to ~40MB, and the duration of the GC
cycle to ~200ms.
Change-Id: I7712244c94240743b266f9eb720c03802799cdd1
Reviewed-on: https://go-review.googlesource.com/9177
Reviewed-by: Rick Hudson <rlh@golang.org>
Change-Id: I429402dd91243cd9415b054ee17bfebccc68ed57
Reviewed-on: https://go-review.googlesource.com/9197
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This may or may not be useful to the end user, but it's incredibly
useful for us to understand the behavior of the pacer. Currently this
is fairly easy (though not trivial) to derive from the other heap
stats we print, but we're about to change how we compute the goal,
which will make it much harder to derive.
Change-Id: I796ef233d470c01f606bd9929820c01ece1f585a
Reviewed-on: https://go-review.googlesource.com/9176
Reviewed-by: Rick Hudson <rlh@golang.org>
The trigger controller computes GC CPU utilization by dividing by the
wall-clock time that's passed since concurrent mark began. Since this
delta is nanoseconds it's borderline impossible for it to be zero, but
if it is zero we'll currently divide by zero. Be robust to this
possibility by ignoring the utilization in the error term if no time
has elapsed.
Change-Id: I93dfc9e84735682af3e637f6538d1e7602634f09
Reviewed-on: https://go-review.googlesource.com/9175
Reviewed-by: Rick Hudson <rlh@golang.org>
To make the gcprog for global data containing variables of types defined in other shared
libraries, we need to know a lot about those types. So read the value of any symbol with
a name starting with "type.". If a type uses a mask, the name of the symbol defining the
mask unfortunately cannot be predicted from the type name so I have to keep track of the
addresses of every such symbol and associate them with the type symbols after the fact.
I'm not very happy about this change, but something like this is needed and this is as
pleasant as I know how to make it.
Change-Id: I408d831b08b3b31e0610688c41367b23998e975c
Reviewed-on: https://go-review.googlesource.com/8334
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
There were 10 implementations of the trivial bool2int function, 9 of which
were the only thing in their file. Remove all of them in favor of one in
cmd/internal/obj.
Change-Id: I9c51d30716239df51186860b9842a5e9b27264d3
Reviewed-on: https://go-review.googlesource.com/9230
Reviewed-by: Ian Lance Taylor <iant@golang.org>
since the "precision" parameter means constant arithmetic is not
necessarily exact.
As requested by gri, within go/types, the local import name 'exact'
has been kept, to reduce the diff with the x/tools branch. This may
be changed later.
Since the go/types.bash script was already obsolete, I added a comment
to this effect.
Tested with all.bash.
Change-Id: I45153688d9d8afa8384fb15229b0124c686059b4
Reviewed-on: https://go-review.googlesource.com/9242
Reviewed-by: Rob Pike <r@golang.org>
We initially added clone0 to handle the case when G or M don't exist, but
it turns out that we could have just modified clone. (It also helps that
the function we're invoking in clone0 no longer needs arguments.)
As a side-effect, newosproc0 is now supported on all linux archs.
Change-Id: Ie603af75d8f164310fc16446052d83743961f3ca
Reviewed-on: https://go-review.googlesource.com/9164
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Added a prec parameter to MakeFromLiteral (which currently must
always be 0). This will permit go/types to provide an upper limit
for the precision of constant values, eventually. Overflows can be
returned with a special Overflow value (very much like the current
Unknown values).
This is a minimal change that should prevent the need for future
backward-incompatible API changes.
Change-Id: I6c9390d7cc4810375e26c53ed3bde5a383392330
Reviewed-on: https://go-review.googlesource.com/9168
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
In the brief window between getConn and persistConn.roundTrip,
a cancel could end up going missing.
Fix by making it possible to inspect if a cancel function was cleared
and checking if we were canceled before entering roundTrip.
Fixes#10511
Change-Id: If6513e63fbc2edb703e36d6356ccc95a1dc33144
Reviewed-on: https://go-review.googlesource.com/9181
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previously all errors were 404 errors, even if the real error had
nothing to do with a file being non-existent.
Fixes#10283
Change-Id: I5b08b471a9064c347510cfcf8557373704eef7c0
Reviewed-on: https://go-review.googlesource.com/9200
Reviewed-by: Daniel Morsing <daniel.morsing@gmail.com>
There used to be a small window where if a server declared it would do
a keep-alive connection but then actually closed the connection before
the roundTrip goroutine scheduled after being sent a response from the
readLoop goroutine, then the readLoop goroutine would loop around and
block forever reading from a channel because the numExpectedResponses
accounting was done too late.
Fixes#10457
Change-Id: Icbae937ffe83c792c295b7f4fb929c6a24a4f759
Reviewed-on: https://go-review.googlesource.com/9169
Reviewed-by: Daniel Morsing <daniel.morsing@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Unlike linux arm32, linux arm64 does not set the condition codes to indicate
whether a system call failed or not. We must check if the return value
is in the error code range (the same as amd64 does).
Fixes runtime.TestBadOpen test.
Change-Id: I97a8b0a17b5f002a3215c535efa91d199cee3309
Reviewed-on: https://go-review.googlesource.com/9220
Reviewed-by: Russ Cox <rsc@golang.org>
Several naming changes and a real issue in asmcgocall_errno.
Change-Id: Ieb0a328a168819fe233d74e0397358384d7e71b3
Reviewed-on: https://go-review.googlesource.com/9212
Reviewed-by: Minux Ma <minux@golang.org>
This change replaces server tests with new ones that require features
introduced after go1 release, such as runtime-integrated network poller,
Dialer, etc.
Change-Id: Icf1f94f08f33caacd499cfccbe74cda8d05eed30
Reviewed-on: https://go-review.googlesource.com/9195
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change deflakes zero byte read/write tests on datagram sockets, and
enables them by default.
Change-Id: I52f1a76f8ff379d90f40a07bb352fae9343ea41a
Reviewed-on: https://go-review.googlesource.com/9194
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change excludes internal UDP header size from a result of number of
bytes written on WriteTo.
Change-Id: I847d57f7f195657b6f14efdf1b4cfab13d4490dd
Reviewed-on: https://go-review.googlesource.com/9196
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
The purpose of this test is to make sure that -buildmode=c-shared
works even when the shared library can be built without invoking cgo.
Change-Id: Id6f95af755992b209aff770440ca9819b74113ab
Reviewed-on: https://go-review.googlesource.com/9166
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This reverts commit 8d7d02f145.
Reverted because it breaks go/build's "deps" test.
Change-Id: I61db6b2431b3ba0d2b3ece5bab7a04194239c34b
Reviewed-on: https://go-review.googlesource.com/9174
Reviewed-by: Alan Donovan <adonovan@google.com>
In external linking mode, the linker automatically imports
runtime/cgo. When the user uses non-standard compilation options,
they have to know to run go install runtime/cgo. When the go tool
adds non-standard compilation options itself, we can't force the user
to do that. So add the dependency ourselves.
Bad news: we don't currently have a clean way to know whether we are
going to use external linking mode. This CL duplicates logic split
between cmd/6l and cmd/internal/ld.
Good news: adding an unnecessary dependency on runtime/cgo does no
real harm. We aren't going to force the linker to pull it in, we're
just going to build it so that its available if the linker wants it.
Change-Id: Ide676339d4e8b1c3d9792884a2cea921abb281b7
Reviewed-on: https://go-review.googlesource.com/9115
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This change refactors reflect.Value to consistently use arrayAt when an element
of an array of bytes is indexed.
This effectively replaces:
arr := unsafe.Pointer(...)
arri := unsafe.Pointer(uintptr(arr) + uintptr(i)*elementSize)
with:
arr := unsafe.Pointer(...)
arri := arrayAt(arr, i, elementSize)
Change-Id: I53ffd0d6de693b43d5c10c0aa4cd6d4f5e95a1e3
Reviewed-on: https://go-review.googlesource.com/9183
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This reduces the number of allocations in the compiler
while building the stdlib by 15.66%.
No functional changes. Passes toolstash -cmp.
Change-Id: Ia21b37134a8906a4e23d53fdc15235b4aa7bbb34
Reviewed-on: https://go-review.googlesource.com/9085
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently, the GC controller computes the mutator assist ratio at the
beginning of the cycle by estimating that the marked heap size this
cycle will be the same as it was the previous cycle. It then uses that
assist ratio for the rest of the cycle. However, this means that if
the mutator is quickly growing its reachable heap, the heap size is
likely to exceed the heap goal and currently there's no additional
pressure on mutator assists when this happens. For example, 6g (with
GOMAXPROCS=1) frequently exceeds the goal heap size by ~25% because of
this.
This change makes GC revise its work estimate and the resulting assist
ratio every 10ms during the concurrent mark. Instead of
unconditionally using the marked heap size from the last cycle as an
estimate for this cycle, it takes the minimum of the previously marked
heap and the currently marked heap. As a result, as the cycle
approaches or exceeds its heap goal, this will increase the assist
ratio to put more pressure on the mutator assist to bring the cycle to
an end. For 6g, this causes the GC to always finish within 5% and
often within 1% of its heap goal.
Change-Id: I4333b92ad0878c704964be42c655c38a862b4224
Reviewed-on: https://go-review.googlesource.com/9070
Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Currently, in accordance with the GC pacing proposal, we schedule
background marking with a goal of achieving 25% utilization *total*
between mutator assists and background marking. This is stricter than
was set out in the Go 1.5 proposal, which suggests that the garbage
collector can use 25% just for itself and anything the mutator does to
help out is on top of that. It also has several technical
drawbacks. Because mutator assist time is constantly changing and we
can't have instantaneous information on background marking time, it
effectively requires hitting a moving target based on out-of-date
information. This works out in the long run, but works poorly for
short GC cycles and on short time scales. Also, this requires
time-multiplexing all Ps between the mutator and background GC since
the goal utilization of background GC constantly fluctuates. This
results in a complicated scheduling algorithm, poor affinity, and
extra overheads from context switching.
This change modifies the way we schedule and run background marking so
that background marking always consumes 25% of GOMAXPROCS and mutator
assist is in addition to this. This enables a much more robust
scheduling algorithm where we pre-determine the number of Ps we should
dedicate to background marking as well as the utilization goal for a
single floating "remainder" mark worker.
Change-Id: I187fa4c03ab6fe78012a84d95975167299eb9168
Reviewed-on: https://go-review.googlesource.com/9013
Reviewed-by: Rick Hudson <rlh@golang.org>
