This avoids confusion with the main findrunnable in the scheduler.
Change-Id: I8cf40657557a8610a2fe5a2f74598518256ca7f0
Reviewed-on: https://go-review.googlesource.com/9305
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, we use a full stop-the-world around enabling write
barriers. This is to ensure that all Gs have enabled write barriers
before any blackening occurs (either in gcBgMarkWorker() or in
gcAssistAlloc()).
However, there's no need to bring the whole world to a synchronous
stop to ensure this. This change replaces the STW with a ragged
barrier that ensures each P has individually observed that write
barriers should be enabled before GC performs any blackening.
Change-Id: If2f129a6a55bd8bdd4308067af2b739f3fb41955
Reviewed-on: https://go-review.googlesource.com/8207
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This adds forEachP, which performs a general-purpose ragged global
barrier. forEachP takes a callback and invokes it for every P at a GC
safe point.
Ps that are idle or in a syscall are considered to be at a continuous
safe point. forEachP ensures that these Ps do not change state by
forcing all syscall Ps into idle and holding the sched.lock.
To ensure that Ps do not enter syscall or idle without running the
safe-point function, this adds checks for a pending callback every
place there is currently a gcwaiting check.
We'll use forEachP to replace the STW around enabling the write
barrier and to replace the current asynchronous per-M wbuf cache with
a cooperatively managed per-P gcWork cache.
Change-Id: Ie944f8ce1fead7c79bf271d2f42fcd61a41bb3cc
Reviewed-on: https://go-review.googlesource.com/8206
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This fixes a bug where the runtime ready()s a goroutine while setting
up a new M that's initially marked as spinning, causing the scheduler
to later panic when it finds work in the run queue of a P associated
with a spinning M. Specifically, the sequence of events that can lead
to this is:
1) sysmon calls handoffp to hand off a P stolen from a syscall.
2) handoffp sees no pending work on the P, so it calls startm with
spinning set.
3) startm calls newm, which in turn calls allocm to allocate a new M.
4) allocm "borrows" the P we're handing off in order to do allocation
and performs this allocation.
5) This allocation may assist the garbage collector, and this assist
may detect the end of concurrent mark and ready() the main GC
goroutine to signal this.
6) This ready()ing puts the GC goroutine on the run queue of the
borrowed P.
7) newm starts the OS thread, which runs mstart and subsequently
mstart1, which marks the M spinning because startm was called with
spinning set.
8) mstart1 enters the scheduler, which panics because there's work on
the run queue, but the M is marked spinning.
To fix this, before marking the M spinning in step 7, add a check to
see if work was been added to the P's run queue. If this is the case,
undo the spinning instead.
Fixes#10573.
Change-Id: I4670495ae00582144a55ce88c45ae71de597cfa5
Reviewed-on: https://go-review.googlesource.com/9332
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
This adds a check that we never put a P on the idle list when it has
work on its local run queue.
Change-Id: Ifcfab750de60c335148a7f513d4eef17be03b6a7
Reviewed-on: https://go-review.googlesource.com/9324
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
This is the optimization made to math/rand in CL 21030043.
Change-Id: I231b24fa77cac1fe74ba887db76313b5efaab3e8
Reviewed-on: https://go-review.googlesource.com/9269
Reviewed-by: Minux Ma <minux@golang.org>
Follows the linux signal forwarding semantics from
http://golang.org/cl/8712, sharing the implementation of sigfwdgo.
Forwarding for 386, arm, and arm64 will follow.
Change-Id: I6bf30d563d19da39b6aec6900c7fe12d82ed4f62
Reviewed-on: https://go-review.googlesource.com/9302
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A previous change to mbitmap.go dropped a return on a
path the seems not to be excersized. This was a mistake that
this CL fixes.
Change-Id: I715ee4ef08f5bf8d9f53cee84e8fb31a237e2d43
Reviewed-on: https://go-review.googlesource.com/9295
Reviewed-by: Austin Clements <austin@google.com>
Currently, each M has a cache of the most recently used *workbuf. This
is used primarily by the write barrier so it doesn't have to access
the global workbuf lists on every write barrier. It's also used by
stack scanning because it's convenient.
This cache is important for write barrier performance, but this
particular approach has several downsides. It's faster than no cache,
but far from optimal (as the benchmarks below show). It's complex:
access to the cache is sprinkled through most of the workbuf list
operations and it requires special care to transform into and back out
of the gcWork cache that's actually used for scanning and marking. It
requires atomic exchanges to take ownership of the cached workbuf and
to return it to the M's cache even though it's almost always used by
only the current M. Since it's per-M, flushing these caches is O(# of
Ms), which may be high. And it has some significant subtleties: for
example, in general the cache shouldn't be used after the
harvestwbufs() in mark termination because it could hide work from
mark termination, but stack scanning can happen after this and *will*
use the cache (but it turns out this is okay because it will always be
followed by a getfull(), which drains the cache).
This change replaces this cache with a per-P gcWork object. This
gcWork cache can be used directly by scanning and marking (as long as
preemption is disabled, which is a general requirement of gcWork).
Since it's per-P, it doesn't require synchronization, which simplifies
things and means the only atomic operations in the write barrier are
occasionally fetching new work buffers and setting a mark bit if the
object isn't already marked. This cache can be flushed in O(# of Ps),
which is generally small. It follows a simple flushing rule: the cache
can be used during any phase, but during mark termination it must be
flushed before allowing preemption. This also makes the dispose during
mutator assist no longer necessary, which eliminates the vast majority
of gcWork dispose calls and reduces contention on the global workbuf
lists. And it's a lot faster on some benchmarks:
benchmark old ns/op new ns/op delta
BenchmarkBinaryTree17 11963668673 11206112763 -6.33%
BenchmarkFannkuch11 2643217136 2649182499 +0.23%
BenchmarkFmtFprintfEmpty 70.4 70.2 -0.28%
BenchmarkFmtFprintfString 364 307 -15.66%
BenchmarkFmtFprintfInt 317 282 -11.04%
BenchmarkFmtFprintfIntInt 512 483 -5.66%
BenchmarkFmtFprintfPrefixedInt 404 380 -5.94%
BenchmarkFmtFprintfFloat 521 479 -8.06%
BenchmarkFmtManyArgs 2164 1894 -12.48%
BenchmarkGobDecode 30366146 22429593 -26.14%
BenchmarkGobEncode 29867472 26663152 -10.73%
BenchmarkGzip 391236616 396779490 +1.42%
BenchmarkGunzip 96639491 96297024 -0.35%
BenchmarkHTTPClientServer 100110 70763 -29.31%
BenchmarkJSONEncode 51866051 52511382 +1.24%
BenchmarkJSONDecode 103813138 86094963 -17.07%
BenchmarkMandelbrot200 4121834 4120886 -0.02%
BenchmarkGoParse 16472789 5879949 -64.31%
BenchmarkRegexpMatchEasy0_32 140 140 +0.00%
BenchmarkRegexpMatchEasy0_1K 394 394 +0.00%
BenchmarkRegexpMatchEasy1_32 120 120 +0.00%
BenchmarkRegexpMatchEasy1_1K 621 614 -1.13%
BenchmarkRegexpMatchMedium_32 209 202 -3.35%
BenchmarkRegexpMatchMedium_1K 54889 55175 +0.52%
BenchmarkRegexpMatchHard_32 2682 2675 -0.26%
BenchmarkRegexpMatchHard_1K 79383 79524 +0.18%
BenchmarkRevcomp 584116718 584595320 +0.08%
BenchmarkTemplate 125400565 109620196 -12.58%
BenchmarkTimeParse 386 387 +0.26%
BenchmarkTimeFormat 580 447 -22.93%
(Best out of 10 runs. The delta of averages is similar.)
This also puts us in a good position to flush these caches when
nearing the end of concurrent marking, which will let us increase the
size of the work buffers while still controlling mark termination
pause time.
Change-Id: I2dd94c8517a19297a98ec280203cccaa58792522
Reviewed-on: https://go-review.googlesource.com/9178
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
When findRunnable considers running a fractional mark worker, it first
checks if there's any work to be done; if there isn't there's no point
in running the worker because it will just reschedule immediately.
However, currently findRunnable just checks work.full and
work.partial, whereas getfull can *also* draw work from m.currentwbuf.
As a result, findRunnable may not start a worker even though there
actually is work.
This problem manifests itself in occasional failures of the
test/init1.go test. This test is unusual because it performs a large
amount of allocation without executing any write barriers, which means
there's nothing to force the pointers in currentwbuf out to the
work.partial/full lists where findRunnable can see them.
This change fixes this problem by making findRunnable also check for a
currentwbuf. This aligns findRunnable with trygetfull's notion of
whether or not there's work.
Change-Id: Ic76d22b7b5d040bc4f58a6b5975e9217650e66c4
Reviewed-on: https://go-review.googlesource.com/9299
Reviewed-by: Russ Cox <rsc@golang.org>
Currently, findRunnable only considers running a mark worker if
there's work in the work queue. In principle, this can delay the start
of the desired number of dedicated mark workers if there's no work
pending. This is unlikely to occur in practice, since there should be
work queued from the scan phase, but if it were to come up, a CPU hog
mutator could slow down or delay garbage collection.
This check makes sense for fractional mark workers, since they'll just
return to the scheduler immediately if there's no work, but we want
the scheduler to start all of the dedicated mark workers promptly,
even if there's currently no queued work. Hence, this change moves the
pending work check after the check for starting a dedicated worker.
Change-Id: I52b851cc9e41f508a0955b3f905ca80f109ea101
Reviewed-on: https://go-review.googlesource.com/9298
Reviewed-by: Rick Hudson <rlh@golang.org>
The motivation is that sysAlloc/Free() currently aren't safe to be
called without a valid G, because arm's xadd64() uses locks that require
a valid G.
The solution here was proposed by Dmitry Vyukov: use xadduintptr()
instead of xadd64(), until arm can support xadd64 on all of its
architectures (not a trivial task for arm).
Change-Id: I250252079357ea2e4360e1235958b1c22051498f
Reviewed-on: https://go-review.googlesource.com/9002
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently, when allocation reaches the GC trigger, the runtime uses
readyExecute to start the GC goroutine immediately rather than wait
for the scheduler to get around to the GC goroutine while the mutator
continues to grow the heap.
Now that the scheduler runs the most recently readied goroutine when a
goroutine yields its time slice, this rigmarole is no longer
necessary. The runtime can simply ready the GC goroutine and yield
from the readying goroutine.
Change-Id: I3b4ebadd2a72a923b1389f7598f82973dd5c8710
Reviewed-on: https://go-review.googlesource.com/9292
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Currently, the main GC goroutine sleeps on a note during concurrent
mark and the first background mark worker or assist to finish marking
use wakes up that note to let the main goroutine proceed into mark
termination. Unfortunately, the latency of this wakeup can be quite
high, since the GC goroutine will typically have lost its P while in
the futex sleep, meaning it will be placed on the global run queue and
will wait there until some P is kind enough to pick it up. This delay
gives the mutator more time to allocate and create floating garbage,
growing the heap unnecessarily. Worse, it's likely that background
marking has stopped at this point (unless GOMAXPROCS>4), so anything
that's allocated and published to the heap during this window will
have to be scanned during mark termination while the world is stopped.
This change replaces the note sleep/wakeup with a gopark/ready
scheme. This keeps the wakeup inside the Go scheduler and lets the
garbage collector take advantage of the new scheduler semantics that
run the ready()d goroutine immediately when the ready()ing goroutine
sleeps.
For the json benchmark from x/benchmarks with GOMAXPROCS=4, this
reduces the delay in waking up the GC goroutine and entering mark
termination once concurrent marking is done from ~100ms to typically
<100µs.
Change-Id: Ib11f8b581b8914f2d68e0094f121e49bac3bb384
Reviewed-on: https://go-review.googlesource.com/9291
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Currently, we use a note sleep with a timeout in a loop in func gc to
periodically revise the GC control variables. Replace this with a
fully blocking note sleep and use a periodic timer to trigger the
revise instead. This is a step toward replacing the note sleep in func
gc.
Change-Id: I2d562f6b9b2e5f0c28e9a54227e2c0f8a2603f63
Reviewed-on: https://go-review.googlesource.com/9290
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Currently, when the runtime ready()s a G, it adds it to the end of the
current P's run queue and continues running. If there are many other
things in the run queue, this can result in a significant delay before
the ready()d G actually runs and can hurt fairness when other Gs in
the run queue are CPU hogs. For example, if there are three Gs sharing
a P, one of which is a CPU hog that never voluntarily gives up the P
and the other two of which are doing small amounts of work and
communicating back and forth on an unbuffered channel, the two
communicating Gs will get very little CPU time.
Change this so that when G1 ready()s G2 and then blocks, the scheduler
immediately hands off the remainder of G1's time slice to G2. In the
above example, the two communicating Gs will now act as a unit and
together get half of the CPU time, while the CPU hog gets the other
half of the CPU time.
This fixes the problem demonstrated by the ping-pong benchmark added
in the previous commit:
benchmark old ns/op new ns/op delta
BenchmarkPingPongHog 684287 825 -99.88%
On the x/benchmarks suite, this change improves the performance of
garbage by ~6% (for GOMAXPROCS=1 and 4), and json by 28% and 36% for
GOMAXPROCS=1 and 4. It has negligible effect on heap size.
This has no effect on the go1 benchmark suite since those benchmarks
are mostly single-threaded.
Change-Id: I858a08eaa78f702ea98a5fac99d28a4ac91d339f
Reviewed-on: https://go-review.googlesource.com/9289
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This benchmark demonstrates a current problem with the scheduler where
a set of frequently communicating goroutines get very little CPU time
in the presence of another goroutine that hogs that CPU, even if one
of those communicating goroutines is always runnable.
Currently it takes about 0.5 milliseconds to switch between
ping-ponging goroutines in the presence of a CPU hog:
BenchmarkPingPongHog 2000 684287 ns/op
Change-Id: I278848c84f778de32344921ae8a4a8056e4898b0
Reviewed-on: https://go-review.googlesource.com/9288
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
There are a variety of places where we check if a P's run queue is
empty. This test is about to get slightly more complicated, so factor
it out into a new function, runqempty. This function is inlinable, so
this has no effect on performance.
Change-Id: If4a0b01ffbd004937de90d8d686f6ded4aad2c6b
Reviewed-on: https://go-review.googlesource.com/9287
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Forward signals to signal handlers installed before Go installs its own,
under certain circumstances. In particular, as iant@ suggests, signals are
forwarded iff:
(1) a non-SIG_DFL signal handler existed before Go, and
(2) signal is synchronous (i.e., one of SIGSEGV, SIGBUS, SIGFPE), and
(3a) signal occured on a non-Go thread, or
(3b) signal occurred on a Go thread but in CGo code.
Supported only on Linux, for now.
Change-Id: I403219ee47b26cf65da819fb86cf1ec04d3e25f5
Reviewed-on: https://go-review.googlesource.com/8712
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This addresses iant's comments from CL 9164.
Change-Id: I7b5b282f61b11aab587402c2d302697e76666376
Reviewed-on: https://go-review.googlesource.com/9222
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, it's possible for the next_gc calculation to underflow.
Since next_gc is unsigned, this wraps around and effectively disables
GC for the rest of the program's execution. Besides being obviously
wrong, this is causing test failures on 32-bit because some tests are
running out of heap.
This underflow happens for two reasons, both having to do with how we
estimate the reachable heap size at the end of the GC cycle.
One reason is that this calculation depends on the value of heap_live
at the beginning of the GC cycle, but we currently only record that
value during a concurrent GC and not during a forced STW GC. Fix this
by moving the recorded value from gcController to work and recording
it on a common code path.
The other reason is that we use the amount of allocation during the GC
cycle as an approximation of the amount of floating garbage and
subtract it from the marked heap to estimate the reachable heap.
However, since this is only an approximation, it's possible for the
amount of allocation during the cycle to be *larger* than the marked
heap size (since the runtime allocates white and it's possible for
these allocations to never be made reachable from the heap). Currently
this causes wrap-around in our estimate of the reachable heap size,
which in turn causes wrap-around in next_gc. Fix this by bottoming out
the reachable heap estimate at 0, in which case we just fall back to
triggering GC at heapminimum (which is okay since this only happens on
small heaps).
Fixes#10555, fixes#10556, and fixes#10559.
Change-Id: Iad07b529c03772356fede2ae557732f13ebfdb63
Reviewed-on: https://go-review.googlesource.com/9286
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
To achieve a 2% improvement in the garbage benchmark this CL removes
an unneeded assert and avoids one hbits.next() call per object
being scanned.
Change-Id: Ibd542d01e9c23eace42228886f9edc488354df0d
Reviewed-on: https://go-review.googlesource.com/9244
Reviewed-by: Austin Clements <austin@google.com>
newosproc0 does not work on android/arm.
See issue #10548.
Change-Id: Ieaf6f5d0b77cddf5bf0b6c89fd12b1c1b8723f9b
Reviewed-on: https://go-review.googlesource.com/9293
Reviewed-by: David Crawshaw <crawshaw@golang.org>
There is an assumption that the function executed in child thread
created by runtime.close should not return. And different systems
enforce that differently: some exit that thread, some exit the
whole process.
The test TestNewOSProc0 introduced in CL 9161 breaks that assumption,
so we need to adjust the code to only exit the thread should the
called function return.
Change-Id: Id631cb2f02ec6fbd765508377a79f3f96c6a2ed6
Reviewed-on: https://go-review.googlesource.com/9246
Reviewed-by: Dave Cheney <dave@cheney.net>
Currently, we set the heap goal for the next GC cycle using the size
of the marked heap at the end of the current cycle. This can lead to a
bad feedback loop if the mutator is rapidly allocating and releasing
pointers that can significantly bloat heap size.
If the GC were STW, the marked heap size would be exactly the
reachable heap size (call it stwLive). However, in concurrent GC,
marked=stwLive+floatLive, where floatLive is the amount of "floating
garbage": objects that were reachable at some point during the cycle
and were marked, but which are no longer reachable by the end of the
cycle. If the GC cycle is short, then the mutator doesn't have much
time to create floating garbage, so marked≈stwLive. However, if the GC
cycle is long and the mutator is allocating and creating floating
garbage very rapidly, then it's possible that marked≫stwLive. Since
the runtime currently sets the heap goal based on marked, this will
cause it to set a high heap goal. This means that 1) the next GC cycle
will take longer because of the larger heap and 2) the assist ratio
will be low because of the large distance between the trigger and the
goal. The combination of these lets the mutator produce even more
floating garbage in the next cycle, which further exacerbates the
problem.
For example, on the garbage benchmark with GOMAXPROCS=1, this causes
the heap to grow to ~500MB and the garbage collector to retain upwards
of ~300MB of heap, while the true reachable heap size is ~32MB. This,
in turn, causes the GC cycle to take upwards of ~3 seconds.
Fix this bad feedback loop by estimating the true reachable heap size
(stwLive) and using this rather than the marked heap size
(stwLive+floatLive) as the basis for the GC trigger and heap goal.
This breaks the bad feedback loop and causes the mutator to assist
more, which decreases the rate at which it can create floating
garbage. On the same garbage benchmark, this reduces the maximum heap
size to ~73MB, the retained heap to ~40MB, and the duration of the GC
cycle to ~200ms.
Change-Id: I7712244c94240743b266f9eb720c03802799cdd1
Reviewed-on: https://go-review.googlesource.com/9177
Reviewed-by: Rick Hudson <rlh@golang.org>
This may or may not be useful to the end user, but it's incredibly
useful for us to understand the behavior of the pacer. Currently this
is fairly easy (though not trivial) to derive from the other heap
stats we print, but we're about to change how we compute the goal,
which will make it much harder to derive.
Change-Id: I796ef233d470c01f606bd9929820c01ece1f585a
Reviewed-on: https://go-review.googlesource.com/9176
Reviewed-by: Rick Hudson <rlh@golang.org>
The trigger controller computes GC CPU utilization by dividing by the
wall-clock time that's passed since concurrent mark began. Since this
delta is nanoseconds it's borderline impossible for it to be zero, but
if it is zero we'll currently divide by zero. Be robust to this
possibility by ignoring the utilization in the error term if no time
has elapsed.
Change-Id: I93dfc9e84735682af3e637f6538d1e7602634f09
Reviewed-on: https://go-review.googlesource.com/9175
Reviewed-by: Rick Hudson <rlh@golang.org>
We initially added clone0 to handle the case when G or M don't exist, but
it turns out that we could have just modified clone. (It also helps that
the function we're invoking in clone0 no longer needs arguments.)
As a side-effect, newosproc0 is now supported on all linux archs.
Change-Id: Ie603af75d8f164310fc16446052d83743961f3ca
Reviewed-on: https://go-review.googlesource.com/9164
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Unlike linux arm32, linux arm64 does not set the condition codes to indicate
whether a system call failed or not. We must check if the return value
is in the error code range (the same as amd64 does).
Fixes runtime.TestBadOpen test.
Change-Id: I97a8b0a17b5f002a3215c535efa91d199cee3309
Reviewed-on: https://go-review.googlesource.com/9220
Reviewed-by: Russ Cox <rsc@golang.org>
Several naming changes and a real issue in asmcgocall_errno.
Change-Id: Ieb0a328a168819fe233d74e0397358384d7e71b3
Reviewed-on: https://go-review.googlesource.com/9212
Reviewed-by: Minux Ma <minux@golang.org>
Currently, the GC controller computes the mutator assist ratio at the
beginning of the cycle by estimating that the marked heap size this
cycle will be the same as it was the previous cycle. It then uses that
assist ratio for the rest of the cycle. However, this means that if
the mutator is quickly growing its reachable heap, the heap size is
likely to exceed the heap goal and currently there's no additional
pressure on mutator assists when this happens. For example, 6g (with
GOMAXPROCS=1) frequently exceeds the goal heap size by ~25% because of
this.
This change makes GC revise its work estimate and the resulting assist
ratio every 10ms during the concurrent mark. Instead of
unconditionally using the marked heap size from the last cycle as an
estimate for this cycle, it takes the minimum of the previously marked
heap and the currently marked heap. As a result, as the cycle
approaches or exceeds its heap goal, this will increase the assist
ratio to put more pressure on the mutator assist to bring the cycle to
an end. For 6g, this causes the GC to always finish within 5% and
often within 1% of its heap goal.
Change-Id: I4333b92ad0878c704964be42c655c38a862b4224
Reviewed-on: https://go-review.googlesource.com/9070
Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Currently, in accordance with the GC pacing proposal, we schedule
background marking with a goal of achieving 25% utilization *total*
between mutator assists and background marking. This is stricter than
was set out in the Go 1.5 proposal, which suggests that the garbage
collector can use 25% just for itself and anything the mutator does to
help out is on top of that. It also has several technical
drawbacks. Because mutator assist time is constantly changing and we
can't have instantaneous information on background marking time, it
effectively requires hitting a moving target based on out-of-date
information. This works out in the long run, but works poorly for
short GC cycles and on short time scales. Also, this requires
time-multiplexing all Ps between the mutator and background GC since
the goal utilization of background GC constantly fluctuates. This
results in a complicated scheduling algorithm, poor affinity, and
extra overheads from context switching.
This change modifies the way we schedule and run background marking so
that background marking always consumes 25% of GOMAXPROCS and mutator
assist is in addition to this. This enables a much more robust
scheduling algorithm where we pre-determine the number of Ps we should
dedicate to background marking as well as the utilization goal for a
single floating "remainder" mark worker.
Change-Id: I187fa4c03ab6fe78012a84d95975167299eb9168
Reviewed-on: https://go-review.googlesource.com/9013
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, the concurrent sweep follows a 1:1 rule: when allocation
needs a span, it sweeps a span (likewise, when a large allocation
needs N pages, it sweeps until it frees N pages). This rule worked
well for the STW collector (especially when GOGC==100) because it did
no more sweeping than necessary to keep the heap from growing, would
generally finish sweeping just before GC, and ensured good temporal
locality between sweeping a page and allocating from it.
It doesn't work well with concurrent GC. Since concurrent GC requires
starting GC earlier (sometimes much earlier), the sweep often won't be
done when GC starts. Unfortunately, the first thing GC has to do is
finish the sweep. In the mean time, the mutator can continue
allocating, pushing the heap size even closer to the goal size. This
worked okay with the 7/8ths trigger, but it gets into a vicious cycle
with the GC trigger controller: if the mutator is allocating quickly
and driving the trigger lower, more and more sweep work will be left
to GC; this both causes GC to take longer (allowing the mutator to
allocate more during GC) and delays the start of the concurrent mark
phase, which throws off the GC controller's statistics and generally
causes it to push the trigger even lower.
As an example of a particularly bad case, the garbage benchmark with
GOMAXPROCS=4 and -benchmem 512 (MB) spends the first 0.4-0.8 seconds
of each GC cycle sweeping, during which the heap grows by between
109MB and 252MB.
To fix this, this change replaces the 1:1 sweep rule with a
proportional sweep rule. At the end of GC, GC knows exactly how much
heap allocation will occur before the next concurrent GC as well as
how many span pages must be swept. This change computes this "sweep
ratio" and when the mallocgc asks for a span, the mcentral sweeps
enough spans to bring the swept span count into ratio with the
allocated byte count.
On the benchmark from above, this entirely eliminates sweeping at the
beginning of GC, which reduces the time between startGC readying the
GC goroutine and GC stopping the world for sweep termination to ~100µs
during which the heap grows at most 134KB.
Change-Id: I35422d6bba0c2310d48bb1f8f30a72d29e98c1af
Reviewed-on: https://go-review.googlesource.com/8921
Reviewed-by: Rick Hudson <rlh@golang.org>
This field used to decrease with sweeps (and potentially go
negative). Now it is always zero or positive, so change it to a
uintptr so it meshes better with other memory stats.
Change-Id: I6a50a956ddc6077eeaf92011c51743cb69540a3c
Reviewed-on: https://go-review.googlesource.com/8899
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, concurrent GC triggers at a fixed 7/8*GOGC heap growth. For
mutators that allocate slowly, this means GC will trigger too early
and run too often, wasting CPU time on GC. For mutators that allocate
quickly, this means GC will trigger too late, causing the program to
exceed the GOGC heap growth goal and/or to exceed CPU goals because of
a high mutator assist ratio.
This change adds a feedback control loop to dynamically adjust the GC
trigger from cycle to cycle. By monitoring the heap growth and GC CPU
utilization from cycle to cycle, this adjusts the Go garbage collector
to target the GOGC heap growth goal and the 25% CPU utilization goal.
Change-Id: Ic82eef288c1fa122f73b69fe604d32cbb219e293
Reviewed-on: https://go-review.googlesource.com/8851
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, the concurrent mark phase is performed by the main GC
goroutine. Prior to the previous commit enabling preemption, this
caused marking to always consume 1/GOMAXPROCS of the available CPU
time. If GOMAXPROCS=1, this meant background GC would consume 100% of
the CPU (effectively a STW). If GOMAXPROCS>4, background GC would use
less than the goal of 25%. If GOMAXPROCS=4, background GC would use
the goal 25%, but if the mutator wasn't using the remaining 75%,
background marking wouldn't take advantage of the idle time. Enabling
preemption in the previous commit made GC miss CPU targets in
completely different ways, but set us up to bring everything back in
line.
This change replaces the fixed GC goroutine with per-P background mark
goroutines. Once started, these goroutines don't go in the standard
run queues; instead, they are scheduled specially such that the time
spent in mutator assists and the background mark goroutines totals 25%
of the CPU time available to the program. Furthermore, this lets
background marking take advantage of idle Ps, which significantly
boosts GC performance for applications that under-utilize the CPU.
This requires also changing how time is reported for gctrace, so this
change splits the concurrent mark CPU time into assist/background/idle
scanning.
This also requires increasing the size of the StackRecord slice used
in a GoroutineProfile test.
Change-Id: I0936ff907d2cee6cb687a208f2df47e8988e3157
Reviewed-on: https://go-review.googlesource.com/8850
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, the entire GC process runs with g.m.preemptoff set. In the
concurrent phases, the parts that actually need preemption disabled
are run on a system stack and there's no overall need to stay on the
same M or P during the concurrent phases. Hence, move the setting of
g.m.preemptoff to when we start mark termination, at which point we
really do need preemption disabled.
This dramatically changes the scheduling behavior of the concurrent
mark phase. Currently, since this is non-preemptible, concurrent mark
gets one dedicated P (so 1/GOMAXPROCS utilization). With this change,
the GC goroutine is scheduled like any other goroutine during
concurrent mark, so it gets 1/<runnable goroutines> utilization.
You might think it's not even necessary to set g.m.preemptoff at that
point since the world is stopped, but stackalloc/stackfree use this as
a signal that the per-P pools are not safe to access without
synchronization.
Change-Id: I08aebe8179a7d304650fb8449ff36262b3771099
Reviewed-on: https://go-review.googlesource.com/8839
Reviewed-by: Rick Hudson <rlh@golang.org>
This time is tracked per P and periodically flushed to the global
controller state. This will be used to compute mutator assist
utilization in order to schedule background GC work.
Change-Id: Ib94f90903d426a02cf488bf0e2ef67a068eb3eec
Reviewed-on: https://go-review.googlesource.com/8837
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, mutator allocation periodically assists the garbage
collector by performing a small, fixed amount of scanning work.
However, to control heap growth, mutators need to perform scanning
work *proportional* to their allocation rate.
This change implements proportional mutator assists. This uses the
scan work estimate computed by the garbage collector at the beginning
of each cycle to compute how much scan work must be performed per
allocation byte to complete the estimated scan work by the time the
heap reaches the goal size. When allocation triggers an assist, it
uses this ratio and the amount allocated since the last assist to
compute the assist work, then attempts to steal as much of this work
as possible from the background collector's credit, and then performs
any remaining scan work itself.
Change-Id: I98b2078147a60d01d6228b99afd414ef857e4fba
Reviewed-on: https://go-review.googlesource.com/8836
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, the "n" in gcDrainN is in terms of objects to scan. This is
used by gchelpwork to perform a limited amount of work on allocation,
but is a pretty arbitrary way to bound this amount of work since the
number of objects has little relation to how long they take to scan.
Modify gcDrainN to perform a fixed amount of scan work instead. For
now, gchelpwork still performs a fairly arbitrary amount of scan work,
but at least this is much more closely related to how long the work
will take. Shortly, we'll use this to precisely control the scan work
performed by mutator assists during allocation to achieve the heap
size goal.
Change-Id: I3cd07fe0516304298a0af188d0ccdf621d4651cc
Reviewed-on: https://go-review.googlesource.com/8835
Reviewed-by: Rick Hudson <rlh@golang.org>
This tracks scan work done by background GC in a global pool. Mutator
assists will draw on this credit to avoid doing work when background
GC is staying ahead.
Unlike the other GC controller tracking variables, this will be both
written and read throughout the cycle. Hence, we can't arbitrarily
delay updates like we can for scan work and bytes marked. However, we
still want to minimize contention, so this global credit pool is
allowed some error from the "true" amount of credit. Background GC
accumulates credit locally up to a limit and only then flushes to the
global pool. Similarly, mutator assists will draw from the credit pool
in batches.
Change-Id: I1aa4fc604b63bf53d1ee2a967694dffdfc3e255e
Reviewed-on: https://go-review.googlesource.com/8834
Reviewed-by: Rick Hudson <rlh@golang.org>
This implements tracking the scan work ratio of a GC cycle and using
this to estimate the scan work that will be required by the next GC
cycle. Currently this estimate is unused; it will be used to drive
mutator assists.
Change-Id: I8685b59d89cf1d83eddfc9b30d84da4e3a7f4b72
Reviewed-on: https://go-review.googlesource.com/8833
Reviewed-by: Rick Hudson <rlh@golang.org>
This tracks the amount of scan work in terms of scanned pointers
during the concurrent mark phase. We'll use this information to
estimate scan work for the next cycle.
Currently this aggregates the work counter in gcWork and dispose
atomically aggregates this into a global work counter. dispose happens
relatively infrequently, so the contention on the global counter
should be low. If this turns out to be an issue, we can reduce the
number of disposes, and if it's still a problem, we can switch to
per-P counters.
Change-Id: Iac0364c466ee35fab781dbbbe7970a5f3c4e1fc1
Reviewed-on: https://go-review.googlesource.com/8832
Reviewed-by: Rick Hudson <rlh@golang.org>
These currently use portable implementations in terms of their uint64
counterparts.
Change-Id: Icba5f7134cfcf9d0429edabcdd73091d97e5e905
Reviewed-on: https://go-review.googlesource.com/8831
Reviewed-by: Rick Hudson <rlh@golang.org>
This change exposes reflect.ArrayOf to create new reflect.Type array
types at runtime, when given a reflect.Type element.
- reflect: implement ArrayOf
- reflect: tests for ArrayOf
- runtime: document that typeAlg is used by reflect and must be kept in
synchronized
Fixes#5996.
Change-Id: I5d07213364ca915c25612deea390507c19461758
Reviewed-on: https://go-review.googlesource.com/4111
Reviewed-by: Keith Randall <khr@golang.org>
Optimized heapBitsForObject by special casing
objects whose size is a power of two. When a
span holding such objects is initialized I
added a mask that when &ed with an interior pointer
results in the base of the pointer. For the garbage
benchmark this resulted in CPU_CLK_UNHALTED in
heapBitsForObject going from 7.7% down to 5.9%
of the total, INST_RETIRED went from 12.2 -> 8.7.
Here are the benchmarks that were at lease plus or minus 1%.
benchmark old ns/op new ns/op delta
BenchmarkFmtFprintfString 249 221 -11.24%
BenchmarkFmtFprintfInt 247 223 -9.72%
BenchmarkFmtFprintfEmpty 76.5 69.6 -9.02%
BenchmarkBinaryTree17 4106631412 3744550160 -8.82%
BenchmarkFmtFprintfFloat 424 399 -5.90%
BenchmarkGoParse 4484421 4242115 -5.40%
BenchmarkGobEncode 8803668 8449107 -4.03%
BenchmarkFmtManyArgs 1494 1436 -3.88%
BenchmarkGobDecode 10431051 10032606 -3.82%
BenchmarkFannkuch11 2591306713 2517400464 -2.85%
BenchmarkTimeParse 361 371 +2.77%
BenchmarkJSONDecode 70620492 68830357 -2.53%
BenchmarkRegexpMatchMedium_1K 54693 53343 -2.47%
BenchmarkTemplate 90008879 91929940 +2.13%
BenchmarkTimeFormat 380 387 +1.84%
BenchmarkRegexpMatchEasy1_32 111 113 +1.80%
BenchmarkJSONEncode 21359159 21007583 -1.65%
BenchmarkRegexpMatchEasy1_1K 603 613 +1.66%
BenchmarkRegexpMatchEasy0_32 127 129 +1.57%
BenchmarkFmtFprintfIntInt 399 393 -1.50%
BenchmarkRegexpMatchEasy0_1K 373 378 +1.34%
Change-Id: I78e297161026f8b5cc7507c965fd3e486f81ed29
Reviewed-on: https://go-review.googlesource.com/8980
Reviewed-by: Austin Clements <austin@google.com>
This CL revises CL 7504 to use explicitly uintptr types for the
struct fields that are going to be updated sometimes without
write barriers. The result is that the fields are now updated *always*
without write barriers.
This approach has two important properties:
1) Now the GC never looks at the field, so if the missing reference
could cause a problem, it will do so all the time, not just when the
write barrier is missed at just the right moment.
2) Now a write barrier never happens for the field, avoiding the
(correct) detection of inconsistent write barriers when GODEBUG=wbshadow=1.
Change-Id: Iebd3962c727c0046495cc08914a8dc0808460e0e
Reviewed-on: https://go-review.googlesource.com/9019
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The callee-saved registers must be saved because for the c-shared case
this code is invoked from C code in the system library, and that code
expects the registers to be saved. The tests were passing because in
the normal case the code calls a cgo function that naturally saves
callee-saved registers anyhow. However, it fails when the code takes
the non-cgo path.
Change-Id: I9c1f5e884f5a72db9614478049b1863641c8b2b9
Reviewed-on: https://go-review.googlesource.com/9114
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Change-Id: I87147ca6bb53e3121cc4245449c519509f107638
Reviewed-on: https://go-review.googlesource.com/9009
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
It's not helping anymore, and it's fooling people who try to
understand performance (like me).
Change-Id: I133a644acae0ddf1bfa17c654cdc01e2089da963
Reviewed-on: https://go-review.googlesource.com/9018
Reviewed-by: Austin Clements <austin@google.com>
readyExecute passes a closure to mcall that captures an argument to
readyExecute. Since mcall is marked noescape, this closure lives on
the stack of the calling goroutine. However, the closure puts the
calling goroutine on the run queue (and switches to a new
goroutine). If the calling goroutine gets scheduled before the mcall
returns, this stack-allocated closure will become invalid while it's
still executing. One consequence of this we've observed is that the
captured gp variable can get overwritten before the call to
execute(gp), causing execute(gp) to segfault.
Fix this by passing the currently captured gp variable through a field
in the calling goroutine's g struct so that the func is no longer a
closure.
To prevent problems like this in the future, this change also removes
the go:noescape annotation from mcall. Due to a compiler bug, this
will currently cause a func closure passed to mcall to be implicitly
allocated rather than refusing the implicit allocation. However, this
is okay because there are no other closures passed to mcall right now
and the compiler bug will be fixed shortly.
Fixes#10428.
Change-Id: I49b48b85de5643323b89e9eaa4df63854e968c32
Reviewed-on: https://go-review.googlesource.com/8866
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Previously we started the Go runtime from a JNI function call, which
eventually called the program's main function. Now the runtime is
initialized by an ELF initialization function as a c-shared library,
and the program's main function is not called. So now we export main
so it can be called from JNI.
This is necessary for all-Go apps because unlike a normal shared
library, the program loading the library is not written by or known
to the programmer. As far as they are concerned, the .so is
everything. In fact the same code is compiled for iOS as a normal Go
program.
Change-Id: I61c6a92243240ed229342362231b1bfc7ca526ba
Reviewed-on: https://go-review.googlesource.com/9015
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Change-Id: Ie7f85873978adf3fd5c739176f501ca219592824
Reviewed-on: https://go-review.googlesource.com/9011
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This memory is untyped and can't be used anymore.
The next version of SWIG won't need it.
Change-Id: I592b287c5f5186975ee09a9b28d8efe3b57134e7
Reviewed-on: https://go-review.googlesource.com/8956
Reviewed-by: Ian Lance Taylor <iant@golang.org>
For some reason the absense of an implementation does not stop arm64
binaries being built. However it comes up with -buildmode=c-archive.
Change-Id: Ic0db5fd8fb4fe8252b5aa320818df0c7aec3db8f
Reviewed-on: https://go-review.googlesource.com/8989
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change-Id: I8e912ff9327a4163b63b8c628aa3546e86ddcc02
Reviewed-on: https://go-review.googlesource.com/8983
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Change-Id: I09e84161d106960a69972f5fc845a1e40c28e58f
Reviewed-on: https://go-review.googlesource.com/8331
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It is faster to execute
MOVQ AX,(DI)
MOVQ AX,8(DI)
MOVQ AX,16(DI)
MOVQ AX,24(DI)
ADDQ $32,DI
than
STOSQ
STOSQ
STOSQ
STOSQ
However, in order to be able to jump into
the middle of a block of MOVQs, the call
site needs to pre-adjust DI.
If we're clearing a small area, the cost
of that DI pre-adjustment isn't repaid.
This CL switches the DUFFZERO implementation
to use a hybrid strategy, in which small
clears use STOSQ as before, but large clears
use mostly MOVQ/ADDQ blocks.
benchmark old ns/op new ns/op delta
BenchmarkClearFat8 0.55 0.55 +0.00%
BenchmarkClearFat12 0.82 0.83 +1.22%
BenchmarkClearFat16 0.55 0.55 +0.00%
BenchmarkClearFat24 0.82 0.82 +0.00%
BenchmarkClearFat32 2.20 1.94 -11.82%
BenchmarkClearFat40 1.92 1.66 -13.54%
BenchmarkClearFat48 2.21 1.93 -12.67%
BenchmarkClearFat56 3.03 2.20 -27.39%
BenchmarkClearFat64 3.26 2.48 -23.93%
BenchmarkClearFat72 3.57 2.76 -22.69%
BenchmarkClearFat80 3.83 3.05 -20.37%
BenchmarkClearFat88 4.14 3.30 -20.29%
BenchmarkClearFat128 5.54 4.69 -15.34%
BenchmarkClearFat256 9.95 9.09 -8.64%
BenchmarkClearFat512 18.7 17.9 -4.28%
BenchmarkClearFat1024 36.2 35.4 -2.21%
Change-Id: Ic786406d9b3cab68d5a231688f9e66fcd1bd7103
Reviewed-on: https://go-review.googlesource.com/2585
Reviewed-by: Keith Randall <khr@golang.org>
By removing type slice, renaming type sliceStruct to type slice and
whacking until it compiles.
Has a pleasing net reduction of conversions.
Fixes#10188
Change-Id: I77202b8df637185b632fd7875a1fdd8d52c7a83c
Reviewed-on: https://go-review.googlesource.com/8770
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes#10450
runtime.cputicks is called from runtime.exitsyscall and must not
split the stack. cputicks is implemented in several ways and the
NOSPLIT annotation was missing from a few of these.
Change-Id: I5cbbb4e5888c5d298fe2fef240782d0e49f59af8
Reviewed-on: https://go-review.googlesource.com/8939
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
When Windows calls externalthreadhandler it expects to receive
return value in AX. We don't set AX anywhere. Change that.
Store ctrlhandler1 and profileloop1 return values into AX before
returning from externalthreadhandler.
Fixes#10215.
Change-Id: Ied04542cc3ebe7d4a26660e970f9f78098143591
Reviewed-on: https://go-review.googlesource.com/8901
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A G will be preempted if it runs for 10ms without blocking. Currently
this constant is hard-coded in retake. Move it to a global const.
We'll use the time slice length in scheduling background GC.
Change-Id: I79a979948af2fad3afe5df9d4af4062f166554b7
Reviewed-on: https://go-review.googlesource.com/8838
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
mHeap_ReclaimList is asked to reclaim at least npages pages, but it
counts the number of spans reclaimed, not the number of pages
reclaimed. The number of spans reclaimed is strictly larger than the
number of pages, so this is not strictly wrong, but it is forcing more
reclamation than was intended by the caller, which delays large
allocations.
Fix this by increasing the count by the number of pages in the swept
span, rather than just increasing it by 1.
Fixes#9048.
Change-Id: I5ae364a9837a6012e68fcd431bba000340cfd50c
Reviewed-on: https://go-review.googlesource.com/8920
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
Commit d7e0ad4 removed the next_gc manipulation from mSpan_Sweep, but
left in the traceNextGC() for recording the updated next_gc
value. Remove this now unnecessary call.
Change-Id: I28e0de071661199be9810d7bdcc81ce50b5a58ae
Reviewed-on: https://go-review.googlesource.com/8894
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
With the new buildmodes c-archive and c-shared, it is possible for a
cgo call to come in early in the lifecycle of a Go program. Calls
before the runtime has been initialized are caught by
_cgo_wait_runtime_init_done. However a call can come in after the
runtime has initialized, but before the program's package init
functions have finished running.
To avoid this cgocallback checks m.ncgo to see if we are on a thread
running Go. If not, we may be a foreign thread and it blocks until
main_init is complete.
Change-Id: I7a9f137fa2a40c322a0b93764261f9aa17fcf5b8
Reviewed-on: https://go-review.googlesource.com/8897
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Avoids shadowing the builtin channel close function.
Change-Id: I7a729b0937c8248fe27222be61318a88db995eee
Reviewed-on: https://go-review.googlesource.com/8898
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Follows http://golang.org/cl/8454, a similar CL for arm architectures.
This CL involves android-specific changes, namely, synthesizing
argv/auxv, as android doesn't provide those to the init functions.
This code is based on crawshaw@ android code in golang.org/x/mobile.
Change-Id: I32364efbb2662e80270a99bd7dfb1d0421b5417d
Reviewed-on: https://go-review.googlesource.com/8457
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Adds the runtime initialization flow for arm akin to amd64.
In particular,we use the library initialization entry point to:
- create a new OS thread and run the "regular" runtime init stack on
that thread
- return immediately from the main (i.e., loader) thread
- at the first CGO invocation, we wait for the runtime initialization
to complete.
Verified to work on a Raspberry Pi and an Android phone.
Change-Id: I32f39228ae30a03ce9569287f234b305790fecf6
Reviewed-on: https://go-review.googlesource.com/8455
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Srdjan Petrovic <spetrovic@google.com>
Related to issue #10410
For some reason, any non-trivial code in _cgo_wait_runtime_init_done
(even fprintf()) will crash that call.
If anybody has any guess why this is happening, please let me know!
For now, I'm clearing the functions for ppc64, as it's currently not used.
Change-Id: I1b11383aaf4f9f9a16f1fd6606842cfeedc9f0b3
Reviewed-on: https://go-review.googlesource.com/8766
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Srdjan Petrovic <spetrovic@google.com>
Just like darwin/arm.
Change-Id: Ic75927bd6457d37cda7dd8279fd9b4cd52edc1d1
Reviewed-on: https://go-review.googlesource.com/8813
Reviewed-by: Minux Ma <minux@golang.org>
Like other arm64 platforms, darwin/arm64 has a different physical
page size to logical page size so it is running into issue 9993. I
hope it can be fixed for Go 1.5, but for now it is demonstrating the
same bug as the other skipped os+arch combinations.
Change-Id: Iedaf9afe56d6954bb4391b6e843d81742a75a00c
Reviewed-on: https://go-review.googlesource.com/8814
Reviewed-by: Minux Ma <minux@golang.org>
Just like darwin/arm.
Change-Id: Ie4998d24b2d891a9f6c8047ec40cd3fdf80622cd
Reviewed-on: https://go-review.googlesource.com/8812
Reviewed-by: Minux Ma <minux@golang.org>
Tested by using -buildmode=c-archive to generate an archive, add it
to an Xcode project and calling a Go function from an iOS app. (I'm
still investigating proper buildmode tests for all.bash.)
Change-Id: I7890df15246df8e90ad27837b8d64ba2cde409fe
Reviewed-on: https://go-review.googlesource.com/8719
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A similar fix was applied in 545686857b
but another instance of 'pc' was missed.
Also adds a test for the goroutine gdb command.
It currently uses goroutine 2 for the test, since goroutine 1 has
its stack pointer set to 0 for some reason.
Change-Id: I53ca22be6952f03a862edbdebd9b5c292e0853ae
Reviewed-on: https://go-review.googlesource.com/8729
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, when allocation reaches the concurrent GC trigger size, we
start the concurrent collector by ready'ing its G. This simply puts it
on the end of the P's run queue, which means we may not actually start
GC for some time as the current G continues to run and then the P
drains other Gs already on its run queue. Since the mutator can
continue to allocate, the heap can potentially be much larger than we
intended by the time GC actually starts. Furthermore, how much larger
is difficult to predict since it depends on the scheduler.
Fix this by preempting the current G and switching directly to the
concurrent GC G as soon as we reach the trigger heap size.
On the garbage benchmark from the benchmarks subrepo with
GOMAXPROCS=4, this reduces the time from triggering the GC to the
beginning of sweep termination by 10 to 30 milliseconds, which reduces
allocation after the trigger by up to 10MB (a large fraction of the
64MB live heap the benchmark tries to maintain).
One other known source of delay before we "really" start GC is the
sweep finalization performed before sweep termination. This has
similar negative effects on heap size and predictability, but is an
orthogonal problem. This change adds a TODO for this.
Change-Id: I8bae98cb43685c1bf353ff55868e4647e3743c47
Reviewed-on: https://go-review.googlesource.com/8513
Reviewed-by: Rick Hudson <rlh@golang.org>
These were appropriate for STW GC, since it interrupted the allocating
Goroutine, but don't apply to concurrent GC, which runs on its own
Goroutine. Forced GC is still STW, but it makes sense to attribute the
GC to the goroutine that called runtime.GC().
Change-Id: If12418ca66dc7e53b8b16025af4e03adb5d9577e
Reviewed-on: https://go-review.googlesource.com/8715
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
exitsyscallfast checks for freezetheworld, but does so only by
checking if stopwait is positive. This can also happen during
stoptheworld, which is harmless, but confusing. Shortly, it will be
important that we get to the p.status cas even if stopwait is set.
Hence, make this test more specific so it only triggers with
freezetheworld and not other uses of stopwait.
Change-Id: Ibb722cd8360c3ed5a9654482519e3ceb87a8274d
Reviewed-on: https://go-review.googlesource.com/8205
Reviewed-by: Russ Cox <rsc@golang.org>
'themoduledata' doesn't really make sense now we support multiple moduledata
objects.
Change-Id: I8263045d8f62a42cb523502b37289b0fba054f62
Reviewed-on: https://go-review.googlesource.com/8521
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This changes all the places that consult themoduledata to consult a
linked list of moduledata objects, as will be necessary for
-linkshared to work.
Obviously, as there is as yet no way of adding moduledata objects to
this list, all this change achieves right now is wasting a few
instructions here and there.
Change-Id: I397af7f60d0849b76aaccedf72238fe664867051
Reviewed-on: https://go-review.googlesource.com/8231
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, the initial heap size reported in the gctrace line is the
heap_live right before sweep termination. However, we triggered GC
when heap_live reached next_gc, and there may have been significant
allocation between that point and the beginning of sweep
termination. Ideally these would be essentially the same, but
currently there's scheduler delay when readying the GC goroutine as
well as delay from background sweep finalization.
We should fix this delay, but in the mean time, to give the user a
better idea of how much the heap grew during the whole of garbage
collection, report the trigger rather than what the heap size happened
to be after the garbage collector finished rolling out of bed. This
will also be more useful for heap growth plots.
Change-Id: I08476b9fbcfb2de90592405e9c9f434dfb9eb1f8
Reviewed-on: https://go-review.googlesource.com/8512
Reviewed-by: Rick Hudson <rlh@golang.org>
According to Go execution modes, a Go program compiled with
-buildmode=c-archive has a main function, but it is ignored on run.
This gives the runtime the information it needs not to run the main.
I have this working with pending linker changes on darwin/amd64.
Change-Id: I49bd7d65aa619ec847c464a872afa5deea7d4d30
Reviewed-on: https://go-review.googlesource.com/8701
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add arm64 assembly implementation of runtime.cmpstring and bytes.Compare.
benchmark old ns/op new ns/op delta
BenchmarkCompareBytesEqual 98.0 27.5 -71.94%
BenchmarkCompareBytesToNil 9.38 10.0 +6.61%
BenchmarkCompareBytesEmpty 13.3 10.0 -24.81%
BenchmarkCompareBytesIdentical 98.0 27.5 -71.94%
BenchmarkCompareBytesSameLength 43.3 16.3 -62.36%
BenchmarkCompareBytesDifferentLength 43.4 16.3 -62.44%
BenchmarkCompareBytesBigUnaligned 6979680 1360979 -80.50%
BenchmarkCompareBytesBig 6915995 1381979 -80.02%
BenchmarkCompareBytesBigIdentical 6781440 1327304 -80.43%
benchmark old MB/s new MB/s speedup
BenchmarkCompareBytesBigUnaligned 150.23 770.46 5.13x
BenchmarkCompareBytesBig 151.62 758.76 5.00x
BenchmarkCompareBytesBigIdentical 154.63 790.01 5.11x
* note, the machine we are benchmarking on has some issues. What is clear is
compared to a few days ago the old MB/s value has increased from ~115 to 150.
I'm less certain about the new MB/s number, which used to be close to 1Gb/s.
Change-Id: I4f31b2c7a06296e13912aacc958525632cb0450d
Reviewed-on: https://go-review.googlesource.com/8541
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Mainly it is simple copy. But I had to change amd64
lastcontinuehandler return value from uint32 to int32.
I don't remember how it happened to be uint32, but new
int32 is matching better with Windows documentation (LONG).
I don't think it matters one way or the others.
Change-Id: I6935224a2470ad6301e27590f2baa86c13bbe8d5
Reviewed-on: https://go-review.googlesource.com/8686
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In cl/8652 I broke darwin/arm and darwin/386 because I removed the *g
parameter, which they both expect and use. This CL adjusts both ports
to look for g0 in m, just as darwin/amd64 does.
Tested on darwin{386,arm,amd64}.
Change-Id: Ia56f3d97e126b40d8bbd2e8f677b008e4a1badad
Reviewed-on: https://go-review.googlesource.com/8666
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is a practice run for darwin/arm.
Similar to the linux/amd64 shared library entry point. With several
pending linker changes I am successfully using this to implement
-buildmode=c-archive on darwin/amd64 with external linking.
The same entry point can be reused to implement -buildmode=c-shared
on darwin/amd64, however that will require further ld changes to
remove all text relocations.
One extra runtime change will follow this. According to the Go
execution modes document, -buildmode=c-archive should ignore the Go
main function. Right now it is being executed (and the process exits
if it doesn't block). I'm still searching for the right way to do
this.
Change-Id: Id97901ddd4d46970996f222bd79731dabff66a3d
Reviewed-on: https://go-review.googlesource.com/8652
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL is quite conservative in some ways. It continues to define
symbols that have no real purpose (e.g. epclntab). These could be
deleted if there is no concern that external tools might look for them.
It would also now be possible to make some changes to the pcln data but
I get the impression that would definitely require some thought and
discussion.
Change-Id: Ib33cde07e4ec38ecc1d6c319a10138c9347933a3
Reviewed-on: https://go-review.googlesource.com/7616
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The return type for bsdthread_register is int32. See
runtime/os_darwin.go.
This change also rewrites declaration comments for go functions to
use go syntax and fixes vet errors in sys_darwin_amd64.s.
Change-Id: I7482105f7562929e0ede30099efac9e76babd8a3
Reviewed-on: https://go-review.googlesource.com/3260
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
At the moment this function does nothing, runtime initialization is
still done in android.c:init_go_runtime.
Fixes#10358
Change-Id: I1d762383ba61efcbcf0bbc7c77895f5c1dbf8968
Reviewed-on: https://go-review.googlesource.com/8510
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
When the gctrace GODEBUG option is enabled, it will now report three
heap sizes: the heap size at the beginning of the GC cycle, the heap
size at the end of the GC cycle before sweeping, and marked heap size,
which is the amount of heap that will be retained until the next GC
cycle.
Change-Id: Ie13f8a6d5c609bc9cc47c7555960ab55b37b5f1c
Reviewed-on: https://go-review.googlesource.com/8430
Reviewed-by: Rick Hudson <rlh@golang.org>
In the STW collector, next_gc was both the heap size to trigger GC at
as well as the goal heap size.
Early in the concurrent collector's development, next_gc was the goal
heap size, but was also used as the heap size to trigger GC at. This
meant we always overshot the goal because of allocation during
concurrent GC.
Currently, next_gc is still the goal heap size, but we trigger
concurrent GC at 7/8*GOGC heap growth. This complicates
shouldtriggergc, but was necessary because of the incremental
maintenance of next_gc.
Now we simply compute next_gc for the next cycle during mark
termination. Hence, it's now easy to take the simpler route and
redefine next_gc as the heap size at which the next GC triggers. We
can directly compute this with the 7/8 backoff during mark termination
and shouldtriggergc can simply test if the live heap size has grown
over the next_gc trigger.
This will also simplify later changes once we start setting next_gc in
more sophisticated ways.
Change-Id: I872be4ae06b4f7a0d7f7967360a054bd36b90eea
Reviewed-on: https://go-review.googlesource.com/8420
Reviewed-by: Russ Cox <rsc@golang.org>
Currently there are two main consumers of memstats.heap_alloc:
updatememstats (aka ReadMemStats) and shouldtriggergc.
updatememstats recomputes heap_alloc from the ground up, so we don't
need to keep heap_alloc up to date for it. shouldtriggergc wants to
know how many bytes were marked by the previous GC plus how many bytes
have been allocated since then, but this *isn't* what heap_alloc
tracks. heap_alloc also includes objects that are not marked and
haven't yet been swept.
Introduce a new memstat called heap_live that actually tracks what
shouldtriggergc wants to know and stop keeping heap_alloc up to date.
Unlike heap_alloc, heap_live follows a simple sawtooth that drops
during each mark termination and increases monotonically between GCs.
heap_alloc, on the other hand, has much more complicated behavior: it
may drop during sweep termination, slowly decreases from background
sweeping between GCs, is roughly unaffected by allocation as long as
there are unswept spans (because we sweep and allocate at the same
rate), and may go up after background sweeping is done depending on
the GC trigger.
heap_live simplifies computing next_gc and using it to figure out when
to trigger garbage collection. Currently, we guess next_gc at the end
of a cycle and update it as we sweep and get a better idea of how much
heap was marked. Now, since we're directly tracking how much heap is
marked, we can directly compute next_gc.
This also corrects bugs that could cause us to trigger GC early.
Currently, in any case where sweep termination actually finds spans to
sweep, heap_alloc is an overestimation of live heap, so we'll trigger
GC too early. heap_live, on the other hand, is unaffected by sweeping.
Change-Id: I1f96807b6ed60d4156e8173a8e68745ffc742388
Reviewed-on: https://go-review.googlesource.com/8389
Reviewed-by: Russ Cox <rsc@golang.org>
This tracks the number of heap bytes marked by a GC cycle. We'll use
this information to precisely trigger the next GC cycle.
Currently this aggregates the work counter in gcWork and dispose
atomically aggregates this into a global work counter. dispose happens
relatively infrequently, so the contention on the global counter
should be low. If this turns out to be an issue, we can reduce the
number of disposes, and if it's still a problem, we can switch to
per-P counters.
Change-Id: I1bc377cb2e802ef61c2968602b63146d52e7f5db
Reviewed-on: https://go-review.googlesource.com/8388
Reviewed-by: Russ Cox <rsc@golang.org>
I guess we need more builders.
Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9422e4
Reviewed-on: https://go-review.googlesource.com/8434
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
This is Part 2 of the change, see Part 1 here: in https://go-review.googlesource.com/#/c/7692/
Suggested by iant@, we use the library initialization entry point to:
- create a new OS thread and run the "regular" runtime init stack on
that thread
- return immediately from the main (i.e., loader) thread
- at the first CGO invocation, we wait for the runtime initialization
to complete.
The above mechanism is implemented only on linux_amd64. Next step is to
support it on linux_arm. Other platforms don't yet support shared library
compiling/linking, but we intend to use the same strategy there as well.
Change-Id: Ib2c81b1b83bee837134084b75a3beecfb8de6bf4
Reviewed-on: https://go-review.googlesource.com/8094
Run-TryBot: Srdjan Petrovic <spetrovic@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This tracks both total CPU time used by GC and the total time
available to all Ps since the beginning of the program and uses this
to derive a cumulative CPU usage percent for the gctrace line.
Change-Id: Ica85372b8dd45f7621909b325d5ac713a9b0d015
Reviewed-on: https://go-review.googlesource.com/8350
Reviewed-by: Russ Cox <rsc@golang.org>
GODEBUG=gctrace=1 turns on a per-GC cycle trace line. The current line
is left over from the STW garbage collector and includes a lot of
information that is no longer meaningful for the concurrent GC and
doesn't include a lot of information that is important.
Replace this line with a new line designed for the new garbage
collector.
This new line is focused more on helping the user understand the
impact of the garbage collector on their program and less on telling
us, the runtime developers, everything that's happening inside
GC. It's designed to fit in 80 columns and intentionally omit some
potentially useful things that were in the old line. We might want a
"verbose" mode that adds information for us.
We'll be able to further simplify the line once we eliminate the STW
around enabling the write barrier. Then we'll have just one STW phase,
one concurrent phase, and one more STW phase, so we'll be able to
reduce the number of times from five to three.
Change-Id: Icc30939fe4576fb4491b4eac811649395727aa2a
Reviewed-on: https://go-review.googlesource.com/8208
Reviewed-by: Russ Cox <rsc@golang.org>
Currently hashmap is riddled with code that attempts to force a GC on
the next allocation if checkgc is set. This no longer works as
originally intended with the concurrent collector, and is apparently
no longer used anyway.
Remove checkgc.
Change-Id: Ia6c17c405fa8821dc2e6af28d506c1133ab1ca0c
Reviewed-on: https://go-review.googlesource.com/8355
Reviewed-by: Keith Randall <khr@golang.org>
This tries to clarify that Alloc and HeapAlloc are tied to how much
freeing has been done by the sweeper.
Change-Id: Id8320074bd75de791f39ec01bac99afe28052d02
Reviewed-on: https://go-review.googlesource.com/8354
Reviewed-by: Rick Hudson <rlh@golang.org>
This makes it easier to experiment with alternative implementations.
While we're here, update the comments.
No functional changes. Passes toolstash -cmp.
Change-Id: I428535754908f0fdd7cc36c214ddb6e1e60f376e
Reviewed-on: https://go-review.googlesource.com/8310
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In preparation for being able to run a go program that has code
in several objects, this changes from having several linker
symbols used by the runtime into having one linker symbol that
points at a structure containing the needed data. Multiple
object support will construct a linked list of such structures.
A follow up will initialize the slices in the themoduledata
structure directly from the linker but I was aiming for a minimal
diff for now.
Change-Id: I613cce35309801cf265a1d5ae5aaca8d689c5cbf
Reviewed-on: https://go-review.googlesource.com/7441
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, gcDrainN is documented saying that it must be run on the
system stack. In fact, the problem and solution here are somewhat
subtler. First, it doesn't have to happen on the system stack, it just
has to be non-stoppable (that is, non-preemptible). Second, this isn't
specific to gcDrainN (though gcDrainN is perhaps the most surprising
instance); it's general to anything that uses the gcWork structure.
Move the comment to gcWork and generalize it.
Change-Id: I5277b5abb070e47f8d783bc15a310b379c6adc22
Reviewed-on: https://go-review.googlesource.com/8247
Reviewed-by: Rick Hudson <rlh@golang.org>
gcDrain used to be passed a *workbuf to start draining from, but now
it takes a gcWork, which hides whether or not there's an initial
workbuf. Update the comment to match this.
Change-Id: I976b58e5bfebc451cfd4fa75e770113067b5cc07
Reviewed-on: https://go-review.googlesource.com/8246
Reviewed-by: Rick Hudson <rlh@golang.org>
Being able to printer pointers to strings means one will able to output
the result of things like the flag library and other components that use
string pointers.
While here, adjusted the tests for gdb to test original string pretty
printing as well as pointers to them. It was doing it via the map before
but for completeness this ensures it's tested as a unit.
Change-Id: I4926547ae4fa6c85ef74301e7d96d49ba4a7b0c6
Reviewed-on: https://go-review.googlesource.com/8217
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime·main·f is normalized by the linker to runtime.main.f, as is
the compiler-generated symbol runtime.main·f. Change the former to
runtime·mainPC instead.
Fixes issue #9934
Change-Id: I656a6fa6422d45385fa2cc55bd036c6affa1abfe
Reviewed-on: https://go-review.googlesource.com/8234
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Extend escape analysis to convT2E and conT2I. If the interface value
does not escape supply runtime with a stack buffer for the object copy.
This is a straight port from .c to .go of Dmitry's patch
Change-Id: Ic315dd50d144d94dd3324227099c116be5ca70b6
Reviewed-on: https://go-review.googlesource.com/8201
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently, various functions are marked with the comment
// May run without a P, so write barriers are not allowed.
However, "running without a P" is ambiguous. We intended these to mean
that m.p may be nil (which is the condition checked by the write
barrier). The comment could also be taken to mean that a
stop-the-world may happen, which is not the case for these functions
because they run in situations where there is in fact a function on
the stack holding a P locally, it just isn't in m.p.
Change these comments to state precisely what we mean, that m.p may be
nil.
Change-Id: I4a4a1d26aebd455e5067540e13b9f96a7482146c
Reviewed-on: https://go-review.googlesource.com/8209
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
When Windows Error Reporting dialog is disabled on amd64
Windows XP or 2003, the continue handler does not fire. Newer
versions work correctly regardless of WER.
Fixes#10162
Change-Id: I84ea36ee188b34d1421a8db6231223cf61b4111b
Reviewed-on: https://go-review.googlesource.com/8165
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Racy tests do not fail currently, they do os.Exit(0).
So if you run go test without -v, you won't even notice.
This was probably introduced with testing.TestMain.
Racy programs do not have the right to finish successfully.
Change-Id: Id133d7424f03d90d438bc3478528683dd02b8846
Reviewed-on: https://go-review.googlesource.com/4371
Reviewed-by: Russ Cox <rsc@golang.org>
Suggested by iant@, this change:
- looks for a symbol _rt0_<GOARCH>_<GOOS>_lib,
- if the symbol is present, adds a new entry into the .init_array ELF
section that points to the symbol.
The end-effect is that the symbol _rt0_<GOARCH>_<GOOS>_lib will be
invoked as soon as the (ELF) shared library is loaded, which will in turn
initialize the runtime. (To be implemented.)
Change-Id: I99911a180215a6df18f8a18483d12b9b497b48f4
Reviewed-on: https://go-review.googlesource.com/7692
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
1) Large allocation in this test caused crash. This was not
detected by builder because builder runs tests with -test.short.
2) The command "go" for forking doesn't exist in some platforms
including android. This change uses the test binary itself which
is guaranteed to exist.
This change also adds logging of the total samples collected in
TestCPUProfileMultithreaded test that is flaky in android-arm
builder.
Change-Id: I225c6b7877d811edef8b25e7eb00559450640c42
Reviewed-on: https://go-review.googlesource.com/8131
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
handoffp by definition runs without a P, so it's not allowed to have
write barriers. It doesn't have any right now, but mark it
nowritebarrier to disallow any creeping in in the future. handoffp in
turns calls startm, newm, and newosproc, all of which are "below Go"
and make sense to run without a P, so disallow write barriers in these
as well.
For most functions, we've done this because they may race with
stoptheworld() and hence must not have write barriers. For these
functions, it's a little different: the world can't stop while we're
in handoffp, so this race isn't present. But we implement this
restriction with a somewhat broader rule that you can't have a write
barrier without a P. We like this rule because it's simple and means
that our write barriers can depend on there being a P, even though
this rule is actually a little broader than necessary. Hence, even
though there's no danger of the race in these functions, we want to
adhere to the broader rule.
Change-Id: Ie22319c30eea37d703eb52f5c7ca5da872030b88
Reviewed-on: https://go-review.googlesource.com/8130
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This should fix the intermittent calling write barrier with mp.p == nil
failures on the nacl/386 builder.
Change-Id: I34aef5ca75ccd2939e6a6ad3f5dacec64903074e
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7973
Reviewed-by: Austin Clements <austin@google.com>
Currently, Darwin's siginfo type uses *byte for the si_addr
field. This results in unwanted write barriers in set_sigaddr. It's
also pointless since it never points to anything real and the get/set
methods return/take uintXX and cast it from/to the pointer.
All other arches use a uint type for this field. Change Darwin to
match. This simplifies the get/set methods and eliminates the unwanted
write barriers.
Change-Id: Ifdb5646d35e1f2f6808b87a3d59745ec9718add1
Reviewed-on: https://go-review.googlesource.com/8086
Reviewed-by: Austin Clements <austin@google.com>
sighandler may run during a stop-the-world without a P, so it's not
allowed to have write barriers. Fix the G write to disable the write
barrier (this is safe because the G is reachable from allgs) and mark
the function nowritebarrier.
Change-Id: I907f05d3829e24eeb15fa4d020598af36710e87e
Reviewed-on: https://go-review.googlesource.com/8020
Reviewed-by: Rick Hudson <rlh@golang.org>
Also invert it, which means it no longer needs to cross the cgo
package boundary.
Change-Id: I393cd073bda02b591a55d6bc6b8bb94970ea71cd
Reviewed-on: https://go-review.googlesource.com/8082
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We do not use SEH to handle Windows exception anymore.
Change-Id: I0ac807a0fed7a5b4c745454246764c524460472b
Reviewed-on: https://go-review.googlesource.com/8071
Reviewed-by: Minux Ma <minux@golang.org>
The SecureRandom named service was removed in
https://codereview.chromium.org/550523002. And the new syscall
was introduced in https://codereview.chromium.org/537543003.
Accepting this will remove the support for older version of
sel_ldr. I've confirmed that both pepper_40 and current
pepper_canary have this syscall.
After this change, we need sel_ldr from pepper_39 or above to
work.
Fixes#9261
Change-Id: I096973593aa302ade61f259a3a71ebc7c1a57913
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/1755
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Also fixes a long-existing problem in the fork/exec path.
Change-Id: Idec40b1cee0cfb1625fe107db3eafdc0d71798f2
Reviewed-on: https://go-review.googlesource.com/8030
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Previously the extra m needed for cgo callbacks was created on the
first callback. This works for cgo, however the cgocallback mechanism
is also borrowed by badsignal which can run before any cgo calls are
made.
Now we initialize the extra M at runtime startup before any signal
handlers are registered, so badsignal cannot be called until the
extra M is ready.
Updates #10207.
Change-Id: Iddda2c80db6dc52d8b60e2b269670fbaa704c7b3
Reviewed-on: https://go-review.googlesource.com/7978
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
There are calls to stdcall when the GC thinks the world is stopped
and stdcall write a *g for the CPU profiler. This produces a write
barrier but the GC is not prepared to deal with write barriers when
it thinks the world is stopped. Since the g is on allg it does not
need a write barrier to keep it alive so eliminate the write barrier.
Change-Id: I937633409a66553d7d292d87d7d58caba1fad0b6
Reviewed-on: https://go-review.googlesource.com/7979
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Rick Hudson <rlh@golang.org>
The test is a simple reproduction of issue 9356.
Update #8948.
Update #9356.
Change-Id: Ia77bc36d12ed0c3c4a8b1214cade8be181c9ad55
Reviewed-on: https://go-review.googlesource.com/7618
Reviewed-by: Minux Ma <minux@golang.org>
windows/386 also wants underscore prefix for external names.
This CL is in preparation of external linking support.
Change-Id: I2d2ea233f976aab3f356f9b508cdd246d5013e2d
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7282
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
When external linking, we must link to implib provided by mingw, so we must use
properly decorated names for stdcalls.
Because the feature is only used in the runtime, I've designed a new decoration
scheme so that we can use the same decorated name for both 386 and amd64.
A stdcall function named FooEx from bar16.dll which takes 3 parameters will be
imported like this:
//go:cgo_import_dynamic runtime._FooEx FooEx%3 "bar16.dll"
Depending on the size of uintptr, the linker will later transform it to _FooEx@12
or _FooEx@24.
This is in prepration for the next CL that adds external linking support for
windows/386.
Change-Id: I2d2ea233f976aab3f356f9b508cdd246d5013e2c
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7163
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
On Unix, when placing a child in a new process group, allow that group
to become the foreground process group. Also, allow a child process to
join a specific process group.
When setting the foreground process group, Ctty is used as the file
descriptor of the controlling terminal. Ctty has been added to the BSD
and Solaris SysProcAttr structures and the handling of Setctty changed
to match Linux.
Change-Id: I18d169a6c5ab8a6a90708c4ff52eb4aded50bc8c
Reviewed-on: https://go-review.googlesource.com/5130
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Implement runtime.atomicand8 for amd64p32 which was overlooked
in CL 7861.
Change-Id: Ic7eccddc6fd6c4682cac1761294893928f5428a2
Reviewed-on: https://go-review.googlesource.com/7920
Reviewed-by: Minux Ma <minux@golang.org>
These can be implemented with just a compare and a move instruction.
Do so, avoiding the overhead of a call into the runtime.
These assertions are a significant cost in Go code that uses interface{}
as a safe alternative to C's void* (or unsafe.Pointer), such as the
current version of the Go compiler.
*T here includes pointer to T but also any Go type represented as
a single pointer (chan, func, map). It does not include [1]*T or struct{*int}.
That requires more work in other parts of the compiler; there is a TODO.
Change-Id: I7ff681c20d2c3eb6ad11dd7b3a37b1f3dda23965
Reviewed-on: https://go-review.googlesource.com/7862
Reviewed-by: Rob Pike <r@golang.org>
Currently, we only exit the getfull barrier if there is work on the
full list, even though the exit path will take work from either the
full or partial list. Change this to exit the barrier if there is work
on either the full or partial lists.
I believe it's currently safe to check only the full list, since
during mark termination there is no reason to put a workbuf on a
partial list. However, checking both is more robust.
Change-Id: Icf095b0945c7cad326a87ff2f1dc49b7699df373
Reviewed-on: https://go-review.googlesource.com/7840
Reviewed-by: Rick Hudson <rlh@golang.org>
The barrier in gcDrain does not account for concurrent gcDrainNs
happening in gchelpwork, so it can actually return while there is
still work being done. It turns out this is okay, but for subtle
reasons involving gcDrainN always being run on the system
stack. Document these reasons.
Change-Id: Ib07b3753cc4e2b54533ab3081a359cbd1c3c08fb
Reviewed-on: https://go-review.googlesource.com/7736
Reviewed-by: Rick Hudson <rlh@golang.org>
Make mask uint32, and move down one line to match atomic_arm64.go.
Change-Id: I4867de494bc4076b7c2b3bf4fd74aa984e3ea0c8
Reviewed-on: https://go-review.googlesource.com/7854
Reviewed-by: Russ Cox <rsc@golang.org>
We're skating on thin ice, and things are finally starting to melt around here.
(I want to avoid the debugging session that will happen when someone
uses atomicand8 expecting it to be atomic with respect to other operations.)
Change-Id: I254f1582be4eb1f2d7fbba05335a91c6bf0c7f02
Reviewed-on: https://go-review.googlesource.com/7861
Reviewed-by: Minux Ma <minux@golang.org>
To reduce lock contention in this mode, makes persistent allocation state per-P,
which means at most 64 kB overhead x $GOMAXPROCS, which should be
completely tolerable.
Change-Id: I34ca95e77d7e67130e30822e5a4aff6772b1a1c5
Reviewed-on: https://go-review.googlesource.com/7740
Reviewed-by: Rick Hudson <rlh@golang.org>
Some type assertions of the form _, ok := i.(T) allow efficient inlining.
Such type assertions commonly show up in type switches.
For example, with this optimization, using 6g, the length of
encoding/binary's intDataSize function shrinks from 2224 to 1728 bytes (-22%).
benchmark old ns/op new ns/op delta
BenchmarkAssertI2E2Blank 4.67 0.82 -82.44%
BenchmarkAssertE2T2Blank 4.38 0.83 -81.05%
BenchmarkAssertE2E2Blank 3.88 0.83 -78.61%
BenchmarkAssertE2E2 14.2 14.4 +1.41%
BenchmarkAssertE2T2 10.3 10.4 +0.97%
BenchmarkAssertI2E2 13.4 13.3 -0.75%
Change-Id: Ie9798c3e85432bb8e0f2c723afc376e233639df7
Reviewed-on: https://go-review.googlesource.com/7697
Reviewed-by: Keith Randall <khr@golang.org>
The distinction between gcWorkProducer and gcWork (producer and
consumer) is not serving us as originally intended, so merge these
into just gcWork.
The original intent was to replace the currentwbuf cache with a
gcWorkProducer. However, with gchelpwork (aka mutator assists),
mutators can both produce and consume work, so it will make more sense
to cache a whole gcWork.
Change-Id: I6e633e96db7cb23a64fbadbfc4607e3ad32bcfb3
Reviewed-on: https://go-review.googlesource.com/7733
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently markroot fetches the wbuf to fill from the per-M wbuf
cache. The wbuf cache is primarily meant for the write barrier because
it produces very little work on each call. There's little point to
using the cache in mark root, since each call to markroot is likely to
produce a large amount of work (so the slight win on getting it from
the cache instead of from the central wbuf lists doesn't matter), and
markroot does not dispose the wbuf back to the cache (so most markroot
calls won't get anything from the wbuf cache anyway).
Instead, just get the wbuf from the central wbuf lists like other work
producers. This will simplify later changes.
Change-Id: I07a18a4335a41e266a6d70aa3a0911a40babce23
Reviewed-on: https://go-review.googlesource.com/7732
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, the GC's concurrent mark phase runs on the system
stack. There's no need to do this, and running it this way ties up the
entire M and P running the GC by preventing the scheduler from
preempting the GC even during concurrent mark.
Fix this by running concurrent mark on the regular G stack. It's still
non-preemptible because we also set preemptoff around the whole GC
process, but this moves us closer to making it preemptible.
Change-Id: Ia9f1245e299b8c5c513a4b1e3ef13eaa35ac5e73
Reviewed-on: https://go-review.googlesource.com/7730
Reviewed-by: Rick Hudson <rlh@golang.org>
"Sync" is not very informative. What's being synchronized and with
whom? Update this comment to explain what we're really doing: enabling
write barriers.
Change-Id: I4f0cbb8771988c7ba4606d566b77c26c64165f0f
Reviewed-on: https://go-review.googlesource.com/7700
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently we harvestwbufs the moment we enter the mark phase, even
before starting the world again. Since cached wbufs are only filled
when we're in mark or mark termination, they should all be empty at
this point, making the harvest pointless. Remove the harvest.
We should, but do not currently harvest at the end of the mark phase
when we're running out of work to do.
Change-Id: I5f4ba874f14dd915b8dfbc4ee5bb526eecc2c0b4
Reviewed-on: https://go-review.googlesource.com/7669
Reviewed-by: Rick Hudson <rlh@golang.org>
One of my earlier versions of finer-grained select locking
failed on this test. If you just naively lock and check channels
one-by-one, it is possible that you skip over ready channels.
Consider that initially c1 is ready and c2 is not. Select checks c2.
Then another goroutine makes c1 not ready and c2 ready (in that order).
Then select checks c1, concludes that no channels are ready and
executes the default case. But there was no point in time when
no channel is ready and so default case must not be executed.
Change-Id: I3594bf1f36cfb120be65e2474794f0562aebcbbd
Reviewed-on: https://go-review.googlesource.com/7550
Reviewed-by: Russ Cox <rsc@golang.org>
The value in question is really a bit pattern
(a pointer with extra bits thrown in),
so treat it as a uintptr instead, avoiding the
generation of a write barrier when there
might not be a p.
Also add the obligatory //go:nowritebarrier.
Change-Id: I4ea097945dd7093a140f4740bcadca3ce7191971
Reviewed-on: https://go-review.googlesource.com/7667
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The GC assumes that there will be no asynchronous write barriers when
the world is stopped. This keeps the synchronization between write
barriers and the GC simple. However, currently, there are a few places
in runtime code where this assumption does not hold.
The GC stops the world by collecting all Ps, which stops all user Go
code, but small parts of the runtime can run without a P. For example,
the code that releases a P must still deschedule its G onto a runnable
queue before stopping. Similarly, when a G returns from a long-running
syscall, it must run code to reacquire a P.
Currently, this code can contain write barriers. This can lead to the
GC collecting reachable objects if something like the following
sequence of events happens:
1. GC stops the world by collecting all Ps.
2. G #1 returns from a syscall (for example), tries to install a
pointer to object X, and calls greyobject on X.
3. greyobject on G #1 marks X, but does not yet add it to a write
buffer. At this point, X is effectively black, not grey, even though
it may point to white objects.
4. GC reaches X through some other path and calls greyobject on X, but
greyobject does nothing because X is already marked.
5. GC completes.
6. greyobject on G #1 adds X to a work buffer, but it's too late.
7. Objects that were reachable only through X are incorrectly collected.
To fix this, we check the invariant that no asynchronous write
barriers happen when the world is stopped by checking that write
barriers always have a P, and modify all currently known sources of
these writes to disable the write barrier. In all modified cases this
is safe because the object in question will always be reachable via
some other path.
Some of the trace code was turned off, in particular the
code that traces returning from a syscall. The GC assumes
that as far as the heap is concerned the thread is stopped
when it is in a syscall. Upon returning the trace code
must not do any heap writes for the same reasons discussed
above.
Fixes#10098Fixes#9953Fixes#9951Fixes#9884
May relate to #9610#9771
Change-Id: Ic2e70b7caffa053e56156838eb8d89503e3c0c8a
Reviewed-on: https://go-review.googlesource.com/7504
Reviewed-by: Austin Clements <austin@google.com>
Some versions of libc, in this case Android's bionic, point environ
directly at the envp memory.
https://android.googlesource.com/platform/bionic/+/master/libc/bionic/libc_init_common.cpp#104
The Go runtime does something surprisingly similar, building the
runtime's envs []string using gostringnocopy. Both libc and the Go
runtime reusing memory interacts badly. When syscall.Setenv uses cgo
to call setenv(3), C modifies the underlying memory of a Go string.
This manifests on android/arm. With GOROOT=/data/local/tmp, a
runtime test calls syscall.Setenv("/os"), resulting in
runtime.GOROOT()=="/os\x00a/local/tmp/goroot".
Avoid this by copying environment string memory into Go.
Covered by runtime.TestFixedGOROOT on android/arm.
Change-Id: Id0cf9553969f587addd462f2239dafca1cf371fa
Reviewed-on: https://go-review.googlesource.com/7663
Reviewed-by: Keith Randall <khr@golang.org>
Channels and sync.Mutex'es allow another goroutine to acquire resource
ahead of an unblocked goroutine. This is good for performance, but
leads to futile wakeups (the unblocked goroutine needs to block again).
Futile wakeups caused user confusion during the very first evaluation
of tracing functionality on a real server (a goroutine as if acquires a mutex
in a loop, while there is no loop in user code).
This change detects futile wakeups on channels and emits a special event
to denote the fact. Later parser finds entire wakeup sequences
(unblock->start->block) and removes them.
sync.Mutex will be supported in a separate change.
Change-Id: Iaaaee9d5c0921afc62b449a97447445030ac19d3
Reviewed-on: https://go-review.googlesource.com/7380
Reviewed-by: Keith Randall <khr@golang.org>
The Go builders (and standard development cycle) for programs on iOS
require running the programs under lldb. Unfortunately lldb intercepts
SIGSEGV and will not give it back.
https://llvm.org/bugs/show_bug.cgi?id=22868
We get around this by never letting lldb see the SIGSEGV. On darwin,
Unix signals are emulated on top of mach exceptions. The debugger
registers a task-level mach exception handler. We register a
thread-level exception handler which acts as a faux signal handler.
The thread-level handler gets precedence over the task-level handler,
so we can turn the exception EXC_BAD_ACCESS into a panic before lldb
can see it.
Fixes#10043
Change-Id: I64d7c310dfa7ecf60eb1e59f094966520d473335
Reviewed-on: https://go-review.googlesource.com/7072
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
When checkmark fails, greyobject dumps both the object that pointed to
the unmarked object and the unmarked object. This code cluttered up
greyobject, was copy-pasted for the two objects, and the copy for
dumping the unmarked object was not entirely correct.
Extract object dumping out to a new function. This declutters
greyobject and fixes the bugs in dumping the unmarked object. The new
function is slightly cleaned up from the original code to have more
natural control flow and shows a marker on the field in the base
object that points to the unmarked object to make it easy to find.
Change-Id: Ib51318a943f50b0b99995f0941d03ee8876b9fcf
Reviewed-on: https://go-review.googlesource.com/7506
Reviewed-by: Rick Hudson <rlh@golang.org>
scanobject no longer returns the new wbuf.
Change-Id: I0da335ae5cd7ef7ea0e0fa965cf0e9f3a650d0e6
Reviewed-on: https://go-review.googlesource.com/7505
Reviewed-by: Rick Hudson <rlh@golang.org>
DragonFlyBSD dropped support for i386 in 4.0 and there is no longer a
dragonfly/386 - as such, remove the Go port.
Fixes#8951Fixes#7580Fixes#7421
Change-Id: I69022ab2262132e8f97153f14dc8c37c98527008
Reviewed-on: https://go-review.googlesource.com/7543
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Joel Sing <jsing@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The kern.rthreads sysctl has not existed for a long time - there is no way to
disable rthreads and __tfork no longer returns ENOTSUP.
Change-Id: Ia50ff01ac86ea83358e72b8f45f7818aaec1e4b1
Reviewed-on: https://go-review.googlesource.com/7490
Reviewed-by: Minux Ma <minux@golang.org>
Fixes#10135.
Change-Id: Ic4c5ab15bcb7b9c3fcc685a788d3b59c60c26e1e
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7400
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Everything has moved to Go, but comments still refer to .c/.h files.
Fix all of those up, at least for these three directories.
Fixes#10138
Change-Id: Ie5efe89b247841e0b3f82aac5256b2c606ef67dc
Reviewed-on: https://go-review.googlesource.com/7431
Reviewed-by: Russ Cox <rsc@golang.org>
This allows to test goroutine analysis code in runtime/pprof tests.
Also fix a nil-deref crash in goroutine analysis code that happens on runtime/pprof tests.
Change-Id: Id7884aa29f7fe4a8d7042482a86fe434e030461e
Reviewed-on: https://go-review.googlesource.com/7301
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Augment ProcStart events with OS thread id.
This helps in scheduler locality analysis.
Change-Id: I93fea75d3072cf68de66110d0b59d07101badcb5
Reviewed-on: https://go-review.googlesource.com/7302
Reviewed-by: Keith Randall <khr@golang.org>
Some of the trace stacks are OS-dependent due to OS-specific code
in net package. Check these stacks only on subset of OSes.
Change-Id: If95e4485839f4120fd6395725374c3a2f8706dfc
Reviewed-on: https://go-review.googlesource.com/7300
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Even though the world is stopped the GC may do pointer
writes that need to be protected by write barriers.
This means that the write barrier must be on
continuously from the time the mark phase starts and
the mark termination phase ends. Checks were added to
ensure that no allocation happens during a GC.
Hoist the logic that clears pools the start of the GC
so that the memory can be reclaimed during this GC cycle.
Change-Id: I9d1551ac5db9bac7bac0cb5370d5b2b19a9e6a52
Reviewed-on: https://go-review.googlesource.com/6990
Reviewed-by: Austin Clements <austin@google.com>
Stip uninteresting bottom and top frames from trace stacks.
This makes both binary and json trace files smaller,
and also makes stacks shorter and more readable in the viewer.
Change-Id: Ib9c80ccc280504f0e235f867f53f1d2652c41583
Reviewed-on: https://go-review.googlesource.com/5523
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Removes a potential data race between os.Setenv and runtime.GOROOT,
along with a bug where os.Setenv would only sometimes change the
value of runtime.GOROOT.
Change-Id: I7d2a905115c667ea6e73f349f3784a1d3e8f810d
Reviewed-on: https://go-review.googlesource.com/6611
Reviewed-by: Keith Randall <khr@golang.org>
Also fixed a stack corruption bug for nacl/amd64p32.
Change-Id: I64b821b16999c296a159137d971af3870053c621
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7073
Reviewed-by: Dave Cheney <dave@cheney.net>
Starting it lazily causes a memory allocation (for the goroutine) during GC.
First use of channels for runtime implementation.
Change-Id: I9cd24dcadbbf0ee5070ee6d0ed7ea415504f316c
Reviewed-on: https://go-review.googlesource.com/6960
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
I asked for this in CL 3742 and it was ignored.
Change-Id: I30ad05f87c7d9eccb11df7e19288e3ed2c7e2e3f
Reviewed-on: https://go-review.googlesource.com/6930
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
For OSes that use elf on intel, 2*Ptrsize bytes are reserved for TLS.
But only one pointer (g) has been stored in the TLS for a while now.
So we can set it to just Ptrsize, which happily matches what happens
when externally linking.
Fixes#9913
Change-Id: Ic816369d3a55a8cdcc23be349b1a1791d53f5f81
Reviewed-on: https://go-review.googlesource.com/6584
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is an experiment to see if removing the boundary bit logic will
lead to fewer cache misses and improved performance. Instead of using
boundary bits we use the span information to get element size and use
some bit whacking to get the boundary without having to touch the
random heap bits which cause cache misses.
Furthermore once the boundary bit is removed we can either use that
bit for a simpler checkmark routine or we can reduce the number of
bits in the GC bitmap to 2 bits per pointer sized work. For example
the 2 bits at the boundary can be used for marking and pointer/scalar
differentiation. Since we don't need the mark bit except at the
boundary nibble of the object other nibbles can use this bit
as a noscan bit to indicate that there are no more pointers in
the object.
Currently the changed included in this CL slows down the garbage
benchmark. With the boundary bits garbage gives 5.78 and without
(this CL) it gives 5.88 which is a 2% slowdown.
Change-Id: Id68f831ad668176f7dc9f7b57b339e4ebb6dc4c2
Reviewed-on: https://go-review.googlesource.com/6665
Reviewed-by: Austin Clements <austin@google.com>
Gc already calculates n as an int, so converting to int64 to call
growslice doesn't serve any purpose except to emit slightly larger
code on 32-bit platforms. Passing n as an int shrinks godoc's text
segment by 8kB (9472633 => 9464133) when building for ARM.
Change-Id: Ief9492c21d01afcb624d3f2a484df741450b788d
Reviewed-on: https://go-review.googlesource.com/6231
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The unbounded list-based defer pool can grow infinitely.
This can happen if a goroutine routinely allocates a defer;
then blocks on one P; and then unblocked, scheduled and
frees the defer on another P.
The scenario was reported on golang-nuts list.
We've been here several times. Any unbounded local caches
are bad and grow to infinite size. This change introduces
central defer pool; local pools become fixed-size
with the only purpose of amortizing accesses to the
central pool.
Freedefer now executes on system stack to not consume
nosplit stack space.
Change-Id: I1a27695838409259d1586a0adfa9f92bccf7ceba
Reviewed-on: https://go-review.googlesource.com/3967
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
The unbounded list-based sudog cache can grow infinitely.
This can happen if a goroutine is routinely blocked on one P
and then unblocked and scheduled on another P.
The scenario was reported on golang-nuts list.
We've been here several times. Any unbounded local caches
are bad and grow to infinite size. This change introduces
central sudog cache; local caches become fixed-size
with the only purpose of amortizing accesses to the
central cache.
The change required to move sudog cache from mcache to P,
because mcache is not scanned by GC.
Change-Id: I3bb7b14710354c026dcba28b3d3c8936a8db4e90
Reviewed-on: https://go-review.googlesource.com/3742
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
Error detection code copied from syscall, where presumably
we actually do it right.
Note that we throw the errno away. The runtime doesn't use it.
Fixes#10052
Change-Id: I8de77dda6bf287276b137646c26b84fa61554ec8
Reviewed-on: https://go-review.googlesource.com/6571
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
OpenBSD's sigprocmask system call passes the signal mask by value
rather than reference, so vars are unnecessary. Additionally,
declaring "var sigset_all = ^sigset_none" means sigset_all won't be
initialized until runtime_init is called, but the first call to
newosproc happens before then.
I've witnessed Go processes on OpenBSD crash from receiving SIGWINCH
on the newly created OS thread before it finished initializing.
Change-Id: I16995e7e466d5e7e50bcaa7d9490173789a0b4cc
Reviewed-on: https://go-review.googlesource.com/6440
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Move type definitions from chan1.go to chan.go and select.go.
Remove underscores from names.
Make c.buf unsafe.Pointer instead of *uint8.
Change-Id: I75cf8385bdb9f79eb5a7f7ad319495abbacbe942
Reviewed-on: https://go-review.googlesource.com/4900
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
This fixes runtime's TestBreakpoint on ppc64:
the Breakpoint frame was not showing up in the trace.
It seems like f.frame should be either the frame size
including the saved LR (if any) or the frame size
not including the saved LR.
On ppc64, f.frame is the frame size not including the saved LR.
On arm, f.frame is the frame size not including the saved LR,
except when that's -4, f.frame is 0 instead.
The code here in the runtime expects that f.frame is the frame
size including the saved LR.
Since all three disagree and nothing else uses f.frame anymore,
stop using it here too. Use funcspdelta, which tells us the exact
difference between the FP and SP. If it's zero, LR has not been
saved yet, so the one saved for sigpanic should be recorded.
This fixes TestBreakpoint on both ppc64 and ppc64le.
I don't really understand how it ever worked there.
Change-Id: I2d2c580d5c0252cc8471e828980aeedcab76858d
Reviewed-on: https://go-review.googlesource.com/6430
Reviewed-by: Minux Ma <minux@golang.org>
Plan 9 provides a /dev/random device to return a
stream of random numbers. However, the method used
to generate random numbers on Plan 9 is slow and
reading from /dev/random may block.
We don't want our Go programs to be significantly
slowed down just to slightly improve the distribution
of hash values.
So, we do the same thing as NaCl and rely exclusively
on extendRandom to generate pseudo-random numbers.
Fixes#10028.
Change-Id: I7e11a9b109c22f23608eb09c406b7c3dba31f26a
Reviewed-on: https://go-review.googlesource.com/6386
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
issue #10017: TestGdbPython 'print mapvar' is reported to fail on ppc64.
issue #10002: TestGdbPython 'print mapvar' is reported to fail on arm hardfloat.
The testcase now uses plain line number in main. Unwinding issues are
unrelated to the GDB map prettyprinter feature.
Remove arch-specific t.Skip()s from those two issues.
Fixes#10017Fixes#10002
Change-Id: I9d50ffe2f3eb7bf65dd17c8c76a2677571de68ba
Reviewed-on: https://go-review.googlesource.com/6267
Reviewed-by: Minux Ma <minux@golang.org>
mv cmd/new5l cmd/5l and so on.
Minimal changes to cmd/dist and cmd/go to keep things building.
More can be deleted in followup CLs.
Change-Id: I1449eca7654ce2580d1f413a56dc4a75f3d4618b
Reviewed-on: https://go-review.googlesource.com/6361
Reviewed-by: Rob Pike <r@golang.org>
We used to not call traceback from goexit1.
But now tracer does it and crashes on amd64p32:
runtime: unexpected return pc for runtime.getg called from 0x108a4240
goroutine 18 [runnable, locked to thread]:
runtime.traceGoEnd()
src/runtime/trace.go:758 fp=0x10818fe0 sp=0x10818fdc
runtime.goexit1()
src/runtime/proc1.go:1540 +0x20 fp=0x10818fe8 sp=0x10818fe0
runtime.getg(0x0)
src/runtime/asm_386.s:2414 fp=0x10818fec sp=0x10818fe8
created by runtime/pprof_test.TestTraceStress
src/runtime/pprof/trace_test.go:123 +0x500
Return PC from goexit1 points right after goexit (+0x6).
It happens to work most of the time somehow.
This change fixes traceback from goexit1 by adding an additional NOP to goexit.
Fixes#9931
Change-Id: Ied25240a181b0a2d7bc98127b3ed9068e9a1a13e
Reviewed-on: https://go-review.googlesource.com/5460
Reviewed-by: Russ Cox <rsc@golang.org>
This is to be used by an lldb script inside go_darwin_arm_exec to pause
the execution of tests on iOS so the working directory can be adjusted
into something resembling a GOROOT.
Change-Id: I69ea2d4d871800ae56634b23ffa48583559ddbc6
Reviewed-on: https://go-review.googlesource.com/6363
Reviewed-by: Minux Ma <minux@golang.org>
Change-Id: I9b08b74214e5a41a7e98866a993b038030a4c073
Reviewed-on: https://go-review.googlesource.com/6251
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Previously, the typeDead check in greyobject was under a separate
!useCheckmark conditional. Put it with the rest of the !useCheckmark
code. Also move a comment about atomic update of the marked bit to
where we actually do that update now.
Change-Id: Ief5f16401a25739ad57d959607b8d81ffe0bc211
Reviewed-on: https://go-review.googlesource.com/6271
Reviewed-by: Rick Hudson <rlh@golang.org>
Change-Id: I1bb0b8b11e8c7686b85657050fd7cf926afe4d29
Reviewed-on: https://go-review.googlesource.com/6200
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Previously, the memory allocator on Plan 9 did
not free memory properly. It was only able to
free the last allocated block.
This change implements a variant of the
Kernighan & Ritchie memory allocator with
coalescing and splitting.
The most notable differences are:
- no header is prefixing the allocated blocks, since
the size is always specified when calling sysFree,
- the free list is nil-terminated instead of circular.
Fixes#9736.
Fixes#9803.
Fixes#9952.
Change-Id: I00d533714e4144a0012f69820d31cbb0253031a3
Reviewed-on: https://go-review.googlesource.com/5524
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Disable the test properly on nacl systems, tested on nacl/amd64p32.
Change-Id: Iffe210be4f9c426bfc47f2dd3a8f0c6b5a398cc3
Reviewed-on: https://go-review.googlesource.com/6093
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Update #9993
If the physical page size of the machine is larger than the logical
heap size, for example 8k logical, 64k physical, then madvise(2) will
round up the requested amount to a 64k boundary and may discard pages
close to the page being madvised.
This patch disables the scavenger in these situations, which at the moment
is only ppc64 and ppc64le systems. NaCl also uses a 64k page size, but
it's not clear if it is affected by this problem.
Change-Id: Ib897f8d3df5bd915ddc0b510f2fd90a30ef329ca
Reviewed-on: https://go-review.googlesource.com/6091
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Needs the Go tool, which we do not have on iOS. (No Fork.)
Change-Id: Iedf69f5ca81d66515647746546c9b304c8ec10c4
Reviewed-on: https://go-review.googlesource.com/6102
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
There is no sense in trying to netpoll while there is
already a thread blocked in netpoll. And in most cases
there must be a thread blocked in netpoll, because
the first otherwise idle thread does blocking netpoll.
On some program I see that netpoll called from findrunnable
consumes 3% of time.
Change-Id: I0af1a73d637bffd9770ea50cb9278839716e8816
Reviewed-on: https://go-review.googlesource.com/4553
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
This makes Go's CPU profiling code somewhat more idiomatic; e.g.,
using := instead of forward declaring variables, using "int" for
element counts instead of "uintptr", and slices instead of C-style
pointer+length. This makes the code easier to read and eliminates a
lot of type conversion clutter.
Additionally, in sigprof we can collect just maxCPUProfStack stack
frames, as cpuprof won't use more than that anyway.
Change-Id: I0235b5ae552191bcbb453b14add6d8c01381bd06
Reviewed-on: https://go-review.googlesource.com/6072
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
The first call is pointless. It appears to simply be a mistake.
benchmark old ns/op new ns/op delta
BenchmarkComplexAlgMap 90.7 76.1 -16.10%
Change-Id: Id0194c9f09cea8b68f17b2ac751a8e3240e47f19
Reviewed-on: https://go-review.googlesource.com/5284
Reviewed-by: Keith Randall <khr@golang.org>
Gives tests a way to find the bundle that contains their testdata, and
is generally useful for finding resources.
Change-Id: Idfa03e8543af927c17bc8ec8aadc5014ec82df28
Reviewed-on: https://go-review.googlesource.com/6000
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Updates #10002
The gdb test added in 1c82e236f5 is failing on most arm systems.
Temporarily disable this test so that we can return to a working arm build.
Change-Id: Iff96ea8d5a99e1ceacf4979e864ff196e5503535
Reviewed-on: https://go-review.googlesource.com/5902
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We return memory to the kernel with madvise(..., DONTNEED).
Also mark returned memory with NOHUGEPAGE to keep the kernel from
merging this memory into a huge page, effectively reallocating it.
Only known to be a problem on linux/{386,amd64,amd64p32} at the moment.
It may come up on other os/arch combinations in the future.
Fixes#8832
Change-Id: Ifffc6627a0296926e3f189a8a9b6e4bdb54c79eb
Reviewed-on: https://go-review.googlesource.com/5660
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
We need to distinguish pointers to free spans, which indicate bugs in
our pointer analysis, from pointers to never-in-the-heap spans, which
can legitimately arise from sysAlloc/mmap/etc. This normally isn't a
problem because the heap is contiguous, but in some situations (32
bit, particularly) the heap must grow around an already allocated
region.
The bad pointer test is disabled so this fix doesn't actually do
anything, but it removes one barrier from reenabling it.
Fixes#9872.
Change-Id: I0a92db4d43b642c58d2b40af69c906a8d9777f88
Reviewed-on: https://go-review.googlesource.com/5780
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Available darwin/arm devices sporadically have trouble mapping 256M.
I would really appreciate it if anyone could check my working on
this, and make sure sure there aren't obviously bad consequences I
haven't considered.
Change-Id: Id1a8edae104d974fcf5f9333274f958625467f79
Reviewed-on: https://go-review.googlesource.com/5752
Reviewed-by: Keith Randall <khr@golang.org>
Since allglock is held in this function, there's no point to
tip-toeing around allgs. Just use a for-range loop.
Change-Id: I1ee61c7e8cac8b8ebc8107c0c22f739db5db9840
Reviewed-on: https://go-review.googlesource.com/5882
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Previously, we had three loops in the garbage collector that all
cleared the per-G GC flags. Consolidate these into one function.
This one function is designed to work in a concurrent setting. As a
result, it's slightly more expensive than the loops it replaces during
STW phases, but these happen at most twice per GC.
Change-Id: Id1ec0074fd58865eb0112b8a0547b267802d0df1
Reviewed-on: https://go-review.googlesource.com/5881
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
The loop in gcMark is redundant with the gcworkdone resetting
performed by markroot, which called a few lines later in gcMark.
Change-Id: Ie0a826a614ecfa79e6e6b866e8d1de40ba515856
Reviewed-on: https://go-review.googlesource.com/5880
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Package runtime's Go code was converted to directly call getcallerpc
and getcallersp in https://golang.org/cl/138740043, but the assembly
implementations were not removed.
Change-Id: Ib2eaee674d594cbbe799925aae648af782a01c83
Reviewed-on: https://go-review.googlesource.com/5901
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
NetBSD's semaphore implementation is derived from OpenBSD's, but has
subsequently diverged due to cleanups that were only applied to the
latter (https://golang.org/cl/137960043, https://golang.org/cl/5563).
This CL applies analogous cleanups for NetBSD.
Notably, we can also remove the scary NetBSD deadlock warning.
NetBSD's manual pages document that lwp_unpark on a not-yet-parked LWP
will cause that LWP's next lwp_park system call to return immediately,
so there's no race hazard.
Change-Id: Ib06844c420d2496ac289748eba13eb4700bbbbb2
Reviewed-on: https://go-review.googlesource.com/5564
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Joel Sing <jsing@google.com>
(gdb) p x
Python Exception <class 'gdb.error'> There is no member named b.:
$2 = map[string]string
->
(gdb) p x
$1 = map[string]string = {["shane"] = "hansen"}
Change-Id: I874d02a029f2ac9afc5ab666afb65760ec2c3177
Reviewed-on: https://go-review.googlesource.com/5522
Reviewed-by: Ian Lance Taylor <iant@golang.org>
OpenBSD's thrsleep system call includes an "abort" parameter, which
specifies a memory address to be tested after being registered on the
sleep channel (i.e., capable of being woken up by thrwakeup). By
passing a pointer to waitsemacount for this parameter, we avoid race
conditions without needing a lock. Instead we just need to use
atomicload, cas, and xadd to mutate the semaphore count.
Change-Id: If9f2ab7cfd682da217f9912783cadea7e72283a8
Reviewed-on: https://go-review.googlesource.com/5563
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Joel Sing <jsing@google.com>
When GODEBUG=gctrace=2 two gcs are preformed. During the first gc
the stack scan sets the g's gcscanvalid and gcworkdone flags to true
indicating that the stacks have to be scanned and do not need to
be rescanned. These need to be reset to false for the second GC so the
stacks are rescanned, otherwise if the only pointer to an object is
on the stack it will not be discovered and the object will be freed.
Typically this will include the object that was just allocated in
the mallocgc call that initiated the GC.
Change-Id: Ic25163f4689905fd810c90abfca777324005c02f
Reviewed-on: https://go-review.googlesource.com/5861
Reviewed-by: Russ Cox <rsc@golang.org>
Currently sync.Mutex is fully cooperative. That is, once contention is discovered,
the goroutine calls into scheduler. This is suboptimal as the resource can become
free soon after (especially if critical sections are short). Server software
usually runs at ~~50% CPU utilization, that is, switching to other goroutines
is not necessary profitable.
This change adds limited active spinning to sync.Mutex if:
1. running on a multicore machine and
2. GOMAXPROCS>1 and
3. there is at least one other running P and
4. local runq is empty.
As opposed to runtime mutex we don't do passive spinning,
because there can be work on global runq on on other Ps.
benchmark old ns/op new ns/op delta
BenchmarkMutexNoSpin 1271 1272 +0.08%
BenchmarkMutexNoSpin-2 702 683 -2.71%
BenchmarkMutexNoSpin-4 377 372 -1.33%
BenchmarkMutexNoSpin-8 197 190 -3.55%
BenchmarkMutexNoSpin-16 131 122 -6.87%
BenchmarkMutexNoSpin-32 170 164 -3.53%
BenchmarkMutexSpin 4724 4728 +0.08%
BenchmarkMutexSpin-2 2501 2491 -0.40%
BenchmarkMutexSpin-4 1330 1325 -0.38%
BenchmarkMutexSpin-8 684 684 +0.00%
BenchmarkMutexSpin-16 414 372 -10.14%
BenchmarkMutexSpin-32 559 469 -16.10%
BenchmarkMutex 19.1 19.1 +0.00%
BenchmarkMutex-2 81.6 54.3 -33.46%
BenchmarkMutex-4 143 100 -30.07%
BenchmarkMutex-8 154 156 +1.30%
BenchmarkMutex-16 140 159 +13.57%
BenchmarkMutex-32 141 163 +15.60%
BenchmarkMutexSlack 33.3 31.2 -6.31%
BenchmarkMutexSlack-2 122 97.7 -19.92%
BenchmarkMutexSlack-4 168 158 -5.95%
BenchmarkMutexSlack-8 152 158 +3.95%
BenchmarkMutexSlack-16 140 159 +13.57%
BenchmarkMutexSlack-32 146 162 +10.96%
BenchmarkMutexWork 154 154 +0.00%
BenchmarkMutexWork-2 89.2 89.9 +0.78%
BenchmarkMutexWork-4 139 86.1 -38.06%
BenchmarkMutexWork-8 177 162 -8.47%
BenchmarkMutexWork-16 170 173 +1.76%
BenchmarkMutexWork-32 176 176 +0.00%
BenchmarkMutexWorkSlack 160 160 +0.00%
BenchmarkMutexWorkSlack-2 103 99.1 -3.79%
BenchmarkMutexWorkSlack-4 155 148 -4.52%
BenchmarkMutexWorkSlack-8 176 170 -3.41%
BenchmarkMutexWorkSlack-16 170 173 +1.76%
BenchmarkMutexWorkSlack-32 175 176 +0.57%
"No work" benchmarks are not very interesting (BenchmarkMutex and
BenchmarkMutexSlack), as they are absolutely not realistic.
Fixes#8889
Change-Id: I6f14f42af1fa48f73a776fdd11f0af6dd2bb428b
Reviewed-on: https://go-review.googlesource.com/5430
Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
This change deletes the C implementations of
the Go compiler and assembler from the master branch.
The Go implementations are a bit slower right now,
due mainly to garbage generated by taking addresses
of stack variables all over the place (it was C code,
after all). That will be cleaned up (mechanically) over the
next week or so, and things will get faster.
Change-Id: I66b2b3477aec8835f9960d0798f5752dcd98d08f
The slow path of heapBitsForObjects somewhat subtly assumes that the
pointer will not point to the first word of the object and will round
the pointer wrong if this assumption is violated. This assumption is
safe because the fast path should always take care of this case, but
there's no benefit to making this assumption, it makes the code more
difficult to experiment with than necessary, and it's trivial to
eliminate.
Change-Id: Iedd336f7d529a27d3abeb83e77dfb32a285ea73a
Reviewed-on: https://go-review.googlesource.com/5636
Reviewed-by: Russ Cox <rsc@golang.org>
The routine mallocgc retrieves objects from freelists. Prefetch
the object that will be returned in the next call to mallocgc.
Experiments indicate that this produces a 1% improvement when using
prefetchnta and less when using prefetcht0, prefetcht1, or prefetcht2.
Benchmark numbers indicate a 1% improvement over no
prefetch, much less over prefetcht0, prefetcht1, and prefetcht2.
These numbers were for the garbage benchmark with MAXPROCS=4
no prefetch >> 5.96 / 5.77 / 5.89
prefetcht0(uintptr(v.ptr().next)) >> 5.88 / 6.17 / 5.84
prefetcht1(uintptr(v.ptr().next)) >> 5.88 / 5.89 / 5.91
prefetcht2(uintptr(v.ptr().next)) >> 5.87 / 6.47 / 5.92
prefetchnta(uintptr(v.ptr().next)) >> 5.72 / 5.84 / 5.85
Change-Id: I54e07172081cccb097d5b5ce8789d74daa055ed9
Reviewed-on: https://go-review.googlesource.com/5350
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Makes them compatible with the new asm.
Applied mechanically from vet diagnostics.
Manual edits: the names for arguments in time·now(SB) in runtime/sys_*_arm.s.
Change-Id: Ib295390d9509d306afc67714e3f50dc832256625
Reviewed-on: https://go-review.googlesource.com/5576
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
With a trivial Golang-built program loaded in gdb-7.8.90.20150214-7.fc23.x86_64
I get this error:
(gdb) source ./src/runtime/runtime-gdb.py
Loading Go Runtime support.
Traceback (most recent call last):
File "./src/runtime/runtime-gdb.py", line 230, in <module>
_rctp_type = gdb.lookup_type("struct reflect.rtype").pointer()
gdb.error: No struct type named reflect.rtype.
(gdb) q
No matter if this struct should or should not be in every Golang-built binary
this change should fix that with no disadvantages.
Change-Id: I0c490d3c9bbe93c65a2183b41bfbdc0c0f405bd1
Reviewed-on: https://go-review.googlesource.com/5521
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trace command allows to visualize and analyze traces.
Run as:
$ go tool trace binary trace.file
The commands opens web browser with the main page,
which contains links for trace visualization,
blocking profiler, network IO profiler and per-goroutine
traces.
Also move trace parser from runtime/pprof/trace_parser_test.go
to internal/trace/parser.go, so that it can be shared between
tests and the command.
Change-Id: Ic97ed59ad6e4c7e1dc9eca5e979701a2b4aed7cf
Reviewed-on: https://go-review.googlesource.com/3601
Reviewed-by: Andrew Gerrand <adg@golang.org>
Restores stack traces in the android/arm builder.
Change-Id: If637aa2ed6f8886126b77cf9cc8a0535ec7c4369
Reviewed-on: https://go-review.googlesource.com/5453
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
In most cases we pass return PC to race detector,
and race runtime subtracts one from them.
However, in manual instrumentation in runtime
we pass function start PC to race runtime.
Race runtime can't distinguish these cases
and so it does not subtract one from top PC.
This leads to bogus line numbers in some cases.
Make it consistent and always pass what looks
like a return PC, so that race runtime can
subtract one and still get PC in the same function.
Also delete two unused functions.
Update #8053
Change-Id: I4242dec5e055e460c9a8990eaca1d085ae240ed2
Reviewed-on: https://go-review.googlesource.com/4902
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is a nice split but more importantly it provides a better
way to fit the checkmark phase into the sequencing.
Also factor out common span copying into gcSpanCopy.
Change-Id: Ia058644974e4ed4ac3cf4b017a3446eb2284d053
Reviewed-on: https://go-review.googlesource.com/5333
Reviewed-by: Austin Clements <austin@google.com>
The loop made more sense when gc_m was not its own function.
Change-Id: I71a7f21d777e69c1924e3b534c507476daa4dfdd
Reviewed-on: https://go-review.googlesource.com/5332
Reviewed-by: Austin Clements <austin@google.com>
See the following issue for context:
https://github.com/golang/go/issues/9729#issuecomment-74648287
In short, RDTSC can produce skewed results without preceding LFENCE/MFENCE.
Information on this matter is very scrappy in the internet.
But this is what linux kernel does (see rdtsc_barrier).
It also fixes the test program on my machine.
Update #9729
Change-Id: I3c1ffbf129fdfdd388bd5b7911b392b319248e68
Reviewed-on: https://go-review.googlesource.com/5033
Reviewed-by: Ian Lance Taylor <iant@golang.org>
