Because profiling signals can arrive at any time, we must
handle the case where a profiling signal arrives halfway
through a goroutine switch. Luckily, although there is much
to think through, very little needs to change.
Fixes#6000.
Fixes#6015.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/13421048
Bug #1:
Issue 5406 identified an interesting case:
defer iface.M()
may end up calling a wrapper that copies an indirect receiver
from the iface value and then calls the real M method. That's
two calls down, not just one, and so recover() == nil always
in the real M method, even during a panic.
[For the purposes of this entire discussion, a wrapper's
implementation is a function containing an ordinary call, not
the optimized tail call form that is somtimes possible. The
tail call does not create a second frame, so it is already
handled correctly.]
Fix this bug by introducing g->panicwrap, which counts the
number of bytes on current stack segment that are due to
wrapper calls that should not count against the recover
check. All wrapper functions must now adjust g->panicwrap up
on entry and back down on exit. This adds slightly to their
expense; on the x86 it is a single instruction at entry and
exit; on the ARM it is three. However, the alternative is to
make a call to recover depend on being able to walk the stack,
which I very much want to avoid. We have enough problems
walking the stack for garbage collection and profiling.
Also, if performance is critical in a specific case, it is already
faster to use a pointer receiver and avoid this kind of wrapper
entirely.
Bug #2:
The old code, which did not consider the possibility of two
calls, already contained a check to see if the call had split
its stack and so the panic-created segment was one behind the
current segment. In the wrapper case, both of the two calls
might split their stacks, so the panic-created segment can be
two behind the current segment.
Fix this by propagating the Stktop.panic flag forward during
stack splits instead of looking backward during recover.
Fixes#5406.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/13367052
This replaces the mcall frame with the badmcall frame instead of
leaving the mcall frame on the stack and adding the badmcall frame.
Because mcall is no longer on the stack, traceback will now report what
called mcall, which is what we would like to see in this situation.
R=golang-dev, cshapiro
CC=golang-dev
https://golang.org/cl/13012044
Actually working to stay within the limit could cause subtle deadlocks.
Crashing avoids the subtlety.
Fixes#4056.
R=golang-dev, r, dvyukov
CC=golang-dev
https://golang.org/cl/13037043
The goal is to stop only those programs that would keep
going and run the machine out of memory, but before they do that.
1 GB on 64-bit, 250 MB on 32-bit.
That seems implausibly large, and it can be adjusted.
Fixes#2556.
Fixes#4494.
Fixes#5173.
R=khr, r, dvyukov
CC=golang-dev
https://golang.org/cl/12541052
Currently it's possible that a goroutine
that periodically executes non-blocking
cgo/syscalls is never preempted.
This change splits scheduler and syscall
ticks to prevent such situation.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12658045
Currently we lose lots of profiling signals.
Most notably, GC is not accounted at all.
But stack splits, scheduler, syscalls, etc are lost as well.
This creates seriously misleading profile.
With this change all profiling signals are accounted.
Now I see these additional entries that were previously absent:
161 29.7% 29.7% 164 30.3% syscall.Syscall
12 2.2% 50.9% 12 2.2% scanblock
11 2.0% 55.0% 11 2.0% markonly
10 1.8% 58.9% 10 1.8% sweepspan
2 0.4% 85.8% 2 0.4% runtime.newstack
It is still impossible to understand what causes stack splits,
but at least it's clear how many time is spent on them.
Update #2197.
Update #5659.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12179043
The mutex, fdMutex, handles locking and lifetime of sysfd,
and serializes Read and Write methods.
This allows to strip 2 sync.Mutex.Lock calls,
2 sync.Mutex.Unlock calls, 1 defer and some amount
of misc overhead from every network operation.
On linux/amd64, Intel E5-2690:
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 9595 9454 -1.47%
BenchmarkTCP4Persistent-2 8978 8772 -2.29%
BenchmarkTCP4ConcurrentReadWrite 4900 4625 -5.61%
BenchmarkTCP4ConcurrentReadWrite-2 2603 2500 -3.96%
In general it strips 70-500 ns from every network operation depending
on processor model. On my relatively new E5-2690 it accounts to ~5%
of network op cost.
Fixes#6074.
R=golang-dev, bradfitz, alex.brainman, iant, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/12418043
Introduce freezetheworld function that is a best-effort attempt to stop any concurrently running goroutines. Call it during crash.
Fixes#5873.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12054044
gcpc/gcsp are used by GC in similar situation.
gcpc/gcsp are also more stable than gp->sched,
because gp->sched is mutated by entersyscall/exitsyscall
in morestack and mcall. So it has higher chances of being inconsistent.
Also, rename gcpc/gcsp to syscallpc/syscallsp.
This is the same as reverted change 12250043
with save marked as textflag 7.
The problem was that if save calls morestack,
then subsequent lessstack spoils g->sched.pc/sp.
And that bad values were remembered in g->syscallpc/sp.
Entersyscallblock had the same problem,
but it was never triggered to date.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12478043
This means that pprof will no longer report profiles on OS X.
That's unfortunate, but the profiles were often wrong and, worse,
it was difficult to tell whether the profile was wrong or not.
The workarounds were making the scheduler more complex,
possibly caused a deadlock (see issue 5519), and did not actually
deliver reliable results.
It may be possible for adventurous users to apply a patch to
their kernels to get working results, or perhaps having no results
will encourage someone to do the work of creating a profiling
thread like on Windows. Issue 6047 has details.
Fixes#5519.
Fixes#6047.
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/12429045
Break all 386 builders.
««« original CL description
runtime: use gcpc/gcsp during traceback of goroutines in syscalls
gcpc/gcsp are used by GC in similar situation.
gcpc/gcsp are also more stable than gp->sched,
because gp->sched is mutated by entersyscall/exitsyscall
in morestack and mcall. So it has higher chances of being inconsistent.
Also, rename gcpc/gcsp to syscallpc/syscallsp.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12250043
»»»
R=rsc
CC=golang-dev
https://golang.org/cl/12424045
It was needed for the old scheduler,
because there temporary could be more threads than gomaxprocs.
In the new scheduler gomaxprocs is always respected.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12438043
gcpc/gcsp are used by GC in similar situation.
gcpc/gcsp are also more stable than gp->sched,
because gp->sched is mutated by entersyscall/exitsyscall
in morestack and mcall. So it has higher chances of being inconsistent.
Also, rename gcpc/gcsp to syscallpc/syscallsp.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12250043
This allows to at least determine goroutine "identity".
Now it looks like:
goroutine 12 [running]:
goroutine running on other thread; stack unavailable
created by testing.RunTests
src/pkg/testing/testing.go:440 +0x88e
R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/12248043
Sysmon thread parks if no goroutines are running (runtime.sched.npidle ==
runtime.gomaxprocs).
Currently it's unparked when a goroutine enters syscall, it was enough
to retake P's from blocking syscalls.
But it's not enough for reliable goroutine preemption. We need to ensure that
sysmon runs if any goroutines are running.
R=rsc
CC=golang-dev
https://golang.org/cl/12176043
Submitted with some unrelated changes that were not intended to go in.
««« original CL description
runtime: do not park sysmon thread if any goroutines are running
Sysmon thread parks if no goroutines are running (runtime.sched.npidle == runtime.gomaxprocs).
Currently it's unparked when a goroutine enters syscall, it was enough
to retake P's from blocking syscalls.
But it's not enough for reliable goroutine preemption. We need to ensure that
sysmon runs if any goroutines are running.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12167043
»»»
R=rsc
CC=golang-dev
https://golang.org/cl/12171044
Sysmon thread parks if no goroutines are running (runtime.sched.npidle == runtime.gomaxprocs).
Currently it's unparked when a goroutine enters syscall, it was enough
to retake P's from blocking syscalls.
But it's not enough for reliable goroutine preemption. We need to ensure that
sysmon runs if any goroutines are running.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12167043
Split stack checks (morestack) corrupt g->sched,
but g->sched must be preserved consistent for GC/traceback.
The change implements runtime.notetsleepg function,
which does entersyscall/exitsyscall and is carefully arranged
to not call any split functions in between.
R=rsc
CC=golang-dev
https://golang.org/cl/11575044
Make it accept type, combine flags.
Several reasons for the change:
1. mallocgc and settype must be atomic wrt GC
2. settype is called from only one place now
3. it will help performance (eventually settype
functionality must be combined with markallocated)
4. flags are easier to read now (no mallocgc(sz, 0, 1, 0) anymore)
R=golang-dev, iant, nightlyone, rsc, dave, khr, bradfitz, r
CC=golang-dev
https://golang.org/cl/10136043
Debugging the Windows breakage I noticed that SEH
only exists on 386, so we can balance the two stacks
a little more on amd64 and reclaim another word.
Now we're down to just one word consumed by
cgocallback_gofunc, having reclaimed 25% of the
overall budget (4 words out of 16).
Separately, fix windows/386 - the SEH must be on the
m0 stack, as must the saved SP, so we are forced to have
a three-word frame for 386. It matters much less for
386, because there 128 bytes gives 32 words to use.
R=dvyukov, alex.brainman
CC=golang-dev
https://golang.org/cl/11551044
Tying preemption to stack splits means that we have to able to
complete the call to exitsyscall (inside cgocallbackg at least for now)
without any stack split checks, meaning that the whole sequence
has to work within 128 bytes of stack, unless we increase the size
of the red zone. This CL frees up 24 bytes along that critical path
on amd64. (The 32-bit systems have plenty of space because all
their words are smaller.)
R=dvyukov
CC=golang-dev
https://golang.org/cl/11676043
If the network is not polled for 10ms, sysmon starts polling network
on every iteration (every 20us) until another thread blocks in netpoll.
Fixes#5922.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11569043
If we start a garbage collection on g0 during a
stack split or unsplit, we'll see morestack or lessstack
at the top of the stack. Record an argument frame size
for those, and record that they terminate the stack.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11533043
Otherwise the tests in pkg/runtime fail:
runtime: unknown argument frame size for runtime.deferreturn called from 0x48657b [runtime_test.func·022]
fatal error: invalid stack
...
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11483043
Update #543
I believe the runtime is strong enough now to reenable
preemption during the function prologue.
Assuming this is or can be made stable, it will be in Go 1.2.
More aggressive preemption is not planned for Go 1.2.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11433045
Currently preemption signal g->stackguard0==StackPreempt
can be lost if it is received when preemption is disabled
(e.g. m->lock!=0). This change duplicates the preemption
signal in g->preempt and restores g->stackguard0
when preemption is enabled.
Update #543.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10792043
With this CL, I believe the runtime always knows
the frame size during the gc walk. There is no fallback
to "assume entire stack frame of caller" anymore.
R=golang-dev, khr, cshapiro, dvyukov
CC=golang-dev
https://golang.org/cl/11374044
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
This is reincarnation of cl/9776044 with the bug fixed.
The bug was due to code added after cl/9776044 was created:
if(tick - (((uint64)tick*0x4325c53fu)>>36)*61 == 0 && runtime·sched.runqsize > 0) {
runtime·lock(&runtime·sched);
gp = globrunqget(m->p, 1);
runtime·unlock(&runtime·sched);
}
If M gets gp from global runq here, it does not reset m->spinning.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10743044
There are various problems, and both Dmitriy and I
will be away for the next week. Make the runtime a bit
more stable while we're gone.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10848043
Currently it replaces GOGCTRACE env var (GODEBUG=gctrace=1).
The plan is to extend it with other type of debug tracing,
e.g. GODEBUG=gctrace=1,schedtrace=100.
R=rsc
CC=bradfitz, daniel.morsing, gobot, golang-dev
https://golang.org/cl/10026045
The last patch for preemptive scheduler,
with this change stoptheworld issues preemption
requests every 100us.
Update #543.
R=golang-dev, daniel.morsing, rsc
CC=golang-dev
https://golang.org/cl/10264044
Failure on bot:
http://build.golang.org/log/f4c648906e1289ec2237c1d0880fb1a8b1852a08
««« original CL description
runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
R=rsc
TBR=rsc
CC=gobot, golang-dev
https://golang.org/cl/9776044
»»»
R=golang-dev
CC=golang-dev
https://golang.org/cl/10692043
