1
0
mirror of https://github.com/golang/go synced 2024-10-04 08:31:22 -06:00
Commit Graph

261 Commits

Author SHA1 Message Date
Russ Cox
14062efb16 runtime: handle runtime.Goexit during init
Fixes #5963.

R=golang-dev, dsymonds, dvyukov
CC=golang-dev
https://golang.org/cl/11879045
2013-07-26 13:54:44 -04:00
Dmitriy Vyukov
f8a850b250 runtime: refactor mallocgc
Make it accept type, combine flags.
Several reasons for the change:
1. mallocgc and settype must be atomic wrt GC
2. settype is called from only one place now
3. it will help performance (eventually settype
functionality must be combined with markallocated)
4. flags are easier to read now (no mallocgc(sz, 0, 1, 0) anymore)

R=golang-dev, iant, nightlyone, rsc, dave, khr, bradfitz, r
CC=golang-dev
https://golang.org/cl/10136043
2013-07-26 21:17:24 +04:00
Keith Randall
3453a2204b runtime: only define SEH when we need it.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11769043
2013-07-24 09:59:47 -07:00
Russ Cox
f011282578 runtime: more cgocallback_gofunc work
Debugging the Windows breakage I noticed that SEH
only exists on 386, so we can balance the two stacks
a little more on amd64 and reclaim another word.

Now we're down to just one word consumed by
cgocallback_gofunc, having reclaimed 25% of the
overall budget (4 words out of 16).

Separately, fix windows/386 - the SEH must be on the
m0 stack, as must the saved SP, so we are forced to have
a three-word frame for 386. It matters much less for
386, because there 128 bytes gives 32 words to use.

R=dvyukov, alex.brainman
CC=golang-dev
https://golang.org/cl/11551044
2013-07-24 09:01:57 -04:00
Russ Cox
dba623b1c7 runtime: reduce frame size for runtime.cgocallback_gofunc
Tying preemption to stack splits means that we have to able to
complete the call to exitsyscall (inside cgocallbackg at least for now)
without any stack split checks, meaning that the whole sequence
has to work within 128 bytes of stack, unless we increase the size
of the red zone. This CL frees up 24 bytes along that critical path
on amd64. (The 32-bit systems have plenty of space because all
their words are smaller.)

R=dvyukov
CC=golang-dev
https://golang.org/cl/11676043
2013-07-23 18:40:02 -04:00
David Symonds
ae5991695c runtime: add a missing newline in a debug printf.
Trivial, but annoying while debugging this code.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/11656043
2013-07-22 12:42:42 +10:00
Dmitriy Vyukov
6857264457 runtime: prevent sysmon from polling network excessivly
If the network is not polled for 10ms, sysmon starts polling network
on every iteration (every 20us) until another thread blocks in netpoll.
Fixes #5922.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11569043
2013-07-19 17:45:34 +04:00
Dmitriy Vyukov
bc31bcccd3 runtime: preempt long-running goroutines
If a goroutine runs for more than 10ms, preempt it.
Update #543.

R=rsc
CC=golang-dev
https://golang.org/cl/10796043
2013-07-19 01:22:26 +04:00
Russ Cox
58f12ffd79 runtime: handle morestack/lessstack in stack trace
If we start a garbage collection on g0 during a
stack split or unsplit, we'll see morestack or lessstack
at the top of the stack. Record an argument frame size
for those, and record that they terminate the stack.

R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11533043
2013-07-18 16:53:45 -04:00
Ian Lance Taylor
1da96a3039 runtime: disable preemption again to fix linux build
Otherwise the tests in pkg/runtime fail:

runtime: unknown argument frame size for runtime.deferreturn called from 0x48657b [runtime_test.func·022]
fatal error: invalid stack
...

R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11483043
2013-07-17 16:15:46 -07:00
Russ Cox
b913cf84dc runtime: re-enable preemption
Update #543

I believe the runtime is strong enough now to reenable
preemption during the function prologue.
Assuming this is or can be made stable, it will be in Go 1.2.
More aggressive preemption is not planned for Go 1.2.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11433045
2013-07-17 14:03:27 -04:00
Dmitriy Vyukov
5887f142a3 runtime: more reliable preemption
Currently preemption signal g->stackguard0==StackPreempt
can be lost if it is received when preemption is disabled
(e.g. m->lock!=0). This change duplicates the preemption
signal in g->preempt and restores g->stackguard0
when preemption is enabled.
Update #543.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10792043
2013-07-17 12:52:37 -04:00
Russ Cox
a83748596c runtime: use new frame argument size information
With this CL, I believe the runtime always knows
the frame size during the gc walk. There is no fallback
to "assume entire stack frame of caller" anymore.

R=golang-dev, khr, cshapiro, dvyukov
CC=golang-dev
https://golang.org/cl/11374044
2013-07-17 12:47:18 -04:00
Dmitriy Vyukov
32fef9908a runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes #5586.
This is reincarnation of cl/9776044 with the bug fixed.
The bug was due to code added after cl/9776044 was created:
if(tick - (((uint64)tick*0x4325c53fu)>>36)*61 == 0 && runtime·sched.runqsize > 0) {
        runtime·lock(&runtime·sched);
        gp = globrunqget(m->p, 1);
        runtime·unlock(&runtime·sched);
}
If M gets gp from global runq here, it does not reset m->spinning.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10743044
2013-07-11 15:57:36 -04:00
Russ Cox
08e064135d runtime: disable preemption
There are various problems, and both Dmitriy and I
will be away for the next week. Make the runtime a bit
more stable while we're gone.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10848043
2013-07-01 17:57:09 -04:00
Dmitriy Vyukov
4b536a550f runtime: introduce GODEBUG env var
Currently it replaces GOGCTRACE env var (GODEBUG=gctrace=1).
The plan is to extend it with other type of debug tracing,
e.g. GODEBUG=gctrace=1,schedtrace=100.

R=rsc
CC=bradfitz, daniel.morsing, gobot, golang-dev
https://golang.org/cl/10026045
2013-06-28 18:37:06 +04:00
Dmitriy Vyukov
1e112cd59f runtime: preempt goroutines for GC
The last patch for preemptive scheduler,
with this change stoptheworld issues preemption
requests every 100us.
Update #543.

R=golang-dev, daniel.morsing, rsc
CC=golang-dev
https://golang.org/cl/10264044
2013-06-28 17:52:17 +04:00
Ian Lance Taylor
ab1270bcfc runtime: remove declaration of function that does not exist
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/10730045
2013-06-27 22:43:30 -07:00
Dmitriy Vyukov
7ebb187e8e undo CL 9776044 / 1e280889f997
Failure on bot:
http://build.golang.org/log/f4c648906e1289ec2237c1d0880fb1a8b1852a08

««« original CL description
runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes #5586.

R=rsc
TBR=rsc
CC=gobot, golang-dev
https://golang.org/cl/9776044
»»»

R=golang-dev
CC=golang-dev
https://golang.org/cl/10692043
2013-06-27 21:03:35 +04:00
Dmitriy Vyukov
15a1c3d1e4 runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes #5586.

R=rsc
CC=gobot, golang-dev
https://golang.org/cl/9776044
2013-06-27 20:52:12 +04:00
Russ Cox
6fa3c89b77 runtime: record proper goroutine state during stack split
Until now, the goroutine state has been scattered during the
execution of newstack and oldstack. It's all there, and those routines
know how to get back to a working goroutine, but other pieces of
the system, like stack traces, do not. If something does interrupt
the newstack or oldstack execution, the rest of the system can't
understand the goroutine. For example, if newstack decides there
is an overflow and calls throw, the stack tracer wouldn't dump the
goroutine correctly.

For newstack to save a useful state snapshot, it needs to be able
to rewind the PC in the function that triggered the split back to
the beginning of the function. (The PC is a few instructions in, just
after the call to morestack.) To make that possible, we change the
prologues to insert a jmp back to the beginning of the function
after the call to morestack. That is, the prologue used to be roughly:

        TEXT myfunc
                check for split
                jmpcond nosplit
                call morestack
        nosplit:
                sub $xxx, sp

Now an extra instruction is inserted after the call:

        TEXT myfunc
        start:
                check for split
                jmpcond nosplit
                call morestack
                jmp start
        nosplit:
                sub $xxx, sp

The jmp is not executed directly. It is decoded and simulated by
runtime.rewindmorestack to discover the beginning of the function,
and then the call to morestack returns directly to the start label
instead of to the jump instruction. So logically the jmp is still
executed, just not by the cpu.

The prologue thus repeats in the case of a function that needs a
stack split, but against the cost of the split itself, the extra few
instructions are noise. The repeated prologue has the nice effect of
making a stack split double-check that the new stack is big enough:
if morestack happens to return on a too-small stack, we'll now notice
before corruption happens.

The ability for newstack to rewind to the beginning of the function
should help preemption too. If newstack decides that it was called
for preemption instead of a stack split, it now has the goroutine state
correctly paused if rescheduling is needed, and when the goroutine
can run again, it can return to the start label on its original stack
and re-execute the split check.

Here is an example of a split stack overflow showing the full
trace, without any special cases in the stack printer.
(This one was triggered by making the split check incorrect.)

runtime: newstack framesize=0x0 argsize=0x18 sp=0x6aebd0 stack=[0x6b0000, 0x6b0fa0]
        morebuf={pc:0x69f5b sp:0x6aebd8 lr:0x0}
        sched={pc:0x68880 sp:0x6aebd0 lr:0x0 ctxt:0x34e700}
runtime: split stack overflow: 0x6aebd0 < 0x6b0000
fatal error: runtime: split stack overflow

goroutine 1 [stack split]:
runtime.mallocgc(0x290, 0x100000000, 0x1)
        /Users/rsc/g/go/src/pkg/runtime/zmalloc_darwin_amd64.c:21 fp=0x6aebd8
runtime.new()
        /Users/rsc/g/go/src/pkg/runtime/zmalloc_darwin_amd64.c:682 +0x5b fp=0x6aec08
go/build.(*Context).Import(0x5ae340, 0xc210030c71, 0xa, 0xc2100b4380, 0x1b, ...)
        /Users/rsc/g/go/src/pkg/go/build/build.go:424 +0x3a fp=0x6b00a0
main.loadImport(0xc210030c71, 0xa, 0xc2100b4380, 0x1b, 0xc2100b42c0, ...)
        /Users/rsc/g/go/src/cmd/go/pkg.go:249 +0x371 fp=0x6b01a8
main.(*Package).load(0xc21017c800, 0xc2100b42c0, 0xc2101828c0, 0x0, 0x0, ...)
        /Users/rsc/g/go/src/cmd/go/pkg.go:431 +0x2801 fp=0x6b0c98
main.loadPackage(0x369040, 0x7, 0xc2100b42c0, 0x0)
        /Users/rsc/g/go/src/cmd/go/pkg.go:709 +0x857 fp=0x6b0f80
----- stack segment boundary -----
main.(*builder).action(0xc2100902a0, 0x0, 0x0, 0xc2100e6c00, 0xc2100e5750, ...)
        /Users/rsc/g/go/src/cmd/go/build.go:539 +0x437 fp=0x6b14a0
main.(*builder).action(0xc2100902a0, 0x0, 0x0, 0xc21015b400, 0x2, ...)
        /Users/rsc/g/go/src/cmd/go/build.go:528 +0x1d2 fp=0x6b1658
main.(*builder).test(0xc2100902a0, 0xc210092000, 0x0, 0x0, 0xc21008ff60, ...)
        /Users/rsc/g/go/src/cmd/go/test.go:622 +0x1b53 fp=0x6b1f68
----- stack segment boundary -----
main.runTest(0x5a6b20, 0xc21000a020, 0x2, 0x2)
        /Users/rsc/g/go/src/cmd/go/test.go:366 +0xd09 fp=0x6a5cf0
main.main()
        /Users/rsc/g/go/src/cmd/go/main.go:161 +0x4f9 fp=0x6a5f78
runtime.main()
        /Users/rsc/g/go/src/pkg/runtime/proc.c:183 +0x92 fp=0x6a5fa0
runtime.goexit()
        /Users/rsc/g/go/src/pkg/runtime/proc.c:1266 fp=0x6a5fa8

And here is a seg fault during oldstack:

SIGSEGV: segmentation violation
PC=0x1b2a6

runtime.oldstack()
        /Users/rsc/g/go/src/pkg/runtime/stack.c:159 +0x76
runtime.lessstack()
        /Users/rsc/g/go/src/pkg/runtime/asm_amd64.s:270 +0x22

goroutine 1 [stack unsplit]:
fmt.(*pp).printArg(0x2102e64e0, 0xe5c80, 0x2102c9220, 0x73, 0x0, ...)
        /Users/rsc/g/go/src/pkg/fmt/print.go:818 +0x3d3 fp=0x221031e6f8
fmt.(*pp).doPrintf(0x2102e64e0, 0x12fb20, 0x2, 0x221031eb98, 0x1, ...)
        /Users/rsc/g/go/src/pkg/fmt/print.go:1183 +0x15cb fp=0x221031eaf0
fmt.Sprintf(0x12fb20, 0x2, 0x221031eb98, 0x1, 0x1, ...)
        /Users/rsc/g/go/src/pkg/fmt/print.go:234 +0x67 fp=0x221031eb40
flag.(*stringValue).String(0x2102c9210, 0x1, 0x0)
        /Users/rsc/g/go/src/pkg/flag/flag.go:180 +0xb3 fp=0x221031ebb0
flag.(*FlagSet).Var(0x2102f6000, 0x293d38, 0x2102c9210, 0x143490, 0xa, ...)
        /Users/rsc/g/go/src/pkg/flag/flag.go:633 +0x40 fp=0x221031eca0
flag.(*FlagSet).StringVar(0x2102f6000, 0x2102c9210, 0x143490, 0xa, 0x12fa60, ...)
        /Users/rsc/g/go/src/pkg/flag/flag.go:550 +0x91 fp=0x221031ece8
flag.(*FlagSet).String(0x2102f6000, 0x143490, 0xa, 0x12fa60, 0x0, ...)
        /Users/rsc/g/go/src/pkg/flag/flag.go:563 +0x87 fp=0x221031ed38
flag.String(0x143490, 0xa, 0x12fa60, 0x0, 0x161950, ...)
        /Users/rsc/g/go/src/pkg/flag/flag.go:570 +0x6b fp=0x221031ed80
testing.init()
        /Users/rsc/g/go/src/pkg/testing/testing.go:-531 +0xbb fp=0x221031edc0
strings_test.init()
        /Users/rsc/g/go/src/pkg/strings/strings_test.go:1115 +0x62 fp=0x221031ef70
main.init()
        strings/_test/_testmain.go:90 +0x3d fp=0x221031ef78
runtime.main()
        /Users/rsc/g/go/src/pkg/runtime/proc.c:180 +0x8a fp=0x221031efa0
runtime.goexit()
        /Users/rsc/g/go/src/pkg/runtime/proc.c:1269 fp=0x221031efa8

goroutine 2 [runnable]:
runtime.MHeap_Scavenger()
        /Users/rsc/g/go/src/pkg/runtime/mheap.c:438
runtime.goexit()
        /Users/rsc/g/go/src/pkg/runtime/proc.c:1269
created by runtime.main
        /Users/rsc/g/go/src/pkg/runtime/proc.c:166

rax     0x23ccc0
rbx     0x23ccc0
rcx     0x0
rdx     0x38
rdi     0x2102c0170
rsi     0x221032cfe0
rbp     0x221032cfa0
rsp     0x7fff5fbff5b0
r8      0x2102c0120
r9      0x221032cfa0
r10     0x221032c000
r11     0x104ce8
r12     0xe5c80
r13     0x1be82baac718
r14     0x13091135f7d69200
r15     0x0
rip     0x1b2a6
rflags  0x10246
cs      0x2b
fs      0x0
gs      0x0

Fixes #5723.

R=r, dvyukov, go.peter.90, dave, iant
CC=golang-dev
https://golang.org/cl/10360048
2013-06-27 11:32:01 -04:00
Dmitriy Vyukov
4bb491b12e runtime: improve scheduler fairness
Currently global runqueue is starved if a group of goroutines
constantly respawn each other (local runqueue never becomes empty).
Fixes #5639.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/10042044
2013-06-15 16:06:28 +04:00
Russ Cox
d67e7e3acf runtime: add lr, ctxt, ret to Gobuf
Add gostartcall and gostartcallfn.
The old gogocall = gostartcall + gogo.
The old gogocallfn = gostartcallfn + gogo.

R=dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/10036044
2013-06-12 15:22:26 -04:00
Dmitriy Vyukov
dbcfed93e7 runtime: fix scheduler race condition
In starttheworld() we assume that P's with local work
are situated in the beginning of idle P list.
However, once we start the first M, it can execute all local G's
and steal G's from other P's.
That breaks the assumption above. Thus starttheworld() will fail
to start some P's with local work.
It seems that it can not lead to very bad things, but still
it's wrong and breaks other assumtions
(e.g. we can have a spinning M with local work).
The fix is to collect all P's with local work first,
and only then start them.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10051045
2013-06-12 18:46:35 +04:00
Russ Cox
e58f798c0c runtime: adjust traceback / garbage collector boundary
The garbage collection routine addframeroots is duplicating
logic in the traceback routine that calls it, sometimes correctly,
sometimes incorrectly, sometimes incompletely.
Pass necessary information to addframeroots instead of
deriving it anew.

Should make addframeroots significantly more robust.
It's certainly smaller.

Also try to standardize on uintptr for saved pc, sp values.

Will make CL 10036044 trivial.

R=golang-dev, dave, dvyukov
CC=golang-dev
https://golang.org/cl/10169045
2013-06-12 08:49:38 -04:00
Ian Lance Taylor
b6e52ecffa runtime: remove unused mid function
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10036047
2013-06-06 18:10:42 -07:00
Dmitriy Vyukov
4a8ef1f65d runtime: disable preemption in several scheduler functions
Required for preemptive scheduler, see the comments for details.

R=golang-dev, khr, iant, khr
CC=golang-dev
https://golang.org/cl/9740051
2013-06-03 14:40:38 +04:00
Dmitriy Vyukov
354ec51666 runtime: introduce preemption function (not used for now)
This is part of preemptive scheduler.

R=golang-dev, cshapiro, iant
CC=golang-dev
https://golang.org/cl/9843046
2013-06-03 13:20:17 +04:00
Dmitriy Vyukov
f5becf4233 runtime: add stackguard0 to G
This is part of preemptive scheduler.
stackguard0 is checked in split stack checks and can be set to StackPreempt.
stackguard is not set to StackPreempt (holds the original value).

R=golang-dev, daniel.morsing, iant
CC=golang-dev
https://golang.org/cl/9875043
2013-06-03 12:28:24 +04:00
Dmitriy Vyukov
573d25a423 runtime: mark runtime.goexit as nosplit
Required for preemptive scheduler, see the comment.

R=golang-dev, daniel.morsing
CC=golang-dev
https://golang.org/cl/9841047
2013-05-30 14:11:49 +04:00
Dmitriy Vyukov
081129e286 runtime: allocate internal symbol table eagerly
we need it for GC anyway.

R=golang-dev, khr, dave, khr
CC=golang-dev
https://golang.org/cl/9728044
2013-05-28 21:10:10 +04:00
Dmitriy Vyukov
34c67eb24e runtime: detect deadlocks in programs using cgo
When cgo is used, runtime creates an additional M to handle callbacks on threads not created by Go.
This effectively disabled deadlock detection, which is a right thing, because Go program can be blocked
and only serve callbacks on external threads.
This also disables deadlock detection under race detector, because it happens to use cgo.
With this change the additional M is created lazily on first cgo call. So deadlock detector
works for programs that import "C", "net" or "net/http/pprof" but do not use them in fact.
Also fixes deadlock detector under race detector.
It should be fine to create the M later, because C code can not call into Go before first cgo call,
because C code does not know when Go initialization has completed. So a Go program need to call into C
first either to create an external thread, or notify a thread created in global ctor that Go
initialization has completed.
Fixes #4973.
Fixes #5475.

R=golang-dev, minux.ma, iant
CC=golang-dev
https://golang.org/cl/9303046
2013-05-22 22:57:47 +04:00
Dmitriy Vyukov
1308194204 runtime: zeroize g->fnstart to not prevent GC of the closure
Fixes #5493.

R=golang-dev, minux.ma, iant
CC=golang-dev
https://golang.org/cl/9557043
2013-05-20 08:17:21 +04:00
Dmitriy Vyukov
fee1d1cda0 runtime: properly set G status after syscall
R=golang-dev, r, dave
CC=golang-dev
https://golang.org/cl/9307045
2013-05-19 19:35:09 +04:00
Anthony Martin
b65271d008 runtime: fix newproc debugging print
R=golang-dev, remyoudompheng, r
CC=golang-dev
https://golang.org/cl/9249044
2013-05-18 15:47:15 -07:00
Dmitriy Vyukov
0b5d55984f runtime: fix deadlock in network poller
The invariant is that there must be at least one running P or a thread polling network.
It was broken.
Fixes #5216.

R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/8459043
2013-04-06 22:27:54 -07:00
Dmitriy Vyukov
54340bf56f runtime: reset dangling typed pointer
+untype it because it can point to different types
Update #5193.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/8454043
2013-04-06 20:01:28 -07:00
Dmitriy Vyukov
77354c39f9 runtime: reset typed dangling pointer
If for whatever reason seh points into Go heap region,
the dangling pointer will cause memory corruption during GC.
Update #5193.

R=golang-dev, alex.brainman, iant
CC=golang-dev
https://golang.org/cl/8402045
2013-04-06 20:00:45 -07:00
Carl Shapiro
c676b8b27b cmd/ld, runtime: restrict stack root scan to locals and arguments
Updates #5134

R=golang-dev, bradfitz, cshapiro, daniel.morsing, ality, iant
CC=golang-dev
https://golang.org/cl/8022044
2013-03-28 14:36:23 -07:00
Dmitriy Vyukov
d4c80d19a8 runtime: faster parallel GC
Use per-thread work buffers instead of global mutex-protected pool. This eliminates contention from parallel scan phase.

benchmark                             old ns/op    new ns/op    delta
garbage.BenchmarkTree2-8               97100768     71417553  -26.45%
garbage.BenchmarkTree2LastPause-8     970931485    714103692  -26.45%
garbage.BenchmarkTree2Pause-8         469127802    345029253  -26.45%
garbage.BenchmarkParser-8            2880950854   2715456901   -5.74%
garbage.BenchmarkParserLastPause-8    137047399    103336476  -24.60%
garbage.BenchmarkParserPause-8         80686028     58922680  -26.97%

R=golang-dev, 0xe2.0x9a.0x9b, dave, adg, rsc, iant
CC=golang-dev
https://golang.org/cl/7816044
2013-03-21 12:48:02 +04:00
Russ Cox
5146a93e72 runtime: accept GOTRACEBACK=crash to mean 'crash after panic'
This provides a way to generate core dumps when people need them.
The settings are:

        GOTRACEBACK=0  no traceback on panic, just exit
        GOTRACEBACK=1  default - traceback on panic, then exit
        GOTRACEBACK=2  traceback including runtime frames on panic, then exit
        GOTRACEBACK=crash traceback including runtime frames on panic, then crash

Fixes #3257.

R=golang-dev, devon.odell, r, daniel.morsing, ality
CC=golang-dev
https://golang.org/cl/7666044
2013-03-15 01:11:03 -04:00
Dmitriy Vyukov
c211884731 runtime: add network polling support into scheduler
This is a part of the bigger change that moves network poller into runtime:
https://golang.org/cl/7326051/

R=golang-dev, bradfitz, mikioh.mikioh, rsc
CC=golang-dev
https://golang.org/cl/7448048
2013-03-12 21:14:26 +04:00
Dmitriy Vyukov
6ee739d7e9 runtime: fix deadlock detector false negative
The issue was that scvg is assigned *after* the scavenger goroutine is started,
so when the scavenger calls entersyscall() the g==scvg check can fail.
Fixes #5025.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7629045
2013-03-12 17:21:44 +04:00
Russ Cox
1a4599b41a runtime: clear locked bit when goroutine exits
Otherwise the next goroutine run on the m
can get inadvertently locked if it executes a cgo call
that turns on the internal lock.

While we're here, fix the cgo panic unwind to
decrement m->ncgo like the non-panic unwind does.

Fixes #4971.

R=golang-dev, iant, dvyukov
CC=golang-dev
https://golang.org/cl/7627043
2013-03-08 11:26:00 -05:00
Dmitriy Vyukov
cb945ba6ba runtime: fix deadlock
The deadlock episodically occurs on misc/cgo/test/TestCthread.
The problem is that starttheworld() leaves some P's with local work
without M's. Then all active M's enter into syscalls, but reject to
wake another M's due to the following check (both in entersyscallblock() and in retake()):
if(p->runqhead == p->runqtail &&
        runtime·atomicload(&runtime·sched.nmspinning) +
        runtime·atomicload(&runtime·sched.npidle) > 0)
        continue;

R=rsc
CC=golang-dev
https://golang.org/cl/7424054
2013-03-07 21:39:59 +04:00
Russ Cox
e0deb2ef7f undo CL 7301062 / 9742f722b558
broke arm garbage collector

traceback_arm fails with a missing pc. It needs CL 7494043.
But that only makes the build break later, this time with
"invalid freelist". Roll back until it can be fixed correctly.

««« original CL description
runtime: restrict stack root scan to locals and arguments

R=rsc
CC=golang-dev
https://golang.org/cl/7301062
»»»

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7493044
2013-03-05 15:36:40 -05:00
Carl Shapiro
db018bf77c runtime: restrict stack root scan to locals and arguments
R=rsc
CC=golang-dev
https://golang.org/cl/7301062
2013-03-04 19:48:50 -08:00
Russ Cox
69c90985b3 runtime: add link to design doc for new scheduler
R=golang-dev, remyoudompheng, bradfitz
CC=golang-dev
https://golang.org/cl/7419049
2013-03-04 10:36:45 -05:00
Russ Cox
3611553c3b runtime: add atomics to fix arm
R=golang-dev, minux.ma
CC=golang-dev
https://golang.org/cl/7429046
2013-03-01 14:57:05 -05:00
Russ Cox
e6a3e22c75 runtime: start all threads with runtime.mstart
Putting the M initialization in multiple places will not scale.
Various code assumes mstart is the start already. Make it so.

R=golang-dev, devon.odell
CC=golang-dev
https://golang.org/cl/7420048
2013-03-01 11:44:43 -05:00
Dmitriy Vyukov
779c45a507 runtime: improved scheduler
Distribute runnable queues, memory cache
and cache of dead G's per processor.
Faster non-blocking syscall enter/exit.
More conservative worker thread blocking/unblocking.

R=dave, bradfitz, remyoudompheng, rsc
CC=golang-dev
https://golang.org/cl/7314062
2013-03-01 13:49:16 +02:00
Russ Cox
f8d49b509b runtime/cgo: make symbol naming consistent
The naming in this package is a disaster.
Make it all consistent.

Remove some 'static' from functions that will
be referred to from other files soon.

This CL is purely renames using global search and replace.

Submitting separately so that real changes will not
be drowned out by these renames in future CLs.

TBR=iant
CC=golang-dev
https://golang.org/cl/7416046
2013-02-28 16:24:38 -05:00
Dmitriy Vyukov
6cdfb00f4e runtime: more changes in preparation to the new scheduler
add per-P cache of dead G's
add global runnable queue (not used for now)
add list of idle P's (not used for now)

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7397061
2013-02-27 21:17:53 +02:00
Dmitriy Vyukov
1e957b6245 runtime: fix windows cpu profiling
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/7407044
2013-02-23 10:07:41 +04:00
Dmitriy Vyukov
353ce60f6e runtime: implement local work queues (in preparation for new scheduler)
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7402047
2013-02-23 08:48:02 +04:00
Dmitriy Vyukov
1d7faf91df runtime: minor changes
to minimize diffs of new scheduler

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7381048
2013-02-23 08:39:31 +04:00
Russ Cox
1903ad7189 cmd/gc, reflect, runtime: switch to indirect func value representation
Step 1 of http://golang.org/s/go11func.

R=golang-dev, r, daniel.morsing, remyoudompheng
CC=golang-dev
https://golang.org/cl/7393045
2013-02-21 17:01:13 -05:00
Dmitriy Vyukov
94fab3cad3 runtime: fix heap corruption
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7397049
2013-02-21 21:59:46 +04:00
Dmitriy Vyukov
a0955a2aa2 runtime: split minit() to mpreinit() and minit()
mpreinit() is called on the parent thread and with mcache (can allocate memory),
minit() is called on the child thread and can not allocate memory.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7389043
2013-02-21 16:24:38 +04:00
Russ Cox
6c976393ae runtime: allow cgo callbacks on non-Go threads
Fixes #4435.

R=golang-dev, iant, alex.brainman, minux.ma, dvyukov
CC=golang-dev
https://golang.org/cl/7304104
2013-02-20 17:48:23 -05:00
Dmitriy Vyukov
e25f19a638 runtime: introduce entersyscallblock()
In preparation for the new scheduler.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7386044
2013-02-20 20:21:45 +04:00
Dmitriy Vyukov
06a488fa97 runtime: fix deadlock detector false negative
Fixes #4819.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7322086
2013-02-20 12:15:02 +04:00
Dmitriy Vyukov
a92e11a256 runtime: ensure forward progress of runtime.Gosched() for locked goroutines
The removed code leads to the situation when M executes the same locked G again
and again.
This is https://golang.org/cl/7310096 but with return instead of break
in the nested switch.
Fixes #4820.

R=golang-dev, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/7304102
2013-02-20 12:13:04 +04:00
Russ Cox
60526ca6d1 undo CL 7310096 / 59da6744d66d
broke windows build

««« original CL description
runtime: ensure forward progress of runtime.Gosched() for locked goroutines
The removed code leads to the situation when M executes the same locked G again and again.
Fixes #4820.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7310096
»»»

TBR=dvyukov
CC=golang-dev
https://golang.org/cl/7343050
2013-02-15 17:54:46 -05:00
Russ Cox
c8214c78bd runtime: make return from main wait for active panic
Arguably if this happens the program is buggy anyway,
but letting the panic continue looks better than interrupting it.
Otherwise things like this are possible, and confusing:

$ go run x.go
panic: $ echo $?
0
$

Fixes #3934.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7322083
2013-02-15 14:48:35 -05:00
Dmitriy Vyukov
f87b7f67b2 runtime: ensure forward progress of runtime.Gosched() for locked goroutines
The removed code leads to the situation when M executes the same locked G again and again.
Fixes #4820.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7310096
2013-02-15 22:22:13 +04:00
Dmitriy Vyukov
691455f780 runtime: move stack management related code to stack.c
No code changes.
This is mainly in preparation to scheduler changes,
oldstack/newstack are not related to scheduling.

R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/7311085
2013-02-14 12:37:55 +04:00
Dmitriy Vyukov
0a40cd2661 runtime/race: switch to explicit race context instead of goroutine id's
Removes limit on maximum number of goroutines ever existed.
code.google.com/p/goexecutor tests now pass successfully.
Also slightly improves performance.
Before: $ time ./flate.test -test.short
real	0m9.314s
After:  $ time ./flate.test -test.short
real	0m8.958s
Fixes #4286.
The runtime is built from llvm rev 174312.

R=rsc
CC=golang-dev
https://golang.org/cl/7218044
2013-02-06 11:40:54 +04:00
Russ Cox
b0a29f393b runtime: cgo-related fixes
* Separate internal and external LockOSThread, for cgo safety.
* Show goroutine that made faulting cgo call.
* Never start a panic due to a signal caused by a cgo call.

Fixes #3774.
Fixes #3775.
Fixes #3797.

R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7228081
2013-02-01 08:34:41 -08:00
Jan Ziak
4da6b36fbf runtime: local allocation in mprof.goc
Binary data in mprof.goc may prevent the garbage collector from freeing
memory blocks. This patch replaces all calls to runtime·mallocgc() with
calls to an allocator private to mprof.goc, thus making the private
memory invisible to the garbage collector. The addrhash variable is
moved outside of the .bss section.

R=golang-dev, dvyukov, rsc, minux.ma
CC=dave, golang-dev, remyoudompheng
https://golang.org/cl/7135063
2013-01-30 09:01:31 -08:00
Dmitriy Vyukov
81221f512d runtime: dump the full stack of a throwing goroutine
Useful for debugging of runtime bugs.
+ Do not print "stack segment boundary" unless GOTRACEBACK>1.
+ Do not traceback system goroutines unless GOTRACEBACK>1.

R=rsc, minux.ma
CC=golang-dev
https://golang.org/cl/7098050
2013-01-29 14:57:11 +04:00
Shenghou Ma
4019d0e424 runtime: avoid defining the same variable in more than one translation unit
For gccgo runtime and Darwin where -fno-common is the default.

R=iant, dave
CC=golang-dev
https://golang.org/cl/7094061
2013-01-26 09:57:06 +08:00
Dmitriy Vyukov
7847f328cd runtime: faster segmented stacks
benchmark                      old ns/op    new ns/op    delta
BenchmarkStackGrowth                 665          548  -17.59%
BenchmarkStackGrowth-2               333          274  -17.72%
BenchmarkStackGrowth-4               224          168  -25.00%
BenchmarkStackGrowth-8               124           91  -26.21%
BenchmarkStackGrowth-16               82           70  -14.55%
BenchmarkStackGrowth-32               73           59  -19.49%

R=nigeltao, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/7026044
2013-01-10 11:36:40 +04:00
Dmitriy Vyukov
f82db7d9e4 runtime: less aggressive per-thread stack segment caching
Introduce global stack segment cache and limit per-thread cache size.
This greatly reduces StackSys memory on workloads that create lots of threads.

benchmark                      old ns/op    new ns/op    delta
BenchmarkStackGrowth                 665          656   -1.35%
BenchmarkStackGrowth-2               333          328   -1.50%
BenchmarkStackGrowth-4               224          172  -23.21%
BenchmarkStackGrowth-8               124           91  -26.13%
BenchmarkStackGrowth-16               82           47  -41.94%
BenchmarkStackGrowth-32               73           40  -44.79%

BenchmarkStackGrowthDeep           97231        94391   -2.92%
BenchmarkStackGrowthDeep-2         47230        58562  +23.99%
BenchmarkStackGrowthDeep-4         24993        49356  +97.48%
BenchmarkStackGrowthDeep-8         15105        30072  +99.09%
BenchmarkStackGrowthDeep-16        10005        15623  +56.15%
BenchmarkStackGrowthDeep-32        12517        13069   +4.41%

TestStackMem#1,MB                  310          12       -96.13%
TestStackMem#2,MB                  296          14       -95.27%
TestStackMem#3,MB                  479          14       -97.08%

TestStackMem#1,sec                 3.22         2.26     -29.81%
TestStackMem#2,sec                 2.43         2.15     -11.52%
TestStackMem#3,sec                 2.50         2.38      -4.80%

R=sougou, no.smile.face, rsc
CC=golang-dev, msolomon
https://golang.org/cl/7029044
2013-01-10 09:57:06 +04:00
Jingcheng Zhang
70e967b7bc runtime: use "mp" and "gp" instead of "m" and "g" for local variable name to avoid confusion with the global "m" and "g".
R=golang-dev, minux.ma, rsc
CC=bradfitz, golang-dev
https://golang.org/cl/6939064
2012-12-19 00:30:29 +08:00
Jan Ziak
51b8edcb37 runtime: use reflect·call() to enter the function gc()
Garbage collection code (to be merged later) is calling functions
which have many local variables. This increases the probability that
the stack capacity won't be big enough to hold the local variables.
So, start gc() on a bigger stack to eliminate a potentially large number
of calls to runtime·morestack().

R=rsc, remyoudompheng, dsymonds, minux.ma, iant, iant
CC=golang-dev
https://golang.org/cl/6846044
2012-11-27 13:04:59 -05:00
Dmitriy Vyukov
320df44f04 runtime: switch to 64-bit goroutine ids
Fixes #4275.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6759053
2012-10-26 10:13:06 +04:00
Jan Ziak
4a191c2c1b runtime: store types of allocated objects
R=rsc
CC=golang-dev
https://golang.org/cl/6569057
2012-10-21 17:41:32 -04:00
Dmitriy Vyukov
2f6cbc74f1 race: runtime changes
This is a part of a bigger change that adds data race detection feature:
https://golang.org/cl/6456044

R=rsc
CC=gobot, golang-dev
https://golang.org/cl/6535050
2012-10-07 22:05:32 +04:00
Russ Cox
0b08c9483f runtime: prepare for 64-bit ints
This CL makes the runtime understand that the type of
the len or cap of a map, slice, or string is 'int', not 'int32',
and it is also careful to distinguish between function arguments
and results of type 'int' vs type 'int32'.

In the runtime, the new typedefs 'intgo' and 'uintgo' refer
to Go int and uint. The C types int and uint continue to be
unavailable (cause intentional compile errors).

This CL does not change the meaning of int, but it should make
the eventual change of the meaning of int on amd64 a bit
smoother.

Update #2188.

R=iant, r, dave, remyoudompheng
CC=golang-dev
https://golang.org/cl/6551067
2012-09-24 14:58:34 -04:00
Dmitriy Vyukov
f20fd87384 runtime: refactor goroutine blocking
The change is a preparation for the new scheduler.
It introduces runtime.park() function,
that will atomically unlock the mutex and park the goroutine.
It will allow to remove the racy readyonstop flag
that is difficult to implement w/o the global scheduler mutex.

R=rsc, remyoudompheng, dave
CC=golang-dev
https://golang.org/cl/6501077
2012-09-18 21:15:46 +04:00
Dmitriy Vyukov
a54f920bfe runtime: move panic/defer/recover-related stuff to a separate file
Move panic/defer/recover-related stuff from proc.c/runtime.c to a new file panic.c.
No semantic changes.
proc.c is 1800+ LOC and is a bit difficult to work with.

R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/6343071
2012-07-04 14:52:51 +04:00
Dmitriy Vyukov
a0c688331f runtime: refactor proc.c
1. Rename 'g' and 'm' local vars to 'gp' and 'mp' (convention already used in some functions)
'g' and 'm' are global vars that mean current goroutine and current machine,
when they are shadowed by local vars, it's confusing, no ability to debug log both, etc.
2. White-space shuffling.
No semantic changes.
In preparation to bigger changes.

R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/6355061
2012-07-03 12:54:13 +04:00
Jan Ziak
334bf95f9e runtime: update field types in preparation for GC changes
R=rsc, remyoudompheng, minux.ma, ality
CC=golang-dev
https://golang.org/cl/6242061
2012-05-30 13:07:52 -04:00
Alex Brainman
afe0e97aa6 runtime: handle windows exceptions, even in cgo programs
Fixes #3543.

R=golang-dev, kardianos, rsc
CC=golang-dev, hectorchu, vcc.163
https://golang.org/cl/6245063
2012-05-30 15:10:54 +10:00
Dmitriy Vyukov
01826280eb runtime: refactor helpgc functionality in preparation for parallel GC
Parallel GC needs to know in advance how many helper threads will be there.
Hopefully it's the last patch before I can tackle parallel sweep phase.
The benchmarks are unaffected.

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6200064
2012-05-15 19:10:16 +04:00
Maxim Pimenov
6727a2ad78 runtime: fix a comment
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5975043
2012-04-02 11:27:50 -04:00
Russ Cox
5eb007dede runtime: work around false negative in deadlock detection
Not a complete fix for issue 3342, but fixes the trivial case.
There may still be a race in the instants before and after
a scavenger-induced garbage collection.

Intended to be "obviously safe": a call to runtime·gosched
before main.main is no different than a call to runtime.Gosched
at the beginning of main.main, and it is (or had better be)
safe to call runtime.Gosched at any point during main.

Update #3342.

R=iant
CC=golang-dev
https://golang.org/cl/5919052
2012-03-27 12:22:19 -04:00
Mikio Hara
4c2614c57c undo CL 5844051 / 5d0322034aa8
Breaks closure test when GOMAXPROCS=2 or more.

««« original CL description
runtime: restore deadlock detection in the simplest case.

Fixes #3342.

R=iant, r, dave, rsc
CC=golang-dev, remy
https://golang.org/cl/5844051

»»»

R=rsc
CC=golang-dev
https://golang.org/cl/5924045
2012-03-27 13:05:17 +09:00
Rémy Oudompheng
84bb2547fb runtime: restore deadlock detection in the simplest case.
Fixes #3342.

R=iant, r, dave, rsc
CC=golang-dev, remy
https://golang.org/cl/5844051
2012-03-26 23:06:20 -04:00
Russ Cox
2e4a035995 runtime: do not handle signals before configuring handler
There was a small window during program initialization
where a signal could come in before the handling mechanisms
were set up to handle it.  Delay the signal-handler installation
until we're ready for the signals.

Fixes #3314.

R=golang-dev, dsymonds, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/5833049
2012-03-15 22:17:54 -04:00
Russ Cox
9e5db8c90a 5l, 6l, 8l: fix stack split logic for stacks near default segment size
Fixes #3310.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5823051
2012-03-15 15:22:30 -04:00
Rémy Oudompheng
aa1aaee7fd runtime: wait for main goroutine before setting GOMAXPROCS.
Fixes #3182.

R=golang-dev, dvyukov, rsc
CC=golang-dev, remy
https://golang.org/cl/5732057
2012-03-05 16:40:27 -05:00
Russ Cox
dc159fabff runtime: run init on main thread
Fixes #3125.

R=golang-dev, r, minux.ma
CC=golang-dev
https://golang.org/cl/5714049
2012-03-01 11:48:17 -05:00
Russ Cox
6e2ae0a12c runtime/pprof: support OS X CPU profiling
Work around profiling kernel bug with signal masks.
Still broken on 64-bit Snow Leopard kernel,
but I think we can ignore that one and let people
upgrade to Lion.

Add new trivial tools addr2line and objdump to take
the place of the GNU tools of the same name, since
those are not installed on OS X.

Adapt pprof to invoke 'go tool addr2line' and
'go tool objdump' if the system tools do not exist.

Clean up disassembly of base register on amd64.

Fixes #2008.

R=golang-dev, bradfitz, mikioh.mikioh, r, iant
CC=golang-dev
https://golang.org/cl/5697066
2012-02-28 16:18:24 -05:00
Russ Cox
e4b02bfdc0 runtime: goroutine profile, stack dumps
R=golang-dev, r, r
CC=golang-dev
https://golang.org/cl/5687076
2012-02-22 21:45:01 -05:00
Russ Cox
8a4c2b3cc4 runtime: fix another memory leak
morebuf holds a pc/sp from the last stack split or
reflect.call or panic/recover.  If the pc is a closure,
the reference will keep it from being collected.

moreargp holds a pointer to the arguments from the
last stack split or reflect.call or panic/recover.
Normally it is a stack pointer and thus not of interest,
but in the case of reflect.call it is an allocated argument
list and holds up the arguments to the call.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5674109
2012-02-19 11:05:19 -05:00
Russ Cox
89b075cc90 runtime: fix tiny memory leak
The m->cret word holds the C return value when returning
across a stack split boundary.  It was not being cleared after
use, which means that the return value (if a C function)
or else the value of AX/R0 at the time of the last stack unsplit
was being kept alive longer than necessary.  Clear it.

I think the effect here should be very small, but worth fixing
anyway.

R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5677092
2012-02-19 00:26:33 -05:00
David Symonds
4b171e5040 runtime: rename Cgocalls and Goroutines to NumCgoCall and NumGoroutine, respectively.
Update some other docs too.

Update #2955.

R=rsc
CC=golang-dev
https://golang.org/cl/5676060
2012-02-17 08:49:41 +11:00
Sébastien Paolacci
5c598d3c9f runtime: release unused memory to the OS.
Periodically browse MHeap's freelists for long unused spans and release them if any.

Current hardcoded settings:
        - GC is forced if none occured over the last 2 minutes.
        - spans are handed back after 5 minutes of uselessness.

SysUnused (for Unix) is a wrapper on madvise MADV_DONTNEED on Linux and MADV_FREE on BSDs.

R=rsc, dvyukov, remyoudompheng
CC=golang-dev
https://golang.org/cl/5451057
2012-02-16 13:30:04 -05:00