Otherwise the tests in pkg/runtime fail:
runtime: unknown argument frame size for runtime.deferreturn called from 0x48657b [runtime_test.func·022]
fatal error: invalid stack
...
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11483043
Update #543
I believe the runtime is strong enough now to reenable
preemption during the function prologue.
Assuming this is or can be made stable, it will be in Go 1.2.
More aggressive preemption is not planned for Go 1.2.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11433045
Currently preemption signal g->stackguard0==StackPreempt
can be lost if it is received when preemption is disabled
(e.g. m->lock!=0). This change duplicates the preemption
signal in g->preempt and restores g->stackguard0
when preemption is enabled.
Update #543.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10792043
With this CL, I believe the runtime always knows
the frame size during the gc walk. There is no fallback
to "assume entire stack frame of caller" anymore.
R=golang-dev, khr, cshapiro, dvyukov
CC=golang-dev
https://golang.org/cl/11374044
I have not done the system call stubs in sys_*.s.
I hope to avoid that, because those do not block, so those
frames will not appear in stack traces during garbage
collection.
R=golang-dev, dvyukov, khr
CC=golang-dev
https://golang.org/cl/11360043
Design at http://golang.org/s/go12symtab.
This enables some cleanup of the garbage collector metadata
that will be done in future CLs.
This CL does not move the old symtab and pclntab back into
an unmapped section of the file. That's a bit tricky and will be
done separately.
Fixes#4020.
R=golang-dev, dave, cshapiro, iant, r
CC=golang-dev, nigeltao
https://golang.org/cl/11085043
A type switch on a value with map index expressions,
could get a spurious instrumentation from a OTYPESW node.
These nodes do not need instrumentation because after
walk the type switch has been turned into a sequence
of ifs.
Fixes#5890.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11308043
If the stack frame size is larger than the known-unmapped region at the
bottom of the address space, then the stack split prologue cannot use the usual
condition:
SP - size >= stackguard
because SP - size may wrap around to a very large number.
Instead, if the stack frame is large, the prologue tests:
SP - stackguard >= size
(This ends up being a few instructions more expensive, so we don't do it always.)
Preemption requests register by setting stackguard to a very large value, so
that the first test (SP - size >= stackguard) cannot possibly succeed.
Unfortunately, that same very large value causes a wraparound in the
second test (SP - stackguard >= size), making it succeed incorrectly.
To avoid *that* wraparound, we have to amend the test:
stackguard != StackPreempt && SP - stackguard >= size
This test is only used for functions with large frames, which essentially
always split the stack, so the cost of the few instructions is noise.
This CL and CL 11085043 together fix the known issues with preemption,
at the beginning of a function, so we will be able to try turning it on again.
R=ken2
CC=golang-dev
https://golang.org/cl/11205043
The current cas64 definition hard-codes the x86 behavior
of updating *old with the new value when the cas fails.
This is inconsistent with cas32 and casp.
Make it consistent.
This means that the cas64 uses will be epsilon less efficient
than they might be, because they have to do an unnecessary
memory load on x86. But so be it. Code clarity and consistency
is more important.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10909045
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
This is reincarnation of cl/9776044 with the bug fixed.
The bug was due to code added after cl/9776044 was created:
if(tick - (((uint64)tick*0x4325c53fu)>>36)*61 == 0 && runtime·sched.runqsize > 0) {
runtime·lock(&runtime·sched);
gp = globrunqget(m->p, 1);
runtime·unlock(&runtime·sched);
}
If M gets gp from global runq here, it does not reset m->spinning.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10743044
Currently it crashes as follows:
fatal error: unknown pc
...
goroutine 71698 [runnable]:
runtime.racegoend()
src/pkg/runtime/race.c:171
runtime.goexit()
src/pkg/runtime/proc.c:1276 +0x9
created by runtime_test.testConcurrentReadsAfterGrowth
src/pkg/runtime/map_test.go:264 +0x332
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10674047
using m->tls[0] to save ucontext pointer is not re-entry safe, and
the old code didn't set it before the early return when signal is
received on non-Go threads.
so misc/cgo/test used to hang when testing issue 5337.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/10076045
When deleting a timer, a panic due to nil deref
would leave a lock held, possibly leading to a deadlock
in a defer. Instead return false on a nil timer.
Fixes#5745.
R=golang-dev, daniel.morsing, dvyukov, rsc, iant
CC=golang-dev
https://golang.org/cl/10373047
There are various problems, and both Dmitriy and I
will be away for the next week. Make the runtime a bit
more stable while we're gone.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10848043
fn can clearly hold a closure in memory.
argp/pc point into stack and so can hold
in memory a block that was previously
a large stack serment.
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/10784043
On amd64 the frames are very close to the limit for a
nosplit (textflag 7) function, in part because the C compiler
does not make any attempt to reclaim space allocated for
completely registerized variables. Avoid a few short-lived
variables to reclaim two words.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10758043
Currently it replaces GOGCTRACE env var (GODEBUG=gctrace=1).
The plan is to extend it with other type of debug tracing,
e.g. GODEBUG=gctrace=1,schedtrace=100.
R=rsc
CC=bradfitz, daniel.morsing, gobot, golang-dev
https://golang.org/cl/10026045
The last patch for preemptive scheduler,
with this change stoptheworld issues preemption
requests every 100us.
Update #543.
R=golang-dev, daniel.morsing, rsc
CC=golang-dev
https://golang.org/cl/10264044
On x86 it is a few words lower on the stack than m->morebuf.sp
so it is a more precise check. Enabling the check requires recording
a valid gp->sched in reflect.call too. This is a good thing in general,
since it will make stack traces during reflect.call work better, and it
may be useful for preemption too.
R=dvyukov
CC=golang-dev
https://golang.org/cl/10709043
runtime.entersyscall() sets g->status = Gsyscall,
then calls runtime.lock() which causes stack split.
runtime.newstack() resets g->status to Grunning.
This will lead to crash during GC (world is not stopped) or GC will scan stack incorrectly.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10696043
Failure on bot:
http://build.golang.org/log/f4c648906e1289ec2237c1d0880fb1a8b1852a08
««« original CL description
runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
R=rsc
TBR=rsc
CC=gobot, golang-dev
https://golang.org/cl/9776044
»»»
R=golang-dev
CC=golang-dev
https://golang.org/cl/10692043
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
R=rsc
CC=gobot, golang-dev
https://golang.org/cl/9776044
Current code can print more arguments than necessary
and also incorrectly prints "...".
Update #5723.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10689043
Until now, the goroutine state has been scattered during the
execution of newstack and oldstack. It's all there, and those routines
know how to get back to a working goroutine, but other pieces of
the system, like stack traces, do not. If something does interrupt
the newstack or oldstack execution, the rest of the system can't
understand the goroutine. For example, if newstack decides there
is an overflow and calls throw, the stack tracer wouldn't dump the
goroutine correctly.
For newstack to save a useful state snapshot, it needs to be able
to rewind the PC in the function that triggered the split back to
the beginning of the function. (The PC is a few instructions in, just
after the call to morestack.) To make that possible, we change the
prologues to insert a jmp back to the beginning of the function
after the call to morestack. That is, the prologue used to be roughly:
TEXT myfunc
check for split
jmpcond nosplit
call morestack
nosplit:
sub $xxx, sp
Now an extra instruction is inserted after the call:
TEXT myfunc
start:
check for split
jmpcond nosplit
call morestack
jmp start
nosplit:
sub $xxx, sp
The jmp is not executed directly. It is decoded and simulated by
runtime.rewindmorestack to discover the beginning of the function,
and then the call to morestack returns directly to the start label
instead of to the jump instruction. So logically the jmp is still
executed, just not by the cpu.
The prologue thus repeats in the case of a function that needs a
stack split, but against the cost of the split itself, the extra few
instructions are noise. The repeated prologue has the nice effect of
making a stack split double-check that the new stack is big enough:
if morestack happens to return on a too-small stack, we'll now notice
before corruption happens.
The ability for newstack to rewind to the beginning of the function
should help preemption too. If newstack decides that it was called
for preemption instead of a stack split, it now has the goroutine state
correctly paused if rescheduling is needed, and when the goroutine
can run again, it can return to the start label on its original stack
and re-execute the split check.
Here is an example of a split stack overflow showing the full
trace, without any special cases in the stack printer.
(This one was triggered by making the split check incorrect.)
runtime: newstack framesize=0x0 argsize=0x18 sp=0x6aebd0 stack=[0x6b0000, 0x6b0fa0]
morebuf={pc:0x69f5b sp:0x6aebd8 lr:0x0}
sched={pc:0x68880 sp:0x6aebd0 lr:0x0 ctxt:0x34e700}
runtime: split stack overflow: 0x6aebd0 < 0x6b0000
fatal error: runtime: split stack overflow
goroutine 1 [stack split]:
runtime.mallocgc(0x290, 0x100000000, 0x1)
/Users/rsc/g/go/src/pkg/runtime/zmalloc_darwin_amd64.c:21 fp=0x6aebd8
runtime.new()
/Users/rsc/g/go/src/pkg/runtime/zmalloc_darwin_amd64.c:682 +0x5b fp=0x6aec08
go/build.(*Context).Import(0x5ae340, 0xc210030c71, 0xa, 0xc2100b4380, 0x1b, ...)
/Users/rsc/g/go/src/pkg/go/build/build.go:424 +0x3a fp=0x6b00a0
main.loadImport(0xc210030c71, 0xa, 0xc2100b4380, 0x1b, 0xc2100b42c0, ...)
/Users/rsc/g/go/src/cmd/go/pkg.go:249 +0x371 fp=0x6b01a8
main.(*Package).load(0xc21017c800, 0xc2100b42c0, 0xc2101828c0, 0x0, 0x0, ...)
/Users/rsc/g/go/src/cmd/go/pkg.go:431 +0x2801 fp=0x6b0c98
main.loadPackage(0x369040, 0x7, 0xc2100b42c0, 0x0)
/Users/rsc/g/go/src/cmd/go/pkg.go:709 +0x857 fp=0x6b0f80
----- stack segment boundary -----
main.(*builder).action(0xc2100902a0, 0x0, 0x0, 0xc2100e6c00, 0xc2100e5750, ...)
/Users/rsc/g/go/src/cmd/go/build.go:539 +0x437 fp=0x6b14a0
main.(*builder).action(0xc2100902a0, 0x0, 0x0, 0xc21015b400, 0x2, ...)
/Users/rsc/g/go/src/cmd/go/build.go:528 +0x1d2 fp=0x6b1658
main.(*builder).test(0xc2100902a0, 0xc210092000, 0x0, 0x0, 0xc21008ff60, ...)
/Users/rsc/g/go/src/cmd/go/test.go:622 +0x1b53 fp=0x6b1f68
----- stack segment boundary -----
main.runTest(0x5a6b20, 0xc21000a020, 0x2, 0x2)
/Users/rsc/g/go/src/cmd/go/test.go:366 +0xd09 fp=0x6a5cf0
main.main()
/Users/rsc/g/go/src/cmd/go/main.go:161 +0x4f9 fp=0x6a5f78
runtime.main()
/Users/rsc/g/go/src/pkg/runtime/proc.c:183 +0x92 fp=0x6a5fa0
runtime.goexit()
/Users/rsc/g/go/src/pkg/runtime/proc.c:1266 fp=0x6a5fa8
And here is a seg fault during oldstack:
SIGSEGV: segmentation violation
PC=0x1b2a6
runtime.oldstack()
/Users/rsc/g/go/src/pkg/runtime/stack.c:159 +0x76
runtime.lessstack()
/Users/rsc/g/go/src/pkg/runtime/asm_amd64.s:270 +0x22
goroutine 1 [stack unsplit]:
fmt.(*pp).printArg(0x2102e64e0, 0xe5c80, 0x2102c9220, 0x73, 0x0, ...)
/Users/rsc/g/go/src/pkg/fmt/print.go:818 +0x3d3 fp=0x221031e6f8
fmt.(*pp).doPrintf(0x2102e64e0, 0x12fb20, 0x2, 0x221031eb98, 0x1, ...)
/Users/rsc/g/go/src/pkg/fmt/print.go:1183 +0x15cb fp=0x221031eaf0
fmt.Sprintf(0x12fb20, 0x2, 0x221031eb98, 0x1, 0x1, ...)
/Users/rsc/g/go/src/pkg/fmt/print.go:234 +0x67 fp=0x221031eb40
flag.(*stringValue).String(0x2102c9210, 0x1, 0x0)
/Users/rsc/g/go/src/pkg/flag/flag.go:180 +0xb3 fp=0x221031ebb0
flag.(*FlagSet).Var(0x2102f6000, 0x293d38, 0x2102c9210, 0x143490, 0xa, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:633 +0x40 fp=0x221031eca0
flag.(*FlagSet).StringVar(0x2102f6000, 0x2102c9210, 0x143490, 0xa, 0x12fa60, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:550 +0x91 fp=0x221031ece8
flag.(*FlagSet).String(0x2102f6000, 0x143490, 0xa, 0x12fa60, 0x0, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:563 +0x87 fp=0x221031ed38
flag.String(0x143490, 0xa, 0x12fa60, 0x0, 0x161950, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:570 +0x6b fp=0x221031ed80
testing.init()
/Users/rsc/g/go/src/pkg/testing/testing.go:-531 +0xbb fp=0x221031edc0
strings_test.init()
/Users/rsc/g/go/src/pkg/strings/strings_test.go:1115 +0x62 fp=0x221031ef70
main.init()
strings/_test/_testmain.go:90 +0x3d fp=0x221031ef78
runtime.main()
/Users/rsc/g/go/src/pkg/runtime/proc.c:180 +0x8a fp=0x221031efa0
runtime.goexit()
/Users/rsc/g/go/src/pkg/runtime/proc.c:1269 fp=0x221031efa8
goroutine 2 [runnable]:
runtime.MHeap_Scavenger()
/Users/rsc/g/go/src/pkg/runtime/mheap.c:438
runtime.goexit()
/Users/rsc/g/go/src/pkg/runtime/proc.c:1269
created by runtime.main
/Users/rsc/g/go/src/pkg/runtime/proc.c:166
rax 0x23ccc0
rbx 0x23ccc0
rcx 0x0
rdx 0x38
rdi 0x2102c0170
rsi 0x221032cfe0
rbp 0x221032cfa0
rsp 0x7fff5fbff5b0
r8 0x2102c0120
r9 0x221032cfa0
r10 0x221032c000
r11 0x104ce8
r12 0xe5c80
r13 0x1be82baac718
r14 0x13091135f7d69200
r15 0x0
rip 0x1b2a6
rflags 0x10246
cs 0x2b
fs 0x0
gs 0x0
Fixes#5723.
R=r, dvyukov, go.peter.90, dave, iant
CC=golang-dev
https://golang.org/cl/10360048
Resubmit 3c2cddfbdaec now that windows callbacks
are not generated during runtime.
Fixes#5494
R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/10487043
- change runtime_pollWait so it does not return
closed or timeout if IO is ready - windows must
know if IO has completed or not even after
interruption;
- add (*pollDesc).Prepare(mode int) that can be
used for both read and write, same for Wait;
- introduce runtime_pollWaitCanceled and expose
it in net as (*pollDesc).WaitCanceled(mode int);
Full windows netpoll changes are
here https://golang.org/cl/8670044/.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/10485043
until we decide what to do with issues 5659/5736.
Profiling with race detector is not very useful in general,
and now it makes race builders red.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10523043
If first GC runs concurrently with setGCPercent,
it can overwrite gcpercent value with default.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/10242047
Currently global runqueue is starved if a group of goroutines
constantly respawn each other (local runqueue never becomes empty).
Fixes#5639.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/10042044
It was used to request large stack segment for GC
when it was running not on g0.
Now GC is running on g0 with large stack,
and it is not needed anymore.
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/10242045
No need to change to Grunnable state.
Add some more checks for Grunning state.
R=golang-dev, rsc, khr, dvyukov
CC=golang-dev
https://golang.org/cl/10186045
The previous implementation would only record access to
the address of the array but the memory access to the whole
memory range must be recorded instead.
R=golang-dev, dvyukov, r
CC=golang-dev
https://golang.org/cl/8053044
Instrumentation of ntest expression should go to ntest->init.
Same for nincr.
Fixes#5340.
R=golang-dev, daniel.morsing
CC=golang-dev
https://golang.org/cl/10026046
Add gostartcall and gostartcallfn.
The old gogocall = gostartcall + gogo.
The old gogocallfn = gostartcallfn + gogo.
R=dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/10036044
In starttheworld() we assume that P's with local work
are situated in the beginning of idle P list.
However, once we start the first M, it can execute all local G's
and steal G's from other P's.
That breaks the assumption above. Thus starttheworld() will fail
to start some P's with local work.
It seems that it can not lead to very bad things, but still
it's wrong and breaks other assumtions
(e.g. we can have a spinning M with local work).
The fix is to collect all P's with local work first,
and only then start them.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10051045
The garbage collection routine addframeroots is duplicating
logic in the traceback routine that calls it, sometimes correctly,
sometimes incorrectly, sometimes incompletely.
Pass necessary information to addframeroots instead of
deriving it anew.
Should make addframeroots significantly more robust.
It's certainly smaller.
Also try to standardize on uintptr for saved pc, sp values.
Will make CL 10036044 trivial.
R=golang-dev, dave, dvyukov
CC=golang-dev
https://golang.org/cl/10169045
There's no reason to use a different name on each architecture,
and doing so makes it impossible for portable code to refer to
the original Go runtime entry point. Rename it _rt0_go everywhere.
This is a global search and replace only.
R=golang-dev, bradfitz, minux.ma
CC=golang-dev
https://golang.org/cl/10196043
Do not synchronize Add(1) with Wait().
Imitate read on first Add(1) and write on Wait(),
it allows to catch common misuses of WaitGroup:
- Add() called in the additional goroutine itself
- incorrect reuse of WaitGroup with multiple waiters
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/10093044
Also reduce FixAlloc allocation granulatiry from 128k to 16k,
small programs do not need that much memory for MCache's and MSpan's.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/10140044
Especially important for Windows because it reserves VM
only in multiple of 64k.
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/10082048
Count only number of frees, everything else is derivable
and does not need to be counted on every malloc.
benchmark old ns/op new ns/op delta
BenchmarkMalloc8 68 66 -3.07%
BenchmarkMalloc16 75 70 -6.48%
BenchmarkMallocTypeInfo8 102 97 -4.80%
BenchmarkMallocTypeInfo16 108 105 -2.78%
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/9776043
Remove unnecessary ( ) around == in && clause.
Add { } around multiline if body, even though it's one statement.
Add runtime: prefix to printed errors.
R=cshapiro, iant
CC=golang-dev
https://golang.org/cl/9685047
This is part of preemptive scheduler.
stackguard0 is checked in split stack checks and can be set to StackPreempt.
stackguard is not set to StackPreempt (holds the original value).
R=golang-dev, daniel.morsing, iant
CC=golang-dev
https://golang.org/cl/9875043
Before this change, grow work was done only
during map writes to ensure multithreaded safety.
This can lead to maps remaining in a partially
grown state for a long time, potentially forever.
This change allows grow work to happen during reads,
which will lead to grow work finishing sooner, making
the resulting map smaller and faster.
Grow work is not done in parallel. Reads can
happen in parallel while grow work is happening.
R=golang-dev, dvyukov, khr, iant
CC=golang-dev
https://golang.org/cl/8852047
instead of regular g stack. We do this so that the g stack
we're currently running on is no longer changing. Cuts
the root set down a bit (g0 stacks are not scanned, and
we don't need to scan gc's internal state). Also an
enabler for copyable stacks.
R=golang-dev, cshapiro, khr, 0xe2.0x9a.0x9b, dvyukov, rsc, iant
CC=golang-dev
https://golang.org/cl/9754044
mheap.map become a pointer, so nelem(h->map) returns 1 rather than the map size.
As the result coalescing with subsequent spans does not happen.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9649046
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
Reincarnation of committed and rolled back https://golang.org/cl/9805043
The latent bugs that it revealed are fixed:
https://golang.org/cl/9837049https://golang.org/cl/9778048
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9778049
Then use the limit to make sure MHeap_LookupMaybe & inlined
copies don't return a span if the pointer is beyond the limit.
Use this fact to optimize all call sites.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/9869045
A nosplits was assumed to have no argument information and no
pointer map. However, nosplits created by the linker often
have both. This change uses the pointer map size as an
alternate source of argument size when processing a nosplit.
In addition, the symbol table construction pointer map size
and argument size consistency check is strengthened. If a
nptrs is greater than 0 it must be equal to the number of
argument words.
R=golang-dev, khr, khr
CC=golang-dev
https://golang.org/cl/9666047
to avoid unintentionally clobber R9/R10.
Thanks Lucio for the suggestion.
PS: yes, this could be considered a big change (but not an API change), but
as it turns out even temporarily changes R9/R10 in user code is unsafe and
leads to very hard to diagnose problems later, better to disable using R9/R10
when the user first uses it.
See CL 6300043 and CL 6305100 for two problems caused by misusing R9/R10.
R=golang-dev, khr, rsc
CC=golang-dev
https://golang.org/cl/9840043
This is needed for preemptive scheduler, because during
stoptheworld we want to wait with timeout and re-preempt
M's on timeout.
R=golang-dev, remyoudompheng, iant
CC=golang-dev
https://golang.org/cl/9375043
With this change the compiler emits a bitmap for each function
covering its stack frame arguments area. If an argument word
is known to contain a pointer, a bit is set. The garbage
collector reads this information when scanning the stack by
frames and uses it to ignores locations known to not contain a
pointer.
R=golang-dev, bradfitz, daniel.morsing, dvyukov, khr, khr, iant, cshapiro
CC=golang-dev
https://golang.org/cl/9223046
This depends on: 9791044: runtime: allocate page table lazily
Once page table is moved out of heap, the heap becomes small.
This removes unnecessary dereferences during heap access.
No logical changes.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9802043
This removes the 256MB memory allocation at startup,
which conflicts with ulimit.
Also will allow to eliminate an unnecessary memory dereference in GC,
because the page table is usually mapped at known address.
Update #5049.
Update #5236.
R=golang-dev, khr, r, khr, rsc
CC=golang-dev
https://golang.org/cl/9791044
The 'n' variable is used during rescan initiation in GC_END case,
but it's overwritten with chan capacity in GC_CHAN case.
As the result rescan is done with the wrong object size.
Fixes#5554.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9831043
multiple failures on amd64
««« original CL description
runtime: introduce helper persistentalloc() function
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
R=golang-dev, daniel.morsing, khr
CC=golang-dev
https://golang.org/cl/9805043
»»»
R=golang-dev
CC=golang-dev
https://golang.org/cl/9822043
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
R=golang-dev, daniel.morsing, khr
CC=golang-dev
https://golang.org/cl/9805043
Variables in data sections of 32-bit executables interfere with
garbage collector's ability to free objects and/or unnecessarily
slow down the garbage collector.
This changeset moves some static variables to .noptr sections.
'files' in symtab.c is now allocated dynamically.
R=golang-dev, dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/9786044
This is needed for preemptive scheduler, because the goroutine
can be preempted at surprising points.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/9376043
When cgo is used, runtime creates an additional M to handle callbacks on threads not created by Go.
This effectively disabled deadlock detection, which is a right thing, because Go program can be blocked
and only serve callbacks on external threads.
This also disables deadlock detection under race detector, because it happens to use cgo.
With this change the additional M is created lazily on first cgo call. So deadlock detector
works for programs that import "C", "net" or "net/http/pprof" but do not use them in fact.
Also fixes deadlock detector under race detector.
It should be fine to create the M later, because C code can not call into Go before first cgo call,
because C code does not know when Go initialization has completed. So a Go program need to call into C
first either to create an external thread, or notify a thread created in global ctor that Go
initialization has completed.
Fixes#4973.
Fixes#5475.
R=golang-dev, minux.ma, iant
CC=golang-dev
https://golang.org/cl/9303046
Currently per-sizeclass stats are lost for destroyed MCache's. This patch fixes this.
Also, only update mstats.heap_alloc on heap operations, because that's the only
stat that needs to be promptly updated. Everything else needs to be up-to-date only in ReadMemStats().
R=golang-dev, remyoudompheng, dave, iant
CC=golang-dev
https://golang.org/cl/9207047
The nlistmin/size thresholds are copied from tcmalloc,
but are unnecesary for Go malloc. We do not do explicit
frees into MCache. For sparse cases when we do (mainly hashmap),
simpler logic will do.
R=rsc, dave, iant
CC=gobot, golang-dev, r, remyoudompheng
https://golang.org/cl/9373043
It contains the LHS of the range clause and gets
instrumented by racewalk, but it doesn't have any meaning.
Fixes#5446.
R=golang-dev, dvyukov, daniel.morsing, r
CC=golang-dev
https://golang.org/cl/9560044
The stack scanner for not started goroutines ignored the arguments
area when its size was unknown. With this change, the distance
between the stack pointer and the stack base will be used instead.
Fixes#5486
R=golang-dev, bradfitz, iant, dvyukov
CC=golang-dev
https://golang.org/cl/9440043
If a slice points to an array embedded in a struct,
the whole struct can be incorrectly scanned as the slice buffer.
Fixes#5443.
R=cshapiro, iant, r, cshapiro, minux.ma
CC=bradfitz, gobot, golang-dev
https://golang.org/cl/9372044
Allocs of size 16 can bypass atomic set of the allocated bit, while allocs of size 8 can not.
Allocs with and w/o type info hit different paths inside of malloc.
Current results on linux/amd64:
BenchmarkMalloc8 50000000 43.6 ns/op
BenchmarkMalloc16 50000000 46.7 ns/op
BenchmarkMallocTypeInfo8 50000000 61.3 ns/op
BenchmarkMallocTypeInfo16 50000000 63.5 ns/op
R=golang-dev, remyoudompheng, minux.ma, bradfitz, iant
CC=golang-dev
https://golang.org/cl/9090045
for checking for page boundary. Also avoid boundary check
when >=16 bytes are hashed.
benchmark old ns/op new ns/op delta
BenchmarkHashStringSpeed 23 22 -0.43%
BenchmarkHashBytesSpeed 44 42 -3.61%
BenchmarkHashStringArraySpeed 71 68 -4.05%
R=iant, khr
CC=gobot, golang-dev, google
https://golang.org/cl/9123046
Finer-grained transfers were relevant with per-M caches,
with per-P caches they are not relevant and harmful for performance.
For few small size classes where it makes difference,
it's fine to grab the whole span (4K).
benchmark old ns/op new ns/op delta
BenchmarkMalloc 42 40 -4.45%
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/9374043
This is needed for preemptive scheduler,
it will preempt only when m->locks==0,
and we do not want to be preempted while
we have not completely unlocked the lock.
R=golang-dev, khr, iant
CC=golang-dev
https://golang.org/cl/9196047
Also change table type from int32[] to int8[] to save space in L1$.
benchmark old ns/op new ns/op delta
BenchmarkMalloc 42 40 -4.68%
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/9199044
Move the documentation from race.go to doc.go, because
race.go uses +build race, so it's not normally parsed by go doc.
Rephrase the documentation for end users, provide link to race
detector manual.
Fixes#5444.
R=golang-dev, minux.ma, adg, r
CC=golang-dev
https://golang.org/cl/9144050
runtime.park() can access freed select descriptor
due to a racing free in another thread.
See the comment for details.
Slightly modified version of dvyukov's CL 9259045.
No test yet. Before this CL, the test described in issue 5422
would fail about every 40 times for me. With this CL, I ran
the test 5900 times with no failures.
Fixes#5422.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9311043
The linker can generate split stack prolog when a textflag 7 function
makes an indirect function call. If it happens, badsignal() crashes
trying to dereference g.
Fixes#5337.
R=bradfitz, dave, adg, iant, r, minux.ma
CC=adonovan, golang-dev
https://golang.org/cl/9226043
runtime.setmg() calls another function (cgo_save_gm), so it must save
LR onto stack.
Re-enabled TestCthread test in misc/cgo/test.
Fixes#4863.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9019043
It works on i386, but fails on amd64 and arm.
««« original CL description
runtime: prevent the GC from seeing the content of a frame in runfinq()
Fixes#5348.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/8954044
»»»
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8695051
This will let us ask people to rebuild the Go system without
precise GC, and then rebuild and retest their program, to see
if precise GC is causing whatever problem they are having.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8700043
UMTX_OP_WAIT expects that the address points to a uintptr, but
the code in lock_futex.c uses a uint32. UMTX_OP_WAIT_UINT is
just like UMTX_OP_WAIT, but the address points to a uint32.
This almost certainly makes no difference on a little-endian
system, but since the kernel supports it we should do the
right thing. And, who knows, maybe it matters.
R=golang-dev, bradfitz, r, ality
CC=golang-dev
https://golang.org/cl/8699043
The race detector uses a global lock to analyze atomic
operations. A panic in the middle of the code leaves the
lock acquired.
Similarly, the sync package may leave the race detectro
inconsistent when methods are called on nil pointers.
R=golang-dev, r, minux.ma, dvyukov, rsc, adg
CC=golang-dev
https://golang.org/cl/7981043
It's not trivial to make a comprehensive check
due to inferior pointers, reflect, gob, etc.
But this is essentially what I've used to debug
the GC issues.
Update #5193.
R=golang-dev, iant, 0xe2.0x9a.0x9b, r
CC=golang-dev
https://golang.org/cl/8455043
Use atomic operations on flags field to make sure we aren't
losing a flag update during parallel map operations.
R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/8377046
The invariant is that there must be at least one running P or a thread polling network.
It was broken.
Fixes#5216.
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/8459043
This makes it an unsafe.Pointer in Go so the garbage collector
will treat it as a pointer to untyped data, not a pointer to
bytes.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/8286045
If for whatever reason seh points into Go heap region,
the dangling pointer will cause memory corruption during GC.
Update #5193.
R=golang-dev, alex.brainman, iant
CC=golang-dev
https://golang.org/cl/8402045
Fixes#5175.
Race detector runtime expects values passed to MapShadow() to be page-aligned,
because they are used in mmap() call. If they are not aligned mmap() trims
either beginning or end of the mapping.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8325043
This changes the map lookup behavior for string maps with 2-8 keys.
There was already previously a fastpath for 0 items and 1 item.
Now, if a string-keyed map has <= 8 items, first check all the
keys for length first. If only one has the right length, then
just check it for equality and avoid hashing altogether. Once
the map has more than 8 items, always hash like normal.
I don't know why some of the other non-string map benchmarks
got faster. This was with benchtime=2s, multiple times. I haven't
anything else getting slower, though.
benchmark old ns/op new ns/op delta
BenchmarkHashStringSpeed 37 34 -8.20%
BenchmarkHashInt32Speed 32 29 -10.67%
BenchmarkHashInt64Speed 31 27 -12.82%
BenchmarkHashStringArraySpeed 105 99 -5.43%
BenchmarkMegMap 274206 255153 -6.95%
BenchmarkMegOneMap 27 23 -14.80%
BenchmarkMegEqMap 148332 116089 -21.74%
BenchmarkMegEmptyMap 4 3 -12.72%
BenchmarkSmallStrMap 22 22 -0.89%
BenchmarkMapStringKeysEight_32 42 23 -43.71%
BenchmarkMapStringKeysEight_64 55 23 -56.96%
BenchmarkMapStringKeysEight_1M 279688 24 -99.99%
BenchmarkIntMap 16 15 -10.18%
BenchmarkRepeatedLookupStrMapKey32 40 37 -8.15%
BenchmarkRepeatedLookupStrMapKey1M 287918 272980 -5.19%
BenchmarkNewEmptyMap 156 130 -16.67%
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/7641057
The expected precision setting for the x87 on Win32 is 53-bit
but MinGW resets the floating point unit to 64-bit. Win32
object code generally expects values to be rounded to double,
not double extended, precision.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/8175044
Doing grow work on reads is not multithreaded safe.
Changed code to do grow work only on inserts & deletes.
This is a short-term fix, eventually we'll want to do
grow work in parallel to recover the space of the old
table.
Fixes#5120.
R=bradfitz, khr
CC=golang-dev
https://golang.org/cl/8242043
Motivated by garbage profiling in HTTP benchmarks. This
changes means new empty maps are just one small allocation
(the HMap) instead the HMap + the relatively larger h->buckets
allocation. This helps maps which remain empty throughout
their life.
benchmark old ns/op new ns/op delta
BenchmarkNewEmptyMap 196 107 -45.41%
benchmark old allocs new allocs delta
BenchmarkNewEmptyMap 2 1 -50.00%
benchmark old bytes new bytes delta
BenchmarkNewEmptyMap 195 50 -74.36%
R=khr, golang-dev, r
CC=golang-dev
https://golang.org/cl/7722046
A HMUL node appears in some constant divisions, but
to observe a false negative in race detector the divisor must be
suitably chosen to make sure the only memory access is
done for HMUL.
R=dvyukov
CC=golang-dev
https://golang.org/cl/7935045
For Go 1.1, stop checking the rlimit, because it broke now
that mheap is allocated using SysAlloc. See issue 5049.
R=r
CC=golang-dev
https://golang.org/cl/7741050
The arm gentraceback mishandled frame linkage values pointing
to the assembly return function. This function is special as
its frame size is zero and it contains only one instruction.
These conditions would preserve the frame pointer and result
in an off by one error when unwinding the caller.
Fixes#5124
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/8023043
Prevents storm of error messages if something goes wrong.
In the case of issue 5073 the epoll fd was closed by the test.
Update #5073.
R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/7966043
Handle interface comparison correctly,
add a few more tests, mark more nodes as impossible.
R=dvyukov, golang-dev
CC=golang-dev
https://golang.org/cl/7942045
This keeps the logic about how to set the thread-local variables
m and g in code compiled and linked by the gc toolchain,
an important property for upcoming cgo changes.
It's also just a nice cleanup: one less place to update when
these details change.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7560048
The right operand of a && and || is only executed conditionnally,
so the instrumentation must be more careful. In particular
it should not turn nodes assumed to be cheap after walk into
expensive ones.
Update #4228
R=dvyukov, golang-dev
CC=golang-dev
https://golang.org/cl/7986043
The ARM implementation of runtime.cgocallback_gofunc diverged
from the calling convention by leaving a word of garbage at
the top of the stack and storing the return PC above the
locals. This change stores the return PC at the top of the
stack and removes the save area above the locals.
Update #5124
This CL fixes first part of the ARM issues and added the unwind test.
R=golang-dev, bradfitz, minux.ma, cshapiro, rsc
CC=golang-dev
https://golang.org/cl/7728045
Adds the new debugging constant 'checkgc'. If its value is non-zero
all calls to mallocgc() from hashmap.c will start a garbage collection.
Fixes#5074.
R=golang-dev, khr
CC=golang-dev, rsc
https://golang.org/cl/7663051
Fixes performance of the current windows network poller
with the new scheduler.
Gives runtime a hint when GetQueuedCompletionStatus() will block.
Fixes#5068.
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 4004000 33906 -99.15%
BenchmarkTCP4Persistent-2 21790 17513 -19.63%
BenchmarkTCP4Persistent-4 44760 34270 -23.44%
BenchmarkTCP4Persistent-6 45280 43000 -5.04%
R=golang-dev, alex.brainman, coocood, rsc
CC=golang-dev
https://golang.org/cl/7612045
I'm not sure how to write a test for this. The change in
behaviour is that if you somehow get a SIGBUS signal for an
address >= 0x1000, the program will now crash rather than
calling panic. As far as I know, on x86 GNU/Linux, the only
way to get a SIGBUS (rather than a SIGSEGV) is to set the
stack pointer to an invalid value.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7906045
On Darwin and FreeBSD, the mmap syscall return value is returned
unmodified. This means that the return value will either be a
valid address or a positive error number.
Also check return value from mmap in SysReserve - the callers of
SysReserve expect nil to be returned if the allocation failed.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7871043
Rather than just checking for ENOMEM, check for a return value of less
than 4096, so that we catch other errors such as EACCES and EINVAL.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7942043
Fixes#5061.
Current code relies on the fact that fd's are automatically removed from epoll set when closed. However, it is not true. Underlying file description is removed from epoll set only when *all* fd's referring to it are closed.
There are 2 bad consequences:
1. Kernel delivers notifications on already closed fd's.
2. The following sequence of events leads to error:
- add fd1 to epoll
- dup fd1 = fd2
- close fd1 (not removed from epoll since we've dup'ed the fd)
- dup fd2 = fd1 (get the same fd as fd1)
- add fd1 to epoll = EEXIST
So, if fd can be potentially dup'ed of fork'ed, it's necessary to explicitly remove the fd from epoll set.
R=golang-dev, bradfitz, dave
CC=golang-dev
https://golang.org/cl/7870043
Add missing CLOSUREVAR in switch.
Mark MAKE, string conversion nodes as impossible.
Control statements do not need instrumentation.
Instrument COM and LROT nodes.
Instrument map length.
Update #4228
R=dvyukov, golang-dev
CC=golang-dev
https://golang.org/cl/7504047
Hashtable is arranged as an array of
8-entry buckets with chained overflow.
Each bucket has 8 extra hash bits
per key to provide quick lookup within
a bucket. Table is grown incrementally.
Update #3885
Go time drops from 0.51s to 0.34s.
R=r, rsc, m3b, dave, bradfitz, khr, ugorji, remyoudompheng
CC=golang-dev
https://golang.org/cl/7504044
Inserting a key-value pair into a hashmap storing keys or values
indirectly can cause the garbage collector to find the hashmap in
an inconsistent state.
Fixes#5074.
R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/7913043
On NetBSD tv_sec is already an int64 so no need for a test.
On OpenBSD, semasleep expects a Unix time as argument,
and 1<<30 is in 2004.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7810044
The current SysAlloc implementation suffers from a signed vs unsigned
comparision bug. Since the error code from mmap is negated, the
unsigned comparision of v < 4096 is always false on error. Fix this
by switching to the darwin/freebsd/linux mmap model and leave the mmap
return value unmodified.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7870044
This provides a way to generate core dumps when people need them.
The settings are:
GOTRACEBACK=0 no traceback on panic, just exit
GOTRACEBACK=1 default - traceback on panic, then exit
GOTRACEBACK=2 traceback including runtime frames on panic, then exit
GOTRACEBACK=crash traceback including runtime frames on panic, then crash
Fixes#3257.
R=golang-dev, devon.odell, r, daniel.morsing, ality
CC=golang-dev
https://golang.org/cl/7666044
NEGL does a negation of the bottom 32 bits and then zero-extends to 64 bits,
resulting in a negative 32-bit number but a positive 64-bit number.
NEGQ does a full 64-bit negation, so that the result is negative both as
a 32-bit and as a 64-bit number.
This doesn't matter for the functions that are declared to return int32.
It only matters for the ones that return int64 or void* [sic].
This will fix the current incorrect error in the OpenBSD/amd64 build.
The build will still be broken, but it won't report a bogus error.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7536046
Bring net/fd_linux.go back (it was deleted this morning)
because it is still needed for ARM.
Fix a few typos in the runtime reorg.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7759046
thread_GOOS.c becomes os_GOOS.c.
signal_GOOS_GOARCH.c becomes os_GOOS_GOARCH.c,
but with non-GOARCH-specific code moved into os_GOOS.c.
The actual arch-specific signal handler moves into signal_GOARCH.c
to avoid per-GOOS duplication.
New files signal_GOOS_GOARCH.h provide macros for
accessing fields of the very system-specific signal info structs.
Lots moving, but nothing changing.
This is a preliminarly cleanup so I can work on the signal
handling code to fix some open issues without having to
make each change 13 times.
Tested on Linux and OS X, 386 and amd64.
Will fix Plan 9, Windows, and ARM after the fact if necessary.
(Plan 9 and Windows should be fine; ARM will probably have some typos.)
Net effect: -1081 lines of code.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7565048
With the global redefinition of runtime·open by CL 7543043,
we need to provide a third argument and remove the cast
to the string.
Fixes build on 386 version of Plan 9.
R=khr, rsc, rminnich, ality
CC=golang-dev
https://golang.org/cl/7644047
Uses AES hardware instructions on 386/amd64 to implement
a fast hash function. Incorporates a random key to
thwart hash collision DOS attacks.
Depends on CL#7548043 for new assembly instructions.
Update #3885
Helps some by making hashing faster. Go time drops from
0.65s to 0.51s.
R=rsc, r, bradfitz, remyoudompheng, khr, dsymonds, minux.ma, elias.naur
CC=golang-dev
https://golang.org/cl/7543043
The issue was that scvg is assigned *after* the scavenger goroutine is started,
so when the scavenger calls entersyscall() the g==scvg check can fail.
Fixes#5025.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7629045
The problem is that there are lots of dead G's from previous tests,
each dead G consumes 1 stack segment.
Fixes#5034.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7749043
The call to the C function runtime.findnull() requires
that we provide the argument at 0(SP).
R=rsc, rminnich, ality
CC=golang-dev
https://golang.org/cl/7559047
Otherwise the next goroutine run on the m
can get inadvertently locked if it executes a cgo call
that turns on the internal lock.
While we're here, fix the cgo panic unwind to
decrement m->ncgo like the non-panic unwind does.
Fixes#4971.
R=golang-dev, iant, dvyukov
CC=golang-dev
https://golang.org/cl/7627043
Now the default startup is that the program begins at _rt0_386_$GOOS,
which behaves as if calling main(argc, argv). Main jumps to _rt0_386.
This makes the _rt0_386 entry match the expected semantics for
the standard C "main" function, which we can now provide for use when
linking against a standard C library.
386 analogue of https://golang.org/cl/7525043
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7551045
Change 231af8ac63aa (CL 7314062) made runtime.enteryscall()
set m->mcache = nil, which means that we can no longer use
syscall.errstr in syscall.Syscall and syscall.Syscall6, since it
requires a new buffer to be allocated for holding the error string.
Instead, we use pre-allocated per-M storage to hold error strings
from syscalls made while in entersyscall mode, and call
runtime.findnull to calculate the lengths.
Fixes#4994.
R=rsc, rminnich, ality, dvyukov, rminnich, r
CC=golang-dev
https://golang.org/cl/7567043
The deadlock episodically occurs on misc/cgo/test/TestCthread.
The problem is that starttheworld() leaves some P's with local work
without M's. Then all active M's enter into syscalls, but reject to
wake another M's due to the following check (both in entersyscallblock() and in retake()):
if(p->runqhead == p->runqtail &&
runtime·atomicload(&runtime·sched.nmspinning) +
runtime·atomicload(&runtime·sched.npidle) > 0)
continue;
R=rsc
CC=golang-dev
https://golang.org/cl/7424054
Still to do: non-linux and non-amd64.
It may work on other ELF-based amd64 systems too, but untested.
"go test -ldflags -hostobj $GOROOT/misc/cgo/test" passes.
Much may yet change, but this seems a reasonable checkpoint.
R=iant
CC=golang-dev
https://golang.org/cl/7369057
Now the default startup is that the program begins at _rt0_amd64_$GOOS,
which sets DI = argc, SI = argv and jumps to _rt0_amd64.
This makes the _rt0_amd64 entry match the expected semantics for
the standard C "main" function, which we can now provide for use when
linking against a standard C library.
R=golang-dev, devon.odell, minux.ma
CC=golang-dev
https://golang.org/cl/7525043
broke arm garbage collector
traceback_arm fails with a missing pc. It needs CL 7494043.
But that only makes the build break later, this time with
"invalid freelist". Roll back until it can be fixed correctly.
««« original CL description
runtime: restrict stack root scan to locals and arguments
R=rsc
CC=golang-dev
https://golang.org/cl/7301062
»»»
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7493044
If the constant CollectStats is non-zero and GOGCTRACE=1
the garbage collector will print basic statistics about executed
GC instructions.
R=golang-dev, dvyukov
CC=golang-dev, rsc
https://golang.org/cl/7413049
Fixes#4904.
The problem was that when the test runs the heap had grown to ~100MB,
so GC allows it to grow to 200MB, and so the test fails.
Moving the test to a separate process makes it much more isolated and stable.
R=golang-dev, minux.ma
CC=golang-dev
https://golang.org/cl/7441046
Also make the crossover point an architecture-dependent constant,
although it's the same everywhere for now.
BenchmarkAppendStr1Byte 416 145 -65.14%
BenchmarkAppendStr4Bytes 743 217 -70.79%
BenchmarkAppendStr8Bytes 421 270 -35.87%
BenchmarkAppendStr16Bytes 415 403 -2.89%
BenchmarkAppendStr32Bytes 415 391 -5.78%
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7459044
Putting the M initialization in multiple places will not scale.
Various code assumes mstart is the start already. Make it so.
R=golang-dev, devon.odell
CC=golang-dev
https://golang.org/cl/7420048
There is a #pragma dynexport crosscall2, to help SWIG,
and 6l cannot export the symbol if it doesn't get to see it.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7448044
There are some function pointers declared by 6c in
package runtime without initialization and then also
declared in package runtime/cgo with initialization,
so that if runtime/cgo is linked in, the function pointers
are non-nil, and otherwise they are nil. We depend on
this property for implementing non-essential cgo hooks
in package runtime.
The declarations in package runtime are 6c-compiled
and end up in .6 files. The declarations in package runtime/cgo
are gcc-compiled and end up in .o files. Since 6l links the .6
and .o files together, this all works.
However, when we switch to "external linking" mode,
6l will not see the .o files, and it would be up to the host linker
to resolve the two into a single initialized symbol.
Not all host linkers will do this (in particular OS X gcc will not).
To fix this, move the cgo declarations into 6c-compiled code,
so that they end up in .6 files, so that 6l gets them no matter what.
R=golang-dev
CC=golang-dev
https://golang.org/cl/7440045
The naming in this package is a disaster.
Make it all consistent.
Remove some 'static' from functions that will
be referred to from other files soon.
This CL is purely renames using global search and replace.
Submitting separately so that real changes will not
be drowned out by these renames in future CLs.
TBR=iant
CC=golang-dev
https://golang.org/cl/7416046
runtime: double-check that symbol table is sorted
If the symbol table is unsorted, the binary search in findfunc
will not find its func, which will make stack traces stop early.
When the garbage collector starts using the stack tracer,
that would be a serious problem.
The unsorted symbol addresses came from from two things:
1. The symbols in an ELF object are not necessarily sorted,
so sort them before adding them to the symbol list.
2. The __i686.get_pc_thunk.bx symbol is present in multiple
object files and was having its address adjusted multiple
times, producing an incorrect address in the symbol table.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7440044
Revision 6a88e1893941 corrupts the argument to
racefuncenter by pushing the data block pointer
to the stack.
Fixes#4885.
R=dvyukov, rsc
CC=golang-dev
https://golang.org/cl/7381053
add per-P cache of dead G's
add global runnable queue (not used for now)
add list of idle P's (not used for now)
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7397061
The current code uses 64-bit pc-relative on 64-bit systems,
but in ELF linkers there is no such thing, so we cannot
express this in a .o file. Change to 32-bit.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7383055
This CL changes the encoding used for the Go symbol table,
stored in the binary and used at run time. It does not change
any of the semantics or structure: the bits are just packed
a little differently.
The comment at the top of runtime/symtab.c describes the new format.
Compared to the Go 1.0 format, the main changes are:
* Store symbol addresses as full-pointer-sized host-endian values.
(For 6g, this means addresses are 64-bit little-endian.)
* Store other values (frame sizes and so on) varint-encoded.
The second change more than compensates for the first:
for the godoc binary on OS X/amd64, the new symbol table
is 8% smaller than the old symbol table (1,425,668 down from 1,546,276).
This is a required step for allowing the host linker (gcc) to write
the final Go binary, since it will have to fill in the symbol address slots
(so the slots must be host-endian) and on 64-bit systems it may
choose addresses above 4 GB.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7403054
Separates the implementation of nanotime on 64-bit
version of Plan 9 from that on the 32-bit version.
The former uses a syscall.
R=rsc, rminnich, ality
CC=golang-dev
https://golang.org/cl/7379051
sigprocmask() is process-wide on darwin, so two concurrent
libcgo_sys_thread_start() can result in all signals permanently
blocked, which in particular blocks handling of nil derefs.
Fixes#4833.
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/7324058
For now, all the callbacks from C use top-level Go functions,
so they use the equivalent C function pointer, and will continue
to do so. But perhaps some day this will be useful for calling
a Go func value (at least if the type is already known).
More importantly, the Windows callback code needs to be able
to use cgocallback_gofunc to call a Go func value.
Should fix the Windows build.
R=ken2
CC=golang-dev
https://golang.org/cl/7388049
Change ARM context register to R7, to get out of the way
of the register allocator during the compilation of the
prologue statements (it wants to use R0 as a temporary).
Step 2 of http://golang.org/s/go11func.
R=ken2
CC=golang-dev
https://golang.org/cl/7369048
runtime: add context argument to gogocall
Too many other things use AX, and at least one
(stack zeroing) cannot be moved onto a different
register. Use the less special DX instead.
Preparation for step 2 of http://golang.org/s/go11func.
Nothing interesting here, just split out so that we can
see it's correct before moving on.
R=ken2
CC=golang-dev
https://golang.org/cl/7395050
Previously, the func structure contained an inaccurate value for
the args member and a 0 value for the locals member.
This change populates the func structure with args and locals
values computed by the compiler. The number of args was
already available in the ATEXT instruction. The number of
locals is now passed through in the new ALOCALS instruction.
This change also switches the unit of args and locals to be
bytes, just like the frame member, instead of 32-bit words.
R=golang-dev, bradfitz, cshapiro, dave, rsc
CC=golang-dev
https://golang.org/cl/7399045
mpreinit() is called on the parent thread and with mcache (can allocate memory),
minit() is called on the child thread and can not allocate memory.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7389043
The removed code leads to the situation when M executes the same locked G again
and again.
This is https://golang.org/cl/7310096 but with return instead of break
in the nested switch.
Fixes#4820.
R=golang-dev, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/7304102
If a test can be placed in the same package ("internal"), it is placed
there. This facilitates testing of package-private details. Because of
dependency cycles some packages cannot be tested by internal tests.
R=golang-dev, rsc, mikioh.mikioh
CC=golang-dev, r
https://golang.org/cl/7323044
* Handle p==nil in signalstack by setting SS_DISABLE flag.
* Make minit only allocate a signal g if there's not one already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7323072
Fix the sa_mask member of the sigaction struct - on FreeBSD this is
declared as a sigset_t, which is an array of four unsigned ints.
Replace the current int64 with Sigset from defs_freebsd_GOARCH, which
has the correct definition.
Unbreaks the FreeBSD builds.
R=golang-dev, dave, minux.ma
CC=golang-dev
https://golang.org/cl/7333047
broke windows build
««« original CL description
runtime: ensure forward progress of runtime.Gosched() for locked goroutines
The removed code leads to the situation when M executes the same locked G again and again.
Fixes#4820.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7310096
»»»
TBR=dvyukov
CC=golang-dev
https://golang.org/cl/7343050
Arguably if this happens the program is buggy anyway,
but letting the panic continue looks better than interrupting it.
Otherwise things like this are possible, and confusing:
$ go run x.go
panic: $ echo $?
0
$
Fixes#3934.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7322083
This is the same logic used in the standard tracebacks.
The caller pc is the pc after the call, so except in the
fake "call" caused by a panic, back up the pc enough
that the lookup will use the previous instruction.
Fixes#4150.
Fixes#4151.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7317047
Before, the mheap structure was in the bss,
but it's quite large (today, 256 MB, much of
which is never actually paged in), and it makes
Go binaries run afoul of exec-time bss size
limits on some BSD systems.
Fixes#4447.
R=golang-dev, dave, minux.ma, remyoudompheng, iant
CC=golang-dev
https://golang.org/cl/7307122
The removed code leads to the situation when M executes the same locked G again and again.
Fixes#4820.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7310096
In addition to the compile failure fixed in signal*.c,
preserving the signal mask led to very strange crashes.
Testing shows that looking for SIG_IGN is all that
matters to get along with nohup, so reintroduce
sigset_zero instead of trying to preserve the signal mask.
TBR=iant
CC=golang-dev
https://golang.org/cl/7323067
There are two ways nohup(1) might be implemented:
it might mask away the signal, or it might set the handler
to SIG_IGN, both of which are inherited across fork+exec.
So two fixes:
* Make sure to preserve the inherited signal mask at
minit instead of clearing it.
* If the SIGHUP handler is SIG_IGN, leave it that way.
Fixes#4491.
R=golang-dev, mikioh.mikioh, iant
CC=golang-dev
https://golang.org/cl/7308102
No code changes.
This is mainly in preparation to scheduler changes,
oldstack/newstack are not related to scheduling.
R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/7311085
With the new scheduler races in the tests are reported during execution of other tests.
The change joins goroutines started during the tests.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7310066
The problem happens when end=0, then end-1 is very big number.
Observed with the new scheduler.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7307073
Removes limit on maximum number of goroutines ever existed.
code.google.com/p/goexecutor tests now pass successfully.
Also slightly improves performance.
Before: $ time ./flate.test -test.short
real 0m9.314s
After: $ time ./flate.test -test.short
real 0m8.958s
Fixes#4286.
The runtime is built from llvm rev 174312.
R=rsc
CC=golang-dev
https://golang.org/cl/7218044
Change the stack unwinding code to compensate for the dynamic
relocation of symbols.
Change the gc instruction GC_CALL to use a relative offset instead of
an absolute address.
R=golang-dev
CC=golang-dev
https://golang.org/cl/7248048
* Separate internal and external LockOSThread, for cgo safety.
* Show goroutine that made faulting cgo call.
* Never start a panic due to a signal caused by a cgo call.
Fixes#3774.
Fixes#3775.
Fixes#3797.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/7228081
Binary data in mprof.goc may prevent the garbage collector from freeing
memory blocks. This patch replaces all calls to runtime·mallocgc() with
calls to an allocator private to mprof.goc, thus making the private
memory invisible to the garbage collector. The addrhash variable is
moved outside of the .bss section.
R=golang-dev, dvyukov, rsc, minux.ma
CC=dave, golang-dev, remyoudompheng
https://golang.org/cl/7135063
This change also resolves some issues with note handling: we now make
sure that there is enough room at the bottom of every goroutine to
execute the note handler, and the `exitstatus' is no longer a global
entity, which resolves some race conditions.
R=rminnich, npe, rsc, ality
CC=golang-dev
https://golang.org/cl/6569068
Range access functions are already available in TSan library
but were not yet used.
Time for go test -race -short:
Before:
compress/flate 24.244s
exp/norm >200s
go/printer 78.268s
After:
compress/flate 17.760s
exp/norm 5.537s
go/printer 5.738s
Fixes#4250.
R=dvyukov, golang-dev, fullung
CC=golang-dev
https://golang.org/cl/7229044
Useful for debugging of runtime bugs.
+ Do not print "stack segment boundary" unless GOTRACEBACK>1.
+ Do not traceback system goroutines unless GOTRACEBACK>1.
R=rsc, minux.ma
CC=golang-dev
https://golang.org/cl/7098050
Mark candidate spans one GC pass earlier.
Move scavenger's code out from mgc0 and constrain it into mheap (where it belongs).
R=rsc, dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/7002049
This is for SPARC64, a 64-bit processor that uses all 64-bits
of virtual addresses. The idea is to use the low order 3 bits
to at least get a small ABA counter. That should work since
pointers are aligned. The idea is for SPARC64 to set CNT_MASK
== 7, PTR_BITS == 0, PTR_MASK == 0xffffffffffffff8.
Also add uintptr casts to avoid GCC warnings. The gccgo
runtime code is compiled with GCC, and GCC warns when casting
between a pointer and a type of a different size.
R=dvyukov
CC=golang-dev
https://golang.org/cl/7225043
Currently it's summed to mark phase.
The change makes it easier to diagnose long stop-the-world phases.
R=golang-dev, bradfitz, dave
CC=golang-dev
https://golang.org/cl/7182043
If the scanned block has no typeinfo the garbage collector will attempt
to get the actual type of the block.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/7093045
Also undo revision a5b96b602690 used to workaround the bug.
Fixes#4643.
R=rsc, golang-dev, dave, minux.ma, lucio.dere, bradfitz
CC=golang-dev
https://golang.org/cl/7090043
The Plan 9 symbol table format defines big-endian symbol values
for portability, but we want to be able to generate an ELF object file
and let the host linker link it, as part of the solution to issue 4069.
The symbol table itself, since it is loaded into memory at run time,
must be filled in by the final host linker, using relocation directives
to set the symbol values. On a little-endian machine, the linker will
only fill in little-endian values during relocation, so we are forced
to use little-endian symbol values.
To preserve most of the original portability of the symbol table
format, we make the table itself say whether it uses big- or
little-endian values. If the table begins with the magic sequence
fe ff ff ff 00 00
then the actual table begins after those six bytes and contains
little-endian symbol values. Otherwise, the table is in the original
format and contains big-endian symbol values. The magic sequence
looks like an "end of table" entry (the fifth byte is zero), so legacy
readers will see a little-endian table as an empty table.
All the gc architectures are little-endian today, so the practical
effect of this CL is to make all the generated tables little-endian,
but if a big-endian system comes along, ld will not generate
the magic sequence, and the various readers will fall back to the
original big-endian interpretation.
R=ken2
CC=golang-dev
https://golang.org/cl/7066043
sysarch requires arguments to be passed on the stack, not in registers.
Credit to Shenghou Ma (minux) for the fix.
R=minux.ma, devon.odell
CC=golang-dev
https://golang.org/cl/7037043
Used to then die on a nil pointer situation. Most Linux standard setups are rather
restrictive regarding the default amount of lockable memory.
R=minux.ma, rsc
CC=golang-dev
https://golang.org/cl/6997049
Currently it silently "succeeds" saying that it run 0 tests
if there are compilations errors.
With this change it fails and outputs the compilation error.
R=golang-dev, remyoudompheng
CC=golang-dev
https://golang.org/cl/7002058
When we release memory to the OS, if the OS doesn't want us
to release it (for example, because the program executed
mlockall(MCL_FUTURE)), madvise will fail. Ignore the failure
instead of crashing.
Fixes#3435.
R=ken2
CC=golang-dev
https://golang.org/cl/6998052
This disables checks for limited address space
and unlimited stack. They are not required for Go.
Fixes#4577.
R=golang-dev, iant
CC=golang-dev, kamil.kisiel, minux.ma
https://golang.org/cl/7003045
Enable cgo on OpenBSD.
The OpenBSD ld.so(1) does not currently support PT_TLS sections. Work
around this by fixing up the TCB that has been provided by librthread
and reallocating a TCB with additional space for TLS. Also provide a
wrapper for pthread_create, allowing zeroed TLS to be allocated for
threads created externally to Go.
Joint work with Shenghou Ma (minux).
Requires change 6846064.
Fixes#3205.
R=golang-dev, minux.ma, iant, rsc, iant
CC=golang-dev
https://golang.org/cl/6853059
With this change the runtime can now read GOMAXPROCS, GOGC, etc.
I'm not quite sure how we missed this.
R=seed, lucio.dere, rsc
CC=golang-dev
https://golang.org/cl/6935062
The code:
func main() {
v := make([]int64, 10)
i := 1
_ = v[(i*4)/3]
}
crashes compiler with:
Program received signal SIGSEGV, Segmentation fault.
0x000000000043c274 in walkexpr (np=0x7fffffffc9b8, init=0x0) at src/cmd/gc/walk.c:587
587 *init = concat(*init, n->ninit);
(gdb) bt
#0 0x000000000043c274 in walkexpr (np=0x7fffffffc9b8, init=0x0) at src/cmd/gc/walk.c:587
#1 0x0000000000432d15 in copyexpr (n=0x7ffff7f69a48, t=<optimized out>, init=0x0) at src/cmd/gc/subr.c:2020
#2 0x000000000043f281 in walkdiv (init=0x0, np=0x7fffffffca70) at src/cmd/gc/walk.c:2901
#3 walkexpr (np=0x7ffff7f69760, init=0x0) at src/cmd/gc/walk.c:956
#4 0x000000000043d801 in walkexpr (np=0x7ffff7f69bc0, init=0x0) at src/cmd/gc/walk.c:988
#5 0x000000000043cc9b in walkexpr (np=0x7ffff7f69d38, init=0x0) at src/cmd/gc/walk.c:1068
#6 0x000000000043c50b in walkexpr (np=0x7ffff7f69f50, init=0x0) at src/cmd/gc/walk.c:879
#7 0x000000000043c50b in walkexpr (np=0x7ffff7f6a0c8, init=0x0) at src/cmd/gc/walk.c:879
#8 0x0000000000440a53 in walkexprlist (l=0x7ffff7f6a0c8, init=0x0) at src/cmd/gc/walk.c:357
#9 0x000000000043d0bf in walkexpr (np=0x7fffffffd318, init=0x0) at src/cmd/gc/walk.c:566
#10 0x00000000004402bf in vmkcall (fn=<optimized out>, t=0x0, init=0x0, va=0x7fffffffd368) at src/cmd/gc/walk.c:2275
#11 0x000000000044059a in mkcall (name=<optimized out>, t=0x0, init=0x0) at src/cmd/gc/walk.c:2287
#12 0x000000000042862b in callinstr (np=0x7fffffffd4c8, init=0x7fffffffd568, wr=0, skip=<optimized out>) at src/cmd/gc/racewalk.c:478
#13 0x00000000004288b7 in racewalknode (np=0x7ffff7f68108, init=0x7fffffffd568, wr=0, skip=0) at src/cmd/gc/racewalk.c:287
#14 0x0000000000428781 in racewalknode (np=0x7ffff7f65840, init=0x7fffffffd568, wr=0, skip=0) at src/cmd/gc/racewalk.c:302
#15 0x0000000000428abd in racewalklist (l=0x7ffff7f65840, init=0x0) at src/cmd/gc/racewalk.c:97
#16 0x0000000000428d0b in racewalk (fn=0x7ffff7f5f010) at src/cmd/gc/racewalk.c:63
#17 0x0000000000402b9c in compile (fn=0x7ffff7f5f010) at src/cmd/6g/../gc/pgen.c:67
#18 0x0000000000419f86 in funccompile (n=0x7ffff7f5f010, isclosure=0) at src/cmd/gc/dcl.c:1414
#19 0x0000000000424161 in p9main (argc=<optimized out>, argv=<optimized out>) at src/cmd/gc/lex.c:431
#20 0x0000000000401739 in main (argc=<optimized out>, argv=<optimized out>) at src/lib9/main.c:35
The problem is nil init passed to mkcall().
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6940045
Details:
- This CL is the conceptual skeleton of code found in CL 6114046
- The garbage collector uses struct Obj to specify memory blocks
- scanblock() is putting found memory blocks into an intermediate buffer
(xbuf) before adding/flushing them to the main work buffer (wbuf)
- The main loop in scanblock() is replaced with a skeleton code that
in the future will be able to recognize the type of objects and
thus will improve the garbage collector's precision.
For now, all objects are simply sequences of pointers so
the precision of the garbage collector remains unchanged.
- The code plugs .gcdata and .gcbss sections into the garbage collector.
scanblock() in this CL is unable to make any use of this.
R=rsc, dvyukov, remyoudompheng
CC=dave, golang-dev, minux.ma
https://golang.org/cl/6856121
