This improves the documentation comment on gcMarkDone, replaces a
recursive call with a simple goto, and disables preemption before
stopping the world in accordance with the documentation comment on
stopTheWorldWithSema.
Updates #13363, but, sadly, doesn't fix it.
Change-Id: I6cb2a5836b35685bf82f7b1ce7e48a7625906656
Reviewed-on: https://go-review.googlesource.com/17149
Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This improves stack barrier debugging messages in various ways:
1) Rather than printing only the remaining stack barriers (of which
there may be none, which isn't very useful), print all of the G's
stack barriers with a marker at the position the stack itself has
unwound to and a marker at the problematic stack barrier (where
applicable).
2) Rather than crashing if we encounter a stack barrier when there are
no more stkbar entries, print the same debug message we would if we
had encountered a stack barrier at an unexpected location.
Hopefully this will help with debugging #12528.
Change-Id: I2e6fe6a778e0d36dd8ef30afd4c33d5d94731262
Reviewed-on: https://go-review.googlesource.com/17147
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
The stack barrier locking functions use a simple cas lock because they
need to support trylock, but currently don't increment g.m.locks. This
is okay right now because they always run on the system stack or the
signal stack and are hence non-preemtible, but this could lead to
difficult-to-reproduce deadlocks if these conditions change in the
future.
Make these functions more robust by incrementing g.m.locks and making
them nosplit to enforce non-preemtibility.
Change-Id: I73d60a35bd2ad2d81c73aeb20dbd37665730eb1b
Reviewed-on: https://go-review.googlesource.com/17058
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ingo Oeser <nightlyone@googlemail.com>
Reviewed-by: Russ Cox <rsc@golang.org>
This test depends on GODEBUG=gcstackbarrierall, which doesn't work on
ppc64.
Updates #13334.
Change-Id: Ie554117b783c4e999387f97dd660484488499d85
Reviewed-on: https://go-review.googlesource.com/17120
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
During a crash showing goroutine stacks of all threads
(with GOTRACEBACK=crash), it can be that f == nil.
Only happens on Solaris; not sure why.
Change-Id: Iee2c394a0cf19fa0a24f6befbc70776b9e42d25a
Reviewed-on: https://go-review.googlesource.com/17110
Reviewed-by: Austin Clements <austin@google.com>
The nosplit stack is now much bigger, so we can afford to allocate
libcall on stack.
Fix asmsysvicall6 to not update errno if g == nil.
These two fixes TestCgoCallbackGC on solaris, which used to stuck
in a loop.
Change-Id: Id1b13be992dae9f059aa3d47ffffd37785300933
Reviewed-on: https://go-review.googlesource.com/17076
Run-TryBot: Minux Ma <minux@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Solaris needs to make system calls without a g,
and Solaris uses asmcgocall to make system calls.
I know, I know.
I hope this makes CL 16915, fixing #12277, work on Solaris.
Change-Id: If988dfd37f418b302da9c7096f598e5113ecea87
Reviewed-on: https://go-review.googlesource.com/17072
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Run-TryBot: Russ Cox <rsc@golang.org>
sysmon runs without a P. This means it can't interact with the garbage
collector, so write barriers not allowed in anything that sysmon does.
Fixes#10600.
Change-Id: I9de1283900dadee4f72e2ebfc8787123e382ae88
Reviewed-on: https://go-review.googlesource.com/17006
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
allocm is a very unusual function: it is specifically designed to
allocate in contexts where m.p is nil by temporarily taking over a P.
Since allocm is used in many contexts where it would make sense to use
nowritebarrierrec, this commit teaches the nowritebarrierrec analysis
to stop at allocm.
Updates #10600.
Change-Id: I8499629461d4fe25712d861720dfe438df7ada9b
Reviewed-on: https://go-review.googlesource.com/17005
Reviewed-by: Russ Cox <rsc@golang.org>
gentraceback is used in many contexts where write barriers are
disallowed. This currently works because the only write barrier is in
assigning frame.argmap in setArgInfo and in practice frame is always
on the stack, so this write barrier is a no-op.
However, we can easily eliminate this write barrier, which will let us
statically disallow write barriers (using go:nowritebarrierrec
annotations) in many more situations. As a bonus, this makes the code
a little more idiomatic.
Updates #10600.
Change-Id: I45ba5cece83697ff79f8537ee6e43eadf1c18c6d
Reviewed-on: https://go-review.googlesource.com/17003
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
The assumption is that there are no nested function calls in complex expressions.
For the most part that assumption is true. It wasn't for these calls inserted during walk.
Fix that.
I looked through all the calls to mkcall in walk and these were the only cases
that emitted calls, that could be part of larger expressions (like not delete),
and that were not already handled.
Fixes#12225.
Change-Id: Iad380683fe2e054d480e7ae4e8faf1078cdd744c
Reviewed-on: https://go-review.googlesource.com/17034
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This adds a test that runs CPU profiling with a high load of stack
barriers and stack barrier insertion/removal operations and checks
that both 1) the runtime doesn't crash and 2) stackBarrier itself
never appears in a profile. Prior to the fix for gentraceback starting
in the middle of stackBarrier, condition 2 often failed.
Change-Id: Ic28860448859029779844c4bf3bb28ca84611e2c
Reviewed-on: https://go-review.googlesource.com/17037
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A sigprof during stack barrier insertion or removal can crash if it
detects an inconsistency between the stkbar array and the stack
itself. Currently we protect against this when scanning another G's
stack using stackLock, but we don't protect against it when unwinding
stack barriers for a recover or a memmove to the stack.
This commit cleans up and improves the stack locking code. It
abstracts out the lock and unlock operations. It uses the lock
consistently everywhere we perform stack operations, and pushes the
lock/unlock down closer to where the stack barrier operations happen
to make it more obvious what it's protecting. Finally, it modifies
sigprof so that instead of spinning until it acquires the lock, it
simply doesn't perform a traceback if it can't acquire it. This is
necessary to prevent self-deadlock.
Updates #11863, which introduced stackLock to fix some of these
issues, but didn't go far enough.
Updates #12528.
Change-Id: I9d1fa88ae3744d31ba91500c96c6988ce1a3a349
Reviewed-on: https://go-review.googlesource.com/17036
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, if a profiling signal happens in the middle of
stackBarrier, gentraceback may see inconsistencies between stkbar and
the barriers on the stack and it will certainly get the wrong return
PC for stackBarrier. In most cases, the return PC won't be a PC at all
and this will immediately abort the traceback (which is considered
okay for a sigprof), but if it happens to be a valid PC this may sent
gentraceback down a rabbit hole.
Fix this by detecting when the gentraceback starts in stackBarrier and
simulating the completion of the barrier to get the correct initial
frame.
Change-Id: Ib11f705ac9194925f63fe5dfbfc84013a38333e6
Reviewed-on: https://go-review.googlesource.com/17035
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Cgo-created threads transition between having associated Go g's and m's and not.
A signal arriving during the transition could think it was safe and appropriate to
run Go signal handlers when it was in fact not.
Avoid the race by masking all signals during the transition.
Fixes#12277.
Change-Id: Ie9711bc1d098391d58362492197a7e0f5b497d14
Reviewed-on: https://go-review.googlesource.com/16915
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Mostly by avoiding CX entirely, sometimes by reloading it.
I also vetted the assembly in other packages, it's all fine.
Change-Id: I50059669aaaa04efa303cf22ac228f9d14d83db0
Reviewed-on: https://go-review.googlesource.com/16386
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Replace the cross platform but unsafe [4]uintptr type with a OS
specific type, sigset. Most OSes already define sigset, and this
change defines a suitable sigset for the OSes that don't (darwin,
openbsd). The OSes that don't use m.sigmask (windows, plan9, nacl)
now defines sigset as the empty type, struct{}.
The gain is strongly typed access to m.sigmask, saving a dynamic
size sanity check and unsafe.Pointer casting. Also, some storage is
saved for each M, since [4]uinptr was conservative for most OSes.
The cost is that OSes that don't need m.sigmask has to define sigset.
completes ./all.bash with GOOS linux, on amd64
completes ./make.bash with GOOSes openbsd, android, plan9, windows,
darwin, solaris, netbsd, freebsd, dragonfly, all amd64.
With GOOS=nacl ./make.bash failed with a seemingly unrelated error.
R=go1.7
Change-Id: Ib460379f063eb83d393e1c5efe7333a643c1595e
Reviewed-on: https://go-review.googlesource.com/16942
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
golang.org/cl/16796 broke android/386 by assuming behaviour specific to glibc's
dynamic linker. Copy bionic by using int $0x80 to invoke syscalls on
android/386 as the old alternative (CALL *runtime_vdso(SB)) cannot be compiled
without text relocations, which we want to get rid of on android.
Also remove "CALL *runtime_vdso(SB)" variant from the syscall package.
Change-Id: I6c01849f8dcbd073d000ddc8f13948a836b8b261
Reviewed-on: https://go-review.googlesource.com/16996
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Not all tests passing yet, but a good chunk are.
Change-Id: I5daebaeabf3aecb380674ece8830a86751a8d139
Reviewed-on: https://go-review.googlesource.com/16458
Reviewed-by: Rahul Chaudhry <rahulchaudhry@google.com>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Currently, if an allocation is large enough that arena_end + size
overflows (which is not hard to do on 32-bit), we go ahead and call
sysReserve with the impossible base and length and depend on this to
either directly fail because the kernel can't possibly fulfill the
requested mapping (causing mheap.sysAlloc to return nil) or to succeed
with a mapping at some other address which will then be rejected as
outside the arena.
In order to make this less subtle, less dependent on the kernel
getting all of this right, and to eliminate the hopeless system call,
add an explicit overflow check.
Updates #13143. This real issue has been fixed by 0de59c2, but this is
a belt-and-suspenders improvement on top of that. It was uncovered by
my symbolic modeling of that bug.
Change-Id: I85fa868a33286fdcc23cdd7cdf86b19abf1cb2d1
Reviewed-on: https://go-review.googlesource.com/16961
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
mcache.tiny is in non-GC'd memory, but points to heap memory. As a
result, there may or may not be write barriers when writing to
mcache.tiny. Make it clearer that funny things are going on by making
mcache.tiny a uintptr instead of an unsafe.Pointer.
Change-Id: I732a5b7ea17162f196a9155154bbaff8d4d00eac
Reviewed-on: https://go-review.googlesource.com/16963
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The tiny alloc cache is maintained in a pointer from non-GC'd memory
(mcache) to heap memory and hence must be handled carefully.
Currently we clear the tiny alloc cache during sweep termination and,
if it is assigned to a non-nil value during concurrent marking, we
depend on a write barrier to keep the new value alive. However, while
the compiler currently always generates this write barrier, we're
treading on thin ice because write barriers may not happen for writes
to non-heap memory (e.g., typedmemmove). Without this lucky write
barrier, the GC may free a current tiny block while it's still
reachable by the tiny allocator, leading to later memory corruption.
Change this code so that, rather than depending on the write barrier,
we simply clear the tiny cache during mark termination when we're
clearing all of the other mcaches. If the current tiny block is
reachable from regular pointers, it will be retained; if it isn't
reachable from regular pointers, it may be freed, but that's okay
because there won't be any pointers in non-GC'd memory to it.
Change-Id: I8230980d8612c35c2997b9705641a1f9f865f879
Reviewed-on: https://go-review.googlesource.com/16962
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If you set GODEBUG=cgocheck=2 the runtime package will use the write
barrier to detect cases where a Go program writes a Go pointer into
non-Go memory. In conjunction with the existing cgo checks, and the
not-yet-implemented cgo check for exported functions, this should
reliably detect all cases (that do not import the unsafe package) in
which a Go pointer is incorrectly shared with C code. This check is
optional because it turns on the write barrier at all times, which is
known to be expensive.
Update #12416.
Change-Id: I549d8b2956daa76eac853928e9280e615d6365f4
Reviewed-on: https://go-review.googlesource.com/16899
Reviewed-by: Russ Cox <rsc@golang.org>
In mheap.sysAlloc, if an allocation at arena_used would exceed
arena_end (but wouldn't yet push us past arena_start+_MaxArean32), it
trie to extend the arena reservation by another 256 MB. It extends the
arena by calling sysReserve, which, on 32-bit, calls mmap without
MAP_FIXED, which means the address is just a hint and the kernel can
put the mapping wherever it wants. In particular, mmap may choose an
address below arena_start (the kernel also chose arena_start, so there
could be lots of space below it). Currently, we don't detect this case
and, if it happens, mheap.sysAlloc will corrupt arena_end and
arena_used then return the low pointer to mheap.grow, which will crash
when it attempts to index in to h_spans with an underflowed index.
Fix this by checking not only that that p+p_size isn't too high, but
that p isn't too low.
Fixes#13143.
Change-Id: I8d0f42bd1484460282a83c6f1a6f8f0df7fb2048
Reviewed-on: https://go-review.googlesource.com/16927
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
If the area returned by sysReserve in mheap.sysAlloc is outside the
usable arena, we sysFree it. We pass a fake stat pointer to sysFree
because we haven't added the allocation to any stat at that point.
However, we pass a 0 stat, so sysFree panics when it decrements the
stat because the fake stat underflows.
Fix this by setting the fake stat to the allocation size.
Updates #13143 (this is a prerequisite to fixing that bug).
Change-Id: I61a6c9be19ac1c95863cf6a8435e19790c8bfc9a
Reviewed-on: https://go-review.googlesource.com/16926
Reviewed-by: Ian Lance Taylor <iant@golang.org>
golang.org/cl/16346 changed the runtime on linux/386 to invoke the vsyscall
helper via a PIC sequence (CALL 0x10(GS)) when dynamically linking. But it's
actually quite easy to make that code sequence work all the time, so do that,
and remove the ugly machinery that passed the buildmode from the go tool to the
assembly.
This means enlarging m.tls so that we can safely access 0x10(GS) (GS is set to
&m.tls + 4, so 0x10(GS) accesses m_tls[5]).
Change-Id: I1345c34029b149cb5f25320bf19a3cdd73a056fa
Reviewed-on: https://go-review.googlesource.com/16796
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A nosplit comment was added to reflect.typelinks accidentally in
https://golang.org/cl/98510044. There is only one caller of
reflect.typelinks, reflect.typesByString, and that function is not
nosplit. There is no reason for reflect.typelinks to be nosplit.
Change-Id: I0fd3cc66fafcd92643e38e53fa586d6b2f868a0a
Reviewed-on: https://go-review.googlesource.com/16932
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
These now live in runtime/internal/sys.
Change-Id: I270597142516512bfc1395419e51d8083ba1663f
Reviewed-on: https://go-review.googlesource.com/16891
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Allows removing fields that aren't relevant to a particular OS or
changing their types to match the underlying OS system calls they'll
be used for.
Change-Id: I5cea89ee77b4e7b985bff41337e561887c3272ff
Reviewed-on: https://go-review.googlesource.com/16176
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
We're allocating TLS storage for m0 anyway, so might as well use it.
Change-Id: I7dc20bbea5320c8ab8a367f18a9540706751e771
Reviewed-on: https://go-review.googlesource.com/16890
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This requires changing the tls access code to match the patterns documented in
the ABI documentation or the system linker will "optimize" it into ridiculousness.
With this change, -buildmode=pie works, although as it is tested in testshared,
the tests are not run yet.
Change-Id: I1efa6687af0a5b8db3385b10f6542a49056b2eb3
Reviewed-on: https://go-review.googlesource.com/15971
Reviewed-by: Russ Cox <rsc@golang.org>
The PowerPC ISA does not have a PC-relative load instruction, which poses
obvious challenges when generating position-independent code. The way the ELFv2
ABI addresses this is to specify that r2 points to a per "module" (shared
library or executable) TOC pointer. Maintaining this pointer requires
cooperation between codegen and the system linker:
* Non-leaf functions leave space on the stack at r1+24 to save the TOC pointer.
* A call to a function that *might* have to go via a PLT stub must be followed
by a nop instruction that the system linker can replace with "ld r1, 24(r1)"
to restore the TOC pointer (only when dynamically linking Go code).
* When calling a function via a function pointer, the address of the function
must be in r12, and the first couple of instructions (the "global entry
point") of the called function use this to derive the address of the TOC
for the module it is in.
* When calling a function that is implemented in the same module, the system
linker adjusts the call to skip over the instructions mentioned above (the
"local entry point"), assuming that r2 is already correctly set.
So this changeset adds the global entry point instructions, sets the metadata so
the system linker knows where the local entry point is, inserts code to save the
TOC pointer at 24(r1), adds a nop after any call not known to be local and copes
with the odd non-local code transfer in the runtime (e.g. the stuff around
jmpdefer). It does not actually compile PIC yet.
Change-Id: I7522e22bdfd2f891745a900c60254fe9e372c854
Reviewed-on: https://go-review.googlesource.com/15967
Reviewed-by: Russ Cox <rsc@golang.org>
darwin/386, freebsd/386, and linux/386 use a setldt system call to
setup each M's thread-local storage area, and they need access to the
M's id for this. The current code copies m.id into m.tls[0] (and this
logic has been cargo culted to OSes like NetBSD and OpenBSD, which
don't even need m.id to configure TLS), and then the 386 assembly
loads m.tls[0]... but since the assembly code already has a pointer to
the M, it might as well just load m.id directly.
Change-Id: I1a7278f1ec8ebda8d1de3aa3a61993070e3a8cdf
Reviewed-on: https://go-review.googlesource.com/16881
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The larger stack frames causes the nosplit stack to overflow so the next change
increases the stackguard.
Change-Id: Ib2b4f24f0649eb1d13e3a58d265f13d1b6cc9bf9
Reviewed-on: https://go-review.googlesource.com/15964
Reviewed-by: Russ Cox <rsc@golang.org>
Larger stack frames mean nosplit functions use more stack and so the limit
needs to increase.
The change to test/nosplit.go is a bit ugly but I can't really think of a
way to make it nicer.
Change-Id: I2616b58015f0b62abbd62951575fcd0d2d8643c2
Reviewed-on: https://go-review.googlesource.com/16504
Reviewed-by: Russ Cox <rsc@golang.org>
Apparently its last use was removed in CL 8899.
Change-Id: I4f3a789b3cc4c249582e81463af62b576a281e40
Reviewed-on: https://go-review.googlesource.com/16880
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The new revision is 389d49d4943780efbfcd2a434f4462b6d0f23c44 (Nov 13, 2015).
The runtimes are built using the new x/build/cmd/racebuild utility.
This update fixes a bug in race detection algorithm that can
lead to occasional false negatives (#10589). But generally just
brings in an up-to-date runtime.
Update #8653Fixes#10589
Change-Id: I7ac9614d014ee89c2302ce5e096d326ef293f367
Reviewed-on: https://go-review.googlesource.com/16827
Reviewed-by: Keith Randall <khr@golang.org>
As per mdempsky's comment on golang.org/cl/14204, textflag.h is
copied to the includes dir by cmd/dist, and the copy in
runtime/internal/atomic is not actually being used.
Updates #11647
Change-Id: Ie95c08903a9df54cea4c70ee9d5291176f7b5609
Reviewed-on: https://go-review.googlesource.com/16871
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Somehow these were left out of the orignial CL.
Updates #11647
Change-Id: I058a30eaa25fbb72d60e7fb6bc9ff0a3b54fdb2a
Reviewed-on: https://go-review.googlesource.com/16870
Reviewed-by: Minux Ma <minux@golang.org>
