This is Part 2 of the change, see Part 1 here: in https://go-review.googlesource.com/#/c/7692/
Suggested by iant@, we use the library initialization entry point to:
- create a new OS thread and run the "regular" runtime init stack on
that thread
- return immediately from the main (i.e., loader) thread
- at the first CGO invocation, we wait for the runtime initialization
to complete.
The above mechanism is implemented only on linux_amd64. Next step is to
support it on linux_arm. Other platforms don't yet support shared library
compiling/linking, but we intend to use the same strategy there as well.
Change-Id: Ib2c81b1b83bee837134084b75a3beecfb8de6bf4
Reviewed-on: https://go-review.googlesource.com/8094
Run-TryBot: Srdjan Petrovic <spetrovic@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This tracks both total CPU time used by GC and the total time
available to all Ps since the beginning of the program and uses this
to derive a cumulative CPU usage percent for the gctrace line.
Change-Id: Ica85372b8dd45f7621909b325d5ac713a9b0d015
Reviewed-on: https://go-review.googlesource.com/8350
Reviewed-by: Russ Cox <rsc@golang.org>
GODEBUG=gctrace=1 turns on a per-GC cycle trace line. The current line
is left over from the STW garbage collector and includes a lot of
information that is no longer meaningful for the concurrent GC and
doesn't include a lot of information that is important.
Replace this line with a new line designed for the new garbage
collector.
This new line is focused more on helping the user understand the
impact of the garbage collector on their program and less on telling
us, the runtime developers, everything that's happening inside
GC. It's designed to fit in 80 columns and intentionally omit some
potentially useful things that were in the old line. We might want a
"verbose" mode that adds information for us.
We'll be able to further simplify the line once we eliminate the STW
around enabling the write barrier. Then we'll have just one STW phase,
one concurrent phase, and one more STW phase, so we'll be able to
reduce the number of times from five to three.
Change-Id: Icc30939fe4576fb4491b4eac811649395727aa2a
Reviewed-on: https://go-review.googlesource.com/8208
Reviewed-by: Russ Cox <rsc@golang.org>
Currently hashmap is riddled with code that attempts to force a GC on
the next allocation if checkgc is set. This no longer works as
originally intended with the concurrent collector, and is apparently
no longer used anyway.
Remove checkgc.
Change-Id: Ia6c17c405fa8821dc2e6af28d506c1133ab1ca0c
Reviewed-on: https://go-review.googlesource.com/8355
Reviewed-by: Keith Randall <khr@golang.org>
This tries to clarify that Alloc and HeapAlloc are tied to how much
freeing has been done by the sweeper.
Change-Id: Id8320074bd75de791f39ec01bac99afe28052d02
Reviewed-on: https://go-review.googlesource.com/8354
Reviewed-by: Rick Hudson <rlh@golang.org>
This makes it easier to experiment with alternative implementations.
While we're here, update the comments.
No functional changes. Passes toolstash -cmp.
Change-Id: I428535754908f0fdd7cc36c214ddb6e1e60f376e
Reviewed-on: https://go-review.googlesource.com/8310
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In preparation for being able to run a go program that has code
in several objects, this changes from having several linker
symbols used by the runtime into having one linker symbol that
points at a structure containing the needed data. Multiple
object support will construct a linked list of such structures.
A follow up will initialize the slices in the themoduledata
structure directly from the linker but I was aiming for a minimal
diff for now.
Change-Id: I613cce35309801cf265a1d5ae5aaca8d689c5cbf
Reviewed-on: https://go-review.googlesource.com/7441
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, gcDrainN is documented saying that it must be run on the
system stack. In fact, the problem and solution here are somewhat
subtler. First, it doesn't have to happen on the system stack, it just
has to be non-stoppable (that is, non-preemptible). Second, this isn't
specific to gcDrainN (though gcDrainN is perhaps the most surprising
instance); it's general to anything that uses the gcWork structure.
Move the comment to gcWork and generalize it.
Change-Id: I5277b5abb070e47f8d783bc15a310b379c6adc22
Reviewed-on: https://go-review.googlesource.com/8247
Reviewed-by: Rick Hudson <rlh@golang.org>
gcDrain used to be passed a *workbuf to start draining from, but now
it takes a gcWork, which hides whether or not there's an initial
workbuf. Update the comment to match this.
Change-Id: I976b58e5bfebc451cfd4fa75e770113067b5cc07
Reviewed-on: https://go-review.googlesource.com/8246
Reviewed-by: Rick Hudson <rlh@golang.org>
Being able to printer pointers to strings means one will able to output
the result of things like the flag library and other components that use
string pointers.
While here, adjusted the tests for gdb to test original string pretty
printing as well as pointers to them. It was doing it via the map before
but for completeness this ensures it's tested as a unit.
Change-Id: I4926547ae4fa6c85ef74301e7d96d49ba4a7b0c6
Reviewed-on: https://go-review.googlesource.com/8217
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
runtime·main·f is normalized by the linker to runtime.main.f, as is
the compiler-generated symbol runtime.main·f. Change the former to
runtime·mainPC instead.
Fixes issue #9934
Change-Id: I656a6fa6422d45385fa2cc55bd036c6affa1abfe
Reviewed-on: https://go-review.googlesource.com/8234
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Extend escape analysis to convT2E and conT2I. If the interface value
does not escape supply runtime with a stack buffer for the object copy.
This is a straight port from .c to .go of Dmitry's patch
Change-Id: Ic315dd50d144d94dd3324227099c116be5ca70b6
Reviewed-on: https://go-review.googlesource.com/8201
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Currently, various functions are marked with the comment
// May run without a P, so write barriers are not allowed.
However, "running without a P" is ambiguous. We intended these to mean
that m.p may be nil (which is the condition checked by the write
barrier). The comment could also be taken to mean that a
stop-the-world may happen, which is not the case for these functions
because they run in situations where there is in fact a function on
the stack holding a P locally, it just isn't in m.p.
Change these comments to state precisely what we mean, that m.p may be
nil.
Change-Id: I4a4a1d26aebd455e5067540e13b9f96a7482146c
Reviewed-on: https://go-review.googlesource.com/8209
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
When Windows Error Reporting dialog is disabled on amd64
Windows XP or 2003, the continue handler does not fire. Newer
versions work correctly regardless of WER.
Fixes#10162
Change-Id: I84ea36ee188b34d1421a8db6231223cf61b4111b
Reviewed-on: https://go-review.googlesource.com/8165
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Racy tests do not fail currently, they do os.Exit(0).
So if you run go test without -v, you won't even notice.
This was probably introduced with testing.TestMain.
Racy programs do not have the right to finish successfully.
Change-Id: Id133d7424f03d90d438bc3478528683dd02b8846
Reviewed-on: https://go-review.googlesource.com/4371
Reviewed-by: Russ Cox <rsc@golang.org>
Suggested by iant@, this change:
- looks for a symbol _rt0_<GOARCH>_<GOOS>_lib,
- if the symbol is present, adds a new entry into the .init_array ELF
section that points to the symbol.
The end-effect is that the symbol _rt0_<GOARCH>_<GOOS>_lib will be
invoked as soon as the (ELF) shared library is loaded, which will in turn
initialize the runtime. (To be implemented.)
Change-Id: I99911a180215a6df18f8a18483d12b9b497b48f4
Reviewed-on: https://go-review.googlesource.com/7692
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
1) Large allocation in this test caused crash. This was not
detected by builder because builder runs tests with -test.short.
2) The command "go" for forking doesn't exist in some platforms
including android. This change uses the test binary itself which
is guaranteed to exist.
This change also adds logging of the total samples collected in
TestCPUProfileMultithreaded test that is flaky in android-arm
builder.
Change-Id: I225c6b7877d811edef8b25e7eb00559450640c42
Reviewed-on: https://go-review.googlesource.com/8131
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
handoffp by definition runs without a P, so it's not allowed to have
write barriers. It doesn't have any right now, but mark it
nowritebarrier to disallow any creeping in in the future. handoffp in
turns calls startm, newm, and newosproc, all of which are "below Go"
and make sense to run without a P, so disallow write barriers in these
as well.
For most functions, we've done this because they may race with
stoptheworld() and hence must not have write barriers. For these
functions, it's a little different: the world can't stop while we're
in handoffp, so this race isn't present. But we implement this
restriction with a somewhat broader rule that you can't have a write
barrier without a P. We like this rule because it's simple and means
that our write barriers can depend on there being a P, even though
this rule is actually a little broader than necessary. Hence, even
though there's no danger of the race in these functions, we want to
adhere to the broader rule.
Change-Id: Ie22319c30eea37d703eb52f5c7ca5da872030b88
Reviewed-on: https://go-review.googlesource.com/8130
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This should fix the intermittent calling write barrier with mp.p == nil
failures on the nacl/386 builder.
Change-Id: I34aef5ca75ccd2939e6a6ad3f5dacec64903074e
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7973
Reviewed-by: Austin Clements <austin@google.com>
Currently, Darwin's siginfo type uses *byte for the si_addr
field. This results in unwanted write barriers in set_sigaddr. It's
also pointless since it never points to anything real and the get/set
methods return/take uintXX and cast it from/to the pointer.
All other arches use a uint type for this field. Change Darwin to
match. This simplifies the get/set methods and eliminates the unwanted
write barriers.
Change-Id: Ifdb5646d35e1f2f6808b87a3d59745ec9718add1
Reviewed-on: https://go-review.googlesource.com/8086
Reviewed-by: Austin Clements <austin@google.com>
sighandler may run during a stop-the-world without a P, so it's not
allowed to have write barriers. Fix the G write to disable the write
barrier (this is safe because the G is reachable from allgs) and mark
the function nowritebarrier.
Change-Id: I907f05d3829e24eeb15fa4d020598af36710e87e
Reviewed-on: https://go-review.googlesource.com/8020
Reviewed-by: Rick Hudson <rlh@golang.org>
Also invert it, which means it no longer needs to cross the cgo
package boundary.
Change-Id: I393cd073bda02b591a55d6bc6b8bb94970ea71cd
Reviewed-on: https://go-review.googlesource.com/8082
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We do not use SEH to handle Windows exception anymore.
Change-Id: I0ac807a0fed7a5b4c745454246764c524460472b
Reviewed-on: https://go-review.googlesource.com/8071
Reviewed-by: Minux Ma <minux@golang.org>
The SecureRandom named service was removed in
https://codereview.chromium.org/550523002. And the new syscall
was introduced in https://codereview.chromium.org/537543003.
Accepting this will remove the support for older version of
sel_ldr. I've confirmed that both pepper_40 and current
pepper_canary have this syscall.
After this change, we need sel_ldr from pepper_39 or above to
work.
Fixes#9261
Change-Id: I096973593aa302ade61f259a3a71ebc7c1a57913
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/1755
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Also fixes a long-existing problem in the fork/exec path.
Change-Id: Idec40b1cee0cfb1625fe107db3eafdc0d71798f2
Reviewed-on: https://go-review.googlesource.com/8030
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Previously the extra m needed for cgo callbacks was created on the
first callback. This works for cgo, however the cgocallback mechanism
is also borrowed by badsignal which can run before any cgo calls are
made.
Now we initialize the extra M at runtime startup before any signal
handlers are registered, so badsignal cannot be called until the
extra M is ready.
Updates #10207.
Change-Id: Iddda2c80db6dc52d8b60e2b269670fbaa704c7b3
Reviewed-on: https://go-review.googlesource.com/7978
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
There are calls to stdcall when the GC thinks the world is stopped
and stdcall write a *g for the CPU profiler. This produces a write
barrier but the GC is not prepared to deal with write barriers when
it thinks the world is stopped. Since the g is on allg it does not
need a write barrier to keep it alive so eliminate the write barrier.
Change-Id: I937633409a66553d7d292d87d7d58caba1fad0b6
Reviewed-on: https://go-review.googlesource.com/7979
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Rick Hudson <rlh@golang.org>
The test is a simple reproduction of issue 9356.
Update #8948.
Update #9356.
Change-Id: Ia77bc36d12ed0c3c4a8b1214cade8be181c9ad55
Reviewed-on: https://go-review.googlesource.com/7618
Reviewed-by: Minux Ma <minux@golang.org>
windows/386 also wants underscore prefix for external names.
This CL is in preparation of external linking support.
Change-Id: I2d2ea233f976aab3f356f9b508cdd246d5013e2d
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7282
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
When external linking, we must link to implib provided by mingw, so we must use
properly decorated names for stdcalls.
Because the feature is only used in the runtime, I've designed a new decoration
scheme so that we can use the same decorated name for both 386 and amd64.
A stdcall function named FooEx from bar16.dll which takes 3 parameters will be
imported like this:
//go:cgo_import_dynamic runtime._FooEx FooEx%3 "bar16.dll"
Depending on the size of uintptr, the linker will later transform it to _FooEx@12
or _FooEx@24.
This is in prepration for the next CL that adds external linking support for
windows/386.
Change-Id: I2d2ea233f976aab3f356f9b508cdd246d5013e2c
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/7163
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
On Unix, when placing a child in a new process group, allow that group
to become the foreground process group. Also, allow a child process to
join a specific process group.
When setting the foreground process group, Ctty is used as the file
descriptor of the controlling terminal. Ctty has been added to the BSD
and Solaris SysProcAttr structures and the handling of Setctty changed
to match Linux.
Change-Id: I18d169a6c5ab8a6a90708c4ff52eb4aded50bc8c
Reviewed-on: https://go-review.googlesource.com/5130
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Implement runtime.atomicand8 for amd64p32 which was overlooked
in CL 7861.
Change-Id: Ic7eccddc6fd6c4682cac1761294893928f5428a2
Reviewed-on: https://go-review.googlesource.com/7920
Reviewed-by: Minux Ma <minux@golang.org>
These can be implemented with just a compare and a move instruction.
Do so, avoiding the overhead of a call into the runtime.
These assertions are a significant cost in Go code that uses interface{}
as a safe alternative to C's void* (or unsafe.Pointer), such as the
current version of the Go compiler.
*T here includes pointer to T but also any Go type represented as
a single pointer (chan, func, map). It does not include [1]*T or struct{*int}.
That requires more work in other parts of the compiler; there is a TODO.
Change-Id: I7ff681c20d2c3eb6ad11dd7b3a37b1f3dda23965
Reviewed-on: https://go-review.googlesource.com/7862
Reviewed-by: Rob Pike <r@golang.org>
Currently, we only exit the getfull barrier if there is work on the
full list, even though the exit path will take work from either the
full or partial list. Change this to exit the barrier if there is work
on either the full or partial lists.
I believe it's currently safe to check only the full list, since
during mark termination there is no reason to put a workbuf on a
partial list. However, checking both is more robust.
Change-Id: Icf095b0945c7cad326a87ff2f1dc49b7699df373
Reviewed-on: https://go-review.googlesource.com/7840
Reviewed-by: Rick Hudson <rlh@golang.org>
The barrier in gcDrain does not account for concurrent gcDrainNs
happening in gchelpwork, so it can actually return while there is
still work being done. It turns out this is okay, but for subtle
reasons involving gcDrainN always being run on the system
stack. Document these reasons.
Change-Id: Ib07b3753cc4e2b54533ab3081a359cbd1c3c08fb
Reviewed-on: https://go-review.googlesource.com/7736
Reviewed-by: Rick Hudson <rlh@golang.org>
Make mask uint32, and move down one line to match atomic_arm64.go.
Change-Id: I4867de494bc4076b7c2b3bf4fd74aa984e3ea0c8
Reviewed-on: https://go-review.googlesource.com/7854
Reviewed-by: Russ Cox <rsc@golang.org>
We're skating on thin ice, and things are finally starting to melt around here.
(I want to avoid the debugging session that will happen when someone
uses atomicand8 expecting it to be atomic with respect to other operations.)
Change-Id: I254f1582be4eb1f2d7fbba05335a91c6bf0c7f02
Reviewed-on: https://go-review.googlesource.com/7861
Reviewed-by: Minux Ma <minux@golang.org>
To reduce lock contention in this mode, makes persistent allocation state per-P,
which means at most 64 kB overhead x $GOMAXPROCS, which should be
completely tolerable.
Change-Id: I34ca95e77d7e67130e30822e5a4aff6772b1a1c5
Reviewed-on: https://go-review.googlesource.com/7740
Reviewed-by: Rick Hudson <rlh@golang.org>
Some type assertions of the form _, ok := i.(T) allow efficient inlining.
Such type assertions commonly show up in type switches.
For example, with this optimization, using 6g, the length of
encoding/binary's intDataSize function shrinks from 2224 to 1728 bytes (-22%).
benchmark old ns/op new ns/op delta
BenchmarkAssertI2E2Blank 4.67 0.82 -82.44%
BenchmarkAssertE2T2Blank 4.38 0.83 -81.05%
BenchmarkAssertE2E2Blank 3.88 0.83 -78.61%
BenchmarkAssertE2E2 14.2 14.4 +1.41%
BenchmarkAssertE2T2 10.3 10.4 +0.97%
BenchmarkAssertI2E2 13.4 13.3 -0.75%
Change-Id: Ie9798c3e85432bb8e0f2c723afc376e233639df7
Reviewed-on: https://go-review.googlesource.com/7697
Reviewed-by: Keith Randall <khr@golang.org>
The distinction between gcWorkProducer and gcWork (producer and
consumer) is not serving us as originally intended, so merge these
into just gcWork.
The original intent was to replace the currentwbuf cache with a
gcWorkProducer. However, with gchelpwork (aka mutator assists),
mutators can both produce and consume work, so it will make more sense
to cache a whole gcWork.
Change-Id: I6e633e96db7cb23a64fbadbfc4607e3ad32bcfb3
Reviewed-on: https://go-review.googlesource.com/7733
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently markroot fetches the wbuf to fill from the per-M wbuf
cache. The wbuf cache is primarily meant for the write barrier because
it produces very little work on each call. There's little point to
using the cache in mark root, since each call to markroot is likely to
produce a large amount of work (so the slight win on getting it from
the cache instead of from the central wbuf lists doesn't matter), and
markroot does not dispose the wbuf back to the cache (so most markroot
calls won't get anything from the wbuf cache anyway).
Instead, just get the wbuf from the central wbuf lists like other work
producers. This will simplify later changes.
Change-Id: I07a18a4335a41e266a6d70aa3a0911a40babce23
Reviewed-on: https://go-review.googlesource.com/7732
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, the GC's concurrent mark phase runs on the system
stack. There's no need to do this, and running it this way ties up the
entire M and P running the GC by preventing the scheduler from
preempting the GC even during concurrent mark.
Fix this by running concurrent mark on the regular G stack. It's still
non-preemptible because we also set preemptoff around the whole GC
process, but this moves us closer to making it preemptible.
Change-Id: Ia9f1245e299b8c5c513a4b1e3ef13eaa35ac5e73
Reviewed-on: https://go-review.googlesource.com/7730
Reviewed-by: Rick Hudson <rlh@golang.org>
"Sync" is not very informative. What's being synchronized and with
whom? Update this comment to explain what we're really doing: enabling
write barriers.
Change-Id: I4f0cbb8771988c7ba4606d566b77c26c64165f0f
Reviewed-on: https://go-review.googlesource.com/7700
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently we harvestwbufs the moment we enter the mark phase, even
before starting the world again. Since cached wbufs are only filled
when we're in mark or mark termination, they should all be empty at
this point, making the harvest pointless. Remove the harvest.
We should, but do not currently harvest at the end of the mark phase
when we're running out of work to do.
Change-Id: I5f4ba874f14dd915b8dfbc4ee5bb526eecc2c0b4
Reviewed-on: https://go-review.googlesource.com/7669
Reviewed-by: Rick Hudson <rlh@golang.org>
