Currently, isSystemGoroutine varies on whether it considers the
finalizer goroutine a user goroutine or a system goroutine. For the
next CL, we're going to want to always consider the finalier goroutine
a user goroutine, so add a flag that indicates that.
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Change-Id: Iafc92e519c13d9f8d879332cb5f0d12164104c33
Reviewed-on: https://go-review.googlesource.com/c/134778
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, setting GODEBUG=gcrescanstacks=1 enables a debugging mode
where the garbage collector re-scans goroutine stacks during mark
termination. This was introduced in Go 1.8 to debug the hybrid write
barrier, but I don't think we ever used it.
Now it's one of the last sources of mark work during mark termination.
This CL removes it.
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Updates #17503.
Change-Id: I6ae04d3738aa9c448e6e206e21857a33ecd12acf
Reviewed-on: https://go-review.googlesource.com/c/134777
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, checkmarks mode uses the full STW GC infrastructure to
perform mark checking. We're about to remove that infrastructure and,
furthermore, since checkmarks is about doing the simplest thing
possible to check concurrent GC, it's valuable for it to be simpler.
Hence, this CL makes checkmarks even simpler by making it non-parallel
and divorcing it from the STW GC infrastructure (including the
gchelper mechanism).
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Change-Id: Iad21158123e025e3f97d7986d577315e994bd43e
Reviewed-on: https://go-review.googlesource.com/c/134776
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This argument is always gcBackgroundMode since only
debug.gcstoptheworld can trigger a STW GC at this point. Remove the
unnecessary argument.
Updates #26903. This is preparation for unifying STW GC and concurrent
GC.
Change-Id: Icb4ba8f10f80c2b69cf51a21e04fa2c761b71c94
Reviewed-on: https://go-review.googlesource.com/c/134775
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, we disable GC work caching during mark termination. This is
no longer necessary with the new mark completion detection because
1. There's no way for any of the GC mark termination helpers to have
any real work queued and,
2. Mark termination has to explicitly flush every P's buffers anyway
in order to flush Ps that didn't run a GC mark termination helper.
Hence, remove the code that disposes gcWork buffers during mark
termination.
Updates #26903. This is a follow-up to eliminating mark 2.
Change-Id: I81f002ee25d5c10f42afd39767774636519007f9
Reviewed-on: https://go-review.googlesource.com/c/134320
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Now that there is no mark 2 phase, gcBlackenPromptly is no longer
used.
Updates #26903. This is a follow-up to eliminating mark 2.
Change-Id: Ib9c534f21b36b8416fcf3cab667f186167b827f8
Reviewed-on: https://go-review.googlesource.com/c/134319
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
The mark 2 phase was originally introduced as a way to reduce the
chance of entering STW mark termination while there was still marking
work to do. It works by flushing and disabling all local work caches
so that all enqueued work becomes immediately globally visible.
However, mark 2 is not only slow–disabling caches makes marking and
the write barrier both much more expensive–but also imperfect. There
is still a rare but possible race (~once per all.bash) that can cause
GC to enter mark termination while there is still marking work. This
race is detailed at
https://github.com/golang/proposal/blob/master/design/17503-eliminate-rescan.md#appendix-mark-completion-race
The effect of this is that mark termination must still cope with the
possibility that there may be work remaining after a concurrent mark
phase. Dealing with this increases STW pause time and increases the
complexity of mark termination.
Furthermore, a similar but far more likely race can cause early
transition from mark 1 to mark 2. This is unfortunate because it
causes performance instability because of the cost of mark 2.
This CL fixes this by replacing mark 2 with a distributed termination
detection algorithm. This algorithm is correct, so it eliminates the
mark termination race, and doesn't require disabling local caches. It
ensures that there are no grey objects upon entering mark termination.
With this change, we're one step closer to eliminating marking from
mark termination entirely (it's still used by STW GC and checkmarks
mode).
This CL does not eliminate the gcBlackenPromptly global flag, though
it is always set to false now. It will be removed in a cleanup CL.
This led to only minor variations in the go1 benchmarks
(https://perf.golang.org/search?q=upload:20180909.1) and compilebench
benchmarks (https://perf.golang.org/search?q=upload:20180910.2).
This significantly improves performance of the garbage benchmark, with
no impact on STW times:
name old time/op new time/op delta
Garbage/benchmem-MB=64-12 2.21ms ± 1% 2.05ms ± 1% -7.38% (p=0.000 n=18+19)
Garbage/benchmem-MB=1024-12 2.30ms ±16% 2.20ms ± 7% -4.51% (p=0.001 n=20+20)
name old STW-ns/GC new STW-ns/GC delta
Garbage/benchmem-MB=64-12 138k ±44% 141k ±23% ~ (p=0.309 n=19+20)
Garbage/benchmem-MB=1024-12 159k ±25% 178k ±98% ~ (p=0.798 n=16+18)
name old STW-ns/op new STW-ns/op delta
Garbage/benchmem-MB=64-12 4.42k ±44% 4.24k ±23% ~ (p=0.531 n=19+20)
Garbage/benchmem-MB=1024-12 591 ±24% 636 ±111% ~ (p=0.309 n=16+18)
(https://perf.golang.org/search?q=upload:20180910.1)
Updates #26903.
Updates #17503.
Change-Id: Icbd1e12b7a12a76f423c9bf033b13cb363e4cd19
Reviewed-on: https://go-review.googlesource.com/c/134318
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Nothing currently consumes the flag, but we'll use it in the
distributed termination detection algorithm.
Updates #26903. This is preparation for eliminating mark 2.
Change-Id: I5e149a05b1c878fe1009150da21f8bd8ae2b9b6a
Reviewed-on: https://go-review.googlesource.com/c/134317
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
It turns out if you set GODEBUG=gctrace=2, it enables an obscure
debugging mode that, in addition to printing gctrace statistics, also
does a second STW GC following each regular GC. This debugging mode
has long since lost its value (you could maybe use it to analyze
floating garbage, except that we don't print the gctrace line on the
second GC), and it interferes substantially with the operation of the
GC by messing up the statistics used to schedule GCs.
It's also a source of mark termination GC work when we're in
concurrent GC mode, so it's going to interfere with eliminating mark
2. And it's going to get in the way of unifying STW and concurrent GC.
This CL removes this debugging mode.
Updates #26903. This is preparation for eliminating mark 2 and
unifying STW GC and concurrent GC.
Change-Id: Ib5bce05d8c4d5b6559c89a65165d49532165df07
Reviewed-on: https://go-review.googlesource.com/c/134316
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, if the gcWork runs out of work, we'll fall out of the GC
worker, even though flushing the write barrier buffer could produce
more work. While this is not a correctness issue, it can lead to
premature mark 2 or mark termination.
Fix this by flushing the write barrier buffer if the local gcWork runs
out of work and then checking the local gcWork again.
This reduces the number of premature mark terminations during all.bash
by about a factor of 10.
Updates #26903. This is preparation for eliminating mark 2.
Change-Id: I48577220b90c86bfd28d498e8603bc379a8cd617
Reviewed-on: https://go-review.googlesource.com/c/134315
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Previously the MaxIdleClosed counter was incremented when added
to the free connection list, rather then when it wasn't added
to the free connection list. Flip this logic to correct.
Fixes#27792
Change-Id: I405302c14fb985369dab48fbe845e5651afc4ccf
Reviewed-on: https://go-review.googlesource.com/c/138578
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Edge supports WebAssembly but not TextEncoder or TextDecoder.
This change adds a comment pointing to a polyfill that could
be used. The polyfill is not added by default, because we want to
let the user decide if/how to include the polyfill.
Fixes#27295
Change-Id: I375f58f2168665f549997b368428c398dfbbca1c
Reviewed-on: https://go-review.googlesource.com/139037
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reverts CL 131718, commit a0e7f12771.
Reason for revert: adds request overhead & dependency on third-party service for all users regardless of whether it's necessary.
Updates #27295
Change-Id: I4a8a9b0c8e4a3198c884dfbd90ba36734f70a9a9
Reviewed-on: https://go-review.googlesource.com/138937
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently there is no example for http.Handle in the
documentation. This adds an example.
Change-Id: I66ee9983bea1f5237757e1ef4956eae9a056e963
Reviewed-on: https://go-review.googlesource.com/137715
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Recent CL 125456 implemented Unix Socket functionality on windows.
But that functionality does not appear to be working when 32-bit
code is used. So disable TestUnixConnLocalWindows.
windows/386 builder does not appear to be complaining about
TestUnixConnLocalWindows, because new functionality requires
Windows 10 Build 17063. windows/386 builder uses Windows 2008.
Fixes#27943
Change-Id: Iea91b86aaa124352d198ca0cd03fff1e7542f949
Reviewed-on: https://go-review.googlesource.com/138676
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Yasuhiro MATSUMOTO <mattn.jp@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
dump/fdump is a reflection-based data structure dumper slightly
customized for the compiler's Node data structure. It dumps the
transitivle closure of Node (and other) data structures using a
recursive descent depth first traversal and permits filtering
options (recursion depth limitation, filtering of struct fields).
I have been using it to diagnose compiler bugs and found it more
useful than the existing node printing code in some cases because
field filtering reduces the output to the interesting parts.
No impact on rest of compiler if functions are not called (which
they only should during a debugging session).
Change-Id: I79d7227f10dd78dbd4bbcdf204db236102fc97a7
Reviewed-on: https://go-review.googlesource.com/136397
Reviewed-by: Alan Donovan <adonovan@google.com>
Previously, some of output from gdb matched with literal string, while
gdb v8.2 print the address of variable (e.g. map key and value) in
output.
This commit fix the regex in testing the output.
Fixes#27608
Change-Id: Ic3fe8280b9f93fda2799116804822616caa66beb
Reviewed-on: https://go-review.googlesource.com/135055
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This reverts commit 067bb443af.
Reason for revert:
Failing Darwin-arm builds because that testing environment does not access testdata
from sibling directories. A future change will likely be made to move this testdata
out of src/testdata to create a solution that doesn't require the single-file directory.
Updates #27151
Change-Id: I8dbf5dd9512c94a605ee749ff4655cb00b0de686
Reviewed-on: https://go-review.googlesource.com/138737
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
R0 isn't the zero register any more. Oops.
Update #27695.
Change-Id: I46a975ed37d5e570afe2e228d3edf74949e08ad7
Reviewed-on: https://go-review.googlesource.com/138580
Reviewed-by: Michael Munday <mike.munday@ibm.com>
During a call to a reflect-generated function or method (via
makeFuncStub or methodValueCall), when should we scan the return
values?
When we're starting a reflect call, the space on the stack for the
return values is not initialized yet, as it contains whatever junk was
on the stack of the caller at the time. The return space must not be
scanned during a GC.
When we're finishing a reflect call, the return values are
initialized, and must be scanned during a GC to make sure that any
pointers in the return values are found and their referents retained.
When the GC stack walk comes across a reflect call in progress on the
stack, it needs to know whether to scan the results or not. It doesn't
know the progress of the reflect call, so it can't decide by
itself. The reflect package needs to tell it.
This CL adds another slot in the frame of makeFuncStub and
methodValueCall so we can put a boolean in there which tells the
runtime whether to scan the results or not.
This CL also adds the args length to reflectMethodValue so the
runtime can restrict its scanning to only the args section (not the
results) if the reflect package says the results aren't ready yet.
Do a delicate dance in the reflect package to set the "results are
valid" bit. We need to make sure we set the bit only after we've
copied the results back to the stack. But we must set the bit before
we drop reflect's copy of the results. Otherwise, we might have a
state where (temporarily) no one has a live copy of the results.
That's the state we were observing in issue #27695 before this CL.
The bitmap used by the runtime currently contains only the args.
(Actually, it contains all the bits, but the size is set so we use
only the args portion.) This is safe for early in a reflect call, but
unsafe late in a reflect call. The test issue27695.go demonstrates
this unsafety. We change the bitmap to always include both args
and results, and decide at runtime which portion to use.
issue27695.go only has a test for method calls. Function calls were ok
because there wasn't a safepoint between when reflect dropped its copy
of the return values and when the caller is resumed. This may change
when we introduce safepoints everywhere.
This truncate-to-only-the-args was part of CL 9888 (in 2015). That
part of the CL fixed the problem demonstrated in issue27695b.go but
introduced the problem demonstrated in issue27695.go.
TODO, in another CL: simplify FuncLayout and its test. stack return
value is now identical to frametype.ptrdata + frametype.gcdata.
Fixes#27695
Change-Id: I2d49b34e34a82c6328b34f02610587a291b25c5f
Reviewed-on: https://go-review.googlesource.com/137440
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Edge supports WASM but not TextEncoder or TextDecoder.
This PR adds a polyfill to `misc/wasm/wasm_exec.js` to fix this.
Fixes#27295
Change-Id: Ie35ee5604529b170a5dc380eb286f71bdd691d3e
GitHub-Last-Rev: a587edae28
GitHub-Pull-Request: golang/go#27296
Reviewed-on: https://go-review.googlesource.com/131718
Reviewed-by: Agniva De Sarker <agniva.quicksilver@gmail.com>
Reviewed-by: Richard Musiol <neelance@gmail.com>
SameFile opens file to discover identifier and volume serial
number that uniquely identify the file. SameFile uses Windows
CreateFile API to open the file, and that works well for files
and directories. But CreateFile always follows symlinks, so
SameFile always opens symlink target instead of symlink itself.
This CL uses FILE_FLAG_OPEN_REPARSE_POINT flag to adjust
CreateFile behavior when handling symlinks.
As per https://docs.microsoft.com/en-us/windows/desktop/FileIO/symbolic-link-effects-on-file-systems-functions#createfile-and-createfiletransacted
"... If FILE_FLAG_OPEN_REPARSE_POINT is specified and:
If an existing file is opened and it is a symbolic link, the handle
returned is a handle to the symbolic link. ...".
I also added new tests for both issue #21854 and #27225.
Issue #27225 is still to be fixed, so skipping the test on
windows for the moment.
Fixes#21854
Updates #27225
Change-Id: I8aaa13ad66ce3b4074991bb50994d2aeeeaa7c95
Reviewed-on: https://go-review.googlesource.com/134195
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This text is used mainly for benchmark compression testing, and in one
net test. The text was prevoiusly in a src/testdata directory, but since
that directory would only include one file, the text is moved to the
existing src/compression/testdata directory.
This does not cause any change to the benchmark results.
Updates #27151
Change-Id: I38ab5089dfe744189a970947d15be50ef1d48517
Reviewed-on: https://go-review.googlesource.com/138495
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
mcache.refill doesn't need to run on the system stack; it just needs
to be non-preemptible. Its only caller, mcache.nextFree, also needs to
be non-preemptible, so we can remove the unnecessary systemstack
switch.
Change-Id: Iba5b3f4444855f1dc134485ba588efff3b54c426
Reviewed-on: https://go-review.googlesource.com/138196
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
mcache.refill acquires g.m.locks, which is pointless because the
caller itself absolutely must have done so already to prevent
ownership of mcache from shifting.
Also, mcache.refill's documentation is generally a bit out-of-date, so
this cleans this up.
Change-Id: Idc8de666fcaf3c3d96006bd23a8f307539587d6c
Reviewed-on: https://go-review.googlesource.com/138195
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This hasn't been true at least since 1.4. Until golang.org/cl/137235
they were lumped together into a random compile unit, now they are
assigned to the correct one.
Change-Id: Ib66539bd67af3e9daeecac8bf5f32c10e62e11b1
Reviewed-on: https://go-review.googlesource.com/138415
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
This is the first commit of a series that will add AIX as an
operating system target for ppc64 architecture.
Updates #25893
Change-Id: I865b67a9c98277c11c1a56107be404ac5253277d
Reviewed-on: https://go-review.googlesource.com/138115
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A simple grep over the codebase for "the the" which is often
missed by humans.
Change-Id: Ie4b4f07abfc24c73dcd51c8ef1edf4f73514a21c
Reviewed-on: https://go-review.googlesource.com/138335
Reviewed-by: Dave Cheney <dave@cheney.net>
This benchmark is odd currently because it uses inconsistent cases
between benchmark iterations, and each iteration actually does a bit of
testing.
This separates the two benchmark cases into two separate benchmarks and
removes the testing done on each iteration. The unit tests above
suffice.
The benchmark being more succinct will make it easier to gauge the
benefits of any future MIME header reading changes.
Change-Id: I2399fab28067f1aeec3d9b16951d39d787f8b39c
Reviewed-on: https://go-review.googlesource.com/134225
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ending a loop with a break is confusing. Rewrite the loop so the
default behavior is to loop and then do the "post-loop" work outside
of the loop.
Change-Id: Ie49b4132541dfb5124c31a8163f2c883aa4abc75
Reviewed-on: https://go-review.googlesource.com/138155
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
When go resolver was changed to use dnsmessage.Parser, LookupTXT
returned two strings in one record as two different records. This change
reverts back to concatenating multiple strings in a single
TXT record.
Fixes#27763
Change-Id: Ice226fcb2be4be58853de34ed35b4627acb429ea
Reviewed-on: https://go-review.googlesource.com/136955
Reviewed-by: Ian Gudger <igudger@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Gudger <igudger@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Also includes a small tweak to test/run.go to allow package names
with Unicode letters (as opposed to just ASCII chars).
Updates #27836
Change-Id: Idbf0bdea24174808cddcb69974dab820eb13e521
Reviewed-on: https://go-review.googlesource.com/138075
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The DIEs for global variables were all assigned to the first emitted
compile unit in debug_info, regardless of what it was. Move them
instead to their respective compile units.
Change-Id: If794fa0ba4702f5b959c6e8c16119b16e7ecf6d8
Reviewed-on: https://go-review.googlesource.com/137235
Reviewed-by: Than McIntosh <thanm@google.com>
Follow-up on https://go-review.googlesource.com/c/go/+/137857/4
which didn't remove this test file after it was removed from the
list of importer tests in importer_test.go.
Change-Id: Ib89cb3a6d976115da42c33443529ea27bd1ce838
Reviewed-on: https://go-review.googlesource.com/137975
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
ggcgo's export format numbers types consecutively, starting at 1.
This makes it trivially possible to use a slice (list) instead of
map for the internal types map.
Change-Id: Ib7814d7fabffac0ad2b56f04a5dad7d6d4c4dd0e
Reviewed-on: https://go-review.googlesource.com/137935
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
