Austin Clements b49b71ae19 runtime: don't rescan globals ... Currently the runtime rescans globals during mark 2 and mark termination. This costs as much as 500µs/MB in STW time, which is enough to surpass the 10ms STW limit with only 20MB of globals. It's also basically unnecessary. The compiler already generates write barriers for global -> heap pointer updates and the regular write barrier doesn't check whether the slot is a global or in the heap. Some less common write barriers do cause problems. heapBitsBulkBarrier, which is used by typedmemmove and related functions, currently depends on having access to the pointer bitmap and as a result ignores writes to globals. Likewise, the reflect-related write barriers reflect_typedmemmovepartial and callwritebarrier ignore non-heap destinations; though it appears they can never be called with global pointers anyway. This commit makes heapBitsBulkBarrier issue write barriers for writes to global pointers using the data and BSS pointer bitmaps, removes the inheap checks from the reflection write barriers, and eliminates the rescans during mark 2 and mark termination. It also adds a test that writes to globals have write barriers. Programs with large data+BSS segments (with pointers) aren't common, but for programs that do have large data+BSS segments, this significantly reduces pause time: name \ 95%ile-time/markTerm old new delta LargeBSS/bss:1GB/gomaxprocs:4 148200µs ± 6% 302µs ±52% -99.80% (p=0.008 n=5+5) This very slightly improves the go1 benchmarks: name old time/op new time/op delta BinaryTree17-12 2.62s ± 3% 2.62s ± 4% ~ (p=0.904 n=20+20) Fannkuch11-12 2.15s ± 1% 2.13s ± 0% -1.29% (p=0.000 n=18+20) FmtFprintfEmpty-12 48.3ns ± 2% 47.6ns ± 1% -1.52% (p=0.000 n=20+16) FmtFprintfString-12 152ns ± 0% 152ns ± 1% ~ (p=0.725 n=18+18) FmtFprintfInt-12 150ns ± 1% 149ns ± 1% -1.14% (p=0.000 n=19+20) FmtFprintfIntInt-12 250ns ± 0% 244ns ± 1% -2.12% (p=0.000 n=20+18) FmtFprintfPrefixedInt-12 219ns ± 1% 217ns ± 1% -1.20% (p=0.000 n=19+20) FmtFprintfFloat-12 280ns ± 0% 281ns ± 1% +0.47% (p=0.000 n=19+19) FmtManyArgs-12 928ns ± 0% 923ns ± 1% -0.53% (p=0.000 n=19+18) GobDecode-12 7.21ms ± 1% 7.24ms ± 2% ~ (p=0.091 n=19+19) GobEncode-12 6.07ms ± 1% 6.05ms ± 1% -0.36% (p=0.002 n=20+17) Gzip-12 265ms ± 1% 265ms ± 1% ~ (p=0.496 n=20+19) Gunzip-12 39.6ms ± 1% 39.3ms ± 1% -0.85% (p=0.000 n=19+19) HTTPClientServer-12 74.0µs ± 2% 73.8µs ± 1% ~ (p=0.569 n=20+19) JSONEncode-12 15.4ms ± 1% 15.3ms ± 1% -0.25% (p=0.049 n=17+17) JSONDecode-12 53.7ms ± 2% 53.0ms ± 1% -1.29% (p=0.000 n=18+17) Mandelbrot200-12 3.97ms ± 1% 3.97ms ± 0% ~ (p=0.072 n=17+18) GoParse-12 3.35ms ± 2% 3.36ms ± 1% +0.51% (p=0.005 n=18+20) RegexpMatchEasy0_32-12 72.7ns ± 2% 72.2ns ± 1% -0.70% (p=0.005 n=19+19) RegexpMatchEasy0_1K-12 246ns ± 1% 245ns ± 0% -0.60% (p=0.000 n=18+16) RegexpMatchEasy1_32-12 72.8ns ± 1% 72.5ns ± 1% -0.37% (p=0.011 n=18+18) RegexpMatchEasy1_1K-12 380ns ± 1% 385ns ± 1% +1.34% (p=0.000 n=20+19) RegexpMatchMedium_32-12 115ns ± 2% 115ns ± 1% +0.44% (p=0.047 n=20+20) RegexpMatchMedium_1K-12 35.4µs ± 1% 35.5µs ± 1% ~ (p=0.079 n=18+19) RegexpMatchHard_32-12 1.83µs ± 0% 1.80µs ± 1% -1.76% (p=0.000 n=18+18) RegexpMatchHard_1K-12 55.1µs ± 0% 54.3µs ± 1% -1.42% (p=0.000 n=18+19) Revcomp-12 386ms ± 1% 381ms ± 1% -1.14% (p=0.000 n=18+18) Template-12 61.5ms ± 2% 61.5ms ± 2% ~ (p=0.647 n=19+20) TimeParse-12 338ns ± 0% 336ns ± 1% -0.72% (p=0.000 n=14+19) TimeFormat-12 350ns ± 0% 357ns ± 0% +2.05% (p=0.000 n=19+18) [Geo mean] 55.3µs 55.0µs -0.41% Change-Id: I57e8720385a1b991aeebd111b6874354308e2a6b Reviewed-on: https://go-review.googlesource.com/20829 Run-TryBot: Austin Clements <austin@google.com> Reviewed-by: Rick Hudson <rlh@golang.org>		2016-04-27 18:48:16 +00:00
..
bench	cmd/compile: enable const division for arm64	2016-04-27 17:47:49 +00:00
bugs	test: migrate remaining tests to run.go	2014-12-22 22:41:34 +00:00
chan	Fix several out of date references to 4g/5g/6g/8g/9g.	2015-06-26 03:38:21 +00:00
ddd2.dir
dwarf
fixedbugs	cmd/compile: treat empty and absent struct field tags as identical	2016-04-25 21:28:45 +00:00
import2.dir
import4.dir	test: adjust gccgo expected import errors	2015-10-07 14:37:44 +00:00
interface	cmd/internal/gc: improve "type *X has no field or method M" message	2015-05-07 16:21:57 +00:00
intrinsic.dir	cmd/compile: added some intrinsics to SSA back end	2016-03-28 16:29:59 +00:00
ken	all: fix spelling mistakes	2016-04-03 17:03:15 +00:00
method4.dir
safe
stress	build: adjustments for move from src/pkg to src	2014-09-08 00:06:45 -04:00
syntax	cmd/compile: use bufio.Reader directly in lexer	2016-03-11 21:27:23 +00:00
64bit.go
235.go
alg.go	test: add test that required algs get generated	2016-04-24 21:36:52 +00:00
alias1.go
alias.go
append.go
args.go
assign1.go
assign.go	cmd/gc: prohibit short variable declarations containing duplicate symbols	2014-10-06 17:16:39 -04:00
atomicload.go	cmd/compile: keep value use counts in SSA	2016-03-17 04:20:02 +00:00
bigalg.go
bigmap.go
blank1.go	cmd/compile: don't allow blank method declarations on builtins	2015-12-02 18:26:38 +00:00
blank.go
bom.go
bombad.go
bounds.go
chancap.go
chanlinear.go	runtime: test for linear enqueue/dequeue behavior	2014-12-08 22:18:17 +00:00
char_lit1.go
char_lit.go
checkbce.go	cmd/compile: bce when max and limit are consts	2016-04-11 16:01:22 +00:00
clearfat.go	[dev.power64] 9g: fix under-zeroing in clearfat	2014-10-31 11:08:27 -04:00
closedchan.go
closure1.go	cmd/gc: capture variables by value	2015-01-29 13:07:30 +00:00
closure2.go	cmd/gc: fix capturing by value for range statements	2015-02-03 15:48:48 +00:00
closure.go
cmp6.go
cmp.go	Fix several out of date references to 4g/5g/6g/8g/9g.	2015-06-26 03:38:21 +00:00
cmplx.go
cmplxdivide1.go
cmplxdivide.c	test: comment the behavior and use of cmplxdivide*	2015-01-15 00:00:06 +00:00
cmplxdivide.go	test: comment the behavior and use of cmplxdivide*	2015-01-15 00:00:06 +00:00
complit1.go	cmd/internal/gc: accept map literals with omitted key type	2015-06-04 02:31:38 +00:00
complit.go
compos.go
const1.go
const2.go
const3.go
const4.go	all: link to https instead of http	2015-07-11 14:36:33 +00:00
const5.go	all: link to https instead of http	2015-07-11 14:36:33 +00:00
const6.go
const.go	cmd/compile: allow huge rsh in constants arithmetic	2015-08-21 20:27:22 +00:00
convert1.go
convert3.go
convert.go
convlit1.go
convlit.go	cmd/internal/gc: unsafe.Pointer constants may only be converted to uintptr	2015-05-07 23:54:28 +00:00
convT2X.go
copy.go
crlf.go
ddd1.go
ddd2.go
ddd.go
decl.go
declbad.go
defer.go
deferfin.go
deferprint.go
deferprint.out
divide.go
divmod.go
empty.go
env.go
eof1.go
eof.go
errchk	Fix several out of date references to 4g/5g/6g/8g/9g.	2015-06-26 03:38:21 +00:00
escape2.go	cmd/compile: better modeling of escape across loop levels	2016-01-13 04:01:00 +00:00
escape2n.go	cmd/compile: better modeling of escape across loop levels	2016-01-13 04:01:00 +00:00
escape3.go
escape4.go	cmd/internal/gc: transitive inlining	2015-02-26 17:36:00 +00:00
escape5.go	cmd/internal/gc: move check for large-hence-heap-allocated types into escape analysis	2015-05-22 02:13:54 +00:00
escape_array.go	cmd/internal/gc: move check for large-hence-heap-allocated types into escape analysis	2015-05-22 02:13:54 +00:00
escape_because.go	cmd/compile: escape analysis explanations added to -m -m output	2016-03-17 13:29:48 +00:00
escape_calls.go	cmd/internal/gc: make indirect calls properly escape-y	2015-05-22 01:36:11 +00:00
escape_closure.go	cmd/compile: note escape of parts of closured-capture vars	2016-04-05 18:10:09 +00:00
escape_field.go	cmd/internal/gc: improve flow of input params to output params	2015-05-01 13:47:20 +00:00
escape_iface.go	cmd/compile: avoid leak of dottype expression if type does not contain pointers.	2016-02-19 16:10:14 +00:00
escape_indir.go	cmd/internal/gc: improve flow of input params to output params	2015-05-01 13:47:20 +00:00
escape_level.go	cmd/internal/gc: improve flow of input params to output params	2015-05-01 13:47:20 +00:00
escape_map.go	test: add tests for escape analysis of interface conversions	2015-03-28 16:15:27 +00:00
escape_param.go	cmd/internal/gc: improve flow of input params to output params	2015-05-01 13:47:20 +00:00
escape_slice.go	cmd/internal/gc: extend escape analysis to pointers in slices	2015-05-18 15:34:39 +00:00
escape_struct_param1.go	cmd/internal/gc: improve flow of input params to output params	2015-05-01 13:47:20 +00:00
escape_struct_param2.go	cmd/internal/gc: improve flow of input params to output params	2015-05-01 13:47:20 +00:00
escape_struct_return.go	cmd/internal/gc: improve flow of input params to output params	2015-05-01 13:47:20 +00:00
escape.go
fibo.go	Fix whitespace (use blanks consistently).	2014-12-08 22:22:58 +00:00
finprofiled.go	runtime: fix finalization and profiling of tiny allocations	2015-11-03 18:57:18 +00:00
float_lit2.go
float_lit3.go
float_lit.go
floatcmp.go
for.go
func1.go
func2.go
func3.go
func4.go
func5.go
func6.go	Fix several out of date references to 4g/5g/6g/8g/9g.	2015-06-26 03:38:21 +00:00
func7.go	Fix several out of date references to 4g/5g/6g/8g/9g.	2015-06-26 03:38:21 +00:00
func8.go	test: use go:noinline consistently	2015-11-03 02:01:34 +00:00
func.go
funcdup2.go
funcdup.go
gc1.go
gc2.go	test: check for build constraints only upto the first blank line	2015-02-06 05:36:26 +00:00
gc.go
gcstring.go
goprint.go	test: make goprint.go wait for goroutine termination	2016-04-06 18:45:00 +00:00
goprint.out
goto.go	[dev.ssa] test: gofmt {goto,label,label1}.go	2015-07-20 20:09:34 +00:00
heapsampling.go	test: really force heap profile update in heapsampling.go	2015-11-12 18:32:13 +00:00
helloworld.go
helloworld.out
if.go
import1.go
import2.go
import4.go
import5.go
import.go
index0.go
index1.go
index2.go
index.go	Fix several out of date references to 4g/5g/6g/8g/9g.	2015-06-26 03:38:21 +00:00
indirect1.go
indirect.go
init1.go	test: use correct value in error message in init1.go	2016-04-14 18:35:09 +00:00
init.go
initcomma.go
initialize.go
initializerr.go
initloop.go	cmd/compile: use []*Node instead of NodeList in sinit	2015-09-05 02:25:01 +00:00
inline.go	cmd/compile: allow inlining of functions with switch statements	2016-03-21 23:05:10 +00:00
int_lit.go
intcvt.go
intrinsic.go	cmd/compile: added some intrinsics to SSA back end	2016-03-28 16:29:59 +00:00
iota.go
label1.go	[dev.ssa] cmd/compile: implement control flow handling	2015-07-23 00:45:26 +00:00
label.go	[dev.ssa] cmd/compile: implement control flow handling	2015-07-23 00:45:26 +00:00
linkmain_run.go	test: fix linkmain test	2015-12-17 23:36:13 +00:00
linkmain.go	cmd/link: reject non-package main toplevel.a file, remove dead code	2015-12-17 20:59:51 +00:00
linkx_run.go	test: print output on link -X failure	2016-04-20 17:37:35 +00:00
linkx.go	cmd/link: detect -X setting non-string variable	2015-06-29 20:28:36 +00:00
literal.go
live1.go
live2.go	[dev.ssa] cmd/compile: default compile+test with SSA	2015-10-30 20:35:20 +00:00
live_ssa.go	cmd/compile: fix x=x assignments	2016-04-06 15:04:32 +00:00
live_syscall.go	cmd/compile: recognize Syscall-like functions for liveness analysis	2016-01-14 01:16:45 +00:00
live.go	[dev.ssa] cmd/compile: default compile+test with SSA	2015-10-30 20:35:20 +00:00
loopbce.go	cmd/compile: bce when max and limit are consts	2016-04-11 16:01:22 +00:00
mallocfin.go
map1.go
map.go
maplinear.go	test: make maplinear more robust	2014-10-27 18:59:02 -04:00
method1.go	cmd/compile: ignore receiver parameters in Eqtype	2016-03-17 00:38:15 +00:00
method2.go
method3.go
method4.go
method5.go
method.go
named1.go
named.go
nil.go
nilcheck.go	[dev.ssa] cmd/compile: make cse faster	2016-01-28 20:59:20 +00:00
nilptr2.go
nilptr3_ssa.go	cmd/compile: don't nilcheck newobject and return values from mapaccess{1,2}	2016-04-22 16:18:42 +00:00
nilptr3.go	cmd/compile: don't nilcheck newobject and return values from mapaccess{1,2}	2016-04-22 16:18:42 +00:00
nilptr4.go
nilptr.go
nosplit.go	test: add s390x case to nosplit test	2016-03-21 08:14:51 +00:00
nul1.go
opt_branchlikely.go	[dev.ssa] cmd/compile: adjust branch likeliness for calls/loops	2016-03-01 20:09:41 +00:00
parentype.go
peano.go
phiopt.go	cmd/compile: introduce bool operations.	2016-04-25 20:43:04 +00:00
print.go
print.out
printbig.go
printbig.out
prove.go	cmd/compile: transform some Phis into Or8.	2016-04-19 22:04:30 +00:00
range.go
recover1.go
recover2.go
recover3.go
recover4.go	test: move allocation before munmap in recover4	2015-09-16 08:51:48 +00:00
recover.go	test: add more cases to recover.go	2014-10-22 08:06:15 -07:00
reflectmethod1.go	cmd/compile: track reflect.Type.Method in deadcode	2016-03-11 21:19:20 +00:00
reflectmethod2.go	cmd/compile: track reflect.Type.Method in deadcode	2016-03-11 21:19:20 +00:00
reflectmethod3.go	cmd/compile: track reflect.Type.Method in deadcode	2016-03-11 21:19:20 +00:00
reflectmethod4.go	cmd/link: treat reflect.Value.Method like Call	2016-03-11 22:07:02 +00:00
rename1.go
rename.go
reorder2.go	test: use go:noinline consistently	2015-11-03 02:01:34 +00:00
reorder.go
return.go
rotate0.go	test: re-enable rotate tests	2015-05-28 19:40:58 +00:00
rotate1.go	test: re-enable rotate tests	2015-05-28 19:40:58 +00:00
rotate2.go	test: re-enable rotate tests	2015-05-28 19:40:58 +00:00
rotate3.go	test: re-enable rotate tests	2015-05-28 19:40:58 +00:00
rotate.go
run.go	cmd/compile: fix build	2016-03-31 18:00:36 +00:00
rune.go
runtime.go
shift1.go
shift2.go
sieve.go
sigchld.go
sigchld.out
simassign.go
sinit_run.go	all: retire architecture letter in file names, public API	2015-05-21 17:32:17 +00:00
sinit.go	cmd/compile: allow static init for unsafe.Pointer(&x) where x is global	2015-07-07 21:51:57 +00:00
sizeof.go
slice3.go	cmd/gc, runtime: treat slices and strings like pointers in garbage collection	2014-08-25 14:38:19 -04:00
slice3err.go
slicecap.go	cmd/gc, runtime: treat slices and strings like pointers in garbage collection	2014-08-25 14:38:19 -04:00
sliceopt.go	[dev.ssa] cmd/compile: default compile+test with SSA	2015-10-30 20:35:20 +00:00
solitaire.go
stack.go
strength.go	cmd/compile: generalize strength reduction of mulq	2016-03-30 22:27:13 +00:00
string_lit.go
stringrange.go
struct0.go
switch2.go	cmd/compile: address several more 1.6 TODOs in parser	2015-11-21 07:21:23 +00:00
switch3.go
switch4.go
switch.go
tinyfin.go	runtime: fix finalization and profiling of tiny allocations	2015-11-03 18:57:18 +00:00
torture.go
turing.go
typecheck.go
typecheckloop.go	cmd/compile: convert typecheck_stack to []*Node	2015-09-06 23:50:51 +00:00
typeswitch1.go
typeswitch2.go
typeswitch3.go
typeswitch.go
undef.go
utf.go
varerr.go
varinit.go
writebarrier.go	runtime: don't rescan globals	2016-04-27 18:48:16 +00:00
zerodivide.go