madvise was missing so implement it in assembler. This change
needs to be extended to the other BSD variantes (Net and Open)
Without this change the scavenger will attempt to pass memory back
to the operating system when it has become idle, but the memory is
not returned and for long running Go processes the total memory used
can grow until OOM occurs.
I have only been able to test the code on FreeBSD AMD64. The ARM
platforms needs testing.
R=golang-dev, mikioh.mikioh, dave, jgc, minux.ma
CC=golang-dev
https://golang.org/cl/6850081
Update OpenBSD runtime to use the new version of the sys___tfork
syscall and switch TLS initialisation from sys_arch to sys___set_tcb
(note that both of these syscalls are available in OpenBSD 5.2).
R=golang-dev, minux.ma
CC=golang-dev
https://golang.org/cl/6843058
This enables to loop over some goroutines, e.g. to print the
backtrace of goroutines 1 to 9:
set $i = 1
while $i < 10
printf "backtrace of goroutine %d:\n", $i
goroutine $i++ bt
end
R=lvd, lvd
CC=golang-dev
https://golang.org/cl/6843071
This significantly decreases amount of shadow memory
mapped by race detector.
I haven't tested on Windows, but on Linux it reduces
virtual memory size from 1351m to 330m for fmt.test.
Fixes#4379.
R=golang-dev, alex.brainman, iant
CC=golang-dev
https://golang.org/cl/6849057
Currently race detector runtime just disables race detection in the finalizer goroutine.
It has false positives when a finalizer writes to shared memory -- the race with finalizer is reported in a normal goroutine that accesses the same memory.
After this change I am going to synchronize the finalizer goroutine with the rest of the world in racefingo(). This is closer to what happens in reality and so
does not have false positives.
And also add README file with instructions how to build the runtime.
R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/6810095
It allows to catch e.g. a data race between atomic write and non-atomic write,
or Mutex.Lock() and mutex overwrite (e.g. mu = Mutex{}).
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6817103
In order to add these, we need to be able to find references
to such types that already exist in the binary. To do that, introduce
a new linker section holding a list of the types corresponding to
arrays, chans, maps, and slices.
To offset the storage cost of this list, and to simplify the code,
remove the interface{} header from the representation of a
runtime type. It was used in early versions of the code but was
made obsolete by the kind field: a switch on kind is more efficient
than a type switch.
In the godoc binary, removing the interface{} header cuts two
words from each of about 10,000 types. Adding back the list of pointers
to array, chan, map, and slice types reintroduces one word for
each of about 500 types. On a 64-bit machine, then, this CL *removes*
a net 156 kB of read-only data from the binary.
This CL does not include the needed support for precise garbage
collection. I have created issue 4375 to track that.
This CL also does not set the 'algorithm' - specifically the equality
and copy functions - for a new array correctly, so I have unexported
ArrayOf for now. That is also part of issue 4375.
Fixes#2339.
R=r, remyoudompheng, mirtchovski, iant
CC=golang-dev
https://golang.org/cl/6572043
Otherwise a poorly timed GC can collect the memory before it
is returned to the Go program.
R=golang-dev, dave, dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/6819119
Re-enable the crash tests on NetBSD now that the issue has been
identified and fixed.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/6813100
Currently race detector runtime maps shadow memory eagerly at process startup.
It works poorly on Windows, because Windows requires reservation in swap file
(especially problematic if several Go program runs at the same, each consuming GBs
of memory).
With this change race detector maps shadow memory lazily, so Go runtime must notify
about all new heap memory.
It will help with Windows port, but also eliminates scary 16TB virtual mememory
consumption in top output (which sometimes confuses some monitoring scripts).
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6811085
When the first result of a type assertion is blank, the compiler would still copy out a potentially large non-interface type.
Fixes#1021.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/6812079
It speedups the race detector somewhat, but also prevents
getcallerpc() from obtaining lessstack().
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/6812091
The deadlock occurs when another goroutine requests GC
during the test. When wait=true the test expects physical parallelism,
that is, that P goroutines are all active at the same time.
If GC is requested, then part of the goroutines are not scheduled,
so other goroutines deadlock.
With wait=false, goroutines finish parallel for w/o waiting for all
other goroutines.
Fixes#3954.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/6820098
The race detector does not understand ParFor synchronization, because it's implemented in C.
If run with -cpu=2 currently race detector says:
WARNING: DATA RACE
Read by goroutine 5:
runtime_test.TestParForParallel()
src/pkg/runtime/parfor_test.go:118 +0x2e0
testing.tRunner()
src/pkg/testing/testing.go:301 +0x8f
Previous write by goroutine 6:
runtime_test.func·024()
src/pkg/runtime/parfor_test.go:111 +0x52
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/6811082
PauseNs is a circular buffer of recent pause times, and the
most recent one is at [((NumGC-1)+256)%256].
Also fix comments cross-linking the Go and C definition of
various structs.
R=golang-dev, rsc, bradfitz
CC=golang-dev
https://golang.org/cl/6657047
If source are not available, then the stack looks like:
stack_test.go:40: /tmp/gobuilder/linux-amd64-race-72b15c5d6f65/go/src/pkg/runtime/debug/bla-bla-bla/src/pkg/runtime/debug/stack_test.go:15 (0x43fb11)
stack_test.go:40: /tmp/gobuilder/linux-amd64-race-72b15c5d6f65/go/src/pkg/runtime/debug/bla-bla-bla/src/pkg/runtime/debug/stack_test.go:18 (0x43fb7a)
stack_test.go:40: /tmp/gobuilder/linux-amd64-race-72b15c5d6f65/go/src/pkg/runtime/debug/bla-bla-bla/src/pkg/runtime/debug/stack_test.go:37 (0x43fbf4)
stack_test.go:40: /tmp/gobuilder/linux-amd64-race-72b15c5d6f65/go/src/pkg/testing/bla-bla-bla/src/pkg/testing/testing.go:301 (0x43b5ba)
stack_test.go:40: /tmp/gobuilder/linux-amd64-race-72b15c5d6f65/go/src/pkg/runtime/bla-bla-bla/src/pkg/runtime/proc.c:276 (0x410670)
stack_test.go:40:
which is 6 lines.
R=golang-dev, minux.ma
CC=golang-dev
https://golang.org/cl/6637060
Check for specific, important misalignment in garbage collector.
Not a complete fix for issue 599 but an important workaround.
Update #599.
R=golang-dev, iant, dvyukov
CC=golang-dev
https://golang.org/cl/6641049
Also add call to GC() to make it easier to re-enable the test.
Update #4155.
When we have precise GC merged, re-enable this test.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/6622058
The profiler collects goroutine blocking information similar to Google Perf Tools.
You may see an example of the profile (converted to svg) attached to
http://code.google.com/p/go/issues/detail?id=3946
The public API changes are:
+pkg runtime, func BlockProfile([]BlockProfileRecord) (int, bool)
+pkg runtime, func SetBlockProfileRate(int)
+pkg runtime, method (*BlockProfileRecord) Stack() []uintptr
+pkg runtime, type BlockProfileRecord struct
+pkg runtime, type BlockProfileRecord struct, Count int64
+pkg runtime, type BlockProfileRecord struct, Cycles int64
+pkg runtime, type BlockProfileRecord struct, embedded StackRecord
R=rsc, dave, minux.ma, r
CC=gobot, golang-dev, r, remyoudompheng
https://golang.org/cl/6443115
The Go run-time assumes that all SSE floating-point exceptions
are masked so that Go programs are not broken by such invalid
operations. By default, the 64-bit version of the Plan 9 kernel
masks only some SSE floating-point exceptions. Here, we mask
them all on a per-thread basis.
R=rsc, rminnich, minux.ma
CC=golang-dev
https://golang.org/cl/6592056
The assembly offsets were converted mechanically using
code.google.com/p/rsc/cmd/asmlint. The instruction
changes were done by hand.
Fixes#2188.
R=iant, r, bradfitz, remyoudompheng
CC=golang-dev
https://golang.org/cl/6550058
This CL makes the runtime understand that the type of
the len or cap of a map, slice, or string is 'int', not 'int32',
and it is also careful to distinguish between function arguments
and results of type 'int' vs type 'int32'.
In the runtime, the new typedefs 'intgo' and 'uintgo' refer
to Go int and uint. The C types int and uint continue to be
unavailable (cause intentional compile errors).
This CL does not change the meaning of int, but it should make
the eventual change of the meaning of int on amd64 a bit
smoother.
Update #2188.
R=iant, r, dave, remyoudompheng
CC=golang-dev
https://golang.org/cl/6551067
Using offsets from Tos is cumbersome and we've had problems
in the past. Since it's only being used to grab the PID, we'll just
get that from the default TLS instead.
R=rsc, rminnich, npe
CC=golang-dev
https://golang.org/cl/6543049
The change is a preparation for the new scheduler.
It introduces runtime.park() function,
that will atomically unlock the mutex and park the goroutine.
It will allow to remove the racy readyonstop flag
that is difficult to implement w/o the global scheduler mutex.
R=rsc, remyoudompheng, dave
CC=golang-dev
https://golang.org/cl/6501077
Fixes#3456.
This proposal is a reformulation of CL 5987063. This CL resets
the default GOARM value to 6 and allows the use of the VFPv3
optimisation if GOARM=7. Binaries built with this CL in place
will abort if GOARM=7 was used and the target host does not
support VFPv3.
R=minux.ma, rsc, ajstarks
CC=golang-dev
https://golang.org/cl/6501099
Fixes#3911.
Requires CL 6449127.
dfc@qnap:~$ ./runtime.test
runtime: this CPU has no floating point hardware, so it cannot run
this GOARM=7 binary. Recompile using GOARM=5.
R=rsc, minux.ma
CC=golang-dev
https://golang.org/cl/6442109
Signal handlers are global resources but many language
environments (Go, C++ at Google, etc) assume they have sole
ownership of a particular handler. Signal handlers in
mixed-language applications must therefore be robust against
unexpected delivery of certain signals, such as SIGPROF.
The default Go signal handler runtime·sigtramp assumes that it
will never be called on a non-Go thread, but this assumption
is violated by when linking in C++ code that spawns threads.
Specifically, the handler asserts the thread has an associated
"m" (Go scheduler).
This CL is a very simple workaround: discard SIGPROF delivered to non-Go threads. runtime.badsignal(int32) now receives the signal number; if it returns without panicking (e.g. sig==SIGPROF) the signal is discarded.
I don't think there is any really satisfactory solution to the
problem of signal-based profiling in a mixed-language
application. It's not only the issue of handler clobbering,
but also that a C++ SIGPROF handler called in a Go thread
can't unwind the Go stack (and vice versa). The best we can
hope for is not crashing.
Note:
- I've ported this to all POSIX platforms, except ARM-linux which already ignores unexpected signals on m-less threads.
- I've avoided tail-calling runtime.badsignal because AFAICT the 6a/6l don't support it.
- I've avoided hoisting 'push sig' (common to both function calls) because it makes the code harder to read.
- Fixed an (apparently incorrect?) docstring.
R=iant, rsc, minux.ma
CC=golang-dev
https://golang.org/cl/6498057
Reverts part of CL 6460082.
If a doc comment describes a type by explaining the
meaning of one instance of the type, a leading article
is fine and makes the text less awkward.
Compare:
// A dog is a kind of animal.
// Dog is a kind of animal.
R=golang-dev, dsymonds, dvyukov, r
CC=golang-dev
https://golang.org/cl/6494066
This set of changes extends the Plan 9 support
to include the AMD64 architecture and should
work on all versions of Plan 9.
R=golang-dev, rminnich, noah.evans, rsc, minux.ma, npe
CC=akskuma, golang-dev, jfflore, noah.evans
https://golang.org/cl/6479052
Use version 2 of the NetBSD signal ABI - both version 2 and version 3
are supported by the kernel, with near identical behaviour. However,
the netbsd32 compat code does not allow version 3 to be used, which
prevents Go netbsd/386 binaries from running in compat mode on a
NetBSD amd64 kernel. Switch to version 2 of the ABI, which is the
same version currently used by NetBSD's libc.
R=minux.ma
CC=golang-dev
https://golang.org/cl/6476068
When manipulating the stack pointer use the UESP register instead
of the ESP register, since the UESP register is the one that gets
restored from the machine context. Fixes broken tests on netbsd/386.
R=golang-dev, minux.ma, r, bsiegert
CC=golang-dev
https://golang.org/cl/6465054
Disable the crash handler test on NetBSD until I can figure out why
it triggers failures in later tests.
R=golang-dev, minux.ma
CC=golang-dev
https://golang.org/cl/6460090
Surrogate halves are part of UTF-16 and should never appear in UTF-8.
(The rune that two combined halves represent in UTF-16 should
be encoded directly.)
Encoding: encode as RuneError.
Decoding: convert to RuneError, consume one byte.
This requires changing:
package unicode/utf8
runtime for range over string
Also added utf8.ValidRune and fixed bug in utf.RuneLen.
Fixes#3927.
R=golang-dev, rsc, bsiegert
CC=golang-dev
https://golang.org/cl/6458099
Depends on CL 6197045.
Result obtained on Core i7 620M, Darwin/amd64:
benchmark old ns/op new ns/op delta
BenchmarkComplex128DivNormal 57 28 -50.78%
BenchmarkComplex128DivNisNaN 49 15 -68.90%
BenchmarkComplex128DivDisNaN 49 15 -67.88%
BenchmarkComplex128DivNisInf 40 12 -68.50%
BenchmarkComplex128DivDisInf 33 13 -61.06%
Result obtained on Core i7 620M, Darwin/386:
benchmark old ns/op new ns/op delta
BenchmarkComplex128DivNormal 89 50 -44.05%
BenchmarkComplex128DivNisNaN 307 802 +161.24%
BenchmarkComplex128DivDisNaN 309 788 +155.02%
BenchmarkComplex128DivNisInf 278 237 -14.75%
BenchmarkComplex128DivDisInf 46 22 -52.46%
Result obtained on 700MHz OMAP4460, Linux/ARM:
benchmark old ns/op new ns/op delta
BenchmarkComplex128DivNormal 1557 465 -70.13%
BenchmarkComplex128DivNisNaN 1443 220 -84.75%
BenchmarkComplex128DivDisNaN 1481 218 -85.28%
BenchmarkComplex128DivNisInf 952 216 -77.31%
BenchmarkComplex128DivDisInf 861 231 -73.17%
The 386 version has a performance regression, but as we have
decided to use SSE2 instead of x87 FPU for 386 too (issue 3912),
I won't address this issue.
R=dsymonds, mchaten, iant, dave, mtj, rsc, r
CC=golang-dev
https://golang.org/cl/6024045
our old choice is not working properly at least on VFPv2 in
ARM1136JF-S (it's not preserved across float64->float32 conversions).
Fixes#3745.
R=dave, rsc
CC=golang-dev
https://golang.org/cl/6344078
Since NUL usually terminates strings in underlying syscalls, allowing
it when converting string arguments is a security risk, especially
when dealing with filenames. For example, a program might reason that
filename like "/root/..\x00/" is a subdirectory or "/root/" and allow
access to it, while underlying syscall will treat "\x00" as an end of
that string and the actual filename will be "/root/..", which might
be unexpected. Returning EINVAL when string arguments have NUL in
them makes sure this attack vector is unusable.
R=golang-dev, r, bradfitz, fullung, rsc, minux.ma
CC=golang-dev
https://golang.org/cl/6458050
When a cgo program calls setuid, setgid, etc., the GNU/Linux
pthread library sends signal SIGSETXID to each thread to tell
it to update its UID info. If Go is permitted to intercept
the default SIGSETXID signal handler, the program will hang.
This patch tells the runtime package to not try to intercept
SIGSETXID on GNU/Linux. This will be odd if a Go program
wants to try to use that signal, but it means that cgo
programs that call setuid, etc., won't hang.
Fixes#3871.
R=rsc, r, minux.ma, bradfitz
CC=golang-dev
https://golang.org/cl/6455050
Move panic/defer/recover-related stuff from proc.c/runtime.c to a new file panic.c.
No semantic changes.
proc.c is 1800+ LOC and is a bit difficult to work with.
R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/6343071
1. Rename 'g' and 'm' local vars to 'gp' and 'mp' (convention already used in some functions)
'g' and 'm' are global vars that mean current goroutine and current machine,
when they are shadowed by local vars, it's confusing, no ability to debug log both, etc.
2. White-space shuffling.
No semantic changes.
In preparation to bigger changes.
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/6355061
There may be further savings if convT2I can avoid the function call
if the cache is good and T is uintptr-shaped, a la convT2E, but that
will be a follow-up CL.
src/pkg/runtime:
benchmark old ns/op new ns/op delta
BenchmarkConvT2ISmall 43 15 -64.01%
BenchmarkConvT2IUintptr 45 14 -67.48%
BenchmarkConvT2ILarge 130 101 -22.31%
test/bench/go1:
benchmark old ns/op new ns/op delta
BenchmarkBinaryTree17 8588997000 8499058000 -1.05%
BenchmarkFannkuch11 5300392000 5358093000 +1.09%
BenchmarkGobDecode 30295580 31040190 +2.46%
BenchmarkGobEncode 18102070 17675650 -2.36%
BenchmarkGzip 774191400 771591400 -0.34%
BenchmarkGunzip 245915100 247464100 +0.63%
BenchmarkJSONEncode 123577000 121423050 -1.74%
BenchmarkJSONDecode 451969800 596256200 +31.92%
BenchmarkMandelbrot200 10060050 10072880 +0.13%
BenchmarkParse 10989840 11037710 +0.44%
BenchmarkRevcomp 1782666000 1716864000 -3.69%
BenchmarkTemplate 798286600 723234400 -9.40%
R=rsc, bradfitz, go.peter.90, daniel.morsing, dave, uriel
CC=golang-dev
https://golang.org/cl/6337058