The timespec passed to thrsleep() needs to be an absolute/realtime
value, so add the current nanotime to ns.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5374048
This looks like it is just moving some code from
time to runtime (and translating it to C), but the
runtime can do a better job managing the goroutines,
and it needs this functionality for its own maintenance
(for example, for the garbage collector to hand back
unused memory to the OS on a time delay).
Might as well have just one copy of the timer logic,
and runtime can't depend on time, so vice versa.
It also unifies Sleep, NewTicker, and NewTimer behind
one mechanism, so that there are no claims that one
is more efficient than another. (For example, today
people recommend using time.After instead of time.Sleep
to avoid blocking an OS thread.)
Fixes#1644.
Fixes#1731.
Fixes#2190.
R=golang-dev, r, hectorchu, iant, iant, jsing, alex.brainman, dvyukov
CC=golang-dev
https://golang.org/cl/5334051
Fixes crash when cgo consumes more than 8K
of stack and makes a callback.
Fixes#1328.
R=golang-dev, rogpeppe, rsc
CC=golang-dev, mpimenov
https://golang.org/cl/5371042
Otherwise some OS X toolchains complain about the redeclaration
of libcgo_thread_start by multiple object files. The real definition
is in util.c.
Fixes#2167.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5364045
- Fix function prototype for thrsleep().
- Provide enums for clock identifiers.
- Provide timespec structure for use with thrsleep().
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/5360042
runtime knows how to get the time of day
without allocating memory.
R=golang-dev, dsymonds, dave, hectorchu, r, cw
CC=golang-dev
https://golang.org/cl/5297078
We only guarantee that the main goroutine runs on the
main OS thread for initialization. Programs that wish to
preserve that property for main.main can call runtime.LockOSThread.
This is what programs used to do before we unleashed
goroutines during init, so it is both a simple fix and keeps
existing programs working.
R=iant, r, dave, dvyukov
CC=golang-dev
https://golang.org/cl/5309070
Revert workaround in compiler and
revert test for compiler workaround.
Tested that the 386 build continues to fail if
the gc change is made without the reflect change.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5312041
The old m[x] = 0, false syntax will be deleted
in a month or so, once people have had time to
change their code (there is a gofix in a separate CL).
R=ken2
CC=golang-dev
https://golang.org/cl/5265048
New DLL and Proc types to manage and call dll functions. These were
used to simplify syscall tests in runtime package. They were also
used to implement LazyDLL and LazyProc.
LazyProc, like Proc, now have Call function, that just a wrapper for
SyscallN. It is not as efficient as Syscall, but easier to use.
NewLazyDLL now supports non-ascii filenames.
LazyDLL and LazyProc now have Load and Find methods. These can be used
during runtime to discover if some dll functions are not present.
All dll functions now return errors that fit os.Error interface. They
also contain Windows error number.
Some of these changes are suggested by jp.
R=golang-dev, jp, rsc
CC=golang-dev
https://golang.org/cl/5272042
The work buffer management used by the garbage
collector during parallel collections leaks buffers.
This CL tests for and fixes the leak.
R=golang-dev, dvyukov, r
CC=golang-dev
https://golang.org/cl/5254059
Use FlagNoPointers and do not zeroize memory when allocate strings.
test/garbage/parser.out old new
run #1 32.923s 32.065s
run #2 33.047s 31.931s
run #3 32.702s 31.841s
run #4 32.718s 31.838s
run #5 32.702s 31.868s
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5259041
Implement a locking model based on the current linux model - a
tri-state mutex with active spinning, passive spinning and sleeping.
R=golang-dev, dvyukov, rsc
CC=golang-dev
https://golang.org/cl/4974043
The malloc sample trigger was not being set in a
new m, so the first allocation in each new m - the
goroutine structure - was being sampled with
probability 1 instead of probability sizeof(G)/rate,
an oversampling of about 5000x for the default
rate of 1 MB. This bug made pprof graphs show
far more G allocations than there actually were.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5224041
Fixes#2337.
Unfortunate sequence of events is:
1. maxcpu=2, mcpu=1, grunning=1
2. starttheworld creates an extra M:
maxcpu=2, mcpu=2, grunning=1
4. the goroutine calls runtime.GOMAXPROCS(1)
maxcpu=1, mcpu=2, grunning=1
5. since it sees mcpu>maxcpu, it calls gosched()
6. schedule() deschedules the goroutine:
maxcpu=1, mcpu=1, grunning=0
7. schedule() call getnextandunlock() which
fails to pick up the goroutine again,
because canaddcpu() fails, because mcpu==maxcpu
8. then it sees that grunning==0,
reports deadlock and terminates
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5191044
When ncpu < 2, work.nproc is always 1 which results in infinite helper
threads being created if gomaxprocs > 1 and MaxGcproc > 1. Avoid this
by using the same limits as imposed helpgc().
R=golang-dev, rsc, dvyukov
CC=golang-dev
https://golang.org/cl/5176044
This change adds the osyield and usleep
functions and code to read the number of
processors from /dev/sysstat.
I also changed SysAlloc to return nil
when brk fails (it was returning -1).
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5177049
The map implementation was using the C idiom of using
a pointer just past the end of its table as a limit pointer.
Unfortunately, the garbage collector sees that pointer as
pointing at the block adjacent to the map table, pinning
in memory a block that would otherwise be freed.
Fix by making limit pointer point at last valid entry, not
just past it.
Reviewed by Mike Burrows.
R=golang-dev, bradfitz, lvd, r
CC=golang-dev
https://golang.org/cl/5158045
Running test/garbage/parser.out.
On a 4-core Lenovo X201s (Linux):
31.12u 0.60s 31.74r 1 cpu, no atomics
32.27u 0.58s 32.86r 1 cpu, atomic instructions
33.04u 0.83s 27.47r 2 cpu
On a 16-core Xeon (Linux):
33.08u 0.65s 33.80r 1 cpu, no atomics
34.87u 1.12s 29.60r 2 cpu
36.00u 1.87s 28.43r 3 cpu
36.46u 2.34s 27.10r 4 cpu
38.28u 3.85s 26.92r 5 cpu
37.72u 5.25s 26.73r 6 cpu
39.63u 7.11s 26.95r 7 cpu
39.67u 8.10s 26.68r 8 cpu
On a 2-core MacBook Pro Core 2 Duo 2.26 (circa 2009, MacBookPro5,5):
39.43u 1.45s 41.27r 1 cpu, no atomics
43.98u 2.95s 38.69r 2 cpu
On a 2-core Mac Mini Core 2 Duo 1.83 (circa 2008; Macmini2,1):
48.81u 2.12s 51.76r 1 cpu, no atomics
57.15u 4.72s 51.54r 2 cpu
The handoff algorithm is really only good for two cores.
Beyond that we will need to so something more sophisticated,
like have each core hand off to the next one, around a circle.
Even so, the code is a good checkpoint; for now we'll limit the
number of gc procs to at most 2.
R=dvyukov
CC=golang-dev
https://golang.org/cl/4641082
The Dwarf info has the full typenames, the go *struct runtime.commonType
has the short name. A more permanent fix would link the two together
but this way the user gets useable stack traces for now.
R=rsc
CC=golang-dev
https://golang.org/cl/5097046
gotest src/pkg/exp/template/html was crashing because the exception handler overflowed the goroutine stack.
R=alex.brainman, golang-dev
CC=golang-dev
https://golang.org/cl/5031049
The Windows implementation of the net package churns through a couple of channels for every read/write operation. This translates into a lot of time spent in the kernel creating and deleting event objects.
R=rsc, dvyukov, alex.brainman, jp
CC=golang-dev
https://golang.org/cl/4997044
My string literal was being rewritten from
"runtime.SysReserve(%p, %D) = error %d"
to
"runtime.SysReserve ( %p , %D ) = error %d"
R=iant
CC=golang-dev
https://golang.org/cl/4972051
- Rename sys_sched_yield() to osyield() as this is now defined in asm.h.
- Only print kern.rtheads message if rfork_thread() failed with ENOTSUP.
- Remove unused variables.
R=rsc
CC=golang-dev
https://golang.org/cl/4973043
cdecl calbacks have been implemented in C/ASM code, just Go function is missing
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/4969047
Make the stack traces more readable for new
Go programmers while preserving their utility for old hands.
- Change status number [4] to string.
- Elide frames in runtime package (internal details).
- Swap file:line and arguments.
- Drop 'created by' for main goroutine.
- Show goroutines in order of allocation:
implies main goroutine first if nothing else.
There is no option to get the extra frames back.
Uncomment 'return 1' at the bottom of symtab.c.
$ 6.out
throw: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:22 +0x8a
goroutine 2 [select (no cases)]:
main.sel()
/Users/rsc/g/go/src/pkg/runtime/x.go:11 +0x18
created by main.main
/Users/rsc/g/go/src/pkg/runtime/x.go:19 +0x23
goroutine 3 [chan receive]:
main.recv(0xf8400010a0, 0x0)
/Users/rsc/g/go/src/pkg/runtime/x.go:15 +0x2e
created by main.main
/Users/rsc/g/go/src/pkg/runtime/x.go:20 +0x50
goroutine 4 [chan receive (nil chan)]:
main.recv(0x0, 0x0)
/Users/rsc/g/go/src/pkg/runtime/x.go:15 +0x2e
created by main.main
/Users/rsc/g/go/src/pkg/runtime/x.go:21 +0x66
$
$ 6.out index
panic: runtime error: index out of range
goroutine 1 [running]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:25 +0xb9
$
$ 6.out nil
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x22ca]
goroutine 1 [running]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:28 +0x211
$
$ 6.out panic
panic: panic
goroutine 1 [running]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:30 +0x101
$
R=golang-dev, qyzhai, n13m3y3r, r
CC=golang-dev
https://golang.org/cl/4907048
Allocate Defer on stack during cgo calls, as suggested
by dvyukov. Also includes some comment corrections.
benchmark old,ns/op new,ns/op
BenchmarkCgoCall 669 330
(Intel Xeon CPU 1.80GHz * 4, Linux 386)
R=dvyukov, rsc
CC=golang-dev
https://golang.org/cl/4910041
The corruption can occur when GOMAXPROCS
is changed from >1 to 1, since GOMAXPROCS=1
does not imply there is only 1 goroutine running,
other goroutines can still be not parked after
the change.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4873050
Every time we enter callback from Windows, it is
possible that go exception handler is not at the top
of per-thread exception handlers chain. So it needs
to be installed again. At this moment this is done
by replacing top SEH frame with SEH frame as at time
of syscall for the time of callback. This is incorrect,
because, if exception strike, we won't be able to call
any exception handlers installed inside syscall,
because they are not in the chain. This changes
procedure to add new SEH frame on top of existing
chain instead.
I also removed m sehframe field, because I don't
think it is needed. We use single global exception
handler everywhere.
R=golang-dev, r
CC=golang-dev, hectorchu
https://golang.org/cl/4832060
Add support for the go runtime on openbsd/amd64. This is based on
the existing freebsd runtime.
Threads are implemented using OpenBSD's rthreads, which are currently
disabled by default, however can be enabled via the kern.rthreads
sysctl.
For now, cgo is disabled.
R=rsc
CC=golang-dev
https://golang.org/cl/4815067
The change adds specialized type algorithms
for slices and types of size 8/16/32/64/128.
It significantly accelerates chan and map operations
for most builtin types as well as user structs.
benchmark old,ns/op new,ns/op
BenchmarkChanUncontended 226 94
(on Intel Xeon E5620, 2.4GHz, Linux 64 bit)
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4815087
The data race can lead to reads of partially
initialized concurrently mutated symbol data.
The change also adds a simple sanity test
for Caller() and FuncForPC().
R=rsc
CC=golang-dev
https://golang.org/cl/4817058
When rnd is called with a second argument of 1, it simply
returns the first argument anyway.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4820045
Replace cas with xadd in scheduler.
Suggested by Dmitriy in last code review.
Verified with Promela model.
When there's actual contention for the atomic word,
this avoids the looping that compare-and-swap requires.
benchmark old ns/op new ns/op delta
runtime_test.BenchmarkSyscall 32 26 -17.08%
runtime_test.BenchmarkSyscall-2 155 59 -61.81%
runtime_test.BenchmarkSyscall-3 112 52 -52.95%
runtime_test.BenchmarkSyscall-4 94 48 -48.57%
runtime_test.BenchmarkSyscallWork 871 872 +0.11%
runtime_test.BenchmarkSyscallWork-2 481 477 -0.83%
runtime_test.BenchmarkSyscallWork-3 338 335 -0.89%
runtime_test.BenchmarkSyscallWork-4 263 256 -2.66%
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/4800047
Drops mallocrep1.go back to a reasonable
amount of time. (154 -> 0.8 seconds on my Mac)
Fixes#2085.
R=golang-dev, dvyukov, r
CC=golang-dev
https://golang.org/cl/4811045
Remove complicated PRNG algorithm
(argument is limited by uint16 and can't be <= 1).
Do not require chansend/chanrecv selgen to be bumped with CAS.
R=rsc, ken
CC=golang-dev
https://golang.org/cl/4816041
pkg/runtime/Makefile:
. Adjusted so "goc2c.c" is built using the Plan 9 libraries.
pkg/runtime/goc2c.c:
. Added/subtracted #include headers to correspond to Plan 9
toolkit.
. Changed fprintf(stderr,...)/exit() combinations to
sysfatal() calls, adjusted the "%u" format to "%ud".
. Added exits(0) at the end of main().
. Made main() a void-returning function and removed the
"return 0" at the end of it.
Tested on UBUNTU and Plan 9 only.
R=r, rsc
CC=golang-dev
https://golang.org/cl/4626093
Used to use mcpu+msyscall but that's
problematic for packing into a single
atomic word. The running goroutine count
(where running == Go code or syscall)
can be maintained separately, always
manipulated under lock.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/4767041
The data race is on newly published Itab nodes, which are
both unsafely published and unsafely acquired. It can
break on IA-32/Intel64 due to compiler optimizations
(most likely not an issue as of now) and on ARM due to
hardware memory access reorderings.
R=rsc
CC=golang-dev
https://golang.org/cl/4673055
runtime.goidgen can be quite frequently modified and
shares cache line with the following variables,
it leads to false sharing.
50c6b0 b nfname
50c6b4 b nfunc
50c6b8 b nfunc$17
50c6bc b nhist$17
50c6c0 B runtime.checking
50c6c4 B runtime.gcwaiting
50c6c8 B runtime.goidgen
50c6cc B runtime.gomaxprocs
50c6d0 B runtime.panicking
50c6d4 B strconv.IntSize
50c6d8 B src/pkg/runtime/_xtest_.ss
50c6e0 B src/pkg/runtime/_xtest_.stop
50c6e8 b addrfree
50c6f0 b addrmem
50c6f8 b argv
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4673054
Change the signature of Split to have no count,
assuming a full split, and rename the existing
Split with a count to SplitN.
Do the same to package bytes.
Add a gofix module.
R=adg, dsymonds, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4661051
grsec needs the FIXED flag to be provided to mmap, which
works now. That said, when the allocation fails to be made
in the specific address, we're still given back a writable
page. This change will unmap that page to avoid using
twice the amount of memory needed.
It'd also be pretty easy to avoid the extra system calls
once we detected that the flag is needed, but I'm not sure
if that edge case is worth the effort.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4634086
All but two packages depend on net:
debug/proc
os/signal
With this change, we can produce
a working build with GOOS=plan9.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4639053
Correct a few error messages (libcgo -> runtime/cgo)
and delete old nacl_386.c file too.
Fixes#1657.
R=iant
CC=golang-dev
https://golang.org/cl/4603057
5a: add SQRTF and SQRTD
5l: add ASQRTF and ASQRTD
Use ARMv7 VFP VSQRT instruction to speed up math.Sqrt
R=rsc, dave, m
CC=golang-dev
https://golang.org/cl/4551082
This change was adapted from gccgo's libgo/runtime/mem.c at
Ian Taylor's suggestion. It fixes all.bash failing with
"address space conflict: map() =" on amd64 Linux with kernel
version 2.6.32.8-grsec-2.1.14-modsign-xeon-64.
With this change, SysMap will use MAP_FIXED to allocate its desired
address space, after first calling mincore to check that there is
nothing else mapped there.
R=iant, dave, n13m3y3r, rsc
CC=golang-dev
https://golang.org/cl/4438091
breaks Mac build
««« original CL description
runtime: use HOST_CC to compile mkversion
HOST_CC is set in Make.inc, so use that rather
than hardcoding quietgcc
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/4515163
»»»
R=iant
CC=golang-dev
https://golang.org/cl/4515168
Works around bug in kernel implementation on old ARM5 kernels.
Bug was fixed on 26 Nov 2007 (between 2.6.23 and 2.6.24) but
old kernels persist.
Fixes#1750.
R=dfc, golang-dev
CC=golang-dev
https://golang.org/cl/4436072
The g->sched.sp saved stack pointer and the
g->stackbase and g->stackguard stack bounds
can change even while "the world is stopped",
because a goroutine has to call functions (and
therefore might split its stack) when exiting a
system call to check whether the world is stopped
(and if so, wait until the world continues).
That means the garbage collector cannot access
those values safely (without a race) for goroutines
executing system calls. Instead, save a consistent
triple in g->gcsp, g->gcstack, g->gcguard during
entersyscall and have the garbage collector refer
to those.
The old code was occasionally seeing (because of
the race) an sp and stk that did not correspond to
each other, so that stk - sp was not the number of
stack bytes following sp. In that case, if sp < stk
then the call scanblock(sp, stk - sp) scanned too
many bytes (anything between the two pointers,
which pointed into different allocation blocks).
If sp > stk then stk - sp wrapped around.
On 32-bit, stk - sp is a uintptr (uint32) converted
to int64 in the call to scanblock, so a large (~4G)
but positive number. Scanblock would try to scan
that many bytes and eventually fault accessing
unmapped memory. On 64-bit, stk - sp is a uintptr (uint64)
promoted to int64 in the call to scanblock, so a negative
number. Scanblock would not scan anything, possibly
causing in-use blocks to be freed.
In short, 32-bit platforms would have seen either
ineffective garbage collection or crashes during garbage
collection, while 64-bit platforms would have seen
either ineffective or incorrect garbage collection.
You can see the invalid arguments to scanblock in the
stack traces in issue 1620.
Fixes#1620.
Fixes#1746.
R=iant, r
CC=golang-dev
https://golang.org/cl/4437075
runtime: memory allocated by OS not in usable range
runtime: out of memory: cannot allocate 1114112-byte block (2138832896 in use)
throw: out of memory
runtime.throw+0x40 /Users/rsc/g/go/src/pkg/runtime/runtime.c:102
runtime.throw(0x1fffd, 0x101)
runtime.mallocgc+0x2af /Users/rsc/g/go/src/pkg/runtime/malloc.c:60
runtime.mallocgc(0x100004, 0x0, 0x1, 0x1, 0xc093, ...)
runtime.mal+0x40 /Users/rsc/g/go/src/pkg/runtime/malloc.c:289
runtime.mal(0x100004, 0x20bc4)
runtime.new+0x26 /Users/rsc/g/go/src/pkg/runtime/malloc.c:296
runtime.new(0x100004, 0x8fe84000, 0x20bc4)
main.main+0x29 /Users/rsc/x.go:11
main.main()
runtime.mainstart+0xf /Users/rsc/g/go/src/pkg/runtime/386/asm.s:93
runtime.mainstart()
runtime.goexit /Users/rsc/g/go/src/pkg/runtime/proc.c:178
runtime.goexit()
----- goroutine created by -----
_rt0_386+0xbf /Users/rsc/g/go/src/pkg/runtime/386/asm.s:80
R=iant, r
CC=golang-dev
https://golang.org/cl/4444073
In a GOROOT path a backslash is a path separator
not an escape character. For example, `C:\go`.
Fixes gotest error:
version.go:3: unknown escape sequence: g
R=rsc
CC=golang-dev
https://golang.org/cl/4437076
Used to fault trying to access l->list->next
when l->list == nil after MCentral_AllocList.
Now prints
runtime: out of memory: no room in arena for 65536-byte allocation (536870912 in use)
throw: out of memory
followed by stack trace.
Fixes#1650.
R=r, dfc
CC=golang-dev
https://golang.org/cl/4446062
Avoid getting out of synch when a function, such as main.init,
has no associated line number information. Without this the
function before main.init can skip the PC all the way to the
next function, which will cause the next function's line table
to be associated with main.init, and leave subsequent
functions with the wrong line numbers.
R=rsc
CC=golang-dev
https://golang.org/cl/4426055
go/types: update for export data format change
reflect: require package qualifiers to match during interface check
runtime: require package qualifiers to match during interface check
test: fixed bug324, adapt to be silent
Fixes#1550.
Issue 1536 remains open.
R=gri, ken2, r
CC=golang-dev
https://golang.org/cl/4442071
* Reduces malloc counts during gob encoder/decoder test from 6/6 to 3/5.
The current reflect uses Set to mean two subtly different things.
(1) If you have a reflect.Value v, it might just represent
itself (as in v = reflect.NewValue(42)), in which case calling
v.Set only changed v, not any other data in the program.
(2) If you have a reflect Value v derived from a pointer
or a slice (as in x := []int{42}; v = reflect.NewValue(x).Index(0)),
v represents the value held there. Changing x[0] affects the
value returned by v.Int(), and calling v.Set affects x[0].
This was not really by design; it just happened that way.
The motivation for the new reflect implementation was
to remove mallocs. The use case (1) has an implicit malloc
inside it. If you can do:
v := reflect.NewValue(0)
v.Set(42)
i := v.Int() // i = 42
then that implies that v is referring to some underlying
chunk of memory in order to remember the 42; that is,
NewValue must have allocated some memory.
Almost all the time you are using reflect the goal is to
inspect or to change other data, not to manipulate data
stored solely inside a reflect.Value.
This CL removes use case (1), so that an assignable
reflect.Value must always refer to some other piece of data
in the program. Put another way, removing this case would
make
v := reflect.NewValue(0)
v.Set(42)
as illegal as
0 = 42.
It would also make this illegal:
x := 0
v := reflect.NewValue(x)
v.Set(42)
for the same reason. (Note that right now, v.Set(42) "succeeds"
but does not change the value of x.)
If you really wanted to make v refer to x, you'd start with &x
and dereference it:
x := 0
v := reflect.NewValue(&x).Elem() // v = *&x
v.Set(42)
It's pretty rare, except in tests, to want to use NewValue and then
call Set to change the Value itself instead of some other piece of
data in the program. I haven't seen it happen once yet while
making the tree build with this change.
For the same reasons, reflect.Zero (formerly reflect.MakeZero)
would also return an unassignable, unaddressable value.
This invalidates the (awkward) idiom:
pv := ... some Ptr Value we have ...
v := reflect.Zero(pv.Type().Elem())
pv.PointTo(v)
which, when the API changed, turned into:
pv := ... some Ptr Value we have ...
v := reflect.Zero(pv.Type().Elem())
pv.Set(v.Addr())
In both, it is far from clear what the code is trying to do. Now that
it is possible, this CL adds reflect.New(Type) Value that does the
obvious thing (same as Go's new), so this code would be replaced by:
pv := ... some Ptr Value we have ...
pv.Set(reflect.New(pv.Type().Elem()))
The changes just described can be confusing to think about,
but I believe it is because the old API was confusing - it was
conflating two different kinds of Values - and that the new API
by itself is pretty simple: you can only Set (or call Addr on)
a Value if it actually addresses some real piece of data; that is,
only if it is the result of dereferencing a Ptr or indexing a Slice.
If you really want the old behavior, you'd get it by translating:
v := reflect.NewValue(x)
into
v := reflect.New(reflect.Typeof(x)).Elem()
v.Set(reflect.NewValue(x))
Gofix will not be able to help with this, because whether
and how to change the code depends on whether the original
code meant use (1) or use (2), so the developer has to read
and think about the code.
You can see the effect on packages in the tree in
https://golang.org/cl/4423043/.
R=r
CC=golang-dev
https://golang.org/cl/4435042
. Missing declaration of runtime.brk_();
. Argument v in runtime.SysReserve() is not used;
(I'd prefer a Plan 9-type solution...)
R=golang-dev, r, r2
CC=golang-dev
https://golang.org/cl/4368076
The list elements are already being allocated out of a
single memory buffer. We can drop the Link* pointer
following and the memory it requires, replacing it with
index operations.
The change also keeps a channel from containing a pointer
back into its own allocation block, which would create a
cycle. Blocks involved in cycles are not guaranteed to be
finalized properly, and channels depend on finalizers to
free OS-level locks on some systems. The self-reference
was keeping channels from being garbage collected.
runtime-gdb.py will need to be updated in order to dump
the content of buffered channels with the new data structure.
Fixes#1676.
R=ken2, r
CC=golang-dev
https://golang.org/cl/4411045
in gdb, 'info goroutines' and 'goroutine <n> <cmd> were crashing
because the 'g' and 'm' structures had changed a bit.
R=rsc
CC=golang-dev
https://golang.org/cl/4289077
On darwin amd64 it was impossible to create more that ~132 threads. While
investigating I noticed that go consumes almost 1TB of virtual memory per
OS thread and the reason for such a small limit of OS thread was because
process was running out of virtual memory. While looking at bsdthread_create
I noticed that on amd64 it wasn't using PTHREAD_START_CUSTOM.
If you look at http://fxr.watson.org/fxr/source/bsd/kern/pthread_synch.c?v=xnu-1228
you will see that in that case darwin will use stack pointer as stack size,
allocating huge amounts of memory for stack. This change fixes the issue
and allows for creation of up to 2560 OS threads (which appears to be some
Mac OS X limit) with relatively small virtual memory consumption.
R=rsc
CC=golang-dev
https://golang.org/cl/4289075
Fixes#1641.
Actually it side steps the real issue, which is that the
setitimer(2) implementation on OS X is not useful for
profiling of multi-threaded programs. I filed the below
using the Apple Bug Reporter.
/*
Filed as Apple Bug Report #9177434.
This program creates a new pthread that loops, wasting cpu time.
In the main pthread, it sleeps on a condition that will never come true.
Before doing so it sets up an interval timer using ITIMER_PROF.
The handler prints a message saying which thread it is running on.
POSIX does not specify which thread should receive the signal, but
in order to be useful in a user-mode self-profiler like pprof or gprof
http://code.google.com/p/google-perftoolshttp://www.delorie.com/gnu/docs/binutils/gprof_25.html
it is important that the thread that receives the signal is the one
whose execution caused the timer to expire.
Linux and FreeBSD handle this by sending the signal to the process's
queue but delivering it to the current thread if possible:
http://lxr.linux.no/linux+v2.6.38/kernel/signal.c#L802
807 /*
808 * Now find a thread we can wake up to take the signal off the queue.
809 *
810 * If the main thread wants the signal, it gets first crack.
811 * Probably the least surprising to the average bear.
812 * /
http://fxr.watson.org/fxr/source/kern/kern_sig.c?v=FREEBSD8;im=bigexcerpts#L1907
1914 /*
1915 * Check if current thread can handle the signal without
1916 * switching context to another thread.
1917 * /
On those operating systems, this program prints:
$ ./a.out
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
signal on cpu-chewing looper thread
$
The OS X kernel does not have any such preference. Its get_signalthread
does not prefer current_thread(), in contrast to the other two systems,
so the signal gets delivered to the first thread in the list that is able to
handle it, which ends up being the main thread in this experiment.
http://fxr.watson.org/fxr/source/bsd/kern/kern_sig.c?v=xnu-1456.1.26;im=excerpts#L1666
$ ./a.out
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
signal on sleeping main thread
$
The fix is to make get_signalthread use the same heuristic as
Linux and FreeBSD, namely to use current_thread() if possible
before scanning the process thread list.
*/
#include <sys/time.h>
#include <sys/signal.h>
#include <pthread.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
static void handler(int);
static void* looper(void*);
static pthread_t pmain, ploop;
int
main(void)
{
struct itimerval it;
struct sigaction sa;
pthread_cond_t cond;
pthread_mutex_t mu;
memset(&sa, 0, sizeof sa);
sa.sa_handler = handler;
sa.sa_flags = SA_RESTART;
memset(&sa.sa_mask, 0xff, sizeof sa.sa_mask);
sigaction(SIGPROF, &sa, 0);
pmain = pthread_self();
pthread_create(&ploop, 0, looper, 0);
memset(&it, 0, sizeof it);
it.it_interval.tv_usec = 10000;
it.it_value = it.it_interval;
setitimer(ITIMER_PROF, &it, 0);
pthread_mutex_init(&mu, 0);
pthread_mutex_lock(&mu);
pthread_cond_init(&cond, 0);
for(;;)
pthread_cond_wait(&cond, &mu);
return 0;
}
static void
handler(int sig)
{
static int nsig;
pthread_t p;
p = pthread_self();
if(p == pmain)
printf("signal on sleeping main thread\n");
else if(p == ploop)
printf("signal on cpu-chewing looper thread\n");
else
printf("signal on %p\n", (void*)p);
if(++nsig >= 10)
exit(0);
}
static void*
looper(void *v)
{
for(;;);
}
R=r
CC=golang-dev
https://golang.org/cl/4273113
Also fix comment.
The only caller of chanrecv initializes the value to false, so
this patch makes no difference at present. But it seems like
the right thing to do.
R=rsc
CC=golang-dev
https://golang.org/cl/4312053
* Change use of m->g0 stack (aka scheduler stack).
* Provide runtime.mcall(f) to invoke f() on m->g0 stack.
* Replace scheduler loop entry with runtime.mcall(schedule).
Runtime.mcall eliminates the need for fake scheduler states that
exist just to run a bit of code on the m->g0 stack
(Grecovery, Gstackalloc).
The elimination of the scheduler as a loop that stops and
starts using gosave and gogo fixes a bad interaction with the
way cgo uses the m->g0 stack. Cgo runs external (gcc-compiled)
C functions on that stack, and then when calling back into Go,
it sets m->g0->sched.sp below the added call frames, so that
other uses of m->g0's stack will not interfere with those frames.
Unfortunately, gogo (longjmp) back to the scheduler loop at
this point would end up running scheduler with the lower
sp, which no longer points at a valid stack frame for
a call to scheduler. If scheduler then wrote any function call
arguments or local variables to where it expected the stack
frame to be, it would overwrite other data on the stack.
I realized this possibility while debugging a problem with
calling complex Go code in a Go -> C -> Go cgo callback.
This wasn't the bug I was looking for, it turns out, but I believe
it is a real bug nonetheless. Switching to runtime.mcall, which
only adds new frames to the stack and never jumps into
functions running in existing ones, fixes this bug.
* Move cgo-related code out of proc.c into cgocall.c.
* Add very large comment describing cgo call sequences.
* Simpilify, regularize cgo function implementations and names.
* Add test suite as misc/cgo/test.
Now the Go -> C path calls cgocall, which calls asmcgocall,
and the C -> Go path calls cgocallback, which calls cgocallbackg.
The shuffling, which affects mainly the callback case, moves
most of the callback implementation to cgocallback running
on the m->curg stack (not the m->g0 scheduler stack) and
only while accounted for with $GOMAXPROCS (between calls
to exitsyscall and entersyscall).
The previous callback code did not block in startcgocallback's
approximation to exitsyscall, so if, say, the garbage collector
were running, it would still barge in and start doing things
like call malloc. Similarly endcgocallback's approximation of
entersyscall did not call matchmg to kick off new OS threads
when necessary, which caused the bug in issue 1560.
Fixes#1560.
R=iant
CC=golang-dev
https://golang.org/cl/4253054
This functionality might be used in environments
where programs are limited to a single thread,
to simulate a select-driven network server. It is
not exposed via the standard runtime API.
R=r, r2
CC=golang-dev
https://golang.org/cl/4254041
Using the kernel-supplied compare-and-swap code
on linux/arm means that runtime doesn't have to care
whether this is GOARM=5 or GOARM=6 anymore.
Fixes#1494.
R=r, r2
CC=golang-dev
https://golang.org/cl/4245043
The pointer will eventually let us find *T given T.
This CL just makes room for it, always storing a zero.
R=r, r2
CC=golang-dev
https://golang.org/cl/4221046
In CL 4188061 I changed malg to allocate the requested
number of bytes n, not n+StackGuard, so that the
allocations would use rounder numbers.
The allocation of the signal stack asks for 32k and
then used g->stackguard as the base, but g->stackguard
is StackGuard bytes above the base. Previously, asking
for 32k meant getting 32k+StackGuard bytes, so using
g->stackguard as the base was safe. Now, the actual base
must be computed, so that the signal handler does not
run StackGuard bytes past the top of the stack.
Was causing flakiness mainly in programs that use the
network, because they sometimes write to closed network
connections, causing SIGPIPEs. Was also causing problems
in the doc/progs test.
Also fix Makefile so that changes to stack.h trigger rebuild.
R=bradfitzgo, r, r2
CC=golang-dev
https://golang.org/cl/4230044
Avoids deadlocks like the one below, in which a stack split happened
in order to call lock(&stacks), but then the stack unsplit cannot run
because stacks is now locked.
The only code calling stackalloc that wasn't on a scheduler
stack already was malg, which creates a new goroutine.
runtime.futex+0x23 /home/rsc/g/go/src/pkg/runtime/linux/amd64/sys.s:139
runtime.futex()
futexsleep+0x50 /home/rsc/g/go/src/pkg/runtime/linux/thread.c:51
futexsleep(0x5b0188, 0x300000003, 0x100020000, 0x4159e2)
futexlock+0x85 /home/rsc/g/go/src/pkg/runtime/linux/thread.c:119
futexlock(0x5b0188, 0x5b0188)
runtime.lock+0x56 /home/rsc/g/go/src/pkg/runtime/linux/thread.c:158
runtime.lock(0x5b0188, 0x7f0d27b4a000)
runtime.stackfree+0x4d /home/rsc/g/go/src/pkg/runtime/malloc.goc:336
runtime.stackfree(0x7f0d27b4a000, 0x1000, 0x8, 0x7fff37e1e218)
runtime.oldstack+0xa6 /home/rsc/g/go/src/pkg/runtime/proc.c:705
runtime.oldstack()
runtime.lessstack+0x22 /home/rsc/g/go/src/pkg/runtime/amd64/asm.s:224
runtime.lessstack()
----- lessstack called from goroutine 2 -----
runtime.lock+0x56 /home/rsc/g/go/src/pkg/runtime/linux/thread.c:158
runtime.lock(0x5b0188, 0x40a5e2)
runtime.stackalloc+0x55 /home/rsc/g/go/src/pkg/runtime/malloc.c:316
runtime.stackalloc(0x1000, 0x4055b0)
runtime.malg+0x3d /home/rsc/g/go/src/pkg/runtime/proc.c:803
runtime.malg(0x1000, 0x40add9)
runtime.newproc1+0x12b /home/rsc/g/go/src/pkg/runtime/proc.c:854
runtime.newproc1(0xf840027440, 0x7f0d27b49230, 0x0, 0x49f238, 0x40, ...)
runtime.newproc+0x2f /home/rsc/g/go/src/pkg/runtime/proc.c:831
runtime.newproc(0x0, 0xf840027440, 0xf800000010, 0x44b059)
...
R=r, r2
CC=golang-dev
https://golang.org/cl/4216045
A terminal panic (one that prints a stack trace and exits)
has been calling runtime.breakpoint before calling exit,
so that if running under a debugger, the debugger can
take control. When not running under a debugger, though,
this causes an additional SIGTRAP on Unix and pop-up
dialogs on Windows.
Support for debugging Go programs has gotten good
enough that we can rely on the debugger to set its own
breakpoint on runtime.exit if it wants to look around.
R=r, r2
CC=golang-dev
https://golang.org/cl/4222043
The existing code assumed that signals only arrived
while executing on the goroutine stack (g == m->curg),
not while executing on the scheduler stack (g == m->g0).
Most of the signal handling trampolines correctly saved
and restored g already, but the sighandler C code did not
have access to it.
Some rewriting of assembly to make the various
implementations as similar as possible.
Will need to change Windows too but I don't
understand how sigtramp gets called there.
R=r
CC=golang-dev
https://golang.org/cl/4203042
With this change, a panic trace due to a signal arriving while
running on the scheduler stack during a lessstack
(a stack unsplit) will trace through the lessstack to show
the state of the goroutine that was unsplitting its stack.
R=r
CC=golang-dev
https://golang.org/cl/4206042
Fix problems found.
On amd64, various library routines had bigger
stack frames than expected, because large function
calls had been added.
runtime.assertI2T: nosplit stack overflow
120 assumed on entry to runtime.assertI2T
8 after runtime.assertI2T uses 112
0 on entry to runtime.newTypeAssertionError
-8 on entry to runtime.morestack01
runtime.assertE2E: nosplit stack overflow
120 assumed on entry to runtime.assertE2E
16 after runtime.assertE2E uses 104
8 on entry to runtime.panic
0 on entry to runtime.morestack16
-8 after runtime.morestack16 uses 8
runtime.assertE2T: nosplit stack overflow
120 assumed on entry to runtime.assertE2T
16 after runtime.assertE2T uses 104
8 on entry to runtime.panic
0 on entry to runtime.morestack16
-8 after runtime.morestack16 uses 8
runtime.newselect: nosplit stack overflow
120 assumed on entry to runtime.newselect
56 after runtime.newselect uses 64
48 on entry to runtime.printf
8 after runtime.printf uses 40
0 on entry to vprintf
-8 on entry to runtime.morestack16
runtime.selectdefault: nosplit stack overflow
120 assumed on entry to runtime.selectdefault
56 after runtime.selectdefault uses 64
48 on entry to runtime.printf
8 after runtime.printf uses 40
0 on entry to vprintf
-8 on entry to runtime.morestack16
runtime.selectgo: nosplit stack overflow
120 assumed on entry to runtime.selectgo
0 after runtime.selectgo uses 120
-8 on entry to runtime.gosched
On arm, 5c was tagging functions NOSPLIT that should
not have been, like the recursive function printpanics:
printpanics: nosplit stack overflow
124 assumed on entry to printpanics
112 after printpanics uses 12
108 on entry to printpanics
96 after printpanics uses 12
92 on entry to printpanics
80 after printpanics uses 12
76 on entry to printpanics
64 after printpanics uses 12
60 on entry to printpanics
48 after printpanics uses 12
44 on entry to printpanics
32 after printpanics uses 12
28 on entry to printpanics
16 after printpanics uses 12
12 on entry to printpanics
0 after printpanics uses 12
-4 on entry to printpanics
R=r, r2
CC=golang-dev
https://golang.org/cl/4188061
