Periodically browse MHeap's freelists for long unused spans and release them if any.
Current hardcoded settings:
- GC is forced if none occured over the last 2 minutes.
- spans are handed back after 5 minutes of uselessness.
SysUnused (for Unix) is a wrapper on madvise MADV_DONTNEED on Linux and MADV_FREE on BSDs.
R=rsc, dvyukov, remyoudompheng
CC=golang-dev
https://golang.org/cl/5451057
Changeset 36c9c7810f14 broke support for grsec-patched kernels.
Those do not give back the address requested without MAP_FIXED,
so when verifying an mmap without this flag for success, the
resulting address must not be compared against the requested
address since it may have succeeded at a different location.
R=golang-dev, rsc, gustavo, iant
CC=golang-dev
https://golang.org/cl/5650072
before after
go test -short time 4.144s 1.215s
go test -short runtime 1.315s 0.351s
go test -short -cpu=1,2,4 runtime 4.376s 1.266s
Partially solves issue 3015.
R=golang-dev, r
CC=golang-dev, remy
https://golang.org/cl/5673045
It is possible that Linux and Windows copy the FP control word
from the parent thread when creating a new thread. Empirically,
Darwin does not. Reset the FP control world in all cases.
Enable the floating-point strconv test.
Fixes#2917 (again).
R=golang-dev, r, iant
CC=golang-dev
https://golang.org/cl/5660047
It's not as pretty, but it deletes some irrelevant information from the
printout and avoids a dependency.
It also means the test binary will stop if a test panics. That's a feature,
not a bug.
Any output printed by the test appears before the panic traceback.
before:
--- FAIL: TestPanic (0.00 seconds)
fmt_test.go:19: HI
testing.go:257: runtime error: index out of range
/Users/r/go/src/pkg/testing/testing.go:257 (0x23998)
_func_003: t.Logf("%s\n%s", err, debug.Stack())
/Users/r/go/src/pkg/runtime/proc.c:1388 (0x10d2d)
panic: reflect·call(d->fn, d->args, d->siz);
/Users/r/go/src/pkg/runtime/runtime.c:128 (0x119b0)
panicstring: runtime·panic(err);
/Users/r/go/src/pkg/runtime/runtime.c:85 (0x11857)
panicindex: runtime·panicstring("index out of range");
/Users/r/go/src/pkg/fmt/fmt_test.go:21 (0x23d72)
TestPanic: a[10]=1
/Users/r/go/src/pkg/testing/testing.go:264 (0x21b75)
tRunner: test.F(t)
/Users/r/go/src/pkg/runtime/proc.c:258 (0xee9e)
goexit: runtime·goexit(void)
FAIL
after:
--- FAIL: TestPanic (0.00 seconds)
fmt_test.go:19: HI
panic: runtime error: index out of range [recovered]
panic: (*testing.T) (0xec3b0,0xf8400001c0)
goroutine 2 [running]:
testing._func_003(0x21f5fa8, 0x21f5100, 0x21f5fb8, 0x21f5e88)
/Users/r/go/src/pkg/testing/testing.go:259 +0x108
----- stack segment boundary -----
fmt_test.TestPanic(0xf8400001c0, 0x27603728)
/Users/r/go/src/pkg/fmt/fmt_test.go:21 +0x6b
testing.tRunner(0xf8400001c0, 0x18edb8, 0x0, 0x0)
/Users/r/go/src/pkg/testing/testing.go:264 +0x6f
created by testing.RunTests
/Users/r/go/src/pkg/testing/testing.go:343 +0x76e
goroutine 1 [chan receive]:
testing.RunTests(0x2000, 0x18edb8, 0x2400000024, 0x100000001, 0x200000001, ...)
/Users/r/go/src/pkg/testing/testing.go:344 +0x791
testing.Main(0x2000, 0x18edb8, 0x2400000024, 0x188a58, 0x800000008, ...)
/Users/r/go/src/pkg/testing/testing.go:275 +0x62
main.main()
/var/folders/++/+++Fn+++6+0++4RjPqRgNE++2Qk/-Tmp-/go-build743922747/fmt/_test/_testmain.go:129 +0x91
exit status 2
R=rsc, dsymonds
CC=golang-dev
https://golang.org/cl/5658048
1, IMO, the fatal error "regfree: not a register" from 5g when
compiling runtime/debug.go is due to gcc miscompile, it doesn't
show up when compiled with -O0. But I still haven't thought of
a way to fix this, should all ARM builds be built with -O0?
2, fixed mksysnum_linux.pl, so zsysnum_linux_arm.go no longer
needs to be hand-generated.
3, regen all in pkg syscall for Linux/ARM on Debian 6.0
This CL is somewhat big, I'd like to split it if necessary.
R=rsc, dave
CC=golang-dev
https://golang.org/cl/5659044
Restore package os/signal, with new API:
Notify replaces Incoming, allowing clients
to ask for certain signals only. Also, signals
go to everyone who asks, not just one client.
This could plausibly move into package os now
that there are no magic side effects as a result
of the import.
Update runtime for new API: move common Unix
signal handling code into signal_unix.c.
(It's so easy to do this now that we don't have
to edit Makefiles!)
Tested on darwin,linux 386,amd64.
Fixes#1266.
R=r, dsymonds, bradfitz, iant, borman
CC=golang-dev
https://golang.org/cl/3749041
Among other things, this avoids putting a testing.go:nnn:
prefix on every line of the stack trace.
R=golang-dev, r, dsymonds, r
CC=golang-dev
https://golang.org/cl/5651081
unsafe: delete Typeof, Reflect, Unreflect, New, NewArray
Part of issue 2955 and issue 2968.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5650069
Otherwise lockorder may be misaligned, since lockorder is a
list of pointers and pollorder is a list of uint16.
Discovered running gccgo (which uses a modified copy of this
code) on SPARC.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5655054
If the values being compared have different concrete types,
then they're clearly unequal without needing to invoke the
actual interface compare routine. This speeds tests for
specific values, like if err == io.EOF, by about 3x.
benchmark old ns/op new ns/op delta
BenchmarkIfaceCmp100 843 287 -65.95%
BenchmarkIfaceCmpNil100 184 182 -1.09%
Fixes#2591.
R=ken2
CC=golang-dev
https://golang.org/cl/5651073
On 64 bit UML it is not possible to reserve memory at 0xF8<<32.
Detect when linux cannot use these high virtual memory addresses
and drop back to the 32 bit memory allocator.
R=rsc, cw
CC=golang-dev
https://golang.org/cl/5634050
Same idea as heap profile: how did each thread get created?
Low memory (256 bytes per OS thread), high reward for
programs that suddenly have many threads running.
Fixes#1477.
R=golang-dev, r, dvyukov
CC=golang-dev
https://golang.org/cl/5639059
Unexports runtime.MemStats and rename MemStatsType to MemStats.
The new accessor requires passing a pointer to a user-allocated
MemStats structure.
Fixes#2572.
R=bradfitz, rsc, bradfitz, gustavo
CC=golang-dev, remy
https://golang.org/cl/5616072
Multiplying by the low 32 bits was a bad idea
no matter what, but it was a particularly unfortunate
choice because those bits are 0 for small integer values.
Fixes#2883.
R=ken2
CC=golang-dev
https://golang.org/cl/5634047
Right now, GOTRACEBACK=0 means do not show any stack traces.
Unset means the default behavior (declutter by hiding runtime routines).
This CL makes GOTRACEBACK=2 mean include the runtime routines.
It avoids having to recompile the runtime when you want to see
the runtime in the tracebacks.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5633050
The go- is redundant now that the directory is required
to be inside $GOROOT. Rob LGTMed the idea.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5618044
This patch adds a function to get the current cpu ticks. This is
deemed to be 'sufficiently random' to use to seed fastrand to mitigate
the algorithmic complexity attacks on the hash table implementation.
On AMD64 we use the RDTSC instruction. For 386, this instruction,
while valid, is not recognized by 8a so I've inserted the opcode by
hand. For ARM, this routine is currently stubbed to return a constant
0 value.
Future work: update 8a to recognize RDTSC.
Fixes#2630.
R=rsc
CC=golang-dev
https://golang.org/cl/5606048
This patch adds a hash seed to the Hmap struct. Each seed is
initialized by runtime.fastrand1(). This is the first step of a
solution to issue 2630. Fastrand1 still needs to be updated to provide
us with actually random bits.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5599046
Consequently, remove many package Makefiles,
and shorten the few that remain.
gomake becomes 'go tool make'.
Turn off test phases of run.bash that do not work,
flagged with $BROKEN. Future CLs will restore these,
but this seemed like a big enough CL already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
We weren't properly deleting the various header
files (that were temporarily renamed) if a $CC
for the current $GOARCH didn't exist. And since
the compiler checks the current directory for
headers before any -I arguments, this had the
unfortunate side effect of including the last
generated headers instead of the correct ones.
R=r, rsc
CC=golang-dev
https://golang.org/cl/5581055
Also delete gotest, since it's messy to fix and slated for deletion anyway.
A couple of things outside src can't be tested any more. "go test" will be
fixed and these tests will be re-enabled. They're noisy for now.
Fixes#284.
R=rsc
CC=golang-dev
https://golang.org/cl/5598049
In order to allow buildscript.sh to generate buildscripts for all
$GOOS/$GOARCH combinations, we have to generate dummy files for cmd/go.
Fixes#2586.
R=rsc, golang-dev
CC=golang-dev
https://golang.org/cl/5557050
- use proper Win64 gcc calling convention when
calling initcgo on amd64
- increase g0 stack size to 64K on amd64 to make
it the same as 386
- implement C.sleep
- do not use C.stat, since it is renamed to C._stat by mingw
- use fopen to implement TestErrno, since C.strtol
always succeeds on windows
- skip TestSetEnv on windows, because os.Setenv
sets windows process environment, while C.getenv
inspects internal C runtime variable instead
R=golang-dev, vcc.163, rsc
CC=golang-dev
https://golang.org/cl/5500094
pkg/runtime/sys_darwin_amd64.s: fixes syscall select nr
pkg/runtime/sys_linux_arm.s: uses newselect instead of the now unimplemented
(old) select, also fixes the wrong div/mod statements in runtime.usleep.
Fixes#2633
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/5504096
If something goes wrong, it should suffice to set
USE_GO_TOOL=false in env.bash to fall back to the
makefiles. I will delete the makefiles in January.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5502047
Also rename -v to -x in the build and install commands,
to match the flag in go test (which we can't change
because -v is taken). Matches sh -x anyway.
R=r, iant, ality
CC=golang-dev
https://golang.org/cl/5504045
This is like the ill-fated CL 5493063 except that
I have written a shell script (autogen.sh) instead of
thinking I could possibly write a correct Makefile.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5496075
That was the last build that was close to working.
I will try that change again next week.
Make is being very subtle today.
At the reverted-to CL, the ARM traceback appears
to be broken. I'll look into that next week too.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5492063
Why it was not failing anywhere else I don't know,
but the Makefile was definitely wrong. The rules
must not run in parallel.
TBR=r
CC=golang-dev
https://golang.org/cl/5489069
I am looking forward to not supporting two build
systems simultaneously. Make complains about
a circular dependency still, but I don't understand it
and it's probably not worth the time to figure out.
TBR=r
CC=golang-dev
https://golang.org/cl/5496058
Collapse the arch,os-specific directories into the main directory
by renaming xxx/foo.c to foo_xxx.c, and so on.
There are no substantial edits here, except to the Makefile.
The assumption is that the Go tool will #define GOOS_darwin
and GOARCH_amd64 and will make any file named something
like signals_darwin.h available as signals_GOOS.h during the
build. This replaces what used to be done with -I$(GOOS).
There is still work to be done to make runtime build with
standard tools, but this is a big step. After this we will have
to write a script to generate all the generated files so they
can be checked in (instead of generated during the build).
R=r, iant, r, lucio.dere
CC=golang-dev
https://golang.org/cl/5490053
Testing total space fails for gccgo when not using split
stacks, because then each goroutine has a large stack, and so
the total memory usage is large.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5487068
This will be nicer to the automatic tools.
It requires a few more assembly stubs
but fewer Go files.
There are a few instances where it looks like
there are new blobs of code, but they are just
being copied out of deleted files.
There is no new code here.
Suppose you have a portable implementation for Sin
and a 386-specific assembly one. The old way to
do this was to write three files
sin_decl.go
func Sin(x float64) float64 // declaration only
sin_386.s
assembly implementation
sin_port.go
func Sin(x float64) float64 { ... } // pure-Go impl
and then link in either sin_decl.go+sin_386.s or
just sin_port.go. The Makefile actually did the magic
of linking in only the _port.go files for those without
assembly and only the _decl.go files for those with
assembly, or at least some of that magic.
The biggest problem with this, beyond being hard
to explain to the build system, is that once you do
explain it to the build system, godoc knows which
of sin_port.go or sin_decl.go are involved on a given
architecture, and it (correctly) ignores the other.
That means you have to put identical doc comments
in both files.
The new approach, which is more like what we did
in the later packages math/big and sync/atomic,
is to have
sin.go
func Sin(x float64) float64 // decl only
func sin(x float64) float64 {...} // pure-Go impl
sin_386.s
// assembly for Sin (ignores sin)
sin_amd64.s
// assembly for Sin: jmp sin
sin_arm.s
// assembly for Sin: jmp sin
Once we abandon Makefiles we can put all the assembly
stubs in one source file, so the number of files will
actually go down.
Chris asked whether the branches cost anything.
Given that they are branching to pure-Go implementations
that are not typically known for their speed, the single
direct branch is not going to be noticeable. That is,
it's on the slow path.
An alternative would have been to preserve the old
"only write assembly files when there's an implementation"
and still have just one copy of the declaration of Sin
(and thus one doc comment) by doing:
sin.go
func Sin(x float64) float64 { return sin(x) }
sin_decl.go
func sin(x float64) float64 // declaration only
sin_386.s
// assembly for sin
sin_port.go
func sin(x float64) float64 { portable code }
In this version everyone would link in sin.go and
then either sin_decl.go+sin_386.s or sin_port.go.
This has an extra function call on all paths, including
the "fast path" to get to assembly, and it triples the
number of Go files involved compared to what I did
in this CL. On the other hand you don't have to
write assembly stubs. After starting down this path
I decided that the assembly stubs were the easier
approach.
As for generating the assembly stubs on the fly, much
of the goal here is to eliminate magic from the build
process, so that zero-configuration tools like goinstall
or the new go tool can handle this package.
R=golang-dev, r, cw, iant, r
CC=golang-dev
https://golang.org/cl/5488057
To allow these types as map keys, we must fill in
equal and hash functions in their algorithm tables.
Structs or arrays that are "just memory", like [2]int,
can and do continue to use the AMEM algorithm.
Structs or arrays that contain special values like
strings or interface values use generated functions
for both equal and hash.
The runtime helper func runtime.equal(t, x, y) bool handles
the general equality case for x == y and calls out to
the equal implementation in the algorithm table.
For short values (<= 4 struct fields or array elements),
the sequence of elementwise comparisons is inlined
instead of calling runtime.equal.
R=ken, mpimenov
CC=golang-dev
https://golang.org/cl/5451105
I had to move readFile into sys_$GOOS.go
since syscall.Open takes only two arguments
on Plan 9.
R=lucio.dere, rsc, alex.brainman
CC=golang-dev
https://golang.org/cl/5447061
Equality on structs will require arbitrary code for type equality,
so change algorithm in type data from uint8 to table pointer.
In the process, trim top-level map structure from
104/80 bytes (64-bit/32-bit) to 24/12.
Equality on structs will require being able to call code generated
by the Go compiler, and C code has no way to access Go return
values, so change the hash and equal algorithm functions to take
a pointer to a result instead of returning the result.
R=ken
CC=golang-dev
https://golang.org/cl/5453043
The environment is needed by package time, which
we want not to depend on os (so that os can use
time.Time), so push down into syscall.
Delete syscall.Sleep, now unnecessary.
The package os environment API is preserved;
it is only the implementation that is moving to syscall.
Delete os.Envs, which was undocumented,
uninitialized on Windows and Plan 9, and
not maintained by Setenv and Clearenv.
Code can call os.Environ instead.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5370091
The timespec passed to thrsleep() needs to be an absolute/realtime
value, so add the current nanotime to ns.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5374048
This looks like it is just moving some code from
time to runtime (and translating it to C), but the
runtime can do a better job managing the goroutines,
and it needs this functionality for its own maintenance
(for example, for the garbage collector to hand back
unused memory to the OS on a time delay).
Might as well have just one copy of the timer logic,
and runtime can't depend on time, so vice versa.
It also unifies Sleep, NewTicker, and NewTimer behind
one mechanism, so that there are no claims that one
is more efficient than another. (For example, today
people recommend using time.After instead of time.Sleep
to avoid blocking an OS thread.)
Fixes#1644.
Fixes#1731.
Fixes#2190.
R=golang-dev, r, hectorchu, iant, iant, jsing, alex.brainman, dvyukov
CC=golang-dev
https://golang.org/cl/5334051
Fixes crash when cgo consumes more than 8K
of stack and makes a callback.
Fixes#1328.
R=golang-dev, rogpeppe, rsc
CC=golang-dev, mpimenov
https://golang.org/cl/5371042
Otherwise some OS X toolchains complain about the redeclaration
of libcgo_thread_start by multiple object files. The real definition
is in util.c.
Fixes#2167.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5364045
- Fix function prototype for thrsleep().
- Provide enums for clock identifiers.
- Provide timespec structure for use with thrsleep().
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/5360042
runtime knows how to get the time of day
without allocating memory.
R=golang-dev, dsymonds, dave, hectorchu, r, cw
CC=golang-dev
https://golang.org/cl/5297078
We only guarantee that the main goroutine runs on the
main OS thread for initialization. Programs that wish to
preserve that property for main.main can call runtime.LockOSThread.
This is what programs used to do before we unleashed
goroutines during init, so it is both a simple fix and keeps
existing programs working.
R=iant, r, dave, dvyukov
CC=golang-dev
https://golang.org/cl/5309070
Revert workaround in compiler and
revert test for compiler workaround.
Tested that the 386 build continues to fail if
the gc change is made without the reflect change.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5312041
The old m[x] = 0, false syntax will be deleted
in a month or so, once people have had time to
change their code (there is a gofix in a separate CL).
R=ken2
CC=golang-dev
https://golang.org/cl/5265048
New DLL and Proc types to manage and call dll functions. These were
used to simplify syscall tests in runtime package. They were also
used to implement LazyDLL and LazyProc.
LazyProc, like Proc, now have Call function, that just a wrapper for
SyscallN. It is not as efficient as Syscall, but easier to use.
NewLazyDLL now supports non-ascii filenames.
LazyDLL and LazyProc now have Load and Find methods. These can be used
during runtime to discover if some dll functions are not present.
All dll functions now return errors that fit os.Error interface. They
also contain Windows error number.
Some of these changes are suggested by jp.
R=golang-dev, jp, rsc
CC=golang-dev
https://golang.org/cl/5272042
The work buffer management used by the garbage
collector during parallel collections leaks buffers.
This CL tests for and fixes the leak.
R=golang-dev, dvyukov, r
CC=golang-dev
https://golang.org/cl/5254059
Use FlagNoPointers and do not zeroize memory when allocate strings.
test/garbage/parser.out old new
run #1 32.923s 32.065s
run #2 33.047s 31.931s
run #3 32.702s 31.841s
run #4 32.718s 31.838s
run #5 32.702s 31.868s
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5259041
Implement a locking model based on the current linux model - a
tri-state mutex with active spinning, passive spinning and sleeping.
R=golang-dev, dvyukov, rsc
CC=golang-dev
https://golang.org/cl/4974043
The malloc sample trigger was not being set in a
new m, so the first allocation in each new m - the
goroutine structure - was being sampled with
probability 1 instead of probability sizeof(G)/rate,
an oversampling of about 5000x for the default
rate of 1 MB. This bug made pprof graphs show
far more G allocations than there actually were.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5224041
Fixes#2337.
Unfortunate sequence of events is:
1. maxcpu=2, mcpu=1, grunning=1
2. starttheworld creates an extra M:
maxcpu=2, mcpu=2, grunning=1
4. the goroutine calls runtime.GOMAXPROCS(1)
maxcpu=1, mcpu=2, grunning=1
5. since it sees mcpu>maxcpu, it calls gosched()
6. schedule() deschedules the goroutine:
maxcpu=1, mcpu=1, grunning=0
7. schedule() call getnextandunlock() which
fails to pick up the goroutine again,
because canaddcpu() fails, because mcpu==maxcpu
8. then it sees that grunning==0,
reports deadlock and terminates
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5191044
When ncpu < 2, work.nproc is always 1 which results in infinite helper
threads being created if gomaxprocs > 1 and MaxGcproc > 1. Avoid this
by using the same limits as imposed helpgc().
R=golang-dev, rsc, dvyukov
CC=golang-dev
https://golang.org/cl/5176044
This change adds the osyield and usleep
functions and code to read the number of
processors from /dev/sysstat.
I also changed SysAlloc to return nil
when brk fails (it was returning -1).
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5177049
