Currently freeOSMemory makes only marking phase of GC, but not sweeping phase.
So recently memory is not released after freeOSMemory.
Do both marking and sweeping during freeOSMemory.
Fixes#8019.
LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews, rsc
https://golang.org/cl/97550043
Rename Seek to seek in asm file, was overlooked in CL 99320043.
LGTM=bradfitz, r
R=r, rsc, bradfitz
CC=golang-codereviews
https://golang.org/cl/99320044
These functions claimed to return error (an interface)
and be implemented entirely in assembly, but it's not
possible to create an interface from assembly
(at least not easily).
In reality the functions were written to return an errno uintptr
despite the Go prototype saying error.
When the errno was 0, they coincidentally filled out a nil error
by writing the 0 to the type word of the interface.
If the errno was ever non-zero, the functions would
create a non-nil error that would crash when trying to
call err.Error().
Luckily these functions (Seek, Time, Gettimeofday) pretty
much never fail, so it was all kind of working.
Found by go vet.
LGTM=bradfitz, r
R=golang-codereviews, bradfitz, r
CC=golang-codereviews
https://golang.org/cl/99320043
TestLargeDefs was issuing over one million small writes to
create a 7MB file (large.go). This is quite slow on Plan 9
since our disk file systems aren't very fast and they're
usually accessed over the network.
Buffering the writes makes the test about six times faster.
Even on Linux, it's about 1.5 times faster.
Here are the results on a slow Plan 9 machine:
Before:
% ./pack.test -test.v -test.run TestLargeDefs
=== RUN TestLargeDefs
--- PASS: TestLargeDefs (125.11 seconds)
PASS
After:
% ./pack.test -test.v -test.run TestLargeDefs
=== RUN TestLargeDefs
--- PASS: TestLargeDefs (20.835 seconds)
PASS
LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/95040044
The introduction of temporaries in order.c was not
quite right for two corner cases:
1) The rewrite that pushed new variables on the lhs of
a receive into the body of the case was dropping the
declaration of the variables. If the variables escape,
the declaration is what allocates them.
Caught by escape analysis sanity check.
In fact the declarations should move into the body
always, so that we only allocate if the corresponding
case is selected. Do that. (This is an optimization that
was already present in Go 1.2. The new order code just
made it stop working.)
Fixes#7997.
2) The optimization to turn a single-recv select into
an ordinary receive assumed it could take the address
of the destination; not so if the destination is _.
Fixes#7998.
LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/100480043
Calling tar.Reader.Read() used to work fine, but without this patch it panics.
Simply return EOF to indicate the tar.Reader.Next() needs to be called.
LGTM=iant, bradfitz
R=golang-codereviews, bradfitz, iant, mikioh.mikioh, dominik.honnef
CC=golang-codereviews
https://golang.org/cl/94530043
This CL restores dropped constants not supported in OpenBSD 5.5
and tris to keep the promise of API compatibility.
Update #7049
LGTM=jsing, bradfitz
R=rsc, jsing, bradfitz
CC=golang-codereviews
https://golang.org/cl/94950043
This CL restores dropped constants not supported in OpenBSD 5.5
and tris to keep the promise of API compatibility.
Update #7049
LGTM=jsing, bradfitz, rsc
R=rsc, jsing, robert.hencke, minux.ma, bradfitz, iant
CC=golang-codereviews
https://golang.org/cl/96970043
- use Init to establish heap invariant on
a non-empty heap
- use Fix to update heap after an element's
properties have been changed
(The old code used Init where it wasn't needed,
and didn't use Fix because Fix was added after
the example was written.)
LGTM=bradfitz
R=adonovan, bradfitz
CC=golang-codereviews
https://golang.org/cl/94520043
for GOOS in darwin freebsd linux nacl netbsd openbsd plan9 solaris windows
do
for GOARCH in 386 amd64 amd64p32 arm
do
go vet
done
done
These are all real mistakes being corrected, but none
of them should be able to cause problems today
due to the NOSPLIT on the functions.
However, vet has also identified a few important problems.
I'm sending this CL to get rid of the trivial 'go vet' results
before attacking the real ones.
LGTM=r
R=golang-codereviews, r, bradfitz
CC=golang-codereviews
https://golang.org/cl/95460046
None of these are real bugs.
The variable name in the reference is not semantically meaningful,
except that 'go vet' will double check the offset against the name for you.
The stack sizes being corrected really are incorrect but they are also
in NOSPLIT functions so they typically don't matter.
Found by vet.
GOOS=linux GOARCH=amd64 go vet sync/atomic
GOOS=linux GOARCH=amd64p32 go vet sync/atomic
GOOS=linux GOARCH=386 go vet sync/atomic
GOOS=linux GOARCH=arm go vet sync/atomic
GOOS=freebsd GOARCH=arm go vet sync/atomic
GOOS=netbsd GOARCH=arm go vet sync/atomic
LGTM=r
R=r, bradfitz
CC=golang-codereviews
https://golang.org/cl/100500043
The GC program describing a data structure sometimes trusts the
pointer base type and other times does not (if not, the garbage collector
must fall back on per-allocation type information stored in the heap).
Make the scanning of a pointer in an interface do the same.
This fixes a crash in a particular use of reflect.SliceHeader.
Fixes#8004.
LGTM=khr
R=golang-codereviews, khr
CC=0xe2.0x9a.0x9b, golang-codereviews, iant, r
https://golang.org/cl/100470045
Globals, function arguments, and results are special cases in
registerization.
Globals must be flushed aggressively, because nearly any
operation can cause a panic, and the recovery code must see
the latest values. Globals also must be loaded aggressively,
because nearly any store through a pointer might be updating a
global: the compiler cannot see all the "address of"
operations on globals, especially exported globals. To
accomplish this, mark all globals as having their address
taken, which effectively disables registerization.
If a function contains a defer statement, the function results
must be flushed aggressively, because nearly any operation can
cause a panic, and the deferred code may call recover, causing
the original function to return the current values of its
function results. To accomplish this, mark all function
results as having their address taken if the function contains
any defer statements. This causes not just aggressive flushing
but also aggressive loading. The aggressive loading is
overkill but the best we can do in the current code.
Function arguments must be considered live at all safe points
in a function, because garbage collection always preserves
them: they must be up-to-date in order to be preserved
correctly. Accomplish this by marking them live at all call
sites. An earlier attempt at this marked function arguments as
having their address taken, which disabled registerization
completely, making programs slower. This CL's solution allows
registerization while preserving safety. The benchmark speedup
is caused by being able to registerize again (the earlier CL
lost the same amount).
benchmark old ns/op new ns/op delta
BenchmarkEqualPort32 61.4 56.0 -8.79%
benchmark old MB/s new MB/s speedup
BenchmarkEqualPort32 521.56 570.97 1.09x
Fixes#1304. (again)
Fixes#7944. (again)
Fixes#7984.
Fixes#7995.
LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews, iant, r
https://golang.org/cl/97500044
The function takes 32 bytes of arguments: 8 for the *block
and then 3*8 for the slice.
The 24 is not causing a bug (today at least) because the
final word is the cap of the slice, which the assembly
does not use.
Identified by 'go vet std'.
LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/96360043
Turns out elf.File.Sections is indexed by the actual
section number, not the number minus one.
I don't know why I thought the -1 was necessary.
Fixes objdump test (and therefore build) on ELF systems.
While we're here, fix bounds on gnuDump so that we
don't crash when asked to disassemble outside
the text segment. May fix Windows build or at least
make the failure more interesting.
TBR=iant
CC=golang-codereviews
https://golang.org/cl/92390043
There is some duplication here with cmd/nm.
There is a TODO to address that after 1.3 is out.
Update #7452
x86 disassembly works and is tested.
The arm disassembler does not exist yet
and is therefore not yet hooked up.
LGTM=crawshaw, iant
R=crawshaw, iant
CC=golang-codereviews
https://golang.org/cl/91360046
Make it a little clearer how they are used, in particular that
it is not enough just to return a nil pointer on error, but also
to return an error value explaining the problem.
Fixes#1963.
LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/97360045
- use previously defined terms (with links) throughout
- specify evaluation order more precisely (in particular,
the evaluation time of rhs expressions in receive cases
was not specified)
- added extra example case
Not a language change.
Description matches observed behavior of code compiled
with gc and gccgo.
Fixes#7669.
LGTM=iant, r, rsc
R=r, rsc, iant, ken, josharian
CC=golang-codereviews
https://golang.org/cl/91230043
Do not use ustar format if we need the GNU one.
Change \000 to \x00 for consistency
Check for "ustar\x00" instead of "ustar\x00\x00" for conistency with tar
and compatiblity with archive generated with older code (which was ustar\x00\x20\x00)
Add test for long name + big file.
LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/99050043
AddressSanitizer says:
AddressSanitizer: heap-buffer-overflow on address 0x60200001b6f3
READ of size 6 at 0x60200001b6f3 thread T0
#0 0x46741b in __interceptor_memcmp asan_interceptors.cc:337
#1 0x4b5794 in compile src/cmd/6g/../gc/pgen.c:177
#2 0x509b81 in funccompile src/cmd/gc/dcl.c:1457
#3 0x520fe2 in p9main src/cmd/gc/lex.c:489
#4 0x5e2e01 in main src/lib9/main.c:57
#5 0x7fab81f7976c in __libc_start_main /build/buildd/eglibc-2.15/csu/libc-start.c:226
#6 0x4b16dc in _start (pkg/tool/linux_amd64/6g+0x4b16dc)
0x60200001b6f3 is located 0 bytes to the right of 3-byte region [0x60200001b6f0,0x60200001b6f3)
allocated by thread T0 here:
#0 0x493ec8 in __interceptor_malloc asan_malloc_linux.cc:75
#1 0x54d64e in mal src/cmd/gc/subr.c:459
#2 0x5260d5 in yylex src/cmd/gc/lex.c:1605
#3 0x52078f in p9main src/cmd/gc/lex.c:402
#4 0x5e2e01 in main src/lib9/main.c:57
If the memory block happens to be at the end of hunk and page bounadry,
this out-of-bounds can lead to a crash.
LGTM=dave, iant
R=golang-codereviews, dave, iant
CC=golang-codereviews
https://golang.org/cl/93370043
The code recurs very deeply in cases like (?:x{1,1000}){1,1000}
Since if much time is spent checking whether one pass is possible, it's not
worth doing at all, a simple fix is proposed: Stop if the check takes too long.
To do this, we simply avoid machines with >1000 instructions.
Benchmarks show a percent or less change either way, effectively zero.
Fixes#7608.
LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/92290043
If a map variable is created with reflect.New it has incorrect type (map[unsafe.Pointer]unsafe.Pointer).
If GC follows such pointer, it scans Hmap and buckets with incorrect type.
This can lead to overscan of up to 120 bytes for map[int8]struct{}.
Which in turn can lead to crash if the memory after a bucket object is unaddressable
or false retention (buckets are scanned as arrays of unsafe.Pointer).
I don't see how it can lead to heap corruptions, though.
LGTM=khr
R=rsc, khr
CC=golang-codereviews
https://golang.org/cl/96270044
mstats.last_gc is unix time now, it is compared with abstract monotonic time.
On my machine GC is forced every 5 mins regardless of last_gc.
LGTM=rsc
R=golang-codereviews
CC=golang-codereviews, iant, rsc
https://golang.org/cl/91350045
I have no test case for this at tip.
The original report included a program crashing at revision 88ac7297d2fa.
I tested this code at that revision and it does fix the crash.
However, at tip the reported code no longer crashes, presumably
because some allocation patterns have changed. I believe the
bug is still present at tip and that this code still fixes it.
Fixes#7143.
LGTM=alex.brainman
R=golang-codereviews, alex.brainman
CC=dvyukov, golang-codereviews
https://golang.org/cl/96300046
