Currently we track the per-G GC assist balance as two monotonically
increasing values: the bytes allocated by the G this cycle (gcalloc)
and the scan work performed by the G this cycle (gcscanwork). The
assist balance is hence assistRatio*gcalloc - gcscanwork.
This works, but has two important downsides:
1) It requires floating-point math to figure out if a G is in debt or
not. This makes it inappropriate to check for assist debt in the
hot path of mallocgc, so we only do this when a G allocates a new
span. As a result, Gs can operate "in the red", leading to
under-assist and extended GC cycle length.
2) Revising the assist ratio during a GC cycle can lead to an "assist
burst". If you think of plotting the scan work performed versus
heaps size, the assist ratio controls the slope of this line.
However, in the current system, the target line always passes
through 0 at the heap size that triggered GC, so if the runtime
increases the assist ratio, there has to be a potentially large
assist to jump from the current amount of scan work up to the new
target scan work for the current heap size.
This commit replaces this approach with directly tracking the GC
assist balance in terms of allocation credit bytes. Allocating N bytes
simply decreases this by N and assisting raises it by the amount of
scan work performed divided by the assist ratio (to get back to
bytes).
This will make it cheap to figure out if a G is in debt, which will
let us efficiently check if an assist is necessary *before* performing
an allocation and hence keep Gs "in the black".
This also fixes assist bursts because the assist ratio is now in terms
of *remaining* work, rather than work from the beginning of the GC
cycle. Hence, the plot of scan work versus heap size becomes
continuous: we can revise the slope, but this slope always starts from
where we are right now, rather than where we were at the beginning of
the cycle.
Change-Id: Ia821c5f07f8a433e8da7f195b52adfedd58bdf2c
Reviewed-on: https://go-review.googlesource.com/15408
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently we ensure a minimum heap distance of 1MB when computing the
assist ratio. Rather than enforcing this minimum on the heap distance,
it makes more sense to enforce that the heap goal itself is at least
1MB over the live heap size at the beginning of GC. Currently the two
approaches are semantically equivalent, but this will let us switch to
basing the assist ratio on current heap distance rather than the
initial heap distance, since we can't enforce this minimum on the
current heap distance (the GC may never finish because the goal posts
will always be 1MB away).
Change-Id: I0027b1c26a41a0152b01e5b67bdb1140d43ee903
Reviewed-on: https://go-review.googlesource.com/15604
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, gcController.scanWork is updated as lazily as possible
since it is only read at the end of the GC cycle. We're about to read
it during the GC cycle to improve the assist ratio revisions, so
modify gcDrain* to regularly flush to gcController.scanWork in much
the same way as we regularly flush to gcController.bgScanCredit.
One consequence of this is that it's difficult to keep gcw.scanWork
monotonic, so we give up on that and simply return the amount of scan
work done by gcDrainN rather than calculating it in the caller.
Change-Id: I7b50acdc39602f843eed0b5c6d2dacd7e762b81d
Reviewed-on: https://go-review.googlesource.com/15407
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently callers of gcDrain control whether it flushes scan work
credit to gcController.bgScanCredit by passing a value other than -1
for the flush threshold. Shortly we're going to make this always flush
scan work to gcController.scanWork and optionally also flush scan work
to gcController.bgScanCredit. This will be much easier if the flush
threshold is simply a constant (which it is in practice) and callers
merely control whether or not the flush includes the background
credit. Hence, replace the flush threshold argument with a flag.
Change-Id: Ia27db17de8a3f1e462a5d7137d4b5dc72f99a04e
Reviewed-on: https://go-review.googlesource.com/15406
Reviewed-by: Rick Hudson <rlh@golang.org>
These functions were nearly identical. Consolidate them by adding a
flags argument. In addition to cleaning up this code, this makes
further changes that affect both functions easier.
Change-Id: I6ec5c947603bbbd3ff4040113b2fbc240e99745f
Reviewed-on: https://go-review.googlesource.com/15405
Reviewed-by: Rick Hudson <rlh@golang.org>
The comment for assistRatio claimed it to be the reciprocal of what it
actually is.
Change-Id: If7f9bb853d75d0097facff3aa6704b224d9108b8
Reviewed-on: https://go-review.googlesource.com/15402
Reviewed-by: Russ Cox <rsc@golang.org>
(*T)(unsafe.Pointer(&t)) === &t
for t of type T
Change-Id: I43c1aa436747dfa0bf4cb0d615da1647633f9536
Reviewed-on: https://go-review.googlesource.com/15656
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The case fixed by this change happens when, in func (pr partReader)
Read, the Peek happens to read so that peek looks like:
"somedata\r\n--Boundary\r"
peekBufferSeparatorIndex was returning (-1, false) because it didn't
find the trailing '\n'.
This was wrong because:
1) It didn't match the documentation: as "\r\n--Boundary" was found, it
should return the index of that pattern, not -1.
2) It lead to an nCopy cut such as:
"somedata\r| |\n--Boundary\r" instead of "somedata| |\r\n--Boundary\r"
which made the subsequent Read miss the boundary, and eventually end
with a "return 0, io.ErrUnexpectedEOF" case, as reported in:
https://github.com/camlistore/camlistore/issues/642
Change-Id: I1ba78a741bc0c7719e160add9cca932d10f8a615
Reviewed-on: https://go-review.googlesource.com/15269
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
obj.ProgInfo is a field inside obj.Prog, which is currently 320 bytes
on 64bit platforms. By moving the Flags field below the other fields
the size of obj.Prog drops into the 288 byte size class, a saving of
32 bytes per value allocated on the heap.
Change-Id: If8bb12f45328996d7df1d0bac9d1c019d2af73bd
Reviewed-on: https://go-review.googlesource.com/15522
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change-Id: Ic19c94fe0af55e17f6c2fcfd36085ccb1584da6f
Reviewed-on: https://go-review.googlesource.com/15608
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Go shared libraries do not support dlclose, and there is no likelihood
that they will suppose dlclose in the future. Set the DF_1_NODELETE
flag to tell the dynamic linker to not attempt to remove them from
memory. This makes the shared library act as though every call to
dlopen passed the RTLD_NODELETE flag.
Fixes#12582.
Update #11100.
Update #12873.
Change-Id: Id4b6e90a1b54e2e6fc8355b5fb22c5978fc762b4
Reviewed-on: https://go-review.googlesource.com/15605
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Improve the aeshash implementation to make it harder to engineer collisions.
1) Scramble the seed before xoring with the input string. This
makes it harder to cancel known portions of the seed (like the size)
because it mixes the per-table seed into those other parts.
2) Use table-dependent seeds for all stripes when hashing >16 byte strings.
For small strings this change uses 4 aesenc ops instead of 3, so it
is somewhat slower. The first two can run in parallel, though, so
it isn't 33% slower.
benchmark old ns/op new ns/op delta
BenchmarkHash64-12 10.2 11.2 +9.80%
BenchmarkHash16-12 5.71 6.13 +7.36%
BenchmarkHash5-12 6.64 7.01 +5.57%
BenchmarkHashBytesSpeed-12 30.3 31.9 +5.28%
BenchmarkHash65536-12 2785 2882 +3.48%
BenchmarkHash1024-12 53.6 55.4 +3.36%
BenchmarkHashStringArraySpeed-12 54.9 56.5 +2.91%
BenchmarkHashStringSpeed-12 18.7 19.2 +2.67%
BenchmarkHashInt32Speed-12 14.8 15.1 +2.03%
BenchmarkHashInt64Speed-12 14.5 14.5 +0.00%
Change-Id: I59ea124b5cb92b1c7e8584008257347f9049996c
Reviewed-on: https://go-review.googlesource.com/14124
Reviewed-by: jcd . <jcd@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Following the C to Go translation, some useless variables
were left in the code. In fmt.go, this was harmless.
In lex.go, it broke the error message related to
non-canonical import paths.
Fix it, and remove the useless variables.
The added test case is ignored in the go/types tests, since
the behavior of the non-canonical import path check seems
to be different.
Fixes#11362
Change-Id: Ic9129139ede90357dc79ebf167af638cf44536fa
Reviewed-on: https://go-review.googlesource.com/15580
Reviewed-by: Marvin Stenger <marvin.stenger94@gmail.com>
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It's particularly nice to get rid of the android special cases in the linker.
Change-Id: I516363af7ce8a6b2f196fe49cb8887ac787a6dad
Reviewed-on: https://go-review.googlesource.com/14197
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Copy math package CL 12230 to cmplx package.
Change-Id: I3345b782b84b5b98e2b6a60d8774c7e7cede2891
Reviewed-on: https://go-review.googlesource.com/15500
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The self tests do not need to build the binary; they won't read it. The
self tests should work on any ELF system.
Use t.Skip instead of panic. Use internal/testenv. Don't worry about a
space in the temporary directory name.
Change-Id: I66ef0af90520d330820afa7b6c6b3a132ab27454
Reviewed-on: https://go-review.googlesource.com/15495
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The <importPath>/_test directory is not actually created in -n mode, so
`go test` fails to write _testmain.go.
Do not write _testmain.go if -n is passed.
Change-Id: I825d5040cacbc9d9a8c89443e5a3f83e6f210ce4
Reviewed-on: https://go-review.googlesource.com/15433
Reviewed-by: Andrew Gerrand <adg@golang.org>
In particular, don't assume that one reflect.Value can be assigned to another just because they have the same reflect.Kind.
Fixes#12401
Change-Id: Ia4605a5c46557ff8f8f1d44f26d492850666c6d1
Reviewed-on: https://go-review.googlesource.com/15420
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In CL 14836, the implementation of duffcopy on amd64
was changed to replace the use of the MOVQ instructions
by MOVUPS.
However, it broke the build on plan9/amd64, since
Plan 9 doesn't allow floating point in note handler.
This change disables the use of duffcopy on Plan 9.
Fixes#12829.
Change-Id: Ifd5b17b17977a1b631b16c3dfe2dc7ab4ad00507
Reviewed-on: https://go-review.googlesource.com/15421
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Motivation:
* The logic to verify the numEntries can overflow and incorrectly
pass, allowing a malicious file to allocate arbitrary memory.
* The use of strconv.ParseInt does not set the integer precision
to 64bit, causing this code to work incorrectly on 32bit machines.
Change-Id: I1b1571a750a84f2dde97cc329ed04fe2342aaa60
Reviewed-on: https://go-review.googlesource.com/15173
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
A recursive call to Reader.Next did not check the error before
trying to use the result, leading to a nil pointer panic.
This specific CL addresses the immediate issue, which is the panic,
but does not solve the root issue, which is due to an integer
overflow in the base-256 parser.
Updates #12435
Change-Id: Ia908671f0f411a409a35e24f2ebf740d46734072
Reviewed-on: https://go-review.googlesource.com/15437
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It is generally expected that the ELF flags of a dynamically linked executable
and the libraries it links against match. Go's linker currently always produces
executables with flags that do not declare a float abi (hard, soft) at all, but
when cgo is involved it is unlikely that this matches the system libraries
being linked against -- really the decision about ABI is made by the C compiler
during the invocation of cgo.
This change is basically a port of the code from binutils that parses the
".ARM.attributes" section to check for the tag that declares that the code is
built for the hard-float ABI.
Fixes#7094
Change-Id: I737c8f3b5ed4af545cfc3e86722d03eb83083402
Reviewed-on: https://go-review.googlesource.com/14860
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Change-Id: Id6cb5e3d40e8a2ded6359aa7fcdc012861cc3994
Reviewed-on: https://go-review.googlesource.com/14545
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Motivation:
* There are an increasing number of "one-off" corrupt files added
to make sure that package does not succeed or crash on them.
Instead, allow for the test to specify the error that is expected
to occur (if any).
* Also, fold in the logic to check the MD5 checksum into this
function.
The following tests are being removed:
* TestIncrementalRead: Done by TestReader by using io.CopyBuffer
with a buffer of 8. This achieves the same behavior as this test.
* TestSparseEndToEnd: Since TestReader checks the MD5 checksums
if the input corpus provides them, then this is redundant.
* TestSparseIncrementalRead: Redundant for the same reasons that
TestIncrementalRead is now redundant
* TestNegativeHdrSize: Added to TestReader corpus
* TestIssue10968: Added to TestReader corpus
* TestIssue11169: Added to TestReader corpus
With this change, code coverage did not change: 85.3%
Change-Id: I8550d48657d4dbb8f47dfc3dc280758ef73b47ec
Reviewed-on: https://go-review.googlesource.com/15176
Reviewed-by: Andrew Gerrand <adg@golang.org>
This lets us re-enable duffzero.
Fixes#12108
Change-Id: Iefd24d26eaa56067caa2c29ff99cd20a42d8714a
Reviewed-on: https://go-review.googlesource.com/14937
Reviewed-by: Keith Randall <khr@golang.org>
Let C be whether c valid
Let E be whether err is non-nil
The old comment explicitly says that (~C → E). However, that does call
into question whether (E → ~C), which causes doubts for users.
Without a comment at all, it is obvious that only (E ↔ ~C) makes sense.
Fixes#11308
Change-Id: I5a7d51ceb509057eccca91f57a7e48c9d1c6d112
Reviewed-on: https://go-review.googlesource.com/15256
Reviewed-by: Andrew Gerrand <adg@golang.org>
The later part of the docstring simply talks about "offset" but does
not disambiguate what it is relative to. For both the return value
and valid offsets to seek to, it only makes sense in the context of
"offset relative to origin of file".
Fixes#11877
Change-Id: Ic238a407cf8e8fdd64991d98a6584cdc8a51cd6b
Reviewed-on: https://go-review.googlesource.com/15257
Reviewed-by: Andrew Gerrand <adg@golang.org>
Usage of all commands is printed to stderr, except go test, which is printed to
stdout. This is inconsistent.
Print `go test -help` to stderr instead.
R=rsc@golang.org
Change-Id: I079f4788134bf9aedcccc26838879eedad1c925e
Reviewed-on: https://go-review.googlesource.com/15434
Reviewed-by: Andrew Gerrand <adg@golang.org>
The net package already has support for limited uses of the strconv
package. Despite this, a few uses of strconv have crept in over time.
Remove them and use the existing net support instead.
Change-Id: Icdb4bdaa8e1197f1119a96cddcf548ed4a551b74
Reviewed-on: https://go-review.googlesource.com/15400
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
cgo panics in Package.rewriteRef for:
var a = C.enum_test(1)
or
p := new(C.enum_test)
when the corresponding enum type is not defined.
Check nil values for Type fields and issue a proper
error instead.
Fixes#11097
Updates #12160
Change-Id: I5821d29097ef0a36076ec5273125b09846c7d832
Reviewed-on: https://go-review.googlesource.com/15264
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The existing serve() method returns a zero-length response body when
it encounters an error, which results in a blank page and no visible
error in browsers.
This change sends a response body explaining the error for display in browsers.
Fixes#12745
Change-Id: I9dc3b95ad88cb92c18ced51f6b52bd3b2c1b974c
Reviewed-on: https://go-review.googlesource.com/15018
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The current heap sampling introduces some bias that interferes
with unsampling, producing unexpected heap profiles.
The solution is to use a Poisson process to generate the
sampling points, using the formulas described at
https://en.wikipedia.org/wiki/Poisson_process
This fixes#12620
Change-Id: If2400809ed3c41de504dd6cff06be14e476ff96c
Reviewed-on: https://go-review.googlesource.com/14590
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
ppc64 codegen assumes that it is OK to stomp on r31 at any time, but it is not
excluded from the set of registers that regopt is allowed to use.
Fixes#12597
Change-Id: I29c7655e32abd22f3c21d88427b73e4fca055233
Reviewed-on: https://go-review.googlesource.com/15245
Reviewed-by: Minux Ma <minux@golang.org>
Turns out the summary information for the ... args was
already correctly computed, all that lacked was to make
use of it and correct tests that documented our prior
deficiencies.
Fixes#12006
Change-Id: Ie8adfab7547f179391d470679598f0904aabf9f7
Reviewed-on: https://go-review.googlesource.com/15200
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
The runtime/zgoos_$GOOS.go and runtime/zgoarch_$GOARCH.go files
are in the repository now, so the message is actually incorrect
(running make.bash won't generate those). The reason is probably
wrong $GOROOT.
Change-Id: I8dc125594c52d666eca91fd5af48b60d12d599b8
Reviewed-on: https://go-review.googlesource.com/15221
Reviewed-by: Ian Lance Taylor <iant@golang.org>
