This trivial change is a prerequisite to
converting Val.U to an interface{}.
No functional changes. Passes toolstash -cmp.
Change-Id: I17ff036f68d29a9ed0097a8b23ae1c91e6ce8c21
Reviewed-on: https://go-review.googlesource.com/10058
Reviewed-by: Russ Cox <rsc@golang.org>
Remove all uses of Node.Val outside of the gc package.
A subsequent, automated commit in the Go 1.6 cycle
will unexport Node.Val.
No functional changes. Passes toolstash -cmp.
Change-Id: Ia92ae6a7766c83ab3e45c69edab24a9581c824f9
Reviewed-on: https://go-review.googlesource.com/9267
Reviewed-by: Russ Cox <rsc@golang.org>
Remove all uses of Mp* outside of the gc package.
A subsequent, automated commit in the Go 1.6
cycle will unexport all Mp* functions and types.
No functional changes. Passes toolstash -cmp.
Change-Id: Ie1604cb5b84ffb30b47f4777d4235570f2c62709
Reviewed-on: https://go-review.googlesource.com/9263
Reviewed-by: Russ Cox <rsc@golang.org>
Preallocating them in reflect means that
(1) if you say _ = PtrTo(ArrayOf(1000000000, reflect.TypeOf(byte(0)))), you just allocated 1GB of data
(2) if you say it again, that's *another* GB of data.
The only use of t.zero in the runtime is for map elements.
Delay the allocation until the creation of a map with that element type,
and share the zeros.
The one downside of the shared zero is that it's not garbage collected,
but it's also never written, so the OS should be able to handle it fairly
efficiently.
Change-Id: I56b098a091abf3ac0945de28ebef9a6c08e76614
Reviewed-on: https://go-review.googlesource.com/10111
Reviewed-by: Keith Randall <khr@golang.org>
According to MSDN RegQueryValueEx page:
If the data has the REG_SZ, REG_MULTI_SZ or REG_EXPAND_SZ type, the
string may not have been stored with the proper terminating null
characters. Therefore, even if the function returns ERROR_SUCCESS, the
application should ensure that the string is properly terminated before
using it; otherwise, it may overwrite a buffer. (Note that REG_MULTI_SZ
strings should have two terminating null characters.)
Test written by Alex Brainman <alex.brainman@gmail.com>
Change-Id: I8c0852e0527e27ceed949134ed5e6de944189986
Reviewed-on: https://go-review.googlesource.com/9806
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
See CL 9962 for the rationale.
Change-Id: I73c714fce258430eea1e61d3835f5c8e9014ca1f
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/9925
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Auto-generated using the following bash script:
for i in z*_*_*.go; do
goosgoarch=`basename ${i/${i/_*/}_/} .go`
goos=${goosgoarch/_*/}
goarch=${goosgoarch/*_/}
echo $i $goos $goarch
[ "$goos" = "windows" ] && continue
sed -i -e "/^package /i\/\/ +build $goarch,$goos\n" "$i"
done
Change-Id: I756fee551d1698080e4591fed8f058ae0450aaa5
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/10113
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Missed a case; just need to call validateType.
Fixes#10800.
Change-Id: I81997ca7a9feb1be31c8b47e631b32712d7ffb86
Reviewed-on: https://go-review.googlesource.com/10031
Reviewed-by: Andrew Gerrand <adg@golang.org>
This reduces the depth of the inlining at a particular call site.
The inliner introduces many temporary variables, and the compiler can do
a better job with fewer. Being verbose in the bodies of these helper functions
seems like a reasonable tradeoff: the uses are still just as readable, and
they run faster in some important cases.
Change-Id: I5323976ed3704d0acd18fb31176cfbf5ba23a89c
Reviewed-on: https://go-review.googlesource.com/9883
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
BenchmarkSkipValue was sensitive to the value of
b.N due to its significant startup cost.
Two adjacent runs before this CL:
BenchmarkSkipValue 50 21047499 ns/op 93.37 MB/s
BenchmarkSkipValue 100 17260554 ns/op 118.05 MB/s
After this CL, using benchtime to recreate the
difference in b.N:
BenchmarkSkipValue 50 15204797 ns/op 131.67 MB/s
BenchmarkSkipValue 100 15332319 ns/op 130.58 MB/s
Change-Id: Iac86f86dd774d535302fa5e4c08f89f8da00be9e
Reviewed-on: https://go-review.googlesource.com/10053
Reviewed-by: Andrew Gerrand <adg@golang.org>
The Transport's writer to the remote server is wrapped in a
bufio.Writer to suppress many small writes while writing headers and
trailers. However, when writing the request body, the buffering may get
in the way if the request body is arriving slowly.
Because the io.Copy from the Request.Body to the writer is already
buffered, the outer bufio.Writer is unnecessary and prevents small
Request.Body.Reads from going to the server right away. (and the
io.Reader contract does say to return when you've got something,
instead of blocking waiting for more). After the body is finished, the
Transport's bufio.Writer is still used for any trailers following.
A previous attempted fix for this made the chunk writer always flush
if the underlying type was a bufio.Writer, but that is not quite
correct. This CL instead makes it opt-in by using a private sentinel
type (wrapping a *bufio.Writer) to the chunk writer that requests
Flushes after each chunk body (the chunk header & chunk body are still
buffered together into one write).
Fixes#6574
Change-Id: Icefcdf17130c9e285c80b69af295bfd3e72c3a70
Reviewed-on: https://go-review.googlesource.com/10021
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I was previously setting GOARM=arm5 (due to confusion with previously
seeing buildall.sh's temporary of "arm5" as a GOARCH and
misremembernig), but GOARM=arm5 was acting like GOARM=5 only on
accident. See https://go-review.googlesource.com/#/c/10023/
Instead, fail if GOARM is not a known value.
Change-Id: I9ba4fd7268df233d40b09f0431f37cd85a049847
Reviewed-on: https://go-review.googlesource.com/10024
Reviewed-by: Minux Ma <minux@golang.org>
Was otherwise absent unless bound to an exported symbol,
as in the BUG with strings.Title.
Fixes#10781.
Change-Id: I1543137073a9dee9e546bc9d648ca54fc9632dde
Reviewed-on: https://go-review.googlesource.com/9899
Reviewed-by: Russ Cox <rsc@golang.org>
Set overflowing integer constants to 1 rather than 0 to avoid
spurious div-zero errors in subsequent constant expressions.
Also: Exclude new test case from go/types test since it's
running too long (go/types doesn't have an upper constant
size limit at the moment).
Fixes#7746.
Change-Id: I3768488ad9909a3cf995247b81ee78a8eb5a1e41
Reviewed-on: https://go-review.googlesource.com/9165
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
One important use case is a pipeline computation that pass values
from one Goroutine to the next and then exits or is placed in a
wait state. If GOMAXPROCS > 1 a Goroutine running on P1 will enable
another Goroutine and then immediately make P1 available to execute
it. We need to prevent other Ps from stealing the G that P1 is about
to execute. Otherwise the Gs can thrash between Ps causing unneeded
synchronization and slowing down throughput.
Fix this by changing the stealing logic so that when a P attempts to
steal the only G on some other P's run queue, it will pause
momentarily to allow the victim P to schedule the G.
As part of optimizing stealing we also use a per P victim queue
move stolen gs. This eliminates the zeroing of a stack local victim
queue which turned out to be expensive.
This CL is a necessary but not sufficient prerequisite to changing
the default value of GOMAXPROCS to something > 1 which is another
CL/discussion.
For highly serialized programs, such as GoroutineRing below this can
make a large difference. For larger and more parallel programs such
as the x/benchmarks there is no noticeable detriment.
~/work/code/src/rsc.io/benchstat/benchstat old.txt new.txt
name old mean new mean delta
GoroutineRing 30.2µs × (0.98,1.01) 30.1µs × (0.97,1.04) ~ (p=0.941)
GoroutineRing-2 113µs × (0.91,1.07) 30µs × (0.98,1.03) -73.17% (p=0.004)
GoroutineRing-4 144µs × (0.98,1.02) 32µs × (0.98,1.01) -77.69% (p=0.000)
GoroutineRingBuf 32.7µs × (0.97,1.03) 32.5µs × (0.97,1.02) ~ (p=0.795)
GoroutineRingBuf-2 120µs × (0.92,1.08) 33µs × (1.00,1.00) -72.48% (p=0.004)
GoroutineRingBuf-4 138µs × (0.92,1.06) 33µs × (1.00,1.00) -76.21% (p=0.003)
The bench benchmarks show little impact.
old new
garbage 7032879 7011696
httpold 25509 25301
splayold 1022073 1019499
jsonold 28230624 28081433
Change-Id: I228c48fed8d85c9bbef16a7edc53ab7898506f50
Reviewed-on: https://go-review.googlesource.com/9872
Reviewed-by: Austin Clements <austin@google.com>
FileConn and FilePacketConn APIs accept user-configured socket
descriptors to make them work together with runtime-integrated network
poller, but there's a limitation. The APIs reject protocol sockets that
are not supported by standard library. It's very hard for the net,
syscall packages to look after all platform, feature-specific sockets.
This change allows various platform, feature-specific socket descriptors
to use runtime-integrated network poller by using SocketConn,
SocketPacketConn APIs that bridge between the net, syscall packages and
platforms.
New exposed APIs:
pkg net, func SocketConn(*os.File, SocketAddr) (Conn, error)
pkg net, func SocketPacketConn(*os.File, SocketAddr) (PacketConn, error)
pkg net, type SocketAddr interface { Addr, Raw }
pkg net, type SocketAddr interface, Addr([]uint8) Addr
pkg net, type SocketAddr interface, Raw(Addr) []uint8
Fixes#10565.
Change-Id: Iec57499b3d84bb5cb0bcf3f664330c535eec11e3
Reviewed-on: https://go-review.googlesource.com/9275
Reviewed-by: Ian Lance Taylor <iant@golang.org>
If __LP64__ is defined then the type "long" is 64-bits, and there is
no need to explicitly request _FILE_OFFSET_BITS == 64. This changes
the definitions of F_GETLK, F_SETLK, and F_SETLKW on PPC to the values
that the kernel requires. The values used in C when _FILE_OFFSET_BITS
== 64 are corrected by the glibc fcntl function before making the
system call.
With this change, regenerate ppc64le files on Ubuntu trusty.
Change-Id: I8dddbd8a6bae877efff818f5c5dd06291ade3238
Reviewed-on: https://go-review.googlesource.com/9962
Reviewed-by: Minux Ma <minux@golang.org>
This will skip system call numbers that are ifdef'ed out in unistd.h,
as occurs on PPC.
Change-Id: I88e640e4621c7a8cc266433f34a7b4be71543ec9
Reviewed-on: https://go-review.googlesource.com/9966
Reviewed-by: Minux Ma <minux@golang.org>
Running these tests on the system stack is problematic because they
allocate Ps, which are large enough to overflow the system stack if
they are stack-allocated. It used to be necessary to run these tests
on the system stack because they were written in C, but since this is
no longer the case, we can fix this problem by simply not running the
tests on the system stack.
This also means we no longer need the hack in one of these tests that
forces the allocated Ps to escape to the heap, so eliminate that as
well.
Change-Id: I9064f5f8fd7f7b446ff39a22a70b172cfcb2dc57
Reviewed-on: https://go-review.googlesource.com/9923
Reviewed-by: Rick Hudson <rlh@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
The line in cgen.go was lost during the ginscmp CL.
The ggen.go change is not strictly necessary, but
it makes the 5g -S output for x[0] match what it said
before the ginscmp CL.
Change-Id: I5890a9ec1ac69a38509416eda5aea13b8b12b94a
Reviewed-on: https://go-review.googlesource.com/9929
Reviewed-by: Russ Cox <rsc@golang.org>
Fix bug on Linux SysProcAttr handling: setting both Pdeathsig and
Credential caused Pdeathsig to be ignored. This is because the kernel
clears the deathsignal field when performing a setuid/setgid
system call.
Avoid this by moving Pdeathsig handling after Credential handling.
Fixes#9686
Change-Id: Id01896ad4e979b8c448e0061f00aa8762ca0ac94
Reviewed-on: https://go-review.googlesource.com/3290
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The code generated for x = append(x, v) is roughly:
t := x
if len(t)+1 > cap(t) {
t = grow(t)
}
t[len(t)] = v
len(t)++
x = t
We used to generate this code as Go pseudocode during walk.
Generate it instead as actual instructions during gen.
Doing so lets us apply a few optimizations. The most important
is that when, as in the above example, the source slice and the
destination slice are the same, the code can instead do:
t := x
if len(t)+1 > cap(t) {
t = grow(t)
x = {base(t), len(t)+1, cap(t)}
} else {
len(x)++
}
t[len(t)] = v
That is, in the fast path that does not reallocate the array,
only the updated length needs to be written back to x,
not the array pointer and not the capacity. This is more like
what you'd write by hand in C. It's faster in general, since
the fast path elides two of the three stores, but it's especially
faster when the form of x is such that the base pointer write
would turn into a write barrier. No write, no barrier.
name old mean new mean delta
BinaryTree17 5.68s × (0.97,1.04) 5.81s × (0.98,1.03) +2.35% (p=0.023)
Fannkuch11 4.41s × (0.98,1.03) 4.35s × (1.00,1.00) ~ (p=0.090)
FmtFprintfEmpty 92.7ns × (0.91,1.16) 86.0ns × (0.94,1.11) -7.31% (p=0.038)
FmtFprintfString 281ns × (0.96,1.08) 276ns × (0.98,1.04) ~ (p=0.219)
FmtFprintfInt 288ns × (0.97,1.06) 274ns × (0.98,1.06) -4.94% (p=0.002)
FmtFprintfIntInt 493ns × (0.97,1.04) 506ns × (0.99,1.01) +2.65% (p=0.009)
FmtFprintfPrefixedInt 423ns × (0.97,1.04) 391ns × (0.99,1.01) -7.52% (p=0.000)
FmtFprintfFloat 598ns × (0.99,1.01) 566ns × (0.99,1.01) -5.27% (p=0.000)
FmtManyArgs 1.89µs × (0.98,1.05) 1.91µs × (0.99,1.01) ~ (p=0.231)
GobDecode 14.8ms × (0.98,1.03) 15.3ms × (0.99,1.02) +3.01% (p=0.000)
GobEncode 12.3ms × (0.98,1.01) 11.5ms × (0.97,1.03) -5.93% (p=0.000)
Gzip 656ms × (0.99,1.05) 645ms × (0.99,1.01) ~ (p=0.055)
Gunzip 142ms × (1.00,1.00) 142ms × (1.00,1.00) -0.32% (p=0.034)
HTTPClientServer 91.2µs × (0.97,1.04) 90.5µs × (0.97,1.04) ~ (p=0.468)
JSONEncode 32.6ms × (0.97,1.08) 32.0ms × (0.98,1.03) ~ (p=0.190)
JSONDecode 114ms × (0.97,1.05) 114ms × (0.99,1.01) ~ (p=0.887)
Mandelbrot200 6.11ms × (0.98,1.04) 6.04ms × (1.00,1.01) ~ (p=0.167)
GoParse 6.66ms × (0.97,1.04) 6.47ms × (0.97,1.05) -2.81% (p=0.014)
RegexpMatchEasy0_32 159ns × (0.99,1.00) 171ns × (0.93,1.07) +7.19% (p=0.002)
RegexpMatchEasy0_1K 538ns × (1.00,1.01) 550ns × (0.98,1.01) +2.30% (p=0.000)
RegexpMatchEasy1_32 138ns × (1.00,1.00) 135ns × (0.99,1.02) -1.60% (p=0.000)
RegexpMatchEasy1_1K 869ns × (0.99,1.01) 879ns × (1.00,1.01) +1.08% (p=0.000)
RegexpMatchMedium_32 252ns × (0.99,1.01) 243ns × (1.00,1.00) -3.71% (p=0.000)
RegexpMatchMedium_1K 72.7µs × (1.00,1.00) 70.3µs × (1.00,1.00) -3.34% (p=0.000)
RegexpMatchHard_32 3.85µs × (1.00,1.00) 3.82µs × (1.00,1.01) -0.81% (p=0.000)
RegexpMatchHard_1K 118µs × (1.00,1.00) 117µs × (1.00,1.00) -0.56% (p=0.000)
Revcomp 920ms × (0.97,1.07) 917ms × (0.97,1.04) ~ (p=0.808)
Template 129ms × (0.98,1.03) 114ms × (0.99,1.01) -12.06% (p=0.000)
TimeParse 619ns × (0.99,1.01) 622ns × (0.99,1.01) ~ (p=0.062)
TimeFormat 661ns × (0.98,1.04) 665ns × (0.99,1.01) ~ (p=0.524)
See next CL for combination with a similar optimization for slice.
The benchmarks that are slower in this CL are still faster overall
with the combination of the two.
Change-Id: I2a7421658091b2488c64741b4db15ab6c3b4cb7e
Reviewed-on: https://go-review.googlesource.com/9812
Reviewed-by: David Chase <drchase@google.com>
This lets us abstract away which arguments can be constants and so on
and lets the back ends reverse the order of arguments if that helps.
Change-Id: I283ec1d694f2dd84eba22e5eb4aad78a2d2d9eb0
Reviewed-on: https://go-review.googlesource.com/9810
Reviewed-by: David Chase <drchase@google.com>
Messages that are too big are rejected when read, so they should
be rejected when written too.
Fixes#10518.
Change-Id: I96678fbe2d94f51b957fe26faef33cd8df3823dd
Reviewed-on: https://go-review.googlesource.com/9965
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The status buffer built by the exit function
was not nil-terminated.
Fixes#10789.
Change-Id: I2d34ac50a19d138176c4b47393497ba7070d5b61
Reviewed-on: https://go-review.googlesource.com/9953
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Once added to the signal queue, the pointer passed to the
signal handler could no longer be valid. Instead of passing
the pointer to the note string, we recopy the value of the
note string to a static array in the signal queue.
Fixes#10784.
Change-Id: Iddd6837b58a14dfaa16b069308ae28a7b8e0965b
Reviewed-on: https://go-review.googlesource.com/9950
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Today's earlier fix can stay, but it's a band-aid over the real problem,
which is that bad code was slipping through the type checker
into the back end (and luckily causing a type error there).
I discovered this because my new append does not use the same
temporaries and failed the test as written.
Fixes#9521.
Change-Id: I7e33e2ea15743406e15c6f3fdf73e1edecda69bd
Reviewed-on: https://go-review.googlesource.com/9921
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Any Git branch can be the default branch not only master. Removing
hardwired 'checkout master', and using 'checkout {tag}' is the best
choice. It works with and without a master branch. Furthermore it
resolves the Github default branch issue. Changing Github default
branch is effectively changing HEAD.
Fixes#9032
Change-Id: I19a1221bcefe0806e7556c124c6da7ac0c2160b5
Reviewed-on: https://go-review.googlesource.com/5312
Reviewed-by: Russ Cox <rsc@golang.org>
registry.ReadSubKeyNames requires QUERY access right in addition to
ENUMERATE_SUB_KEYS.
This was making TestLocalZoneAbbr fail on Windows 7 in Paris/Madrid
timezone. It succeeded on Windows 8 because timezone name changed from
"Paris/Madrid" to "Romance Standard Time", the latter being matched by
an abbrs entry.
Change-Id: I791287ba9d1b3556246fa4e9e1604a1fbba1f5e6
Reviewed-on: https://go-review.googlesource.com/9809
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TestAcceptIgnoreSomeErrors was created to test that network
accept function ignores some errors. But conditions created
by the test also affects network reads. Change the test to
ignore these read errors when acceptable.
Fixes#10785
Change-Id: I3da85cb55bd3e78c1980ad949e53e82391f9b41e
Reviewed-on: https://go-review.googlesource.com/9942
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This:
1) Defines the ABI hash of a package (as the SHA1 of the __.PKGDEF)
2) Defines the ABI hash of a shared library (sort the packages by import
path, concatenate the hashes of the packages and SHA1 that)
3) When building a shared library, compute the above value and define a
global symbol that points to a go string that has the hash as its value.
4) When linking against a shared library, read the abi hash from the
library and put both the value seen at link time and a reference
to the global symbol into the moduledata.
5) During runtime initialization, check that the hash seen at link time
still matches the hash the global symbol points to.
Change-Id: Iaa54c783790e6dde3057a2feadc35473d49614a5
Reviewed-on: https://go-review.googlesource.com/8773
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
addmoduledata is called from a .init_array function and need to follow the
platform ABI. It contains accesses to global data which are rewritten to use
R15 by the assembler, and as R15 is callee-save we need to save it.
Change-Id: I03893efb1576aed4f102f2465421f256f3bb0f30
Reviewed-on: https://go-review.googlesource.com/9941
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I89778988baec1cf4a35d9342c7dbe8c4c08ff3cd
Reviewed-on: https://go-review.googlesource.com/9893
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Instead of errors like:
./blank2.go:15: cannot use ~b1 (type []int) as type int in assignment
we now have:
./blank2.go:15: cannot use _ (type []int) as type int in assignment
Less confusing for users.
Fixes#9521
Change-Id: Ieab9859040e8e0df95deeaee7eeb408d3be61c0f
Reviewed-on: https://go-review.googlesource.com/9902
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
DWARF generation appears to assume Cpos is cheap and this makes linking godoc
about 8% faster and linking the standard library into a single shared library
about 22% faster on my machine.
Updates #10571
Change-Id: I3f81efd0174e356716e7971c4f59810b72378177
Reviewed-on: https://go-review.googlesource.com/9913
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When running the client header timeout test, there is a race between
us timing out and waiting on the remaining requests to be serviced. If
the client times out before the server blocks on the channel in the
handler, we will be simultaneously adding to a waitgroup with the
value 0 and waiting on it when we call TestServer.Close().
This is largely a theoretical race. We have to time out before we
enter the handler and the only reason we would time out if we're
blocked on the channel. Nevertheless, make the race detector happy
by turning the close into a channel send. This turns the defer call
into a synchronization point and we can be sure that we've entered
the handler before we close the server.
Fixes#10780
Change-Id: Id73b017d1eb7503e446aa51538712ef49f2f5c9e
Reviewed-on: https://go-review.googlesource.com/9905
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The current implementation of typedmemmove walks the ptrmask
in the type to find out where pointers are. This led to turning off
GC programs for the Go 1.5 dev cycle, so that there would always
be a ptrmask. Instead of also interpreting the GC programs,
interpret the heap bitmap, which we know must be available and
up to date. (There is no point to write barriers when writing outside
the heap.)
This CL is only about correctness. The next CL will optimize the code.
Change-Id: Id1305c7c071fd2734ab96634b0e1c745b23fa793
Reviewed-on: https://go-review.googlesource.com/9886
Reviewed-by: Austin Clements <austin@google.com>
We want typedmemmove to use the heap bitmap to determine
where pointers are, instead of reinterpreting the type information.
The heap bitmap is simpler to access.
In general, typedmemmove will need to be able to look up the bits
for any word and find valid pointer information, so fill even after the
dead marker. Not filling after the dead marker was an optimization
I introduced only a few days ago, when reintroducing the dead marker
code. At the time I said it probably wouldn't last, and it didn't.
Change-Id: I6ba01bff17ddee1ff429f454abe29867ec60606e
Reviewed-on: https://go-review.googlesource.com/9885
Reviewed-by: Austin Clements <austin@google.com>
Moving them up makes them properly aligned on 32-bit systems.
There are some odd fields above them right now
(like fixalloc and mutex maybe).
Change-Id: I57851a5bbb2e7cc339712f004f99bb6c0cce0ca5
Reviewed-on: https://go-review.googlesource.com/9889
Reviewed-by: Austin Clements <austin@google.com>
Dead code.
This field is left over from Go 1.4, when we elided the fake write
barrier in this case. Today, it's unused (always false).
The upcoming append/slice changes handle this case again,
but without needing this field.
Change-Id: Ic6f160b64efdc1bbed02097ee03050f8cd0ab1b8
Reviewed-on: https://go-review.googlesource.com/9789
Reviewed-by: David Chase <drchase@google.com>
If you are using -h to get a stack trace at the site of the failure,
Yyerror will never return. Dump the register allocation sites
before calling Yyerror.
Change-Id: I51266c03e06cb5084c2eaa89b367b9ed85ba286a
Reviewed-on: https://go-review.googlesource.com/9788
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Dave Cheney <dave@cheney.net>
The new(uint64) was moving to the stack, which may not be aligned.
Change-Id: Iad070964202001b52029494d43e299fed980f939
Reviewed-on: https://go-review.googlesource.com/9787
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David Chase <drchase@google.com>
The -g mode is a debugging mode that prints instructions
as they are constructed. Gbranch was just missing the print.
Change-Id: I3fb45fd9bd3996ed96df5be903b9fd6bd97148b0
Reviewed-on: https://go-review.googlesource.com/9827
Reviewed-by: Rick Hudson <rlh@golang.org>
Reintroduce an optimization discarded during the initial conversion
from 4-bit heap bitmaps to 2-bit heap bitmaps: when we reach the
place in the bitmap where there are no more pointers, mark that position
for the GC so that it can avoid scanning past that place.
During heapBitsSetType we can also avoid initializing heap bitmap
beyond that location, which gives a bit of a win compared to Go 1.4.
This particular optimization (not initializing the heap bitmap) may not last:
we might change typedmemmove to use the heap bitmap, in which
case it would all need to be initialized. The early stop in the GC scan
will stay no matter what.
Compared to Go 1.4 (github.com/rsc/go, branch go14bench):
name old mean new mean delta
SetTypeNode64 80.7ns × (1.00,1.01) 57.4ns × (1.00,1.01) -28.83% (p=0.000)
SetTypeNode64Dead 80.5ns × (1.00,1.01) 13.1ns × (0.99,1.02) -83.77% (p=0.000)
SetTypeNode64Slice 2.16µs × (1.00,1.01) 1.54µs × (1.00,1.01) -28.75% (p=0.000)
SetTypeNode64DeadSlice 2.16µs × (1.00,1.01) 1.52µs × (1.00,1.00) -29.74% (p=0.000)
Compared to previous CL:
name old mean new mean delta
SetTypeNode64 56.7ns × (1.00,1.00) 57.4ns × (1.00,1.01) +1.19% (p=0.000)
SetTypeNode64Dead 57.2ns × (1.00,1.00) 13.1ns × (0.99,1.02) -77.15% (p=0.000)
SetTypeNode64Slice 1.56µs × (1.00,1.01) 1.54µs × (1.00,1.01) -0.89% (p=0.000)
SetTypeNode64DeadSlice 1.55µs × (1.00,1.01) 1.52µs × (1.00,1.00) -2.23% (p=0.000)
This is the last CL in the sequence converting from the 4-bit heap
to the 2-bit heap, with all the same optimizations reenabled.
Compared to before that process began (compared to CL 9701 patch set 1):
name old mean new mean delta
BinaryTree17 5.87s × (0.94,1.09) 5.91s × (0.96,1.06) ~ (p=0.578)
Fannkuch11 4.32s × (1.00,1.00) 4.32s × (1.00,1.00) ~ (p=0.474)
FmtFprintfEmpty 89.1ns × (0.95,1.16) 89.0ns × (0.93,1.10) ~ (p=0.942)
FmtFprintfString 283ns × (0.98,1.02) 298ns × (0.98,1.06) +5.33% (p=0.000)
FmtFprintfInt 284ns × (0.98,1.04) 286ns × (0.98,1.03) ~ (p=0.208)
FmtFprintfIntInt 486ns × (0.98,1.03) 498ns × (0.97,1.06) +2.48% (p=0.000)
FmtFprintfPrefixedInt 400ns × (0.99,1.02) 408ns × (0.98,1.02) +2.23% (p=0.000)
FmtFprintfFloat 566ns × (0.99,1.01) 587ns × (0.98,1.01) +3.69% (p=0.000)
FmtManyArgs 1.91µs × (0.99,1.02) 1.94µs × (0.99,1.02) +1.81% (p=0.000)
GobDecode 15.5ms × (0.98,1.05) 15.8ms × (0.98,1.03) +1.94% (p=0.002)
GobEncode 11.9ms × (0.97,1.03) 12.0ms × (0.96,1.09) ~ (p=0.263)
Gzip 648ms × (0.99,1.01) 648ms × (0.99,1.01) ~ (p=0.992)
Gunzip 143ms × (1.00,1.00) 143ms × (1.00,1.01) ~ (p=0.585)
HTTPClientServer 89.2µs × (0.99,1.02) 90.3µs × (0.98,1.01) +1.24% (p=0.000)
JSONEncode 32.3ms × (0.97,1.06) 31.6ms × (0.99,1.01) -2.29% (p=0.000)
JSONDecode 106ms × (0.99,1.01) 107ms × (1.00,1.01) +0.62% (p=0.000)
Mandelbrot200 6.02ms × (1.00,1.00) 6.03ms × (1.00,1.01) ~ (p=0.250)
GoParse 6.57ms × (0.97,1.06) 6.53ms × (0.99,1.03) ~ (p=0.243)
RegexpMatchEasy0_32 162ns × (1.00,1.00) 161ns × (1.00,1.01) -0.80% (p=0.000)
RegexpMatchEasy0_1K 561ns × (0.99,1.02) 541ns × (0.99,1.01) -3.67% (p=0.000)
RegexpMatchEasy1_32 145ns × (0.95,1.04) 138ns × (1.00,1.00) -5.04% (p=0.000)
RegexpMatchEasy1_1K 864ns × (0.99,1.04) 887ns × (0.99,1.01) +2.57% (p=0.000)
RegexpMatchMedium_32 255ns × (0.99,1.04) 253ns × (0.99,1.01) -1.05% (p=0.012)
RegexpMatchMedium_1K 73.9µs × (0.98,1.04) 72.8µs × (1.00,1.00) -1.51% (p=0.005)
RegexpMatchHard_32 3.92µs × (0.98,1.04) 3.85µs × (1.00,1.01) -1.88% (p=0.002)
RegexpMatchHard_1K 120µs × (0.98,1.04) 117µs × (1.00,1.01) -2.02% (p=0.001)
Revcomp 936ms × (0.95,1.08) 922ms × (0.97,1.08) ~ (p=0.234)
Template 130ms × (0.98,1.04) 126ms × (0.99,1.01) -2.99% (p=0.000)
TimeParse 638ns × (0.98,1.05) 628ns × (0.99,1.01) -1.54% (p=0.004)
TimeFormat 674ns × (0.99,1.01) 668ns × (0.99,1.01) -0.80% (p=0.001)
The slowdown of the first few benchmarks seems to be due to the new
atomic operations for certain small size allocations. But the larger
benchmarks mostly improve, probably due to the decreased memory
pressure from having half as much heap bitmap.
CL 9706, which removes the (never used anymore) wbshadow mode,
gets back what is lost in the early microbenchmarks.
Change-Id: I37423a209e8ec2a2e92538b45cac5422a6acd32d
Reviewed-on: https://go-review.googlesource.com/9705
Reviewed-by: Rick Hudson <rlh@golang.org>
For the conversion of the heap bitmap from 4-bit to 2-bit fields,
I replaced heapBitsSetType with the dumbest thing that could possibly work:
two atomic operations (atomicand8+atomicor8) per 2-bit field.
This CL replaces that code with a proper implementation that
avoids the atomics whenever possible. Benchmarks vs base CL
(before the conversion to 2-bit heap bitmap) and vs Go 1.4 below.
Compared to Go 1.4, SetTypePtr (a 1-pointer allocation)
is 10ns slower because a race against the concurrent GC requires the
use of an atomicor8 that used to be an ordinary write. This slowdown
was present even in the base CL.
Compared to both Go 1.4 and base, SetTypeNode8 (a 10-word allocation)
is 10ns slower because it too needs a new atomic, because with the
denser representation, the byte on the end of the allocation is now shared
with the object next to it; this was not true with the 4-bit representation.
Excluding these two (fundamental) slowdowns due to the use of atomics,
the new code is noticeably faster than both Go 1.4 and the base CL.
The next CL will reintroduce the ``typeDead'' optimization.
Stats are from 5 runs on a MacBookPro10,2 (late 2012 Core i5).
Compared to base CL (** = new atomic)
name old mean new mean delta
SetTypePtr 14.1ns × (0.99,1.02) 14.7ns × (0.93,1.10) ~ (p=0.175)
SetTypePtr8 18.4ns × (1.00,1.01) 18.6ns × (0.81,1.21) ~ (p=0.866)
SetTypePtr16 28.7ns × (1.00,1.00) 22.4ns × (0.90,1.27) -21.88% (p=0.015)
SetTypePtr32 52.3ns × (1.00,1.00) 33.8ns × (0.93,1.24) -35.37% (p=0.001)
SetTypePtr64 79.2ns × (1.00,1.00) 55.1ns × (1.00,1.01) -30.43% (p=0.000)
SetTypePtr126 118ns × (1.00,1.00) 100ns × (1.00,1.00) -15.97% (p=0.000)
SetTypePtr128 130ns × (0.92,1.19) 98ns × (1.00,1.00) -24.36% (p=0.008)
SetTypePtrSlice 726ns × (0.96,1.08) 760ns × (1.00,1.00) ~ (p=0.152)
SetTypeNode1 14.1ns × (0.94,1.15) 12.0ns × (1.00,1.01) -14.60% (p=0.020)
SetTypeNode1Slice 135ns × (0.96,1.07) 88ns × (1.00,1.00) -34.53% (p=0.000)
SetTypeNode8 20.9ns × (1.00,1.01) 32.6ns × (1.00,1.00) +55.37% (p=0.000) **
SetTypeNode8Slice 414ns × (0.99,1.02) 244ns × (1.00,1.00) -41.09% (p=0.000)
SetTypeNode64 80.0ns × (1.00,1.00) 57.4ns × (1.00,1.00) -28.23% (p=0.000)
SetTypeNode64Slice 2.15µs × (1.00,1.01) 1.56µs × (1.00,1.00) -27.43% (p=0.000)
SetTypeNode124 119ns × (0.99,1.00) 100ns × (1.00,1.00) -16.11% (p=0.000)
SetTypeNode124Slice 3.40µs × (1.00,1.00) 2.93µs × (1.00,1.00) -13.80% (p=0.000)
SetTypeNode126 120ns × (1.00,1.01) 98ns × (1.00,1.00) -18.19% (p=0.000)
SetTypeNode126Slice 3.53µs × (0.98,1.08) 3.02µs × (1.00,1.00) -14.49% (p=0.002)
SetTypeNode1024 726ns × (0.97,1.09) 740ns × (1.00,1.00) ~ (p=0.451)
SetTypeNode1024Slice 24.9µs × (0.89,1.37) 23.1µs × (1.00,1.00) ~ (p=0.476)
Compared to Go 1.4 (** = new atomic)
name old mean new mean delta
SetTypePtr 5.71ns × (0.89,1.19) 14.68ns × (0.93,1.10) +157.24% (p=0.000) **
SetTypePtr8 19.3ns × (0.96,1.10) 18.6ns × (0.81,1.21) ~ (p=0.638)
SetTypePtr16 30.7ns × (0.99,1.03) 22.4ns × (0.90,1.27) -26.88% (p=0.005)
SetTypePtr32 51.5ns × (1.00,1.00) 33.8ns × (0.93,1.24) -34.40% (p=0.001)
SetTypePtr64 83.6ns × (0.94,1.12) 55.1ns × (1.00,1.01) -34.12% (p=0.001)
SetTypePtr126 137ns × (0.87,1.26) 100ns × (1.00,1.00) -27.10% (p=0.028)
SetTypePtrSlice 865ns × (0.80,1.23) 760ns × (1.00,1.00) ~ (p=0.243)
SetTypeNode1 15.2ns × (0.88,1.12) 12.0ns × (1.00,1.01) -20.89% (p=0.014)
SetTypeNode1Slice 156ns × (0.93,1.16) 88ns × (1.00,1.00) -43.57% (p=0.001)
SetTypeNode8 23.8ns × (0.90,1.18) 32.6ns × (1.00,1.00) +36.76% (p=0.003) **
SetTypeNode8Slice 502ns × (0.92,1.10) 244ns × (1.00,1.00) -51.46% (p=0.000)
SetTypeNode64 85.6ns × (0.94,1.11) 57.4ns × (1.00,1.00) -32.89% (p=0.001)
SetTypeNode64Slice 2.36µs × (0.91,1.14) 1.56µs × (1.00,1.00) -33.96% (p=0.002)
SetTypeNode124 130ns × (0.91,1.12) 100ns × (1.00,1.00) -23.49% (p=0.004)
SetTypeNode124Slice 3.81µs × (0.90,1.22) 2.93µs × (1.00,1.00) -23.09% (p=0.025)
There are fewer benchmarks vs Go 1.4 because unrolling directly
into the heap bitmap is not yet implemented, so those would not
be meaningful comparisons.
These benchmarks were not present in Go 1.4 as distributed.
The backport to Go 1.4 is in github.com/rsc/go's go14bench branch,
commit 71d5ee5.
Change-Id: I95ed05a22bf484b0fc9efad549279e766c98d2b6
Reviewed-on: https://go-review.googlesource.com/9704
Reviewed-by: Rick Hudson <rlh@golang.org>
Previous CLs changed the representation of the non-heap type bitmaps
to be 1-bit bitmaps (pointer or not). Before this CL, the heap bitmap
stored a 2-bit type for each word and a mark bit and checkmark bit
for the first word of the object. (There used to be additional per-word bits.)
Reduce heap bitmap to 2-bit, with 1 dedicated to pointer or not,
and the other used for mark, checkmark, and "keep scanning forward
to find pointers in this object." See comments for details.
This CL replaces heapBitsSetType with very slow but obviously correct code.
A followup CL will optimize it. (Spoiler: the new code is faster than Go 1.4 was.)
Change-Id: I999577a133f3cfecacebdec9cdc3573c235c7fb9
Reviewed-on: https://go-review.googlesource.com/9703
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
The type information in reflect.Type and the GC programs is now
1 bit per word, down from 2 bits.
The in-memory unrolled type bitmap representation are now
1 bit per word, down from 4 bits.
The conversion from the unrolled (now 1-bit) bitmap to the
heap bitmap (still 4-bit) is not optimized. A followup CL will
work on that, after the heap bitmap has been converted to 2-bit.
The typeDead optimization, in which a special value denotes
that there are no more pointers anywhere in the object, is lost
in this CL. A followup CL will bring it back in the final form of
heapBitsSetType.
Change-Id: If61e67950c16a293b0b516a6fd9a1c755b6d5549
Reviewed-on: https://go-review.googlesource.com/9702
Reviewed-by: Austin Clements <austin@google.com>
There was an old benchmark that measured this indirectly
via allocation, but I don't understand how to factor out the
allocation cost when interpreting the numbers.
Replace with a benchmark that only calls heapBitsSetType,
that does not allocate. This was not possible when the
benchmark was first written, because heapBitsSetType had
not been factored out of mallocgc.
Change-Id: I30f0f02362efab3465a50769398be859832e6640
Reviewed-on: https://go-review.googlesource.com/9701
Reviewed-by: Austin Clements <austin@google.com>
Since we now have stack information for code running on the
systemstack, we can traceback over it. To make cpu profiles useful,
add a case in gentraceback to jump over systemstack switches.
Fixes#10609.
Change-Id: I21f47fcc802c07c5d4a1ada56374314e388a6dc7
Reviewed-on: https://go-review.googlesource.com/9506
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
I have around twenty of such values on a Windows 7 development machine.
regedit displays (translated): "invalid 32-bits DWORD value".
Change-Id: Ib37a414ee4c85e891b0a25fed2ddad9e105f5f4e
Reviewed-on: https://go-review.googlesource.com/9901
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
On darwin/arm, the test sometimes fails with:
Process 557 resuming
--- FAIL: TestWriteTimeoutFluctuation (1.64s)
timeout_test.go:706: Write took over 1s; expected 0.1s
FAIL
Process 557 exited with status = 1 (0x00000001)
go_darwin_arm_exec: timeout running tests
This change increaes timeout on iOS builders from 1s to 3s as a
temporarily fix.
Updates #10775.
Change-Id: Ifdaf99cf5b8582c1a636a0f7d5cc66bb276efd72
Reviewed-on: https://go-review.googlesource.com/9915
Reviewed-by: Minux Ma <minux@golang.org>
ExpandString correctly loops on the syscall until it reaches the
required buffer size but truncates it before converting it back to
string. The truncation limit is increased to 2^15 bytes which is the
documented maximum ExpandEnvironmentStrings output size.
This fixes TestExpandString on systems where len($PATH) > 1024.
Change-Id: I2a6f184eeca939121b458bcffe1a436a50f3298e
Reviewed-on: https://go-review.googlesource.com/9805
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
There is no SYS_INOTIFY_INIT on linux/arm64, only SYS_INOTIFY_INIT1.
Change-Id: I97f430f2c2b910fb19dce495ff1adf591b8634fc
Reviewed-on: https://go-review.googlesource.com/9870
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
The html package uses some specific code to escape special characters.
Actually, the strings.Replacer can be used instead, and is much more
efficient. The converse operation is more complex but can still be
slightly optimized.
Credits to Ken Bloom (kabloom@google.com), who first submitted a
similar patch at https://codereview.appspot.com/141930043
Added benchmarks and slightly optimized UnescapeString.
benchmark old ns/op new ns/op delta
BenchmarkEscape-4 118713 19825 -83.30%
BenchmarkEscapeNone-4 87653 3784 -95.68%
BenchmarkUnescape-4 24888 23417 -5.91%
BenchmarkUnescapeNone-4 14423 157 -98.91%
benchmark old allocs new allocs delta
BenchmarkEscape-4 9 2 -77.78%
BenchmarkEscapeNone-4 0 0 +0.00%
BenchmarkUnescape-4 2 2 +0.00%
BenchmarkUnescapeNone-4 0 0 +0.00%
benchmark old bytes new bytes delta
BenchmarkEscape-4 24800 12288 -50.45%
BenchmarkEscapeNone-4 0 0 +0.00%
BenchmarkUnescape-4 10240 10240 +0.00%
BenchmarkUnescapeNone-4 0 0 +0.00%
Fixes#8697
Change-Id: I208261ed7cbe9b3dee6317851f8c0cf15528bce4
Reviewed-on: https://go-review.googlesource.com/9808
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Delete the colon from RUN: for examples, since it's not there for tests.
Add spaces to line up RUN and PASS: lines.
Before:
=== RUN TestCount
--- PASS: TestCount (0.00s)
=== RUN: ExampleFields
--- PASS: ExampleFields (0.00s)
After:
=== RUN TestCount
--- PASS: TestCount (0.00s)
=== RUN ExampleFields
--- PASS: ExampleFields (0.00s)
Fixes#10594.
Change-Id: I189c80a5d99101ee72d8c9c3a4639c07e640cbd8
Reviewed-on: https://go-review.googlesource.com/9846
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Pipelines are altered by inserting sanitizers if they are not
already present. The code makes the assumption that the first
operands of each commands are function identifiers.
This is wrong, since they can also be methods. It results in
a panic with templates such as {{1|print 2|.f 3}}
Adds an extra type assertion to make sure only identifiers
are compared with sanitizers.
Fixes#10673
Change-Id: I3eb820982675231dbfa970f197abc5ef335ce86b
Reviewed-on: https://go-review.googlesource.com/9801
Reviewed-by: Rob Pike <r@golang.org>
This will make it possible for C++ code to #include the export header
file and see the correct declarations.
The preamble remains the user's responsibility. It would not be
appropriate to wrap the preamble in extern "C", because it might
include header files that work with both C and C++. Putting those
header files in an extern "C" block would break them.
Change-Id: Ifb40879d709d26596d5c80b1307a49f1bd70932a
Reviewed-on: https://go-review.googlesource.com/9850
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
With this patch, gdb seems to be able to corretly backtrace Go
process on at least linux/{arm,arm64,ppc64}.
Change-Id: Ic40a2a70e71a19c4a92e4655710f38a807b67e9a
Reviewed-on: https://go-review.googlesource.com/9822
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It was testing the mark bits on what roots pointed at,
but not the remainder of the live heap, because in
CL 2991 I accidentally inverted this check during
refactoring.
The next CL will turn it back off by default again,
but I want one run on the builders with the full
checkmark checks.
Change-Id: Ic166458cea25c0a56e5387fc527cb166ff2e5ada
Reviewed-on: https://go-review.googlesource.com/9824
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently the heap minimum is set to 4MB which prevents our ability to
collect at every allocation by setting GOGC=0. This adjust the
heap minimum to 4MB*GOGC/100 thus reenabling collecting at every allocation.
Fixes#10681
Change-Id: I912d027dac4b14ae535597e8beefa9ac3fb8ad94
Reviewed-on: https://go-review.googlesource.com/9814
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
This was added during testing but is unnecessary.
Thanks to gravis on GitHub for catching it.
See #10574.
Change-Id: I4a8f76d237e67f5a0ea189a0f3cadddbf426778a
Reviewed-on: https://go-review.googlesource.com/9841
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The code already handled high widths but not high precisions.
Also make sure it handles the harder cases of %U.
Fixes#10745.
Change-Id: Ib4d394d49a9941eeeaff866dc59d80483e312a98
Reviewed-on: https://go-review.googlesource.com/9769
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When
using -buildmode=c-archive or c-shared, and
when installing packages that use cgo, and
when those packages export some functions via //export comments,
then
for each such package, install a pkg.h header file that declares the
functions.
This permits C code to #include the header when calling the Go
functions.
This is a little awkward to use when there are multiple packages that
export functions, as you have to "go install" your c-archive/c-shared
object and then pull it out of the package directory. When compiling
your C code you have to -I pkg/$GOOS_$GOARCH. I haven't thought of
any more convenient approach. It's simpler when only the main package
has exported functions.
When using c-shared you currently have to use a _shared suffix in the
-I option; it would be nice to fix that somehow.
Change-Id: I5d8cf08914b7d3c2b194120c77791d2732ffd26e
Reviewed-on: https://go-review.googlesource.com/9798
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Try to provide hints for common areas, either *interface
were interface would have been better, and note incorrect
capitalization (but don't be more ambitious than that, at
least not today).
Added code and test for cases
ptrInterface.ExistingMethod
ptrInterface.unexportedMethod
ptrInterface.MissingMethod
ptrInterface.withwRongcASEdMethod
interface.withwRongcASEdMethod
ptrStruct.withwRongcASEdMethod
struct.withwRongcASEdMethod
also included tests for related errors to check for
unintentional changes and consistent wording.
Somewhat simplified from previous versions to avoid second-
guessing user errors, yet also biased to point out most-likely
root cause.
Fixes#10700
Change-Id: I16693e93cc8d8ca195e7742a222d640c262105b4
Reviewed-on: https://go-review.googlesource.com/9731
Reviewed-by: Russ Cox <rsc@golang.org>
Current code just checks the consistency (that the functab is correctly
sorted by PC, etc) of the moduledata object that the runtime belongs to.
Change to check all of them.
Change-Id: I544a44c5de7445fff87d3cdb4840ff04c5e2bf75
Reviewed-on: https://go-review.googlesource.com/9773
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When AttrByteSize is not present for a type, we can still determine the
size in two more cases: when the type is a Typedef referring to another
type, and when the type is a pointer and we know the default address
size.
entry.go: return after setting an error if the offset is out of range.
Change-Id: I63a922ca4e4ad2fc9e9be3e5b47f59fae7d0eb5c
Reviewed-on: https://go-review.googlesource.com/9663
Reviewed-by: Austin Clements <austin@google.com>
No code changes, but the test passes here.
And TryBots are happy.
Fixes#8662 maybe
Change-Id: Id37380f72a951c9ad7cf96c0db153c05167e62ed
Reviewed-on: https://go-review.googlesource.com/9778
Reviewed-by: Minux Ma <minux@golang.org>
The -exportheader option tells cgo to generate a header file declaring
expoted functions. The header file is only created if there are, in
fact, some exported functions, so it also serves as a signal as to
whether there were any.
In future CLs the go tool will use this option to install header files
for packages that use cgo and export functions.
Change-Id: I5b04357d453a9a8f0e70d37f8f18274cf40d74c9
Reviewed-on: https://go-review.googlesource.com/9796
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Similar for linux/386 with GO386=387.
Change-Id: If8b6f8a0659a1b3e078d87a43fcfe8a38af20308
Reviewed-on: https://go-review.googlesource.com/9821
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change doesn't work perfectly on IPv6-only kernels including CLAT
enabled kernels, but works enough on IPv4-only kernels.
Fixes#10721.
Updates #10729.
Change-Id: I7db0e572e252aa0a9f9f54c8e557955077b72e44
Reviewed-on: https://go-review.googlesource.com/9777
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Also adds missing temporary file deletion.
Change-Id: Ia644b0898022e05d2f5232af38f51d55e40c6fb5
Reviewed-on: https://go-review.googlesource.com/9772
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Makes searching in source code easier.
Change-Id: Ie2e85934d23920ac0bc01d28168bcfbbdc465580
Reviewed-on: https://go-review.googlesource.com/9774
Reviewed-by: Daniel Morsing <daniel.morsing@gmail.com>
Reviewed-by: Minux Ma <minux@golang.org>
Also copy doc comments from Go code to _cgo_export.h.
This is a step toward installing this generated file when using
-buildmode=c-archive or c-shared, so that C code can #include it.
Change-Id: I3a243f7b386b58ec5c5ddb9a246bb9f9eddc5fb8
Reviewed-on: https://go-review.googlesource.com/9790
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Already done for constants and funcs, but I didn't realize that some
global vars were also not in the global list. This fixes
go doc build.Default
Change-Id: I768bde13a400259df3e46dddc9f58c8f0e993c72
Reviewed-on: https://go-review.googlesource.com/9764
Reviewed-by: Andrew Gerrand <adg@golang.org>
Currently, the GC uses a moving average of recent scan work ratios to
estimate the total scan work required by this cycle. This is in turn
used to compute how much scan work should be done by mutators when
they allocate in order to perform all expected scan work by the time
the allocated heap reaches the heap goal.
However, our current scan work estimate can be arbitrarily wrong if
the heap topography changes significantly from one cycle to the
next. For example, in the go1 benchmarks, at the beginning of each
benchmark, the heap is dominated by a 256MB no-scan object, so the GC
learns that the scan density of the heap is very low. In benchmarks
that then rapidly allocate pointer-dense objects, by the time of the
next GC cycle, our estimate of the scan work can be too low by a large
factor. This in turn lets the mutator allocate faster than the GC can
collect, allowing it to get arbitrarily far ahead of the scan work
estimate, which leads to very long GC cycles with very little mutator
assist that can overshoot the heap goal by large margins. This is
particularly easy to demonstrate with BinaryTree17:
$ GODEBUG=gctrace=1 ./go1.test -test.bench BinaryTree17
gc #1 @0.017s 2%: 0+0+0+0+0 ms clock, 0+0+0+0/0/0+0 ms cpu, 4->262->262 MB, 4 MB goal, 1 P
gc #2 @0.026s 3%: 0+0+0+0+0 ms clock, 0+0+0+0/0/0+0 ms cpu, 262->262->262 MB, 524 MB goal, 1 P
testing: warning: no tests to run
PASS
BenchmarkBinaryTree17 gc #3 @1.906s 0%: 0+0+0+0+7 ms clock, 0+0+0+0/0/0+7 ms cpu, 325->325->287 MB, 325 MB goal, 1 P (forced)
gc #4 @12.203s 20%: 0+0+0+10067+10 ms clock, 0+0+0+0/2523/852+10 ms cpu, 430->2092->1950 MB, 574 MB goal, 1 P
1 9150447353 ns/op
Change this estimate to instead use the *current* scannable heap
size. This has the advantage of being based solely on the current
state of the heap, not on past densities or reachable heap sizes, so
it isn't susceptible to falling behind during these sorts of phase
changes. This is strictly an over-estimate, but it's better to
over-estimate and get more assist than necessary than it is to
under-estimate and potentially spiral out of control. Experiments with
scaling this estimate back showed no obvious benefit for mutator
utilization, heap size, or assist time.
This new estimate has little effect for most benchmarks, including
most go1 benchmarks, x/benchmarks, and the 6g benchmark. It has a huge
effect for benchmarks that triggered the bad pacer behavior:
name old mean new mean delta
BinaryTree17 10.0s × (1.00,1.00) 3.5s × (0.98,1.01) -64.93% (p=0.000)
Fannkuch11 2.74s × (1.00,1.01) 2.65s × (1.00,1.00) -3.52% (p=0.000)
FmtFprintfEmpty 56.4ns × (0.99,1.00) 57.8ns × (1.00,1.01) +2.43% (p=0.000)
FmtFprintfString 187ns × (0.99,1.00) 185ns × (0.99,1.01) -1.19% (p=0.010)
FmtFprintfInt 184ns × (1.00,1.00) 183ns × (1.00,1.00) (no variance)
FmtFprintfIntInt 321ns × (1.00,1.00) 315ns × (1.00,1.00) -1.80% (p=0.000)
FmtFprintfPrefixedInt 266ns × (1.00,1.00) 263ns × (1.00,1.00) -1.22% (p=0.000)
FmtFprintfFloat 353ns × (1.00,1.00) 353ns × (1.00,1.00) -0.13% (p=0.035)
FmtManyArgs 1.21µs × (1.00,1.00) 1.19µs × (1.00,1.00) -1.33% (p=0.000)
GobDecode 9.69ms × (1.00,1.00) 9.59ms × (1.00,1.00) -1.07% (p=0.000)
GobEncode 7.89ms × (0.99,1.01) 7.74ms × (1.00,1.00) -1.92% (p=0.000)
Gzip 391ms × (1.00,1.00) 392ms × (1.00,1.00) ~ (p=0.522)
Gunzip 97.1ms × (1.00,1.00) 97.0ms × (1.00,1.00) -0.10% (p=0.000)
HTTPClientServer 55.7µs × (0.99,1.01) 56.7µs × (0.99,1.01) +1.81% (p=0.001)
JSONEncode 19.1ms × (1.00,1.00) 19.0ms × (1.00,1.00) -0.85% (p=0.000)
JSONDecode 66.8ms × (1.00,1.00) 66.9ms × (1.00,1.00) ~ (p=0.288)
Mandelbrot200 4.13ms × (1.00,1.00) 4.12ms × (1.00,1.00) -0.08% (p=0.000)
GoParse 3.97ms × (1.00,1.01) 4.01ms × (1.00,1.00) +0.99% (p=0.000)
RegexpMatchEasy0_32 114ns × (1.00,1.00) 115ns × (0.99,1.00) ~ (p=0.070)
RegexpMatchEasy0_1K 376ns × (1.00,1.00) 376ns × (1.00,1.00) ~ (p=0.900)
RegexpMatchEasy1_32 94.9ns × (1.00,1.00) 96.3ns × (1.00,1.01) +1.53% (p=0.001)
RegexpMatchEasy1_1K 568ns × (1.00,1.00) 567ns × (1.00,1.00) -0.22% (p=0.001)
RegexpMatchMedium_32 159ns × (1.00,1.00) 159ns × (1.00,1.00) ~ (p=0.178)
RegexpMatchMedium_1K 46.4µs × (1.00,1.00) 46.6µs × (1.00,1.00) +0.29% (p=0.000)
RegexpMatchHard_32 2.37µs × (1.00,1.00) 2.37µs × (1.00,1.00) ~ (p=0.722)
RegexpMatchHard_1K 71.1µs × (1.00,1.00) 71.2µs × (1.00,1.00) ~ (p=0.229)
Revcomp 565ms × (1.00,1.00) 562ms × (1.00,1.00) -0.52% (p=0.000)
Template 81.0ms × (1.00,1.00) 80.2ms × (1.00,1.00) -0.97% (p=0.000)
TimeParse 380ns × (1.00,1.00) 380ns × (1.00,1.00) ~ (p=0.148)
TimeFormat 405ns × (0.99,1.00) 385ns × (0.99,1.00) -5.00% (p=0.000)
Change-Id: I11274158bf3affaf62662e02de7af12d5fb789e4
Reviewed-on: https://go-review.googlesource.com/9696
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
This tracks the number of scannable bytes in the allocated heap. That
is, bytes that the garbage collector must scan before reaching the
last pointer field in each object.
This will be used to compute a more robust estimate of the GC scan
work.
Change-Id: I1eecd45ef9cdd65b69d2afb5db5da885c80086bb
Reviewed-on: https://go-review.googlesource.com/9695
Reviewed-by: Russ Cox <rsc@golang.org>
The garbage collector predicts how much "scan work" must be done in a
cycle to determine how much work should be done by mutators when they
allocate. Most code doesn't care what units the scan work is in: it
simply knows that a certain amount of scan work has to be done in the
cycle. Currently, the GC uses the number of pointer slots scanned as
the scan work on the theory that this is the bulk of the time spent in
the garbage collector and hence reflects real CPU resource usage.
However, this metric is difficult to estimate at the beginning of a
cycle.
Switch to counting the total number of bytes scanned, including both
pointer and scalar slots. This is still less than the total marked
heap since it omits no-scan objects and no-scan tails of objects. This
metric may not reflect absolute performance as well as the count of
scanned pointer slots (though it still takes time to scan scalar
fields), but it will be much easier to estimate robustly, which is
more important.
Change-Id: Ie3a5eeeb0384a1ca566f61b2f11e9ff3a75ca121
Reviewed-on: https://go-review.googlesource.com/9694
Reviewed-by: Russ Cox <rsc@golang.org>
Currently, we only flush the per-P gcWork caches in gcMark, at the
beginning of mark termination. This is necessary to ensure that no
work is held up in these caches.
However, this flush happens after we update the GC controller state,
which depends on statistics about marked heap size and scan work that
are only updated by this flush. Hence, the controller is missing the
bulk of heap marking and scan work. This bug was introduced in commit
1b4025f, which introduced the per-P gcWork caches.
Fix this by flushing these caches before we update the GC controller
state. We continue to flush them at the beginning of mark termination
as well to be robust in case any write barriers happened between the
previous flush and entering mark termination, but this should be a
no-op.
Change-Id: I8f0f91024df967ebf0c616d1c4f0c339c304ebaa
Reviewed-on: https://go-review.googlesource.com/9646
Reviewed-by: Russ Cox <rsc@golang.org>
Ramp up the delay on subsequent attempts. Fast builders have the same delay.
Not a perfect fix, but should make it better. And this easy.
Fixes#9903 maybe
Fixes#10680 maybe
Change-Id: I967380c2cb8196e6da9a71116961229d37b36335
Reviewed-on: https://go-review.googlesource.com/9795
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Android has (had?) its own local DNS resolver daemon, also my fault:
007e987fee
And you access that via libc, not DNS.
Fixes#10714
Change-Id: Iaff752872ce19bb5c7771ab048fd50e3f72cb73c
Reviewed-on: https://go-review.googlesource.com/9793
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Improving the usability further.
Before:
$ go doc bytes.Read
doc: symbol Read not present in package bytes installed in "bytes"
$
After:
$ go doc bytes.Read
func (b *Buffer) Read(p []byte) (n int, err error)
Read reads the next len(p) bytes from the buffer or until the buffer is drained.
The return value n is the number of bytes read. If the buffer has no data to
return, err is io.EOF (unless len(p) is zero); otherwise it is nil.
func (r *Reader) Read(b []byte) (n int, err error)
$
Change-Id: I646511fada138bd09e9b39820da01a5ccef4a90f
Reviewed-on: https://go-review.googlesource.com/9656
Reviewed-by: Russ Cox <rsc@golang.org>
GOROOT is not dependably set.
When I first wrote this test, I thought it was a waste of time
because the function can't fail if the other environment functions
work, but I didn't want to add functionality without testing it.
Of course, the test broke, and I learned something: GOROOT is not
set on iOS or, to put it more broadly, the world continues to
surprise me with its complexity and horror, such as a version of
cat with syntax coloring.
In that vein, I built this test around smallpox.
Change-Id: Ifa6c218a927399d05c47954fdcaea1015e558fb6
Reviewed-on: https://go-review.googlesource.com/9791
Reviewed-by: Russ Cox <rsc@golang.org>
During development some tracing routines were added that are not
needed in the release. These included GCstarttimes, GCendtimes, and
GCprinttimes.
Fixes#10462
Change-Id: I0788e6409d61038571a5ae0cbbab793102df0a65
Reviewed-on: https://go-review.googlesource.com/9689
Reviewed-by: Austin Clements <austin@google.com>
Fixes#10221.
Change-Id: Ib23805494d8af1946360bfea767f9727e2504dc5
Reviewed-on: https://go-review.googlesource.com/7941
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Unfortunately Oracle Solaris does not have TCP_KEEPIDLE and
TCP_KEEPINTVL. TCP_KEEPIDLE is equivalent to TCP_KEEPALIVE_THRESHOLD,
but TCP_KEEPINTVL does not have a direct equivalent, so we don't set
TCP_KEEPINTVL any more.
Old Darwin versions also lack TCP_KEEPINTVL, but the code tries to set
it anyway so that it works on newer versions. We can't do that because
Oracle might assign the number illumos uses for TCP_KEEPINTVL to a
constant with a different meaning.
Unfortunately there's nothing we can do if we want to support both
illumos and Oracle Solaris with the same GOOS.
Updates #9614.
Change-Id: Id39eb5147f7afa8e951f886c0bf529d00f0e1bd4
Reviewed-on: https://go-review.googlesource.com/7690
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Before CL 8214 (use .plt instead of .got on Solaris) Solaris used a
dynamic linking scheme that didn't permit lazy binding. To speed program
startup, Go binaries only used it for a small number of symbols required
by the runtime. Other symbols were resolved on demand on first use, and
were cached for subsequent use. This required some moderately complex
code in the syscall package.
CL 8214 changed the way dynamic linking is implemented, and now lazy
binding is supported. As now all symbols are resolved lazily by the
dynamic loader, there is no need for the complex code in the syscall
package that did the same. This CL makes Go programs link directly
with the necessary shared libraries and deletes the lazy-loading code
implemented in Go.
Change-Id: Ifd7275db72de61b70647242e7056dd303b1aee9e
Reviewed-on: https://go-review.googlesource.com/9184
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
ELF normally requires this and Solaris runtime loader will crash if we
don't do it.
Fixes Solaris build.
Change-Id: I0482eed890aff2d346136ae7f9caf8f094f502ed
Reviewed-on: https://go-review.googlesource.com/8216
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The linker always uses .plt for externals, so libcFunc is now an actual
external symbol instead of a pointer to one.
Fixes most of the breakage introduced in previous CL.
Change-Id: I64b8c96f93127f2d13b5289b024677fd3ea7dbea
Reviewed-on: https://go-review.googlesource.com/8215
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Solaris requires all external procedures to be accessed through the
PLT. If 6l won't do it, /bin/ld will, so all the code written with .GOT
in mind won't work with the external linker.
This CL makes external linking work, opening the path to cgo support
on Solaris.
This CL breaks the Solaris build, this is fixed in subsequent CLs in
this series.
Change-Id: If370a79f49fdbe66d28b89fa463b4f3e91685f69
Reviewed-on: https://go-review.googlesource.com/8214
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
This change simplifies unnecessarily redundant error messages in tests.
There's no need to worry any more because package APIs now return
consistent, self-descriptive error values.
Alos renames ambiguous test functions and makes use of test tables.
Change-Id: I7b61027607c4ae2a3cf605d08d58cf449fa27eb2
Reviewed-on: https://go-review.googlesource.com/9662
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
This change makes TestDualStack{TCP,UDP}Listener work more properly by
attempting to book an available service port before testing.
Also simplifies error messages in tests.
Fixes#5001.
Change-Id: If13b0d0039878c9bd32061a0440664e4fa7abaf7
Reviewed-on: https://go-review.googlesource.com/9661
Reviewed-by: Ian Lance Taylor <iant@golang.org>
isn't (0, 0).
Also fix a s/b.Min.X/b.Max.X/ typo in bounds checking.
Fixes#10676
Change-Id: Ie5ff7ec20ca30367a8e65d32061959a2d8e089e9
Reviewed-on: https://go-review.googlesource.com/9712
Reviewed-by: Rob Pike <r@golang.org>
An ELF linker handles a PC-relative reference to an STT_FUNC defined in a
shared library by building a PLT entry and referring to that, so do the
same in 6l.
Fixes#10690
Change-Id: I061a96fd4400d957e301d0ac86760ce256910e1d
Reviewed-on: https://go-review.googlesource.com/9711
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Catch some malformed pipelines at parsing time.
The current code accepts pipelines such as:
{{12|.}}
{{"hello"|print|false}}
{{.|"blah blah"}}
Such pipelines generate panic in html/template at execution time.
Add an extra check to verify all the commands of the pipeline are executable
(except for the first one).
Fixes#10610
Change-Id: Id72236ba8f76a59fa284fe3d4c2cb073e50b51f1
Reviewed-on: https://go-review.googlesource.com/9626
Reviewed-by: Rob Pike <r@golang.org>
This makes the intermediate object file a little bigger but it doesn't waste
any space in the final shared library.
Fixes#10691
Change-Id: Ic51a571d60291f1ac2dad1b50dba4679643168ae
Reviewed-on: https://go-review.googlesource.com/9710
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This changes the action graph when shared libraries are involved to always have
an action for the shared library (which does nothing when the shared library
is up to date).
Change-Id: Ibbc70fd01cbb3f4e8c0ef96e62a151002d446144
Reviewed-on: https://go-review.googlesource.com/8934
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes#10660
Fix the clang only builder by passing -extld down to the linker when needed.
The build passed on most hosts because gcc is almost always present. The bug
was verified by symlinking bin/false in place of gcc in my $PATH and running
the build.
Also, resolve a TODO and move the support logic into its own function.
Tested manually
env CC=clang-3.5 ./all.bash # linux/amd64
env CC=gcc-4.8 ./all.bash # linux/amd64
./all.bash # linux/amd64
./all.bash # darwin/amd64
Change-Id: I4e27a1119356e295500a0d19ad7a4ec14207bf10
Reviewed-on: https://go-review.googlesource.com/9526
Run-TryBot: Dave Cheney <dave@cheney.net>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I just submitted the real fix for #10641.
This reverts commit 3120adc212.
Change-Id: I55051515f697e27ca887ed21c2ac985f0b9b062b
Reviewed-on: https://go-review.googlesource.com/9720
Reviewed-by: Joel Sing <jsing@google.com>
No semantic change.
Fixes#8708.
Change-Id: Ieda04a86a19bb69bfc2519d381a2f025e7cb8279
Reviewed-on: https://go-review.googlesource.com/9740
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I forgot there is already a ptrSize constant.
Rename field to avoid some confusion.
Change-Id: I098fdcc8afc947d6c02c41c6e6de24624cc1c8ff
Reviewed-on: https://go-review.googlesource.com/9700
Reviewed-by: Austin Clements <austin@google.com>
The old one was inferior.
Fixes#10695.
Change-Id: Ia7fb88c9ceb1b10197b77a54f729865385288d98
Reviewed-on: https://go-review.googlesource.com/9709
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
When a parse error occurred, the lexing goroutine would lay idle.
It's not likely a problem but if the program is for some reason
accepting badly formed data repeatedly, it's wasteful.
The solution is easy: Just drain the input on error. We know this
will succeed because the input is always a string and is therefore
guaranteed finite.
With debugging prints in the package tests I've shown this is effective,
shutting down 79 goroutines that would otherwise linger, out of 123 total.
Fixes#10574.
Change-Id: I8aa536e327b219189a7e7f604a116fa562ae1c39
Reviewed-on: https://go-review.googlesource.com/9658
Reviewed-by: Russ Cox <rsc@golang.org>
Freezetheworld still has stuff to do when gomaxprocs=1.
In particular, signals can come in on other Ms (like the GC M, say)
and the single user M is still running.
Fixes#10546
Change-Id: I2f07f17d1c81e93cf905df2cb087112d436ca7e7
Reviewed-on: https://go-review.googlesource.com/9551
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
When emulating ARM FSQRT instruction, the sqrt function itself
should not use any floating point arithmetics, otherwise it will
clobber the user software FP registers.
Fortunately, the sqrt function only uses floating point instructions
to test for corner cases, so it's easy to make that function does
all it job using pure integer arithmetic only. I've verified that
after this change, runtime.stepflt and runtime.sqrt doesn't contain
any call to _sfloat. (Perhaps we should add //go:nosfloat to make
the compiler enforce this?)
Fixes#10641.
Change-Id: Ida4742c49000fae4fea4649f28afde630ce4c576
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/9570
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Keith Randall <khr@golang.org>
OSQRT currently produces incorrect results when used on arm with softfloat.
Disable it on GOARM=5 until the actual problem is found and fixed.
Updates #10641
Change-Id: Ia6f6879fbbb05cb24399c2feee93c1be21113e73
Reviewed-on: https://go-review.googlesource.com/9524
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Whenever we introduce a new GOARCH, older Go releases won't
recognize them and this causes trouble for both our users and
us (we need to add unnecessary build tags).
Go 1.5 has introduced three new GOARCHes so far: arm64 ppc64
ppc64le, we can take the time to introduce GOARCHes for all
common architectures that Go might support in the future to
avoid the problem.
Fixes#10165.
Change-Id: Ida4f9112897cfb1e85b06538db79125955ad0f4c
Reviewed-on: https://go-review.googlesource.com/9644
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Update #10652
This proposal deletes cmd/internal/ld.Biobuf and replaces all uses with
cmd/internal/obj.Biobuf. As cmd/internal/ld already imported cmd/internal/obj
there are no additional dependencies created.
Notes:
- ld.Boffset included more checks, so it was merged into obj.Boffset
- obj.Bflush was removed in 8d16253c90, so replaced all calls to
ld.Bflush, with obj.Biobuf.Flush.
- Almost all of this change was prepared with sed.
Change-Id: I814854d52f5729a5a40c523c8188e465246b88da
Reviewed-on: https://go-review.googlesource.com/9660
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Dave Cheney <dave@cheney.net>
Write should return ErrWriteAfterClose instead
of ErrWriteTooLong when called after Close.
Change-Id: If5ec4ef924e4c56489e0d426976f7e5fad79be9b
Reviewed-on: https://go-review.googlesource.com/9259
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds a field to the runtime type structure that records the size
of the prefix of objects of that type containing pointers. Any data
after this offset is scalar data.
This is necessary for shrinking the type bitmaps to 1 bit and will
help the garbage collector efficiently estimate the amount of heap
that needs to be scanned.
Change-Id: I1318d79e6360dca0ac980245016c562e61f52ff5
Reviewed-on: https://go-review.googlesource.com/9691
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Move the one instance of type structure decoding in the linker that
doesn't live decodesym.go in to decodesym.go.
Change-Id: Ic6a23500deb72f0e9c8227ab611511e9781fac70
Reviewed-on: https://go-review.googlesource.com/9690
Reviewed-by: Russ Cox <rsc@golang.org>
shouldtriggergc is slightly expensive due to the call overhead
and the use of an atomic. This CL reduces the number of time
one checks if a GC should be done from one at each allocation
to once when a span is allocated. Since shouldtriggergc is an
important abstraction simply hand inlining it, along with its
atomic instruction would lose the abstraction.
Change-Id: Ia3210655b4b3d433f77064a21ecb54e4d9d435f7
Reviewed-on: https://go-review.googlesource.com/9403
Reviewed-by: Austin Clements <austin@google.com>
At the end of lexInsideAction(), we return lexInsideAction: this is the default
behaviour when we are still parsing an action. But some switch branches return
lexInsideAction too.
So let's ensure code consistency by always reaching the end of the
lexInsideAction function when needed.
Change-Id: I7e9d8d6e51f29ecd6db6bdd63b36017845d95368
Reviewed-on: https://go-review.googlesource.com/9441
Reviewed-by: Rob Pike <r@golang.org>
We shouldn't sort the slots array, as it is used each time the
test is run. Tests after the first should continue to use the
unsorted ordering.
Note that this doesn't fix the flaky test. Just a bug I saw
while investigating.
Change-Id: Ic03cca637829d569d50d3a2278d19410d4dedba9
Reviewed-on: https://go-review.googlesource.com/9637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
global color table.
Change-Id: Ia38f75708ed5e5b430680a1eecafb4fc8047269c
Reviewed-on: https://go-review.googlesource.com/9467
Reviewed-by: Rob Pike <r@golang.org>
The script was renamed in b3000b6.
Change-Id: I45ecafff7400e4bff14f31906278609abf2bcb9f
Reviewed-on: https://go-review.googlesource.com/9667
Reviewed-by: Andrew Gerrand <adg@golang.org>
Ideal constants in the template package are a little different from Go.
This is a case that slipped through the cracks: A huge integer number
was accepted as a floating-point number, but this loses precision
and is confusing. Also, the code in the template package (as opposed
to the parse package) wasn't expecting it.
Root this out at the source: If an integer doesn't fit an int64 or uint64,
complain right away.
Change-Id: I375621e6f5333c4d53f053a3c84a9af051711b7a
Reviewed-on: https://go-review.googlesource.com/9651
Reviewed-by: Russ Cox <rsc@golang.org>
The siz argument to both runtime.newproc and runtime.deferproc is
int32, not uintptr. This problem won't manifest on little-endian
systems because that stack slot is uintptr sized anyway. However,
on big-endian systems, it will make a difference.
Change-Id: I2351d1ec81839abe25375cff95e327b80764c2b5
Reviewed-on: https://go-review.googlesource.com/9647
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The current parser ignores obvious errors such as:
{{0.1.E}}
{{true.any}}
{{"hello".wrong}}
{{nil.E}}
The common problem is that a chain is built from
a literal value. It then panics at execution time.
Furthermore, a double dot triggers the same behavior:
{{..E}}
Addresses a TODO left in Tree.operand to catch these
errors at parsing time.
Note that identifiers can include a '.', and pipelines
could return an object which a field can be derived
from (like a variable), so they are excluded from the check.
Fixes#10615
Change-Id: I903706d1c17861b5a8354632c291e73c9c0bc4e1
Reviewed-on: https://go-review.googlesource.com/9621
Reviewed-by: Rob Pike <r@golang.org>
There are three problems:
1. There is no CR at the end of the message.
2. The message is unconditionally printed.
3. The message is printed to stdout.
Change-Id: Ib2d880eea03348e8a69720aad7752302a75bd277
Reviewed-on: https://go-review.googlesource.com/9622
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change applies CL 9365 to the copy of Biobuf in cmd/internal/obj.
In the process I discovered that some of the methods that should have been
checking the unget buffer before reading were not and it was probably just
dumb luck that we handn't hit these issues before; Bungetc is only used in
one place in cmd/internal/gc and only an unlikely code path.
Change-Id: Ifa0c5c08442e9fe951a5078c6e9ec77a8a4dc2ff
Reviewed-on: https://go-review.googlesource.com/9529
Reviewed-by: Daniel Morsing <daniel.morsing@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Run-TryBot: Dave Cheney <dave@cheney.net>
The test was measuring something, assuming other goroutines had
already scheduled.
Fixes#10427
Change-Id: I2a4d3906f9d4b5ea44b57d972e303bbe2b0b1cde
Reviewed-on: https://go-review.googlesource.com/9561
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
An unmatched {{else}} should trigger a parsing error.
The top level parser is able to issue an error in case
of unmatched {{end}}. It does it a posteriori (i.e. after having
parsed the action).
Extend this behavior to also check for unmatched {{else}}
Fixes#10611
Change-Id: I1d4f433cc64e11bea5f4d61419ccc707ac01bb1d
Reviewed-on: https://go-review.googlesource.com/9620
Reviewed-by: Rob Pike <r@golang.org>
The usage messages for the flags in gc and ld are using the old
flag argument syntax:
"arg: description using arg"
Update them to the Go 1.5 flag package's syntax:
"description using arg"
Fixes#10505
Change-Id: Ifa54ff91e1fd644cfc9a3b41e10176eac3654137
Reviewed-on: https://go-review.googlesource.com/9505
Reviewed-by: Rob Pike <r@golang.org>
This adds a detailed debug dump of the state of the GC controller and
a GODEBUG flag to enable it.
Change-Id: I562fed7981691a84ddf0f9e6fcd9f089f497ac13
Reviewed-on: https://go-review.googlesource.com/9640
Reviewed-by: Russ Cox <rsc@golang.org>
(1) Count pointer-free objects found during scanning roots
as marked bytes, by not zeroing the mark total after scanning roots.
(2) Don't count the bytes for the roots themselves, by not adding
them to the mark total in scanblock (the zeroing removed by (1)
was aimed at that add but hitting more).
Combined, (1) and (2) fix the calculation of the marked heap size.
This makes the GC trigger much less often in the Go 1 benchmarks,
which have a global []byte pointing at 256 MB of data.
That 256 MB allocation was not being included in the heap size
in the current code, but was included in Go 1.4.
This is the source of much of the relative slowdown in that directory.
(3) Count the bytes for the roots as scanned work, by not zeroing
the scan total after scanning roots. There is no strict justification
for this, and it probably doesn't matter much either way,
but it was always combined with another buggy zeroing
(removed in (1)), so guilty by association.
Austin noticed this.
name old mean new mean delta
BenchmarkBinaryTree17 13.1s × (0.97,1.03) 5.9s × (0.97,1.05) -55.19% (p=0.000)
BenchmarkFannkuch11 4.35s × (0.99,1.01) 4.37s × (1.00,1.01) +0.47% (p=0.032)
BenchmarkFmtFprintfEmpty 84.6ns × (0.95,1.14) 85.7ns × (0.94,1.05) ~ (p=0.521)
BenchmarkFmtFprintfString 320ns × (0.95,1.06) 283ns × (0.99,1.02) -11.48% (p=0.000)
BenchmarkFmtFprintfInt 311ns × (0.98,1.03) 288ns × (0.99,1.02) -7.26% (p=0.000)
BenchmarkFmtFprintfIntInt 554ns × (0.96,1.05) 478ns × (0.99,1.02) -13.70% (p=0.000)
BenchmarkFmtFprintfPrefixedInt 434ns × (0.96,1.06) 393ns × (0.98,1.04) -9.60% (p=0.000)
BenchmarkFmtFprintfFloat 620ns × (0.99,1.03) 584ns × (0.99,1.01) -5.73% (p=0.000)
BenchmarkFmtManyArgs 2.19µs × (0.98,1.03) 1.94µs × (0.99,1.01) -11.62% (p=0.000)
BenchmarkGobDecode 21.2ms × (0.97,1.06) 15.2ms × (0.99,1.01) -28.17% (p=0.000)
BenchmarkGobEncode 18.1ms × (0.94,1.06) 11.8ms × (0.99,1.01) -35.00% (p=0.000)
BenchmarkGzip 650ms × (0.98,1.01) 649ms × (0.99,1.02) ~ (p=0.802)
BenchmarkGunzip 143ms × (1.00,1.01) 143ms × (1.00,1.01) ~ (p=0.438)
BenchmarkHTTPClientServer 110µs × (0.98,1.04) 101µs × (0.98,1.02) -8.79% (p=0.000)
BenchmarkJSONEncode 40.3ms × (0.97,1.03) 31.8ms × (0.98,1.03) -20.92% (p=0.000)
BenchmarkJSONDecode 119ms × (0.97,1.02) 108ms × (0.99,1.02) -9.15% (p=0.000)
BenchmarkMandelbrot200 6.03ms × (1.00,1.01) 6.03ms × (0.99,1.01) ~ (p=0.750)
BenchmarkGoParse 8.58ms × (0.89,1.10) 6.80ms × (1.00,1.00) -20.71% (p=0.000)
BenchmarkRegexpMatchEasy0_32 162ns × (1.00,1.01) 162ns × (0.99,1.02) ~ (p=0.131)
BenchmarkRegexpMatchEasy0_1K 540ns × (0.99,1.02) 559ns × (0.99,1.02) +3.58% (p=0.000)
BenchmarkRegexpMatchEasy1_32 139ns × (0.98,1.04) 139ns × (1.00,1.00) ~ (p=0.466)
BenchmarkRegexpMatchEasy1_1K 889ns × (0.99,1.01) 885ns × (0.99,1.01) -0.50% (p=0.022)
BenchmarkRegexpMatchMedium_32 252ns × (0.99,1.02) 252ns × (0.99,1.01) ~ (p=0.469)
BenchmarkRegexpMatchMedium_1K 72.9µs × (0.99,1.01) 73.6µs × (0.99,1.03) ~ (p=0.168)
BenchmarkRegexpMatchHard_32 3.87µs × (1.00,1.01) 3.86µs × (1.00,1.00) ~ (p=0.055)
BenchmarkRegexpMatchHard_1K 118µs × (0.99,1.01) 117µs × (0.99,1.00) ~ (p=0.133)
BenchmarkRevcomp 995ms × (0.94,1.10) 949ms × (0.99,1.01) -4.64% (p=0.000)
BenchmarkTemplate 141ms × (0.97,1.02) 127ms × (0.99,1.01) -10.00% (p=0.000)
BenchmarkTimeParse 641ns × (0.99,1.01) 623ns × (0.99,1.01) -2.79% (p=0.000)
BenchmarkTimeFormat 729ns × (0.98,1.03) 679ns × (0.99,1.00) -6.93% (p=0.000)
Change-Id: I839bd7356630d18377989a0748763414e15ed057
Reviewed-on: https://go-review.googlesource.com/9602
Reviewed-by: Austin Clements <austin@google.com>
This is a follow up to rev 443a32e707 which reduces some of the
duplication between methods and functions that operate on obj.Biobuf.
obj.Biobuf has Flush and Write methods as well as helpers which duplicate
those methods, consolidate on the former and remove the latter.
Also, address a final comment from CL 9525.
Change-Id: I67deaf3a163bb489a9bb21bb39524785d7a2f6c5
Reviewed-on: https://go-review.googlesource.com/9527
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ensures that parameter flow bits are not set for tags EscScope, EscHeap, EscNever;
crash the compiler earl to expose faulty logic, rather than flake out silently downstream.
Change-Id: I1428129980ae047d02975f033d56cbbd04f49579
Reviewed-on: https://go-review.googlesource.com/9601
Reviewed-by: Russ Cox <rsc@golang.org>
This reverts commit c26fc88d56.
This broke pprof. See the comments at 9491.
Change-Id: Ic99ce026e86040c050a9bf0ea3024a1a42274ad1
Reviewed-on: https://go-review.googlesource.com/9565
Reviewed-by: Daniel Morsing <daniel.morsing@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Kind of a hack, but makes the non-optimized builds pass.
Fixes#10079
Change-Id: I26f41c546867f8f3f16d953dc043e784768f2aff
Reviewed-on: https://go-review.googlesource.com/9552
Reviewed-by: Russ Cox <rsc@golang.org>
This includes the following information in the per-function summary:
outK = paramJ encoded in outK bits for paramJ
outK = *paramJ encoded in outK bits for paramJ
heap = paramJ EscHeap
heap = *paramJ EscContentEscapes
Note that (currently) if the address of a parameter is taken and
returned, necessarily a heap allocation occurred to contain that
reference, and the heap can never refer to stack, therefore the
parameter and everything downstream from it escapes to the heap.
The per-function summary information now has a tuneable number of bits
(2 is probably noticeably better than 1, 3 is likely overkill, but it
is now easy to check and the -m debugging output includes information
that allows you to figure out if more would be better.)
A new test was added to check pointer flow through struct-typed and
*struct-typed parameters and returns; some of these are sensitive to
the number of summary bits, and ought to yield better results with a
more competent escape analysis algorithm. Another new test checks
(some) correctness with array parameters, results, and operations.
The old analysis inferred a piece of plan9 runtime was non-escaping by
counteracting overconservative analysis with buggy analysis; with the
bug fixed, the result was too conservative (and it's not easy to fix
in this framework) so the source code was tweaked to get the desired
result. A test was added against the discovered bug.
The escape analysis was further improved splitting the "level" into
3 parts, one tracking the conventional "level" and the other two
computing the highest-level-suffix-from-copy, which is used to
generally model the cancelling effect of indirection applied to
address-of.
With the improved escape analysis enabled, it was necessary to
modify one of the runtime tests because it now attempts to allocate
too much on the (small, fixed-size) G0 (system) stack and this
failed the test.
Compiling src/std after touching src/runtime/*.go with -m logging
turned on shows 420 fewer heap allocation sites (10538 vs 10968).
Profiling allocations in src/html/template with
for i in {1..5} ;
do go tool 6g -memprofile=mastx.${i}.prof -memprofilerate=1 *.go;
go tool pprof -alloc_objects -text mastx.${i}.prof ;
done
showed a 15% reduction in allocations performed by the compiler.
Update #3753
Update #4720Fixes#10466
Change-Id: I0fd97d5f5ac527b45f49e2218d158a6e89951432
Reviewed-on: https://go-review.googlesource.com/8202
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
App Store policy requires programs do not reference the exc_server
symbol. (Some public forum threads show that Unity ran into this
several years ago and it is a hard policy rule.) While some research
suggests that I could write my own version of exc_server, the
expedient course is to disable the exception handler by default.
Go programs only need it when running under lldb, which is primarily
used by tests. So enable the exception handler in cmd/dist when we
are running the tests.
Fixes#10646
Change-Id: I853905254894b5367edb8abd381d45585a78ee8b
Reviewed-on: https://go-review.googlesource.com/9549
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Fix the various builds which don't have a real filesystem or don't support forking.
Change-Id: I3075c662fe6191ecbe70ba359b73d9a88bb06f35
Reviewed-on: https://go-review.googlesource.com/9528
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Forgot to update the references to the old cover package. No excuse.
Change-Id: If17b7521f0bf70bc0c8da9c5adf246d90f644637
Reviewed-on: https://go-review.googlesource.com/9564
Reviewed-by: Rob Pike <r@golang.org>
Fixes#10592
Calling gc.Fatal before gc.Main has been called ends up flushing gc.bstdout before
it is properly set up. Ideally obj.Bflush would handle this case, but that type
and its callers are rather convoluted, so take the simpler route and avoid calling
gc.Fatal altogether.
Change-Id: I338b469e86edba558b6bedff35bb904bfc3d6990
Reviewed-on: https://go-review.googlesource.com/9525
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Run-TryBot: Dave Cheney <dave@cheney.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This was disallowed for error-checking reasons but people ask for
it, it's easy, and it's clear what it all means.
Fixes#7323.
Change-Id: I26542f5ac6519e45b335ad789713a4d9e356279b
Reviewed-on: https://go-review.googlesource.com/9537
Reviewed-by: Russ Cox <rsc@golang.org>
This required dealing with the ill-advised split of the profile code
into a separate package. I just copied it over unchanged. The package
does not deserve to be in the standard repository. We can cope
with the duplication.
Also update the go command to know about the new location.
Fixes#10528.
Change-Id: I05170ef3663326d57b9c18888d01163acd9256b6
Reviewed-on: https://go-review.googlesource.com/9560
Reviewed-by: Russ Cox <rsc@golang.org>
Per resolv.conf man page, "If this file does not exist, only the name
server on the local machine will be queried."
This behavior also occurs if file is present but unreadable,
or if no nameservers are listed.
Fixes#10566
Change-Id: Id5716da0eae534d5ebfafea111bbc657f302e307
Reviewed-on: https://go-review.googlesource.com/9380
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change-Id: I611f7dec2109dc7e2f090ced0a1dca3d4b577134
Reviewed-on: https://go-review.googlesource.com/9520
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Fix several warnings generated on the linux-amd64-clang builder
and make it clear to clang that -znow is a linker only flag.
Tested with
env CC=clang-3.5 ./all.bash
env CC=gcc-4.8 ./all.bash
Change-Id: I5ca7366ba8bf6221a36d25a2157dda4b4f3e16fa
Reviewed-on: https://go-review.googlesource.com/9523
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This extends the cgo changes in http://golang.org/cl/8094 to gccgo.
It also adds support for setting runtime_iscgo correctly for gccgo;
the gc runtime bases the variable on the runtime/cgo package, but
gccgo has no equivalent to that package.
The go tool supports -buildmode=c-archive for gccgo by linking all the
Go objects together using -r. For convenience this object is then put
into an archive file.
The go tool now passes -fsplit-stack when building C code for gccgo on
386 and amd64. This is required for using -r and will also cut down
on unnecessary stack splits.
The go tool no longer applies standard package cgo LDFLAGS when using
gccgo. This is mainly to avoid getting confused by the LDFLAGS in the
runtime/cgo package that gccgo does not use.
Change-Id: I1d0865b2a362818a033ca9e9e901d0ce250784e7
Reviewed-on: https://go-review.googlesource.com/9511
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The current implementation of the tSpecialTagEnd function
is inefficient since it generates plenty of memory allocations
and converts the whole buffer to lowercase at each call.
If the number of special tags increases linearly with the
template size, the complexity becomes quadratic.
This CL provides an alternative implementation.
While the algorithm is probably still not optimal, it avoids
the quadratic behavior and the memory allocations.
benchmark old ns/op new ns/op delta
BenchmarkTemplateSpecialTags-4 19326431 532190 -97.25%
benchmark old allocs new allocs delta
BenchmarkTemplateSpecialTags-4 2650 190 -92.83%
benchmark old bytes new bytes delta
BenchmarkTemplateSpecialTags-4 4106460 46568 -98.87%
While we are there, make sure we respect the HTML tokenization algorithm.
An end tag needs to be followed by a space, tab, CR, FF, /, or > as described
in https://html.spec.whatwg.org/multipage/syntax.html#tokenization
Explicitly add this check.
Fixes#10605
Change-Id: Ia33ddee164ab608a69ac4183e16ec506bbeaa54c
Reviewed-on: https://go-review.googlesource.com/9502
Reviewed-by: Rob Pike <r@golang.org>
gcDumpObject is used to print the source and destination objects when
checkmark find a missing mark. However, gcDumpObject currently assumes
the given pointer will point to a heap object. This is not true of the
source object during root marking and may not even be true of the
destination object in the limited situations where the heap points
back in to the stack.
If the pointer isn't a heap object, gcDumpObject will attempt an
out-of-bounds access to h_spans. This will cause a panicslice, which
will attempt to construct a useful panic message. This will cause a
string allocation, which will lead mallocgc to panic because the GC is
in mark termination (checkmark only happens during mark termination).
Fix this by checking that the pointer points into the heap arena
before attempting to use it as an arena pointer.
Change-Id: I09da600c380d4773f1f8f38e45b82cb229ea6382
Reviewed-on: https://go-review.googlesource.com/9498
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently the packages have the following index functions:
func Index(s, sep []byte) int
func IndexAny(s []byte, chars string) int
func IndexByte(s []byte, c byte) int
func IndexFunc(s []byte, f func(r rune) bool) int
func IndexRune(s []byte, r rune) int
func LastIndex(s, sep []byte) int
func LastIndexAny(s []byte, chars string) int
func LastIndexFunc(s []byte, f func(r rune) bool) int
Searching for the last occurrence of a byte is quite common
for string parsing algorithms (e.g. find the last paren on a line).
Also addition of LastIndexByte makes the set more orthogonal.
Change-Id: Ida168849acacf8e78dd70c1354bef9eac5effafe
Reviewed-on: https://go-review.googlesource.com/9500
Reviewed-by: Rob Pike <r@golang.org>
This CL copies golang.org/x/sys/windows/registry into
internal/syscall/windows/registry (minus KeyInfo.ModTime to prevent
dependency cycles). New registry package is used in mime and time
packages instead of calling Windows API directly.
Change-Id: I965a5a41d4739b3ba38e539a7b8d96d3223e3d56
Reviewed-on: https://go-review.googlesource.com/9271
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This change adds Int.ModSqrt to compute modular square-roots via the
standard Tonelli-Shanks algorithm, and the Jacobi function that this and
many other modular-arithmetic algorithms depend on.
This is needed by change 1883 (https://golang.org/cl/1883), to add
support for ANSI-standard compressed encoding of elliptic curve points.
Change-Id: Icc4805001bba0b3cb7200e0b0a7f87b14a9e9439
Reviewed-on: https://go-review.googlesource.com/1886
Reviewed-by: Adam Langley <agl@golang.org>
The X.509 parser was allowing trailing data after a number of structures
in certificates and public keys. There's no obvious security issue here,
esp in certificates which are signed anyway, but this change makes
trailing data an error just in case.
Fixes#10583
Change-Id: Idc289914899600697fc6d30482227ff4bf479241
Reviewed-on: https://go-review.googlesource.com/9473
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
This is the second in a two-part change. See https://golang.org/cl/9415
for details of the overall change.
This change updates the supported signature algorithms to include
SHA-384 and updates all the testdata/ files accordingly. Even some of
the testdata/ files named “TLS1.0” and “TLS1.1” have been updated
because they have TLS 1.2 ClientHello's even though the server picks a
lower version.
Fixes#9757.
Change-Id: Ia76de2b548d3b39cd4aa3f71132b0da7c917debd
Reviewed-on: https://go-review.googlesource.com/9472
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Prior to TLS 1.2, the handshake had a pleasing property that one could
incrementally hash it and, from that, get the needed hashes for both
the CertificateVerify and Finished messages.
TLS 1.2 introduced negotiation for the signature and hash and it became
possible for the handshake hash to be, say, SHA-384, but for the
CertificateVerify to sign the handshake with SHA-1. The problem is that
one doesn't know in advance which hashes will be needed and thus the
handshake needs to be buffered.
Go ignored this, always kept a single handshake hash, and any signatures
over the handshake had to use that hash.
However, there are a set of servers that inspect the client's offered
signature hash functions and will abort the handshake if one of the
server's certificates is signed with a hash function outside of that
set. https://robertsspaceindustries.com/ is an example of such a server.
Clearly not a lot of thought happened when that server code was written,
but its out there and we have to deal with it.
This change decouples the handshake hash from the CertificateVerify
hash. This lays the groundwork for advertising support for SHA-384 but
doesn't actually make that change in the interests of reviewability.
Updating the advertised hash functions will cause changes in many of the
testdata/ files and some errors might get lost in the noise. This change
only needs to update four testdata/ files: one because a SHA-384-based
handshake is now being signed with SHA-256 and the others because the
TLS 1.2 CertificateRequest message now includes SHA-1.
This change also has the effect of adding support for
client-certificates in SSLv3 servers. However, SSLv3 is now disabled by
default so this should be moot.
It would be possible to avoid much of this change and just support
SHA-384 for the ServerKeyExchange as the SKX only signs over the nonces
and SKX params (a design mistake in TLS). However, that would leave Go
in the odd situation where it advertised support for SHA-384, but would
only use the handshake hash when signing client certificates. I fear
that'll just cause problems in the future.
Much of this code was written by davidben@ for the purposes of testing
BoringSSL.
Partly addresses #9757
Change-Id: I5137a472b6076812af387a5a69fc62c7373cd485
Reviewed-on: https://go-review.googlesource.com/9415
Run-TryBot: Adam Langley <agl@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
This exercises the linker as well as the compiler.
Credit to Matthew Dempsky; see #10418.
Change-Id: I793947c0c617a34e23df766bff5238ff3ac3c0af
Reviewed-on: https://go-review.googlesource.com/9530
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Just a first basic test, I'll extend this to test more but want to get an
opinion on basic approach first.
Change-Id: Idab9ebd7d9960b000b81a01a1e53258bf4bce755
Reviewed-on: https://go-review.googlesource.com/9386
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This change deflakes timeout, deadline tests, especially fixes socket
and goroutine leaks. Also adds a few missing tests that use features
introduced after go1 release.
Change-Id: Ibf73a4859f8d4a0ee494ca2fd180cbce72a7a2c7
Reviewed-on: https://go-review.googlesource.com/9464
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change merges unicast_posix_test.go and multicast_test.go into
listen_test.go before deflaking tests for Listen functions.
No code changes.
Change-Id: Ic4cd6531b95dfb5b6e6e254241692eca61a71e94
Reviewed-on: https://go-review.googlesource.com/9460
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change adds missing CloseRead test and Close tests on Conn,
Listener and PacketConn with various networks.
Change-Id: Iadf99eaf526a323f853d203edc7c8d0577f67972
Reviewed-on: https://go-review.googlesource.com/9469
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fixes an issue where Response.Write writes out a Content-Length: -1
header when the corresponding Request is a POST or PUT and the
ContentLength was not previously set.
This was encountered when using httputil.DumpResponse
to write out the response from a server that responded to a PUT
request with no Content-Length header. The dumped output is
thus invalid.
Change-Id: I52c6ae8ef3443f1f9de92aeee9f9581dabb05991
Reviewed-on: https://go-review.googlesource.com/9496
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Not only by network, transport-layer intermediaries but by
virtualization stuff in a node, it is hard to identify the root cause of
weird faults without information of packet flows after disaster
happened.
This change adds Source field to OpError to be able to represent a
5-tuple of internet transport protocols for helping dealing with
complicated systems.
Also clarifies the usage of Source and Addr fields.
Updates #4856.
Change-Id: I96a523fe391ed14406bfb21604c461d4aac2fa19
Reviewed-on: https://go-review.googlesource.com/9231
Reviewed-by: Ian Lance Taylor <iant@golang.org>
RFC 1035 3.3.14 allows a TXT record to contain one or more <character-string>s.
The current implementation returns a "no such host" error if there is more
than one <character-string> in the TXT record.
Fixes#10482
Change-Id: I0ded258005e6b7ba45f687fecd10afa2b321bb77
Reviewed-on: https://go-review.googlesource.com/8966
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
The TestAfterQueueing test is inherently flaky because it relies on
independent kernel threads being scheduled within the "delta" duration
of each other. Normally, delta is 100ms but during "short" testing,
it's reduced to 20ms.
On at least OpenBSD, the CPU scheduler operates in 10ms time slices,
so high system load (e.g., from running multiple Go unit tests in
parallel, as happens during all.bash) can occasionally cause >20ms
scheduling delays and result in test flaking. This manifests as issue
9903, which is the currently the most common OpenBSD flake.
To mitigate this delay, only reduce the delta duration to 20ms for the
first attempt during short testing. If this fails and the test is
reattempted, subsequent attempts instead use a full 100ms delta.
Fixes#9903.
Change-Id: I11bdfa939e5be915f67ffad8a8aef6ed8772159a
Reviewed-on: https://go-review.googlesource.com/9510
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Sequence of operations:
- Go code does a systemstack call
- during the systemstack call, receive a signal
- signal requests a traceback of all goroutines
The orignal G is still marked as _Grunning, so the traceback code
refuses to print its stack.
Fix by allowing traceback of Gs whose caller is on the same M as G is.
G can't be modifying its stack if that is the case.
Fixes#10546
Change-Id: I2bcea48c0197fbf78ab6fa080027cd80181083ad
Reviewed-on: https://go-review.googlesource.com/9435
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The problem is not actually specific to android/arm. Linux/ARM's
runtime.clone set the stack pointer to child_stk-4 before calling
the fn. And then when fn returns, it tries to write to 4(R13) to
provide argument for runtime.exit, which is just beyond the allocated
child stack, and thus it will corrupt the heap randomly or trigger
segfault if that memory happens to be unmapped.
While we're at here, shorten the test polling interval to 0.1s to
speed up the test (it was only checking at 1s interval, which means
the test takes at least 1s).
Fixes#10548.
Change-Id: I57cd63232022b113b6cd61e987b0684ebcce930a
Reviewed-on: https://go-review.googlesource.com/9457
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Print it out much like godoc so there isn't a single block of text.
Print the symbol before its comment and indent the comment so
individual symbols separate visually.
Buffer the output.
Add a -c option to force case-sensitive matching.
Allow two arguments, like godoc, to help disambiguate cases
where path and symbol may be confused.
Improve the documentation printed by go help doc.
Change-Id: If687aad04bbacdf7dbe4bf7636de9fe96f756fd0
Reviewed-on: https://go-review.googlesource.com/9471
Reviewed-by: Russ Cox <rsc@golang.org>
The heap statistics were only written if asked for a profile with debug > 0,
but that also prints a stack trace for each profile line, which is comparatively
much noisier. The statistics are short enough and separate enough
(they only appear at the end) and useful enough that we can print them
always.
This means that people using -test.memprofile in tests will get a memory
profile with statistics included now. Pprof won't care, but if people care to
look, the numbers will be there.
This avoids the need for hacks like using -memprofilerate=1 to find
the number of allocations.
Change-Id: I10a4f593403d0315aad11b37c6e554b734caa73f
Reviewed-on: https://go-review.googlesource.com/9491
Reviewed-by: David Chase <drchase@google.com>
Removes the unused *bufio.Reader from the object controlling the
linker's primary output.
Change-Id: If91d9f60752f3dc4b280f35d6eb441f3c47574b2
Reviewed-on: https://go-review.googlesource.com/9362
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Disable disassembly with external linking test on openbsd/arm, since this
platform does not currently support cgo/external linking.
Change-Id: I6eab6fcaac21407ce05075a4a1407fbfe0e6142b
Reviewed-on: https://go-review.googlesource.com/9481
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When reading the object files for linking, liblink takes care of
generate the data for them.
This is a port of https://golang.org/cl/3101 to Go.
Change-Id: Ie3e2d6515bd7d253a8c1e25c70ef8fed064436d8
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/8383
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There's no need to call/ret to the body implementation.
It can write the result to the right place. Just jump to
it and have it return to our caller.
Old:
call body implementation
compute result
put result in a register
return
write register to result location
return
New:
load address of result location into a register
jump to body implementation
compute result
write result to passed-in address
return
It's a bit tricky on 386 because there is no free register
with which to pass the result location. Free up a register
by keeping around blen-alen instead of both alen and blen.
Change-Id: If2cf0682a5bf1cc592bdda7c126ed4eee8944fba
Reviewed-on: https://go-review.googlesource.com/9202
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Gdb is not able to backtrace our non-standard stack frames on RISC
architectures without frame pointer.
Change-Id: Id62a566ce2d743602ded2da22ff77b9ae34bc5ae
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/9456
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
GIF's bounds.
Also change the implicit Config Width and Height to be the
Rectangle.Max, not the Dx and Dy, of the first frame's bounds. For the
case where the first frame's bounds is something like (5,5)-(8,8), the
overall width should be 8, not 3.
Change-Id: I3affc484f5e32941a36f15517a92ca8d189d9c22
Reviewed-on: https://go-review.googlesource.com/9465
Reviewed-by: Rob Pike <r@golang.org>
Also, please be informed that the Write method on both connected and
unconnected-mode sockets may return a positive number of bytes written
with timeout or use of closed network connection error.
Change-Id: I2e2e6192e29cef4e9389eb0422c605c6d12e6a3c
Reviewed-on: https://go-review.googlesource.com/9466
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This started out as trying to remove Bool2int calls, which it does a bit, but
mostly it ended up being removing the Link.Symmorestack array which seemed a
pointless bit of caching.
Change-Id: I91a51eb08cb4b08f3f9f093b575306499267b67a
Reviewed-on: https://go-review.googlesource.com/9239
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
With 128KB stack reservation, on 32-bit Windows, the maximum number
threads is ~9000.
The original 65535-byte stack commit is causing problem on Windows
XP where it makes the stack reservation to be 1MB despite the fact
that the runtime specified 128KB.
While we're at here, also fix the extra spacings in the unable to
create more OS thread error message: println will insert a space
between each argument.
See #9457 for more information.
Change-Id: I3a82f7d9717d3d55211b6eb1c34b00b0eaad83ed
Reviewed-on: https://go-review.googlesource.com/2237
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Minux Ma <minux@golang.org>
In the followup changes, tests that require external facilities such as
DNS servers and RRs will move into external_test.go.
Change-Id: Ib460b0c51961159830357652dbf5430e1ba01514
Reviewed-on: https://go-review.googlesource.com/9461
Reviewed-by: Dave Cheney <dave@cheney.net>
They have to read the boolean into a register first and then do
the comparison.
Fixes#10598.
Change-Id: I2b808837a8c6393e1e0778296b6592aaab2b04bf
Signed-off-by: Shenghou Ma <minux@golang.org>
Reviewed-on: https://go-review.googlesource.com/9453
Reviewed-by: Dave Cheney <dave@cheney.net>
To avoid confusion with the runtime concept of copying stack.
Change-Id: I33442377b71012c2482c2d0ddd561492c71e70d0
Reviewed-on: https://go-review.googlesource.com/8639
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Russ Cox <rsc@golang.org>
The previous CL implemented decoding, but not encoding.
Also return the global color map (if present) for DecodeConfig.
Change-Id: I3b99c93720246010c9fe0924dc40a67875dfc852
Reviewed-on: https://go-review.googlesource.com/9389
Reviewed-by: Rob Pike <r@golang.org>
Technically you must initialize static pthread_mutex_t and
pthread_cond_t variables with the appropriate INITIALIZER macro. In
practice the default initializers are zero anyhow, but it's still good
code hygiene.
Change-Id: I517304b16c2c7943b3880855c1b47a9a506b4bdf
Reviewed-on: https://go-review.googlesource.com/9433
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Fixes#10366 (how to set custom headers)
Fixes#9836 (PATCH in PostForm)
Fixes#9276 (generating a server-side Request for testing)
Update #8991 (clarify Response.Write for now; export ReverseProxy's copy later?)
Change-Id: I95a11bf3bb3eeeeb72775b6ebfbc761641addc35
Reviewed-on: https://go-review.googlesource.com/9410
Reviewed-by: David Crawshaw <crawshaw@golang.org>
These were lost in the transition from 8a/6a to asm.
Also, in the process, discover more aliases. I'm betting the missing
ones were a casualty of the recent merge of 386 and amd64.
Update #10385.
Change-Id: I1681034b25af3ffc103f75e5fc57baca5feb3fcd
Reviewed-on: https://go-review.googlesource.com/9431
Reviewed-by: Russ Cox <rsc@golang.org>
This change corrects the serialization of asn1.Flag values, so that
when set, they serialize to an empty value, and when unset, they are
omitted. It also adds a format parameter that allows calling code
to control whether time.Time values are serialized as UTCTime or
GeneralizedTime.
Change-Id: I6d97abf009ea317338dab30c80f35a2de7e07104
Reviewed-on: https://go-review.googlesource.com/5970
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
Previously, unknown critical extensions were a parse error. However, for
some cases one wishes to parse and use a certificate that may contain
these extensions. For example, when using a certificate in a TLS server:
it's the client's concern whether it understands the critical extensions
but the server still wishes to parse SNI values out of the certificate
etc.
This change moves the rejection of unknown critical extensions from
ParseCertificate to Certificate.Verify. The former will now record the
OIDs of unknown critical extensions in the Certificate and the latter
will fail to verify certificates with them. If a user of this package
wishes to handle any unknown critical extensions themselves, they can
extract the extensions from Certificate.Extensions, process them and
remove known OIDs from Certificate.UnknownCriticalExtensions.
See discussion at
https://groups.google.com/forum/#!msg/golang-nuts/IrzoZlwalTQ/qdK1k-ogeHIJ
and in the linked bug.
Fixes#10459
Change-Id: I762521a44c01160fa0901f990ba2f5d4977d7977
Reviewed-on: https://go-review.googlesource.com/9390
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Add the new go doc command to the go command, installed in
the tool directory.
(Still to do: tests)
Fix cmd/dist to remove old "package documentation" code that was
stopping it from including cmd/go/doc.go in the build.
Implement the doc command. Here is the help info from "go help doc":
===
usage: go doc [-u] [package|[package.]symbol[.method]]
Doc accepts at most one argument, indicating either a package, a symbol within a
package, or a method of a symbol.
go doc
go doc <pkg>
go doc <sym>[.<method>]
go doc [<pkg>].<sym>[.<method>]
Doc interprets the argument to see what it represents, determined by its syntax
and which packages and symbols are present in the source directories of GOROOT and
GOPATH.
The first item in this list that succeeds is the one whose documentation is printed.
For packages, the order of scanning is determined by the file system, however the
GOROOT tree is always scanned before GOPATH.
If there is no package specified or matched, the package in the current directory
is selected, so "go doc" shows the documentation for the current package and
"go doc Foo" shows the documentation for symbol Foo in the current package.
Doc prints the documentation comments associated with the top-level item the
argument identifies (package, type, method) followed by a one-line summary of each
of the first-level items "under" that item (package-level declarations for a
package, methods for a type, etc.)
The package paths must be either a qualified path or a proper suffix of a path
(see examples below). The go tool's usual package mechanism does not apply: package
path elements like . and ... are not implemented by go doc.
When matching symbols, lower-case letters match either case but upper-case letters
match exactly.
Examples:
go doc
Show documentation for current package.
go doc Foo
Show documentation for Foo in the current package.
(Foo starts with a capital letter so it cannot match a package path.)
go doc json
Show documentation for the encoding/json package.
go doc json
Shorthand for encoding/json assuming only one json package
is present in the tree.
go doc json.Number (or go doc json.number)
Show documentation and method summary for json.Number.
go doc json.Number.Int64 (or go doc json.number.int64)
Show documentation for the Int64 method of json.Number.
Flags:
-u
Show documentation for unexported as well as exported
symbols and methods.
===
Still to do:
Tests.
Disambiguation when there is both foo and Foo.
Flag for case-sensitive matching.
Change-Id: I83d409a68688a5445f54297a7e7c745f749b9e66
Reviewed-on: https://go-review.googlesource.com/9227
Reviewed-by: Russ Cox <rsc@golang.org>
Some race tests were sensitive to the goroutine scheduling order.
When this changed in commit e870f06, these tests started to fail.
Fix TestRaceHeapParam by ensuring that the racing goroutine has
run before the test exits. Fix TestRaceRWMutexMultipleReaders by
adding a third reader to ensure that two readers wind up on the
same side of the writer (and race with each other) regardless of
the schedule. Fix TestRaceRange by ensuring that the racing
goroutine runs before the main goroutine exits the loop it races
with.
Change-Id: Iaf002f8730ea42227feaf2f3c51b9a1e57ccffdd
Reviewed-on: https://go-review.googlesource.com/9402
Reviewed-by: Russ Cox <rsc@golang.org>
This makes the OS X firewall box pop up.
Not run during all.bash so hasn't been noticed before.
Change-Id: I78feb4fd3e1d3c983ae3419085048831c04de3da
Reviewed-on: https://go-review.googlesource.com/9401
Reviewed-by: Austin Clements <austin@google.com>
ReadMemStats accounts for stacks slightly differently than the runtime
does internally. Internally, only stacks allocated by newosproc0 are
accounted in memstats.stacks_sys and other stacks are accounted in
heap_sys. readmemstats_m shuffles the statistics so all stacks are
accounted in StackSys rather than HeapSys.
However, currently, readmemstats_m assumes StackSys will be zero when
it does this shuffle. This was true until commit 6ad33be. If it isn't
(e.g., if something called newosproc0), StackSys+HeapSys will be
different before and after this shuffle, and the Sys sum that was
computed earlier will no longer agree with the sum of its components.
Fix this by making the shuffle in readmemstats_m not assume that
StackSys is zero.
Fixes#10585.
Change-Id: If13991c8de68bd7b85e1b613d3f12b4fd6fd5813
Reviewed-on: https://go-review.googlesource.com/9366
Reviewed-by: Russ Cox <rsc@golang.org>
I introduced this build failure in golang.org/cl/9302 but failed to
notice due to the other failures on the dashboard.
Change-Id: I84bf00f664ba572c1ca722e0136d8a2cf21613ca
Reviewed-on: https://go-review.googlesource.com/9363
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Also call runtime.GC before exit to ensure
that the profiler picks up all allocations.
Fixes#10537.
Change-Id: Ibfbfc88652ac0ce30a6d1ae392f919df6c1e8126
Reviewed-on: https://go-review.googlesource.com/9261
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Currently TestRaceCrawl fails to wg.Done for every wg.Adds if the
depth ever reaches 0. This causes the test to deadlock. Under the race
detector, this deadlock is not detected, so the test eventually times
out.
This only recently became a problem. Prior to commit e870f06 the depth
would never reach 0 because the strict round-robin goroutine schedule
ensured that all of the URLs were already "seen" by depth 2. Now that
the runtime prefers scheduling the most recently started goroutine,
the test is able to reach depth 0 and trigger this deadlock.
Change-Id: I5176302a89614a344c84d587073b364833af6590
Reviewed-on: https://go-review.googlesource.com/9344
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Replaced code.google.com/p/re2/ with github.com/google/re2/ and
updated the file names (re2-exhaustive.txt.bz2 not re2.txt.gz)
as well as the re2 make command (make log).
Change-Id: I15937b0b8a898d78d45366857ed86421c8d69960
Reviewed-on: https://go-review.googlesource.com/9372
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The master goroutine was returning before
the child goroutine had done its final i < b.N
(the one that fails and causes it to exit the loop)
and then the benchmark harness was updating
b.N, causing a read+write race on b.N.
Change-Id: I2504270a0de30544736f6c32161337a25b505c3e
Reviewed-on: https://go-review.googlesource.com/9368
Reviewed-by: Austin Clements <austin@google.com>
This is a follow-up to CL 9269, as suggested
by dvyukov.
There is probably even more that can be done
to speed up this shuffle. It will matter more
once CL 7570 (fine-grained locking in select)
is in and can be revisited then, with benchmarks.
Change-Id: Ic13a27d11cedd1e1f007951214b3bb56b1644f02
Reviewed-on: https://go-review.googlesource.com/9393
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
This avoids confusion with the main findrunnable in the scheduler.
Change-Id: I8cf40657557a8610a2fe5a2f74598518256ca7f0
Reviewed-on: https://go-review.googlesource.com/9305
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, we use a full stop-the-world around enabling write
barriers. This is to ensure that all Gs have enabled write barriers
before any blackening occurs (either in gcBgMarkWorker() or in
gcAssistAlloc()).
However, there's no need to bring the whole world to a synchronous
stop to ensure this. This change replaces the STW with a ragged
barrier that ensures each P has individually observed that write
barriers should be enabled before GC performs any blackening.
Change-Id: If2f129a6a55bd8bdd4308067af2b739f3fb41955
Reviewed-on: https://go-review.googlesource.com/8207
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This adds forEachP, which performs a general-purpose ragged global
barrier. forEachP takes a callback and invokes it for every P at a GC
safe point.
Ps that are idle or in a syscall are considered to be at a continuous
safe point. forEachP ensures that these Ps do not change state by
forcing all syscall Ps into idle and holding the sched.lock.
To ensure that Ps do not enter syscall or idle without running the
safe-point function, this adds checks for a pending callback every
place there is currently a gcwaiting check.
We'll use forEachP to replace the STW around enabling the write
barrier and to replace the current asynchronous per-M wbuf cache with
a cooperatively managed per-P gcWork cache.
Change-Id: Ie944f8ce1fead7c79bf271d2f42fcd61a41bb3cc
Reviewed-on: https://go-review.googlesource.com/8206
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Instead of running:
go test -short runtime -cpu=1
go test -short runtime -cpu=2
go test -short runtime -cpu=4
Run just:
go test -short runtime -cpu=1,2,4
This is a return to the Go 1.4.2 behavior.
We lose incremental display of progress and
per-cpu timing information, but we don't have
to recompile and relink the runtime test,
which is slow.
This cuts about 10s off all.bash.
Updates #10571.
Change-Id: I6e8c7149780d47439f8bcfa888e6efc84290c60a
Reviewed-on: https://go-review.googlesource.com/9350
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reduces allocs linking cmd/go and runtime.test
by ~13%. No functional changes.
The most easily addressed sources of allocations
after this are expandpkg, rdstring, and symbuf
string conversion.
These can be reduced by interning strings,
but that increases the overall memory footprint.
Change-Id: Ifedefc9f2a0403bcc75460d6b139e8408374e058
Reviewed-on: https://go-review.googlesource.com/9391
Reviewed-by: David Crawshaw <crawshaw@golang.org>
There is no need to escape newlines in char data -
it makes the XML larger and harder to read.
Change-Id: I1c1fcee1bdffc705c7428f89ca90af8085d6fb73
Reviewed-on: https://go-review.googlesource.com/9310
Reviewed-by: Nigel Tao <nigeltao@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This fixes a bug where the runtime ready()s a goroutine while setting
up a new M that's initially marked as spinning, causing the scheduler
to later panic when it finds work in the run queue of a P associated
with a spinning M. Specifically, the sequence of events that can lead
to this is:
1) sysmon calls handoffp to hand off a P stolen from a syscall.
2) handoffp sees no pending work on the P, so it calls startm with
spinning set.
3) startm calls newm, which in turn calls allocm to allocate a new M.
4) allocm "borrows" the P we're handing off in order to do allocation
and performs this allocation.
5) This allocation may assist the garbage collector, and this assist
may detect the end of concurrent mark and ready() the main GC
goroutine to signal this.
6) This ready()ing puts the GC goroutine on the run queue of the
borrowed P.
7) newm starts the OS thread, which runs mstart and subsequently
mstart1, which marks the M spinning because startm was called with
spinning set.
8) mstart1 enters the scheduler, which panics because there's work on
the run queue, but the M is marked spinning.
To fix this, before marking the M spinning in step 7, add a check to
see if work was been added to the P's run queue. If this is the case,
undo the spinning instead.
Fixes#10573.
Change-Id: I4670495ae00582144a55ce88c45ae71de597cfa5
Reviewed-on: https://go-review.googlesource.com/9332
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
This adds a check that we never put a P on the idle list when it has
work on its local run queue.
Change-Id: Ifcfab750de60c335148a7f513d4eef17be03b6a7
Reviewed-on: https://go-review.googlesource.com/9324
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
This is the optimization made to math/rand in CL 21030043.
Change-Id: I231b24fa77cac1fe74ba887db76313b5efaab3e8
Reviewed-on: https://go-review.googlesource.com/9269
Reviewed-by: Minux Ma <minux@golang.org>
This change causes the GetCertificate callback to be called if
Certificates is empty. Previously this configuration would result in an
error.
This allows people to have servers that depend entirely on dynamic
certificate selection, even when the client doesn't send SNI.
Fixes#9208.
Change-Id: I2f5a5551215958b88b154c64a114590300dfc461
Reviewed-on: https://go-review.googlesource.com/8792
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
The OCSP response is currently only exposed via a method on Conn,
which makes it inaccessible when using wrappers like net/http. The
ConnectionState structure is typically available even when using
wrappers and contains many of the other handshake details, so this
change exposes the stapled OCSP response in that structure.
Change-Id: If8dab49292566912c615d816321b4353e711f71f
Reviewed-on: https://go-review.googlesource.com/9361
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
At present, Unmarshal does not check that the point it unmarshals
is actually *on* the curve. (It may be on the curve's twist.)
This can, as Daniel Bernstein has pointed out at great length,
lead to quite devastating attacks. And 3 out of the 4 curves
supported by crypto/elliptic have twists with cofactor != 1;
P-224, in particular, has a sufficiently large cofactor that it
is likely that conventional dlog attacks might be useful.
This closes#2445, filed by Watson Ladd.
To explain why this was (partially) rejected before being accepted:
In the general case, for curves with cofactor != 1, verifying subgroup
membership is required. (This is expensive and hard-to-implement.)
But, as recent discussion during the CFRG standardization process
has brought out, small-subgroup attacks are much less damaging than
a twist attack.
Change-Id: I284042eb9954ff9b7cde80b8b693b1d468c7e1e8
Reviewed-on: https://go-review.googlesource.com/2421
Reviewed-by: Adam Langley <agl@golang.org>
This implements a method for x509.CertificateRequest to prevent
certain attacks and to allow a CA/RA to properly check the validity
of the binding between an end entity and a key pair, to prove that
it has possession of (i.e., is able to use) the private key
corresponding to the public key for which a certificate is requested.
RFC 2986 section 3 states:
"A certification authority fulfills the request by authenticating the
requesting entity and verifying the entity's signature, and, if the
request is valid, constructing an X.509 certificate from the
distinguished name and public key, the issuer name, and the
certification authority's choice of serial number, validity period,
and signature algorithm."
Change-Id: I37795c3b1dfdfdd455d870e499b63885eb9bda4f
Reviewed-on: https://go-review.googlesource.com/7371
Reviewed-by: Adam Langley <agl@golang.org>
This change adds a new method to tls.Config, SetSessionTicketKeys, that
changes the key used to encrypt session tickets while the server is
running. Additional keys may be provided that will be used to maintain
continuity while rotating keys. If a ticket encrypted with an old key is
provided by the client, the server will resume the session and provide
the client with a ticket encrypted using the new key.
Fixes#9994
Change-Id: Idbc16b10ff39616109a51ed39a6fa208faad5b4e
Reviewed-on: https://go-review.googlesource.com/9072
Reviewed-by: Jonathan Rudenberg <jonathan@titanous.com>
Reviewed-by: Adam Langley <agl@golang.org>
This change adds support for serving and receiving Signed Certificate
Timestamps as described in RFC 6962.
The server is now capable of serving SCTs listed in the Certificate
structure. The client now asks for SCTs and, if any are received,
they are exposed in the ConnectionState structure.
Fixes#10201
Change-Id: Ib3adae98cb4f173bc85cec04d2bdd3aa0fec70bb
Reviewed-on: https://go-review.googlesource.com/8988
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
Reviewed-by: Jonathan Rudenberg <jonathan@titanous.com>
Currently parseRecord will always start with a nil
slice and then resize the slice on append. For input
with a fixed number of fields per record we can preallocate
the slice to avoid having to resize the slice.
This change implements this optimization by using
FieldsPerRecord as capacity if it's > 0 and also adds a
benchmark to better show the differences.
benchmark old ns/op new ns/op delta
BenchmarkRead 19741 17909 -9.28%
benchmark old allocs new allocs delta
BenchmarkRead 59 41 -30.51%
benchmark old bytes new bytes delta
BenchmarkRead 6276 5844 -6.88%
Change-Id: I7c2abc9c80a23571369bcfcc99a8ffc474eae7ab
Reviewed-on: https://go-review.googlesource.com/8880
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Follows the linux signal forwarding semantics from
http://golang.org/cl/8712, sharing the implementation of sigfwdgo.
Forwarding for 386, arm, and arm64 will follow.
Change-Id: I6bf30d563d19da39b6aec6900c7fe12d82ed4f62
Reviewed-on: https://go-review.googlesource.com/9302
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit is largely cosmetic in the sense that it is the remnants
of a change proposal I had prepared for testing/quick, until I
discovered that 3e9ed27 already implemented the feature I was looking
for: quick.Value() for reflect.Kind Array. What you see is a merger
and manual cleanup; the cosmetic cleanups are as follows:
(1.) Keeping the TestCheckEqual and its associated input functions
in the same order as type kinds defined in reflect.Kind. Since
3e9ed27 was committed, the test case began to diverge from the
constant's ordering.
(2.) The `Intptr` derivatives existed to exercise quick.Value with
reflect.Kind's `Ptr` constant. All `Intptr` (unrelated to `uintptr`)
in the test have been migrated to ensure the parallelism of the
listings and to convey that `Intptr` is not special.
(3.) Correct a misspelling (transposition) of "alias", whereby it is
named as "Alais".
Change-Id: I441450db16b8bb1272c52b0abcda3794dcd0599d
Reviewed-on: https://go-review.googlesource.com/8804
Reviewed-by: Russ Cox <rsc@golang.org>
The long comment block in obj6.go:progedit talked about the two code sequences
for accessing g as "local exec" and "initial exec", but really they are both forms
of local exec. This stuff is confusing enough without using the wrong words for
things, so rewrite it to talk about 2-instruction and 1-instruction sequences.
Unfortunately the confusion has made it into code, with the R_TLS_IE relocation
now doing double duty as meaning actual initial exec when externally linking and
boring old local exec when linking internally (half of this is my fault). So this
stops using R_TLS_IE in the local exec case. There is a chance this might break
plan9 or windows, but I don't think so. Next step is working out what the heck is
going on on ARM...
Change-Id: I09da4388210cf49dbc99fd25f5172bbe517cee57
Reviewed-on: https://go-review.googlesource.com/9273
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
A previous change to mbitmap.go dropped a return on a
path the seems not to be excersized. This was a mistake that
this CL fixes.
Change-Id: I715ee4ef08f5bf8d9f53cee84e8fb31a237e2d43
Reviewed-on: https://go-review.googlesource.com/9295
Reviewed-by: Austin Clements <austin@google.com>
I think this should fix the arm build. A proper fix involves making the handling
of tlsg less fragile, I'll try that tomorrow.
Update #10557
Change-Id: I9b1b666737fb40aebb6f284748509afa8483cce5
Reviewed-on: https://go-review.googlesource.com/9272
Reviewed-by: Dave Cheney <dave@cheney.net>
Run-TryBot: Dave Cheney <dave@cheney.net>
Currently, each M has a cache of the most recently used *workbuf. This
is used primarily by the write barrier so it doesn't have to access
the global workbuf lists on every write barrier. It's also used by
stack scanning because it's convenient.
This cache is important for write barrier performance, but this
particular approach has several downsides. It's faster than no cache,
but far from optimal (as the benchmarks below show). It's complex:
access to the cache is sprinkled through most of the workbuf list
operations and it requires special care to transform into and back out
of the gcWork cache that's actually used for scanning and marking. It
requires atomic exchanges to take ownership of the cached workbuf and
to return it to the M's cache even though it's almost always used by
only the current M. Since it's per-M, flushing these caches is O(# of
Ms), which may be high. And it has some significant subtleties: for
example, in general the cache shouldn't be used after the
harvestwbufs() in mark termination because it could hide work from
mark termination, but stack scanning can happen after this and *will*
use the cache (but it turns out this is okay because it will always be
followed by a getfull(), which drains the cache).
This change replaces this cache with a per-P gcWork object. This
gcWork cache can be used directly by scanning and marking (as long as
preemption is disabled, which is a general requirement of gcWork).
Since it's per-P, it doesn't require synchronization, which simplifies
things and means the only atomic operations in the write barrier are
occasionally fetching new work buffers and setting a mark bit if the
object isn't already marked. This cache can be flushed in O(# of Ps),
which is generally small. It follows a simple flushing rule: the cache
can be used during any phase, but during mark termination it must be
flushed before allowing preemption. This also makes the dispose during
mutator assist no longer necessary, which eliminates the vast majority
of gcWork dispose calls and reduces contention on the global workbuf
lists. And it's a lot faster on some benchmarks:
benchmark old ns/op new ns/op delta
BenchmarkBinaryTree17 11963668673 11206112763 -6.33%
BenchmarkFannkuch11 2643217136 2649182499 +0.23%
BenchmarkFmtFprintfEmpty 70.4 70.2 -0.28%
BenchmarkFmtFprintfString 364 307 -15.66%
BenchmarkFmtFprintfInt 317 282 -11.04%
BenchmarkFmtFprintfIntInt 512 483 -5.66%
BenchmarkFmtFprintfPrefixedInt 404 380 -5.94%
BenchmarkFmtFprintfFloat 521 479 -8.06%
BenchmarkFmtManyArgs 2164 1894 -12.48%
BenchmarkGobDecode 30366146 22429593 -26.14%
BenchmarkGobEncode 29867472 26663152 -10.73%
BenchmarkGzip 391236616 396779490 +1.42%
BenchmarkGunzip 96639491 96297024 -0.35%
BenchmarkHTTPClientServer 100110 70763 -29.31%
BenchmarkJSONEncode 51866051 52511382 +1.24%
BenchmarkJSONDecode 103813138 86094963 -17.07%
BenchmarkMandelbrot200 4121834 4120886 -0.02%
BenchmarkGoParse 16472789 5879949 -64.31%
BenchmarkRegexpMatchEasy0_32 140 140 +0.00%
BenchmarkRegexpMatchEasy0_1K 394 394 +0.00%
BenchmarkRegexpMatchEasy1_32 120 120 +0.00%
BenchmarkRegexpMatchEasy1_1K 621 614 -1.13%
BenchmarkRegexpMatchMedium_32 209 202 -3.35%
BenchmarkRegexpMatchMedium_1K 54889 55175 +0.52%
BenchmarkRegexpMatchHard_32 2682 2675 -0.26%
BenchmarkRegexpMatchHard_1K 79383 79524 +0.18%
BenchmarkRevcomp 584116718 584595320 +0.08%
BenchmarkTemplate 125400565 109620196 -12.58%
BenchmarkTimeParse 386 387 +0.26%
BenchmarkTimeFormat 580 447 -22.93%
(Best out of 10 runs. The delta of averages is similar.)
This also puts us in a good position to flush these caches when
nearing the end of concurrent marking, which will let us increase the
size of the work buffers while still controlling mark termination
pause time.
Change-Id: I2dd94c8517a19297a98ec280203cccaa58792522
Reviewed-on: https://go-review.googlesource.com/9178
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
When findRunnable considers running a fractional mark worker, it first
checks if there's any work to be done; if there isn't there's no point
in running the worker because it will just reschedule immediately.
However, currently findRunnable just checks work.full and
work.partial, whereas getfull can *also* draw work from m.currentwbuf.
As a result, findRunnable may not start a worker even though there
actually is work.
This problem manifests itself in occasional failures of the
test/init1.go test. This test is unusual because it performs a large
amount of allocation without executing any write barriers, which means
there's nothing to force the pointers in currentwbuf out to the
work.partial/full lists where findRunnable can see them.
This change fixes this problem by making findRunnable also check for a
currentwbuf. This aligns findRunnable with trygetfull's notion of
whether or not there's work.
Change-Id: Ic76d22b7b5d040bc4f58a6b5975e9217650e66c4
Reviewed-on: https://go-review.googlesource.com/9299
Reviewed-by: Russ Cox <rsc@golang.org>
Currently, findRunnable only considers running a mark worker if
there's work in the work queue. In principle, this can delay the start
of the desired number of dedicated mark workers if there's no work
pending. This is unlikely to occur in practice, since there should be
work queued from the scan phase, but if it were to come up, a CPU hog
mutator could slow down or delay garbage collection.
This check makes sense for fractional mark workers, since they'll just
return to the scheduler immediately if there's no work, but we want
the scheduler to start all of the dedicated mark workers promptly,
even if there's currently no queued work. Hence, this change moves the
pending work check after the check for starting a dedicated worker.
Change-Id: I52b851cc9e41f508a0955b3f905ca80f109ea101
Reviewed-on: https://go-review.googlesource.com/9298
Reviewed-by: Rick Hudson <rlh@golang.org>
Also rename and update mkdoc.sh to mkalldocs.sh
Change-Id: Ief3673c22d45624e173fc65ee279cea324da03b5
Reviewed-on: https://go-review.googlesource.com/9226
Reviewed-by: Russ Cox <rsc@golang.org>
The motivation is that sysAlloc/Free() currently aren't safe to be
called without a valid G, because arm's xadd64() uses locks that require
a valid G.
The solution here was proposed by Dmitry Vyukov: use xadduintptr()
instead of xadd64(), until arm can support xadd64 on all of its
architectures (not a trivial task for arm).
Change-Id: I250252079357ea2e4360e1235958b1c22051498f
Reviewed-on: https://go-review.googlesource.com/9002
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
- main3.c tests main.main is exported when compiled for GOOS=android.
- wait longer for main2.c (it's slow on android/arm)
- rearranged test.bash
Fixes#10070.
Change-Id: I6e5a98d1c5fae776afa54ecb5da633b59b269316
Reviewed-on: https://go-review.googlesource.com/9296
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
This lets us avoid loading string constants via the GOT and (together with
http://golang.org/cl/9102) results in the fannkuch benchmark having very similar
register usage with -dynlink as without.
Change-Id: Ic3892b399074982b76773c3e547cfbba5dabb6f9
Reviewed-on: https://go-review.googlesource.com/9103
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Currently, when allocation reaches the GC trigger, the runtime uses
readyExecute to start the GC goroutine immediately rather than wait
for the scheduler to get around to the GC goroutine while the mutator
continues to grow the heap.
Now that the scheduler runs the most recently readied goroutine when a
goroutine yields its time slice, this rigmarole is no longer
necessary. The runtime can simply ready the GC goroutine and yield
from the readying goroutine.
Change-Id: I3b4ebadd2a72a923b1389f7598f82973dd5c8710
Reviewed-on: https://go-review.googlesource.com/9292
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
Currently, the main GC goroutine sleeps on a note during concurrent
mark and the first background mark worker or assist to finish marking
use wakes up that note to let the main goroutine proceed into mark
termination. Unfortunately, the latency of this wakeup can be quite
high, since the GC goroutine will typically have lost its P while in
the futex sleep, meaning it will be placed on the global run queue and
will wait there until some P is kind enough to pick it up. This delay
gives the mutator more time to allocate and create floating garbage,
growing the heap unnecessarily. Worse, it's likely that background
marking has stopped at this point (unless GOMAXPROCS>4), so anything
that's allocated and published to the heap during this window will
have to be scanned during mark termination while the world is stopped.
This change replaces the note sleep/wakeup with a gopark/ready
scheme. This keeps the wakeup inside the Go scheduler and lets the
garbage collector take advantage of the new scheduler semantics that
run the ready()d goroutine immediately when the ready()ing goroutine
sleeps.
For the json benchmark from x/benchmarks with GOMAXPROCS=4, this
reduces the delay in waking up the GC goroutine and entering mark
termination once concurrent marking is done from ~100ms to typically
<100µs.
Change-Id: Ib11f8b581b8914f2d68e0094f121e49bac3bb384
Reviewed-on: https://go-review.googlesource.com/9291
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Currently, we use a note sleep with a timeout in a loop in func gc to
periodically revise the GC control variables. Replace this with a
fully blocking note sleep and use a periodic timer to trigger the
revise instead. This is a step toward replacing the note sleep in func
gc.
Change-Id: I2d562f6b9b2e5f0c28e9a54227e2c0f8a2603f63
Reviewed-on: https://go-review.googlesource.com/9290
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Currently, when the runtime ready()s a G, it adds it to the end of the
current P's run queue and continues running. If there are many other
things in the run queue, this can result in a significant delay before
the ready()d G actually runs and can hurt fairness when other Gs in
the run queue are CPU hogs. For example, if there are three Gs sharing
a P, one of which is a CPU hog that never voluntarily gives up the P
and the other two of which are doing small amounts of work and
communicating back and forth on an unbuffered channel, the two
communicating Gs will get very little CPU time.
Change this so that when G1 ready()s G2 and then blocks, the scheduler
immediately hands off the remainder of G1's time slice to G2. In the
above example, the two communicating Gs will now act as a unit and
together get half of the CPU time, while the CPU hog gets the other
half of the CPU time.
This fixes the problem demonstrated by the ping-pong benchmark added
in the previous commit:
benchmark old ns/op new ns/op delta
BenchmarkPingPongHog 684287 825 -99.88%
On the x/benchmarks suite, this change improves the performance of
garbage by ~6% (for GOMAXPROCS=1 and 4), and json by 28% and 36% for
GOMAXPROCS=1 and 4. It has negligible effect on heap size.
This has no effect on the go1 benchmark suite since those benchmarks
are mostly single-threaded.
Change-Id: I858a08eaa78f702ea98a5fac99d28a4ac91d339f
Reviewed-on: https://go-review.googlesource.com/9289
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This benchmark demonstrates a current problem with the scheduler where
a set of frequently communicating goroutines get very little CPU time
in the presence of another goroutine that hogs that CPU, even if one
of those communicating goroutines is always runnable.
Currently it takes about 0.5 milliseconds to switch between
ping-ponging goroutines in the presence of a CPU hog:
BenchmarkPingPongHog 2000 684287 ns/op
Change-Id: I278848c84f778de32344921ae8a4a8056e4898b0
Reviewed-on: https://go-review.googlesource.com/9288
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
There are a variety of places where we check if a P's run queue is
empty. This test is about to get slightly more complicated, so factor
it out into a new function, runqempty. This function is inlinable, so
this has no effect on performance.
Change-Id: If4a0b01ffbd004937de90d8d686f6ded4aad2c6b
Reviewed-on: https://go-review.googlesource.com/9287
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
We can expand the test cases as we discover problems.
This is some basic tests plus all the things I got wrong
in some recent work.
Change-Id: Id875fcfaf74eb087ae42b441fe47a34c5b8ccb39
Reviewed-on: https://go-review.googlesource.com/9158
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Explicitly specify that we represent polynomial in reversed notation
Fixes#8229
Change-Id: Idf094c01fd82f133cd0c1b50fa967d12c577bdb5
Reviewed-on: https://go-review.googlesource.com/9237
Reviewed-by: David Chase <drchase@google.com>
Some machines are so slow that even with the default timeoutScale,
they still timeout some tests. For example, currently some linux/arm
builders and the openbsd/arm builder are timing out the runtime
test and CL 8397 was proposed to skip some tests on openbsd/arm
to fix the build.
Instead of increasing timeoutScale or skipping tests, this CL
introduces an environment variable $GO_TEST_TIMEOUT_SCALE that
could be set to manually set a larger timeoutScale for those
machines/builders.
Fixes#10314.
Change-Id: I16c9a9eb980d6a63309e4cacd79eee2fe05769ee
Reviewed-on: https://go-review.googlesource.com/9223
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Forward signals to signal handlers installed before Go installs its own,
under certain circumstances. In particular, as iant@ suggests, signals are
forwarded iff:
(1) a non-SIG_DFL signal handler existed before Go, and
(2) signal is synchronous (i.e., one of SIGSEGV, SIGBUS, SIGFPE), and
(3a) signal occured on a non-Go thread, or
(3b) signal occurred on a Go thread but in CGo code.
Supported only on Linux, for now.
Change-Id: I403219ee47b26cf65da819fb86cf1ec04d3e25f5
Reviewed-on: https://go-review.googlesource.com/8712
Reviewed-by: Ian Lance Taylor <iant@golang.org>
They are vestiges of the c2go transition.
Change-Id: I22672e40373ef77d7a0bf69cfff8017e46353055
Reviewed-on: https://go-review.googlesource.com/9265
Reviewed-by: Minux Ma <minux@golang.org>
This addresses iant's comments from CL 9164.
Change-Id: I7b5b282f61b11aab587402c2d302697e76666376
Reviewed-on: https://go-review.googlesource.com/9222
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, it's possible for the next_gc calculation to underflow.
Since next_gc is unsigned, this wraps around and effectively disables
GC for the rest of the program's execution. Besides being obviously
wrong, this is causing test failures on 32-bit because some tests are
running out of heap.
This underflow happens for two reasons, both having to do with how we
estimate the reachable heap size at the end of the GC cycle.
One reason is that this calculation depends on the value of heap_live
at the beginning of the GC cycle, but we currently only record that
value during a concurrent GC and not during a forced STW GC. Fix this
by moving the recorded value from gcController to work and recording
it on a common code path.
The other reason is that we use the amount of allocation during the GC
cycle as an approximation of the amount of floating garbage and
subtract it from the marked heap to estimate the reachable heap.
However, since this is only an approximation, it's possible for the
amount of allocation during the cycle to be *larger* than the marked
heap size (since the runtime allocates white and it's possible for
these allocations to never be made reachable from the heap). Currently
this causes wrap-around in our estimate of the reachable heap size,
which in turn causes wrap-around in next_gc. Fix this by bottoming out
the reachable heap estimate at 0, in which case we just fall back to
triggering GC at heapminimum (which is okay since this only happens on
small heaps).
Fixes#10555, fixes#10556, and fixes#10559.
Change-Id: Iad07b529c03772356fede2ae557732f13ebfdb63
Reviewed-on: https://go-review.googlesource.com/9286
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
To achieve a 2% improvement in the garbage benchmark this CL removes
an unneeded assert and avoids one hbits.next() call per object
being scanned.
Change-Id: Ibd542d01e9c23eace42228886f9edc488354df0d
Reviewed-on: https://go-review.googlesource.com/9244
Reviewed-by: Austin Clements <austin@google.com>
newosproc0 does not work on android/arm.
See issue #10548.
Change-Id: Ieaf6f5d0b77cddf5bf0b6c89fd12b1c1b8723f9b
Reviewed-on: https://go-review.googlesource.com/9293
Reviewed-by: David Crawshaw <crawshaw@golang.org>
If the machine's network configuration files (resolv.conf,
nsswitch.conf) don't have any unsupported options, prefer Go's DNS
resolver, which doesn't have the cgo & thread over.
It means users can have more than 500 DNS requests outstanding (our
current limit for cgo lookups) and not have one blocked thread per
outstanding request.
Discussed in thread https://groups.google.com/d/msg/golang-dev/2ZUi792oztM/Q0rg_DkF5HMJ
Change-Id: I3f685d70aff6b47bec30b63e9fba674b20507f95
Reviewed-on: https://go-review.googlesource.com/8945
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
They are vestiges of the c2go translation.
Change-Id: I9a10536f5986b751a35cc7d84b5ba69ae0c2ede7
Reviewed-on: https://go-review.googlesource.com/9262
Reviewed-by: Minux Ma <minux@golang.org>
There is an assumption that the function executed in child thread
created by runtime.close should not return. And different systems
enforce that differently: some exit that thread, some exit the
whole process.
The test TestNewOSProc0 introduced in CL 9161 breaks that assumption,
so we need to adjust the code to only exit the thread should the
called function return.
Change-Id: Id631cb2f02ec6fbd765508377a79f3f96c6a2ed6
Reviewed-on: https://go-review.googlesource.com/9246
Reviewed-by: Dave Cheney <dave@cheney.net>
It was only visible when you run godoc with explicit GOOS=windows,
which is less useful for people developing portable application on
non-windows platforms.
Also added a note that log/syslog is not supported on NaCl.
Change-Id: I81650445fb2a5ee161da7e0608c3d3547d5ac2a6
Reviewed-on: https://go-review.googlesource.com/9245
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The constants in cmd/internal/goobj had gone stale (we had three copies of
these constants, working on reducing that was what got me to noticing this).
Some of the changes to link.hello.darwin.amd64 are the change from absolute
to %rip-relative addressing, a change which happened quite a while ago...
Depends on http://golang.org/cl/9113.
Fixes#10501.
Change-Id: Iaa1511f458a32228c2df2ccd0076bb9ae212a035
Reviewed-on: https://go-review.googlesource.com/9105
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, we set the heap goal for the next GC cycle using the size
of the marked heap at the end of the current cycle. This can lead to a
bad feedback loop if the mutator is rapidly allocating and releasing
pointers that can significantly bloat heap size.
If the GC were STW, the marked heap size would be exactly the
reachable heap size (call it stwLive). However, in concurrent GC,
marked=stwLive+floatLive, where floatLive is the amount of "floating
garbage": objects that were reachable at some point during the cycle
and were marked, but which are no longer reachable by the end of the
cycle. If the GC cycle is short, then the mutator doesn't have much
time to create floating garbage, so marked≈stwLive. However, if the GC
cycle is long and the mutator is allocating and creating floating
garbage very rapidly, then it's possible that marked≫stwLive. Since
the runtime currently sets the heap goal based on marked, this will
cause it to set a high heap goal. This means that 1) the next GC cycle
will take longer because of the larger heap and 2) the assist ratio
will be low because of the large distance between the trigger and the
goal. The combination of these lets the mutator produce even more
floating garbage in the next cycle, which further exacerbates the
problem.
For example, on the garbage benchmark with GOMAXPROCS=1, this causes
the heap to grow to ~500MB and the garbage collector to retain upwards
of ~300MB of heap, while the true reachable heap size is ~32MB. This,
in turn, causes the GC cycle to take upwards of ~3 seconds.
Fix this bad feedback loop by estimating the true reachable heap size
(stwLive) and using this rather than the marked heap size
(stwLive+floatLive) as the basis for the GC trigger and heap goal.
This breaks the bad feedback loop and causes the mutator to assist
more, which decreases the rate at which it can create floating
garbage. On the same garbage benchmark, this reduces the maximum heap
size to ~73MB, the retained heap to ~40MB, and the duration of the GC
cycle to ~200ms.
Change-Id: I7712244c94240743b266f9eb720c03802799cdd1
Reviewed-on: https://go-review.googlesource.com/9177
Reviewed-by: Rick Hudson <rlh@golang.org>
Change-Id: I429402dd91243cd9415b054ee17bfebccc68ed57
Reviewed-on: https://go-review.googlesource.com/9197
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This may or may not be useful to the end user, but it's incredibly
useful for us to understand the behavior of the pacer. Currently this
is fairly easy (though not trivial) to derive from the other heap
stats we print, but we're about to change how we compute the goal,
which will make it much harder to derive.
Change-Id: I796ef233d470c01f606bd9929820c01ece1f585a
Reviewed-on: https://go-review.googlesource.com/9176
Reviewed-by: Rick Hudson <rlh@golang.org>
The trigger controller computes GC CPU utilization by dividing by the
wall-clock time that's passed since concurrent mark began. Since this
delta is nanoseconds it's borderline impossible for it to be zero, but
if it is zero we'll currently divide by zero. Be robust to this
possibility by ignoring the utilization in the error term if no time
has elapsed.
Change-Id: I93dfc9e84735682af3e637f6538d1e7602634f09
Reviewed-on: https://go-review.googlesource.com/9175
Reviewed-by: Rick Hudson <rlh@golang.org>
To make the gcprog for global data containing variables of types defined in other shared
libraries, we need to know a lot about those types. So read the value of any symbol with
a name starting with "type.". If a type uses a mask, the name of the symbol defining the
mask unfortunately cannot be predicted from the type name so I have to keep track of the
addresses of every such symbol and associate them with the type symbols after the fact.
I'm not very happy about this change, but something like this is needed and this is as
pleasant as I know how to make it.
Change-Id: I408d831b08b3b31e0610688c41367b23998e975c
Reviewed-on: https://go-review.googlesource.com/8334
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
There were 10 implementations of the trivial bool2int function, 9 of which
were the only thing in their file. Remove all of them in favor of one in
cmd/internal/obj.
Change-Id: I9c51d30716239df51186860b9842a5e9b27264d3
Reviewed-on: https://go-review.googlesource.com/9230
Reviewed-by: Ian Lance Taylor <iant@golang.org>
