Android O black-lists the select system call because its libc, Bionic,
does not use this system call. Replace our use of select with pselect6
(which is allowed) on the platforms that support targeting Android.
linux/arm64 already uses pselect6 because there is no select on arm64,
so only linux/amd64 and linux/arm need changing. pselect6 has been
available since Linux 2.6.16, which is before Go's minimum
requirement.
Fixes#20409.
Change-Id: Ic526b5b259a9e01d2f145a1f4d2e76e8c49ce809
Reviewed-on: https://go-review.googlesource.com/43641
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change makes {TCP,UDP,IP,Unix}Conn types compliant of
syscall.Conn interface and adds type rawConn as an implementation of
syscall.RawConn interface.
By this change, the long-standing issues regarding unsupported socket
options and system calls can be solved partly and the broken x/net
packages due to https://go-review.googlesource.com/36799 can be
repaired.
Fixes#3661.
Updates #9661.
Updates #19051.
Updates #19435.
Change-Id: Ic996b040418b54f6d043bc70591789d5a5b23270
Reviewed-on: https://go-review.googlesource.com/37039
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change adds RawControl, RawRead and RawWrite methods to type FD
to make the runtime-integrated network poller work together with a
user-defined function. The methods are used via the net package from
external packages and type FD is considered as an implementation of
syscall.Conn and syscall.RawConn interfaces.
Updates #19435.
Change-Id: I4ad04b10ffddb2b54fa8d70587440960d73c0a2d
Reviewed-on: https://go-review.googlesource.com/37038
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change adds Conn and RawConn interfaces which can be used to
manipulate raw network connection end points typically represented as
socket descriptors.
Fixes#19435.
Change-Id: Ide2d28eeab91bfd27473ab47a87bec69950b64c9
Reviewed-on: https://go-review.googlesource.com/37913
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Recent CL 41834 made windows Stat work for all symlinks.
But CL 41834 also made Stat slow.
John Starks sugested
(see https://github.com/golang/go/issues/19922#issuecomment-300031421)
to use GetFileAttributesEx for files and directories instead.
This makes Stat as fast as at go1.9.
I see these improvements on my Windows 7
name old time/op new time/op delta
StatDot 26.5µs ± 1% 20.6µs ± 2% -22.37% (p=0.000 n=9+10)
StatFile 22.8µs ± 2% 6.2µs ± 1% -72.69% (p=0.000 n=10+10)
StatDir 21.0µs ± 2% 6.1µs ± 3% -71.12% (p=0.000 n=10+9)
LstatDot 20.1µs ± 1% 20.7µs ± 6% +3.37% (p=0.000 n=9+10)
LstatFile 6.23µs ± 1% 6.36µs ± 8% ~ (p=0.587 n=9+10)
LstatDir 6.10µs ± 0% 6.14µs ± 4% ~ (p=0.590 n=9+10)
and on my Windows XP
name old time/op new time/op delta
StatDot-2 20.6µs ± 0% 10.8µs ± 0% -47.44% (p=0.000 n=10+10)
StatFile-2 20.2µs ± 0% 7.9µs ± 0% -60.91% (p=0.000 n=8+10)
StatDir-2 19.3µs ± 0% 7.6µs ± 0% -60.51% (p=0.000 n=10+9)
LstatDot-2 10.8µs ± 0% 10.8µs ± 0% -0.48% (p=0.000 n=10+8)
LstatFile-2 7.83µs ± 0% 7.83µs ± 0% ~ (p=0.844 n=10+8)
LstatDir-2 7.59µs ± 0% 7.56µs ± 0% -0.46% (p=0.000 n=10+10)
Updates #19922
Change-Id: Ice1fb5825defb05c79bab4dec0692e0fd1bcfcd5
Reviewed-on: https://go-review.googlesource.com/43071
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TestScopeRanges has been added in CL 40095. This
test is failing on Plan 9 because executables don't
have a DWARF symbol table.
Fixes#20418.
Change-Id: I6dd3baa636998134ccd042203c8b5c3199a4d6e1
Reviewed-on: https://go-review.googlesource.com/43670
Run-TryBot: David du Colombier <0intro@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Change compiler and linker to emit DWARF lexical blocks in .debug_info
section when compiling with -N -l.
Version of debug_info is updated from DWARF v2 to DWARF v3 since
version 2 does not allow lexical blocks with discontinuous PC ranges.
Remaining open problems:
- scope information is removed from inlined functions
- variables records do not have DW_AT_start_scope attributes so a
variable will shadow other variables with the same name as soon as its
containing scope begins, even before its declaration.
Updates #6913.
Updates #12899.
Change-Id: Idc6808788512ea20e7e45bcf782453acb416fb49
Reviewed-on: https://go-review.googlesource.com/40095
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
A number of issues in decoder.Read and newlineFilteringReader.Read were
preventing errors from the reader supplying the encoded data from being
propagated to the caller. Fixing these issues revealed some additional
problems in which valid decoded data was not always returned to the user
when errors were actually propagated.
This commit fixes both the error propagation and the lost decoded data
problems. It also adds some new unit tests to ensure errors are handled
correctly by decoder.Read. The new unit tests increase the test coverage
of this package from 96.2% to 97.9%.
Fixes#20044
Change-Id: I1a8632da20135906e2d191c2a8825b10e7ecc4c5
Reviewed-on: https://go-review.googlesource.com/42094
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously all arguments were passed through driver.IsValid.
This checked arguments against a few fundamental go types and
prevented others from being passed in as arguments.
The new interface driver.NamedValueChecker may be implemented
by both driver.Stmt and driver.Conn. This allows
this new interface to completely supersede the
driver.ColumnConverter interface as it can be used for
checking arguments known to a prepared statement and
arbitrary query arguments. The NamedValueChecker may be
skipped with driver.ErrSkip after all special cases are
exhausted to use the default argument converter.
In addition if driver.ErrRemoveArgument is returned
the argument will not be passed to the query at all,
useful for passing in driver specific per-query options.
Add a canonical Out argument wrapper to be passed
to OUTPUT parameters. This will unify checks that need to
be written in the NameValueChecker.
The statement number check is also moved to the argument
converter so the NamedValueChecker may remove arguments
passed to the query.
Fixes#13567Fixes#18079
Updates #18417
Updates #17834
Updates #16235
Updates #13067
Updates #19797
Change-Id: I89088bd9cca4596a48bba37bfd20d987453ef237
Reviewed-on: https://go-review.googlesource.com/38533
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The writebarrier pass processes WB ops from beginning to end,
replacing them by other values.
But it also checks whether there are more ops to process
by walking from beginning to end.
This is quadratic, so walk from end to beginning instead.
This speeds up compiling the code in issue 13554:
name old time/op new time/op delta
Pkg 11.9s ± 2% 8.3s ± 3% -29.88% (p=0.000 n=18+17)
Updates #13554
Passes toolstash-check.
Change-Id: I5f8a872ddc4b783540220d89ea2ee188a6d2b2ff
Reviewed-on: https://go-review.googlesource.com/43571
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
SWI only support "SWI $imm", but currently "SWI (Reg)" is also
accepted. This patch fixes it.
And more instruction tests are added to cmd/asm/internal/asm/testdata/arm.s
fixes#20375
Change-Id: Id437d853924a403e41da9b6cbddd20d994b624ff
Reviewed-on: https://go-review.googlesource.com/43552
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Second attempt to fix#14710.
CL 35272 already tried to fix this issue. But CL 35272 assumed
that runtime.epclntab type is STEXT, while it is actually SRODATA.
This CL uses Symbol.Sect.Seg to determine if symbol is part
of Segtext or Segdata.
Fixes#14710
Change-Id: Ic6b6f657555c87a64d2bc36cc4c07ab0591d00c4
Reviewed-on: https://go-review.googlesource.com/42390
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
profileBuilder.locForPC returns 0 to mean "no location" because 0 is
an invalid location index. However, the code to build count profiles
doesn't check the result of locForPC, so this 0 location index ends up
in the profile's location list. This, in turn, causes problems later
when we decode the profile because it puts a nil *Location in the
sample's location slice, which can later lead to a nil pointer panic.
Fix this by making printCountProfile correctly discard the result of
locForPC if it returns 0. This makes this call match the other two
calls of locForPC.
Updates #15156.
Change-Id: I4492b3652b513448bc56f4cfece4e37da5e42f94
Reviewed-on: https://go-review.googlesource.com/43630
Reviewed-by: Michael Matloob <matloob@golang.org>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TestGoroutineCounts was flaky when running on a system under load.
This happened on three builds the last couple of days.
Fix this by running this test with a single operating system thread, so
we do not depend on the operating system scheduler. 50 000 tests ran
without failure with the new version, the old version failed 0.5% of the
time.
Fixes#15156.
Change-Id: I1e5a18d0fef4f72cc9a56e376822b2849cdb0f8b
Reviewed-on: https://go-review.googlesource.com/43590
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
fuseBlockPlain was accidentally quadratic.
If you had plain blocks b1 -> b2 -> b3 -> b4,
each containing single values v1, v2, v3, and v4 respectively,
fuseBlockPlain would move v1 from b1 to b2 to b3 to b4,
then v2 from b2 to b3 to b4, etc.
There are two obvious fixes.
* Look for runs of blocks in fuseBlockPlain
and handle them in a single go.
* Fuse from end to beginning; any given value in a run
of blocks to fuse then moves only once.
The latter is much simpler, so that's what this CL does.
Somewhat surprisingly, this change does not pass toolstash-check.
The resulting set of blocks is the same,
and the values in them are the same,
but the order of values in them differ,
and that order of values (while arbitrary)
is enough to change the compiler's output.
This may be due to #20178; deadstore is the next pass after fuse.
Adding basic sorting to the beginning of deadstore
is enough to make this CL pass toolstash-check:
for _, b := range f.Blocks {
obj.SortSlice(b.Values, func(i, j int) bool { return b.Values[i].ID < b.Values[j].ID })
}
Happily, this CL appears to result in better code on average,
if only by accident. It cuts 4k off of cmd/go; go1 benchmarks
are noisy as always but don't regress (numbers below).
No impact on the standard compilebench benchmarks.
For the code in #13554, this speeds up compilation dramatically:
name old time/op new time/op delta
Pkg 53.1s ± 2% 12.8s ± 3% -75.92% (p=0.008 n=5+5)
name old user-time/op new user-time/op delta
Pkg 55.0s ± 2% 14.9s ± 3% -73.00% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Pkg 2.04GB ± 0% 2.04GB ± 0% +0.18% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Pkg 6.21M ± 0% 6.21M ± 0% ~ (p=0.222 n=5+5)
name old object-bytes new object-bytes delta
Pkg 28.4M ± 0% 28.4M ± 0% +0.00% (p=0.008 n=5+5)
name old export-bytes new export-bytes delta
Pkg 208 ± 0% 208 ± 0% ~ (all equal)
Updates #13554
go1 benchmarks:
name old time/op new time/op delta
BinaryTree17-8 2.29s ± 2% 2.26s ± 2% -1.43% (p=0.000 n=48+50)
Fannkuch11-8 2.74s ± 2% 2.79s ± 2% +1.63% (p=0.000 n=50+49)
FmtFprintfEmpty-8 36.6ns ± 3% 34.6ns ± 4% -5.29% (p=0.000 n=49+50)
FmtFprintfString-8 58.3ns ± 3% 59.1ns ± 3% +1.35% (p=0.000 n=50+49)
FmtFprintfInt-8 62.4ns ± 2% 63.2ns ± 3% +1.19% (p=0.000 n=49+49)
FmtFprintfIntInt-8 95.1ns ± 2% 96.7ns ± 3% +1.61% (p=0.000 n=49+50)
FmtFprintfPrefixedInt-8 118ns ± 3% 113ns ± 2% -4.00% (p=0.000 n=50+49)
FmtFprintfFloat-8 191ns ± 2% 192ns ± 2% +0.40% (p=0.034 n=50+50)
FmtManyArgs-8 419ns ± 2% 420ns ± 2% ~ (p=0.228 n=49+49)
GobDecode-8 5.26ms ± 3% 5.19ms ± 2% -1.33% (p=0.000 n=50+49)
GobEncode-8 4.12ms ± 2% 4.15ms ± 3% +0.68% (p=0.007 n=49+50)
Gzip-8 198ms ± 2% 197ms ± 2% -0.50% (p=0.018 n=48+48)
Gunzip-8 31.9ms ± 3% 31.8ms ± 3% -0.47% (p=0.024 n=50+50)
HTTPClientServer-8 64.4µs ± 0% 64.0µs ± 0% -0.55% (p=0.000 n=43+46)
JSONEncode-8 10.6ms ± 2% 10.6ms ± 3% ~ (p=0.543 n=49+49)
JSONDecode-8 43.3ms ± 3% 43.1ms ± 2% ~ (p=0.079 n=50+50)
Mandelbrot200-8 3.70ms ± 2% 3.70ms ± 2% ~ (p=0.553 n=47+50)
GoParse-8 2.70ms ± 2% 2.71ms ± 3% ~ (p=0.843 n=49+50)
RegexpMatchEasy0_32-8 70.5ns ± 4% 70.4ns ± 4% ~ (p=0.867 n=48+50)
RegexpMatchEasy0_1K-8 162ns ± 3% 162ns ± 2% ~ (p=0.739 n=48+48)
RegexpMatchEasy1_32-8 66.1ns ± 5% 66.2ns ± 4% ~ (p=0.970 n=50+50)
RegexpMatchEasy1_1K-8 297ns ± 7% 296ns ± 7% ~ (p=0.406 n=50+50)
RegexpMatchMedium_32-8 105ns ± 5% 105ns ± 5% ~ (p=0.702 n=50+50)
RegexpMatchMedium_1K-8 32.3µs ± 4% 32.2µs ± 3% ~ (p=0.614 n=49+49)
RegexpMatchHard_32-8 1.75µs ±18% 1.74µs ±12% ~ (p=0.738 n=50+48)
RegexpMatchHard_1K-8 52.2µs ±14% 51.3µs ±13% ~ (p=0.230 n=50+50)
Revcomp-8 366ms ± 3% 367ms ± 3% ~ (p=0.745 n=49+49)
Template-8 48.5ms ± 4% 48.5ms ± 4% ~ (p=0.824 n=50+48)
TimeParse-8 263ns ± 2% 256ns ± 2% -2.98% (p=0.000 n=48+49)
TimeFormat-8 265ns ± 3% 262ns ± 3% -1.35% (p=0.000 n=48+49)
[Geo mean] 41.1µs 40.9µs -0.48%
Change-Id: Ib35fa15b54282abb39c077d150beee27f610891a
Reviewed-on: https://go-review.googlesource.com/43570
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
When the race detector is enabled,
the compiler randomizes the order in which functions are compiled,
in an attempt to shake out bugs.
But we never re-seed the rand source, so every execution is identical.
Fix that to get more coverage.
Change-Id: If5cdde03ef4f1bab5f45e07f03fb6614945481d7
Reviewed-on: https://go-review.googlesource.com/43572
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently, cgo converts integer macros into int64 if it's possible.
As a result, some macros which satisfy
math.MaxInt64 < x <= math.MaxUint64
will lose their original values.
This CL introduces the new probe to check signs,
so we can handle signed ints and unsigned ints separately.
Fixes#20369
Change-Id: I002ba452a82514b3a87440960473676f842cc9ee
Reviewed-on: https://go-review.googlesource.com/43476
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The generated file runtime/internal/sys/zversion.go is deleted by
`go tool cmd dist clean` as part of running clean.bash. Don't treat
a missing file as a reason to stop running the go tool; just treat
is as meaning that runtime/internal/sys is stale.
No test because I don't particularly want to clobber $GOROOT.
Fixes#20385.
Change-Id: I5251a99542cc93c33f627f133d7118df56e18af1
Reviewed-on: https://go-review.googlesource.com/43559
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Restore the handling of io.ErrShortWrite in (*File).Write:
if we write less than the requested amount, and there is no error from
the syscall, then return io.ErrShortWrite.
I can't figure out how to write a test for this. It would require a
non-pollable file (not a pipe) on a device that is almost but not
quite entirely full. The original code (https://golang.org/cl/36800043,
committed as part of https://golang.org/cl/36930044) does not have a test.
Fixes#20386.
Change-Id: Ied7b411e621e1eaf49f864f8db90069f276256f5
Reviewed-on: https://go-review.googlesource.com/43558
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In low memory situations mmap(2) on Illumos[2] can return EAGAIN when it
is unable to reserve the necessary space for the requested mapping. Go
was not previously handling this correctly for Illumos and would fail to
recognize it was in a low-memory situation, the result being the program
would terminate with a panic instead of running the GC.
Fixes: #14930
[1]: https://www.illumos.org/man/2/mmap
Change-Id: I889cc0547e23f9d6c56e4fdd7bcbd0e15403873a
Reviewed-on: https://go-review.googlesource.com/43461
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
On other systems we use "SWI $n". Change Plan 9 files to be
consistent. Generated binary is unchanged.
Fixes#20378.
Change-Id: Ia2a722061da2450c7b30cb707ed4f172fafecf74
Reviewed-on: https://go-review.googlesource.com/43533
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
LLV and SCV are 64-bit load-linked and store-conditional. They
were used in runtime as #define WORD. Change them to normal
instruction form.
NOOP is hardware no-op. It was written as WORD $0. Make a name
for it for better disassembly output.
Fixes#12561.
Fixes#18238.
Change-Id: I82c667ce756fa83ef37b034b641e8c4366335e83
Reviewed-on: https://go-review.googlesource.com/40297
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change adds several examples, with emphasis on special or edge
cases such as a directory parameter consisting of an empty string.
Change-Id: Ib4ac3d0f6d503493eeed0c4fda7c12acf782e9e2
Reviewed-on: https://go-review.googlesource.com/43010
Reviewed-by: Steve Francia <spf@golang.org>
Run-TryBot: Jaana Burcu Dogan <jbd@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Noticed while looking at #20356.
Cuts 160k (1%) off of the cmd/compile binary.
Change-Id: If2397bc6971d6be9be6975048adecb0b5efa6d66
Reviewed-on: https://go-review.googlesource.com/43501
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Previous CL (cmd/internal/objabi: shrink SymType down to a uint8) shrinks
SymType down to a uint8 but forgot making according change in goobj.
Fixes#20296
Also add a test to catch such Goobj format inconsistency bug
Change-Id: Ib43dd7122cfcacf611a643814e95f8c5a924941f
Reviewed-on: https://go-review.googlesource.com/42971
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Mark variables as used even when they appear within an expression
context which we can't type-check; e.g., because the expression is
erroneous, or comes from an import "C" declaration.
Fixes#20358.
Change-Id: Ib28cc78d3867c597c7a1ace54de09ada02f5b33a
Reviewed-on: https://go-review.googlesource.com/43500
Reviewed-by: Alan Donovan <adonovan@google.com>
Attaching positions to SB, SP, initial mem can result in
less-good line-numbering when compiled for debugging.
This "fix" also removes source position from a zero-valued
struct (but not from its fields) and from a zero-length
array constant.
This may be a general problem for constants in entry blocks.
Fixes#20367.
Change-Id: I7e9df3341be2e2f60f127d35bb31e43cdcfce9a1
Reviewed-on: https://go-review.googlesource.com/43531
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Enhance the one-live-memory-at-a-time check to run during many
more phases of the SSA backend. Also make it work in an interblock
fashion.
Change types.IsMemory to return true for tuples containing a memory type.
Fix trim pass to build the merged phi correctly. Doesn't affect
code but allows the check to pass after trim runs.
Switch the AddTuple* ops to take the memory-containing tuple argument second.
Update #20335
Change-Id: I5b03ef3606b75a9e4f765276bb8b183cdc172b43
Reviewed-on: https://go-review.googlesource.com/43495
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Documentation of build.Import says:
// If the path is a local import path naming a package that can be imported
// using a standard import path, the returned package will set p.ImportPath
// to that path.
// ...
// If an error occurs, Import returns a non-nil error and a non-nil
// *Package containing partial information.
That behavior was previously untested, and broken by change in CL 33158.
Fix that by avoiding returning early on error for local import paths.
First, gather partial information, and only then check that the p.Dir
directory exists.
Add tests for this behavior.
Fixes#19769.
Fixes#20175 (duplicate of #19769).
Updates #17863.
Change-Id: I169cb35291099d05e02aaa3cb23a7403d1cc3657
Reviewed-on: https://go-review.googlesource.com/42350
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently proto symbolization uses runtime.FuncForPC and assumes each
PC maps to a single frame. This isn't true in the presence of inlining
(even with leaf-only inlining this can get incorrect results).
Change PC symbolization to use runtime.CallersFrames to expand each PC
to all of the frames at that PC.
Change-Id: I8d20dff7495a5de495ae07f569122c225d433ced
Reviewed-on: https://go-review.googlesource.com/41256
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Proto profile conversion is inconsistent about call vs return PCs in
profile locations. The proto defines locations to be call PCs. This is
what we do when proto-izing CPU profiles, but we fail to convert the
return PCs in memory and count profile stacks to call PCs when
converting them to proto locations.
Fix this in the heap and count profile conversion functions.
TestConvertMemProfile also hard-codes this failure to convert from
return PCs to call PCs, so fix up the addresses in the synthesized
profile to be return PCs while checking that we get call PCs out of
the conversion.
Change-Id: If1fc028b86fceac6d71a2d9fa6c41ff442c89296
Reviewed-on: https://go-review.googlesource.com/42951
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Adjust finddebugruntimepath to look for runtime/debug.go file
instead of runtime/runtime.go. This actually finds runtime.GOMAXPROCS
in every Go executable (including windows).
I also included "-Wl,-T,fix_debug_gdb_scripts.ld" parameter to gcc
invocation on windows to work around gcc bug (see #20183 for details).
This CL only fixes windows -buildmode=exe, buildmode=c-archive
is still broken.
Thanks to Egon Elbre and Nick Clifton for investigation.
Fixes#20183Fixes#20218
Change-Id: I5369a4db3913226aef3d9bd6317446856b0a1c34
Reviewed-on: https://go-review.googlesource.com/43331
Reviewed-by: Ian Lance Taylor <iant@golang.org>
For #20292. (See discussion in that issue.)
Change-Id: I44cd69394fc47a01776905ec34305ba524c89883
Reviewed-on: https://go-review.googlesource.com/43452
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Make yellow the last highlight color rather than the first.
Yellow is also the color that Chrome uses to highlight
search results, which can be confusing.
Also, when Night Shift is on on macOS,
yellow highlighting is completely invisible.
I suppose should be sleeping instead.
Also, remove a completed TODO.
Change-Id: I0eb4439272fad9ccb5fe8e2cf409fdd5dc15b26e
Reviewed-on: https://go-review.googlesource.com/43463
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When compiling concurrently, we walk all functions before compiling
any of them. Walking functions can cause variables to switch from
being non-addrtaken to addrtaken, e.g. to prepare for a runtime call.
Typechecking propagates addrtaken-ness of closure variables to
their outer variables, so that capturevars can decide whether to
pass the variable's value or a pointer to it.
When all functions are compiled immediately, as long as the containing
function is compiled prior to the closure, this propagation has no effect.
When compilation is deferred, though, in rare cases, this results in
a change in the addrtaken-ness of a variable in the outer function,
which in turn changes the compiler's output.
(This is rare because in a great many cases, a temporary has been
introduced, insulating the outer variable from modification.)
But concurrent compilation must generate identical results.
To fix this, track whether capturevars has run.
If it has, there is no need to update outer variables
when closure variables change.
Capturevars always runs before any functions are walked or compiled.
The remainder of the changes in this CL are to support the test.
In particular, -d=compilelater forces the compiler to walk all
functions before compiling any of them, despite being non-concurrent.
This is useful because -live is fundamentally incompatible with
concurrent compilation, but we want -c=1 to have no behavior changes.
Fixes#20250
Change-Id: I89bcb54268a41e8588af1ac8cc37fbef856a90c2
Reviewed-on: https://go-review.googlesource.com/42853
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
When writing tar files by using the FileInfoHeader
the type bits was set in the mode field of the header
This is not correct according to the standard (GNU/Posix) and
other implementations.
Fixed#20150
Change-Id: I3be7d946a1923ad5827cf45c696546a5e287ebba
Reviewed-on: https://go-review.googlesource.com/42093
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
While at it, unindent source text so column values are easier
to read, remove unnecessary text in output, and simplify the
loop.
Fixes#20346.
Change-Id: I0fde02b9e4242383da427f4cf4c6c13dd0ab3b47
Reviewed-on: https://go-review.googlesource.com/43450
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In some cases it is desirable to customize the way the DNS server is
contacted, for instance to use a specific LocalAddr. While most
operating-system level resolvers do not allow this, we have the
opportunity to do so with the Go resolver. Most of the code was
already in place to allow tests to override the dialer. This exposes
that functionality, and as a side effect eliminates the need for a
testing hook.
Fixes#17404
Change-Id: I1c5e570f8edbcf630090f8ec6feb52e379e3e5c0
Reviewed-on: https://go-review.googlesource.com/37260
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime.gchelper depends on the non-atomic load of work.ndone
happening strictly before the atomic add of work.nwait. Until very
recently (commit 978af9c2db, fixing #20334), the compiler reordered
these operations. This created a race since work.ndone can change as
soon as work.nwait is equal to work.ndone. If that happened, more than
one gchelper could attempt to wake up the work.alldone note, causing a
"double wakeup" panic.
This was fixed in the compiler, but to make this code less subtle,
make the load of work.ndone atomic. This clearly forces the order of
these operations, ensuring the race doesn't happen.
Fixes#19305 (though really 978af9c2db fixed it).
Change-Id: Ieb1a84e1e5044c33ac612c8a5ab6297e7db4c57d
Reviewed-on: https://go-review.googlesource.com/43311
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This adds debugging information when we panic with "heapBitsForSpan:
base out of range".
Updates #20259.
Change-Id: I0dc1a106aa9e9531051c7d08867ace5ef230eb3f
Reviewed-on: https://go-review.googlesource.com/43310
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
On Windows the drive letter is sometime "c:" and sometimes "C:".
Fixes#20336.
Change-Id: I38c86999af9522c51470d60016729d41cfec6b25
Reviewed-on: https://go-review.googlesource.com/43390
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TestCgoContainsSpace builds a small program which mimics $CC.
Usually, $CC attempts to compile a trivial code to detect its own
supported flags (i.e. "-no-pie", which must be passed on some systems),
however the mimic didn't consider these cases.
This CL solve the issue.
Also, use the same name as $CC, it may solve other potential problems.
Fixes#20324
Change-Id: I7a00ac016a5fd0667540f2a715371f8152edc395
Reviewed-on: https://go-review.googlesource.com/43330
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Tuple ops are weird. They are essentially a pair of ops,
one which consumes a mem and one which generates a mem (the Select1).
The schedule pass didn't handle these quite right.
Fix the scheduler to include both parts of the paired op in
the store chain. That makes sure that loads are correctly ordered
with respect to the first of the pair.
Add a check for the ssacheck builder, that there is only one
live store at a time. I thought we already had such a check, but
apparently not...
Fixes#20335
Change-Id: I59eb3446a329100af38d22820b1ca2190ca46a78
Reviewed-on: https://go-review.googlesource.com/43294
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The writebarrier test has to change.
Now that T23 composite literals are passed to the backend,
they get SSA'd, so writes to their fields are treated separately,
so the relevant part of the first write to t23 is now a dead store.
Preserve the intent of the test by splitting it up into two functions.
Reduces code size a bit:
name old object-bytes new object-bytes delta
Template 386k ± 0% 386k ± 0% ~ (all equal)
Unicode 202k ± 0% 202k ± 0% ~ (all equal)
GoTypes 1.16M ± 0% 1.16M ± 0% ~ (all equal)
Compiler 3.92M ± 0% 3.91M ± 0% -0.19% (p=0.008 n=5+5)
SSA 7.91M ± 0% 7.91M ± 0% ~ (all equal)
Flate 228k ± 0% 228k ± 0% -0.05% (p=0.008 n=5+5)
GoParser 283k ± 0% 283k ± 0% ~ (all equal)
Reflect 952k ± 0% 952k ± 0% -0.06% (p=0.008 n=5+5)
Tar 188k ± 0% 188k ± 0% -0.09% (p=0.008 n=5+5)
XML 406k ± 0% 406k ± 0% -0.02% (p=0.008 n=5+5)
[Geo mean] 649k 648k -0.04%
Fixes#18872
Change-Id: Ifeed0f71f13849732999aa731cc2bf40c0f0e32a
Reviewed-on: https://go-review.googlesource.com/43154
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
v is not a pointer receiver, and v.typ isn't used in the lines below.
The assignment is dead. Remove it.
Keep the comment, as it refers to the whole case block and not just the
removed line.
Change-Id: Icb2d20c287d9a41bf620ebe5cdec764cd84178a7
Reviewed-on: https://go-review.googlesource.com/43134
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
They are not exported and not used in the compiler or standard library.
Change-Id: Ie1d210464f826742d282f12258ed1792cbd2d188
Reviewed-on: https://go-review.googlesource.com/43135
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reuse block head or preceding instruction's line number for
register allocator's spill, fill, copy, rematerialization
instructionsl; and also for phi, and for no-src-pos
instructions. Assembler creates same line number tables
for copy-predecessor-line and for no-src-pos,
but copy-predecessor produces better-looking assembly
language output with -S and with GOSSAFUNC, and does not
require changes to tests of existing assembly language.
Split "copyInto" into two cases, one for register allocation,
one for otherwise. This caused the test score line change
count to increase by one, which may reflect legitimately
useful information preserved. Without any special treatment
for copyInto, the change count increases by 21 more, from
51 to 72 (i.e., quite a lot).
There is a test; using two naive "scores" for line number
churn, the old numbering is 2x or 4x worse.
Fixes#18902.
Change-Id: I0a0a69659d30ee4e5d10116a0dd2b8c5df8457b1
Reviewed-on: https://go-review.googlesource.com/36207
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Fixes#7906
Change-Id: Ibcf9cd670593241921ab3c426ff7357f799ebc3e
Reviewed-on: https://go-review.googlesource.com/43072
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Implements detection of x86 cpu features that
are used in the go standard library.
Changes all standard library packages to use the new cpu package
instead of using runtime internal variables to check x86 cpu features.
Updates: #15403
Change-Id: I2999a10cb4d9ec4863ffbed72f4e021a1dbc4bb9
Reviewed-on: https://go-review.googlesource.com/41476
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Eliminates stores of values that have just been loaded from the same
location. Handles the common case where there are up to 3 intermediate
stores to non-overlapping struct fields.
For example the loads and stores of x.a, x.b and x.d in the following
function are now removed:
type T struct {
a, b, c, d int
}
func f(x *T) {
y := *x
y.c += 8
*x = y
}
Before this CL (s390x):
TEXT "".f(SB)
MOVD "".x(R15), R5
MOVD (R5), R1
MOVD 8(R5), R2
MOVD 16(R5), R0
MOVD 24(R5), R4
ADD $8, R0, R3
STMG R1, R4, (R5)
RET
After this CL (s390x):
TEXT "".f(SB)
MOVD "".x(R15), R1
MOVD 16(R1), R0
ADD $8, R0, R0
MOVD R0, 16(R1)
RET
In total these rules are triggered ~5091 times during all.bash,
which is broken down as:
Intermediate stores | Triggered
--------------------+----------
0 | 1434
1 | 2508
2 | 888
3 | 261
--------------------+----------
Change-Id: Ia4721ae40146aceec1fdd3e65b0e9283770bfba5
Reviewed-on: https://go-review.googlesource.com/38793
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The files PPC64.rules and rewritePPC64.go were out of sync due to
conflicts between CL 41630 and CL 42145 (i.e. running 'go run *.go'
in the gen directory resulted in unexpected changes).
Change-Id: I1d409656b66afeab6cb9c6df9b3dcab7859caa75
Reviewed-on: https://go-review.googlesource.com/43091
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
This can make life easier for Delve (and other debuggers),
and can help them with bug reports.
Sample producer field (from objdump):
<48> DW_AT_producer : Go cmd/compile devel +8a59dbf41a Mon May 8 16:02:44 2017 -0400
Change-Id: I0605843c959b53a60a25a3b870aa8755bf5d5b13
Reviewed-on: https://go-review.googlesource.com/33588
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We do a division by the elem type size to check if the array size would
be too large for the virtual address space. This is a silly check if the
size is 0, but the problem is that it means a division by zero and a
panic.
Since arrays of empty structs are valid in a regular program, make them
also work in reflect.
Use a separate, explicit test with struct{}{} to make sure the test for
a zero-sized type is not confused with the rest.
Fixes#20313.
Change-Id: I47b8b87e6541631280b79227bdea6a0f6035c9e0
Reviewed-on: https://go-review.googlesource.com/43131
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This adds math/bits intrinsics for OnesCount, Len, TrailingZeros on
ppc64x.
benchmark old ns/op new ns/op delta
BenchmarkLeadingZeros-16 4.26 1.71 -59.86%
BenchmarkLeadingZeros16-16 3.04 1.83 -39.80%
BenchmarkLeadingZeros32-16 3.31 1.82 -45.02%
BenchmarkLeadingZeros64-16 3.69 1.71 -53.66%
BenchmarkTrailingZeros-16 2.55 1.62 -36.47%
BenchmarkTrailingZeros32-16 2.55 1.77 -30.59%
BenchmarkTrailingZeros64-16 2.78 1.62 -41.73%
BenchmarkOnesCount-16 3.19 0.93 -70.85%
BenchmarkOnesCount32-16 2.55 1.18 -53.73%
BenchmarkOnesCount64-16 3.22 0.93 -71.12%
Update #18616
I also made a change to bits_test.go because when debugging some failures
the output was not quite providing the right argument information.
Change-Id: Ia58d31d1777cf4582a4505f85b11a1202ca07d3e
Reviewed-on: https://go-review.googlesource.com/41630
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Keith Randall <khr@golang.org>
When constructing a new type for an array type in ArrayOf, we don't
reset tflag to 0. All the other methods in the package, such as SliceOf,
do this already. This results in the new array type having weird issues
when being printed, such as having tflagExtraStar set when it shouldn't.
That flag removes the first char to get rid of '*', but when used
incorrectly in this case it eats the '[' character leading to broken
strings like "3]int".
This was fixed in 56752eb2 for issue #16722, but ArrayOf was missed.
Also make the XM test struct have a non-zero size as that leads to a
division by zero panic in ArrayOf.
Fixes#20311.
Change-Id: I18f1027fdbe9f71767201e7424269c3ceeb23eb5
Reviewed-on: https://go-review.googlesource.com/43130
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Also change type from map[*types.Type]bool to map[*types.Type]struct{}.
This is basically a clean-up.
Change-Id: I167583eff0fa1070a7522647219476033b52b840
Reviewed-on: https://go-review.googlesource.com/41859
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
OpVarXXX Values don't generate instructions,
so there's no reason not to duplicate them,
and duplicating them generates better code
(fewer branches).
This requires changing the start/end accounting
to correctly handle the case in which we have run
of Values beginning with an OpVarXXX, e.g.
OpVarDef, OpZeroWB, OpMoveWB.
In that case, the sequence of values should begin
at the OpZeroWB, not the OpVarDef.
This also lays the groundwork for experimenting
with allowing duplication of some scalar stores.
Shrinks function text sizes a tiny amount:
name old object-bytes new object-bytes delta
Template 381k ± 0% 381k ± 0% -0.01% (p=0.008 n=5+5)
Unicode 203k ± 0% 203k ± 0% -0.04% (p=0.008 n=5+5)
GoTypes 1.17M ± 0% 1.17M ± 0% -0.01% (p=0.008 n=5+5)
SSA 8.24M ± 0% 8.24M ± 0% -0.00% (p=0.008 n=5+5)
Flate 230k ± 0% 230k ± 0% ~ (all equal)
GoParser 286k ± 0% 286k ± 0% ~ (all equal)
Reflect 1.00M ± 0% 1.00M ± 0% ~ (all equal)
Tar 189k ± 0% 189k ± 0% ~ (all equal)
XML 415k ± 0% 415k ± 0% -0.01% (p=0.008 n=5+5)
Updates #19838
Change-Id: Ic5ef30855919f1468066eba08ae5c4bd9a01db27
Reviewed-on: https://go-review.googlesource.com/42011
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
In PPC64 ELF files, the st_other field indicates the number of
prologue instructions between the global and local entry points.
We add the instructions in the compiler and assembler if -shared is used.
We were assuming that the instructions were present when building a
c-archive or PIE or doing dynamic linking, on the assumption that those
are the cases where the go tool would be building with -shared.
That assumption fails when using some other tool, such as Bazel,
that does not necessarily use -shared in exactly the same way.
This CL records in the object file whether a symbol was compiled
with -shared (this will be the same for all symbols in a given compilation)
and uses that information when setting the st_other field.
Fixes#20290.
Change-Id: Ib2b77e16aef38824871102e3c244fcf04a86c6ea
Reviewed-on: https://go-review.googlesource.com/43051
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
When package ssa was created, Type was in package gc.
To avoid circular dependencies, we used an interface (ssa.Type)
to represent type information in SSA.
In the Go 1.9 cycle, gri extricated the Type type from package gc.
As a result, we can now use it in package ssa.
Now, instead of package types depending on package ssa,
it is the other way.
This is a more sensible dependency tree,
and helps compiler performance a bit.
Though this is a big CL, most of the changes are
mechanical and uninteresting.
Interesting bits:
* Add new singleton globals to package types for the special
SSA types Memory, Void, Invalid, Flags, and Int128.
* Add two new Types, TSSA for the special types,
and TTUPLE, for SSA tuple types.
ssa.MakeTuple is now types.NewTuple.
* Move type comparison result constants CMPlt, CMPeq, and CMPgt
to package types.
* We had picked the name "types" in our rules for the handy
list of types provided by ssa.Config. That conflicted with
the types package name, so change it to "typ".
* Update the type comparison routine to handle tuples and special
types inline.
* Teach gc/fmt.go how to print special types.
* We can now eliminate ElemTypes in favor of just Elem,
and probably also some other duplicated Type methods
designed to return ssa.Type instead of *types.Type.
* The ssa tests were using their own dummy types,
and they were not particularly careful about types in general.
Of necessity, this CL switches them to use *types.Type;
it does not make them more type-accurate.
Unfortunately, using types.Type means initializing a bit
of the types universe.
This is prime for refactoring and improvement.
This shrinks ssa.Value; it now fits in a smaller size class
on 64 bit systems. This doesn't have a giant impact,
though, since most Values are preallocated in a chunk.
name old alloc/op new alloc/op delta
Template 37.9MB ± 0% 37.7MB ± 0% -0.57% (p=0.000 n=10+8)
Unicode 28.9MB ± 0% 28.7MB ± 0% -0.52% (p=0.000 n=10+10)
GoTypes 110MB ± 0% 109MB ± 0% -0.88% (p=0.000 n=10+10)
Flate 24.7MB ± 0% 24.6MB ± 0% -0.66% (p=0.000 n=10+10)
GoParser 31.1MB ± 0% 30.9MB ± 0% -0.61% (p=0.000 n=10+9)
Reflect 73.9MB ± 0% 73.4MB ± 0% -0.62% (p=0.000 n=10+8)
Tar 25.8MB ± 0% 25.6MB ± 0% -0.77% (p=0.000 n=9+10)
XML 41.2MB ± 0% 40.9MB ± 0% -0.80% (p=0.000 n=10+10)
[Geo mean] 40.5MB 40.3MB -0.68%
name old allocs/op new allocs/op delta
Template 385k ± 0% 386k ± 0% ~ (p=0.356 n=10+9)
Unicode 343k ± 1% 344k ± 0% ~ (p=0.481 n=10+10)
GoTypes 1.16M ± 0% 1.16M ± 0% -0.16% (p=0.004 n=10+10)
Flate 238k ± 1% 238k ± 1% ~ (p=0.853 n=10+10)
GoParser 320k ± 0% 320k ± 0% ~ (p=0.720 n=10+9)
Reflect 957k ± 0% 957k ± 0% ~ (p=0.460 n=10+8)
Tar 252k ± 0% 252k ± 0% ~ (p=0.133 n=9+10)
XML 400k ± 0% 400k ± 0% ~ (p=0.796 n=10+10)
[Geo mean] 428k 428k -0.01%
Removing all the interface calls helps non-trivially with CPU, though.
name old time/op new time/op delta
Template 178ms ± 4% 173ms ± 3% -2.90% (p=0.000 n=94+96)
Unicode 85.0ms ± 4% 83.9ms ± 4% -1.23% (p=0.000 n=96+96)
GoTypes 543ms ± 3% 528ms ± 3% -2.73% (p=0.000 n=98+96)
Flate 116ms ± 3% 113ms ± 4% -2.34% (p=0.000 n=96+99)
GoParser 144ms ± 3% 140ms ± 4% -2.80% (p=0.000 n=99+97)
Reflect 344ms ± 3% 334ms ± 4% -3.02% (p=0.000 n=100+99)
Tar 106ms ± 5% 103ms ± 4% -3.30% (p=0.000 n=98+94)
XML 198ms ± 5% 192ms ± 4% -2.88% (p=0.000 n=92+95)
[Geo mean] 178ms 173ms -2.65%
name old user-time/op new user-time/op delta
Template 229ms ± 5% 224ms ± 5% -2.36% (p=0.000 n=95+99)
Unicode 107ms ± 6% 106ms ± 5% -1.13% (p=0.001 n=93+95)
GoTypes 696ms ± 4% 679ms ± 4% -2.45% (p=0.000 n=97+99)
Flate 137ms ± 4% 134ms ± 5% -2.66% (p=0.000 n=99+96)
GoParser 176ms ± 5% 172ms ± 8% -2.27% (p=0.000 n=98+100)
Reflect 430ms ± 6% 411ms ± 5% -4.46% (p=0.000 n=100+92)
Tar 128ms ±13% 123ms ±13% -4.21% (p=0.000 n=100+100)
XML 239ms ± 6% 233ms ± 6% -2.50% (p=0.000 n=95+97)
[Geo mean] 220ms 213ms -2.76%
Change-Id: I15c7d6268347f8358e75066dfdbd77db24e8d0c1
Reviewed-on: https://go-review.googlesource.com/42145
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
These collectively fire a few hundred times during make.bash,
mostly rewriting XOR SETNE -> SETEQ.
Fixes#17905.
Change-Id: Ic5eb241ee93ed67099da3de11f59e4df9fab64a3
Reviewed-on: https://go-review.googlesource.com/42491
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
CL 39915 introduced sorting of signats by ShortString
for reproducible builds. But ShortString treats types
byte and uint8 identically; same for rune and uint32.
CL 39915 attempted to compensate for this by only
adding the underlying type (uint8) to signats in addsignat.
This only works for byte and uint8. For e.g. *byte and *uint,
both get added, and their sort order is random,
leading to non-reproducible builds.
One fix would be to add yet another type printing mode
that doesn't eliminate byte and rune, and use it
for sorting signats. But the formatting routines
are complicated enough as it is.
Instead, just sort first by ShortString and then by String.
We can't just use String, because ShortString makes distinctions
that String doesn't. ShortString is really preferred here;
String is serving only as a backstop for handling of bytes and runes.
The long series of types in the test helps increase the odds of
failure, allowing a smaller number of iterations in the test.
On my machine, a full test takes 700ms.
Passes toolstash-check.
Updates #19961Fixes#20272
name old alloc/op new alloc/op delta
Template 37.9MB ± 0% 37.9MB ± 0% +0.12% (p=0.032 n=5+5)
Unicode 28.9MB ± 0% 28.9MB ± 0% ~ (p=0.841 n=5+5)
GoTypes 110MB ± 0% 110MB ± 0% ~ (p=0.841 n=5+5)
Compiler 463MB ± 0% 463MB ± 0% ~ (p=0.056 n=5+5)
SSA 1.11GB ± 0% 1.11GB ± 0% +0.02% (p=0.016 n=5+5)
Flate 24.7MB ± 0% 24.8MB ± 0% +0.14% (p=0.032 n=5+5)
GoParser 31.1MB ± 0% 31.1MB ± 0% ~ (p=0.421 n=5+5)
Reflect 73.9MB ± 0% 73.9MB ± 0% ~ (p=1.000 n=5+5)
Tar 25.8MB ± 0% 25.8MB ± 0% +0.15% (p=0.016 n=5+5)
XML 41.2MB ± 0% 41.2MB ± 0% ~ (p=0.310 n=5+5)
[Geo mean] 72.0MB 72.0MB +0.07%
name old allocs/op new allocs/op delta
Template 384k ± 0% 385k ± 1% ~ (p=0.056 n=5+5)
Unicode 343k ± 0% 344k ± 0% ~ (p=0.548 n=5+5)
GoTypes 1.16M ± 0% 1.16M ± 0% ~ (p=0.421 n=5+5)
Compiler 4.43M ± 0% 4.44M ± 0% +0.26% (p=0.032 n=5+5)
SSA 9.86M ± 0% 9.87M ± 0% +0.10% (p=0.032 n=5+5)
Flate 237k ± 1% 238k ± 0% +0.49% (p=0.032 n=5+5)
GoParser 319k ± 1% 320k ± 1% ~ (p=0.151 n=5+5)
Reflect 957k ± 0% 957k ± 0% ~ (p=1.000 n=5+5)
Tar 251k ± 0% 252k ± 1% +0.49% (p=0.016 n=5+5)
XML 399k ± 0% 401k ± 1% ~ (p=0.310 n=5+5)
[Geo mean] 739k 741k +0.26%
Change-Id: Ic27995a8d374d012b8aca14546b1df9d28d30df7
Reviewed-on: https://go-review.googlesource.com/42955
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
If there were more unused imports than
the maximum default number of errors to report,
the set of reported imports was non-deterministic.
Fix by accumulating and sorting them prior to output.
Fixes#20298
Change-Id: Ib3d5a15fd7dc40009523fcdc1b93ddc62a1b05f2
Reviewed-on: https://go-review.googlesource.com/42954
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
ARM64 assembler backend only accepts loads and stores with small
or aligned offset. The compiler therefore can only fold small or
aligned offsets into loads and stores. For locals and args, their
offsets to SP are not known until very late, and the compiler
makes conservative decision not folding some of them. However,
in most cases, the offset is indeed small or aligned, and can
be folded into load and store (but actually not).
This CL adds support of loads and stores with large and unaligned
offsets. When the offset doesn't fit into the instruction, it
uses two instructions and (for very large offset) the constant
pool. This way, the compiler doesn't need to be conservative,
and can simply fold the offset.
To make it work, the assembler's optab matching rules need to be
changed. Before, MOVD accepts C_UAUTO32K which matches multiple
of 8 between 0 and 32K, and also C_UAUTO16K, which may not be
multiple of 8 and does not fit into MOVD instruction. The
assembler errors in the latter case. This change makes it only
matches multiple of 8 (or offsets within ±256, which also fits
in instruction), and uses the large-or-unaligned-offset rule
for things doesn't fit (without error). Other sized move rules
are changed similarly.
Class C_UAUTO64K and C_UOREG64K are removed, as they are never
used.
In shared library, load/store of global is rewritten to using
GOT and temp register, which conflicts with the use of temp
register for assembling large offset. So the folding is disabled
for globals in shared library mode.
Reduce cmd/go binary size by 2%.
name old time/op new time/op delta
BinaryTree17-8 8.67s ± 0% 8.61s ± 0% -0.60% (p=0.000 n=9+10)
Fannkuch11-8 6.24s ± 0% 6.19s ± 0% -0.83% (p=0.000 n=10+9)
FmtFprintfEmpty-8 116ns ± 0% 116ns ± 0% ~ (all equal)
FmtFprintfString-8 196ns ± 0% 192ns ± 0% -1.89% (p=0.000 n=10+10)
FmtFprintfInt-8 199ns ± 0% 198ns ± 0% -0.35% (p=0.001 n=9+10)
FmtFprintfIntInt-8 294ns ± 0% 293ns ± 0% -0.34% (p=0.000 n=8+8)
FmtFprintfPrefixedInt-8 318ns ± 1% 318ns ± 1% ~ (p=1.000 n=10+10)
FmtFprintfFloat-8 537ns ± 0% 531ns ± 0% -1.17% (p=0.000 n=9+10)
FmtManyArgs-8 1.19µs ± 1% 1.18µs ± 1% -1.41% (p=0.001 n=10+10)
GobDecode-8 17.2ms ± 1% 17.3ms ± 2% ~ (p=0.165 n=10+10)
GobEncode-8 14.7ms ± 1% 14.7ms ± 2% ~ (p=0.631 n=10+10)
Gzip-8 837ms ± 0% 836ms ± 0% -0.14% (p=0.006 n=9+10)
Gunzip-8 141ms ± 0% 139ms ± 0% -1.24% (p=0.000 n=9+10)
HTTPClientServer-8 256µs ± 1% 253µs ± 1% -1.35% (p=0.000 n=10+10)
JSONEncode-8 40.1ms ± 1% 41.3ms ± 1% +3.06% (p=0.000 n=10+9)
JSONDecode-8 157ms ± 1% 156ms ± 1% -0.83% (p=0.001 n=9+8)
Mandelbrot200-8 8.94ms ± 0% 8.94ms ± 0% +0.02% (p=0.000 n=9+9)
GoParse-8 8.69ms ± 0% 8.54ms ± 1% -1.69% (p=0.000 n=8+10)
RegexpMatchEasy0_32-8 227ns ± 1% 228ns ± 1% +0.48% (p=0.016 n=10+9)
RegexpMatchEasy0_1K-8 1.92µs ± 0% 1.63µs ± 0% -15.08% (p=0.000 n=10+9)
RegexpMatchEasy1_32-8 256ns ± 0% 251ns ± 0% -2.19% (p=0.000 n=10+9)
RegexpMatchEasy1_1K-8 2.38µs ± 0% 2.09µs ± 0% -12.49% (p=0.000 n=10+9)
RegexpMatchMedium_32-8 352ns ± 0% 354ns ± 0% +0.39% (p=0.002 n=10+9)
RegexpMatchMedium_1K-8 106µs ± 0% 106µs ± 0% -0.05% (p=0.005 n=10+9)
RegexpMatchHard_32-8 5.92µs ± 0% 5.89µs ± 0% -0.40% (p=0.000 n=9+8)
RegexpMatchHard_1K-8 180µs ± 0% 179µs ± 0% -0.14% (p=0.000 n=10+9)
Revcomp-8 1.20s ± 0% 1.13s ± 0% -6.29% (p=0.000 n=9+8)
Template-8 159ms ± 1% 154ms ± 1% -3.14% (p=0.000 n=9+10)
TimeParse-8 800ns ± 3% 769ns ± 1% -3.91% (p=0.000 n=10+10)
TimeFormat-8 826ns ± 2% 817ns ± 2% -1.04% (p=0.050 n=10+10)
[Geo mean] 145µs 143µs -1.79%
Change-Id: I5fc42087cee9b54ea414f8ef6d6d020b80eb5985
Reviewed-on: https://go-review.googlesource.com/42172
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
It can be disabled by setting the environment variable
GO19CONCURRENTCOMPILATION=0, or with -gcflags=-c=1.
Fixes#15756.
Change-Id: I7acbf16330512b62ee14ecbab1f46b53ec5a67b6
Reviewed-on: https://go-review.googlesource.com/41820
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
TestGoroutineCounts currently depends on timing to get 100 goroutines
to a known blocking point before taking a profile. This fails
frequently, with different goroutines captured at different stacks.
The test is disabled on openbsd because it was too flaky, but in fact
it flakes on all platforms.
Fix this by using Gosched instead of timing. This is both much more
reliable and makes the test run faster.
Fixes#15156.
Change-Id: Ia6e894196d717655b8fb4ee96df53f6cc8bc5f1f
Reviewed-on: https://go-review.googlesource.com/42953
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This permits the user to override the code generation flag when they
know better. This is always a good policy for all flags automatically
inserted by the build system.
Doing this now so that I can write a test for #20290.
Update #20290
Change-Id: I5c6708a277238d571b8d037993a5a59e2a442e98
Reviewed-on: https://go-review.googlesource.com/42952
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Issue #15228 describes that reserved address blocks should be used for
documentation purposes. This change updates the prefix length so the
IPv4 address adheres to this.
Change-Id: I237d9cce1a71f4fd95f927ec894ce53fa806047f
Reviewed-on: https://go-review.googlesource.com/42991
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Further examples to support the net package.
See issue #5757
Change-Id: I839fd97a468c8d9195e8f4a0ee886ba50ca3f382
Reviewed-on: https://go-review.googlesource.com/42912
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
As necessary, math functions were structured to use stubs, so that they can
be accelerated with assembly on any platform.
Technique used was minimax polynomial approximation using tables of
polynomial coefficients, with argument range reduction.
Benchmark New Old Speedup
BenchmarkAcos 12.2 47.5 3.89
BenchmarkAcosh 18.5 56.2 3.04
BenchmarkAsin 13.1 40.6 3.10
BenchmarkAsinh 19.4 62.8 3.24
BenchmarkAtan 10.1 23 2.28
BenchmarkAtanh 19.1 53.2 2.79
BenchmarkAtan2 16.5 33.9 2.05
BenchmarkCbrt 14.8 58 3.92
BenchmarkErf 10.8 20.1 1.86
BenchmarkErfc 11.2 23.5 2.10
BenchmarkExp 8.77 53.8 6.13
BenchmarkExpm1 10.1 38.3 3.79
BenchmarkLog 13.1 40.1 3.06
BenchmarkLog1p 12.7 38.3 3.02
BenchmarkPowInt 31.7 40.5 1.28
BenchmarkPowFrac 33.1 141 4.26
BenchmarkTan 11.5 30 2.61
Accuracy was tested against a high precision
reference function to determine maximum error.
Note: ulperr is error in "units in the last place"
max
ulperr
Acos 1.15
Acosh 1.07
Asin 2.22
Asinh 1.72
Atan 1.41
Atanh 3.00
Atan2 1.45
Cbrt 1.18
Erf 1.29
Erfc 4.82
Exp 1.00
Expm1 2.26
Log 0.94
Log1p 2.39
Tan 3.14
Pow will have 99.99% correctly rounded results with reasonable inputs
producing numeric (non Inf or NaN) results
Change-Id: I850e8cf7b70426e8b54ec49d74acd4cddc8c6cb2
Reviewed-on: https://go-review.googlesource.com/38585
Reviewed-by: Michael Munday <munday@ca.ibm.com>
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The test "TestTryGrowByResliceInlined" introduced in c08ac36 broke the
noopt builder as it fails when inlining is disabled.
Since there are currently no other options at hand for checking
inlined-ness other than looking at emited symbols of the compilation,
we for now skip the problem causing test by default and only run
it on one specific builder ("linux-amd64").
Also see CL 42813, which introduced the test and contains comments
suggesting this temporary solution.
Change-Id: I3978ab0831da04876cf873d78959f821c459282b
Reviewed-on: https://go-review.googlesource.com/42820
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Also switch "stating" to "statting" to describe applying os.Stat to
a resource; the former is more confusable than the latter.
Change-Id: I9d8e3506bd383f8f1479c05948c03b8c633dc4af
Reviewed-on: https://go-review.googlesource.com/42855
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
In the common case, the grow method only needs to reslice the internal
buffer. Making another function call to grow can be expensive when Write
is called very often with small pieces of data (like a byte or rune).
Thus, we add a tryGrowByReslice method that is inlineable so that we can
avoid an extra call in most cases.
name old time/op new time/op delta
WriteByte-4 35.5µs ± 0% 17.4µs ± 1% -51.03% (p=0.000 n=19+20)
WriteRune-4 55.7µs ± 1% 38.7µs ± 1% -30.56% (p=0.000 n=18+19)
BufferNotEmptyWriteRead-4 304µs ± 5% 283µs ± 3% -6.86% (p=0.000 n=19+17)
BufferFullSmallReads-4 87.0µs ± 5% 66.8µs ± 2% -23.26% (p=0.000 n=17+17)
name old speed new speed delta
WriteByte-4 115MB/s ± 0% 235MB/s ± 1% +104.19% (p=0.000 n=19+20)
WriteRune-4 221MB/s ± 1% 318MB/s ± 1% +44.01% (p=0.000 n=18+19)
Fixes#17857
Change-Id: I08dfb10a1c7e001817729dbfcc951bda12fe8814
Reviewed-on: https://go-review.googlesource.com/42813
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
MOVSD is properly handled but its encoding test wasn't enabled. Enable
it.
For reference this was found with a little tool I wrote [1] to explore
which instructions are missing or not tested in the go obj package and
assembler:
"which SSE2 instructions aren't tested? And don't list instructions
which can take MMX operands"
$ x86db-gogen list --extension SSE2 --not-tested --not-mmx
CLFLUSH mem [m: np 0f ae /7] WILLAMETTE,SSE2
MOVSD xmmreg,xmmreg [rm: f2 0f 10 /r] WILLAMETTE,SSE2
MOVSD xmmreg,xmmreg [mr: f2 0f 11 /r] WILLAMETTE,SSE2
MOVSD mem64,xmmreg [mr: f2 0f 11 /r] WILLAMETTE,SSE2
MOVSD xmmreg,mem64 [rm: f2 0f 10 /r] WILLAMETTE,SSE2
(CLFLUSH was introduced with SSE2, but has its own CPUID bit)
[1] https://github.com/dlespiau/x86db
Change-Id: Ic3af3028cb8d4f02e53fdebb9b30fb311f4ee454
Reviewed-on: https://go-review.googlesource.com/42814
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently windows Stat uses combination of Lstat and Readlink to
walk symlinks until it reaches file or directory. Windows Readlink
is implemented via Windows DeviceIoControl(FSCTL_GET_REPARSE_POINT, ...)
call, but that call does not work on network shares or inside of
Docker container (see issues #18555 ad #19922 for details).
But Raymond Chen suggests different approach:
https://blogs.msdn.microsoft.com/oldnewthing/20100212-00/?p=14963/
- he suggests to use Windows I/O manager to dereferences the
symbolic link.
This appears to work for all normal symlinks, but also for network
shares and inside of Docker container.
This CL implements described procedure.
I also had to adjust TestStatSymlinkLoop, because the test is
expecting Stat to return syscall.ELOOP for symlink with a loop.
But new Stat returns Windows error of ERROR_CANT_RESOLVE_FILENAME
= 1921 instead. I could map ERROR_CANT_RESOLVE_FILENAME into
syscall.ELOOP, but I suspect the former is broader than later.
And ERROR_CANT_RESOLVE_FILENAME message text of "The name of
the file cannot be resolved by the system." sounds fine to me.
Fixes#10935Fixes#18555Fixes#19922
Change-Id: I979636064cdbdb9c7c840cf8ae73fe2c24499879
Reviewed-on: https://go-review.googlesource.com/41834
Reviewed-by: Harshavardhana <hrshvardhana@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
gcc on windows writes invalid location for .debug_gdb_scripts
which causes the executable loading to fail.
Fixes#20183
Change-Id: I5134013bc926b44a55b528f66ab79555855d1f4d
Reviewed-on: https://go-review.googlesource.com/42651
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
As discussion in issue #19141, the addend should be the third
argument of MULA. This patch fixes it in both the front end
and the back end of the assembler. And also tests are added to
the encoding test.
Fixes#19141
Change-Id: Idbc6f338b8fdfcad97a135f27a98c5b375b27d43
Reviewed-on: https://go-review.googlesource.com/42028
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
NewPackage required through documentation that the package name not
be blank (which wasn't true since each time we check a new package
we create one with a blank name (api.go:350). NewPackage also asserted
that a package name not be "_". While it is invalid for a package name
to be "_", one could conceivably create a package named "_" through
export data manipulation. Furthermore, it is ok to import a package
with package path "_" as long as the package itself is not named "_".
- removed misleading documentation
- removed unnecessary assertion
- added safety checks when we actually do the import
Fixes#20231.
Change-Id: I1eb1ab7b5e3130283db715374770cf05d749d159
Reviewed-on: https://go-review.googlesource.com/42852
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Generated hash and eq routines don't need nil checks.
Prior to this CL, this was accomplished by
temporarily incrementing the global variable disable_checknil.
However, that increment lasted only the lifetime of the
call to funccompile. After CL 41503, funccompile may
do nothing but enqueue the function for compilation,
resulting in nil checks being generated.
Fix this by adding an explicit flag to a function
indicating whether nil checks should be disabled
for that function.
While we're here, allow concurrent compilation
with the -w and -W flags, since that was needed
to investigate this issue.
Fixes#20242
Change-Id: Ib9140c22c49e9a09e62fa3cf350f5d3eff18e2bd
Reviewed-on: https://go-review.googlesource.com/42591
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marvin Stenger <marvin.stenger94@gmail.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
The Power processor manual states that "Branches not from the last instruction
of an aligned quadword and not to the first instruction of an aligned quadword
cause inefficiencies in the IBuffer". This changes the function alignment from 8
to 16 bytes to comply with that.
Fixes#18963
Change-Id: Ibce9bf8302110a86c6ab05948569af9ffdfcf4bb
Reviewed-on: https://go-review.googlesource.com/36390
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Allow the predefined escapers "html", "urlquery", and "js" to be used
in pipelines when they have no potential to affect the correctness or
safety of the escaped pipeline output. Specifically:
- "urlquery" may be used if it is the last command in the pipeline.
- "html" may be used if it is the last command in the pipeline, and
the pipeline does not occur in an unquoted HTML attribute value
context.
- "js" may be used in any pipeline, since it does not affect the
merging of contextual escapers.
This change will loosens the restrictions on predefined escapers
introduced in golang.org/cl/37880, which will hopefully ease the
upgrade path for existing template users.
This change brings back the escaper-merging logic, and associated
unit tests, that were removed in golang.org/cl/37880. However, a
few notable changes have been made:
- "_html_template_nospaceescaper" is no longer considered
equivalent to "html", since the former escapes spaces, while
the latter does not (see #19345). This change should not silently
break any templates, since pipelines where this substituion will
happen will already trigger an explicit error.
- An "_eval_args_" internal directive has been added to
handle pipelines containing a single explicit call to a
predefined escaper, e.g. {{html .X}} (see #19353).
Also, the HTMLEscape function called by the predefined
text/template "html" function now escapes the NULL character as
well. This effectively makes it as secure as the internal
html/template HTML escapers (see #19345). While this change is
backward-incompatible, it will only affect illegitimate uses
of this escaper, since the NULL character is always illegal in
valid HTML.
Fixes#19952
Change-Id: I9b5570a80a3ea284b53901e6a1f842fc59b33d3a
Reviewed-on: https://go-review.googlesource.com/40936
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Execute incurs separate writes for each "step", e.g. each
variable that needs to be printed, and the final newline.
While it is correct to state that templates can be executed
concurrently, there is a more subtle nuance that is easily missed:
when writing to the same writer, the writes from concurrent execute
calls can be interleaved, leading to unexpected output.
Change-Id: I0abbd7960d8a8d15e109a8a3eeff3b43b852bbbf
Reviewed-on: https://go-review.googlesource.com/37444
Reviewed-by: Rob Pike <r@golang.org>
The external darwin linker has been printing:
ld: warning: -read_only_relocs cannot be used with x86_64
for a long time. Now that it is printed by CL 33301, we may as
well get rid of it.
Fixes#20246
Change-Id: I1147cf1ff197fdfda228a1349f13627bcf9fc72f
Reviewed-on: https://go-review.googlesource.com/42730
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Todd Neal <todd@tneal.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
For .bss section symbol ldelf does not set P (raw symbol data).
Make ldpe do the same.
Change-Id: Ib3d558456f505ee568d0972465fa9b08b5794a87
Reviewed-on: https://go-review.googlesource.com/42631
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If we've already complained about a type T,
don't complain again about further expressions
involving it.
Fixes#20245 and hopefully all of its ilk.
Change-Id: Ic0abe8235d52e8a7ac40e3615aea8f3a54fd7cec
Reviewed-on: https://go-review.googlesource.com/42690
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
We already set it for mips32 objects. The native ELF linker warns when
linking PIC objects with non-PIC objects. Our objects are PIC, but we
were not marking them as such.
Fixes#20243.
Change-Id: Ifab131200b263e4c72cf81f7b131a65ac02a13a9
Reviewed-on: https://go-review.googlesource.com/42710
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change passes runtime.Version from the go tool to the compiler.
If the versions do not match, the compilation fails.
The result is a go tool from one GOROOT will complain loudly if it
is invoked with a different GOROOT value.
Only release versions are checked, so that when developing Go
you can still use "go install cmd/go" and "go install cmd/compile"
separately.
Fixes#19064
Change-Id: I17e184d07d3c1092b1d9af53ba55ed3ecf67791d
Reviewed-on: https://go-review.googlesource.com/42595
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Before this change, building a GOROOT using make.bash, and then
moving the entire to a new path confused the go tool. Correct
operation of the go tool under these conditions required either
running make.bash again (not always possible if the new location
was owned by a different system user) or setting the GOROOT
environment variable. Setting GOROOT is unfortunate and
discouraged, as it makes it too easy to use the go tool from
one GOROOT and the compiler from another GOROOT.
With this change, the go tool finds its GOROOT relative to its
own location, using os.Executable. It checks it is in a GOROOT
by searching for the GOROOT/pkg/tool directory, to avoid two
plausible situations:
ln -s $GOROOT/bin/go /usr/local/bin/go
and
PATH=$HOME/bin:$PATH
GOPATH=$HOME
ln -s $GOROOT/bin/go $HOME/bin/go
Additionally, if the current executable path is not in a GOROOT,
the tool will follow any symlinks for the executable and check
to see if its original path is a GOROOT.
Fixes#18678
Change-Id: I151d7d449d213164f98193cc176b616849e6332c
Reviewed-on: https://go-review.googlesource.com/42533
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Compile:
package p
var f = func(...A)
Before this CL:
x.go:3:13: type %!v(PANIC=runtime error: invalid memory address or nil pointer dereference) is not an expression
x.go:3:17: undefined: A
After this CL:
x.go:3:13: type func(...<T>) is not an expression
x.go:3:17: undefined: A
Found with go-fuzz.
Fixes#20233
Change-Id: Ibb232b3954c4091071440eba48b44c4022a8083f
Reviewed-on: https://go-review.googlesource.com/42610
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This change explicitly documents that DES, MD5, RC4 and SHA-1 are
insecure / broken - at all or at least within a commonly used scenario.
Fixes#14395
Change-Id: Id1d543c85d67968ba64ed7495313501953c3ef3a
Reviewed-on: https://go-review.googlesource.com/42511
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add the ability to override the default file and directory from
which certificates are loaded by setting the OpenSSL compatible
environment variables: SSL_CERT_FILE, SSL_CERT_DIR.
If the variables are set the default locations are not checked.
Added new default file "/usr/local/etc/ssl/cert.pem" for FreeBSD.
Certificates in the first valid location found for both file and
directory are added, instead of only the first file location if
a valid one was found, which is consistent with OpenSSL.
Fixes#3905Fixes#14022Fixes#14311Fixes#16920Fixes#18813 - If user sets SSL_CERT_FILE.
Change-Id: Ia24fb7c1c2ffff4338b4cf214bd040326ce27bb0
Reviewed-on: https://go-review.googlesource.com/36093
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When compiling the program:
package p
func _(){
*;:=
}
Before:
x.go:4:3: syntax error: unexpected semicolon, expecting expression
x.go:4:4: non-name *%!v(PANIC=runtime error: invalid memory address or nil pointer dereference) on left side of :=
x.go:5:1: syntax error: unexpected }, expecting expression
After:
x.go:4:3: syntax error: unexpected semicolon, expecting expression
x.go:4:4: non-name *<N> on left side of :=
x.go:5:1: syntax error: unexpected }, expecting expression
No test because:
(1) we don't have a good mechanism to check for the
absence of the string "PANIC" in an error message
(2) the string "*<N>", while better, is itself ugly enough
that I don't want to actively check for it
(3) the bug isn't very important, the kind of thing only fuzzers encounter
(4) the fix is obvious and trivial
Fixes#20220
Change-Id: I35faa986b60b671414ee999d6264b06937f250e3
Reviewed-on: https://go-review.googlesource.com/42498
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Because the hint parameter is supposed to be treated
purely as a hint, if it doesn't meet the requirements
we disregard it and continue as if there was no hint
at all.
Fixes#19926
Change-Id: I86e7f99472fad6b99ba4e2fd33e4a9e55d55115e
Reviewed-on: https://go-review.googlesource.com/40854
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This CL modifies how MOV[DWHB] instructions that store a constant to
memory are assembled to avoid them clobbering the condition code
(flags). It also modifies zeroAuto to use MOVD instructions instead of
CLEAR (which is assembled as XC).
MOV[DWHB]storeconst ops also no longer clobbers flags.
Note: this CL modifies the assembler so that it can no longer handle
immediates outside the range of an int16 or offsets from SB, which
reflects what the machine instructions support. The compiler doesn't
need this capability any more and I don't think this affects any existing
assembly, but it is easy to workaround if it does.
Fixes#20187.
Change-Id: Ie54947ff38367bd6a19962bf1a6d0296a4accffb
Reviewed-on: https://go-review.googlesource.com/42179
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
5 shards, each of which spins up NumCPU processes,
each of which is running at GOMAXPROCS=NumCPU,
is too much for one machine. It makes my laptop unusable.
It might also be in part responsible for test flakes
that require a moderately responsive system,
like #18589 (backedge scheduling) and #19276 (locklinear).
It's possible that Go should be a better neighbor in general;
that's #17969. In the meantime, fix this corner of the world.
Builders snapshot the world and run shards on different
machines, so keeping sharding high for them is good.
This is a partial reversion of CL 18199.
Fixes#20141.
Change-Id: I123cf9436f4f4da3550372896265c38117b78071
Reviewed-on: https://go-review.googlesource.com/42431
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Prior to this CL, the compiler and assembler
were sloppy about the LSym.Type for LSyms
containing static data.
The linker then fixed this up, converting
Sxxx and SBSS to SDATA, and SNOPTRBSS to SNOPTRDATA
if it noticed that the symbol had associated data.
It is preferable to just get this right in cmd/compile
and cmd/asm, because it removes an unnecessary traversal
of the symbol table from the linker (see #14624).
Do this by touching up the LSym.Type fixes in
LSym.prepwrite and Link.Globl.
I have confirmed by instrumenting the linker
that the now-eliminated code paths were unreached.
And an additional check in the object file writing code
will help preserve that invariant.
There was a case in the Windows linker,
with internal linking and cgo,
where we were generating SNOPTRBSS symbols with data.
For now, convert those at the site at which they occur
into SNOPTRDATA, just like they were.
Does not pass toolstash-check,
but does generate identical linked binaries.
No compiler performance changes.
Change-Id: I77b071ab103685ff8e042cee9abb864385488872
Reviewed-on: https://go-review.googlesource.com/40864
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Changes all cpu features to be detected and stored in bools in rt0_go.
Updates: #15403
Change-Id: I5a9961cdec789b331d09c44d86beb53833d5dc3e
Reviewed-on: https://go-review.googlesource.com/41950
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
Reviewed-by: Keith Randall <khr@golang.org>
DUFFZERO on 386 is not marked as clobbering flags, but rewriteToUseGot rewrote
"ADUFFZERO $offset" to "MOVL runtime.duffxxx@GOT, CX; ADDL $offset, CX; CALL CX"
which does. Luckily the fix is easier than figuring out what the problem was:
replace the ADDL $offset, CX with LEAL $offset(CX), CX.
On amd64 DUFFZERO clobbers flags, on arm, arm64 and ppc64 ADD does not clobber
flags and s390x does not use the duff functions, so I'm fairly confident this
is the only fix required.
I don't know how to write a test though.
Change-Id: I69b0958f5f45771d61db5f5ecb4ded94e8960d4d
Reviewed-on: https://go-review.googlesource.com/41821
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
ANDPS, like all others PS (Packed Single precision floats) instructions,
need Ym: they don't use the 0x66 prefix.
From the manual:
NP 0F 54 /r ANDPS xmm1, xmm2/m128
NP meaning, quoting the manual:
NP - Indicates the use of 66/F2/F3 prefixes (beyond those already part
of the instructions opcode) are not allowed with the instruction.
And indeed, the same instruction prefixed by 0x66 is ANDPD.
Updates #14069
Change-Id: If312a6f1e77113ab8c0febe66bdb1b4171e41e0a
Reviewed-on: https://go-review.googlesource.com/42090
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We generate code that calls each user init function one at a time.
When there are lots of user init functions,
usually due to generated code, like test/rotate* or
github.com/juju/govmomi/vim25/types,
we can end up with a giant function,
which can be slow to compile.
This CL puts in an escape valve.
When there are more than 500 functions, instead of doing:
init.0()
init.1()
// ...
we construct a static array of functions:
var fns = [...]func(){init.0, init.1, ... }
and call them in a loop.
This generates marginally bigger, marginally worse code,
so we restrict it to cases in which it might start to matter.
500 was selected as a mostly arbitrary threshold for "lots".
Each call uses two Progs, one for PCDATA and one for the call,
so at 500 calls we use ~1000 Progs.
At concurrency==8, we get a Prog cache of about
1000 Progs per worker.
So a threshold of 500 should more or less avoid
exhausting the Prog cache in most cases.
Change-Id: I276b887173ddbf65b2164ec9f9b5eb04d8c753c2
Reviewed-on: https://go-review.googlesource.com/41500
Reviewed-by: Keith Randall <khr@golang.org>
overLoadFactor used a uintptr for its calculations.
When the number of potential buckets was large,
perhaps due to a coding error or corrupt/malicious user input
leading to a very large map size hint,
this led to overflow on 32 bit systems.
This overflow resulted in an infinite loop.
Prevent it by always using a 64 bit calculation.
Updates #20195
Change-Id: Iaabc710773cd5da6754f43b913478cc5562d89a2
Reviewed-on: https://go-review.googlesource.com/42185
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Noticed while adding to the bitset implementation
in cmd/compile/internal/gc.
The (Com (Const)) optimizations were already present
in the AMD64 lowered optimizations.
They trigger 118, 44, 262, and 108 times
respectively for int sizes 8, 16, 32, and 64
in a run of make.bash.
The (Or (And)) optimization is new.
It triggers 3 times for int size 8
and once for int size 64 during make.bash,
in packages internal/poll, reflect,
encoding/asn1, and go/types,
so there is a bit of natural test coverage.
Change-Id: I44072864ff88831d5ec7dce37c516d29df056e98
Reviewed-on: https://go-review.googlesource.com/41758
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Per gri's suggestion on CL 41623,
add a comment to trackAllTypes
about the trade-offs of enabling it.
Change-Id: Iec42b0da7933543200729003d1b2c6e0d9dcc5f0
Reviewed-on: https://go-review.googlesource.com/42186
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The generated test cases had their arguments reversed, putting them back
in order makes those tests pass.
CMPPS SRC, DEST, CC
Change-Id: Ie15021edc533d5681a6a78d10d88b665e3de9017
Reviewed-on: https://go-review.googlesource.com/42097
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I want to move the SHIDDEN type bit into Attribute, but AttrHidden is already
there and means something completely different, so rename it. (I'll give the
SHIDDEN bit a better name when it moves too).
Change-Id: I075403d9542b7626d4c1f6db9094329c4181aad3
Reviewed-on: https://go-review.googlesource.com/42024
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
since there current version suffers from a copy'n'paste error.
Change-Id: I1ef97a4413198dde10c5c926a2a9806724cd6d44
Reviewed-on: https://go-review.googlesource.com/42095
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Without this, T can sneak through to the backend
with its width unknown.
Fixes#20174
Change-Id: I9b21e0e2641f75e360cc5e45dcb4eefe8255b675
Reviewed-on: https://go-review.googlesource.com/42175
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Fixes#7121.
CL 31173 inadvertently fixed this issue in the Go1.8 cycle.
This CL enables the test for that issue.
Change-Id: I8e73b41f646fa89c75bff833c1b07e1cdb2b1e62
Reviewed-on: https://go-review.googlesource.com/42149
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently Go sets the system-wide timer resolution to 1ms the whole
time it's running. This has negative affects on system performance and
power consumption. Unfortunately, simply reducing the timer resolution
to the default 15ms interferes with several sleeps in the runtime
itself, including sysmon's ability to interrupt goroutines.
This commit takes a hybrid approach: it only reduces the timer
resolution when the Go process is entirely idle. When the process is
idle, nothing needs a high resolution timer. When the process is
non-idle, it's already consuming CPU so it doesn't really matter if
the OS also takes timer interrupts more frequently.
Updates #8687.
Change-Id: I0652564b4a36d61a80e045040094a39c19da3b06
Reviewed-on: https://go-review.googlesource.com/38403
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
(ADDconst [c] x) && !isARMImmRot(uint32(c)) && isARMImmRot(uint32(-c)) -> (SUBconst [int64(int32(-c))] x)
(SUBconst [c] x) && !isARMImmRot(uint32(c)) && isARMImmRot(uint32(-c)) -> (ADDconst [int64(int32(-c))] x)
Currently
a = a + 0xfffffff1 is compiled to （variable a is in R0)
MVN $14, R11
ADD R11, R0, R0
After applying the above 2 rules, it becomes
SUB $15, R0, R0
(BICconst [c] (BICconst [d] x)) -> (BICconst [int64(int32(c|d))] x)
This rule also optimizes the generated ARM code.
The other rules are added to avoid to generate less optimized ARM code
when substitutions ADD->SUB happen.
Change-Id: I3ead9aae2b446b674e2ab42d37259d38ceb93a4d
Reviewed-on: https://go-review.googlesource.com/41679
Reviewed-by: Keith Randall <khr@golang.org>
Instead of playing whack-a-mole finding all
the non-dowidth'd expressions that can sneak
out of the frontend and then deciding on
just the right place to handle them,
use a big hammer.
Fixes#20152
Change-Id: Id452d9e8c4e9585216bd8bf0e0004c85aba4f9f7
Reviewed-on: https://go-review.googlesource.com/42021
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently the pprof tests re-symbolize PCs in profiles, and do so in a
way that can't handle inlining. Proto profiles already contain full
symbol information, so this modifies the tests to use the symbol
information already present in the profile.
Change-Id: I63cd491de7197080fd158b1e4f782630f1bbbb56
Reviewed-on: https://go-review.googlesource.com/41255
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
When running a Query on Stmt a dependency is added to the stmt and
rows. To do that it needs a reference to Rows, so the releaseConn
function is defined after the definition. However the
rows.initContextClose was set to run before the releaseConn was
set on rows, setting up a situation where the connection could
be canceled before the releaseConn was set and resulting in
a segfault.
Fixes#20160
Change-Id: I5592e7db2cf653dfc48d42cbc2b03ca20501b1a0
Reviewed-on: https://go-review.googlesource.com/42139
Run-TryBot: Daniel Theophanes <kardianos@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently _TinySizeClass is untyped, which means it can accidentally
be used as a spanClass (not that I would know this from experience or
anything). Make it an int8 to avoid this mix up.
This is a cherry-pick of dev.garbage commit 81b74bf9c5.
Change-Id: I1e69eccee436ea5aa45e9a9828a013e369e03f1a
Reviewed-on: https://go-review.googlesource.com/41254
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
This is no longer necessary now that we can more efficiently consult
the span's noscan bit.
This is a cherry-pick of dev.garbage commit 312aa09996.
Change-Id: Id0b00b278533660973f45eb6efa5b00f373d58af
Reviewed-on: https://go-review.googlesource.com/41252
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Currently, we mix objects with pointers and objects without pointers
("noscan" objects) together in memory. As a result, for every object
we grey, we have to check that object's heap bits to find out if it's
noscan, which adds to the per-object cost of GC. This also hurts the
TLB footprint of the garbage collector because it decreases the
density of scannable objects at the page level.
This commit improves the situation by using separate spans for noscan
objects. This will allow a much simpler noscan check (in a follow up
CL), eliminate the need to clear the bitmap of noscan objects (in a
follow up CL), and improves TLB footprint by increasing the density of
scannable objects.
This is also a step toward eliminating dead bits, since the current
noscan check depends on checking the dead bit of the first word.
This has no effect on the heap size of the garbage benchmark.
We'll measure the performance change of this after the follow-up
optimizations.
This is a cherry-pick from dev.garbage commit d491e550c3. The only
non-trivial merge conflict was in updatememstats in mstats.go, where
we now have to separate the per-spanclass stats from the per-sizeclass
stats.
Change-Id: I13bdc4869538ece5649a8d2a41c6605371618e40
Reviewed-on: https://go-review.googlesource.com/41251
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
In particular, this says that Frames.Function uniquely identifies a
function within a program. We depend on this in various places that
use runtime.Frames in std, but it wasn't actually written down.
Change-Id: Ie7ede348c17673e11ae513a094862b60c506abc5
Reviewed-on: https://go-review.googlesource.com/41610
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This reverts commit 3d86d45dd6.
Reason for revert: Fixing a different way. See #20008.
Change-Id: I6f4382ffdfff83a4504b013ba37100319b2c1aad
Reviewed-on: https://go-review.googlesource.com/42138
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The code in #20162 contains an embedded interface.
It didn't get dowidth'd by the frontend,
and during DWARF generation, ngotype asked
for a string description of it,
which triggered a request for the number of fields
in the interface, which triggered a dowidth,
which is disallowed in the backend.
The other changes in this CL are to support the test.
Fixes#20162
Change-Id: I4d0be5bd949c361d4cdc89a8ed28b10977e40cf9
Reviewed-on: https://go-review.googlesource.com/42131
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Now that it only takes small values.
Change-Id: I08086d392529d8775b470d65afc2475f8d0e7f4a
Reviewed-on: https://go-review.googlesource.com/42030
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It used to be simple, and then it got complicated for speed (to reduce
allocations, mostly), but that involved a mutex and hurt multi-core
performance, contending on the mutex.
A change was sent to try to improve that mutex contention in
https://go-review.googlesource.com/c/42110/2/src/net/http/server.go
but that introduced its own allocations (the string->interface{}
boxing for the sync.Map key), which runs counter to the whole point of
that statusLine function: to remove allocations.
Instead, make the code simple again and not have a mutex. It's a bit
slower for the single-core case, but nobody with a single-user HTTP
server cares about 50 nanoseconds:
name old time/op new time/op delta
ResponseStatusLine 37.5ns ± 2% 87.1ns ± 2% +132.42% (p=0.029 n=4+4)
ResponseStatusLine-2 63.1ns ± 1% 43.1ns ±12% -31.67% (p=0.029 n=4+4)
ResponseStatusLine-4 53.8ns ± 8% 40.2ns ± 2% -25.29% (p=0.029 n=4+4)
name old alloc/op new alloc/op delta
ResponseStatusLine 0.00B ±NaN% 0.00B ±NaN% ~ (all samples are equal)
ResponseStatusLine-2 0.00B ±NaN% 0.00B ±NaN% ~ (all samples are equal)
ResponseStatusLine-4 0.00B ±NaN% 0.00B ±NaN% ~ (all samples are equal)
name old allocs/op new allocs/op delta
ResponseStatusLine 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal)
ResponseStatusLine-2 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal)
ResponseStatusLine-4 0.00 ±NaN% 0.00 ±NaN% ~ (all samples are equal)
(Note the code could be even simpler with fmt.Fprintf, but that is
relatively slow and involves a bunch of allocations getting arguments
into interface{} for the call)
Change-Id: I1fa119132dbbf97a8e7204ce3e0707d433060da2
Reviewed-on: https://go-review.googlesource.com/42133
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Using 'go build -tags "foo,bar"' might seem to work when you wanted
-tags "foo bar", since they make up a single tag that doesn't exist and
the build is unaffected.
Instead, error on any tag that contains a comma.
Fixes#18800.
Change-Id: I6641e03e2ae121c8878d6301c4311aef97026b73
Reviewed-on: https://go-review.googlesource.com/41951
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The comment for Cmd.Stdout and Cmd.Stderr says that it's safe to
set both to the same writer, but it doesn't say that this only
works when both writers are comparable.
This change updates the comment to explain that using a
non-comparable writer may still lead to a race.
Fixes#19804
Change-Id: I63b420034666209a2b6fab48b9047c9d07b825e2
Reviewed-on: https://go-review.googlesource.com/42052
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Profile labels added by the user using pprof.Do, if present will
be in a *labelMap stored in the unsafe.Pointer 'tag' field of
the profile map entry. This change extracts the labels from the tag
field and writes them to the profile proto.
Change-Id: Ic40fdc58b66e993ca91d5d5effe0e04ffbb5bc46
Reviewed-on: https://go-review.googlesource.com/39613
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
If g1 sets its labels and then they are copied into a profile buffer
and then g2 reads the profile buffer and inspects the labels,
the race detector must understand that g1's recording of the labels
happens before g2's use of the labels. Make that so.
Fixes race test failure in CL 39613.
Change-Id: Id7cda1c2aac6f8eef49213b5ca414f7154b4acfa
Reviewed-on: https://go-review.googlesource.com/42111
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
More specifically, allow Unicode letters in the directories of GitHub
repositories, which can occur and don't have a valid reason to be
disallowed by go get.
Do so by using a predefined character class, the Unicode character
property class \p{L} that describes the Unicode characters that are
letters:
http://www.regular-expressions.info/unicode.html#category
Since it's not possible to create GitHub usernames or repositories
containing Unicode letters at this time, those parts of the import path
are still restricted to ASCII letters only.
Fix name of tested func in t.Errorf messages.
Fixes#18660.
Change-Id: Ia0ef4742bfd8317d989ef1eb1d7065e382852fe2
Reviewed-on: https://go-review.googlesource.com/41822
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is the same technique used in CL 24466. By adding a little bit of
size to the binary, we can remove a function call and gain a lot of
performance.
A raw array ([128]bool) would be faster, but is also be 128 bytes
instead of 16.
Running tip on a Mac:
name old time/op new time/op delta
QuoteMetaAll-4 192ns ±12% 120ns ±11% -37.27% (p=0.000 n=10+10)
QuoteMetaNone-4 186ns ± 6% 64ns ± 6% -65.52% (p=0.000 n=10+10)
name old speed new speed delta
QuoteMetaAll-4 73.2MB/s ±11% 116.6MB/s ±10% +59.21% (p=0.000 n=10+10)
QuoteMetaNone-4 139MB/s ± 6% 405MB/s ± 6% +190.74% (p=0.000 n=10+10)
Change-Id: I68ce9fe2ef1c28e2274157789b35b0dd6ae3efb5
Reviewed-on: https://go-review.googlesource.com/41495
Run-TryBot: Kevin Burke <kev@inburke.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Use it to ensure that dowidth is not called
from the backend on a type whose size
has not yet been calculated.
This is an alternative to CL 42016.
Change-Id: I8c7b4410ee4c2a68573102f6b9b635f4fdcf392e
Reviewed-on: https://go-review.googlesource.com/42018
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Now it's not very different from syscall_dragonfly.go in
golang.org/x/sys/unix repository.
Change-Id: I8dfd22e1ebce9dc2cc71ab9ab7f0c92d93b2b762
Reviewed-on: https://go-review.googlesource.com/41835
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL mainly moves some work to the switch on w.Op,
to make a follow-up change simpler and clearer.
Updates #19838
Change-Id: I86f3181c380dd60960afcc24224f655276b8956c
Reviewed-on: https://go-review.googlesource.com/42010
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Node.Used was written to from the backend
concurrently with reads of Node.Class
for the same ONAME Nodes.
I do not know why it was not failing consistently
under the race detector, but it is a race.
This is likely also a problem with Node.HasVal and Node.HasOpt.
They will be handled in a separate CL.
Fix Used by moving it to gc.Name and making it a separate bool.
There was one non-Name use of Used, marking OLABELs as used.
That is no longer needed, now that goto and label checking
happens early in the front end.
Leave the getters and setters in place,
to ease changing the representation in the future
(or changing to an interface!).
Updates #20144
Change-Id: I9bec7c6d33dcb129a4cfa9d338462ea33087f9f7
Reviewed-on: https://go-review.googlesource.com/42015
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Type.Size and Type.Alignment are for the front end:
They calculate size and alignment if needed.
Type.MustSize and Type.MustAlignment are for the back end:
They call Fatal if size and alignment are not already calculated.
Most uses are of MustSize and MustAlignment,
but that's because the back end is newer,
and this API was added to support it.
This CL was mostly generated with sed and selective reversion.
The only mildly interesting bit is the change of the ssa.Type interface
and the supporting ssa dummy types.
Follow-up to review feedback on CL 41970.
Passes toolstash-check.
Change-Id: I0d9b9505e57453dae8fb6a236a07a7a02abd459e
Reviewed-on: https://go-review.googlesource.com/42016
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
dowidth is fundamentally unsafe to call from the back end;
it will cause data races.
Replace all calls to dowidth in the backend with
assertions that the width has been calculated.
Then fix all the cases in which that was not so,
including the cases from #20145.
Fixes#20145.
Change-Id: Idba3d19d75638851a30ec2ebcdb703c19da3e92b
Reviewed-on: https://go-review.googlesource.com/41970
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Many (most!) of the values of objapi.SymKind are used only in the linker, so
this creates a separate cmd/link/internal/ld.SymKind type, removes most values
from SymKind and maps one to the other when reading object files in the linker.
Two of the remaining objapi.SymKind values are only checked for, never set and
so will never be actually found but I wanted to keep this to the most
mechanical change possible.
Change-Id: I4bbc5aed6713cab3e8de732e6e288eb77be0474c
Reviewed-on: https://go-review.googlesource.com/40985
Run-TryBot: Michael Hudson-Doyle <michael.hudson@canonical.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Currently, the following two codes generate the identical dwarf info
for type Foo.
prog 1)
type Foo struct {
Bar
}
prog 2)
type Foo struct {
Bar Bar
}
This change adds a go-specific attribute DW_AT_go_embedded_field
to annotate each member entry. Its absence or false value indicates
the corresponding member is not an embedded field.
Update #20037
Change-Id: Ibcbd2714f3e4d97c7b523d7398f29ab2301cc897
Reviewed-on: https://go-review.googlesource.com/41873
Reviewed-by: David Chase <drchase@google.com>
There's been one failure on the race builder so far,
before we started sorting functions by length.
The race detector can only detect actual races,
and ordering functions by length might reduce the odds
of catching some kinds of races. Give it more to chew on.
Updates #20144
Change-Id: I0206ac182cb98b70a729dea9703ecb0fef54d2d0
Reviewed-on: https://go-review.googlesource.com/41973
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Its sole use is in walk.go. 100% code movement.
gsubr.go increasingly contains backend-y things.
With a few more relocations, it could probably be
fruitfully renamed progs.go.
Change-Id: I61ec5c2bc1f8cfdda64c6d6f580952c154ff60e0
Reviewed-on: https://go-review.googlesource.com/41972
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
They were used only in esc.go. 100% code movement.
Also, remove the rather outdated comment at the top of gen.go.
It's not really clear what gen.go is for any more.
Change-Id: Iaedfe7015ef6f5c11c49f3e6721b15d779a00faa
Reviewed-on: https://go-review.googlesource.com/41971
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When a constant doesn't fit in a single instruction, use two
paired instructions instead of the constant pool. For example
ADD $0xaa00bb, R0, R1
Used to rewrite to:
MOV ?(IP), R11
ADD R11, R0, R1
Instead, do:
ADD $0xaa0000, R0, R1
ADD $0xbb, R1, R1
Same number of instructions.
Good:
4 less bytes (no constant pool entry)
One less load.
Bad:
Critical path is one instruction longer.
It's probably worth it to avoid the loads, they are expensive.
Dave Cheney got us some performance numbers: https://perf.golang.org/search?q=upload:20170426.1
TL;DR mean 1.37% improvement.
Change-Id: Ib206836161fdc94a3962db6f9caa635c87d57cf1
Reviewed-on: https://go-review.googlesource.com/41612
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The current code treats condition as special register and write
its raw data directly into instruction.
The fix converts the raw data into correct condition encoding.
Also fix the operand catogery of FCCMP.
Add tests to cover all cases.
Change-Id: Ib194041bd9017dd0edbc241564fe983082ac616b
Reviewed-on: https://go-review.googlesource.com/41511
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Previously, the package did not distinguish between baseline and
extended sequential images. Both are non-progressive images, but the Th
range differs between the two, as per Annex B of
https://www.w3.org/Graphics/JPEG/itu-t81.pdf
Extended sequential images are often emitted by the Guetzli encoder.
Fixes#19913
Change-Id: I3d0f9e16d5d374ee1c65e3a8fb87519de61cff94
Reviewed-on: https://go-review.googlesource.com/41831
Reviewed-by: David Symonds <dsymonds@golang.org>
When using a concurrent backend,
the overall compilation time is bounded
in part by the slowest function to compile.
The number of top-level statements in a function
is an easily calculated and fairly reliable
proxy for compilation time.
Here's a standard compilecmp output for -c=8 with this CL:
name old time/op new time/op delta
Template 127ms ± 4% 125ms ± 6% -1.33% (p=0.000 n=47+50)
Unicode 84.8ms ± 4% 84.5ms ± 4% ~ (p=0.217 n=49+49)
GoTypes 289ms ± 3% 287ms ± 3% -0.78% (p=0.002 n=48+50)
Compiler 1.36s ± 3% 1.34s ± 2% -1.29% (p=0.000 n=49+47)
SSA 2.95s ± 3% 2.77s ± 4% -6.23% (p=0.000 n=50+49)
Flate 70.7ms ± 3% 70.9ms ± 2% ~ (p=0.112 n=50+49)
GoParser 85.0ms ± 3% 83.0ms ± 4% -2.31% (p=0.000 n=48+49)
Reflect 229ms ± 3% 225ms ± 4% -1.83% (p=0.000 n=49+49)
Tar 70.2ms ± 3% 69.4ms ± 3% -1.17% (p=0.000 n=49+49)
XML 115ms ± 7% 114ms ± 6% ~ (p=0.158 n=49+47)
name old user-time/op new user-time/op delta
Template 352ms ± 5% 342ms ± 8% -2.74% (p=0.000 n=49+50)
Unicode 117ms ± 5% 118ms ± 4% +0.88% (p=0.005 n=46+48)
GoTypes 986ms ± 3% 980ms ± 4% ~ (p=0.110 n=46+48)
Compiler 4.39s ± 2% 4.43s ± 4% +0.97% (p=0.002 n=50+50)
SSA 12.0s ± 2% 13.3s ± 3% +11.33% (p=0.000 n=49+49)
Flate 222ms ± 5% 219ms ± 6% -1.56% (p=0.002 n=50+50)
GoParser 271ms ± 5% 268ms ± 4% -0.83% (p=0.036 n=49+48)
Reflect 560ms ± 4% 571ms ± 3% +1.90% (p=0.000 n=50+49)
Tar 183ms ± 3% 183ms ± 3% ~ (p=0.903 n=45+50)
XML 364ms ±13% 391ms ± 4% +7.16% (p=0.000 n=50+40)
A more interesting way of viewing the data is by
looking at the ratio of the time taken to compile
the slowest-to-compile function to the overall
time spent compiling functions.
If this ratio is small (near 0), then increased concurrency might help.
If this ratio is big (near 1), then we're bounded by that single function.
I instrumented the compiler to emit this ratio per-package,
ran 'go build -a -gcflags=-c=C -p=P std cmd' three times,
for varying values of C and P,
and collected the ratios encountered into an ASCII histogram.
Here's c=1 p=1, which is a non-concurrent backend, single process at a time:
90%|
80%|
70%|
60%|
50%|
40%|
30%|
20%|**
10%|***
0%|*********
----+----------
|0123456789
The x-axis is floor(10*ratio), so the first column indicates the percent of
ratios that fell in the 0% to 9.9999% range.
We can see in this histogram that more concurrency will help;
in most cases, the ratio is small.
Here's c=8 p=1, before this CL:
90%|
80%|
70%|
60%|
50%|
40%|
30%| *
20%| *
10%|* * *
0%|**********
----+----------
|0123456789
In 30-40% of cases, we're mostly bound by the compilation time
of a single function.
Here's c=8 p=1, after this CL:
90%|
80%|
70%|
60%|
50%| *
40%| *
30%| *
20%| *
10%| *
0%|**********
----+----------
|0123456789
The sorting pays off; we are bound by the
compilation time of a single function in over half of packages.
The single * in the histogram indicates 0-10%.
The actual values for this chart are:
0: 5%, 1: 1%, 2: 1%, 3: 4%, 4: 5%, 5: 7%, 6: 7%, 7: 7%, 8: 9%, 9: 55%
This indicates that efforts to increase or enable more concurrency,
e.g. by optimizing mutexes or increasing the value of c,
will probably not yield fruit.
That matches what compilecmp tells us.
Further optimization efforts should thus focus instead on one of:
(1) making more functions compile concurrently
(2) improving the compilation time of the slowest functions
(3) speeding up the remaining serial parts of the compiler
(4) automatically splitting up some large autogenerated functions
into small ones, as discussed in #19751
I hope to spend more time on (1) before the freeze.
Adding process parallelism doesn't change the story much.
For example, here's c=8 p=8, after this CL:
90%|
80%|
70%|
60%|
50%|
40%| *
30%| *
20%| *
10%| ***
0%|**********
----+----------
|0123456789
Since we don't need to worry much about p,
these histograms can help us select a good
general value of c to use as a default,
assuming we're not bounded by GOMAXPROCS.
Here are some charts after this CL, for c from 1 to 8:
c=1 p=1
90%|
80%|
70%|
60%|
50%|
40%|
30%|
20%|**
10%|***
0%|*********
----+----------
|0123456789
c=2 p=1
90%|
80%|
70%|
60%|
50%|
40%|
30%|
20%|
10%| **** *
0%|**********
----+----------
|0123456789
c=3 p=1
90%|
80%|
70%|
60%|
50%|
40%|
30%|
20%| *
10%| ** * *
0%|**********
----+----------
|0123456789
c=4 p=1
90%|
80%|
70%|
60%|
50%|
40%|
30%| *
20%| *
10%| * *
0%|**********
----+----------
|0123456789
c=5 p=1
90%|
80%|
70%|
60%|
50%|
40%|
30%| *
20%| *
10%| * *
0%|**********
----+----------
|0123456789
c=6 p=1
90%|
80%|
70%|
60%|
50%|
40%| *
30%| *
20%| *
10%| *
0%|**********
----+----------
|0123456789
c=7 p=1
90%|
80%|
70%|
60%|
50%| *
40%| *
30%| *
20%| *
10%| **
0%|**********
----+----------
|0123456789
c=8 p=1
90%|
80%|
70%|
60%|
50%| *
40%| *
30%| *
20%| *
10%| *
0%|**********
----+----------
|0123456789
Given the increased user-CPU costs as
c increases, it looks like c=4 is probably
the sweet spot, at least for now.
Pleasingly, this matches (and explains)
the results of the standard benchmarking
that I have done.
Updates #15756
Change-Id: I82b606c06efd34a5dbd1afdbcf66a605905b2aeb
Reviewed-on: https://go-review.googlesource.com/41192
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL adds initial support for concurrent backend compilation.
BACKGROUND
The compiler currently consists (very roughly) of the following phases:
1. Initialization.
2. Lexing and parsing into the cmd/compile/internal/syntax AST.
3. Translation into the cmd/compile/internal/gc AST.
4. Some gc AST passes: typechecking, escape analysis, inlining,
closure handling, expression evaluation ordering (order.go),
and some lowering and optimization (walk.go).
5. Translation into the cmd/compile/internal/ssa SSA form.
6. Optimization and lowering of SSA form.
7. Translation from SSA form to assembler instructions.
8. Translation from assembler instructions to machine code.
9. Writing lots of output: machine code, DWARF symbols,
type and reflection info, export data.
Phase 2 was already concurrent as of Go 1.8.
Phase 3 is planned for eventual removal;
we hope to go straight from syntax AST to SSA.
Phases 5–8 are per-function; this CL adds support for
processing multiple functions concurrently.
The slowest phases in the compiler are 5 and 6,
so this offers the opportunity for some good speed-ups.
Unfortunately, it's not quite that straightforward.
In the current compiler, the latter parts of phase 4
(order, walk) are done function-at-a-time as needed.
Making order and walk concurrency-safe proved hard,
and they're not particularly slow, so there wasn't much reward.
To enable phases 5–8 to be done concurrently,
when concurrent backend compilation is requested,
we complete phase 4 for all functions
before starting later phases for any functions.
Also, in reality, we automatically generate new
functions in phase 9, such as method wrappers
and equality and has routines.
Those new functions then go through phases 4–8.
This CL disables concurrent backend compilation
after the first, big, user-provided batch of
functions has been compiled.
This is done to keep things simple,
and because the autogenerated functions
tend to be small, few, simple, and fast to compile.
USAGE
Concurrent backend compilation still defaults to off.
To set the number of functions that may be backend-compiled
concurrently, use the compiler flag -c.
In future work, cmd/go will automatically set -c.
Furthermore, this CL has been intentionally written
so that the c=1 path has no backend concurrency whatsoever,
not even spawning any goroutines.
This helps ensure that, should problems arise
late in the development cycle,
we can simply have cmd/go set c=1 always,
and revert to the original compiler behavior.
MUTEXES
Most of the work required to make concurrent backend
compilation safe has occurred over the past month.
This CL adds a handful of mutexes to get the rest of the way there;
they are the mutexes that I didn't see a clean way to avoid.
Some of them may still be eliminable in future work.
In no particular order:
* gc.funcsymsmu. The global funcsyms slice is populated
lazily when we need function symbols for closures.
This occurs during gc AST to SSA translation.
The function funcsym also does a package lookup,
which is a source of races on types.Pkg.Syms;
funcsymsmu also covers that package lookup.
This mutex is low priority: it adds a single global,
it is in an infrequently used code path, and it is low contention.
Since funcsyms may now be added in any order,
we must sort them to preserve reproducible builds.
* gc.largeStackFramesMu. We don't discover until after SSA compilation
that a function's stack frame is gigantic.
Recording that error happens basically never,
but it does happen concurrently.
Fix with a low priority mutex and sorting.
* obj.Link.hashmu. ctxt.hash stores the mapping from
types.Syms (compiler symbols) to obj.LSyms (linker symbols).
It is accessed fairly heavily through all the phases.
This is the only heavily contended mutex.
* gc.signatlistmu. The global signatlist map is
populated with types through several of the concurrent phases,
including notably via ngotype during DWARF generation.
It is low priority for removal.
* gc.typepkgmu. Looking up symbols in the types package
happens a fair amount during backend compilation
and DWARF generation, particularly via ngotype.
This mutex helps us to avoid a broader mutex on types.Pkg.Syms.
It has low-to-moderate contention.
* types.internedStringsmu. gc AST to SSA conversion and
some SSA work introduce new autotmps.
Those autotmps have their names interned to reduce allocations.
That interning requires protecting types.internedStrings.
The autotmp names are heavily re-used, and the mutex
overhead and contention here are low, so it is probably
a worthwhile performance optimization to keep this mutex.
TESTING
I have been testing this code locally by running
'go install -race cmd/compile'
and then doing
'go build -a -gcflags=-c=128 std cmd'
for all architectures and a variety of compiler flags.
This obviously needs to be made part of the builders,
but it is too expensive to make part of all.bash.
I have filed #19962 for this.
REPRODUCIBLE BUILDS
This version of the compiler generates reproducible builds.
Testing reproducible builds also needs automation, however,
and is also too expensive for all.bash.
This is #19961.
Also of note is that some of the compiler flags used by 'toolstash -cmp'
are currently incompatible with concurrent backend compilation.
They still work fine with c=1.
Time will tell whether this is a problem.
NEXT STEPS
* Continue to find and fix races and bugs,
using a combination of code inspection, fuzzing,
and hopefully some community experimentation.
I do not know of any outstanding races,
but there probably are some.
* Improve testing.
* Improve performance, for many values of c.
* Integrate with cmd/go and fine tune.
* Support concurrent compilation with the -race flag.
It is a sad irony that it does not yet work.
* Minor code cleanup that has been deferred during
the last month due to uncertainty about the
ultimate shape of this CL.
PERFORMANCE
Here's the buried lede, at last. :)
All benchmarks are from my 8 core 2.9 GHz Intel Core i7 darwin/amd64 laptop.
First, going from tip to this CL with c=1 has almost no impact.
name old time/op new time/op delta
Template 195ms ± 3% 194ms ± 5% ~ (p=0.370 n=30+29)
Unicode 86.6ms ± 3% 87.0ms ± 7% ~ (p=0.958 n=29+30)
GoTypes 548ms ± 3% 555ms ± 4% +1.35% (p=0.001 n=30+28)
Compiler 2.51s ± 2% 2.54s ± 2% +1.17% (p=0.000 n=28+30)
SSA 5.16s ± 3% 5.16s ± 2% ~ (p=0.910 n=30+29)
Flate 124ms ± 5% 124ms ± 4% ~ (p=0.947 n=30+30)
GoParser 146ms ± 3% 146ms ± 3% ~ (p=0.150 n=29+28)
Reflect 354ms ± 3% 352ms ± 4% ~ (p=0.096 n=29+29)
Tar 107ms ± 5% 106ms ± 3% ~ (p=0.370 n=30+29)
XML 200ms ± 4% 201ms ± 4% ~ (p=0.313 n=29+28)
[Geo mean] 332ms 333ms +0.10%
name old user-time/op new user-time/op delta
Template 227ms ± 5% 225ms ± 5% ~ (p=0.457 n=28+27)
Unicode 109ms ± 4% 109ms ± 5% ~ (p=0.758 n=29+29)
GoTypes 713ms ± 4% 721ms ± 5% ~ (p=0.051 n=30+29)
Compiler 3.36s ± 2% 3.38s ± 3% ~ (p=0.146 n=30+30)
SSA 7.46s ± 3% 7.47s ± 3% ~ (p=0.804 n=30+29)
Flate 146ms ± 7% 147ms ± 3% ~ (p=0.833 n=29+27)
GoParser 179ms ± 5% 179ms ± 5% ~ (p=0.866 n=30+30)
Reflect 431ms ± 4% 429ms ± 4% ~ (p=0.593 n=29+30)
Tar 124ms ± 5% 123ms ± 5% ~ (p=0.140 n=29+29)
XML 243ms ± 4% 242ms ± 7% ~ (p=0.404 n=29+29)
[Geo mean] 415ms 415ms +0.02%
name old obj-bytes new obj-bytes delta
Template 382k ± 0% 382k ± 0% ~ (all equal)
Unicode 203k ± 0% 203k ± 0% ~ (all equal)
GoTypes 1.18M ± 0% 1.18M ± 0% ~ (all equal)
Compiler 3.98M ± 0% 3.98M ± 0% ~ (all equal)
SSA 8.28M ± 0% 8.28M ± 0% ~ (all equal)
Flate 230k ± 0% 230k ± 0% ~ (all equal)
GoParser 287k ± 0% 287k ± 0% ~ (all equal)
Reflect 1.00M ± 0% 1.00M ± 0% ~ (all equal)
Tar 190k ± 0% 190k ± 0% ~ (all equal)
XML 416k ± 0% 416k ± 0% ~ (all equal)
[Geo mean] 660k 660k +0.00%
Comparing this CL to itself, from c=1 to c=2
improves real times 20-30%, costs 5-10% more CPU time,
and adds about 2% alloc.
The allocation increase comes from allocating more ssa.Caches.
name old time/op new time/op delta
Template 202ms ± 3% 149ms ± 3% -26.15% (p=0.000 n=49+49)
Unicode 87.4ms ± 4% 84.2ms ± 3% -3.68% (p=0.000 n=48+48)
GoTypes 560ms ± 2% 398ms ± 2% -28.96% (p=0.000 n=49+49)
Compiler 2.46s ± 3% 1.76s ± 2% -28.61% (p=0.000 n=48+46)
SSA 6.17s ± 2% 4.04s ± 1% -34.52% (p=0.000 n=49+49)
Flate 126ms ± 3% 92ms ± 2% -26.81% (p=0.000 n=49+48)
GoParser 148ms ± 4% 107ms ± 2% -27.78% (p=0.000 n=49+48)
Reflect 361ms ± 3% 281ms ± 3% -22.10% (p=0.000 n=49+49)
Tar 109ms ± 4% 86ms ± 3% -20.81% (p=0.000 n=49+47)
XML 204ms ± 3% 144ms ± 2% -29.53% (p=0.000 n=48+45)
name old user-time/op new user-time/op delta
Template 246ms ± 9% 246ms ± 4% ~ (p=0.401 n=50+48)
Unicode 109ms ± 4% 111ms ± 4% +1.47% (p=0.000 n=44+50)
GoTypes 728ms ± 3% 765ms ± 3% +5.04% (p=0.000 n=46+50)
Compiler 3.33s ± 3% 3.41s ± 2% +2.31% (p=0.000 n=49+48)
SSA 8.52s ± 2% 9.11s ± 2% +6.93% (p=0.000 n=49+47)
Flate 149ms ± 4% 161ms ± 3% +8.13% (p=0.000 n=50+47)
GoParser 181ms ± 5% 192ms ± 2% +6.40% (p=0.000 n=49+46)
Reflect 452ms ± 9% 474ms ± 2% +4.99% (p=0.000 n=50+48)
Tar 126ms ± 6% 136ms ± 4% +7.95% (p=0.000 n=50+49)
XML 247ms ± 5% 264ms ± 3% +6.94% (p=0.000 n=48+50)
name old alloc/op new alloc/op delta
Template 38.8MB ± 0% 39.3MB ± 0% +1.48% (p=0.008 n=5+5)
Unicode 29.8MB ± 0% 30.2MB ± 0% +1.19% (p=0.008 n=5+5)
GoTypes 113MB ± 0% 114MB ± 0% +0.69% (p=0.008 n=5+5)
Compiler 443MB ± 0% 447MB ± 0% +0.95% (p=0.008 n=5+5)
SSA 1.25GB ± 0% 1.26GB ± 0% +0.89% (p=0.008 n=5+5)
Flate 25.3MB ± 0% 25.9MB ± 1% +2.35% (p=0.008 n=5+5)
GoParser 31.7MB ± 0% 32.2MB ± 0% +1.59% (p=0.008 n=5+5)
Reflect 78.2MB ± 0% 78.9MB ± 0% +0.91% (p=0.008 n=5+5)
Tar 26.6MB ± 0% 27.0MB ± 0% +1.80% (p=0.008 n=5+5)
XML 42.4MB ± 0% 43.4MB ± 0% +2.35% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Template 379k ± 0% 378k ± 0% ~ (p=0.421 n=5+5)
Unicode 322k ± 0% 321k ± 0% ~ (p=0.222 n=5+5)
GoTypes 1.14M ± 0% 1.14M ± 0% ~ (p=0.548 n=5+5)
Compiler 4.12M ± 0% 4.11M ± 0% -0.14% (p=0.032 n=5+5)
SSA 9.72M ± 0% 9.72M ± 0% ~ (p=0.421 n=5+5)
Flate 234k ± 1% 234k ± 0% ~ (p=0.421 n=5+5)
GoParser 316k ± 1% 315k ± 0% ~ (p=0.222 n=5+5)
Reflect 980k ± 0% 979k ± 0% ~ (p=0.095 n=5+5)
Tar 249k ± 1% 249k ± 1% ~ (p=0.841 n=5+5)
XML 392k ± 0% 391k ± 0% ~ (p=0.095 n=5+5)
From c=1 to c=4, real time is down ~40%, CPU usage up 10-20%, alloc up ~5%:
name old time/op new time/op delta
Template 203ms ± 3% 131ms ± 5% -35.45% (p=0.000 n=50+50)
Unicode 87.2ms ± 4% 84.1ms ± 2% -3.61% (p=0.000 n=48+47)
GoTypes 560ms ± 4% 310ms ± 2% -44.65% (p=0.000 n=50+49)
Compiler 2.47s ± 3% 1.41s ± 2% -43.10% (p=0.000 n=50+46)
SSA 6.17s ± 2% 3.20s ± 2% -48.06% (p=0.000 n=49+49)
Flate 126ms ± 4% 74ms ± 2% -41.06% (p=0.000 n=49+48)
GoParser 148ms ± 4% 89ms ± 3% -39.97% (p=0.000 n=49+50)
Reflect 360ms ± 3% 242ms ± 3% -32.81% (p=0.000 n=49+49)
Tar 108ms ± 4% 73ms ± 4% -32.48% (p=0.000 n=50+49)
XML 203ms ± 3% 119ms ± 3% -41.56% (p=0.000 n=49+48)
name old user-time/op new user-time/op delta
Template 246ms ± 9% 287ms ± 9% +16.98% (p=0.000 n=50+50)
Unicode 109ms ± 4% 118ms ± 5% +7.56% (p=0.000 n=46+50)
GoTypes 735ms ± 4% 806ms ± 2% +9.62% (p=0.000 n=50+50)
Compiler 3.34s ± 4% 3.56s ± 2% +6.78% (p=0.000 n=49+49)
SSA 8.54s ± 3% 10.04s ± 3% +17.55% (p=0.000 n=50+50)
Flate 149ms ± 6% 176ms ± 3% +17.82% (p=0.000 n=50+48)
GoParser 181ms ± 5% 213ms ± 3% +17.47% (p=0.000 n=50+50)
Reflect 453ms ± 6% 499ms ± 2% +10.11% (p=0.000 n=50+48)
Tar 126ms ± 5% 149ms ±11% +18.76% (p=0.000 n=50+50)
XML 246ms ± 5% 287ms ± 4% +16.53% (p=0.000 n=49+50)
name old alloc/op new alloc/op delta
Template 38.8MB ± 0% 40.4MB ± 0% +4.21% (p=0.008 n=5+5)
Unicode 29.8MB ± 0% 30.9MB ± 0% +3.68% (p=0.008 n=5+5)
GoTypes 113MB ± 0% 116MB ± 0% +2.71% (p=0.008 n=5+5)
Compiler 443MB ± 0% 455MB ± 0% +2.75% (p=0.008 n=5+5)
SSA 1.25GB ± 0% 1.27GB ± 0% +1.84% (p=0.008 n=5+5)
Flate 25.3MB ± 0% 26.9MB ± 1% +6.31% (p=0.008 n=5+5)
GoParser 31.7MB ± 0% 33.2MB ± 0% +4.61% (p=0.008 n=5+5)
Reflect 78.2MB ± 0% 80.2MB ± 0% +2.53% (p=0.008 n=5+5)
Tar 26.6MB ± 0% 27.9MB ± 0% +5.19% (p=0.008 n=5+5)
XML 42.4MB ± 0% 44.6MB ± 0% +5.20% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Template 380k ± 0% 379k ± 0% -0.39% (p=0.032 n=5+5)
Unicode 321k ± 0% 321k ± 0% ~ (p=0.841 n=5+5)
GoTypes 1.14M ± 0% 1.14M ± 0% ~ (p=0.421 n=5+5)
Compiler 4.12M ± 0% 4.14M ± 0% +0.52% (p=0.008 n=5+5)
SSA 9.72M ± 0% 9.76M ± 0% +0.37% (p=0.008 n=5+5)
Flate 234k ± 1% 234k ± 1% ~ (p=0.690 n=5+5)
GoParser 316k ± 0% 317k ± 1% ~ (p=0.841 n=5+5)
Reflect 981k ± 0% 981k ± 0% ~ (p=1.000 n=5+5)
Tar 250k ± 0% 249k ± 1% ~ (p=0.151 n=5+5)
XML 393k ± 0% 392k ± 0% ~ (p=0.056 n=5+5)
Going beyond c=4 on my machine tends to increase CPU time and allocs
without impacting real time.
The CPU time numbers matter, because when there are many concurrent
compilation processes, that will impact the overall throughput.
The numbers above are in many ways the best case scenario;
we can take full advantage of all cores.
Fortunately, the most common compilation scenario is incremental
re-compilation of a single package during a build/test cycle.
Updates #15756
Change-Id: I6725558ca2069edec0ac5b0d1683105a9fff6bea
Reviewed-on: https://go-review.googlesource.com/40693
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When using Lstat against symlinks that point to a directory,
the function returns FileInfo with both ModeDir and ModeSymlink set.
Change that to never set ModeDir if ModeSymlink is set.
Fixes#10424Fixes#17540Fixes#17541
Change-Id: Iba280888aad108360b8c1f18180a24493fe7ad2b
Reviewed-on: https://go-review.googlesource.com/41830
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Taken from the Intel Software Development Manual (of course, in the line
below it's ADC DST, SRC; The opposite of the commit subject).
12 /r ADC r8, r/m8
We need 0x12 for the corresponding ytab line, not 0x10.
{Ymb, Ynone, Yrb, Zm_r, 1},
Updates #14069
Change-Id: Id37cbd0c581c9988c2de355efa908956278e2189
Reviewed-on: https://go-review.googlesource.com/41857
Reviewed-by: Keith Randall <khr@golang.org>
Delete old TestRuntimeFunctionTrimming, which is testing a dead API
and is now handled in end-to-end tests.
Change-Id: I64fc2991ed4a7690456356b5f6b546f36935bb67
Reviewed-on: https://go-review.googlesource.com/41815
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This is a direct port of the version from
commit a60ad46e0ed33d02e09bda439efaf9c9727dbc6c
(https://go-review.googlesource.com/c/37342/).
updates #17973
updates #18177
Change-Id: I63fa5ef6951b1edd39f84927d1181a4df9b15385
Reviewed-on: https://go-review.googlesource.com/36617
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Follow-up to review comments on CL 41797.
Mask the input to set2 and set3, so that at the very least,
we won't corrupt the rest of the flags in case of a bad input.
It also seems more semantically appropriate.
Do minor cleanup in addrescapes. I started on larger cleanup,
but it wasn't clear that it was an improvement.
Add warning comments and sanity checks to Initorder and Class constants,
to attempt to prevent them from overflowing their allotted flag bits.
Passes toolstash-check.
Change-Id: I57b9661ba36f56406aa7a1d8da9b7c70338f9119
Reviewed-on: https://go-review.googlesource.com/41817
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When the stack register is decremented to acquire stack space at
the beginning of a function, a MOVDU should be used so it is done
atomically, unless the size of the stack frame is too large for
that instruction. The code to determine whether to use MOVDU
or MOVD was checking if the function was a leaf and always generating MOVD
when it was. The choice of MOVD vs. MOVDU should only depend on the stack
frame size. This fixes that problem.
Change-Id: I0e49c79036f1e8f7584179e1442b938fc6da085f
Reviewed-on: https://go-review.googlesource.com/41813
Reviewed-by: Michael Munday <munday@ca.ibm.com>
In many cases the records returned by Reader.Read will only be used between calls
to Read and become garbage once a new record is read. In this case, instead of
allocating a new slice on each call to Read, we can reuse the last allocated slice
for successive calls to avoid unnecessary allocations.
This change adds a new field ReuseRecord to the Reader struct to enable this reuse.
ReuseRecord is false by default to avoid breaking existing code which dependss on
the current behaviour.
I also added 4 new benchmarks, corresponding to the existing Read benchmarks, which
set ReuseRecord to true.
Benchstat on my local machine (old is ReuseRecord = false, new is ReuseRecord = true)
name old time/op new time/op delta
Read-8 2.75µs ± 2% 1.88µs ± 1% -31.52% (p=0.000 n=14+15)
ReadWithFieldsPerRecord-8 2.75µs ± 0% 1.89µs ± 1% -31.43% (p=0.000 n=13+13)
ReadWithoutFieldsPerRecord-8 2.77µs ± 1% 1.88µs ± 1% -32.06% (p=0.000 n=15+15)
ReadLargeFields-8 55.4µs ± 1% 54.2µs ± 0% -2.07% (p=0.000 n=15+14)
name old alloc/op new alloc/op delta
Read-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadLargeFields-8 3.94kB ± 0% 2.98kB ± 0% -24.39% (p=0.000 n=15+15)
name old allocs/op new allocs/op delta
Read-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadLargeFields-8 24.0 ± 0% 12.0 ± 0% -50.00% (p=0.000 n=15+15)
Fixes#19721
Change-Id: I79b14128bb9bb3465f53f40f93b1b528a9da6f58
Reviewed-on: https://go-review.googlesource.com/41730
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some large testing/build systems require some form of test discovery before
running tests. This usually allows for analytics, history, and stats on a per
tests basis. Typically these systems are meant used in multi-language
environments and the original source code is not known or available.
This adds a -test.list option which takes a regular expression as an
argument. Any tests, benchmarks, or examples that match that regular
expression will be printed, one per line, to stdout and then the program
will exit.
Since subtests are named/discovered at run time this will only show
top-level tests names and is a known limitation.
Fixes#17209
Change-Id: I7e607f5f4f084d623a1cae88a1f70e7d92b7f13e
Reviewed-on: https://go-review.googlesource.com/41195
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The Context definition to date has not defined what Err returns
before the Done channel is closed. Define that it returns nil,
as most implementations do.
All the standard context implementations (those in package
context and in golang.org/x/net/context) return Err() == nil
when Done is not yet closed. However, some non-standard
implementations may exist that return Err() != nil in this case,
as permitted by the Context definition before this date.
Call these "errorful implementations".
Because all the standard context implementations ensure that
Err() == nil when Done is not yet closed, clients now exist that
assume Err() != nil implies Done is closed and use calling Err
as a quick short-circuit check instead of first doing a non-blocking
receive from Done and then, if that succeeds, needing to call Err.
This assumption holds for all the standard Context implementations,
so these clients work fine in practice, even though they are making
unwarranted assumptions about the Context implementations.
Call these "technically incorrect clients".
If a technically incorrect client encounters an errorful
implementation, the client misbehaves. Because there are few
errorful implementations, over time we expect that many clients
will end up being technically incorrect without realizing it,
leading to latent, subtle bugs. If we want to eliminate these
latent, subtle bugs, there are two ways to do this:
either make errorful implementations more common
(exposing the client bugs more often) or redefine the Context
interface so that the clients are not buggy after all.
If we make errorful implementations more common, such
as by changing the standard context implementations to
return ErrNotDone instead of nil when Err is called before
Done is closed, this will shake out essentially all of the
technically incorrect clients, forcing people to find and fix
those clients during the transition to Go 1.9.
Technically this is allowed by the compatibility policy,
but we expect there are many pieces of code assuming
that Err() != nil means done, so updating will cause real pain.
If instead we disallow errorful implementations, then they
will need to be fixed as they are discovered, but the fault
will officially lie in the errorful Context implementation,
not in the clients. Technically this is disallowed by the compatibility
policy, because these errorful implementations were "correct"
in earlier versions of Go, except that they didn't work with
common client code. We expect there are hardly any errorful
implementations, so that disallowing them will be less disruptive
and more in the spirit of the compatibility policy.
This CL takes the path of expected least disruption,
narrowing the Context interface semantics and potentially
invalidating existing implementations. A survey of the
go-corpus v0.01 turned up only five Context implementations,
all trivial and none errorful (details in #19856).
We are aware of one early Context implementation inside Google,
from before even golang.org/x/net/context existed,
that is errorful. The misbehavior of an open-source library
when passed such a context is what prompted #19856.
That context implementation would be disallowed after this CL
and would need to be corrected. We are aware of no other
affected context implementations. On the other hand, a survey
of the go-corpus v0.01 turned up many instances of client
code assuming that Err() == nil implies not done yet
(details also in #19856). On balance, narrowing Context and
thereby allowing Err() == nil checks should invalidate significantly
less code than a push to flush out all the currently technically
incorrect Err() == nil checks.
If release feedback shows that we're wrong about this balance,
we can roll back this CL and try again in Go 1.10.
Fixes#19856.
Change-Id: Id45d126fac70e1fcc42d73e5a87ca1b66935b831
Reviewed-on: https://go-review.googlesource.com/40291
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Sameer Ajmani <sameer@golang.org>
Since close errors have been cleaned up in CL 39997,
TestCloseError is failing on Plan 9, because
TCPListener.Close didn't check that the listener
has already been closed before writing the "hangup"
string to the listener control file.
This change fixes TCPListener.Close on Plan 9,
by closing poll.FD before writing the "hangup"
string.
Fixes#20128.
Change-Id: I13862b23a9055dd1be658acef7066707d98c591f
Reviewed-on: https://go-review.googlesource.com/41850
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This updates sha256.block and sha512.block to use vector instructions. While
each round must still be performed independently, this allows for the use of
the vshasigma{w,d} crypto acceleration instructions.
For crypto/sha256:
benchmark old ns/op new ns/op delta
BenchmarkHash8Bytes 570 300 -47.37%
BenchmarkHash1K 7529 3018 -59.91%
BenchmarkHash8K 55308 21938 -60.33%
benchmark old MB/s new MB/s speedup
BenchmarkHash8Bytes 14.01 26.58 1.90x
BenchmarkHash1K 136.00 339.23 2.49x
BenchmarkHash8K 148.11 373.40 2.52x
For crypto/sha512:
benchmark old ns/op new ns/op delta
BenchmarkHash8Bytes 725 394 -45.66%
BenchmarkHash1K 5062 2107 -58.38%
BenchmarkHash8K 34711 13918 -59.90%
benchmark old MB/s new MB/s speedup
BenchmarkHash8Bytes 11.03 20.29 1.84x
BenchmarkHash1K 202.28 485.84 2.40x
BenchmarkHash8K 236.00 588.56 2.49x
Fixes#20069
Change-Id: I28bffe6e9eb484a83a004116fce84acb4942abca
Reviewed-on: https://go-review.googlesource.com/41391
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
This may improve perormance during concurrent access
to mheap.central array from multiple CPU cores.
Change-Id: I8f48dd2e72aa62e9c32de07ae60fe552d8642782
Reviewed-on: https://go-review.googlesource.com/41550
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Moved the relevant file.close() usages close to after the
file opens and put them in defer statements, so that readers
don't have to think too much as to where the file is
being closed.
Change-Id: Ic4190b02ea2f5ac281b9ba104e0023e9f87ca8c7
Reviewed-on: https://go-review.googlesource.com/41796
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Catch all the cases where a file operation might return ErrFileClosing,
and convert to ErrClosed. Use a new method for the conversion, which
permits us to remove some KeepAlive calls.
Change-Id: I584178f297efe6cb86f3090b2341091b412f1041
Reviewed-on: https://go-review.googlesource.com/41793
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The LocalAddrContext should have the network address of the actual
interface.
Fixes#18686
Change-Id: I9c401eda312f3a0e7e65b013af827aeeef3b4d3d
Reviewed-on: https://go-review.googlesource.com/35490
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Node.Walkdef is 0, 1, or 2, so it only requires two bits.
Add support for 2-bit values to bitset,
and use it for Node.Walkdef.
Class, Embedded, Typecheck, and Initorder will follow suit
in subsequent CLs.
The multi-bit flags will go at the beginning,
since that generates (marginally) more efficient code.
Change-Id: Id6e2e66e437f10aaa05b8a6e1652efb327d06128
Reviewed-on: https://go-review.googlesource.com/41791
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In the past we returned "use of closed network connection" when using
a closed network descriptor in some way. In CL 36799 that was changed
to return "use of closed file or network connection". Because programs
have no access to a value of this error type (see issue #4373) they
resort to doing direct string comparisons (see issue #19252). This CL
restores the old error string so that we don't break programs
unnecessarily with the 1.9 release.
This adds a test to the net package for the expected string.
For symmetry check that the os package returns the expected error,
which for os already exists as os.ErrClosed.
Updates #4373.
Fixed#19252.
Change-Id: I5b83fd12cfa03501a077cad9336499b819f4a38b
Reviewed-on: https://go-review.googlesource.com/39997
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
