This change adds support for the splice system call on Linux,
for the purpose of optimizing (*TCPConn).ReadFrom by reducing
copies of data from and to userspace. It does so by creating a
temporary pipe and splicing data from the source connection to the
pipe, then from the pipe to the destination connection. The pipe
serves as an in-kernel buffer for the data transfer.
No new API is added to package net, but a new Splice function is
added to package internal/poll, because using splice requires help
from the network poller. Users of the net package should benefit
from the change transparently.
This change only enables the optimization if the Reader in ReadFrom
is a TCP connection. Since splice is a more general interface, it
could, in theory, also be enabled if the Reader were a unix socket,
or the read half of a pipe.
However, benchmarks show that enabling it for unix sockets is most
likely not a net performance gain. The tcp <- unix case is also
fairly unlikely to be used very much by users of package net.
Enabling the optimization for pipes is also problematic from an
implementation perspective, since package net cannot easily get at
the *poll.FD of an *os.File. A possible solution to this would be
to dup the pipe file descriptor, register the duped descriptor with
the network poller, and work on that *poll.FD instead of the original.
However, this seems too intrusive, so it has not been done. If there
was a clean way to do it, it would probably be worth doing, since
splicing from a pipe to a socket can be done directly.
Therefore, this patch only enables the optimization for what is likely
the most common use case: tcp <- tcp.
The following benchmark compares the performance of the previous
userspace genericReadFrom code path to the new optimized code path.
The sub-benchmarks represent chunk sizes used by the writer on the
other end of the Reader passed to ReadFrom.
benchmark old ns/op new ns/op delta
BenchmarkTCPReadFrom/1024-4 4727 4954 +4.80%
BenchmarkTCPReadFrom/2048-4 4389 4301 -2.01%
BenchmarkTCPReadFrom/4096-4 4606 4534 -1.56%
BenchmarkTCPReadFrom/8192-4 5219 4779 -8.43%
BenchmarkTCPReadFrom/16384-4 8708 8008 -8.04%
BenchmarkTCPReadFrom/32768-4 16349 14973 -8.42%
BenchmarkTCPReadFrom/65536-4 35246 27406 -22.24%
BenchmarkTCPReadFrom/131072-4 72920 52382 -28.17%
BenchmarkTCPReadFrom/262144-4 149311 95094 -36.31%
BenchmarkTCPReadFrom/524288-4 306704 181856 -40.71%
BenchmarkTCPReadFrom/1048576-4 674174 357406 -46.99%
benchmark old MB/s new MB/s speedup
BenchmarkTCPReadFrom/1024-4 216.62 206.69 0.95x
BenchmarkTCPReadFrom/2048-4 466.61 476.08 1.02x
BenchmarkTCPReadFrom/4096-4 889.09 903.31 1.02x
BenchmarkTCPReadFrom/8192-4 1569.40 1714.06 1.09x
BenchmarkTCPReadFrom/16384-4 1881.42 2045.84 1.09x
BenchmarkTCPReadFrom/32768-4 2004.18 2188.41 1.09x
BenchmarkTCPReadFrom/65536-4 1859.38 2391.25 1.29x
BenchmarkTCPReadFrom/131072-4 1797.46 2502.21 1.39x
BenchmarkTCPReadFrom/262144-4 1755.69 2756.68 1.57x
BenchmarkTCPReadFrom/524288-4 1709.42 2882.98 1.69x
BenchmarkTCPReadFrom/1048576-4 1555.35 2933.84 1.89x
Fixes#10948
Change-Id: I3ce27f21f7adda8b696afdc48a91149998ae16a5
Reviewed-on: https://go-review.googlesource.com/107715
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The caller of epollctl expects it to return a negative errno value,
but it returns a positive errno value on mips, mips64 and ppc64.
The change fixes this.
Updates #23446
Change-Id: Ie6372eca6c23de21964caaaa433c9a45ef93531e
Reviewed-on: https://go-review.googlesource.com/89235
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ever since we added sleep to the runtime back in 2008, we've
implemented it on GNU/Linux with the select (or pselect or pselect6)
system call. But the Linux kernel has a nanosleep system call,
which should be a tiny bit more efficient since it doesn't have to
check to see whether there are any file descriptors. So use it.
Change-Id: Icc3430baca46b082a4d33f97c6c47e25fa91cb9a
Reviewed-on: https://go-review.googlesource.com/108538
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The go/types API exposes what package objects were declared in, which
includes struct fields, interface methods, and function parameters.
The compiler implicitly tracks these for non-exported identifiers
(through the Sym's associated Pkg), but exported identifiers always
use localpkg. To simplify identifying this, add an explicit package
field to struct, interface, and function types.
Change-Id: I6adc5dc653e78f058714259845fb3077066eec82
Reviewed-on: https://go-review.googlesource.com/107622
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Rewrite x<<1+c into x+x+c, which can be expressed as a single LEAQ/LEAL.
Bit of a special case, but the single-instruction
LEA is both shorter and faster than SHL then ADD.
Triggers 293 times during make.bash.
Change-Id: I3f09c8e9a8f3859d1eeed336f095fc3ada79c2c1
Reviewed-on: https://go-review.googlesource.com/108938
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
The language spec requires the RHS operand of shift expressions to be unsigned integers.
The changes in CL 60230 and the related CL 81277 refer to a variable s of type uint.
The "untyped constant" here refers to 1.0, not s.
Change-Id: Id2b884816af7f79f453afcb8c34ade2d34e18bc2
GitHub-Last-Rev: b26c853cae
GitHub-Pull-Request: golang/go#24989
Reviewed-on: https://go-review.googlesource.com/108676
Reviewed-by: Robert Griesemer <gri@golang.org>
For struct fields and methods, Field.Nname was only used to store
position information, which means we're allocating an entire ONAME
Node+Name+Param structure just for one field. We can optimize away
these ONAME allocations by instead adding a Field.Pos field.
Unfortunately, we can't get rid of Field.Nname, because it's needed
for function parameters, so Field grows a little bit and now has more
redundant information in those cases. However, that was already the
case (e.g., Field.Sym and Field.Nname.Sym), and it's still a net win
for allocations as demonstrated by the benchmarks below.
Additionally, by moving the ONAME allocation for function parameters
to funcargs, we can avoid allocating them for function parameters that
aren't used in corresponding function bodies (e.g., interface methods,
function-typed variables, and imported functions/methods without
inline bodies).
name old time/op new time/op delta
Template 254ms ± 6% 251ms ± 6% -1.04% (p=0.000 n=487+488)
Unicode 128ms ± 7% 128ms ± 7% ~ (p=0.294 n=482+467)
GoTypes 862ms ± 5% 860ms ± 4% ~ (p=0.075 n=488+471)
Compiler 3.91s ± 4% 3.90s ± 4% -0.39% (p=0.000 n=468+473)
name old user-time/op new user-time/op delta
Template 339ms ±14% 336ms ±14% -1.02% (p=0.001 n=498+494)
Unicode 176ms ±18% 176ms ±25% ~ (p=0.940 n=491+499)
GoTypes 1.13s ± 8% 1.13s ± 9% ~ (p=0.157 n=496+493)
Compiler 5.24s ± 6% 5.21s ± 6% -0.57% (p=0.000 n=485+489)
name old alloc/op new alloc/op delta
Template 38.3MB ± 0% 37.3MB ± 0% -2.58% (p=0.000 n=499+497)
Unicode 29.1MB ± 0% 29.1MB ± 0% -0.03% (p=0.000 n=500+493)
GoTypes 116MB ± 0% 115MB ± 0% -0.65% (p=0.000 n=498+499)
Compiler 492MB ± 0% 487MB ± 0% -1.00% (p=0.000 n=497+498)
name old allocs/op new allocs/op delta
Template 364k ± 0% 360k ± 0% -1.15% (p=0.000 n=499+499)
Unicode 336k ± 0% 336k ± 0% -0.01% (p=0.000 n=500+493)
GoTypes 1.16M ± 0% 1.16M ± 0% -0.30% (p=0.000 n=499+499)
Compiler 4.54M ± 0% 4.51M ± 0% -0.58% (p=0.000 n=494+495)
Passes toolstash-check -gcflags=-dwarf=false. Changes DWARF output
because position information is now tracked more precisely for
function parameters.
Change-Id: Ib8077d70d564cc448c5e4290baceab3a4396d712
Reviewed-on: https://go-review.googlesource.com/108217
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Currently Liveness.compact rewrites the Liveness.livevars slice in
place. However, we're about to add register maps, which we'll want to
track in livevars, but compact independently from the stack maps.
Hence, this CL modifies Liveness.compact to consume Liveness.livevars
and produce a new slice of deduplicated stack maps. This is somewhat
clearer anyway because it avoids potential confusion over how
Liveness.livevars is indexed.
Passes toolstash -cmp.
For #24543.
Change-Id: I7093fbc71143f8a29e677aa30c96e501f953ca2b
Reviewed-on: https://go-review.googlesource.com/108498
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
We were using file descriptor 100, which requires the Linux kernel to
grow the fdtable size. That step may sometimes require a long time,
causing the test to fail. Switch to file descriptor 30, which should
not require growing the fdtable.
Fixes#23784
Change-Id: I3ac40d6f8569c70d34b470cfca34eff149bf8229
Reviewed-on: https://go-review.googlesource.com/108537
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Ignored reports whether sig is currently ignored.
This implementation only works applies on Unix systems for now. However, at
the moment that is also the case for Ignore() and several other signal
interaction methods, so that seems fair.
Fixes#22497
Change-Id: I7c1b1a5e12373ca5da44709500ff5acedc6f1316
Reviewed-on: https://go-review.googlesource.com/108376
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This triggers three times while building std,
once in image/png and twice in go/internal/gccgoimporter.
There are no instances in std in which a more aggressive
optimization would have triggered.
This doesn't necessarily avoid an allocation,
because escape analysis is already able in many cases
to use a temporary backing for the string,
but it does at a minimum avoid the runtime call and copy.
Fixes#24937
Change-Id: I7019e85638ba8cd7e2f03890e672558b858579bc
Reviewed-on: https://go-review.googlesource.com/108035
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Change-Id: I6e1fa67dc9d4d151c90eb19a6f736e4daa7d4fb3
Reviewed-on: https://go-review.googlesource.com/107615
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reject to compile I386/AMD64 asm code that contains
(Register)(PseudoReg*scale) forms of memory operands.
Example of such program: "CALL (AX)(PC*2)".
PseudoReg is one of the PC, FP, SB (but not SP).
When pseudo-register is used in register indirect as
scaled index base, x86 backend will panic because
its register file misses SB/FP/PC registers.
Fixes#12657.
Change-Id: I30fca797b537cbc86ab47583ae96c6a0c59acaa1
Reviewed-on: https://go-review.googlesource.com/107835
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL moves all of the logic for wiring up imported declarations
into export.go, so that it can be reused by the indexed importer
code. While here, increase symmetry across routines.
Passes toolstash-check.
Change-Id: I1ccec5c3999522b010e4d04ed56b632fd4d712d9
Reviewed-on: https://go-review.googlesource.com/107621
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Add a new DWARF attribute, DW_AT_go_runtime_type, that gives the offset
of the runtime type structure, if any, for a DWARF type. This should
allow debuggers to decode interface content without having to do awkward
name matching.
Fixes#24814
Change-Id: Ic7a66524d2be484154c584afa9697111618efea4
Reviewed-on: https://go-review.googlesource.com/106775
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
These will appear when tracking live pointers in registers, so we need
to know whether they have pointers.
For #24543.
Change-Id: I2edccee39ca989473db4b3e7875ff166808ac141
Reviewed-on: https://go-review.googlesource.com/108497
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: David Chase <drchase@google.com>
Currently, each architecture lowers OpConvert to an arch-specific
OpXXXconvert. This is silly because OpConvert means the same thing on
all architectures and is logically a no-op that exists only to keep
track of conversions to and from unsafe.Pointer. Furthermore, lowering
it makes it harder to recognize in other analyses, particularly
liveness analysis.
This CL eliminates the lowering of OpConvert, leaving it as the
generic op until code generation time.
The main complexity here is that we still need to register-allocate
OpConvert operations. Currently, each arch's lowered OpConvert
specifies all GP registers in its register mask. Ideally, OpConvert
wouldn't affect value homing at all, and we could just copy the home
of OpConvert's source, but this can potentially home an OpConvert in a
LocalSlot, which neither regalloc nor stackalloc expect. Rather than
try to disentangle this assumption from regalloc and stackalloc, we
continue to register-allocate OpConvert, but teach regalloc that
OpConvert can be allocated to any allocatable GP register.
For #24543.
Change-Id: I795a6aee5fd94d4444a7bafac3838a400c9f7bb6
Reviewed-on: https://go-review.googlesource.com/108496
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
These refer to old function names.
Change-Id: Ic4507ff836b442e953a21c8a2d09def54e1e43a7
Reviewed-on: https://go-review.googlesource.com/108495
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
All test cases are commented-out until real implementation is merged.
These files are required to make x86avxgen work: it expects that
for each new instruction it enables end2end test cases are available.
This test suite is automatically generated.
Additional AVX512 tests will be added later.
Change-Id: I5f5cb6b90540834585ee5ad4c00ebfbb6efa8094
Reviewed-on: https://go-review.googlesource.com/107217
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
No changes in the actual generated compiler code.
Change-Id: Ibae71b6de35722792cc94fea0cdfa5e53cf3c83e
Reviewed-on: https://go-review.googlesource.com/108476
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Due to some recent optimizations related to the compare
instruction, DS-form load instructions started to be used
to load 8-byte go.strings. This can cause link time errors
if the go.string is not aligned to 4 bytes.
For DS-form instructions, the value in the offset field must
be a multiple of 4. If the offset is known at the time the
rules are processed, a DS-form load will not be chosen. But for
go.strings, the offset is not known at that time, but a
relocation is generated indicating that the linker should fill
in the DS relocation. When the linker tries to fill in the
relocation, if the offset is not aligned properly, a link error
will occur.
To fix this, when loading a go.string using MOVDload, the full
address of the go.string is generated and loaded into the base
register. Then the go.string is loaded with a 0 offset field.
Added a testcase that reproduces this problem.
Fixes#24799
Change-Id: I6a154e8e1cba64eae290be0fbcb608b75884ecdd
Reviewed-on: https://go-review.googlesource.com/107855
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Current assembler encodes "ADD $0xaaaaaaaa, Rx" to "MOVD off(PC), Rtmp"
+ "ADD Rtmp, Rx", and a 64-bit item is stored in the constant pool.
This patch optimizes it to "MOVWU off(PC), Rtmp" + "ADD Rtmp, Rx",
and a 32-bit item is stored.
The total size of the executable binary go and the library files in
pkg/linux_arm64 decreased about 3KB by this patch.
Change-Id: Ieb1592f78ef9ed52f5d3ad232d6cdf87d0923de1
Reviewed-on: https://go-review.googlesource.com/107516
Reviewed-by: Wei Xiao <Wei.Xiao@arm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The patch rewrites the content of doc.go file. The file describes some
general rules of the mapping between Go assembly syntax and GNU syntax.
And it gives some Go assembly examples and corresponding GNU assembly
examples.
The patch changes the doc.go to use standard doc comment format so that
the link https://golang.org/cmd/internal/obj/arm64/ can display it.
Assembly document framework is mainly contributed by Eric Fang <Eric.Fang@arm.com>
Documentation work is contributed by Eric Fang and Fannie Zhang <Fannie.Zhang@arm.com>
Change-Id: I8b3f6d6c6b91afdc2c44602e8f796beea905085e
Reviewed-on: https://go-review.googlesource.com/102055
Reviewed-by: Rob Pike <r@golang.org>
There are a bunch of places where we generate functions: equality and
hash functions; method expression and promoted method wrappers; and
print/delete wrappers for defer/go statements.
This CL brings them in sync by:
1) Always using dclfunc and funcbody. Most were already using this,
but makepartialcall needed some changes.
2) Removing duplicate types.Markdcl/types.Popdcl calls. These are
already handled by dclfunc and funcbody.
3) Using structargs (already used by genwrapper) to construct new
param/result lists from existing types.
4) Always accessing the parameter ONAME nodes through Field.Nname
instead of poking into the ODCLFIELD. Also, since creating a slice of
the entire parameter list is common, extract this out into a
paramNnames helper function.
5) Add a Type.IsVariadic method to simplify identifying variadic
function types.
Passes toolstash-check -gcflags=-dwarf=false. DWARF output changes
because using structargs in makepartialcall changes the generated
parameter names.
Change-Id: I6661d3699afdbe7852ad60db5a4ec6eeb2b696e4
Reviewed-on: https://go-review.googlesource.com/108216
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
On Unix systems, the underlying socket is no longer forced into blocking
mode.
Fixes#24942
Change-Id: I3e0c503c72df0844e30a63af298691dedacd1f46
Reviewed-on: https://go-review.googlesource.com/108297
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
By simply rearranging the logic, we avoid the overhead of a superfluous
call to getsockopt. For, if p is already non empty, there's no point
in having to check if we need to attach dummy payload. This has
performance benefits when using send/recvmsg for high speed
communications.
Change-Id: Id85cff17328ecbf6d09dd52fbeeaa691dbe69b75
Reviewed-on: https://go-review.googlesource.com/108338
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is to allow followup refactorings that will replace
Field.Nname.Pos with Field.Pos.
Passes toolstash-check.
Change-Id: I1060b6a37c60273892f7af5369809057cff61881
Reviewed-on: https://go-review.googlesource.com/108215
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Fixes golint warning about "if block ends with a return statement,
so drop this else and outdent its block".
Change-Id: Iac4fd324e04e3e3fe3e3933f5e59095041d292c5
Reviewed-on: https://go-review.googlesource.com/107115
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Daniel Theophanes <kardianos@gmail.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes#23373
Fix interfacing with latest(1.4+) pkgconf versions, as they have change the
output format, by extending parsing function splitPkgConfigOutput to accommodate
more possible fragment escaping formats. Function is based on pkgconfigs own
implementation at
https://github.com/pkgconf/pkgconf/blob/master/libpkgconf/argvsplit.c. Along
with this change test case TestSplitPkgConfigOutput have been expanded. Thanks
to ignatenko for help on test cases and insights in to the pkgconfig.
Change-Id: I55301bb564b07128d5564ec1454dd247f84a95c3
Reviewed-on: https://go-review.googlesource.com/86541
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 99337 introduced a file with CRLF line endings. Convert them to LF
line endings as in all other Go files.
Change-Id: I68b28fd443f05bebbbd9280d1821c4ccd33a4a8a
Reviewed-on: https://go-review.googlesource.com/108075
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
