Since CL 38bf89161a72 raw socket tests are not executed
on windows builders. This change re-enable them again.
It will attempt to run raw socket tests only if user
is permitted to create raw socket by OS.
Fixes#6392
R=golang-dev
CC=golang-dev, mikioh.mikioh, rsc
https://golang.org/cl/13422044
This CL adds minimal support of Happy Eyeballs-like TCP connection
setup to Dialer API. Happy Eyeballs and derivation techniques are
described in the following:
- Happy Eyeballs: Success with Dual-Stack Hosts
http://tools.ietf.org/html/rfc6555
- Analysing Dual Stack Behaviour and IPv6 Quality
http://www.potaroo.net/presentations/2012-04-17-dual-stack-quality.pdf
Usually, the techniques consist of three components below.
- DNS query racers, that run A and AAAA queries in parallel or series
- A short list of destination addresses
- TCP SYN racers, that run IPv4 and IPv6 transport in parallel or series
This CL implements only the latter two. The existing DNS query
component gathers together A and AAAA records in series, so we don't
touch it here. This CL just uses extended resolveInternetAddr and makes
it possible to run multiple Dial racers in parallel.
For example, when the given destination is a DNS name and the name has
multiple address family A and AAAA records, and it happens on the TCP
wildcard network "tcp" with DualStack=true like the following:
(&net.Dialer{DualStack: true}).Dial("tcp", "www.example.com:80")
The function will return a first established connection either TCP over
IPv4 or TCP over IPv6, and close the other connection internally.
Fixes#3610.
Fixes#5267.
Benchmark results on freebsd/amd64 virtual machine, tip vs. tip+12416043:
benchmark old ns/op new ns/op delta
BenchmarkTCP4OneShot 50696 52141 +2.85%
BenchmarkTCP4OneShotTimeout 65775 66426 +0.99%
BenchmarkTCP4Persistent 10986 10457 -4.82%
BenchmarkTCP4PersistentTimeout 11207 10445 -6.80%
BenchmarkTCP6OneShot 62009 63718 +2.76%
BenchmarkTCP6OneShotTimeout 78351 79138 +1.00%
BenchmarkTCP6Persistent 14695 14659 -0.24%
BenchmarkTCP6PersistentTimeout 15032 14646 -2.57%
BenchmarkTCP4ConcurrentReadWrite 7215 6217 -13.83%
BenchmarkTCP6ConcurrentReadWrite 7528 7493 -0.46%
benchmark old allocs new allocs delta
BenchmarkTCP4OneShot 36 36 0.00%
BenchmarkTCP4OneShotTimeout 36 36 0.00%
BenchmarkTCP4Persistent 0 0 n/a%
BenchmarkTCP4PersistentTimeout 0 0 n/a%
BenchmarkTCP6OneShot 37 37 0.00%
BenchmarkTCP6OneShotTimeout 37 37 0.00%
BenchmarkTCP6Persistent 0 0 n/a%
BenchmarkTCP6PersistentTimeout 0 0 n/a%
BenchmarkTCP4ConcurrentReadWrite 0 0 n/a%
BenchmarkTCP6ConcurrentReadWrite 0 0 n/a%
benchmark old bytes new bytes delta
BenchmarkTCP4OneShot 2500 2503 0.12%
BenchmarkTCP4OneShotTimeout 2508 2505 -0.12%
BenchmarkTCP4Persistent 0 0 n/a%
BenchmarkTCP4PersistentTimeout 0 0 n/a%
BenchmarkTCP6OneShot 2713 2707 -0.22%
BenchmarkTCP6OneShotTimeout 2722 2720 -0.07%
BenchmarkTCP6Persistent 0 0 n/a%
BenchmarkTCP6PersistentTimeout 0 0 n/a%
BenchmarkTCP4ConcurrentReadWrite 0 0 n/a%
BenchmarkTCP6ConcurrentReadWrite 0 0 n/a%
R=golang-dev, bradfitz, nightlyone, rsc
CC=golang-dev
https://golang.org/cl/12416043
This CL adds the netaddr interface that will carry a single network
endpoint address or a short list of IP addresses to dial helper
functions in the upcoming CLs.
This is in preparation for TCP connection setup with fast failover on
dual IP stack node as described in RFC 6555.
Update #3610
Update #5267
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/13368044
Also avoids platform-dependent datagram truncation in raw IP tests.
At least it's different between Windows and others.
Fixes#6122.
R=alex.brainman
CC=golang-dev
https://golang.org/cl/12843043
The mutex, fdMutex, handles locking and lifetime of sysfd,
and serializes Read and Write methods.
This allows to strip 2 sync.Mutex.Lock calls,
2 sync.Mutex.Unlock calls, 1 defer and some amount
of misc overhead from every network operation.
On linux/amd64, Intel E5-2690:
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 9595 9454 -1.47%
BenchmarkTCP4Persistent-2 8978 8772 -2.29%
BenchmarkTCP4ConcurrentReadWrite 4900 4625 -5.61%
BenchmarkTCP4ConcurrentReadWrite-2 2603 2500 -3.96%
In general it strips 70-500 ns from every network operation depending
on processor model. On my relatively new E5-2690 it accounts to ~5%
of network op cost.
Fixes#6074.
R=golang-dev, bradfitz, alex.brainman, iant, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/12418043
Unlike the existing net package own pollster, runtime-integrated
network pollster on BSD variants, actually kqueue, requires a socket
that has beed passed to syscall.Listen previously for a stream
listener.
This CL separates pollDesc.Init (actually runtime_pollOpen) from newFD
to allow control of each state of sockets and adds init method to netFD
instead. Upcoming CLs will rearrange the call order of runtime-integrated
pollster and syscall functions like the following;
- For dialers that open active connections, runtime_pollOpen will be
called in between syscall.Bind and syscall.Connect.
- For stream listeners that open passive stream connections,
runtime_pollOpen will be called just after syscall.Listen.
- For datagram listeners that open datagram connections,
runtime_pollOpen will be called just after syscall.Bind.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=dvyukov, alex.brainman, minux.ma
CC=golang-dev
https://golang.org/cl/8608044
Embed all data necessary for read/write operations directly into netFD.
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 27669 23341 -15.64%
BenchmarkTCP4Persistent-2 18173 12558 -30.90%
BenchmarkTCP4Persistent-4 10390 7319 -29.56%
This change will intentionally break all builders to see
how many allocations they do per read/write.
This will be fixed soon afterwards.
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/12413043
Runtime netpoll supports at most one read waiter
and at most one write waiter. It's responsibility
of net package to ensure that. Currently windows
implementation allows more than one waiter in Accept.
It leads to "fatal error: netpollblock: double wait".
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12400045
Close netpoll descriptor along with socket.
Ensure that error paths close the descriptor as well.
R=golang-dev, mikioh.mikioh, alex.brainman
CC=golang-dev
https://golang.org/cl/11987043
Fixes performance of the current windows network poller
with the new scheduler.
Gives runtime a hint when GetQueuedCompletionStatus() will block.
Fixes#5068.
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 4004000 33906 -99.15%
BenchmarkTCP4Persistent-2 21790 17513 -19.63%
BenchmarkTCP4Persistent-4 44760 34270 -23.44%
BenchmarkTCP4Persistent-6 45280 43000 -5.04%
R=golang-dev, alex.brainman, coocood, rsc
CC=golang-dev
https://golang.org/cl/7612045
64bit atomics are broken on 32bit systems. This is issue 599.
linux/arm builders all broke with this change, I am concerned that the other 32bit builders are silently impacted.
««« original CL description
net: fix data races on deadline vars
Fixes#4434.
R=mikioh.mikioh, bradfitz, dvyukov, alex.brainman
CC=golang-dev
https://golang.org/cl/6855110
»»»
R=rsc, mikioh.mikioh, dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/6852105
Thank you zhoumichaely for original CL 5175042.
Fixes#1740.
Fixes#2315.
R=golang-dev, bradfitz, mikioh.mikioh
CC=golang-dev, zhoumichaely
https://golang.org/cl/6822045
It is common to close network connection while another goroutine is
blocked reading on another goroutine. This sequence corresponds to
windows calls to WSARecv to start io, followed by GetQueuedCompletionStatus
that blocks until io completes, and, finally, closesocket called from
another thread. We were expecting that closesocket would unblock
GetQueuedCompletionStatus, and it does, but not always
(http://code.google.com/p/go/issues/detail?id=4170#c5). Also that sequence
results in connection is being reset.
This CL inserts CancelIo between GetQueuedCompletionStatus and closesocket,
and waits for both WSARecv and GetQueuedCompletionStatus to complete before
proceeding to closesocket. This seems to fix both connection resets and
issue 4170. It also makes windows code behave similar to unix version.
Unfortunately, CancelIo needs to be called on the same thread as WSARecv.
So we have to employ strategy we use for connections with deadlines to
every connection now. It means, there are 2 unavoidable thread switches
for every io. Some newer versions of windows have new CancelIoEx api that
doesn't have these drawbacks, and this CL uses this capability when available.
As time goes by, we should have less of CancelIo and more of CancelIoEx
systems. Computers with CancelIoEx are also not affected by issue 4195 anymore.
Fixes#3710Fixes#3746Fixes#4170
Partial fix for issue 4195
R=golang-dev, mikioh.mikioh, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/6604072
The set of errors forwarded by the os package varied with system and
was therefore non-portable.
Three helpers added for portable error checking: IsExist, IsNotExist, and IsPermission.
One or two more may need to come, but let's keep the set very small to discourage
thinking about errors that way.
R=mikioh.mikioh, gustavo, r, rsc
CC=golang-dev
https://golang.org/cl/5672047
Once we've evicted all the blocked I/O, the ref count
should go to zero quickly, so it should be safe to
postpone the close(2) until then.
Fixes#1898.
Fixes#2116.
Fixes#2122.
R=golang-dev, mikioh.mikioh, bradfitz, fullung, iant
CC=golang-dev
https://golang.org/cl/5649076
Both are unused and undocumented.
InvalidConnError is also non-idiomatic: a FooError type can
typically describe something, else it would be an ErrFoo
variable.
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/5609045
It works with NewTicker too, but is potentially a bit less efficient,
and reads wrong.
This is what happens when you TBR Windows changes, I guess.
R=golang-dev, gri, iant
CC=golang-dev
https://golang.org/cl/5536060
Previously, a timeout (in int64 nanoseconds) applied to a granularity
even smaller than one operation: a 100 byte read with a 1 second timeout
could take 100 seconds, if the bytes all arrived on the network 1 second
apart. This was confusing.
Rather than making the timeout granularity be per-Read/Write,
this CL makes callers set an absolute deadline (in time.Time)
after which operations will fail. This makes it possible to
set deadlines at higher levels, without knowing exactly how
many read/write operations will happen in e.g. reading an HTTP
request.
Fixes#2723
R=r, rsc, dave
CC=golang-dev
https://golang.org/cl/5555048
- syscall (not os) now defines the Errno type.
- the low-level assembly functions Syscall, Syscall6, and so on
return Errno, not uintptr
- syscall wrappers all return error, not uintptr.
R=golang-dev, mikioh.mikioh, r, alex.brainman
CC=golang-dev
https://golang.org/cl/5372080
