Just for readability reasons; to prevent overlooking deadline stuff
across over platforms.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/8656044
Malformed domain attributes are not sent in a Set-Cookie header.
Instead the domain attribute is dropped which turns the cookie
into a host-only cookie. This is much safer than dropping characters
from domain attribute.
Domain attributes with a leading dot '.' are still allowed, even
if discouraged by RFC 6265 section 4.1.1.
Fixes#6013
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12745043
Probably we should remove this type before Go 1 contract has settled,
but too late. Instead, keep InvalidAddrError close to package generic
error types.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/12670044
lookup_plan9.go's lookupSRV is using the wrong order for srv results. order should be weight, priority, port, following the response from /net/dns:
chi Aug 9 20:31:13 Rread tag 20 count 61 '_xmpp-client._tcp.offblast.org srv 5 0 5222 iota.offblast.org' 72
R=golang-dev, bradfitz
CC=ality, golang-dev, r, rsc
https://golang.org/cl/12708043
The mutex, fdMutex, handles locking and lifetime of sysfd,
and serializes Read and Write methods.
This allows to strip 2 sync.Mutex.Lock calls,
2 sync.Mutex.Unlock calls, 1 defer and some amount
of misc overhead from every network operation.
On linux/amd64, Intel E5-2690:
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 9595 9454 -1.47%
BenchmarkTCP4Persistent-2 8978 8772 -2.29%
BenchmarkTCP4ConcurrentReadWrite 4900 4625 -5.61%
BenchmarkTCP4ConcurrentReadWrite-2 2603 2500 -3.96%
In general it strips 70-500 ns from every network operation depending
on processor model. On my relatively new E5-2690 it accounts to ~5%
of network op cost.
Fixes#6074.
R=golang-dev, bradfitz, alex.brainman, iant, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/12418043
Unlike the existing net package own pollster, runtime-integrated
network pollster on BSD variants, actually kqueue, requires a socket
that has beed passed to syscall.Listen previously for a stream
listener.
This CL separates pollDesc.Init of Unix network pollster from newFD
to avoid any breakages in the transition from Unix network pollster
to runtime-integrated pollster. Upcoming CLs will rearrange the call
order of pollster and syscall functions like the following;
- For dialers that open active connections, pollDesc.Init will be
called in between syscall.Bind and syscall.Connect.
- For stream listeners that open passive stream connections,
pollDesc.Init will be called just after syscall.Listen.
- For datagram listeners that open datagram connections,
pollDesc.Init will be called just after syscall.Bind.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=dvyukov, bradfitz
CC=golang-dev
https://golang.org/cl/12663043
Having a trailing dot in the string doesn't really simplify
the checking loop in isDomainName. Avoid this unnecessary allocation.
Also make the valid domain names more explicit by adding some more
test cases.
benchmark old ns/op new ns/op delta
BenchmarkDNSNames 2420.0 983.0 -59.38%
benchmark old allocs new allocs delta
BenchmarkDNSNames 12 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkDNSNames 336 0 -100.00%
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12662043
The ResponseWriter's ReadFrom method was causing side effects on
the output before any data was read.
Now, bail out early and do a normal copy (which does a read
before writing) when our input and output are known to not to
be the pair of types we need for sendfile.
Fixes#5660
R=golang-dev, rsc, nightlyone
CC=golang-dev
https://golang.org/cl/12632043
There were some issues with the code sometimes using base64.StdEncoding,
and sometimes base64.URLEncoding.
Encoding basic authentication is now always done by the same code.
Fixes#5970.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12397043
HTTP/1.0 connections are closed implicitly, unless otherwise specified.
Note that this change does not test or fix "request too large" responses.
Reasoning: (a) it complicates tests and fixes, (b) they should be rare,
and (c) this is just a minor wire optimization, and thus not really worth worrying
about in this context.
Fixes#5955.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12435043
A response to a HEAD request is supposed to look the same as a
response to a GET request, just without a body.
HEAD requests are incredibly rare in the wild.
The Go net/http package has so far treated HEAD requests
specially: a Write on our default ResponseWriter returned
ErrBodyNotAllowed, telling handlers that something was wrong.
This was to optimize the fast path for HEAD requests, but:
1) because HEAD requests are incredibly rare, they're not
worth having a fast path for.
2) Letting the http.Handler handle but do nop Writes is still
very fast.
3) this forces ugly error handling into the application.
e.g. https://code.google.com/p/go/source/detail?r=6f596be7a31e
and related.
4) The net/http package nowadays does Content-Type sniffing,
but you don't get that for HEAD.
5) The net/http package nowadays does Content-Length counting
for small (few KB) responses, but not for HEAD.
6) ErrBodyNotAllowed was useless. By the time you received it,
you had probably already done all your heavy computation
and I/O to calculate what to write.
So, this change makes HEAD requests like GET requests.
We now count content-length and sniff content-type for HEAD
requests. If you Write, it doesn't return an error.
If you want a fast-path in your code for HEAD, you have to do
it early and set all the response headers yourself. Just like
before. If you choose not to Write in HEAD requests, be sure
to set Content-Length if you know it. We won't write
"Content-Length: 0" because you might've just chosen to not
write (or you don't know your Content-Length in advance).
Fixes#5454
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/12583043
This CL refactors the existing listenerSockaddr function into several
methods on netFD.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave, alex.brainman, dvyukov, remyoudompheng
CC=golang-dev
https://golang.org/cl/12023043
Looks like latest FreeBSD doesn't set address family identifer
for RTAX_NETMASK stuff; probably RTAX_GENMASK too, not confirmed.
This CL tries to identify address families by using the length of
each socket address if possible.
The issue is confirmed on FreeBSD 9.1.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12332043
Unlike the existing net package own pollster, runtime-integrated
network pollster on BSD variants, actually kqueue, requires a socket
that has beed passed to syscall.Listen previously for a stream
listener.
This CL separates pollDesc.Init (actually runtime_pollOpen) from newFD
to allow control of each state of sockets and adds init method to netFD
instead. Upcoming CLs will rearrange the call order of runtime-integrated
pollster and syscall functions like the following;
- For dialers that open active connections, runtime_pollOpen will be
called in between syscall.Bind and syscall.Connect.
- For stream listeners that open passive stream connections,
runtime_pollOpen will be called just after syscall.Listen.
- For datagram listeners that open datagram connections,
runtime_pollOpen will be called just after syscall.Bind.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=dvyukov, alex.brainman, minux.ma
CC=golang-dev
https://golang.org/cl/8608044
Embed all data necessary for read/write operations directly into netFD.
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 27669 23341 -15.64%
BenchmarkTCP4Persistent-2 18173 12558 -30.90%
BenchmarkTCP4Persistent-4 10390 7319 -29.56%
This change will intentionally break all builders to see
how many allocations they do per read/write.
This will be fixed soon afterwards.
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/12413043
This means that in the common case (modern kernel), we only
make 1 system call to dup instead of two, and we also avoid
grabbing the syscall.ForkLock.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/12476043
Runtime netpoll supports at most one read waiter
and at most one write waiter. It's responsibility
of net package to ensure that. Currently windows
implementation allows more than one waiter in Accept.
It leads to "fatal error: netpollblock: double wait".
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12400045
Whether the keys are concatenated or separate (or a mixture) depends on the server.
Fixes#5979.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12433043
Windows dynamic priority boosting assumes that a process has different types
of dedicated threads -- GUI, IO, computational, etc. Go processes use
equivalent threads that all do a mix of GUI, IO, computations, etc.
In such context dynamic priority boosting does nothing but harm, so turn it off.
In particular, if 2 goroutines do heavy IO on a server uniprocessor machine,
windows rejects to schedule timer thread for 2+ seconds when priority boosting is enabled.
Fixes#5971.
R=alex.brainman
CC=golang-dev
https://golang.org/cl/12406043
This CL makes IPAddr, UDPAddr and TCPAddr implement sockaddr
interface, UnixAddr is already sockaddr interface compliant, and
reduces unnecessary conversions between net.Addr, net.sockaddr and
syscall.Sockaddr.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave, bradfitz
CC=golang-dev
https://golang.org/cl/12010043
If netFD is closed by finalizer, runtime netpoll descriptor is not freed.
R=golang-dev, dave, alex.brainman
CC=golang-dev
https://golang.org/cl/12037043
Close netpoll descriptor along with socket.
Ensure that error paths close the descriptor as well.
R=golang-dev, mikioh.mikioh, alex.brainman
CC=golang-dev
https://golang.org/cl/11987043
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, fvbommel, dave
CC=golang-dev
https://golang.org/cl/11984043
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11932044
This CL extends existing sockaddr interface to accommodate not only
internet protocol family endpoint addressess but unix network family
endpoint addresses.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11979043
When making an HTTPS client request, respect the
ServerName field in the tls.Config.
Fixes#5829
R=golang-dev, agl, adg
CC=golang-dev
https://golang.org/cl/11691043
Phrases like "returns whether or not the image is opaque" could be
describing what the function does (it always returns, regardless of
the opacity) or what it returns (a boolean indicating the opacity).
Even when the "or not" is missing, the phrasing is bizarre.
Go with "reports whether", which is still clunky but at least makes
it clear we're talking about the return value.
These were edited by hand. A few were cleaned up in other ways.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/11699043
Sets both the duration from the last data packet to the first
keep alive packet and the duration between keep alive packets to be
the passed duration.
I've tested the function on both Darwin (10.8.4) and 4.2 Linux.
I've compiled (make.bash) for all the OS's and tested (all.bash)
on Darwin and Linux.
R=golang-dev, dave, rsc, dvyukov, presotto+facebook, nick
CC=golang-dev, veyron-team
https://golang.org/cl/11130044
ReadMIMEHeader is used by net/http, net/mail, and
mime/multipart.
Don't do so many small allocations. Calculate up front
how much we'll probably need.
benchmark old ns/op new ns/op delta
BenchmarkReadMIMEHeader 8433 7467 -11.45%
benchmark old allocs new allocs delta
BenchmarkReadMIMEHeader 23 14 -39.13%
benchmark old bytes new bytes delta
BenchmarkReadMIMEHeader 1705 1343 -21.23%
R=golang-dev, r, iant, adg
CC=golang-dev
https://golang.org/cl/8179043
- change runtime_pollWait so it does not return
closed or timeout if IO is ready - windows must
know if IO has completed or not even after
interruption;
- add (*pollDesc).Prepare(mode int) that can be
used for both read and write, same for Wait;
- introduce runtime_pollWaitCanceled and expose
it in net as (*pollDesc).WaitCanceled(mode int);
Full windows netpoll changes are
here https://golang.org/cl/8670044/.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/10485043
If a server response contains a Content-Length and the body is short,
the Transport should end in io.ErrUnexpectedEOF, not io.EOF.
Fixes#5738
R=golang-dev, kevlar, r
CC=golang-dev
https://golang.org/cl/10237050
trivial: it is not a serious problem to leak a fd in a short lived process, but it was obscuring my investigation of issue 5593.
R=golang-dev, iant, bradfitz
CC=golang-dev
https://golang.org/cl/10391043
The old code worked, somewhat on accident, but was confusing,
and had a useless assignment to the inner err. It worked
because url.Parse parses just about anything, so the outer err
was always nil, so it always fell through to the bottom return
statement, even without the "err = nil" line.
Instead, just have two return statements, and add a comment.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10448044
use correct field count when resolving nameservers via /net/dns on Plan 9.
we incorrectly check for 4 fields instead of 3 when parsing the result of /net/dns, and get no results
R=golang-dev, ality
CC=golang-dev
https://golang.org/cl/10182044
In Issue 5625, Russ says: "We should at least have a cache of
inflight lookups, so that 100 simultaneous dials of one host
name don't do the work 100x. That's easy and (assume we forget
the answer once they all get it) doesn't pose any consistency
problems. It just merges simultaneous work."
This brings in singleflight (unexported) from Google /
Camlistore, but without its tests. Maybe we should put it
somewhere in the standard library. But not now.
Update #5625
R=golang-dev, iant, cespare, rsc, dave, rogpeppe, remyoudompheng
CC=golang-dev
https://golang.org/cl/10079043
Several places used io.WriteString unnecessarily when the
static type already implemented WriteString. No need to
check for it at runtime.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9608043
Makes it possible to return the spent runtime.PollDesc to
runtime.pollcache descriptor pool when netFD.connect or
syscall.Listen fails.
Fixes#5219.
R=dvyukov, dave, bradfitz, adg
CC=golang-dev
https://golang.org/cl/8318044
We've decided to leave logging to third-parties (there are too
many formats), which others have done.
And we can't change the behavior of the various response
fields at this point anyway. Plus I argue they're correct and
match their documention.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8391043
The smtp package originally allowed PLAIN whenever, but then
the TLS check was added for paranoia, but it's too paranoid:
it prevents using PLAIN auth even from localhost to localhost
when the server advertises PLAIN support.
This CL also permits the client to send PLAIN if the server
advertises it.
Fixes#5184
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8279043
Save an allocation per GET request and don't call io.LimitedReader(r, 0)
just to read 0 bytes. There's already an eofReader global variable
for when we just want a non-nil io.Reader to immediately EOF.
(Sorry, I know Rob told me to stop, but I was bored on the plane and
wrote this before I received the recent "please, really stop" email.)
benchmark old ns/op new ns/op delta
BenchmarkServerHandlerTypeLen 13888 13279 -4.39%
BenchmarkServerHandlerNoLen 12912 12229 -5.29%
BenchmarkServerHandlerNoType 13348 12632 -5.36%
BenchmarkServerHandlerNoHeader 10911 10261 -5.96%
benchmark old allocs new allocs delta
BenchmarkServerHandlerTypeLen 20 19 -5.00%
BenchmarkServerHandlerNoLen 18 17 -5.56%
BenchmarkServerHandlerNoType 18 17 -5.56%
BenchmarkServerHandlerNoHeader 13 12 -7.69%
benchmark old bytes new bytes delta
BenchmarkServerHandlerTypeLen 1913 1878 -1.83%
BenchmarkServerHandlerNoLen 1878 1843 -1.86%
BenchmarkServerHandlerNoType 1878 1844 -1.81%
BenchmarkServerHandlerNoHeader 1085 1051 -3.13%
Fixes#5188
R=golang-dev, adg, r
CC=golang-dev
https://golang.org/cl/8297044
It was unnecessarily cloning and then mutating a map that had
a very short lifetime (just that function).
No new tests, because they were added in revision 833bf2ef1527
(TestHeaderToWire). The benchmarks below are from the earlier
commit, revision 52e3407d.
I noticed this inefficiency when reviewing a change Peter Buhr
is looking into, which will also use these benchmarks.
benchmark old ns/op new ns/op delta
BenchmarkServerHandlerTypeLen 12547 12325 -1.77%
BenchmarkServerHandlerNoLen 12466 11167 -10.42%
BenchmarkServerHandlerNoType 12699 11800 -7.08%
BenchmarkServerHandlerNoHeader 11901 9210 -22.61%
benchmark old allocs new allocs delta
BenchmarkServerHandlerTypeLen 21 20 -4.76%
BenchmarkServerHandlerNoLen 20 18 -10.00%
BenchmarkServerHandlerNoType 20 18 -10.00%
BenchmarkServerHandlerNoHeader 17 13 -23.53%
benchmark old bytes new bytes delta
BenchmarkServerHandlerTypeLen 1930 1913 -0.88%
BenchmarkServerHandlerNoLen 1912 1879 -1.73%
BenchmarkServerHandlerNoType 1912 1878 -1.78%
BenchmarkServerHandlerNoHeader 1491 1086 -27.16%
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/8268046
Closes the API documentation gap between platforms.
Also makes the code textual representation same between platforms.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8148043
Closes the API documentation gap between platforms.
Also makes the code textual representation same between platforms.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8147043
Since we can't properly handle anything except 100, treat all
1xx informational responses as sketchy and don't reuse the
connection for future requests.
The only other 1xx response code currently in use in the wild
is WebSockets' use of "101 Switching Protocols", but our
code.google.com/p/go.net/websockets doesn't use Client or
Transport: it uses ReadResponse directly, so is unaffected by
this CL. (and its tests still pass)
So this CL is entirely just future-proofing paranoia.
Also: the Internet is weird.
Update #2184
Update #3665
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/8208043
Whoops. I'm surprised it even worked before. (Need two pipes,
not one.)
Also, remove the whole pipe registration business, since it
wasn't even required in the previous version. (I'd later fixed
it at the end of send100Response, but forgot to delete it)
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8191044
This CL ensures we use the correct socket options for
passive and active open sockets.
For the passive open sockets created by Listen functions,
additional SO_REUSEADDR, SO_REUSEPORT options are required
for the quick service restart and/or multicasting.
For the active open sockets created by Dial functions, no
additional options are required.
R=golang-dev, dave, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/7795050
"There are only two hard problems in computer science:
cache invalidation, naming things, and off-by-one errors."
The HTTP server code already strips Expect: 100-continue on
requests, so httputil.ReverseProxy should be unaffected, but
some servers send unsolicited HTTP/1.1 100 Continue responses,
so we need to skip over them if they're seen to avoid getting
off-by-one on Transport requests/responses.
This does change the behavior of people who were using Client
or Transport directly and explicitly setting "Expect: 100-continue"
themselves, but it didn't work before anyway. Now instead of the
user code seeing a 100 response and then things blowing up, now
it basically works, except the Transport will still blast away
the full request body immediately. That's the part that needs
to be finished to close this issue.
This is the safe quick fix.
Update #3665
R=golang-dev, dsymonds, dave, jgrahamc
CC=golang-dev
https://golang.org/cl/8166045
Adds the missing wildcard port assignment description to ListenUDP.
Also updates the wildcard port description on ListenTCP.
R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/8063043
Also removes redundant tests that run Go 1.0 non-IPv6 support
Windows code on IPv6 enabled Windows kernels.
R=alex.brainman, golang-dev, bradfitz, dave
CC=golang-dev
https://golang.org/cl/7812052
Saves both the textproto.Reader allocation, and its internal
scratch buffer growing.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 10324 10149 -1.70%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 19 17 -10.53%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1559 1492 -4.30%
R=golang-dev, r, gri
CC=golang-dev
https://golang.org/cl/8094046
Removes another per-request allocation. Also makes the code more
readable, IMO. And more testable.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 10539 10324 -2.04%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 20 19 -5.00%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1609 1559 -3.11%
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8118044
We already depend on strings in this file, so use it.
Plus strings.Index will be faster than a manual loop
once issue 3751 is finished.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/8116043
A chunkWriter and a response are 1:1. Make them contiguous in
memory and save an allocation.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 10715 10539 -1.64%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 21 20 -4.76%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1626 1609 -1.05%
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8114043
Add more tests around the various orders handlers can access
and flush response headers.
Also clarify the documentation on fields of response and
chunkWriter.
While there, remove an allocation (a header clone) for simple
handlers.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 15245 14966 -1.83%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 24 23 -4.17%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1717 1668 -2.85%
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8101043
There was another bufio.Writer not being reused, found with
GOGC=off and -test.memprofile.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAlive 18270 16046 -12.17%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAlive 38 36 -5.26%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAlive 4598 2488 -45.89%
Update #5100
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8038047
This CL avoids test data sharing in repetitive test runs;
e.g., go test net -cpu=1,1,1
R=golang-dev, fullung, bradfitz
CC=golang-dev
https://golang.org/cl/8011043
Fixes performance of the current windows network poller
with the new scheduler.
Gives runtime a hint when GetQueuedCompletionStatus() will block.
Fixes#5068.
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 4004000 33906 -99.15%
BenchmarkTCP4Persistent-2 21790 17513 -19.63%
BenchmarkTCP4Persistent-4 44760 34270 -23.44%
BenchmarkTCP4Persistent-6 45280 43000 -5.04%
R=golang-dev, alex.brainman, coocood, rsc
CC=golang-dev
https://golang.org/cl/7612045
