ReadMIMEHeader is used by net/http, net/mail, and
mime/multipart.
Don't do so many small allocations. Calculate up front
how much we'll probably need.
benchmark old ns/op new ns/op delta
BenchmarkReadMIMEHeader 8433 7467 -11.45%
benchmark old allocs new allocs delta
BenchmarkReadMIMEHeader 23 14 -39.13%
benchmark old bytes new bytes delta
BenchmarkReadMIMEHeader 1705 1343 -21.23%
R=golang-dev, r, iant, adg
CC=golang-dev
https://golang.org/cl/8179043
- change runtime_pollWait so it does not return
closed or timeout if IO is ready - windows must
know if IO has completed or not even after
interruption;
- add (*pollDesc).Prepare(mode int) that can be
used for both read and write, same for Wait;
- introduce runtime_pollWaitCanceled and expose
it in net as (*pollDesc).WaitCanceled(mode int);
Full windows netpoll changes are
here https://golang.org/cl/8670044/.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/10485043
If a server response contains a Content-Length and the body is short,
the Transport should end in io.ErrUnexpectedEOF, not io.EOF.
Fixes#5738
R=golang-dev, kevlar, r
CC=golang-dev
https://golang.org/cl/10237050
trivial: it is not a serious problem to leak a fd in a short lived process, but it was obscuring my investigation of issue 5593.
R=golang-dev, iant, bradfitz
CC=golang-dev
https://golang.org/cl/10391043
The old code worked, somewhat on accident, but was confusing,
and had a useless assignment to the inner err. It worked
because url.Parse parses just about anything, so the outer err
was always nil, so it always fell through to the bottom return
statement, even without the "err = nil" line.
Instead, just have two return statements, and add a comment.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10448044
use correct field count when resolving nameservers via /net/dns on Plan 9.
we incorrectly check for 4 fields instead of 3 when parsing the result of /net/dns, and get no results
R=golang-dev, ality
CC=golang-dev
https://golang.org/cl/10182044
In Issue 5625, Russ says: "We should at least have a cache of
inflight lookups, so that 100 simultaneous dials of one host
name don't do the work 100x. That's easy and (assume we forget
the answer once they all get it) doesn't pose any consistency
problems. It just merges simultaneous work."
This brings in singleflight (unexported) from Google /
Camlistore, but without its tests. Maybe we should put it
somewhere in the standard library. But not now.
Update #5625
R=golang-dev, iant, cespare, rsc, dave, rogpeppe, remyoudompheng
CC=golang-dev
https://golang.org/cl/10079043
Several places used io.WriteString unnecessarily when the
static type already implemented WriteString. No need to
check for it at runtime.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9608043
Makes it possible to return the spent runtime.PollDesc to
runtime.pollcache descriptor pool when netFD.connect or
syscall.Listen fails.
Fixes#5219.
R=dvyukov, dave, bradfitz, adg
CC=golang-dev
https://golang.org/cl/8318044
We've decided to leave logging to third-parties (there are too
many formats), which others have done.
And we can't change the behavior of the various response
fields at this point anyway. Plus I argue they're correct and
match their documention.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8391043
The smtp package originally allowed PLAIN whenever, but then
the TLS check was added for paranoia, but it's too paranoid:
it prevents using PLAIN auth even from localhost to localhost
when the server advertises PLAIN support.
This CL also permits the client to send PLAIN if the server
advertises it.
Fixes#5184
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8279043
Save an allocation per GET request and don't call io.LimitedReader(r, 0)
just to read 0 bytes. There's already an eofReader global variable
for when we just want a non-nil io.Reader to immediately EOF.
(Sorry, I know Rob told me to stop, but I was bored on the plane and
wrote this before I received the recent "please, really stop" email.)
benchmark old ns/op new ns/op delta
BenchmarkServerHandlerTypeLen 13888 13279 -4.39%
BenchmarkServerHandlerNoLen 12912 12229 -5.29%
BenchmarkServerHandlerNoType 13348 12632 -5.36%
BenchmarkServerHandlerNoHeader 10911 10261 -5.96%
benchmark old allocs new allocs delta
BenchmarkServerHandlerTypeLen 20 19 -5.00%
BenchmarkServerHandlerNoLen 18 17 -5.56%
BenchmarkServerHandlerNoType 18 17 -5.56%
BenchmarkServerHandlerNoHeader 13 12 -7.69%
benchmark old bytes new bytes delta
BenchmarkServerHandlerTypeLen 1913 1878 -1.83%
BenchmarkServerHandlerNoLen 1878 1843 -1.86%
BenchmarkServerHandlerNoType 1878 1844 -1.81%
BenchmarkServerHandlerNoHeader 1085 1051 -3.13%
Fixes#5188
R=golang-dev, adg, r
CC=golang-dev
https://golang.org/cl/8297044
It was unnecessarily cloning and then mutating a map that had
a very short lifetime (just that function).
No new tests, because they were added in revision 833bf2ef1527
(TestHeaderToWire). The benchmarks below are from the earlier
commit, revision 52e3407d.
I noticed this inefficiency when reviewing a change Peter Buhr
is looking into, which will also use these benchmarks.
benchmark old ns/op new ns/op delta
BenchmarkServerHandlerTypeLen 12547 12325 -1.77%
BenchmarkServerHandlerNoLen 12466 11167 -10.42%
BenchmarkServerHandlerNoType 12699 11800 -7.08%
BenchmarkServerHandlerNoHeader 11901 9210 -22.61%
benchmark old allocs new allocs delta
BenchmarkServerHandlerTypeLen 21 20 -4.76%
BenchmarkServerHandlerNoLen 20 18 -10.00%
BenchmarkServerHandlerNoType 20 18 -10.00%
BenchmarkServerHandlerNoHeader 17 13 -23.53%
benchmark old bytes new bytes delta
BenchmarkServerHandlerTypeLen 1930 1913 -0.88%
BenchmarkServerHandlerNoLen 1912 1879 -1.73%
BenchmarkServerHandlerNoType 1912 1878 -1.78%
BenchmarkServerHandlerNoHeader 1491 1086 -27.16%
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/8268046
Closes the API documentation gap between platforms.
Also makes the code textual representation same between platforms.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8148043
Closes the API documentation gap between platforms.
Also makes the code textual representation same between platforms.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8147043
Since we can't properly handle anything except 100, treat all
1xx informational responses as sketchy and don't reuse the
connection for future requests.
The only other 1xx response code currently in use in the wild
is WebSockets' use of "101 Switching Protocols", but our
code.google.com/p/go.net/websockets doesn't use Client or
Transport: it uses ReadResponse directly, so is unaffected by
this CL. (and its tests still pass)
So this CL is entirely just future-proofing paranoia.
Also: the Internet is weird.
Update #2184
Update #3665
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/8208043
Whoops. I'm surprised it even worked before. (Need two pipes,
not one.)
Also, remove the whole pipe registration business, since it
wasn't even required in the previous version. (I'd later fixed
it at the end of send100Response, but forgot to delete it)
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8191044
This CL ensures we use the correct socket options for
passive and active open sockets.
For the passive open sockets created by Listen functions,
additional SO_REUSEADDR, SO_REUSEPORT options are required
for the quick service restart and/or multicasting.
For the active open sockets created by Dial functions, no
additional options are required.
R=golang-dev, dave, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/7795050
"There are only two hard problems in computer science:
cache invalidation, naming things, and off-by-one errors."
The HTTP server code already strips Expect: 100-continue on
requests, so httputil.ReverseProxy should be unaffected, but
some servers send unsolicited HTTP/1.1 100 Continue responses,
so we need to skip over them if they're seen to avoid getting
off-by-one on Transport requests/responses.
This does change the behavior of people who were using Client
or Transport directly and explicitly setting "Expect: 100-continue"
themselves, but it didn't work before anyway. Now instead of the
user code seeing a 100 response and then things blowing up, now
it basically works, except the Transport will still blast away
the full request body immediately. That's the part that needs
to be finished to close this issue.
This is the safe quick fix.
Update #3665
R=golang-dev, dsymonds, dave, jgrahamc
CC=golang-dev
https://golang.org/cl/8166045
Adds the missing wildcard port assignment description to ListenUDP.
Also updates the wildcard port description on ListenTCP.
R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/8063043
Also removes redundant tests that run Go 1.0 non-IPv6 support
Windows code on IPv6 enabled Windows kernels.
R=alex.brainman, golang-dev, bradfitz, dave
CC=golang-dev
https://golang.org/cl/7812052
Saves both the textproto.Reader allocation, and its internal
scratch buffer growing.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 10324 10149 -1.70%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 19 17 -10.53%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1559 1492 -4.30%
R=golang-dev, r, gri
CC=golang-dev
https://golang.org/cl/8094046
Removes another per-request allocation. Also makes the code more
readable, IMO. And more testable.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 10539 10324 -2.04%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 20 19 -5.00%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1609 1559 -3.11%
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8118044
We already depend on strings in this file, so use it.
Plus strings.Index will be faster than a manual loop
once issue 3751 is finished.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/8116043
A chunkWriter and a response are 1:1. Make them contiguous in
memory and save an allocation.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 10715 10539 -1.64%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 21 20 -4.76%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1626 1609 -1.05%
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8114043
Add more tests around the various orders handlers can access
and flush response headers.
Also clarify the documentation on fields of response and
chunkWriter.
While there, remove an allocation (a header clone) for simple
handlers.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAliveLite 15245 14966 -1.83%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAliveLite 24 23 -4.17%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAliveLite 1717 1668 -2.85%
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8101043
There was another bufio.Writer not being reused, found with
GOGC=off and -test.memprofile.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnWithKeepAlive 18270 16046 -12.17%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnWithKeepAlive 38 36 -5.26%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnWithKeepAlive 4598 2488 -45.89%
Update #5100
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/8038047
This CL avoids test data sharing in repetitive test runs;
e.g., go test net -cpu=1,1,1
R=golang-dev, fullung, bradfitz
CC=golang-dev
https://golang.org/cl/8011043
Fixes performance of the current windows network poller
with the new scheduler.
Gives runtime a hint when GetQueuedCompletionStatus() will block.
Fixes#5068.
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 4004000 33906 -99.15%
BenchmarkTCP4Persistent-2 21790 17513 -19.63%
BenchmarkTCP4Persistent-4 44760 34270 -23.44%
BenchmarkTCP4Persistent-6 45280 43000 -5.04%
R=golang-dev, alex.brainman, coocood, rsc
CC=golang-dev
https://golang.org/cl/7612045
This CL updates CL 7511043;
- adds new test cases for both UDPConn and UnixConn,
- makes sure unnamed UnixAddr handling,
- replaces t.Errorf with t.Fatalf in sockname related test cases.
Fixes#3721 (again).
Fixes#3838 (again).
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7627048
Saves over 8KB of allocations per new connection.
benchmark old ns/op new ns/op delta
BenchmarkServerFakeConnNoKeepAlive 28777 24927 -13.38%
benchmark old allocs new allocs delta
BenchmarkServerFakeConnNoKeepAlive 52 46 -11.54%
benchmark old bytes new bytes delta
BenchmarkServerFakeConnNoKeepAlive 13716 5286 -61.46%
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/7799047
If a handler didn't consume all its Request.Body, child.go was
closing the socket while the host was still writing to it,
causing the child to send a RST and the host (at least nginx)
to send an empty response body.
Now, we tell the host we're done with the request/response
first, and then close our input pipe after consuming a bit of
it. Consuming the body fixes the problem, and flushing to the
host first to tell it that we're done increases the chance
that the host cuts off further data to us, meaning we won't
have much to consume.
No new tests, because this package is lacking in tests.
Tested by hand with nginx. See issue for testing details.
Fixes#4183
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7939045
Fixes#5061.
Current code relies on the fact that fd's are automatically removed from epoll set when closed. However, it is not true. Underlying file description is removed from epoll set only when *all* fd's referring to it are closed.
There are 2 bad consequences:
1. Kernel delivers notifications on already closed fd's.
2. The following sequence of events leads to error:
- add fd1 to epoll
- dup fd1 = fd2
- close fd1 (not removed from epoll since we've dup'ed the fd)
- dup fd2 = fd1 (get the same fd as fd1)
- add fd1 to epoll = EEXIST
So, if fd can be potentially dup'ed of fork'ed, it's necessary to explicitly remove the fd from epoll set.
R=golang-dev, bradfitz, dave
CC=golang-dev
https://golang.org/cl/7870043
Found this inconsistency from net/http's Server while
debugging Issue 4183
Unfortunately this package lacks testing around this,
or most of child.go. :/
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/7735046
No net package involved. And with ReportAllocs we can see
how much garbage is created per request.
R=adg, dave
CC=golang-dev
https://golang.org/cl/7913044
The example is the same as the FileServer one, but
it's relevant for both.
Also use strings.TrimPrefix while I'm here.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7598046
Bring net/fd_linux.go back (it was deleted this morning)
because it is still needed for ARM.
Fix a few typos in the runtime reorg.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/7759046
The problem is that new network poller can have spurious
rediness notifications. This implementation ensures that
the socket is actually connected.
R=golang-dev, rsc, akumar
CC=golang-dev
https://golang.org/cl/7785043
The system call takes an int, but the kernel stores it in a uint16.
At least one Linux system sets /proc/sys/net/core/somaxconn
to 262144, which ends up being 0 in the uint16. Avoid being tricked.
FreeBSD sources also store the backlog in a uint16.
Assume the problem is systemic and fix it everywhere.
Fixes#5030.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7480046
This isn't as bad as it used to be, but add a bit
more detail to close the issue.
Fixes#3359
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/7606044
This includes a simplified resolvePath function and tests for all normal and abnormal path resolution examples described in RFC 3986, sections 5.4.1 and 5.4.2 [1]. Some of those examples failed before (see http://play.golang.org/p/F0ApSaXniv).
Also, parsing a reference "//foo" now works as expected. It was treated as an absolute path with very weird results (see http://play.golang.org/p/089b-_xoNe).
During path resolution, all dot segments are removed as described by the RFC.
A few existing tests had to be changed because they expected the wrong output.
Fixes#4700.
Fixes#4706.
[1] http://tools.ietf.org/html/rfc3986#section-5.4.1
R=rsc, adg, bradfitz
CC=golang-dev
https://golang.org/cl/7203059
Previously it was evaluated once, so re-using the timeout option
repeatedly would always generate the same deadine.
Also switch to doing just one pass over the options, making
the private interface actually useful.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/7608045
Implement what Chrome calls socket "late binding". See:
https://insouciant.org/tech/connection-management-in-chromium/
In a nutshell, if our HTTP client needs a TCP connection to a
remote host and there's not an idle one available, rather than
kick off a dial and wait for that specific dial, we instead
kick off a dial and wait for either our own dial to finish, or
any other TCP connection to that same host to become
available.
The implementation looks like a classic "Learning Go
Concurrency" slide.
Chrome's commit and numbers:
http://src.chromium.org/viewvc/chrome?view=rev&revision=36230
R=golang-dev, daniel.morsing, adg
CC=golang-dev
https://golang.org/cl/7587043
This CL splits multicast listener tests into two; for IPv4 and
for IPv6. It also removes redundant test inputs and makes sure
that assignment of multicast interface to stablize the tests.
Fixes#4059.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7565043
Move pollServer from fd_unix.go to fd_poll_unix.go.
Add pollServerInit(*NetFD) to allow custom initialization.
Add pollServer.Close(*NetFD) to allow custom finalization.
Move setDeadline() to fd_poll_unix.go to allow custom handling of deadlines.
Move newPollServer() to fd_poll_unix.go to allow custom initialization.
No logical code changes.
The next step will be to turn off fd_poll_unix.go for some platform
(I have changes for darwin/linux) and redirect it into runtime. See:
https://golang.org/cl/7569043/diff/2001/src/pkg/net/fd_poll_runtime.go
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7513045
Ensure that accept/connect respect deadline,
even if the operation can be executed w/o blocking.
Note this changes external behavior, but it makes
it consistent with read/write.
Factor out deadline check into pollServer.PrepareRead/Write,
in preparation for edge triggered pollServer.
Ensure that pollServer.WaitRead/Write are not called concurrently
by adding rio/wio locks around connect/accept.
R=golang-dev, mikioh.mikioh, bradfitz, iant
CC=golang-dev
https://golang.org/cl/7436048
Some IDS somewhere thinks "Go http package" is a virus.
Make it something else for Go 1.1. Dumb but easy.
R=golang-dev, bradfitz, minux.ma
CC=golang-dev
https://golang.org/cl/7532043
Previously the HTTP client's (*Response).Body.Close would try
to keep reading until EOF, hoping to reuse the keep-alive HTTP
connection, but the EOF might never come, or it might take a
long time. Now we immediately close the TCP connection if we
haven't seen EOF.
This shifts the burden onto clients to read their whole response
bodies if they want the advantage of reusing TCP connections.
In the future maybe we could decide on heuristics to read some
number of bytes for some max amount of time before forcefully
closing, but I'd rather not for now.
Statistically, touching this code makes things regress, so I
wouldn't be surprised if this introduces new bugs, but all the
tests pass, and I think the code is simpler now too. Maybe.
Please test your HTTP client code before Go 1.1.
Fixes#3672
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/7419050
Use a 17 MB payload instead of a 1 MB payload, since
OS X can apparently buffer up to 16 MB in its pipes.
Fixes#4958 maybe
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/7453049
net/http/cookiejar.
This is a straight rename. There are no code changes.
Fixes#1960.
R=rsc, adg
CC=dr.volker.dobler, golang-dev
https://golang.org/cl/7436043
The test suite for ReadResponse was not checking the error return on the io.Copy
on the body. This was masking two errors: the handling of chunked responses to
HEAD requests and the handling of Content-Length > 0 to HEAD.
The former manifested itself as an 'unexpected EOF' when doing the io.Copy
because a chunked reader was assigned but there were no chunks to read. The
latter cause (*http.Response).Write to report an error on HEAD requests
because it saw a Content-Length > 0 and expected a body.
There was also a missing \r\n in one chunked test that meant that the chunked
encoding was malformed. This does not appear to have been intentional.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7407046
syscall: Use NewError for all system errors and introduce
some new errors for compatibility with other packages
and proper error handling in net. Also introduce
Temporary and Timeout methods on ErrorString.
net: Make errors from dial, accept, listen functions follow the
OpError standard and discern whether the underlying
error came from syscall. Since Plan 9 uses a correspondence
between file and network operations, all system error
reporting happens through the underlying file operation.
In Go code, we go through package os for file operations,
so there is another level of indirection in error types.
This change allows us to compare the errors with those in
package syscall, when appropriate.
os: Just use the error string already present in package os,
instead of calling out to package syscall.
R=rsc, ality, rminnich, bradfitz
CC=golang-dev
https://golang.org/cl/7398054
