1) Move StateHijacked callback earlier, to make it
panic-proof. A Hijack followed by a panic didn't previously
result in ConnState getting fired for StateHijacked. Move it
earlier, to the time of hijack.
2) Don't fire StateActive unless any bytes were read off the
wire while waiting for a request. This means we don't
transition from New or Idle to Active if the client
disconnects or times out. This was documented before, but not
implemented properly.
This CL is required for an pending fix for Issue 7264
LGTM=josharian
R=josharian
CC=golang-codereviews
https://golang.org/cl/69860049
In Go 1.2, closing a request body without reading to EOF
causes the underlying TCP connection to not be reused. This
client code following redirects was never updated when that
happened.
This was part of a previous CL but moved to its own CL at
Josh's request. Now with test too.
LGTM=josharian
R=josharian
CC=golang-codereviews
https://golang.org/cl/70800043
The addition of Server.ConnState provides all the necessary
hooks to stop a Server gracefully, but StateNew previously
could fire concurrently with Serve exiting (as it does when
its net.Listener is closed). This previously meant one
couldn't use a WaitGroup incremented in the StateNew hook
along with calling Wait after Serve. Now you can.
Update #4674
LGTM=bradfitz
R=bradfitz
CC=golang-codereviews
https://golang.org/cl/70410044
Update #4674
This allows for all sorts of graceful shutdown policies,
without picking a policy (e.g. lameduck period) and without
adding lots of locking to the server core. That policy and
locking can be implemented outside of net/http now.
LGTM=adg
R=golang-codereviews, josharian, r, adg, dvyukov
CC=golang-codereviews
https://golang.org/cl/69260044
See golang.org/s/go13nacl for design overview.
This CL is the mostly mechanical changes from rsc's Go 1.2 based NaCl branch, specifically 39cb35750369 to 500771b477cf from https://code.google.com/r/rsc-go13nacl. This CL does not include working NaCl support, there are probably two or three more large merges to come.
CL 15750044 is not included as it involves more invasive changes to the linker which will need to be merged separately.
The exact change lists included are
15050047: syscall: support for Native Client
15360044: syscall: unzip implementation for Native Client
15370044: syscall: Native Client SRPC implementation
15400047: cmd/dist, cmd/go, go/build, test: support for Native Client
15410048: runtime: support for Native Client
15410049: syscall: file descriptor table for Native Client
15410050: syscall: in-memory file system for Native Client
15440048: all: update +build lines for Native Client port
15540045: cmd/6g, cmd/8g, cmd/gc: support for Native Client
15570045: os: support for Native Client
15680044: crypto/..., hash/crc32, reflect, sync/atomic: support for amd64p32
15690044: net: support for Native Client
15690048: runtime: support for fake time like on Go Playground
15690051: build: disable various tests on Native Client
LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/68150047
Update #3362
Also set a 30 second timeout, instead of relying on the
operating system's timeout, which if often but not always 3
minutes.
LGTM=crawshaw
R=rsc, crawshaw
CC=golang-codereviews
https://golang.org/cl/68330046
This lays the groundwork for making Go robust when the system's
calendar time jumps around. All input values to the runtimeTimer
struct now use the runtime clock as a common reference point.
This affects net.Conn.Set[Read|Write]Deadline(), time.Sleep(),
time.Timer, etc. Under normal conditions, behavior is unchanged.
Each platform and architecture's implementation of runtime·nanotime()
should be modified to use a monotonic system clock when possible.
Platforms/architectures modified and tested with monotonic clock:
linux/x86 - clock_gettime(CLOCK_MONOTONIC)
Update #6007
LGTM=dvyukov, rsc
R=golang-codereviews, dvyukov, alex.brainman, stephen.gutekanst, dave, rsc, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/53010043
TestDNSThreadLimit creates tons of DNS queries and it occasionally
causes an unintentional traffic jam and/or crash of some virtual
machine software, especially its builtin networking stuff.
We can run TestDNSThreadLimit with -dnsflood flag instead.
LGTM=dave, rsc
R=rsc, dave
CC=golang-codereviews
https://golang.org/cl/63600043
4 KB is a bit too small in some situations (e.g. panic during a
template execution), and ends up with an unhelpfully-truncated trace.
64 KB should be much more likely to capture the useful information.
There's not a garbage generation issue, since this code should only
be triggered when there's something seriously wrong with the program.
LGTM=bradfitz
R=bradfitz
CC=golang-codereviews
https://golang.org/cl/63520043
TestLookupHost expects that no duplicate addresses are returned. when cs is consulted for a name, e.g net!localhost!1, it will possibly return multiple available paths, e.g. via il and tcp. this confuses the tests.
LGTM=aram
R=jas, 0intro, aram
CC=golang-codereviews
https://golang.org/cl/58120045
Allow clients to check for timeouts without relying on error substring
matching.
Fixes#6185.
LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/55470048
Whitespace characters are allowed in quoted-string according to RFC 5322 without
being "Q"-encoding. Address.String() already always formats the name portion in
quoted string, so whitespace characters should be allowed in there.
Fixes#6641.
LGTM=dave, dsymonds
R=golang-codereviews, gobot, dsymonds, dave
CC=golang-codereviews
https://golang.org/cl/55770043
The Transport's idle connection cache is keyed by a string,
for pre-Go 1.0 reasons. Ever since Go has been able to use
structs as map keys, there's been a TODO in the code to use
structs instead of allocating strings. This change does that.
Saves 3 allocatins and ~100 bytes of garbage per client
request. But because string hashing is so fast these days
(thanks, Keith), the performance is a wash: what we gain
on GC and not allocating, we lose in slower hashing. (hashing
structs of strings is slower than 1 string)
This seems a bit faster usually, but I've also seen it be a
bit slower. But at least it's how I've wanted it now, and it
the allocation improvements are consistent.
LGTM=adg
R=adg
CC=golang-codereviews
https://golang.org/cl/58260043
This is the chunked half of https://golang.org/cl/49570044 .
We want full reads to return EOF as early as possible, when we
know we're at the end, so http.Transport client connections are eagerly
re-used in the common case, even if no Read or Close follows.
To do this, make the chunkedReader.Read fill up its argument p []byte
buffer as much as possible, as long as that doesn't involve doing
any more blocking reads to read chunk headers. That means if we
have a chunk EOF ("0\r\n") sitting in the incoming bufio.Reader,
we see it and set EOF on our final Read.
LGTM=adg
R=adg
CC=golang-codereviews
https://golang.org/cl/58240043
Set EOF on the final Read of a body with a Content-Length, which
will cause clients to recycle their connection immediately upon
the final Read, rather than waiting for another Read or Close
(neither of which might come). This happens often when client
code is simply something like:
err := json.NewDecoder(resp.Body).Decode(&dest)
...
Then there's usually no subsequent Read. Even if the client
calls Close (which they should): in Go 1.1, the body was
slurped to EOF, but in Go 1.2, that was then treated as a
Close-before-EOF and the underlying connection was closed.
But that's assuming the user even calls Close. Many don't.
Reading to EOF also causes a connection be reused. Now the EOF
arrives earlier.
This CL only addresses the Content-Length case. A future CL
will address the chunked case.
LGTM=adg
R=adg
CC=golang-codereviews
https://golang.org/cl/49570044
Use the smaller read-only bytes.NewReader/strings.NewReader instead
of a bytes.Buffer when possible.
LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/54660045
This change include updates to the probeIPv4Stack
and probeIPv6Stack to ensure that one or both
protocols are supported by ip(3).
The addition of fdMutex to netFD fixes the
TestTCPConcurrentAccept failures.
Additional changes add support for keepalive.
R=golang-codereviews, 0intro
CC=golang-codereviews, rsc
https://golang.org/cl/49920048
Use testing.AllocsPerRun now that it exists, instead of doing it by hand.
Fixes#6076
R=golang-codereviews, alex.brainman
CC=golang-codereviews
https://golang.org/cl/53810043
Falsely claimed an old, no longer true condition that the first argument
must be a pointer.
Fixes#6697
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/53480043
Update #5001
This test is flakey on linux servers and fails otherwise good builds. Mikio has some proposals to fix the test, but they require additional plumbing.
In the meantime, disable this test in -short mode so it will run during the full net test suite, but not during builder ci.
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/53410043
Status codes 204, 304, and 1xx don't allow bodies. We already
had a function for this, but we were hard-coding just 304
(StatusNotModified) in a few places. Use the function
instead, and flesh out tests for all codes.
Fixes#6685
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/53290044
Apparently this is expensive on Windows.
Fixes#7020
R=golang-codereviews, alex.brainman, mattn.jp, dvyukov
CC=golang-codereviews
https://golang.org/cl/52840043
Previously, filenames containing special characters could:
1) Escape the <a> tag, with a file called something like: ">foo
2) Break the links in the index by prematurely ending the path portion
of the url, with a file called: foo?bar
In order to avoid a forbidden dependency on the html package, I'm
using htmlReplacer from net/http/server.go, which is equivalent to
html.EscapeString.
This change also expands fakeFile.Readdir to better emulate
os.File.Readdir.
R=golang-codereviews, rsc, gobot, bradfitz, josharian, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/37440043
Still work to do. See http://golang.org/issue/7125
««« original CL description
net/http/cookiejar: document format of domain in PublicSuffix
Document what values a PublicSuffixList must accept as
a domain in a call to PublicSuffix.
R=bradfitz, nigeltao
CC=golang-codereviews
https://golang.org/cl/47560044
»»»
R=golang-codereviews, minux.ma
CC=golang-codereviews
https://golang.org/cl/51770044
Fix another issue (similar to Issue 6995) where there was a
data race when sharing a server handler's Request.Body with
another goroutine that out-lived the Handler's goroutine.
In some cases we were not closing the incoming Request.Body
(which would've required reading it until the end) if we
thought it we thought we were going to be forcibly closing the
underlying net.Conn later anyway. But that optimization
largely moved to the transfer.go *body later, and locking was
added to *body which then detected read-after-close, so now
calling the (*body).Close always is both cheap and correct.
No new test because TestTransportAndServerSharedBodyRace caught it,
albeit only sometimes. Running:
while ./http.test -test.cpu=8 -test.run=TestTransportAndServerSharedBodyRace; do true; done
... would reliably cause a race before, but not now.
Update #6995Fixes#7092
R=golang-codereviews, khr
CC=golang-codereviews
https://golang.org/cl/51700043
There were no docs explaining the meaning of Readdir's count
argument, for instance. Clarify that these mean the same as
the methods on *os.File.
R=golang-codereviews, minux.ma
CC=golang-codereviews
https://golang.org/cl/51630043
Our default behavior for the common cases shouldn't lead to
leaked TCP connections (e.g. from people closing laptops) when
their Go servers are exposed to the open Internet without a
proxy in front.
Too many users on golang-nuts have learned this the hard way.
No API change. Only ListenAndServe and ListenAndServeTLS are
updated.
R=golang-codereviews, cespare, gobot, rsc, minux.ma
CC=golang-codereviews
https://golang.org/cl/48300043
A server Handler (e.g. a proxy) can receive a Request, and
then turn around and give a copy of that Request.Body out to
the Transport. So then two goroutines own that Request.Body
(the server and the http client), and both think they can
close it on failure. Therefore, all incoming server requests
bodies (always *http.body from transfer.go) need to be
thread-safe.
Fixes#6995
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/46570043
Adds tests for branches handling call ordering which
were shown to be untested by the cover tool.
This is part of the refactoring of form parsing discussed
in CL 44040043. These tests may need to be changed later but
should help lock in the current behaviour.
R=golang-codereviews, dave, bradfitz
CC=golang-codereviews
https://golang.org/cl/46750043
The Domain and Path field of a parsed cookie have been
the unprocessed wire data since Go 1.0; this seems to
be okay for most applications so let's keep it.
Returning the unprocessed wire data makes it easy to
handle nonstandard or even broken clients without
consulting Raw or Unparsed of a cookie.
The RFC 6265 parsing rules for domain and path are
currently buried in net/http/cookiejar but could be
exposed in net/http if necessary.
R=bradfitz, nigeltao
CC=golang-codereviews
https://golang.org/cl/48060043
Document what values a PublicSuffixList must accept as
a domain in a call to PublicSuffix.
R=bradfitz, nigeltao
CC=golang-codereviews
https://golang.org/cl/47560044
On Solaris, if you do a in-progress connect, and then the
server accepts and closes the socket, the client's later
attempt to complete the connect will fail with EINVAL. Handle
this case by assuming that the connect succeeded. This code
is weird enough that it is implemented as Solaris-only so that
it doesn't hide a real error on a different OS.
Update #6828
R=golang-codereviews, bradfitz, dave
CC=golang-codereviews
https://golang.org/cl/46160043
This avoids problems with systems that take a long time to
find out nothing is listening, while still testing for the
self-connect misfeature since a self-connect should be fast.
With this we may be able to remove the test for non-Linux
systems.
Tested (on GNU/Linux) by editing selfConnect in
tcpsock_posix.go to always return false and verifying that
TestSelfConnect then fails with and without this change.
Idea from Uros Bizjak.
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/39200044
Capture log output (and test it while at it),
and quiet unnecessary t.Logf.
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/45850043
When a connection is hijacked, release the reference to the bufio.Writer
that is used with the chunkWriter. The chunkWriter is not used after
the connection is hijacked.
Also add a test to check that double Hijack calls do something sensible.
benchmark old ns/op new ns/op delta
BenchmarkServerHijack 24137 20629 -14.53%
benchmark old allocs new allocs delta
BenchmarkServerHijack 21 19 -9.52%
benchmark old bytes new bytes delta
BenchmarkServerHijack 11774 9667 -17.90%
R=bradfitz, dave, chris.cahoon
CC=golang-codereviews
https://golang.org/cl/39440044
RFC 2616, section 7.2.1 - empty type SHOULD be treated as
application/octet-stream.
Fixes#6616.
R=golang-codereviews, gobot, bradfitz, josharian
CC=golang-codereviews
https://golang.org/cl/31810043
Per RFC 4291, 'The use of "::" indicates one or more groups of 16 bits of zeros.'
Fixes#6628
R=golang-dev, rsc, minux.ma, mikioh.mikioh
CC=golang-dev
https://golang.org/cl/15990043
Notably, to show allocs. Currently: 11766 B/op, 21 allocs/op,
at least one alloc of which is in the benchmark loop itself.
R=golang-dev, jnewlin
CC=golang-dev
https://golang.org/cl/40370057
Because TestDNSThreadLimit consumes tons of file descriptors and
makes other tests flaky when CGO_ENABLE=0 or being with netgo tag.
Fixes#6580.
R=golang-dev, bradfitz, adg, minux.ma
CC=golang-dev
https://golang.org/cl/14639044
Protocol keywords are case-insensitive,
but the Ndb database is case-sensitive.
Also use the generic net protocol instead
of tcp in lookupHost.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/40600047
If the return was reached, then hostsPath would not be properly restored
to its original value. See the (lengthy) discussion at
https://golang.org/cl/15960047/
I assume that this is not for Go 1.2; mailing now since I promised to do so.
I will plan to ping once Go 1.2 is out.
R=rsc, bradfitz
CC=golang-dev
https://golang.org/cl/16200043
Encoded query strings are always sorted by key; the example wasn't.
R=golang-dev, dsymonds, minux.ma, bradfitz
CC=golang-dev
https://golang.org/cl/16430043
Only add a slash to path if it's a separator between
a host and path.
Fixes#6609
R=golang-dev, dsymonds, r
CC=golang-dev
https://golang.org/cl/14815043
