This CL adds skipped failing tests, showing differences between HTTP/1
and HTTP/2 behavior. They'll be fixed in later commits.
Only a tiny fraction of the net/http tests have been split into their
"_h1" and "_h2" variants. That will also continue. (help welcome)
Updates #6891
Updates #13315
Updates #13316
Updates #13317
Change-Id: I16c3c381dbe267a3098fb266ab0d804c36473a64
Reviewed-on: https://go-review.googlesource.com/17046
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix regression from https://golang.org/cl/16829 ("require valid methods
in NewRequest and Transport.RoundTrip").
An empty string is a valid method (it means "GET", per the docs).
Fixes#13311
Change-Id: I26b71dc4ccc146498b5d7e38fbe31ed11dd5a6cf
Reviewed-on: https://go-review.googlesource.com/16952
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Inspect the crypto/tls error to recognize this case and give a more
helpful error.
Fixes#11111.
Change-Id: I63f6af8c375aa892326ccccbd29655d54d68df0b
Reviewed-on: https://go-review.googlesource.com/16079
Run-TryBot: Caleb Spare <cespare@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
mips64 strace doesn't support sendfile64 and will error out if we
specify that with `-e trace='. So we use sendfile for mips64 here.
Change-Id: If5e2bb39866ca3a77dcc40e4db338ba486921d89
Reviewed-on: https://go-review.googlesource.com/14455
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(This relands commit a4dcc692011bf1ceca9b1a363fd83f3e59e399ee.)
https://tools.ietf.org/html/rfc6066#section-3 states:
“Literal IPv4 and IPv6 addresses are not permitted in "HostName".”
However, if an IP literal was set as Config.ServerName (which could
happen as easily as calling Dial with an IP address) then the code would
send the IP literal as the SNI value.
This change filters out IP literals, as recognised by net.ParseIP, from
being sent as the SNI value.
Fixes#13111.
Change-Id: I6e544a78a01388f8fe98150589d073b917087f75
Reviewed-on: https://go-review.googlesource.com/16776
Run-TryBot: Adam Langley <agl@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Show more than one character when we recieve a unsolicited
response on an idle HTTP channel. Showing more than one
byte is really useful when you want to debug your program
when you get this message.
Change-Id: I3caf9f06420e7c2a2de3e4eb302c5dab95428fdb
Reviewed-on: https://go-review.googlesource.com/13959
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Updates to git rev 042ba42f (https://golang.org/cl/16734)
This moves all the code for glueing the HTTP1 and HTTP2 transports
together out of net/http and into x/net/http2 where others can use it,
and where it has tests.
Change-Id: I143ac8bb61eed36c87fd838b682ebb37b81b8c2c
Reviewed-on: https://go-review.googlesource.com/16735
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Mathieu Lonjaret <mathieu.lonjaret@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The GODEBUG option remains, for now, but only for turning it off.
We'll decide what to do with it before release.
This CL includes the dependent http2 change (https://golang.org/cl/16692)
in the http2 bundle (h2_bundle.go).
Updates golang/go#6891
Change-Id: If9723ef627c7ba4f7343dc8cb89ca88ef0fbcb10
Reviewed-on: https://go-review.googlesource.com/16693
Reviewed-by: Blake Mizerany <blake.mizerany@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The Server's server goroutine was panicing (but recovering) when
cleaning up after handling a request. It was pretty harmless (it just
closed that one connection and didn't kill the whole process) but it
was distracting.
Updates #13135
Change-Id: I2a0ce9e8b52c8d364e3f4ce245e05c6f8d62df14
Reviewed-on: https://go-review.googlesource.com/16572
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Errors with http.Redirect and http.StatusOk seem
to occur from time to time on the irc channel.
This change adds documentation suggesting
to use one of the 3xx codes and not StatusOk
with Redirect.
Change-Id: I6b900a8eb868265fbbb846ee6a53e426d90a727d
Reviewed-on: https://go-review.googlesource.com/15980
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This part got dropped when we were debating between two solutions
in https://golang.org/cl/15151Fixes#13032
Change-Id: I820b94f6c0c102ccf9342abf957328ea01f49a26
Reviewed-on: https://go-review.googlesource.com/16313
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In earlier versions of Go, times were only encoded as an ASN.1 UTCTIME and
crypto/tls/generate_cert.go limited times to the maximum UTCTIME value.
Revision 050b60a3 added support for ASN.1 GENERALIZEDTIME, allowing larger
time values to be represented (per RFC 5280).
As a result, when the httptest certificate was regenerated in revision
9b2d84ef, the Not After date changed to Jan 29 16:00:00 2084 GMT. Update
the comment to reflect this.
Change-Id: I1bd66e011f2749f9372b5c7506f52ea34e264ce9
Reviewed-on: https://go-review.googlesource.com/16193
Reviewed-by: Adam Langley <agl@golang.org>
In https://golang.org/cl/15860 http2.ConfigureServer was changed to
return an error if explicit CipherSuites are listed and they're not
compliant with the HTTP/2 spec.
This is the net/http side of the change, to look at the return value
from ConfigureServer and propagate it in Server.Serve.
h2_bundle.go will be updated in a future CL. There are too many other
http2 changes pending to be worth updating it now. Instead,
h2_bundle.go is minimally updated by hand in this CL so at least the
net/http change will compile.
Updates #12895
Change-Id: I4df7a097faff2d235742c2d310c333bd3fd5c08e
Reviewed-on: https://go-review.googlesource.com/16065
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
* detect Content-Type on ReponseRecorder.Write[String] call
if header wasn't written yet, Content-Type header is not set and
Transfer-Encoding is not set.
* fix typos in serve_test.go
Updates #12986
Change-Id: Id2ed8b1994e64657370fed71eb3882d611f76b31
Reviewed-on: https://go-review.googlesource.com/16096
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is the start of wiring up the HTTP/2 Transport. It is still
disabled in this commit.
This change does two main things:
1) Transport.RegisterProtocol now permits registering "http" or
"https" (they previously paniced), and the semantics of the
registered RoundTripper have been extended to say that the new
sentinel error value (ErrSkipAltProtocol, added in this CL) means
that the Transport's RoundTrip method proceeds as if the alternate
protocol had not been registered. This gives us a place to register
an alternate "https" RoundTripper which gets first dibs on using
HTTP/2 if there's already a cached connection.
2) adds Transport.TLSNextProto, a map keyed by TLS NPN/ALPN protocol
strings, similar in feel to the existing Server.TLSNextProto map.
This map is the glue between the HTTP/1 and HTTP/2 clients, since
we don't know which protocol we're going to speak (and thus which
Transport type to use) until we've already made the TCP connection.
Updates #6891
Change-Id: I7328c7ff24f52d9fe4899facabf7ecc5dcb989f3
Reviewed-on: https://go-review.googlesource.com/16090
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
With this CL, httptest.Server now uses connection-level accounting of
outstanding requests instead of ServeHTTP-level accounting. This is
more robust and results in a non-racy shutdown.
This is much easier now that net/http.Server has the ConnState hook.
Fixes#12789Fixes#12781
Change-Id: I098cf334a6494316acb66cd07df90766df41764b
Reviewed-on: https://go-review.googlesource.com/15151
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It was generating the wrong error message, always defaulting to "500
Internal Server Error", since the err variable used was always nil.
Fixes#12991
Change-Id: I94b0e516409c131ff3b878bcb91e65f0259ff077
Reviewed-on: https://go-review.googlesource.com/16060
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Current http client doesn't support Expect: 100-continue request
header(RFC2616-8/RFC7231-5.1.1). So even if the client have the header,
the head of the request body is consumed prematurely.
Those are my intentions to avoid premature consuming body in this change.
- If http.Request header contains body and Expect: 100-continue
header, it blocks sending body until it gets the first response.
- If the first status code to the request were 100, the request
starts sending body. Otherwise, sending body will be cancelled.
- Tranport.ExpectContinueTimeout specifies the amount of the time to
wait for the first response.
Fixes#3665
Change-Id: I4c04f7d88573b08cabd146c4e822061764a7cd1f
Reviewed-on: https://go-review.googlesource.com/10091
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Fixes#7782Fixes#9554
Updates #7237 (original metabug, before we switched to specific bugs)
Updates #11932 (plan9 still doesn't have net I/O deadline support)
Change-Id: I96f311b88b1501d884ebc008fd31ad2cf1e16d75
Reviewed-on: https://go-review.googlesource.com/15941
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The clues to this were already there, but as a user I was still unsure.
Make this more explicit.
Change-Id: I68564f3498dcd4897772a303588f03a6b65f111d
Reviewed-on: https://go-review.googlesource.com/15172
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This enables HTTP/2 by default (for https only) if the user didn't
configure anything in their NPN/ALPN map. If they're using SPDY or an
alternate http2 or a newer http2 from x/net/http2, we do nothing
and don't use the standard library's vendored copy of x/net/http2.
Upstream remains golang.org/x/net/http2.
Update #6891
Change-Id: I69a8957a021a00ac353f9d7fdb9a40a5b69f2199
Reviewed-on: https://go-review.googlesource.com/15828
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The PROXY protocol is supported by several proxy servers such as haproxy
and Amazon ELB. This protocol allows services running behind a proxy to
learn the remote address of the actual client connecting to the proxy,
by including a single textual line at the beginning of the TCP
connection.
http://www.haproxy.org/download/1.5/doc/proxy-protocol.txt
There are several Go libraries for this protocol (such as
https://github.com/armon/go-proxyproto), which operate by wrapping a
net.Conn with an implementation whose RemoteAddr method reads the
protocol line before returning. This means that RemoteAddr is a blocking
call.
Before this change, http.Serve called RemoteAddr from the main Accepting
goroutine, not from the per-connection goroutine. This meant that it
would not Accept another connection until RemoteAddr returned, which is
not appropriate if RemoteAddr needs to do a blocking read from the
socket first.
Fixes#12943.
Change-Id: I1a242169e6e4aafd118b794e7c8ac45d0d573421
Reviewed-on: https://go-review.googlesource.com/15835
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Allow all CGI environment settings from the inherited set and default
inherited set to be overridden including PATH by Env.
Change-Id: Ief8d33247b879fa87a8bfd6416d4813116db98de
Reviewed-on: https://go-review.googlesource.com/14959
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
These were proposed in the RFC over three years ago, then proposed to
be added to Go in https://codereview.appspot.com/7678043/ 2 years and
7 months ago, and the spec hasn't been updated or retracted the whole
time.
Time to export them.
Of note, HTTP/2 uses code 431 (Request Header Fields Too Large).
Updates #12843
Change-Id: I78c2fed5fab9540a98e845ace73f21c430a48809
Reviewed-on: https://go-review.googlesource.com/15732
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes#12866
net/http.Client returns some errors wrapped in a *url.Error. To avoid
the requirement to unwrap these errors to determine if the cause was
temporary or a timeout, make *url.Error implement net.Error directly.
Change-Id: I1ba84ecc7ad5147a40f056ff1254e60290152408
Reviewed-on: https://go-review.googlesource.com/15672
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
The existing serve() method returns a zero-length response body when
it encounters an error, which results in a blank page and no visible
error in browsers.
This change sends a response body explaining the error for display in browsers.
Fixes#12745
Change-Id: I9dc3b95ad88cb92c18ced51f6b52bd3b2c1b974c
Reviewed-on: https://go-review.googlesource.com/15018
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
As stated in FastCGI specifications:
FastCGI transmits a name-value pair as the length of the name,
followed by the length of the value, followed by the name,
followed by the value.
The current implementation trusts the name and value length
provided in the record, leading to a panic if the record
is malformed.
Added an explicit check on the lengths.
Test case and fix suggested by diogin@gmail.com (Jingcheng Zhang)
Fixes#11824
Change-Id: I883a1982ea46465e1fb02e0e02b6a4df9e529ae4
Reviewed-on: https://go-review.googlesource.com/15015
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Optimize two calls of io.Copy which cannot make use of neither
io.ReaderFrom nor io.WriterTo optimization tricks by replacing them with
io.CopyBuffer with reusable buffers.
First is fallback call to io.Copy when server misses the optimized case
of using sendfile to copy from a regular file to net.TCPConn; second is
use of io.Copy on piped reader/writer when handler implementation uses
http.CloseNotifier interface. One of the notable users of
http.CloseNotifier is httputil.ReverseProxy.
benchmark old ns/op new ns/op delta
BenchmarkCloseNotifier-4 309591 303388 -2.00%
benchmark old allocs new allocs delta
BenchmarkCloseNotifier-4 50 49 -2.00%
benchmark old bytes new bytes delta
BenchmarkCloseNotifier-4 36168 3140 -91.32%
Fixes#12455
Change-Id: I512e6aa2f1aeed2ed00246afb3350c819b65b87e
Reviewed-on: https://go-review.googlesource.com/14177
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Accepting a request with a nil body was never explicitly supported but
happened to work in the past.
This doesn't happen in most cases because usually people pass
a Server's incoming Request to the ReverseProxy's ServeHTTP method,
and incoming server requests are guaranteed to have non-nil bodies.
Still, it's a regression, so fix.
Fixes#12344
Change-Id: Id9a5a47aea3f2875d195b66c9a5f8581c4ca2aed
Reviewed-on: https://go-review.googlesource.com/13935
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
byte is unsigned so the comparison against zero is always true.
Change-Id: I8fa60245972be362ae920507a291f92c0f9831ad
Reviewed-on: https://go-review.googlesource.com/13941
Run-TryBot: Todd Neal <todd@tneal.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Could go in 1.5, although not critical.
See also #12107
Change-Id: I7f1608b58581d21df4db58f0db654fef79e33a90
Reviewed-on: https://go-review.googlesource.com/13481
Reviewed-by: Dave Cheney <dave@cheney.net>
Found in a Google program running under the race detector.
No test, but verified that this fixes the race with go run -race of:
package main
import (
"crypto/tls"
"fmt"
"net"
"net/http"
"net/http/httptest"
)
func main() {
for {
ts := httptest.NewTLSServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {}))
conf := &tls.Config{} // non-nil
a, b := net.Pipe()
go func() {
sconn := tls.Server(a, conf)
sconn.Handshake()
}()
tr := &http.Transport{
TLSClientConfig: conf,
}
req, _ := http.NewRequest("GET", ts.URL, nil)
_, err := tr.RoundTrip(req)
println(fmt.Sprint(err))
a.Close()
b.Close()
ts.Close()
}
}
Also modified cmd/vet to report the copy-of-mutex bug statically
in CL 13646, and fixed two other instances in the code found by vet.
But vet could not have told us about cloneTLSConfig vs cloneTLSClientConfig.
Confirmed that original report is also fixed by this.
Fixes#12099.
Change-Id: Iba0171549e01852a5ec3438c25a1951c98524dec
Reviewed-on: https://go-review.googlesource.com/13453
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
Prior to this change, broken trailers would be handled by body.Read, and
an error would be returned to its caller (likely a Handler), but that
error would go completely unnoticed by the rest of the server flow
allowing a broken connection to be reused. This is a possible request
smuggling vector.
Fixes#12027.
Change-Id: I077eb0b8dff35c5d5534ee5f6386127c9954bd58
Reviewed-on: https://go-review.googlesource.com/13148
Reviewed-by: Russ Cox <rsc@golang.org>
It was failing with multiple goroutines a few out of every thousand
runs (with errRequestCanceled) because it was using the same
*http.Request for all 5 RoundTrips, but the RoundTrips' goroutines
(notably the readLoop method) were all still running, sharing that
same pointer. Because the response has no body (which is what
TestZeroLengthPostAndResponse tests), the readLoop was marking the
connection as reusable early (before the caller read until the body's
EOF), but the Transport code was clearing the Request's cancelation
func *AFTER* the caller had already received it from RoundTrip. This
let the test continue looping and do the next request with the same
pointer, fetch a connection, and then between getConn and roundTrip
have an invariant violated: the Request's cancelation func was nil,
tripping this check:
if !pc.t.replaceReqCanceler(req.Request, pc.cancelRequest) {
pc.t.putIdleConn(pc)
return nil, errRequestCanceled
}
The solution is to clear the request cancelation func in the readLoop
goroutine in the no-body case before it's returned to the caller.
This now passes reliably:
$ go test -race -run=TestZeroLengthPostAndResponse -count=3000
I think we've only seen this recently because we now randomize scheduling
of goroutines in race mode (https://golang.org/cl/11795). This race
has existed for a long time but the window was hard to hit.
Change-Id: Idb91c582919f85aef5b9e5ef23706f1ba9126e9a
Reviewed-on: https://go-review.googlesource.com/13070
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Introduced in https://go-review.googlesource.com/12865 (git rev c2db5f4c).
This fix doesn't add any new lock acquistions: it just moves the
existing one taken by the unreadDataSize method and moves it out
wider.
It became flaky at rev c2db5f4c, but now reliably passes again:
$ go test -v -race -run=TestTransportAndServerSharedBodyRace -count=100 net/http
Fixes#11985
Change-Id: I6956d62839fd7c37e2f7441b1d425793f4a0db30
Reviewed-on: https://go-review.googlesource.com/12909
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
HTTP servers attempt to entirely consume a request body before sending a
response. However, when doing so, it previously would ignore any errors
encountered.
Unfortunately, the errors triggered at this stage are indicative of at
least a couple problems: read timeouts and chunked encoding errors.
This means properly crafted and/or timed requests could lead to a
"smuggled" request.
The fix is to inspect the errors created by the response body Reader,
and treat anything other than io.EOF or ErrBodyReadAfterClose as
fatal to the connection.
Fixes#11930
Change-Id: I0bf18006d7d8f6537529823fc450f2e2bdb7c18e
Reviewed-on: https://go-review.googlesource.com/12865
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This makes the receiver name consistent with the rest of the methods on
type Server.
Change-Id: Ic2a007d3b5eb50bd87030e15405e9856109cf590
Reviewed-on: https://go-review.googlesource.com/13035
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
From https://github.com/golang/go/issues/11745#issuecomment-123555313
this implements option (b), having the server pause slightly after
sending the final response on a TCP connection when we're about to close
it when we know there's a request body outstanding. This biases the
client (which might not be Go) to prefer our response header over the
request body write error.
Updates #11745
Change-Id: I07cb0b74519d266c8049d9e0eb23a61304eedbf8
Reviewed-on: https://go-review.googlesource.com/12658
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
From https://github.com/golang/go/issues/11745#issuecomment-123555313 :
The http.RoundTripper interface says you get either a *Response, or an
error.
But in the case of a client writing a large request and the server
replying prematurely (e.g. 403 Forbidden) and closing the connection
without reading the request body, what does the client want? The 403
response, or the error that the body couldn't be copied?
This CL implements the aforementioned comment's option c), making the
Transport give an N millisecond advantage to responses over body write
errors.
Updates #11745
Change-Id: I4485a782505d54de6189f6856a7a1f33ce4d5e5e
Reviewed-on: https://go-review.googlesource.com/12590
Reviewed-by: Russ Cox <rsc@golang.org>
The "add a Request.Cancel channel" change (https://golang.org/cl/11601)
added support for "race free" cancellation, but introduced a data race. :)
Noticed while running "go test -race net/http". The test is skipped in
short mode, so we never saw it on the dashboard.
Change-Id: Ica14579d8723f8f9d1691e8d56c30b585b332c64
Reviewed-on: https://go-review.googlesource.com/12663
Reviewed-by: Aaron Jacobs <jacobsa@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Move tracing functions from runtime/pprof to the new runtime/trace package.
Fixes#9710
Change-Id: I718bcb2ae3e5959d9f72cab5e6708289e5c8ebd5
Reviewed-on: https://go-review.googlesource.com/12511
Reviewed-by: Russ Cox <rsc@golang.org>
If we receive an HTTP request with "Expect: 100-continue" and the
Handler never read to EOF, the conn is in an unknown state.
Don't reuse that connection.
Fixes#11549
Change-Id: I5be93e7a54e899d615b05f72bdcf12b25304bc60
Reviewed-on: https://go-review.googlesource.com/12262
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
A malformed Host header can result in a malformed HTTP request.
Clean them to avoid this.
Updates #11206. We may come back and make this stricter for 1.6.
Change-Id: I23c7d821cd9dbf66c3c15d26750f305e3672d984
Reviewed-on: https://go-review.googlesource.com/11241
Reviewed-by: Russ Cox <rsc@golang.org>
The previous commit (git 2ae77376) just did golang.org. This one
includes golang.org subdomains like blog, play, and build.
Change-Id: I4469f7b307ae2a12ea89323422044e604c5133ae
Reviewed-on: https://go-review.googlesource.com/12071
Reviewed-by: Rob Pike <r@golang.org>
The one in misc/makerelease/makerelease.go is particularly bad and
probably warrants rotating our keys.
I didn't update old weekly notes, and reverted some changes involving
test code for now, since we're late in the Go 1.5 freeze. Otherwise,
the rest are all auto-generated changes, and all manually reviewed.
Change-Id: Ia2753576ab5d64826a167d259f48a2f50508792d
Reviewed-on: https://go-review.googlesource.com/12048
Reviewed-by: Rob Pike <r@golang.org>
The recent https://golang.org/cl/11810 is reportedly a bit too
aggressive.
Apparently some HTTP requests in the wild do contain both a
Transfer-Encoding along with a bogus Content-Length. Instead of
returning a 400 Bad Request error, we should just ignore the
Content-Length like we did before.
Change-Id: I0001be90d09f8293a34f04691f608342875ff5c4
Reviewed-on: https://go-review.googlesource.com/11962
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Go's continuous build system depends on HTTP trailers for the buildlet
interface.
Andrew rewrote the makerelease tool to work in terms of Go's builder
system (now at x/build/cmd/release), but it previously could only
create GCE-based buildlets, which meant x/build/cmd/release couldn't
build the release for Darwin.
https://golang.org/cl/11901 added support for proxying buildlet
connections via the coordinator, but that exposed the fact that
httputil.ReverseProxy couldn't proxy Trailers. A fork of that code
also wasn't possible because net/http needlessly deleted the "Trailer"
response header in the Transport code. This mistake goes back to
"release-branch.r56" and earlier but was never noticed because nobody
ever uses Trailers, and servers via ResponseWriter never had the
ability to even set trailers before this Go 1.5. Note that setting
trailers requires pre-declaring (in the response header) which
trailers you'll set later (after the response body). Because you could
never set them, before this release you could also never proxy them.
Change-Id: I2410a099921790dcd391675ae8610300efa19108
Reviewed-on: https://go-review.googlesource.com/11940
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
This was originally done in https://codereview.appspot.com/5690059
(Feb 2012) to deal with bad response headers coming back from webcams,
but it presents a potential security problem with HTTP request
smuggling for request headers containing "Content Length" instead of
"Content-Length".
Part of overall HTTP hardening for request smuggling. See RFC 7230.
Thanks to Régis Leroy for the report.
Change-Id: I92b17fb637c9171c5774ea1437979ae2c17ca88a
Reviewed-on: https://go-review.googlesource.com/11772
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
RawPath is a hint to the desired encoding of Path.
It is ignored when it is not a valid encoding of Path,
such as when Path has been changed but RawPath has not.
It is not ignored but also not useful when it matches
the url package's natural choice of encoding.
In this latter case, set it to the empty string.
This should help drive home the point that clients
cannot in general depend on it being present and
that they should use the EncodedPath method instead.
This also reduces the impact of the change on tests,
especially tests that use reflect.DeepEqual on parsed URLs.
Change-Id: I437c51a33b85439a31c307caf1436118508ea196
Reviewed-on: https://go-review.googlesource.com/11760
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This allows for "race free" cancellation, in the sense discussed in
issue #11013: in contrast to Transport.CancelRequest, the cancellation
will not be lost if the user cancels before the request is put into the
transport's internal map.
Fixes#11013.
Change-Id: I0b5e7181231bdd65d900e343f764b4d1d7c422cd
Reviewed-on: https://go-review.googlesource.com/11601
Run-TryBot: David Symonds <dsymonds@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TestTransportAndServerSharedBodyRace got flaky after
issue #9662 was fixed by https://golang.org/cl/11412, which made
servers hang up on clients when a Handler stopped reading its body
early.
This test was affected by a race between the the two goroutines in the
test both only reading part of the request, which was an unnecessary
detail for what the test was trying to test (concurrent Read/Close
races on an *http.body)
Also remove an unused remnant from an old test from which this one was
derived. And make the test not deadlock when it fails. (which was why
the test was showing up as 2m timeouts on the dashboard)
Fixes#11418
Change-Id: Ic83d18aef7e09a9cd56ac15e22ebed75713026cb
Reviewed-on: https://go-review.googlesource.com/11610
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
The race occurs rarely, but by putting some delays and more reads/writes
of prePendingDial/postPendingDial in the handlePendingDial function I
could reproduce it.
Fixes#11136
Change-Id: I8da9e66c88fbda049eaaaaffa2717264ef327768
Reviewed-on: https://go-review.googlesource.com/11250
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Includes a new net/http test too.
Fixes#11202
Change-Id: I61edc594f4de8eb6780b8dfa221269dd482e8f35
Reviewed-on: https://go-review.googlesource.com/11492
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
All other exported errors in net/http are commented. This change adds
documentation to ErrNoCookie and ErrNoLocation to explain where they are
returned, and why.
Change-Id: I21fa0d070dd35256681ad0714000f238477d4af1
Reviewed-on: https://go-review.googlesource.com/11044
Reviewed-by: Russ Cox <rsc@golang.org>
The consecutive calls to Head would sometimes get different
connections depending on if the readLoop had finished executing
and placed its connection on the idle list or not. This change
ensures that readLoop completes before we make our second connection.
Fixes#11250
Change-Id: Ibdbc4d3d0aba0162452f6dec5928355a37dda70a
Reviewed-on: https://go-review.googlesource.com/11170
Reviewed-by: Russ Cox <rsc@golang.org>
Some old buggy browsers sent extra CRLF(s) after POST bodies. Skip
over them before reading subsequent requests.
Fixes#10876
Change-Id: I62eacf2b3e985caffa85aee3de39d8cd3548130b
Reviewed-on: https://go-review.googlesource.com/11491
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
If a client sent a POST with a huge request body, calling
req.Body.Close in the handler (which is implicit at the end of a
request) would end up consuming it all.
Put a cap on that, using the same threshold used elsewhere for similar
cases.
Fixes#9662
Change-Id: I26628413aa5f623a96ef7c2609a8d03c746669e5
Reviewed-on: https://go-review.googlesource.com/11412
Reviewed-by: Andrew Gerrand <adg@golang.org>
Instead of ambiguously referring to "the Client's CheckRedirect
function" in Head, describe the default behavior like for Get as users
aren't expected to change DefaultClient.CheckRedirect.
While here, use consistent punctuation for the Get and Head Client
method documentation.
Change-Id: I9e7046c73b0d0bc4de002234924d9e7c59aceb41
Reviewed-on: https://go-review.googlesource.com/11362
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The ListenAndServerTLS function still requires the certFile and
keyFile, but the Server.ListenAndServerTLS method doesn't need to
require the certFile and keyFile if the Server.TLSConfig.Certificates
are already populated.
Fixes#8599
Change-Id: Id2e3433732f93e2619bfd78891f775d89f1d651e
Reviewed-on: https://go-review.googlesource.com/11413
Reviewed-by: Andrew Gerrand <adg@golang.org>
This appears to be some legacy which is no longer used.
Change-Id: I469beb59a90853e8de910158f179b32f1aa14c7d
Reviewed-on: https://go-review.googlesource.com/11304
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>
These were found by grepping the comments from the go code and feeding
the output to aspell.
Change-Id: Id734d6c8d1938ec3c36bd94a4dbbad577e3ad395
Reviewed-on: https://go-review.googlesource.com/10941
Reviewed-by: Aamir Khan <syst3m.w0rm@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The Error function is a potential XSS vector if a user can control the
error message.
For example, an http.FileServer when given a request for this path
/<script>alert("xss!")</script>
may return a response with a body like this
open <script>alert("xss!")</script>: no such file or directory
Browsers that sniff the content may interpret this as HTML and execute
the script. The nosniff header added by this CL should help, but we
should also try santizing the output entirely.
Change-Id: I447f701531329a2fc8ffee2df2f8fa69d546f893
Reviewed-on: https://go-review.googlesource.com/10640
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
