This CL makes code like this work:
res, err := http.Get("https://фу.бар/баз")
So far, IDNA support is limited to the http1 and http2 Transports.
The http package is currently responsible for converting domain names
into Punycode before calling the net layer. The http package also has
to Punycode-ify the hostname for the Host & :authority headers for
HTTP/1 and HTTP/2, respectively.
No automatic translation from Punycode back to Unicode is performed,
per Go's historical behavior. Docs are updated where relevant. No
changes needed to the Server package. Things are already in ASCII
at that point.
No changes to the net package, at least yet.
Updates x/net/http2 to git rev 57c7820 for https://golang.org/cl/29071
Updates #13835
Change-Id: I1e9a74c60d00a197ea951a9505da5c3c3187099b
Reviewed-on: https://go-review.googlesource.com/29072
Reviewed-by: Chris Broadfoot <cbro@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The CL 28951 added TestLookupPort_Minimal, which was failing
on Plan 9, because lookupPort was not case-insensitive.
Change-Id: Ic80dd29dad4ffd1c84c2590e3d5d0e588ab2e6c2
Reviewed-on: https://go-review.googlesource.com/29051
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: David du Colombier <0intro@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Remove the naked returns and goto statements from parse.
Make tests more consistent in the got/want ordering, and clean up some
unnecessary helper functions.
Change-Id: Iaa244cb8c00dd6b42836d95448bf02caa72bfabd
Reviewed-on: https://go-review.googlesource.com/28890
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Also, flesh out the baked-in /etc/services table for LookupPort a bit.
This services map moves from a unix-specific file to a portable file
where nacl can use it.
Also, remove the duplicated entries in the protocol map in different
cases, and just canonicalize the input before looking in the map. Now
it handles any case, including MiXeD cAse.
In the process, add a test that service names for LookupPort are case
insensitive. They were on Windows, but not cgo. Now there's a test and
they're case insensitive in all 3+ paths. Maybe it breaks plan9. We'll
see.
Fixes#17045
Change-Id: Idce7d68703f371727c7505cda03a32bd842298cd
Reviewed-on: https://go-review.googlesource.com/28951
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
CL 28484 mistakenly assumed that WSARecv returns WSAEINPROGRESS
when there is nothing to read. But the error is ERROR_IO_PENDING.
Fix that mistake.
I was about to write a test for it. But I have found
TestTCPReadWriteAllocs in net package that does nearly what I need,
but was conveniently disabled. So enable and extend the test.
Fixes#16988
Change-Id: I55e5cf8998a9cf29e92b398d702280bdf7d6fc85
Reviewed-on: https://go-review.googlesource.com/28990
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Copy all of the original request's headers on redirect, unless they're
sensitive. Only send sensitive ones to the same origin, or subdomains
thereof.
Fixes#4800
Change-Id: Ie9fa75265c9d5e4c1012c028d31fd1fd74465712
Reviewed-on: https://go-review.googlesource.com/28930
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Reviewed-by: Francesc Campoy Flores <campoy@golang.org>
Reviewed-by: Ross Light <light@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently, path resolution is done using the un-escaped version of
paths. This means that path elements like one%2ftwo%2fthree are
handled incorrectly, and optional encodings (%2d vs. -) are dropped.
This function makes escaped handling consistent with Parse: provided
escapings are honoured, and RawPath is only set if necessary.
A helper method setPath is introduced to handle the correct setting of
Path and RawPath given the encoded path.
Fixes#16947
Change-Id: I40b1215e9066e88ec868b41635066eee220fde37
Reviewed-on: https://go-review.googlesource.com/28343
Run-TryBot: Dave Day <djd@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Hop-by-hop headers (explicitly mentioned in RFC 2616) were already
removed from the response. This removes the custom hop-by-hop
headers listed in the "Connection" header of the response.
Updates #16875
Change-Id: I6b8f261d38b8d72040722f3ded29755ef0303427
Reviewed-on: https://go-review.googlesource.com/28810
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
We were already making a copy of the map before removing
hop-by-hop headers. This commit does the same for proxied
headers mentioned in the "Connection" header.
A test is added to ensure request headers are not modified.
Updates #16875
Change-Id: I85329d212787958d5ad818915eb0538580a4653a
Reviewed-on: https://go-review.googlesource.com/28493
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
https://golang.org/cl/27206 fixed the dtoi function such that
it now properly parses negative number. Ironically, this causes
several other functions that depended on dtoi to now (incorrectly)
parse negative numbers.
For example, ParseCIDR("-1.0.0.0/32") used to be rejected prior to the
above CL, but is now accepted even though it is an invalid CIDR notation.
This CL fixes that regression.
We fix this by removing the signed parsing logic entirely from dtoi.
It was introduced relatively recently in https://golang.org/cl/12447
to fix a bug where an invalid port was improperly being parsed as OK.
It seems to me that the fix in that CL to the port handling logic was
sufficient such that a change to dtoi was unnecessary.
Updates #16350
Change-Id: I414bb1aa27d0a226ebd4b05a09cb40d784691b43
Reviewed-on: https://go-review.googlesource.com/28414
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
The http.Transport's retry can't retry requests with non-nil
bodies. When cloning an incoming server request into an outgoing
client request, nil out the Body field if the ContentLength is 0. (For
server requests, Body is always non-nil, even for GET, HEAD, etc.)
Also, don't use the deprecated CancelRequest and use Context instead.
And don't set Proto, ProtoMajor, ProtoMinor. Those are ignored in
client requests, which was probably a later documentation
clarification.
Fixes#16036
Updates #16696 (remove useless Proto lines)
Change-Id: I70a869e9bd4bf240c5838e82fb5aa695a539b343
Reviewed-on: https://go-review.googlesource.com/28412
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
RFC 2616, section 14.10 says:
>>>
HTTP/1.1 proxies MUST parse the Connection header field before a message
is forwarded and, for each connection-token in this field, remove any
header field(s) from the message with the same name as the
connection-token. Connection options are signaled by the presence of a
connection-token in the Connection header field, not by any
corresponding additional header field(s), since the additional header
field may not be sent if there are no parameters associated with that
connection option.
<<<
The same requirement was included in RFC 7230, section 6.1.
Fixes#16875
Change-Id: I57ad4a4a17775537c8810d0edd7de1604317b5fa
Reviewed-on: https://go-review.googlesource.com/27970
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Per RFC 7230, Section 3.3.2: "A server MUST NOT send a Content-Length
header field in any response with a status code of 1xx (Informational)
or 204 (No Content).".
Fixes#16942
Change-Id: I8006c76c126304e13618966e6eafb08a3885d3cd
Reviewed-on: https://go-review.googlesource.com/28351
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In Go 1.0, the Config struct consisted only of exported fields.
In Go 1.1, it started to grow private, uncopyable fields (sync.Once,
sync.Mutex, etc).
Ever since, people have been writing their own private Config.Clone
methods, or risking it and doing a language-level shallow copy and
copying the unexported sync variables.
Clean this up and export the Config.clone method as Config.Clone.
This matches the convention of Template.Clone from text/template and
html/template at least.
Fixes#15771
Updates #16228 (needs update in x/net/http2 before fixed)
Updates #16492 (not sure whether @agl wants to do more)
Change-Id: I48c2825d4fef55a75d2f99640a7079c56fce39ca
Reviewed-on: https://go-review.googlesource.com/28075
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
As @pmarks-net said in the bug, this is something of a prisoner's
dilemma, but it does help people who occasionally report problems.
This is temporary. IPv6 is happening regardless of our decision here,
so we'll do this for now.
Fixes#15324
Change-Id: I8cc29c6efa56222970996c71182fc9ee89d78539
Reviewed-on: https://go-review.googlesource.com/28077
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
This change makes sure that tests are run with the correct
version of the go tool. The correct version is the one that
we invoked with "go test", not the one that is first in our path.
Fixes#16577
Change-Id: If22c8f8c3ec9e7c35d094362873819f2fbb8559b
Reviewed-on: https://go-review.googlesource.com/28089
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ordering fix: this CL swaps the order of the log write and the channel close
in WroteRequest. I could reproduce the bug by putting a sleep between the two
when the channel close was first. It needs to happen after the log.
Data race: use the log buffer's mutex when reading too. Not really
important once the ordering fix above is fixed (since nobody is
concurrently writing anymore), but for consistency.
Fixes#16414
Change-Id: If6657884e67be90b4455c8f5a6f7bc6981999ee4
Reviewed-on: https://go-review.googlesource.com/28078
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The handling of "options timeout:n" is supposed to be per individual
DNS server exchange, not per Lookup call.
Fixes#16865.
Change-Id: I2304579b9169c1515292f142cb372af9d37ff7c1
Reviewed-on: https://go-review.googlesource.com/28057
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Add support for writing TLS client random and master secret
in NSS key log format.
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format
Normally this is enabled by a developer debugging TLS based
applications, especially HTTP/2, by setting the KeyLogWriter
to an open file. The keys negotiated in handshake are then
logged and can be used to decrypt TLS sessions e.g. in Wireshark.
Applications may choose to add support similar to NSS where this
is enabled by environment variable, but no such mechanism is
built in to Go. Instead each application must explicitly enable.
Fixes#13057.
Change-Id: If6edd2d58999903e8390b1674ba4257ecc747ae1
Reviewed-on: https://go-review.googlesource.com/27434
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
RFC 7233, section 4.4 says:
>>>
For byte ranges, failing to overlap the current extent means that the
first-byte-pos of all of the byte-range-spec values were greater than the
current length of the selected representation. When this status code is
generated in response to a byte-range request, the sender SHOULD generate a
Content-Range header field specifying the current length of the selected
representation
<<<
Thus, we should send the Content-Range only if none of the ranges
overlap.
Fixes#15798.
Change-Id: Ic9a3e1b3a8730398b4bdff877a8f2fd2e30149e3
Reviewed-on: https://go-review.googlesource.com/24212
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The net package sets a finalizer on *netFD. I looked through all the
uses of *netFD in the package, looking for each case where a *netFD
was passed as an argument and the final reference to the argument was
not a function or method call. I added a call to runtime.KeepAlive after
each such final reference (there were only three).
The code is safe today without the KeepAlive calls because the compiler
keeps arguments alive for the duration of the function. However, that is
not a language requirement, so adding the KeepAlive calls ensures that
this code remains safe even if the compiler changes in the future.
Change-Id: I4e2bd7c5a946035dc509ccefb4828f72335a9ee3
Reviewed-on: https://go-review.googlesource.com/27650
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
On some error when using io.Copy with TCPConn, it displays an error
correlation like the following:
read tcp 192.0.2.1:1111->192.0.2.2:2222: read tcp [2001:db8::2]:2222->[2001:db8::3]:3333 read: connection reset by peer
the correlation "some error on reading after reading operation" looks a
bit confusing because the operation on the ReadFrom method of TCPConn is
actually "writing after reading." To clarify and avoid confusion, this
change sets "readfrom" to the Op field of outer-most OpError instead of
"read."
Change-Id: I6bf4e2e7247143fa54bbcf9cef7a8ae1ede1b35c
Reviewed-on: https://go-review.googlesource.com/25220
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This permits the error message to distinguish between a context that was
canceled and a context that timed out.
Updates #16381.
Change-Id: I3994b98e32952abcd7ddb5fee08fa1535999be6d
Reviewed-on: https://go-review.googlesource.com/24978
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
example for httptest.Recorder was inspecting Recoder directly.
Using Result() to convert Recorder into a http.Response yields a much
better user experience.
Closes#16837
Change-Id: Id0e636c12cd6adb1ba11f89953ff2b0f43758cf3
Reviewed-on: https://go-review.googlesource.com/27495
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
The old deprecation docs were referencing another deprecated field.
Fixes#16752
Change-Id: I44a690048e00ddc790a80214ecb7f5bb0a5b7b34
Reviewed-on: https://go-review.googlesource.com/27510
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Go 1.7 crashed after Transport.IdleConnTimeout if an HTTP/2 connection
was established but but its caller no longer wanted it. (Assuming the
connection cache was enabled, which it is by default)
Fixes#16208
Change-Id: I9628757f7669e344f416927c77f00ed3864839e3
Reviewed-on: https://go-review.googlesource.com/27450
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Certain browsers (Chrome 53, Safari 9.1.2, Firefox 46) won't correctly
follow a directory listing's links if the file name begins with a run
of characters then a colon, e.g. "foo:bar". Probably mistaking it for
a URI. However, they are happy to follow "./foo:bar", so this change
prepends "./" to all link hrefs in the directory listing of
FileServer.
Change-Id: I60ee8e1ebac73cbd3a3ac0f23e80fdf52e3dc352
Reviewed-on: https://go-review.googlesource.com/27440
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The RFC is clear that the Parameters in an AlgorithmIdentifer for an RSA
public key must be NULL. BoringSSL enforces this so we have strong
evidence that this is a widely compatible change.
Embarrassingly enough, the major source of violations of this is us. Go
used to get this correct in only one of two places. This was only fixed
in 2013 (with 4874bc9b). That's why lots of test certificates are
updated in this change.
Fixes#16166.
Change-Id: Ib9a4551349354c66e730d44eb8cee4ec402ea8ab
Reviewed-on: https://go-review.googlesource.com/27312
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
.local addresses are used by things like Kubernetes and Weave DNS; Go
should not avoid resolving them.
This is a partial revert of https://golang.org/cl/21328 which was too
strict of an interpretation of RFC 6762.
Fixes#16739
Change-Id: I349415b4eab5d61240dd18217bd95dc7d2105cd5
Reviewed-on: https://go-review.googlesource.com/27250
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
BIND libresolv allows values from 0 to 15.
For invalid values and negative numbers, 0 is used.
For numbers greater than 15, 15 is used.
Fixes#15419
Change-Id: I1009bc119c3e87919bcb55a80a35532e9fc3ba52
Reviewed-on: https://go-review.googlesource.com/24901
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If the server failed on us before we even tried to write any bytes,
it's safe to retry the request on a new connection, regardless of the
HTTP method/idempotence.
Fixes#15723
Change-Id: I25360f82aac530d12d2b3eef02c43ced86e62906
Reviewed-on: https://go-review.googlesource.com/27117
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Let the kernel pick a port for testing, and have the server in the
child process tell the parent (benchmarking) process the port that
was selected.
Fixes flakes like seen in https://golang.org/cl/27050 (and previously)
Change-Id: Ia2b705dc4152f70e0a5725015bdae09984d09d53
Reviewed-on: https://go-review.googlesource.com/27051
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The Go HTTP/1 client will make as many new TCP connections as the user requests.
The HTTP/2 client tried to have that behavior, but the policy of
whether a connection is re-usable didn't take into account the extra 1
stream counting against SETTINGS_MAX_CONCURRENT_STREAMS so in practice
users were getting errors.
For example, if the server's advertised max concurrent streams is 100
and 200 concurrrent Go HTTP requests ask for a connection at once, all
200 will think they can reuse that TCP connection, but then 100 will
fail later when the number of concurrent streams exceeds 100.
Instead, recognize the "no cached connections" error value in the
shouldRetryRequest method, so those 100 will retry a new connection.
This is the conservative fix for Go 1.7 so users don't get errors, and
to match the HTTP/1 behavior. Issues #13957 and #13774 are the more
involved bugs for Go 1.8.
Updates #16582
Updates #13957
Change-Id: I1f15a7ce60c07a4baebca87675836d6fe03993e8
Reviewed-on: https://go-review.googlesource.com/25580
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Chris Broadfoot <cbro@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
