Previously Request and Response had redundant fields for
Referer, UserAgent, and cookies which caused confusion and
bugs. It also didn't allow us to expand the package over
time, since the way to access fields would be in the Headers
one day and promoted to a field the next day. That would be
hard to gofix, especially with code ranging over Headers.
After a discussion on the mail package's design with a similar
problem, we've designed to make the Headers be the source of
truth and add accessors instead.
Request:
change: Referer -> Referer()
change: UserAgent -> UserAgent()
change: Cookie -> Cookies()
new: Cookie(name) *Cookie
new: AddCookie(*Cookie)
Response:
change: Cookie -> Cookies()
Cookie:
new: String() string
R=rsc
CC=golang-dev
https://golang.org/cl/4620049
It was always a weird interface but I didn't know what I
was doing at the time. rsc questioned me about it then
but didn't press on it during review. Then adg bugged me
about it too recently.
So clean it up. It parallels the Writer struct too.
R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/4602063
Create a new spdy.Error type that includes the enumerated error type and
the associated stream id (0 if not associated with a specific stream).
This will let users handle errors differently (RST_STREAM vs GOAWAY).
R=bradfitz, rsc, rogpeppe
CC=golang-dev
https://golang.org/cl/4532131
With the ReadFrom change in the sendfile CL, it became
possible to illegally send a response to a HEAD request if you
did it via io.Copy.
Fixes#1939
R=rsc
CC=golang-dev
https://golang.org/cl/4584049
Via Russ Ross' bug report on golang-nuts, it was not possible
to send an HTTP request with a zero length body with either a
Content-Length (it was stripped) or chunking (it wasn't set).
This means Go couldn't upload 0-length objects to Amazon S3.
(which aren't as silly as they might sound, as S3 objects can
have key/values associated with them, set in the headers)
Amazon further doesn't supported chunked uploads. (not Go's
problem, but we should be able to let users set an explicit
Content-Length, even if it's zero.)
To fix the ambiguity of an explicit zero Content-Length and
the Request struct's default zero value, users need to
explicit set TransferEncoding to []string{"identity"} to force
the Request.Write to include a Content-Length: 0. identity is
in RFC 2616 but is ignored pretty much everywhere. We don't
even then serialize it on the wire, since it's kinda useless,
except as an internal sentinel value.
The "identity" value is then documented, but most users can
ignore that because NewRequest now sets that.
And adds more tests.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4603041
This replaces most the map[string][]string usage with
a new Values type name, with the usual methods.
It also changes client.PostForm to take a Values, rather
than a map[string]string, closing a TODO in the code.
R=rsc
CC=golang-dev
https://golang.org/cl/4532123
This change moves a number of common PKIX structures into
crypto/x509/pkix, from where x509, and ocsp can reference
them, saving duplication. It also removes x509/crl and merges it into
x509 and x509/pkix.
x509 is changed to take advantage of the big.Int support that now
exists in asn1. Because of this, the public/private key pair in
http/httptest/server.go had to be updated because it was serialised
with an old version of the code that didn't zero pad ASN.1 INTEGERs.
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4532115
It gets annoying to do this in caller code otherwise,
especially having to remember to Close one side.
R=rsc
CC=golang-dev
https://golang.org/cl/4517134
This removes exec.Run and replaces exec.Cmd with a
new implementation. The new exec.Cmd represents
both a currently-running command and also a command
being prepared. It has a good zero value.
You can Start + Wait on a Cmd, or simply Run it.
Start (and Run) deal with copying stdout, stdin,
and stderr between the Cmd's io.Readers and
io.Writers.
There are convenience methods to capture a command's
stdout and/or stderr.
R=r, n13m3y3r, rsc, gustavo, alex.brainman, dsymonds, r, adg, duzy.chan, mike.rosset, kevlar
CC=golang-dev
https://golang.org/cl/4552052
flate's reader greedily reads from the shared io.Reader in Framer. This leads to a data race on Framer.r. Fix this by providing a corkedReader to zlib.NewReaderDict(). We uncork the reader and allow it to read the number of bytes in the compressed payload.
Fixes#1884.
R=bradfitz, rsc, go.peter.90
CC=golang-dev
https://golang.org/cl/4530089
Also some cleanup, removing redundant code. Make more
things use NewRequest. Add some tests, docs.
R=golang-dev, adg, rsc
CC=golang-dev
https://golang.org/cl/4561047
Added a new Framer to handle reading/writing Frames. This is necessary since we have to maintain a compression context across streams.
TODO:
* Separate the types and read/write routines into different files.
* Improve error handling.
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4503042
I had a report that this was broken. It seems fine.
I think the reporter was just never flushing their response
headers. If I omit the test server's initial Flush I get the
same behavior as reported. (a hang at Client.Get)
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4552062
A user pointed out that Go didn't work with their
corp proxy, always throwing 400 Bad Request errors.
Looking at the RFC 2616, Host is always required,
even with proxies.
The old code assumed that writing an absolute URL
in the first line of an HTTP request implied
that the Host header was no longer necessary.
Double-checked behavior with curl.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4539075
This appears to have been a long-standing formatting bug.
The test cases has misformatted golden files.
Applied gofmt -w src misc .
Fixes#1839.
R=iant
CC=golang-dev
https://golang.org/cl/4515113
It's documented as such, but it was never wired up
after Transport went in and Head was fixed.
If people don't want redirects, that's what RoundTripper/
Transport are for. Or a custom redirect policy.
R=golang-dev, kevlar
CC=golang-dev
https://golang.org/cl/4526065
HEAD requests should in my opinion have the ability to follow redirects
like the implementation of GET requests does. My use case is polling
several thousand severs to check if they respond with 200 status codes.
Using GET requests is neither efficient in running time of the task nor
for bandwidth consumption.
This suggested patch changes the return signature of http.Head() to match
that of http.Get(), providing the final URL in a redirect chain.
`curl -IL http://google.com` follows redirects with HEAD requests just fine.
Fixes#1806.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4517058
This CL:
-- removes Response.RequestMethod string
-- adds Response.Request *Request
-- removes the finalURL result parameter from client.Get()
-- adds a gofix rule for callers of http.Get which assign
the final url to the blank identifier; warning otherwise
Caller who did:
res, finalURL, err := http.Get(...)
now need to do:
res, err := http.Get(...)
if err != nil {
...
}
finalURL := res.Request.URL.String()
R=rsc
CC=golang-dev
https://golang.org/cl/4535056
There were a couple issues:
-- HEAD requests were attempting to be ungzipped,
despite having no content. That was fixed in
the previous patch version, but ultimately was
fixed as a result of other refactoring:
-- persist.go's ClientConn "lastbody" field was
remembering the wrong body, since we were
mucking with it later. Instead, ditch
ClientConn's readRes func field and add a new
method passing it in, so we can use a closure
and do all our bodyEOFSignal + gunzip stuff
in one place, simplifying a lot of code and
not requiring messing with ClientConn's innards.
-- closing the gzip reader didn't consume its
contents. if the caller wasn't done reading
all the response body and ClientConn closed it
(thinking it'd move past those bytes in the
TCP stream), it actually wouldn't. so introduce
a new wrapper just for gzip reader to have its
Close method do an ioutil.Discard on its body
first, before the close.
Fixes#1725Fixes#1804
R=rsc, eivind
CC=golang-dev
https://golang.org/cl/4523058
RFC 6265 requires that user agents MUST NOT send more than
one Cookie header in a request.
Note, this change also fixes an issue when sending requests
with more than one cookie header line to a php script served
by an apache web server. Apache concatenates the cookies
with ", ", but php tries to split them only at ";". E.g.
two cookies: "a=b, c=d" are seen by php as one cookie "a"
with the value "b, c=d".
Fixes#1801
R=bradfitz
CC=golang-dev
https://golang.org/cl/4535048
An optimization in Transport which re-uses TCP
connections early in the case where there is
no response body interacted poorly with
ErrBodyReadAfterClose. Upon recycling the TCP
connection early we would Close the Response.Body
(in case the user forgot to), but in the case
of a zero-lengthed body, the user's handler might
not have run yet.
This CL makes sure the Transport doesn't try
to Close requests when we're about to immediately
re-use the TCP connection.
This also includes additional tests I wrote
while debugging.
R=rsc, bradfitzgoog
CC=golang-dev
https://golang.org/cl/4529050
Add local URI path support, which isn't as fringe
as I originally thought. (it's supported by Apache)
Send an implicit 302 status on redirects (not 200).
Fixes#1597
R=rsc, r
CC=golang-dev
https://golang.org/cl/4442089
NewRequest will save a lot of boilerplate code.
This also updates some docs on Request.Write and
adds some tests.
R=rsc, petar-m, r
CC=golang-dev
https://golang.org/cl/4406047
Ubuntu and/or GNOME have some bug that likes
to set the "http_proxy" environment variable
and forgets to unset it. This is annoying
to debug. Be clear in the error message that
a proxy was in use.
R=rsc
CC=golang-dev
https://golang.org/cl/4409045
This fixes our http behavior (even if Handlers forget to
consume a request body, we do it for them before we send
their response header), fixes the racy TestServerExpect,
and adds TestServerConsumesRequestBody.
With GOMAXPROCS>1, the http tests now seem race-free.
R=rsc
CC=golang-dev
https://golang.org/cl/4419042
This mostly adds Expect 100-continue tests (from
the perspective of server correctness) that were
missing before.
It also fixes a few missing cases that will
probably never come up in practice, but it's nice
to have handled correctly.
Proper 100-continue client support remains a TODO.
R=rsc, bradfitzwork
CC=golang-dev
https://golang.org/cl/4399044
It matches encoding/line exactly and the tests are copied from there.
If we land this, then encoding/line will get marked as deprecated then
deleted in time.
R=rsc, rog, peterGo
CC=golang-dev
https://golang.org/cl/4389046
We replace the current Open with:
OpenFile(name, flag, perm) // same as old Open
Open(name) // same as old Open(name, O_RDONLY, 0)
Create(name) // same as old Open(name, O_RDWR|O_TRUNC|O_CREAT, 0666)
This CL includes a gofix module and full code updates: all.bash passes.
(There may be a few comments I missed.)
The interesting packages are:
gofix
os
Everything else is automatically generated except for hand tweaks to:
src/pkg/io/ioutil/ioutil.go
src/pkg/io/ioutil/tempfile.go
src/pkg/crypto/tls/generate_cert.go
src/cmd/goyacc/goyacc.go
src/cmd/goyacc/units.y
R=golang-dev, bradfitzwork, rsc, r2
CC=golang-dev
https://golang.org/cl/4357052
Amazon S3 sends Transfer-Encoding "chunked"
on its 404 responses to HEAD requests for
missing objects.
We weren't ignoring the Transfer-Encoding
and were thus interpretting the subsequent
response headers as a chunk header from the
previous responses body (but a HEAD response
can't have a body)
R=rsc, adg
CC=golang-dev
https://golang.org/cl/4346050
A connection shouldn't be made available
for re-use until its body has been consumed.
(except in the case of pipelining, which isn't
implemented yet)
This CL fixes some issues seen with heavy load
against Amazon S3.
Subtle implementation detail: to prevent a race
with the client requesting a new connection
before previous one is returned, we actually
have to call putIdleConnection _before_ we
return from the final Read/Close call on the
http.Response.Body.
R=rsc, adg
CC=golang-dev
https://golang.org/cl/4351048
The transport readLoop was waiting forever for the client to
read the non-existent body before proceeding to read the next
request.
R=rsc
CC=golang-dev
https://golang.org/cl/4357051
According to RFC 3986: "For consistency, URI producers
and normalizers should use uppercase hexadecimal digits
for all percent-encodings." Using lower case characters
makes it incompatible with Google APIs when signing OAuth requests.
R=golang-dev, rsc1, rsc
CC=golang-dev
https://golang.org/cl/4352044
This patch adds a connection cache and keep-alive
support to Transport, which is used by the
HTTP client.
It's also structured such that it's easy to add
HTTP pipelining in the future.
R=rsc, petar-m, bradfitzwork, r
CC=golang-dev
https://golang.org/cl/4272045
This just returns a ClientConn suitable for writing
proxy requests. To be used in Transport.
R=rsc, petar-m
CC=golang-dev
https://golang.org/cl/4290052
Transport.Do -> RoundTripper.RoundTrip
This makes way for a subsequent CL to export the
currently private RoundTripper implementation
as struct Transport.
R=rsc, r
CC=golang-dev
https://golang.org/cl/4286043
The Flush functionality wasn't removed, but now you have
to test if your ResponseWriter is also a Flusher:
func ServeHTTP(rw http.ResponseWriter, req *http.Request) {
if f, ok := rw.(http.Flusher); ok {
f.Flush()
}
}
R=rsc, bradfitzwork
CC=gburd, golang-dev
https://golang.org/cl/4239077
Caller code needs to change:
rw.SetHeader("Content-Type", "text/plain")
to:
rw.Header().Set("Content-Type", "text/plain")
This now permits returning multiple headers
with the same name using Add:
rw.Header().Add("Set-Cookie", "..")
rw.Header().Add("Set-Cookie", "..")
This patch also fixes serialization of headers, removing newline characters.
Fixes#488Fixes#914
R=rsc
CC=gburd, golang-dev
https://golang.org/cl/4239076
The http/cgi package now supports both being
a CGI host or being a CGI child process.
R=rsc, adg, bradfitzwork
CC=golang-dev
https://golang.org/cl/4245070
Was only breaking on some dashboard builds because
not all run the network tests.
R=bradfitzgo, bradfitzwork
CC=golang-dev
https://golang.org/cl/4240086
The Hijack functionality wasn't removed, but now you have
to test if your ResponseWriter is also a Hijacker:
func ServeHTTP(rw http.ResponseWriter, req *http.Request) {
if hj, ok := rw.(http.Hijacker); ok {
hj.Hijack(..)
}
}
R=rsc
CC=golang-dev
https://golang.org/cl/4245064
The path package now contains only functions which
deal with slashed paths, sensible for any OS when dealing
with network paths or URLs. OS-specific functionality
has been moved into the new path/filepath package.
This also includes fixes for godoc, goinstall and other
packages which were mixing slashed and OS-specific paths.
R=rsc, gri, mattn, brainman
CC=golang-dev
https://golang.org/cl/4252044
This also breaks fs_test into two parts
as the range tests test http's private httpRange
and I had to change the fs_test package from
"http" to "http_test" to use httptest which otherwise
has a cyclic depedency back on http.
Aside: we should start exposing the Range
stuff in the future.
R=rsc
CC=golang-dev
https://golang.org/cl/4261047
Currently all http handlers reply to HTTP/1.1 requests with
chunked responses. This patch allows handlers to opt-out of
that behavior by pre-declaring their Content-Length (which is
then enforced) and unsetting their Transfer-Encoding or
setting it to the "identity" encoding.
R=rsc, bradfitzwork
CC=golang-dev
https://golang.org/cl/4245058
This is again an intentionally minimal change.
The plan is to keep Client's zero value be a usable
client, with optional fields being added over time
(e.g. cookie manager, redirect policy, auth)
R=rsc
CC=golang-dev
https://golang.org/cl/4239044
