This adds http.MaxBytesReader, similar to io.LimitReader,
but specific to http, and for preventing a class of DoS
attacks.
This also makes the 10MB ParseForm limit optional (if
not already set by a MaxBytesReader), documents it,
and also adds "PUT" as a valid verb for parsing forms
in the request body.
Improves issue 2093 (DoS protection)
Fixes#2165 (PUT form parsing)
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/4921049
ParseMediaType previously documented that it always returned
a non-nil map, but also documented that it returned a nil map
to signal an error.
That is confusing, contradictory and not Go-like.
Now it returns (mediatype string, params map, os.Error).
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4867054
This is just moving the URL code from package http into its own package,
which has been planned for a while.
Besides clarity, this also breaks a nascent dependency cycle the new template
package was about to introduce.
Add a gofix module, url, and use it to generate changes outside http and url.
Sadness about the churn, gladness about some of the naming improvements.
R=dsymonds, bradfitz, rsc, gustavo, r
CC=golang-dev
https://golang.org/cl/4893043
Fixes#2140.
The http.DefaultTransport's RoundTrip method leaves the http.Request object
in an altered state after performing the round trip. This patch removes
the header from the Request before returning to the client.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/4857041
calling filepath.FromSlash(x) make invalid character to serve file.
Fixes#2128
R=golang-dev, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4810064
serveFile was using absolute redirects, which didn't work under StripPrefix.
Now it uses relative redirects.
R=golang-dev, rsc, bradfitz
CC=golang-dev, kevlar
https://golang.org/cl/4789042
ServeMux depends on having a URL
in order to mux. It might be that the right
fix is to have CONNECT handlers just not
look at URL.
««« original CL description
http: do not parse req.URL for CONNECT
CONNECT's argument is not a URL.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4808044
»»»
TBR=bradfitz
CC=golang-dev
https://golang.org/cl/4798046
Something is broken, and investigation is underway.
In the meantime, godoc is broken, so disable sniffing for now
by reverting to the pre-sniffer state.
R=r
CC=golang-dev
https://golang.org/cl/4809046
This follows draft-ietf-websec-mime-sniff-03 in its intent,
though not its algorithmic specification.
R=rsc
CC=golang-dev
https://golang.org/cl/4746042
Previously, during the anti-zombie test, a
CGI process had to finish within ~625ms.
Now it gets ~5.6 seconds.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4741041
ServeFile() pass empty string to serveFile(). serveFile() should get
file extension via joining root and filename.
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4654089
This causes the child, if still writing, to get an error or
SIGPIPE and most likely exit so our subsequent wait can
finish.
A more guaranteed fix would be putting a time limit on the
child's overall execution, but this fixes the problem
I was having.
Fixes#2059
R=rsc
CC=golang-dev
https://golang.org/cl/4675081
CGI.pm used to be part of the Perl core,
but some distros now bundle perl-core
and perl-modules separately.
It's not worth depending on.
R=golang-dev, mattn.jp
CC=golang-dev
https://golang.org/cl/4675063
Change the signature of Split to have no count,
assuming a full split, and rename the existing
Split with a count to SplitN.
Do the same to package bytes.
Add a gofix module.
R=adg, dsymonds, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4661051
Permits serving from virtual filesystems, such as files linked
into a binary, or from a zip file.
Also adds a gofix for:
http.FileServer(root, prefix) -> http.StripPrefix(prefix, http.FileServer(http.Dir(root)))
R=r, rsc, gri, adg, dsymonds, r, gri
CC=golang-dev
https://golang.org/cl/4629047
Previously we were snapshotting the TLS state into *Request
before we did the HTTP ReadRequest, the first Read of which
triggered the TLS handshake implicitly.
Fixes#1956
R=golang-dev, rsc
CC=agl, golang-dev
https://golang.org/cl/4630072
As rsc suggested after change 58a6bdac3d12 was committed, we
now read the first byte of Request.Body when the
Request.ContentLength is 0 to disambiguate between a truly
zero-length body and a body of unknown length where the user
didn't set the ContentLength field.
This was also causing the reverse proxy problem where incoming
requests (which always have a body, of private type http.body,
even for 0-lengthed requests) were being relayed to the http
Transport for fetching, which was serializing the request as a
chunked request (since ContentLength was 0 and Body was
non-nil)
Fixes#1999
R=golang-dev, kevlar
CC=golang-dev
https://golang.org/cl/4628063
This is a core API change.
1) gofix misc src
2) Manual adjustments to the following files under src/pkg:
gob/decode.go
rpc/client.go
os/error.go
io/io.go
bufio/bufio.go
http/request.go
websocket/client.go
as well as:
src/cmd/gofix/testdata/*.go.in (reverted)
test/fixedbugs/bug243.go
3) Implemented gofix patch (oserrorstring.go) and test case (oserrorstring_test.go)
Compiles and runs all tests.
R=r, rsc, gri
CC=golang-dev
https://golang.org/cl/4607052
We'll do the right thing by default, but people wanting minimal
response sizes can explicitly remove the Date header.
(empty fields aren't written out)
R=rsc
CC=golang-dev
https://golang.org/cl/4634048
Previously Request and Response had redundant fields for
Referer, UserAgent, and cookies which caused confusion and
bugs. It also didn't allow us to expand the package over
time, since the way to access fields would be in the Headers
one day and promoted to a field the next day. That would be
hard to gofix, especially with code ranging over Headers.
After a discussion on the mail package's design with a similar
problem, we've designed to make the Headers be the source of
truth and add accessors instead.
Request:
change: Referer -> Referer()
change: UserAgent -> UserAgent()
change: Cookie -> Cookies()
new: Cookie(name) *Cookie
new: AddCookie(*Cookie)
Response:
change: Cookie -> Cookies()
Cookie:
new: String() string
R=rsc
CC=golang-dev
https://golang.org/cl/4620049
It was always a weird interface but I didn't know what I
was doing at the time. rsc questioned me about it then
but didn't press on it during review. Then adg bugged me
about it too recently.
So clean it up. It parallels the Writer struct too.
R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/4602063
Create a new spdy.Error type that includes the enumerated error type and
the associated stream id (0 if not associated with a specific stream).
This will let users handle errors differently (RST_STREAM vs GOAWAY).
R=bradfitz, rsc, rogpeppe
CC=golang-dev
https://golang.org/cl/4532131
With the ReadFrom change in the sendfile CL, it became
possible to illegally send a response to a HEAD request if you
did it via io.Copy.
Fixes#1939
R=rsc
CC=golang-dev
https://golang.org/cl/4584049
Via Russ Ross' bug report on golang-nuts, it was not possible
to send an HTTP request with a zero length body with either a
Content-Length (it was stripped) or chunking (it wasn't set).
This means Go couldn't upload 0-length objects to Amazon S3.
(which aren't as silly as they might sound, as S3 objects can
have key/values associated with them, set in the headers)
Amazon further doesn't supported chunked uploads. (not Go's
problem, but we should be able to let users set an explicit
Content-Length, even if it's zero.)
To fix the ambiguity of an explicit zero Content-Length and
the Request struct's default zero value, users need to
explicit set TransferEncoding to []string{"identity"} to force
the Request.Write to include a Content-Length: 0. identity is
in RFC 2616 but is ignored pretty much everywhere. We don't
even then serialize it on the wire, since it's kinda useless,
except as an internal sentinel value.
The "identity" value is then documented, but most users can
ignore that because NewRequest now sets that.
And adds more tests.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4603041
This replaces most the map[string][]string usage with
a new Values type name, with the usual methods.
It also changes client.PostForm to take a Values, rather
than a map[string]string, closing a TODO in the code.
R=rsc
CC=golang-dev
https://golang.org/cl/4532123
This change moves a number of common PKIX structures into
crypto/x509/pkix, from where x509, and ocsp can reference
them, saving duplication. It also removes x509/crl and merges it into
x509 and x509/pkix.
x509 is changed to take advantage of the big.Int support that now
exists in asn1. Because of this, the public/private key pair in
http/httptest/server.go had to be updated because it was serialised
with an old version of the code that didn't zero pad ASN.1 INTEGERs.
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4532115
It gets annoying to do this in caller code otherwise,
especially having to remember to Close one side.
R=rsc
CC=golang-dev
https://golang.org/cl/4517134
This removes exec.Run and replaces exec.Cmd with a
new implementation. The new exec.Cmd represents
both a currently-running command and also a command
being prepared. It has a good zero value.
You can Start + Wait on a Cmd, or simply Run it.
Start (and Run) deal with copying stdout, stdin,
and stderr between the Cmd's io.Readers and
io.Writers.
There are convenience methods to capture a command's
stdout and/or stderr.
R=r, n13m3y3r, rsc, gustavo, alex.brainman, dsymonds, r, adg, duzy.chan, mike.rosset, kevlar
CC=golang-dev
https://golang.org/cl/4552052
flate's reader greedily reads from the shared io.Reader in Framer. This leads to a data race on Framer.r. Fix this by providing a corkedReader to zlib.NewReaderDict(). We uncork the reader and allow it to read the number of bytes in the compressed payload.
Fixes#1884.
R=bradfitz, rsc, go.peter.90
CC=golang-dev
https://golang.org/cl/4530089
Also some cleanup, removing redundant code. Make more
things use NewRequest. Add some tests, docs.
R=golang-dev, adg, rsc
CC=golang-dev
https://golang.org/cl/4561047
Added a new Framer to handle reading/writing Frames. This is necessary since we have to maintain a compression context across streams.
TODO:
* Separate the types and read/write routines into different files.
* Improve error handling.
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4503042
I had a report that this was broken. It seems fine.
I think the reporter was just never flushing their response
headers. If I omit the test server's initial Flush I get the
same behavior as reported. (a hang at Client.Get)
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4552062
A user pointed out that Go didn't work with their
corp proxy, always throwing 400 Bad Request errors.
Looking at the RFC 2616, Host is always required,
even with proxies.
The old code assumed that writing an absolute URL
in the first line of an HTTP request implied
that the Host header was no longer necessary.
Double-checked behavior with curl.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4539075
This appears to have been a long-standing formatting bug.
The test cases has misformatted golden files.
Applied gofmt -w src misc .
Fixes#1839.
R=iant
CC=golang-dev
https://golang.org/cl/4515113
It's documented as such, but it was never wired up
after Transport went in and Head was fixed.
If people don't want redirects, that's what RoundTripper/
Transport are for. Or a custom redirect policy.
R=golang-dev, kevlar
CC=golang-dev
https://golang.org/cl/4526065
HEAD requests should in my opinion have the ability to follow redirects
like the implementation of GET requests does. My use case is polling
several thousand severs to check if they respond with 200 status codes.
Using GET requests is neither efficient in running time of the task nor
for bandwidth consumption.
This suggested patch changes the return signature of http.Head() to match
that of http.Get(), providing the final URL in a redirect chain.
`curl -IL http://google.com` follows redirects with HEAD requests just fine.
Fixes#1806.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4517058
This CL:
-- removes Response.RequestMethod string
-- adds Response.Request *Request
-- removes the finalURL result parameter from client.Get()
-- adds a gofix rule for callers of http.Get which assign
the final url to the blank identifier; warning otherwise
Caller who did:
res, finalURL, err := http.Get(...)
now need to do:
res, err := http.Get(...)
if err != nil {
...
}
finalURL := res.Request.URL.String()
R=rsc
CC=golang-dev
https://golang.org/cl/4535056
There were a couple issues:
-- HEAD requests were attempting to be ungzipped,
despite having no content. That was fixed in
the previous patch version, but ultimately was
fixed as a result of other refactoring:
-- persist.go's ClientConn "lastbody" field was
remembering the wrong body, since we were
mucking with it later. Instead, ditch
ClientConn's readRes func field and add a new
method passing it in, so we can use a closure
and do all our bodyEOFSignal + gunzip stuff
in one place, simplifying a lot of code and
not requiring messing with ClientConn's innards.
-- closing the gzip reader didn't consume its
contents. if the caller wasn't done reading
all the response body and ClientConn closed it
(thinking it'd move past those bytes in the
TCP stream), it actually wouldn't. so introduce
a new wrapper just for gzip reader to have its
Close method do an ioutil.Discard on its body
first, before the close.
Fixes#1725Fixes#1804
R=rsc, eivind
CC=golang-dev
https://golang.org/cl/4523058
RFC 6265 requires that user agents MUST NOT send more than
one Cookie header in a request.
Note, this change also fixes an issue when sending requests
with more than one cookie header line to a php script served
by an apache web server. Apache concatenates the cookies
with ", ", but php tries to split them only at ";". E.g.
two cookies: "a=b, c=d" are seen by php as one cookie "a"
with the value "b, c=d".
Fixes#1801
R=bradfitz
CC=golang-dev
https://golang.org/cl/4535048
An optimization in Transport which re-uses TCP
connections early in the case where there is
no response body interacted poorly with
ErrBodyReadAfterClose. Upon recycling the TCP
connection early we would Close the Response.Body
(in case the user forgot to), but in the case
of a zero-lengthed body, the user's handler might
not have run yet.
This CL makes sure the Transport doesn't try
to Close requests when we're about to immediately
re-use the TCP connection.
This also includes additional tests I wrote
while debugging.
R=rsc, bradfitzgoog
CC=golang-dev
https://golang.org/cl/4529050
