1
0
mirror of https://github.com/golang/go synced 2024-10-05 12:21:22 -06:00
Commit Graph

9420 Commits

Author SHA1 Message Date
Shenghou Ma
c274ff6761 fmt: fix go syntax formatting of []byte(nil)
Fixes #7639.

LGTM=rsc
R=r, adg, rsc
CC=golang-codereviews
https://golang.org/cl/81240043
2014-04-03 16:11:03 -04:00
Russ Cox
5fb39cc6a2 net: accept a few more errors in Accept4 wrapper
Fixes #7271.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/84170043
2014-04-03 16:10:45 -04:00
Russ Cox
844b625ebc net/url: add test of "Windows" file URL
This is just testing the status quo, so that any future attempt
to change it will make the test break and redirect the person
making the change to look at issue 6027.

Fixes #6027.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/83930046
2014-04-03 16:10:33 -04:00
David Thomas
730db0affc archive/tar: add support for GNU sparse files.
Supports all the current GNU tar sparse formats, including the
old GNU format and the GNU PAX format versions 0.0, 0.1, and 1.0.
Fixes #3864.

LGTM=rsc
R=golang-codereviews, dave, gobot, dsymonds, rsc
CC=golang-codereviews
https://golang.org/cl/64740043
2014-04-03 20:01:04 +00:00
Russ Cox
b2cbf49343 runtime: fix fault during arm software floating point
The software floating point runs with m->locks++
to avoid being preempted; recognize this case in panic
and undo it so that m->locks is maintained correctly
when panicking.

Fixes #7553.

LGTM=dvyukov
R=golang-codereviews, dvyukov
CC=golang-codereviews
https://golang.org/cl/84030043
2014-04-03 15:39:48 -04:00
Brad Fitzpatrick
9dbb185fb6 net/http: clarify Request fields' client-vs-server semantics
Fixes #7682

LGTM=adg
R=golang-codereviews, adg
CC=dsymonds, golang-codereviews, iant
https://golang.org/cl/83800043
2014-04-02 21:05:41 -07:00
Russ Cox
c40480ddd9 runtime: print up to 10 words of arguments
The old limit of 5 was chosen because we didn't actually know how
many bytes of arguments there were; 5 was a halfway point between
printing some useful information and looking ridiculous.

Now we know how many bytes of arguments there are, and we stop
the printing when we reach that point, so the "looking ridiculous" case
doesn't happen anymore: we only print actual argument words.
The cutoff now serves only to truncate very long (but real) argument lists.

In multiple debugging sessions recently (completely unrelated bugs)
I have been frustrated by not seeing more of the long argument lists:
5 words is only 2.5 interface values or strings, and not even 2 slices.
Double the max amount we'll show.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews, iant, r
https://golang.org/cl/83850043
2014-04-02 23:00:40 -04:00
Dave Cheney
9121e7e4df runtime: check that new slice cap doesn't overflow
Fixes #7550.

LGTM=iant
R=golang-codereviews, iant, josharian
CC=golang-codereviews
https://golang.org/cl/83520043
2014-04-03 13:44:44 +11:00
Ian Lance Taylor
7d299d031d net: don't export netFD readFrom, writeTo, readMsg, writeMsg methods
There is no way to call them from outside the net package.
They are used to implement UCPConn.ReadMsgUDP and similar.

LGTM=mikioh.mikioh
R=golang-codereviews, mikioh.mikioh
CC=golang-codereviews
https://golang.org/cl/83730044
2014-04-02 17:06:51 -07:00
Brad Fitzpatrick
84db9e09d9 crypto/tls: deflake TestConnReadNonzeroAndEOF
Fixes #7683

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/83080048
2014-04-02 14:31:57 -07:00
Russ Cox
81bc9b3ffd runtime: revert change to PoisonPtr value
Submitted accidentally in CL 83630044.
Fixes various builds.

TBR=khr
CC=golang-codereviews
https://golang.org/cl/83100047
2014-04-02 16:55:30 -04:00
Russ Cox
4676fae525 cmd/gc, cmd/ld, runtime: compact liveness bitmaps
Reduce footprint of liveness bitmaps by about 5x.

1. Mark all liveness bitmap symbols as 4-byte aligned
(they were aligned to a larger size by default).

2. The bitmap data is a bitmap count n followed by n bitmaps.
Each bitmap begins with its own count m giving the number
of bits. All the m's are the same for the n bitmaps.
Emit this bitmap length once instead of n times.

3. Many bitmaps within a function have the same bit values,
but each call site was given a distinct bitmap. Merge duplicate
bitmaps so that no bitmap is written more than once.

4. Many functions end up with the same aggregate bitmap data.
We used to name the bitmap data funcname.gcargs and funcname.gclocals.
Instead, name it gclocals.<md5 of data> and mark it dupok so
that the linker coalesces duplicate sets. This cut the bitmap
data remaining after step 3 by 40%; I was not expecting it to
be quite so dramatic.

Applied to "go build -ldflags -w code.google.com/p/go.tools/cmd/godoc":

                bitmaps           pclntab           binary on disk
before this CL  1326600           1985854           12738268
4-byte align    1154288 (0.87x)   1985854 (1.00x)   12566236 (0.99x)
one bitmap len   782528 (0.54x)   1985854 (1.00x)   12193500 (0.96x)
dedup bitmap     414748 (0.31x)   1948478 (0.98x)   11787996 (0.93x)
dedup bitmap set 245580 (0.19x)   1948478 (0.98x)   11620060 (0.91x)

While here, remove various dead blocks of code from plive.c.

Fixes #6929.
Fixes #7568.

LGTM=khr
R=khr
CC=golang-codereviews
https://golang.org/cl/83630044
2014-04-02 16:49:27 -04:00
Mikio Hara
e88e7ed6ba net: enable unixpacket test on available platforms
DragonFlyBSD, FreeBSD 9 and beyond, NetBSD 6 and beyond, and
Solaris (illumos) support AF_UNIX+SOCK_SEQPACKET socket.

LGTM=dave
R=golang-codereviews, dave
CC=golang-codereviews
https://golang.org/cl/83390043
2014-04-02 19:43:39 +09:00
Mikio Hara
67a5181045 net: make WriteTo, WriteToUnix and WriteMsgUnix fail when connectionless-mode UnixConn is already connected
This CL tries to fill the gap between Linux and other Unix-like systems
in the same way UDPConn already did.

Fixes #7677.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/83330045
2014-04-02 19:42:05 +09:00
Dmitriy Vyukov
f4ef6977ff runtime: ignore pointers to global objects in SetFinalizer
Update #7656

LGTM=rsc
R=rsc, iant
CC=golang-codereviews
https://golang.org/cl/82560043
2014-04-02 10:19:28 +04:00
Andrew Gerrand
2f3776ac27 time: increase timeout in negative sleep duration test
There's enough jitter in the scheduler on overloaded machines
that 25ms is not enough.

LGTM=dave
R=golang-codereviews, gobot, rsc, dave
CC=golang-codereviews
https://golang.org/cl/83300044
2014-04-02 08:23:35 +11:00
Keith Randall
6c7cbf086c runtime: get rid of most uses of REP for copying/zeroing.
REP MOVSQ and REP STOSQ have a really high startup overhead.
Use a Duff's device to do the repetition instead.

benchmark                 old ns/op     new ns/op     delta
BenchmarkClearFat32       7.20          1.60          -77.78%
BenchmarkCopyFat32        6.88          2.38          -65.41%
BenchmarkClearFat64       7.15          3.20          -55.24%
BenchmarkCopyFat64        6.88          3.44          -50.00%
BenchmarkClearFat128      9.53          5.34          -43.97%
BenchmarkCopyFat128       9.27          5.56          -40.02%
BenchmarkClearFat256      13.8          9.53          -30.94%
BenchmarkCopyFat256       13.5          10.3          -23.70%
BenchmarkClearFat512      22.3          18.0          -19.28%
BenchmarkCopyFat512       22.0          19.7          -10.45%
BenchmarkCopyFat1024      36.5          38.4          +5.21%
BenchmarkClearFat1024     35.1          35.0          -0.28%

TODO: use for stack frame zeroing
TODO: REP prefixes are still used for "reverse" copying when src/dst
regions overlap.  Might be worth fixing.

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews, r
https://golang.org/cl/81370046
2014-04-01 12:51:02 -07:00
Russ Cox
cfb347fc0a runtime: use correct pc to obtain liveness info during stack copy
The old code was using the PC of the instruction after the CALL.
Variables live during the call but not live when it returns would
not be seen as live during the stack copy, which might lead to
corruption. The correct PC to use is the one just before the
return address. After this CL the lookup matches what mgc0.c does.

The only time this matters is if you have back to back CALL instructions:

        CALL f1 // x live here
        CALL f2 // x no longer live

If a stack copy occurs during the execution of f1, the old code will
use the liveness bitmap intended for the execution of f2 and will not
treat x as live.

The only way this situation can arise and cause a problem in a stack copy
is if x lives on the stack has had its address taken but the compiler knows
enough about the context to know that x is no longer needed once f1
returns. The compiler has never known that much, so using the f2 context
cannot currently cause incorrect execution. For the same reason, it is not
possible to write a test for this today.

CL 83090046 will make the compiler precise enough in some cases
that this distinction will start mattering. The existing stack growth tests
in package runtime will fail if that CL is submitted without this one.

While we're here, print the frame PC in debug mode and update the
bitmap interpretation strings.

LGTM=khr
R=khr
CC=golang-codereviews
https://golang.org/cl/83250043
2014-04-01 14:57:58 -04:00
Russ Cox
1ec4d5e9e7 runtime: adjust GODEBUG=allocfreetrace=1 and GODEBUG=gcdead=1
GODEBUG=allocfreetrace=1:

The allocfreetrace=1 mode prints a stack trace for each block
allocated and freed, and also a stack trace for each garbage collection.

It was implemented by reusing the heap profiling support: if allocfreetrace=1
then the heap profile was effectively running at 1 sample per 1 byte allocated
(always sample). The stack being shown at allocation was the stack gathered
for profiling, meaning it was derived only from the program counters and
did not include information about function arguments or frame pointers.
The stack being shown at free was the allocation stack, not the free stack.
If you are generating this log, you can find the allocation stack yourself, but
it can be useful to see exactly the sequence that led to freeing the block:
was it the garbage collector or an explicit free? Now that the garbage collector
runs on an m0 stack, the stack trace for the garbage collector was never interesting.

Fix all these problems:

1. Decouple allocfreetrace=1 from heap profiling.
2. Print the standard goroutine stack traces instead of a custom format.
3. Print the stack trace at time of allocation for an allocation,
   and print the stack trace at time of free (not the allocation trace again)
   for a free.
4. Print all goroutine stacks at garbage collection. Having all the stacks
   means that you can see the exact point at which each goroutine was
   preempted, which is often useful for identifying liveness-related errors.

GODEBUG=gcdead=1:

This mode overwrites dead pointers with a poison value.
Detect the poison value as an invalid pointer during collection,
the same way that small integers are invalid pointers.

LGTM=khr
R=khr
CC=golang-codereviews
https://golang.org/cl/81670043
2014-04-01 13:30:10 -04:00
Shenghou Ma
16b95507a5 html/template: fix broken links
Fixes #7562.

LGTM=nigeltao
R=nigeltao
CC=golang-codereviews
https://golang.org/cl/81190044
2014-04-01 02:57:51 -04:00
Mikio Hara
83ac901fb9 net: tweak the ephemeral port range on dragonfly
On DragonFly BSD, we adjust the ephemeral port range because
unlike other BSD systems its default ephemeral port range
doesn't conform to IANA recommendation as described in RFC 6355
and is pretty narrow.

On DragonFly BSD 3.6: default range [1024, 5000], high range [49152, 65535]
On FreeBSD 10: default range [10000, 65535], high range [49152, 65535]
On Linux 3.11: default range [32768, 61000]

Fixes #7541.

LGTM=iant
R=jsing, gobot, iant
CC=golang-codereviews
https://golang.org/cl/80610044
2014-03-29 13:04:25 +09:00
Mikio Hara
a7858a40a5 syscall: add SendmsgN for BSD variants, Linux and Solaris
SendmsgN is an alternate version Sendmsg that also returns
the number of bytes transferred, instead of just the error.

Update #7645

LGTM=aram, iant
R=iant, aram, bradfitz
CC=golang-codereviews
https://golang.org/cl/81210043
2014-03-29 09:28:40 +09:00
Brad Fitzpatrick
f074565158 bytes, strings: allow Reader.Seek past 1<<31
Fixes #7654

LGTM=rsc
R=rsc, dan.kortschak
CC=golang-codereviews
https://golang.org/cl/81530043
2014-03-28 12:23:51 -07:00
Adam Langley
ea3353b64c crypto/x509: unbreak Windows build.
This change sets systemSkip on a test where Go and CAPI have different
chain building behaviour. CAPI is correct, but aligning the Go code is
probably too large a change prior to 1.3.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/81620043
2014-03-28 10:36:52 -04:00
Alex Brainman
2dc7552f57 runtime: ignore windows exception if not in Go binary
LGTM=minux.ma
R=golang-codereviews, minux.ma
CC=golang-codereviews
https://golang.org/cl/80400043
2014-03-28 17:35:00 +11:00
Mikio Hara
a05ffd85aa net: make IPv6 capability test more suitable for address family selection on the dual IP stack node
For now we strictly use IPV6_V6ONLY=1 for IPv6-only communications
and IPV6_V6ONLY=0 for both IPv4 and IPv6 communications. So let the
capability test do the same.

LGTM=iant
R=golang-codereviews, gobot, iant
CC=golang-codereviews
https://golang.org/cl/80140044
2014-03-28 13:27:51 +09:00
Mikio Hara
753bdc0f47 syscall: don't generate RTF_BITS constant on OS X Mavericks and beyond
LGTM=iant
R=iant, bradfitz
CC=golang-codereviews
https://golang.org/cl/80700044
2014-03-28 13:27:14 +09:00
Alex Brainman
277a7b22f1 runtime: do not crash when GetQueuedCompletionStatusEx is missing
Fixes #7635

LGTM=minux.ma
R=golang-codereviews, minux.ma
CC=golang-codereviews
https://golang.org/cl/80390043
2014-03-28 12:37:14 +11:00
Adam Langley
0f272d1378 crypto/x509: update tests because Windows removed the Verisign root.
The root update on 3/11/2014 removed the Verisign root cert that the Go
tests use. This only affects the 'TestSystemVerify' test in
crypto/x509.

Fixes #7523.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/80000044
2014-03-27 17:56:02 -04:00
Russ Cox
5a23a7e52c runtime: enable 'bad pointer' check during garbage collection of Go stack frames
This is the same check we use during stack copying.
The check cannot be applied to C stack frames, even
though we do emit pointer bitmaps for the arguments,
because (1) the pointer bitmaps assume all arguments
are always live, not true of outputs during the prologue,
and (2) the pointer bitmaps encode interface values as
pointer pairs, not true of interfaces holding integers.

For the rest of the frames, however, we should hold ourselves
to the rule that a pointer marked live really is initialized.
The interface scanning already implicitly checks this
because it interprets the type word  as a valid type pointer.

This may slow things down a little because of the extra loads.
Or it may speed things up because we don't bother enqueuing
nil pointers anymore. Enough of the rest of the system is slow
right now that we can't measure it meaningfully.
Enable for now, even if it is slow, to shake out bugs in the
liveness bitmaps, and then decide whether to turn it off
for the Go 1.3 release (issue 7650 reminds us to do this).

The new m->traceback field lets us force printing of fp=
values on all goroutine stack traces when we detect a
bad pointer. This makes it easier to understand exactly
where in the frame the bad pointer is, so that we can trace
it back to a specific variable and determine what is wrong.

Update #7650

LGTM=khr
R=khr
CC=golang-codereviews
https://golang.org/cl/80860044
2014-03-27 14:06:15 -04:00
Russ Cox
ef3c0e7e61 regexp/syntax: remove InstLast
This was added by the one-pass CL (post Go 1.2)
so it can still be removed.

Removing because surely there will be new operations
added later, and we can't change the constant value
once we define it, so "last" is a bad concept to expose.

Nothing uses it.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/81160043
2014-03-27 14:05:14 -04:00
Russ Cox
fc6befba3c runtime: initialize complete Hiter during mapiterinit
The garbage collector will scan these pointers,
so make sure they are initialized.

LGTM=bradfitz, khr
R=khr, bradfitz
CC=golang-codereviews
https://golang.org/cl/80960047
2014-03-26 21:52:29 -04:00
Dmitriy Vyukov
7c75a862b4 runtime: eliminate false retention due to m->moreargp/morebuf
m->moreargp/morebuf were not cleared in case of preemption and stack growing,
it can lead to persistent leaks of large memory blocks.

It seems to fix the sync.Pool finalizer failures. I've run the test 500'000 times
w/o a single failure; previously it would fail dozens of times.

Fixes #7633.
Fixes #7533.

LGTM=rsc
R=golang-codereviews
CC=golang-codereviews, khr, rsc
https://golang.org/cl/80480044
2014-03-26 19:06:15 +04:00
Dmitriy Vyukov
d89a738378 runtime: support channel-based mutex in race detector
Update channel race annotations to support change in
cl/75130045: doc: allow buffered channel as semaphore without initialization
The new annotations are added only for channels with capacity 1.
Strictly saying it's possible to construct a counter-example that
will produce a false positive with capacity > 1. But it's hardly can
lead to false positives in real programs, at least I would like to see such programs first.
Any additional annotations also increase probability of false negatives,
so I would prefer to add them lazily.

LGTM=rsc
R=golang-codereviews
CC=golang-codereviews, iant, khr, rsc
https://golang.org/cl/76970043
2014-03-26 19:05:48 +04:00
Dmitriy Vyukov
f8c350873c runtime: fix yet another race in bgsweep
Currently it's possible that bgsweep finishes before all spans
have been swept (we only know that sweeping of all spans has *started*).
In such case bgsweep may fail wake up runfinq goroutine when it needs to.
finq may still be nil at this point, but some finalizers may be queued later.
Make bgsweep to wait for sweeping to *complete*, then it can decide
whether it needs to wake up runfinq for sure.
Update #7533

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/75960043
2014-03-26 15:11:36 +04:00
Dmitriy Vyukov
40f5e67571 runtime: minor improvement of string scanning
If we set obj, then it will be enqueued for marking at the end of the scanning loop.
This is not necessary, since we've already marked it.
This can wait for 1.4 if you wish.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/80030043
2014-03-26 15:03:58 +04:00
Rob Pike
274fc7600d fmt: document order of application of %T, %p and the special interfaces
Their priority was not documented.

Fixes #7571.

LGTM=adg
R=golang-codereviews, adg
CC=golang-codereviews
https://golang.org/cl/80360043
2014-03-26 17:18:24 +11:00
Mikio Hara
4f1aecf2c4 net: deflake TestTCPConcurrentAccept
Some platform that implements inp_localgroup-like shared internet
protocol control block group looks a bit sensitive about transport
layer protocol's address:port reuse. Sometimes it rejects a TCP SYN
packet using TCP RST, and sometimes silence.

For now, until test case refactoring, we admit few Dial failures on
TestTCPConcurrentAccept as a workaround.

Update #7400
Update #7541

LGTM=jsing
R=jsing
CC=golang-codereviews
https://golang.org/cl/75920043
2014-03-25 02:56:37 +09:00
Mikio Hara
d1e3ad8bc1 net: avoid multiple calling of syscall connect on Unix variants
The previous fix CL 69340044 still leaves a possibility of it.
This CL prevents the kernel, especially DragonFly BSD, from
performing unpredictable asynchronous connection establishment
on stream-based transport layer protocol sockets.

Update #7541
Update #7474

LGTM=jsing
R=jsing
CC=golang-codereviews
https://golang.org/cl/75930043
2014-03-25 02:56:10 +09:00
Mikio Hara
4ffa021965 runtime: slience warning on 32-bit platforms
LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/80250043
2014-03-26 10:21:22 +09:00
Alex Brainman
a837347dd9 runtime: use VEH for windows/amd64 exception handling
Fixes windows/amd64 build.

LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/79470046
2014-03-26 11:13:50 +11:00
Brad Fitzpatrick
aa2d4dee60 net/http: disable recently-introduced flaky test on Windows
Disable it until it's debugged so it doesn't hide other real
problems on Windows. The test was known to be unreliable
anyway (which is why it only needed 1 of 20 runs to pass), but
apparently it never passes on Windows. Figure out why later.

Update #7634

LGTM=alex.brainman
R=adg, alex.brainman
CC=golang-codereviews
https://golang.org/cl/80110043
2014-03-25 15:19:58 -07:00
Keith Randall
fff63c2448 runtime: WriteHeapDump dumps the heap to a file.
See http://golang.org/s/go13heapdump for the file format.

LGTM=rsc
R=rsc, bradfitz, dvyukov, khr
CC=golang-codereviews
https://golang.org/cl/37540043
2014-03-25 15:09:49 -07:00
Ian Lance Taylor
bee3848f4e runtime: fix windows build (buggy commit in 19543:d68b79ccbfed)
TBR=rsc
CC=golang-codereviews
https://golang.org/cl/80090043
2014-03-25 14:17:00 -07:00
Keith Randall
1b45cc45e3 runtime: redo stack map entries to avoid false retention
Change two-bit stack map entries to encode:
0 = dead
1 = scalar
2 = pointer
3 = multiword

If multiword, the two-bit entry for the following word encodes:
0 = string
1 = slice
2 = iface
3 = eface

That way, during stack scanning we can check if a string
is zero length or a slice has zero capacity.  We can avoid
following the contained pointer in those cases.  It is safe
to do so because it can never be dereferenced, and it is
desirable to do so because it may cause false retention
of the following block in memory.

Slice feature turned off until issue 7564 is fixed.

Update #7549

LGTM=rsc
R=golang-codereviews, bradfitz, rsc
CC=golang-codereviews
https://golang.org/cl/76380043
2014-03-25 14:11:34 -07:00
Ian Lance Taylor
4ebfa83199 runtime: accurately record whether heap memory is reserved
The existing code did not have a clear notion of whether
memory has been actually reserved.  It checked based on
whether in 32-bit mode or 64-bit mode and (on GNU/Linux) the
requested address, but it confused the requested address and
the returned address.

LGTM=rsc
R=rsc, dvyukov
CC=golang-codereviews, michael.hudson
https://golang.org/cl/79610043
2014-03-25 13:22:19 -07:00
Brad Fitzpatrick
cc2c5fc3d2 net/http: don't re-use Transport connections if we've seen an EOF
This the second part of making persistent HTTPS connections to
certain servers (notably Amazon) robust.

See the story in part 1: https://golang.org/cl/76400046/

This is the http Transport change that notes whether our
net.Conn.Read has ever seen an EOF. If it has, then we use
that as an additional signal to not re-use that connection (in
addition to the HTTP response headers)

Fixes #3514

LGTM=rsc
R=agl, rsc
CC=golang-codereviews
https://golang.org/cl/79240044
2014-03-25 10:59:09 -07:00
Brad Fitzpatrick
f61f18d694 crypto/tls: make Conn.Read return (n, io.EOF) when EOF is next in buffer
Update #3514

An io.Reader is permitted to return either (n, nil)
or (n, io.EOF) on EOF or other error.

The tls package previously always returned (n, nil) for a read
of size n if n bytes were available, not surfacing errors at
the same time.

Amazon's HTTPS frontends like to hang up on clients without
sending the appropriate HTTP headers. (In their defense,
they're allowed to hang up any time, but generally a server
hangs up after a bit of inactivity, not immediately.) In any
case, the Go HTTP client tries to re-use connections by
looking at whether the response headers say to keep the
connection open, and because the connection looks okay, under
heavy load it's possible we'll reuse it immediately, writing
the next request, just as the Transport's always-reading
goroutine returns from tls.Conn.Read and sees (0, io.EOF).

But because Amazon does send an AlertCloseNotify record before
it hangs up on us, and the tls package does its own internal
buffering (up to 1024 bytes) of pending data, we have the
AlertCloseNotify in an unread buffer when our Conn.Read (to
the HTTP Transport code) reads its final bit of data in the
HTTP response body.

This change makes that final Read return (n, io.EOF) when
an AlertCloseNotify record is buffered right after, if we'd
otherwise return (n, nil).

A dependent change in the HTTP code then notes whether a
client connection has seen an io.EOF and uses that as an
additional signal to not reuse a HTTPS connection. With both
changes, the majority of Amazon request failures go
away. Without either one, 10-20 goroutines hitting the S3 API
leads to such an error rate that empirically up to 5 retries
are needed to complete an API call.

LGTM=agl, rsc
R=agl, rsc
CC=golang-codereviews
https://golang.org/cl/76400046
2014-03-25 10:58:35 -07:00
Ian Lance Taylor
8de04c78b7 runtime: change nproc local variable to uint32
The nproc and ndone fields are uint32.  This makes the type
consistent.

LGTM=minux.ma
R=golang-codereviews, minux.ma
CC=golang-codereviews
https://golang.org/cl/79340044
2014-03-25 05:18:08 -07:00
Nigel Tao
50ca1a52ca database/sql: add "defer rows.Close()" to the example code.
Strictly speaking, it's not necessary in example_test.go, as the
Rows.Close docs say that "If Next returns false, the Rows are closed
automatically". However, if the for loop breaks or returns early, it's
not obvious that you'll leak unless you explicitly call Rows.Close.

LGTM=bradfitz
R=bradfitz
CC=golang-codereviews, rsc
https://golang.org/cl/79330043
2014-03-25 13:32:18 +11:00
Russ Cox
3750904a7e runtime: use VEH, not SEH, for windows/386 exception handling
Structured Exception Handling (SEH) was the first way to handle
exceptions (memory faults, divides by zero) on Windows.
The S might as well stand for "stack-based": the implementation
interprets stack addresses in a few different ways, and it gets
subtly confused by Go's management of stacks. It's also something
that requires active maintenance during cgo switches, and we've
had bugs in that maintenance in the past.

We have recently come to believe that SEH cannot work with
Go's stack usage. See http://golang.org/issue/7325 for details.

Vectored Exception Handling (VEH) is more like a Unix signal
handler: you set it once for the whole process and forget about it.

This CL drops all the SEH code and replaces it with VEH code.
Many special cases and 7 #ifdefs disappear.

VEH was introduced in Windows XP, so Go on windows/386 will
now require Windows XP or later. The previous requirement was
Windows 2000 or later. Windows 2000 immediately preceded
Windows XP, so Windows 2000 is the only affected version.
Microsoft stopped supporting Windows 2000 in 2010.
See http://golang.org/s/win2000-golang-nuts for details.

Fixes #7325.

LGTM=alex.brainman, r
R=golang-codereviews, alex.brainman, stephen.gutekanst, dave
CC=golang-codereviews, iant, r
https://golang.org/cl/74790043
2014-03-24 21:22:16 -04:00
Russ Cox
3b27343c14 time: add comment explaining rejection of years outside [0000,9999]
This has come up twice now. Redirect future questions
to the explanation in the issue tracker.

LGTM=iant, r
R=r, iant
CC=golang-codereviews
https://golang.org/cl/79550043
2014-03-24 20:34:17 -04:00
Rob Pike
a9014ba415 math/cmplx: define Pow(0, x) for problematic values of x.
Currently it's always zero, but that is inconsistent with math.Pow
and also plain wrong.
This is a proposal for how it should be defined.
Fixes #7583.

LGTM=rsc
R=golang-codereviews, iant, gobot, rsc
CC=golang-codereviews
https://golang.org/cl/76940044
2014-03-25 11:25:20 +11:00
Rob Pike
929ee59fce regexp/syntax: document the upper limit of n in x{n}
Fixes #7252.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/77990044
2014-03-25 11:19:25 +11:00
Rob Pike
c790b029e6 regexp: document that it is linear in the input size.
Fixes #7488.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/78050043
2014-03-25 11:17:30 +11:00
Rui Ueyama
4ffc799295 bufio: fix bug that ReadFrom stops before EOF or error
ReadFrom should not return until it receives a non-nil error
or too many contiguous (0, nil)s from a given reader.
Currently it immediately returns if it receives one (0, nil).
Fixes #7611.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/76400048
2014-03-24 11:48:34 -07:00
Rui Ueyama
de1ea5de86 strings: minor cleanup
bi is a slice and not an array, so bi[:] does not make much sense.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/79280043
2014-03-23 18:58:35 -07:00
Rui Ueyama
446d90d727 unicode/utf8: minor code simplification
It's a little bit waste to check if r is not a surrogate
code point because RuneError is not a surrogate code point.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/79230043
2014-03-23 15:44:29 -07:00
Rui Ueyama
160649ff9a unicode/utf16: remove unnecessary type conversions
LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/79080044
2014-03-23 15:07:26 -07:00
Rui Ueyama
fa445849d1 bufio: fix typo in test
LGTM=dave
R=golang-codereviews, dave
CC=golang-codereviews
https://golang.org/cl/79120043
2014-03-22 17:40:17 -07:00
Rui Ueyama
ed97788af9 regexp: remove unused variables
"min" and "max" in "case '{'" clause are fresh variables.
The variables defined in the outer scope never get value
other than 0.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/78750044
2014-03-22 11:05:40 -07:00
Rui Ueyama
b89a9fff5e text/scanner: handle non-io.EOF errors
Currently Scan ignores an error returned from source if the number
of bytes source has read is 0.

Fixes #7594.

LGTM=gri
R=golang-codereviews, bradfitz, gri
CC=golang-codereviews
https://golang.org/cl/78120043
2014-03-21 17:05:57 -07:00
David du Colombier
f182a6eec8 runtime: skip stack growth test in short mode
LGTM=dvyukov
R=dvyukov
CC=golang-codereviews
https://golang.org/cl/78410043
2014-03-21 18:13:23 +01:00
Adam Langley
050b60a369 encoding/asn1: use GeneralizedTime for times outside the range of UTCTime.
Fixes issue #6976.

LGTM=r
R=golang-codereviews, r
CC=golang-codereviews
https://golang.org/cl/72080044
2014-03-21 11:14:38 -04:00
Rémy Oudompheng
ea7d801130 reflect: correct alignment of call arguments on amd64p32.
Changes adapted from original CL 15680044.

LGTM=iant
R=rsc, iant, dave
CC=golang-codereviews
https://golang.org/cl/76150044
2014-03-20 22:22:07 +01:00
Brad Fitzpatrick
ca2cb5190b crypto/tls: clarify concurrent use of Config
LGTM=r, agl
R=agl, r
CC=golang-codereviews
https://golang.org/cl/77530044
2014-03-20 08:32:06 -07:00
Rui Ueyama
a2770af447 base64: fix bug that decoder fails to detect corruption
Encoding.Decode() failed to detect trailing garbages if input contains "==" followed by garbage smaller than 3 bytes (for example, it failed to detect "x" in "AA==x"). This patch fixes the bug and adds a few tests.

LGTM=nigeltao
R=golang-codereviews, bradfitz, nigeltao
CC=golang-codereviews
https://golang.org/cl/75340044
2014-03-20 16:00:34 +11:00
Rui Ueyama
90a66fa6af base32: remove unnecessary "if" condition
This is a patch to apply the same change as CL 76610045.

LGTM=nigeltao
R=nigeltao
CC=golang-codereviews
https://golang.org/cl/77460044
2014-03-20 15:54:17 +11:00
Rob Pike
b00d967706 fmt: make %F a synonym for %f
Rationale:
        It already is for scanning.
        It is accepted for complexes already, but doesn't work.
        It's analogous to %G and %E.
        C accepts it too, and we try to be roughly compatible.
Fixes #7518.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/77580044
2014-03-20 08:51:06 +11:00
Rui Ueyama
cc4bdf0226 strings, bytes: ReadAt should not mutate receiver
CL 77580046 caused a data race issue with tests that assumes ReadAt
does not mutate receiver. This patch partially revert CL 77580046
to fix it.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/77900043
2014-03-19 12:13:47 -07:00
Rui Ueyama
79b3daa977 unicode: minor cleanup
These test cases are redundant because TestSimpleFold tests
all possible rotations of test data, so no need to add
rotated strings.

Also updated the comment as it's guaranteed that SimpleFold
returns values in increasing order.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/77730043
2014-03-19 10:14:04 -07:00
Rui Ueyama
a509026ff0 strings, bytes: fix Reader.UnreadRune
UnreadRune should return an error if previous operation is not
ReadRune.

Fixes #7579.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/77580046
2014-03-19 09:00:58 -07:00
Dmitriy Vyukov
8a908efd74 runtime: add stack growth tests
Also move generated code into a separate file,
because it's difficult to work with the file otherwise.

LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews
https://golang.org/cl/76080044
2014-03-19 17:22:56 +04:00
Dmitriy Vyukov
1895014257 runtime: fix stack split detection around fork
If runtime_BeforeFork splits stack, it will unsplit it
with spoiled g->stackguard. It leads to check failure in oldstack:

fatal error: stackfree: bad fixed size

runtime stack:
runtime.throw(0xadf3cd)
runtime.stackfree(0xc208040480, 0xfffffffffffff9dd, 0x1b00fa8)
runtime.oldstack()
runtime.lessstack()

goroutine 311 [stack unsplit]:
syscall.forkAndExecInChild(0xc20802eea0, 0xc208192c00, 0x5, 0x5, 0xc208072a80, ...)
syscall.forkExec(0xc20802ed80, 0x54, 0xc2081ccb40, 0x4, 0x4, ...)

Fixes #7567.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews, khr, rsc
https://golang.org/cl/77340045
2014-03-19 17:04:51 +04:00
Rui Ueyama
1a21dbc572 bytes: fix panic in Map
utf8.RuneLen returns -1 for an invalid rune. In that case we
need to extend the internal buffer at least by 3 for \uFFFD.

Fixes #7577.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/77420044
2014-03-18 20:52:58 -07:00
Rob Pike
f34251a91c strconv: CanBackquote should reject \x7F
It's a control character.
Fixes #7565.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/77300043
2014-03-19 10:16:48 +11:00
Brad Fitzpatrick
666f5b4a89 expvar: don't recursively acquire Map.RLock
Fixes #7575

LGTM=iant
R=dvyukov, iant
CC=golang-codereviews
https://golang.org/cl/77540044
2014-03-18 11:38:39 -07:00
Mikio Hara
d7039b71a9 net/http: make use of testing.Errorf instead of testing.Fatalf in non-test function goroutines
See testing.FailNow for further information.
Also avoid nil pointer derefernce in TestTransportMaxPerHostIdleConns.

LGTM=dave
R=golang-codereviews, dave
CC=golang-codereviews
https://golang.org/cl/76470043
2014-03-18 21:12:00 +09:00
Rui Ueyama
41818f8fcc base64: refactor encoder.Write
"nn" can never be zero for any input "p", so no check is needed.
This change should improve readability a bit.

LGTM=nigeltao
R=golang-codereviews, bradfitz, nigeltao
CC=golang-codereviews
https://golang.org/cl/76610045
2014-03-18 16:26:23 +11:00
Rob Pike
0de0de0907 fmt: document GoStringer and explain application of formats to compound objects
%q quotes each element of a string slice; this was never explained in the docs.
Fixes #7015.

LGTM=josharian
R=golang-codereviews, josharian
CC=golang-codereviews
https://golang.org/cl/77140044
2014-03-18 11:25:04 +11:00
Luka Zakrajšek
2701dadbac net/http: Request Body error should not be ignored.
Fixes #7521.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/76320043
2014-03-17 15:52:52 -07:00
Catalin Patulea
a387f91553 net/http/fcgi: fix handling of request ID reuse
Request ID reuse is allowed by the FastCGI spec [1]. In particular nginx uses
the same request ID, 1, for all requests on a given connection. Because
serveRequest does not remove the request from conn.requests, this causes it to
treat the second request as a duplicate and drops the connection immediately
after beginRequest. This manifests with nginx option 'fastcgi_keep_conn on' as
the following message in nginx error log:

2014/03/17 01:39:13 [error] 730#0: *109 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: x.x.x.x, server: example.org, request: "GET / HTTP/1.1", upstream: "fastcgi://127.0.0.1:9001", host: "example.org"

Because handleRecord and serveRequest run in different goroutines, access to
conn.requests must now be synchronized.

[1] http://www.fastcgi.com/drupal/node/6?q=node/22#S3.3

LGTM=bradfitz
R=bradfitz
CC=golang-codereviews
https://golang.org/cl/76800043
2014-03-17 15:47:16 -07:00
Rui Ueyama
01d49dc2dd encoding/hex: add error check for write error.
I believe the original author of this code just forgot to check for error here.

LGTM=bradfitz
R=golang-codereviews, bradfitz
CC=golang-codereviews
https://golang.org/cl/76760043
2014-03-17 12:07:30 -07:00
Mikio Hara
f0433e422b net: make use of testing.Errorf instead of testing.Fatalf in non-test function goroutines
See testing.FailNow for further information.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/75900043
2014-03-15 13:43:02 +09:00
Rémy Oudompheng
6436df0d81 runtime: fix 386 assembly for syscall.naclWrite
It was using the wrong offset and returned random values
making "runoutput" compiler tests crash.

LGTM=iant
R=golang-codereviews, iant
CC=golang-codereviews
https://golang.org/cl/76250043
2014-03-14 22:49:46 +01:00
Rémy Oudompheng
88f84b3e41 liblink, runtime: make nacl/386 binaries valid.
They were rejected by NaCl due to AES instructions and
accesses to %gs:0x8, caused by wrong tlsoffset value.

LGTM=iant
R=rsc, dave, iant
CC=golang-codereviews
https://golang.org/cl/76050044
2014-03-14 21:33:55 +01:00
Dmitriy Vyukov
fed5428c4a runtime: fix another race in bgsweep
It's possible that bgsweep constantly does not catch up for some reason,
in this case runfinq was not woken at all.

R=rsc
CC=golang-codereviews
https://golang.org/cl/75940043
2014-03-14 23:32:12 +04:00
Dmitriy Vyukov
8d321625fd runtime: fix spans corruption
The problem was that spans end up in wrong lists after split
(e.g. in h->busy instead of h->central->empty).
Also the span can be non-swept before split,
I don't know what it can cause, but it's safer to operate on swept spans.
Fixes #7544.

R=golang-codereviews, rsc
CC=golang-codereviews, khr
https://golang.org/cl/76160043
2014-03-14 23:25:48 +04:00
Rémy Oudompheng
f210fd1fa9 cmd/6g, runtime: alignment fixes for amd64p32.
LGTM=rsc
R=rsc, dave, iant, khr
CC=golang-codereviews
https://golang.org/cl/75820044
2014-03-14 19:37:39 +01:00
Dmitriy Vyukov
d8e6881166 runtime: report "out of memory" in efence mode
Currently processes crash with obscure message.
Say that it's "out of memory".

LGTM=rsc
R=golang-codereviews
CC=golang-codereviews, khr, rsc
https://golang.org/cl/75820045
2014-03-14 21:22:03 +04:00
Dmitriy Vyukov
0da73b9f07 runtime: fix a race in bgsweep
See the comment for description.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/75670044
2014-03-14 21:21:44 +04:00
Dmitriy Vyukov
b8d40172ce runtime: do not shrink stacks GOCOPYSTACK=0
LGTM=rsc
R=golang-codereviews
CC=golang-codereviews, khr, rsc
https://golang.org/cl/76070043
2014-03-14 21:11:04 +04:00
Aram Hăvărneanu
199e703083 runtime: fix use after close race in Solaris network poller
The Solaris network poller uses event ports, which are
level-triggered. As such, it has to re-arm itself after each
wakeup. The arming mechanism (which runs in its own thread) raced
with the closing of a file descriptor happening in a different
thread. When a network file descriptor is about to be closed,
the network poller is awaken to give it a chance to remove its
association with the file descriptor. Because the poller always
re-armed itself, it raced with code that closed the descriptor.

This change makes the network poller check before re-arming if
the file descriptor is about to be closed, in which case it will
ignore the re-arming request. It uses the per-PollDesc lock in
order to serialize access to the PollDesc.

This change also adds extensive documentation describing the
Solaris implementation of the network poller.

Fixes #7410.

LGTM=dvyukov, iant
R=golang-codereviews, bradfitz, iant, dvyukov, aram.h, gobot
CC=golang-codereviews
https://golang.org/cl/69190044
2014-03-14 17:53:05 +04:00
Keith Randall
4c6d2d6aa8 debug/macho: handle missing __debug_str section
debug/elf does the same thing, use []byte{} for
any missing sections.

Fixes #7510

LGTM=rsc
R=golang-codereviews, iant
CC=golang-codereviews, rsc
https://golang.org/cl/75230043
2014-03-13 14:04:29 -07:00
Anthony Martin
41aa887be5 runtime: fix signal handling on Plan 9
LGTM=rsc
R=rsc, 0intro, aram, jeremyjackins, iant
CC=golang-codereviews, lucio.dere, minux.ma, paurea, r
https://golang.org/cl/9796043
2014-03-13 09:00:12 -07:00
Dmitriy Vyukov
c115cda22c runtime: improve efence
Mark free memory blocks as unused.
On amd64 it allows the process to eat all 128 GB of heap
without killing the machine.

LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/74070043
2014-03-13 19:04:00 +04:00
Dmitriy Vyukov
e678ab4e37 runtime: detect stack split after fork
This check would allowed to easily prevent issue 7511.
Update #7511

LGTM=rsc
R=rsc, aram
CC=golang-codereviews
https://golang.org/cl/75260043
2014-03-13 17:41:08 +04:00
Dmitriy Vyukov
1569628725 runtime: harden conditions when runtime panics on crash
This is especially important for SetPanicOnCrash,
but also useful for e.g. nil deref in mallocgc.
Panics on such crashes can't lead to anything useful,
only to deadlocks, hangs and obscure crashes.
This is a copy of broken but already LGTMed
https://golang.org/cl/68540043/

TBR=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/75320043
2014-03-13 13:25:59 +04:00
Dmitriy Vyukov
5daffee17f runtime: fix stack size check
When we copy stack, we check only new size of the top segment.
This is incorrect, because we can have other segments below it.

LGTM=khr
R=golang-codereviews, khr
CC=golang-codereviews, rsc
https://golang.org/cl/73980045
2014-03-13 13:16:02 +04:00
Aram Hăvărneanu
28792f5d83 runtime: avoid runtime·cgocall in functions called by forkAndExecInChild
Calling runtime·cgocall could trigger a GC in the child while
gclock was held by the parent.

Fixes #7511

LGTM=bradfitz, dvyukov, dave
R=golang-codereviews, bradfitz, dvyukov, dave
CC=golang-codereviews, rsc
https://golang.org/cl/75210044
2014-03-13 18:26:01 +11:00