1
0
mirror of https://github.com/golang/go synced 2024-11-07 00:36:15 -07:00
Commit Graph

41806 Commits

Author SHA1 Message Date
Austin Clements
62e53b7922 runtime: use signals to preempt Gs for suspendG
This adds support for pausing a running G by sending a signal to its
M.

The main complication is that we want to target a G, but can only send
a signal to an M. Hence, the protocol we use is to simply mark the G
for preemption (which we already do) and send the M a "wake up and
look around" signal. The signal checks if it's running a G with a
preemption request and stops it if so in the same way that stack check
preemptions stop Gs. Since the preemption may fail (the G could be
moved or the signal could arrive at an unsafe point), we keep a count
of the number of received preemption signals. This lets stopG detect
if its request failed and should be retried without an explicit
channel back to suspendG.

For #10958, #24543.

Change-Id: I3e1538d5ea5200aeb434374abb5d5fdc56107e53
Reviewed-on: https://go-review.googlesource.com/c/go/+/201760
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02 21:51:18 +00:00
Austin Clements
d16ec13756 runtime: scan stacks conservatively at async safe points
This adds support for scanning the stack when a goroutine is stopped
at an async safe point. This is not yet lit up because asyncPreempt is
not yet injected, but prepares us for that.

This works by conservatively scanning the registers dumped in the
frame of asyncPreempt and its parent frame, which was stopped at an
asynchronous safe point.

Conservative scanning works by only marking words that are pointers to
valid, allocated heap objects. One complication is pointers to stack
objects. In this case, we can't determine if the stack object is still
"allocated" or if it was freed by an earlier GC. Hence, we need to
propagate the conservative-ness of scanning stack objects: if all
pointers found to a stack object were found via conservative scanning,
then the stack object itself needs to be scanned conservatively, since
its pointers may point to dead objects.

For #10958, #24543.

Change-Id: I7ff84b058c37cde3de8a982da07002eaba126fd6
Reviewed-on: https://go-review.googlesource.com/c/go/+/201761
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02 21:51:16 +00:00
Austin Clements
a3ffb0d9eb runtime: asynchronous preemption function for x86
This adds asynchronous preemption function for amd64 and 386. These
functions spill and restore all register state that can be used by
user Go code.

For the moment we stub out the other arches.

For #10958, #24543.

Change-Id: I6f93fabe9875f4834922a5712362e79045c00aca
Reviewed-on: https://go-review.googlesource.com/c/go/+/201759
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02 21:51:14 +00:00
Austin Clements
2d031dc559 runtime: support for injecting calls at signals on x86
This adds a sigctxt.pushCall method that pushes a call at the signaled
site. We'll use this to inject asynchronous preemptions and in some
places we use it to clean up preparePanic.

For the moment this only works on 386 and amd64. We stub it out on
other platforms and will avoid calling the stubbed version.

For #10958, #24543.

Change-Id: I49e0e853f935d32dd67a70c6cafbae44ee68af8e
Reviewed-on: https://go-review.googlesource.com/c/go/+/201758
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02 21:51:12 +00:00
Austin Clements
61fa79885b cmd/compile: fix missing unsafe-points
Currently, the compiler fails to mark any unsafe-points in the initial
instructions of a function as unsafe points. This happens because
unsafe points are encoded as a stack map index of -2 and the compiler
emits PCDATA instructions when there's a change in the stack map
index, but I had set the initial stack map index to -2. The actual
initial PCDATA value assumed by the PCDATA encoder and the runtime is
-1. Hence, if the first instructions had a stack map index of -2, no
PCDATA was emitted, which cause the runtime to assume the index was -1
instead.

This was particularly problematic in the runtime, where the compiler
was supposed to mark only calls as safe-points and everything else as
unsafe-points. Runtime leaf functions, for example, should have been
marked as entirely unsafe-points, but were instead marked entirely as
safe-points.

Fix this by making the PCDATA instruction generator assume the initial
PCDATA value is -1 instead of -2, so it will emit a PCDATA instruction
right away if the first real instruction is an unsafe-point.

This increases the size of the cmd/go binary by 0.02% since we now
emit slightly more PCDATA than before.

For #10958, #24543.

Change-Id: I92222107f799130072b36d49098d2686f1543699
Reviewed-on: https://go-review.googlesource.com/c/go/+/202084
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02 21:51:09 +00:00
Austin Clements
40b7455877 runtime: add GODEBUG=asyncpreemptoff=1
This doesn't do anything yet, but it will provide a way to disable
non-cooperative preemption.

For #10958, #24543.

Change-Id: Ifdef303f103eabd0922ced8d9bebbd5f0aa2cda4
Reviewed-on: https://go-review.googlesource.com/c/go/+/201757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-02 21:51:07 +00:00
Harshavardhana
bdb5e9d170 net/http/httputil: fix missing Transfer-Encoding header
Current implementation of httputil.DumpRequestOut
incorrectly resets the Request.Body prematurely
before Content-Length/Transfer-Encoding detection
in newTransferWriter()

This fix avoids resetting the Request.Body when
Request.ContentLength is set to '0' by the caller
and Request.Body is set to a custom reader. To allow
newTransferWriter() to treat this situation as
'Transfer-Encoding: chunked'.

Fixes #34504

Change-Id: Ieab6bf876ced28c32c084e0f4c8c4432964181f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/197898
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-02 19:53:12 +00:00
Alberto Donizetti
afe50c1196 doc: update ports list description to reflect current status
This change updates the GOARCH/GOOS discussion at the top of the
"Installing Go from source" document to better reflect the current
status. In particular:

- The GOARCH list now focuses on simply listing the supported
architectures, with no notes about their supposed "maturity", since
the same GOARCH can be mature on a GOOS and not so mature on another.

- Outdated notes about some archs being new and "not well-exercised"
have been removed in favour of a following list of which ports are
first class.

- The list of supported OS has been updated (added: AIX, Illumos),
and sorted in alphabetical order.

- A note about the runtime support being the same for all ARCHS,
"including garbage collection and efficient array slicing and" etc etc
has been removed, since it doesn't seem particularly relevant in a
"install from source" instruction page, and it's likely a leftover
from the time this doc page was the landing place for new people and
it felt the need to "sell" Go.

Updates #27689
Fixes #35009

Change-Id: Ic4eca91dca3135adc7bed4fe00b4f157768f0e81
Reviewed-on: https://go-review.googlesource.com/c/go/+/202197
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-02 19:25:54 +00:00
Keith Randall
35cfe059a1 hash/maphash: move bytes/hash to hash/maphash
Fixes #34778

Change-Id: If8225a7c41cb2af3f67157fb9670eef86272e85e
Reviewed-on: https://go-review.googlesource.com/c/go/+/204997
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-02 18:30:37 +00:00
Katie Hockman
dc0c23ec9d crypto/dsa: change bitwise checks to mod operations
Even though bitwise operations may be slightly more
performant, the readability improvement of a mod
operation is worth the tradeoff.

Change-Id: I352c92ad355c6eb6ef99e3da00e1eff2d2ea5812
Reviewed-on: https://go-review.googlesource.com/c/go/+/204739
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-02 15:46:47 +00:00
Ian Lance Taylor
8de0bb77eb runtime: clear preemptStop in dropm
Updates #10958
Updates #24543
Fixes #35294

Change-Id: I60f024d08451565df6d9751dab9832b50cbf637a
Reviewed-on: https://go-review.googlesource.com/c/go/+/204957
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-02 05:52:33 +00:00
Joshua M. Clulow
971ec8728e runtime: check for events when port_getn fails with ETIME
On illumos systems, and at least historically on Solaris systems, it is
possible for port_getn(3C) calls to return some number of events and
then fail with error ETIME.

Generally we expect this to happen if the caller passes an nget value
larger than 1 and calls with a timeout; if less than the requested
number of events accumulate the system will still return them after
timeout failure so the caller must check the updated nget value in the
ETIME case.  Note that although less likely this can still happen even
when requesting just 1 event, especially with a short timeout value or
on a busy system.

Fixes #35261

Change-Id: I0d83251b69a2fadc64c4e8e280aa596e2e1548ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/204801
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-02 05:36:43 +00:00
Constantin Konstantinidis
1e4a358454 net/http: deflake TestCancelRequestWithChannelBeforeDo_Cancel
Goroutines clean up takes longer when using deprecated CloseNotifier.

Fixes #35122

Change-Id: Id820a3012b5c781ddfb294b38ee3b009624e398c
Reviewed-on: https://go-review.googlesource.com/c/go/+/204661
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-02 03:00:20 +00:00
Ian Lance Taylor
dc39be8b85 runtime: use atomic.Cas to change timerRemoved to timerWaiting
If multiple goroutines call time.(*Timer).Reset then the timer will go
from timerWaiting to timerDeleted to timerModifying to timerModifiedLater.
The timer can be on a different P, meaning that simultaneously cleantimers
could change it from timerDeleted to timerRemoving to timerRemoved.
If Reset sees timerRemoved, it was doing an atomic.Store of timerWaiting,
meaning that it did not necessarily see the other values set in the timer,
so the timer could appear to be in an inconsistent state. Use atomic.Cas
to avoid that possibility.

Updates #6239
Updates #27707
Fixes #35272

Change-Id: I1d59a13dc4f2ff4af110fc6e032c8c9d59cfc270
Reviewed-on: https://go-review.googlesource.com/c/go/+/204717
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-01 22:07:56 +00:00
Cuong Manh Le
130f3c0617 cmd/compile: revert change to "bound" in typecheckarraylit
In CL 204617, I intend to make "bound" parameter to have special meaning
in typecheckarraylit, so we can distinguish between type-checks array
literal and slice literal. But we end up with other solution. The CL was
submitted without reverting the "bound" parameter in case of slice
literal.

Technically, it's not harmful, but causes the code harder to read and maintain.

Change-Id: Ia522ccc9a6b8e25d7eaad4aa4957cb4fa18edc60
Reviewed-on: https://go-review.googlesource.com/c/go/+/204618
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-11-01 22:04:05 +00:00
Domas Tamašauskas
9bde9b4d15 time: document an alternative way of setting the system's time zone
User's program was mutating time.Local variable and crashing
itself as a consequence. Instead of documenting that time.Local
variable should not be mutated, recommended way of setting the
system's time zone has been documented.

Fixes #34814

Change-Id: I7781189855c3bf2ea979dfa07f86c283eed27091
Reviewed-on: https://go-review.googlesource.com/c/go/+/200457
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-01 21:41:41 +00:00
Ian Lance Taylor
4ff45821ae runtime: unlock sched lock when checkdead throws due to a deadlock
I was doing some testing with GODEBUG=schedtrace=1,scheddetail=1 and I
noticed that the program hung after a throw with "all goroutines are
asleep". This is because when doing a throw or fatal panic with schedtrace
the panic code does a final schedtrace, which needs to acquire the
scheduler lock. The checkdead function is always called with the scheduler
lock held. So checkdead would throw with the scheduler lock held, then
the panic code would call schedtrace, which would block trying to acquire
the scheduler lock.

This problem will only happen for people debugging the runtime, but
it's easy to avoid by having checkdead unlock the scheduler lock before
it throws. I only did this for the throws that can happen for a normal
program, not for throws that indicate some corruption in the scheduler data.

Change-Id: Ic62277b3ca6bee6f0fca8d5eb516c59cb67855cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/204778
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-01 21:38:07 +00:00
Michael Anthony Knyszek
08a72c851c runtime: turn off scavenger when there's <1 physical page of work
This change turns off the scavenger if there's less than one physical
page of work to do. If there's less than one phyiscal page of work
today, then the computed time for the work to be done will be zero,
resulting in a floating point division by zero.

This is bad on two accounts. On the one hand it could cause a fault on
some systems. On the other hand, it could cause the pacing computations
done by the scavenger to be nonsense. While this is generally harmless
in the case where there's a very small amount of work to do anyway (the
scavenger might just back off expontentially forever, or do some work
and immediately sleep, because there's not much of it to do), it causes
problems for the deadlock checker. On platforms with a larger physical
page size, such as 64 KiB, we might hit this path in a deadlock
scenario, in which case the deadlock checker will never fire and we'll
just hang.

Specifically, this happens on ppc64 trybot tests, which is where the
issue was discovered.

Fixes #34575.

Change-Id: I8677db539447b2f0e75b8cfcbe33932244e1508c
Reviewed-on: https://go-review.googlesource.com/c/go/+/203517
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-11-01 21:32:33 +00:00
Brad Fitzpatrick
d0160bc32d make.bash: show bootstrap version
For debugging. (The "go1.4" can be misleading since it might actually
be go1.4.3 or go1.11 or go1.12 or master)

Change-Id: I27520b931a2be018de577a299592d082260aa467
Reviewed-on: https://go-review.googlesource.com/c/go/+/204757
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-01 20:45:04 +00:00
Bryan C. Mills
617b4168b8 cmd/go: adjust module-related logging
Suppress “finding” messages unless they are unusually slow, and
“extracting” messages always (they almost always occur conjunction
with “downloading”, which is already logged).

Log “found” messages for module dependencies added to satisfy missing
import paths.

Log top-level version changes in 'go get' when the selected version
is not identical to the version requested on the command line.

Updates #26152
Updates #33284

Change-Id: I4d0de60fab58d7cc7df8a2aff05c8b5b2220e626
Reviewed-on: https://go-review.googlesource.com/c/go/+/204777
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-11-01 20:38:31 +00:00
Lorenz Bauer
c9d89f6bac encoding/binary: cache struct sizes to speed up Read and Write
A majority of work is spent in dataSize when en/decoding the same
struct over and over again. This wastes a lot of work, since
the result doesn't change for a given reflect.Value.

Cache the result of the function for structs, so that subsequent
calls to dataSize can avoid doing work.

    name         old time/op    new time/op     delta
    ReadStruct     1.00µs ± 1%     0.37µs ± 1%   -62.99%  (p=0.029 n=4+4)
    WriteStruct    1.00µs ± 3%     0.37µs ± 1%   -62.69%  (p=0.008 n=5+5)

    name         old speed      new speed       delta
    ReadStruct   75.1MB/s ± 1%  202.9MB/s ± 1%  +170.16%  (p=0.029 n=4+4)
    WriteStruct  74.8MB/s ± 3%  200.4MB/s ± 1%  +167.96%  (p=0.008 n=5+5)

Fixes #34471

Change-Id: Ic5d987ca95f1197415ef93643a0af6fc1224fdf0
Reviewed-on: https://go-review.googlesource.com/c/go/+/199539
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-01 20:16:01 +00:00
Bryan C. Mills
2c8529cb4d cmd/go: default to mod=readonly when the go.mod file is read-only
Updates #30185
Updates #33326
Updates #34822

Change-Id: Ie13651585898d1bbbf4f779b97ee50b6c7e7ad50
Reviewed-on: https://go-review.googlesource.com/c/go/+/204521
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-11-01 19:01:14 +00:00
Tobias Klauser
8d45e612f1 syscall: add support for linux/riscv64
Based on the riscv-go port and the linux/riscv64 files in x/sys/unix.

Updates #27532

Change-Id: Ib33a59a61f6b2721b12292c18f1fc9f9d0509cd3
Reviewed-on: https://go-review.googlesource.com/c/go/+/204659
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-01 16:05:22 +00:00
empijei
a570fcf74b net/http: clarify Header.WriteSubset behavior
The current implementation performs a plain map lookup,
but other header methods canonicalize header keys before
using them.

Fixes #34918

Change-Id: Id4120488b8b39ecee97fa7a6ad8a34158687ffcd
Reviewed-on: https://go-review.googlesource.com/c/go/+/201357
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-01 14:48:28 +00:00
Tobias Klauser
8405cd3005 internal/syscall/unix: add support for linux/riscv64
Based on the riscv-go port.

Updates #27532

Change-Id: I3a4d86783fbd625e3ade16d08f87d66e4502f3f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/204660
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-01 14:19:56 +00:00
Jay Conrod
24e9c53396 cmd/go/internal/modfetch: add zip sums and hashes to TestCodeRepo
Updates #35290

Change-Id: I09cad17f09e78c2bf6a9de98b01f13ed383ca006
Reviewed-on: https://go-review.googlesource.com/c/go/+/204643
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-11-01 13:38:23 +00:00
Tobias Klauser
5b31021525 os: gofmt
Change-Id: Ie76303e403f0539bdfe14f6bb5f32896df916bce
Reviewed-on: https://go-review.googlesource.com/c/go/+/204657
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-01 07:59:09 +00:00
Ian Lance Taylor
e96fd13264 runtime: use correct state machine in addAdjustedTimers
The addAdjustedTimers function was a late addition, and it got some of
the state machine wrong, leading to failures like
https://storage.googleapis.com/go-build-log/930576b6/windows-amd64-2016_53d0319e.log

Updates #6239
Updates #27707

Change-Id: I9e94e563b4698ff3035ce609055ca292b9cab3df
Reviewed-on: https://go-review.googlesource.com/c/go/+/204280
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-01 05:38:51 +00:00
Cuong Manh Le
efd395f9fb cmd/compile: make duplicate index error distinguish arrays and slices
Fixes #35291

Change-Id: I11ae367b6e972cd9e7a22bbc2cb23d32f4d72b98
Reviewed-on: https://go-review.googlesource.com/c/go/+/204617
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-11-01 01:51:26 +00:00
Audrius Butkevicius
4a09a9b054 os: allow case only renames on case-insensitive filesystems
Fixes #35222

Change-Id: I8be45092ac4079d21ff54661637a3aa8ec4eb9bc
GitHub-Last-Rev: 954a016c9b
GitHub-Pull-Request: golang/go#35298
Reviewed-on: https://go-review.googlesource.com/c/go/+/204601
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-01 00:04:51 +00:00
Josh Bleecher Snyder
ce49f9506a cmd/compile: remove amd64p32 rules
And simplify the remaining rules.

Updates #30439

Change-Id: Ib89dce16b17ae881824178346ed6ab895b79627e
Reviewed-on: https://go-review.googlesource.com/c/go/+/204600
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-31 23:37:57 +00:00
Andrew Bonventre
15bff20829 doc: document Go 1.13.4
Change-Id: Ib29e642c56eca96826187f5737d74f8c0430ac3d
Reviewed-on: https://go-review.googlesource.com/c/go/+/204638
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2019-10-31 21:47:51 +00:00
Andrew Bonventre
ef03c4476e doc: document Go 1.12.13
Change-Id: Ic65a74e56320adbd76aeef1cf3b19d7906ffe8fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/204637
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-31 21:47:08 +00:00
Brad Fitzpatrick
d6f2f833c9 strings: remove a redundant and flaky part of a test
Fixes #35280

Change-Id: I6fa3747ff7b92c6fcabdf8692d85e103de55859f
Reviewed-on: https://go-review.googlesource.com/c/go/+/204598
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-31 20:45:40 +00:00
Cuong Manh Le
48c0cef6dc bufio: return the underlying error in ReadFrom if not nil
If a previous Write returned an error, any subsequent Write or ReadFrom
must return that error before any operations.

However, only Write behaved correctly and this change fixes that problem
by making sure that ReadFrom firstly checks for the underlying error.

Fixes #35194

Change-Id: I31356a9e8bd945bc0168b2e3be470f3ae69d4813
Reviewed-on: https://go-review.googlesource.com/c/go/+/204000
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-31 17:21:56 +00:00
Austin Clements
7de15e362b runtime: atomically set span state and use as publication barrier
When everything is working correctly, any pointer the garbage
collector encounters can only point into a fully initialized heap
span, since the span must have been initialized before that pointer
could escape the heap allocator and become visible to the GC.

However, in various cases, we try to be defensive against bad
pointers. In findObject, this is just a sanity check: we never expect
to find a bad pointer, but programming errors can lead to them. In
spanOfHeap, we don't necessarily trust the pointer and we're trying to
check if it really does point to the heap, though it should always
point to something. Conservative scanning takes this to a new level,
since it can only guess that a word may be a pointer and verify this.

In all of these cases, we have a problem that the span lookup and
check can race with span initialization, since the span becomes
visible to lookups before it's fully initialized.

Furthermore, we're about to start initializing the span without the
heap lock held, which is going to introduce races where accesses were
previously protected by the heap lock.

To address this, this CL makes accesses to mspan.state atomic, and
ensures that the span is fully initialized before setting the state to
mSpanInUse. All loads are now atomic, and in any case where we don't
trust the pointer, it first atomically loads the span state and checks
that it's mSpanInUse, after which it will have synchronized with span
initialization and can safely check the other span fields.

For #10958, #24543, but a good fix in general.

Change-Id: I518b7c63555b02064b98aa5f802c92b758fef853
Reviewed-on: https://go-review.googlesource.com/c/go/+/203286
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-31 17:09:50 +00:00
Austin Clements
a9b37ae026 runtime: fully initialize span in alloc_m
Currently, several important fields of a heap span are set by
heapBits.initSpan, which happens after the span has already been
published and returned from the locked region of alloc_m. In
particular, allocBits is set very late, which makes mspan.isFree
unsafe even if you were to lock the heap because it tries to access
allocBits.

This CL fixes this by populating these fields in alloc_m. The next CL
builds on this to only publish the span once it is fully initialized.
Together, they'll make it safe to check allocBits even if there is a
race with alloc_m.

For #10958, #24543, but a good fix in general.

Change-Id: I7fde90023af0f497e826b637efa4d19c32840c08
Reviewed-on: https://go-review.googlesource.com/c/go/+/203285
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-31 17:09:48 +00:00
Cuong Manh Le
d5caea771e bufio: save error from underlying ReadFrom
Updates #35194

Change-Id: Ib854bc6250ddeb606d6ff6240179e23b98e4ac62
Reviewed-on: https://go-review.googlesource.com/c/go/+/203999
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-31 16:02:25 +00:00
Olivier Poitrey
02a5502ab8 crypto/tls: send ec_points_format extension in ServerHello
Follow the recommandation from RFC 8422, section 5.1.2 of sending back the
ec_points_format extension when requested by the client. This is to fix
some clients declining the handshake if omitted.

Fixes #31943

Change-Id: I7b04dbac6f9af75cda094073defe081e1e9a295d
Reviewed-on: https://go-review.googlesource.com/c/go/+/176418
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Olivier Poitrey <rs@rhapsodyk.net>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-31 15:14:45 +00:00
Daniel Martí
54697702e4 encoding/json: avoid work when unquoting strings, take 2
This is a re-submission of CL 151157, since it was reverted in CL 190909
due to an introduced crash found by a fuzzer. The revert CL included
regression tests, while this CL includes a fixed version of the original
change.

In particular, what we forgot in the original optimization was that we
still need the length and trailing quote checks at the beginning of
unquoteBytes. Without those, we could end up in a crash later on.

We can work out how many bytes can be unquoted trivially in
rescanLiteral, which already iterates over a string's bytes.

Removing the extra loop in unquoteBytes simplifies the function and
speeds it up, especially when decoding simple strings, which are common.

While at it, we can remove the check that s[0]=='"', since all call
sites already meet that condition.

name           old time/op    new time/op    delta
CodeDecoder-8    10.6ms ± 2%    10.5ms ± 1%  -1.01%  (p=0.004 n=20+10)

name           old speed      new speed      delta
CodeDecoder-8   183MB/s ± 2%   185MB/s ± 1%  +1.02%  (p=0.003 n=20+10)

Updates #28923.

Change-Id: I8c6b13302bcd86a364bc998d72451332c0809cde
Reviewed-on: https://go-review.googlesource.com/c/go/+/190659
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Peter Weinberger <pjw@google.com>
2019-10-31 12:02:57 +00:00
Michael Munday
10855608bc runtime/internal/atomic: add tests for And8 and Or8
Add some simple unit tests for these atomic operations. These can't
catch all the bugs that are possible with these operations but at
least they provide some coverage.

Change-Id: I94b9f451fcc9fecdb2a1448c5357b019563ad275
Reviewed-on: https://go-review.googlesource.com/c/go/+/204317
Run-TryBot: Michael Munday <mike.munday@ibm.com>
Reviewed-by: Austin Clements <austin@google.com>
2019-10-31 09:49:57 +00:00
Ian Lance Taylor
6becb03334 runtime: switch to using new timer code
No big changes in the runtime package benchmarks.

Changes in the time package benchmarks:

name                      old time/op  new time/op  delta
AfterFunc-12              1.57ms ± 1%  0.07ms ± 1%  -95.42%  (p=0.000 n=10+8)
After-12                  1.63ms ± 3%  0.11ms ± 1%  -93.54%  (p=0.000 n=9+10)
Stop-12                   78.3µs ± 3%  73.6µs ± 3%   -6.01%  (p=0.000 n=9+10)
SimultaneousAfterFunc-12   138µs ± 1%   111µs ± 1%  -19.57%  (p=0.000 n=10+9)
StartStop-12              28.7µs ± 1%  31.5µs ± 5%   +9.64%  (p=0.000 n=10+7)
Reset-12                  6.78µs ± 1%  4.24µs ± 7%  -37.45%  (p=0.000 n=9+10)
Sleep-12                   183µs ± 1%   125µs ± 1%  -31.67%  (p=0.000 n=10+9)
Ticker-12                 5.40ms ± 2%  0.03ms ± 1%  -99.43%  (p=0.000 n=10+10)
Sub-12                     114ns ± 1%   113ns ± 3%     ~     (p=0.069 n=9+10)
Now-12                    37.2ns ± 1%  36.8ns ± 3%     ~     (p=0.287 n=8+8)
NowUnixNano-12            38.1ns ± 2%  37.4ns ± 3%   -1.87%  (p=0.020 n=10+9)
Format-12                  252ns ± 2%   195ns ± 3%  -22.61%  (p=0.000 n=9+10)
FormatNow-12               234ns ± 1%   177ns ± 2%  -24.34%  (p=0.000 n=10+10)
MarshalJSON-12             320ns ± 2%   250ns ± 0%  -21.94%  (p=0.000 n=8+8)
MarshalText-12             320ns ± 2%   245ns ± 2%  -23.30%  (p=0.000 n=9+10)
Parse-12                   206ns ± 2%   208ns ± 4%     ~     (p=0.084 n=10+10)
ParseDuration-12          89.1ns ± 1%  86.6ns ± 3%   -2.78%  (p=0.000 n=10+10)
Hour-12                   4.43ns ± 2%  4.46ns ± 1%     ~     (p=0.324 n=10+8)
Second-12                 4.47ns ± 1%  4.40ns ± 3%     ~     (p=0.145 n=9+10)
Year-12                   14.6ns ± 1%  14.7ns ± 2%     ~     (p=0.112 n=9+9)
Day-12                    20.1ns ± 3%  20.2ns ± 1%     ~     (p=0.404 n=10+9)

Updates #6239
Updates #27707

Change-Id: I51e25a90f941574f1a9cf83a22e84ac8c678537d
Reviewed-on: https://go-review.googlesource.com/c/go/+/171883
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-10-30 21:43:57 +00:00
Damien Neil
81a74b4e8d testing: provide additional information when test funcs panic
Flush the output log up to the root when a test panics. Prior to
this change, only the current test's output log was flushed to its
parent, resulting in no output when a subtest panics.

For the following test function:

	func Test(t *testing.T) {
		for i, test := range []int{1, 0, 2} {
			t.Run(fmt.Sprintf("%v/%v", i, test), func(t *testing.T) {
				_ = 1 / test
			})
		}
	}

Output before this change:

	panic: runtime error: integer divide by zero [recovered]
		panic: runtime error: integer divide by zero
	(stack trace follows)

Output after this change:

	--- FAIL: Test (0.00s)
	    --- FAIL: Test/1/0 (0.00s)
	panic: runtime error: integer divide by zero [recovered]
	(stack trace follows)

Fixes #32121

Change-Id: Ifee07ccc005f0493a902190a8be734943123b6b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/179599
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-30 20:46:44 +00:00
Filippo Valsorda
cd18da451f crypto/tls: improve error messages for invalid certificates and signatures
Also, fix the alert value sent when a signature by a client certificate
is invalid in TLS 1.0-1.2.

Fixes #35190

Change-Id: I2ae1d5593dfd5ee2b4d979664aec74aab4a8a704
Reviewed-on: https://go-review.googlesource.com/c/go/+/204157
Reviewed-by: Katie Hockman <katie@golang.org>
2019-10-30 20:18:59 +00:00
Sam Whited
a05934639b encoding/xml: fix token decoder on early EOF
The documentation for TokenReader suggests that implementations of the
interface may return a token and io.EOF together, indicating that it is
the last token in the stream. This is similar to io.Reader. However, if
you wrap such a TokenReader in a Decoder it complained about the EOF.
A test was added to ensure this behavior on Decoder's.

Change-Id: I9083c91d9626180d3bcf5c069a017050f3c7c4a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/130556
Run-TryBot: Sam Whited <sam@samwhited.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-30 19:47:52 +00:00
Ian Lance Taylor
cf6e6abc68 runtime: clear js idle timeout before new one and after event handler
Updates #6239
Updates #27707

Change-Id: I0a62c1374db485dd830bf02e59625997d9247fc3
Reviewed-on: https://go-review.googlesource.com/c/go/+/203890
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-30 16:37:41 +00:00
Clément Chigot
cc4b824e53 runtime: fix nbpipe_test for AIX
Fcntl can't be called using syscall.Syscall as it doesn't work on AIX.
Moreover, fcntl isn't exported by syscall package.

However, it can be accessed by exporting it from runtime package
using export_aix_test.go.

Change-Id: Ib6af66d9d7eacb9ca0525ebc4cd4c92951735f1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/204059
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-30 14:51:03 +00:00
Clément Chigot
301bc66a47 runtime: fix netpollBreak for AIX
Change-Id: I2629711ce02d935130fb2aab29f9028b62ba9fe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/204318
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-30 14:15:21 +00:00
Lorenz Bauer
17190de085 syscall: treat ENFILE as a temporary error
ENFILE is returned from accept when the whole system has run out of
file descriptors. Mark the error as temporary, so that accept loops
continue working.

Fixes #35131
Updates #1891

Change-Id: Idf44c084731898ff4c720d06c250d3b8a42de312
Reviewed-on: https://go-review.googlesource.com/c/go/+/203117
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-30 14:06:04 +00:00
Elias Naur
f4e32aeed1 cmd/link/internal/ld: remove flags incompatible with -fembed-bitcode
The flags -headerpad, -Wl,-no_pie and -pagezero_size are incompatible with
the -fembed-bitcode flag used by `gomobile build`. Than McIntosh
suggested we might not need the offending flags; this change removes
the flags on darwin/arm64 and -headerpad, -pagezero_size on darwin/arm.

The -Wl,-no_pie flag is left for darwin/arm because linking fails
without it:

ld: warning: PIE disabled. Absolute addressing (perhaps -mdynamic-no-pic) not allowed in code signed PIE, but used in _runtime.rodata from /var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/tmp/go-link-225285265/go.o. To fix this warning, don't compile with -mdynamic-no-pic or link with -Wl,-no_pie

Discussion: https://groups.google.com/d/msg/golang-dev/U1jK3xmmGAk/j0_ty46EDAAJ

I've verified the CL on the builders, built the "flappy" example from
gomobile with `gomobile build`, and verified that flappy runs on an
iPhone 5S.

Updates #32963

Change-Id: I783abc93ccf3c1d2b7ca00144b7164ba223d3529
Reviewed-on: https://go-review.googlesource.com/c/go/+/201358
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-30 08:17:29 +00:00