1
0
mirror of https://github.com/golang/go synced 2024-11-12 00:20:22 -07:00
Commit Graph

41765 Commits

Author SHA1 Message Date
Bryan C. Mills
649f341e95 cmd: update x/mod to CL 205497
Also revert an incidental 'gofmt' of a vendored file from CL 205240.

Updates #34822

Change-Id: I82a015d865db4d865b4776a8013312f25dbb9181
Reviewed-on: https://go-review.googlesource.com/c/go/+/205539
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-05 21:26:19 +00:00
Bryan C. Mills
14a133fe77 cmd/go/internal/modfetch: prune +incompatible versions more aggressively
codeRepo.Versions previously checked every possible +incompatible
version for a 'go.mod' file. That is wasteful and counterproductive.

It is wasteful because typically, a project will adopt modules at some
major version, after which they will (be required to) use semantic
import paths for future major versions.

It is counterproductive because it causes an accidental
'+incompatible' tag to exist, and no compatible tag can have higher
semantic precedence.

This change prunes out some of the +incompatible versions in
codeRepo.Versions, eliminating the “wasteful” part but not all of the
“counterproductive” part: the extraneous versions can still be fetched
explicitly, and proxies may include them in the @v/list endpoint.

Updates #34165
Updates #34189
Updates #34533

Change-Id: Ifc52c725aa396f7fde2afc727d0d5950acd06946
Reviewed-on: https://go-review.googlesource.com/c/go/+/204439
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-11-05 21:16:50 +00:00
Filippo Valsorda
81559af51a doc: mention the anti-spam bypass in security.html
We had some issues with reports being marked as spam, so I added a
filter to never mark as spam something that mentions the word
"vulnerability". We get too much spam at that address to disable the
filter entirely, so instead meantion the bypass in the docs.

Change-Id: Idb4dabcf51a9dd8234a2d571cd020c970b0a582c
Reviewed-on: https://go-review.googlesource.com/c/go/+/205538
Reviewed-by: Katie Hockman <katie@golang.org>
2019-11-05 20:56:05 +00:00
Ian Lance Taylor
fb37821014 runtime/cgo: add -Wno-nullability-completeness on Darwin
Fixes #35247

Change-Id: I4f2e243c89e9f745b82bcd181add87fad1443171
Reviewed-on: https://go-review.googlesource.com/c/go/+/205457
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2019-11-05 20:47:22 +00:00
Katie Hockman
552987fdbf crypto/dsa: prevent bad public keys from causing panic
dsa.Verify might currently use a nil s inverse in a
multiplication if the public key contains a non-prime Q,
causing a panic. Change this to check that the mod
inverse exists before using it.

Fixes CVE-2019-17596

Fixes #34960

Change-Id: I94d5f3cc38f1b5d52d38dcb1d253c71b7fd1cae7
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/572809
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/205441
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2019-11-05 20:21:34 +00:00
Bryan C. Mills
7e71c9c3ed cmd/go/internal/modget: synchronize writes to modOnly map in runGet
Fixes #35317

Change-Id: Id858a25dc16a1bbff1802d25bcd4aca31c1133bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/205067
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-11-05 19:06:28 +00:00
Cherry Zhang
f07cbc7f88 runtime: don't fetch G from signal stack when using cgo
When using cgo, we save G to TLS, and when a signal happens, we
load G from TLS in sigtramp. This should give us a valid G. Don't
try to fetch from the signal stack. In particular, C code may
change the signal stack or call our signal handler directly (e.g.
TSAN), so we are not necessarily running on the original gsignal
stack where we saved G.

Also skip saving G on the signal stack when using cgo.

Updates #35249.

Change-Id: I40749ce6682709bd4ebfdfd9f23bd0f317fc197d
Reviewed-on: https://go-review.googlesource.com/c/go/+/204519
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-05 18:58:41 +00:00
Cherry Zhang
fb05264fe1 runtime: setg after sigFetchG
In the normal case, sigFetchG just returns the G register. But in
the case that sigFetchG fetches the G from somewhere else, the G
register still holding an invalid value. Setg here to make sure
they match.

This is particularly useful because setGsignalStack, called by
adjustSignalStack from sigtrampgo before setg to gsignal,
accesses the G register.

Should fix #35249.

Change-Id: I64c85143cb05cdb2ecca7f9936dbd8bfec186c2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/204441
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-05 18:58:29 +00:00
Ian Lance Taylor
c3cef0bfe5 runtime: keep adjusted timers in timerMoving status until moved
Before this CL adjustTimers left timers being moved in an inconsistent
state: status timerWaiting but not on a P. Simplify the code by
leaving the timers in timerMoving status until they are actually moved.
Other functions (deltimer, modtimer) will wait until the move is complete
before changing anything on the timer. This does leave timers in timerMoving
state for longer, but still not all that long.

Fixes #35367

Change-Id: I31851002fb4053bd6914139125b4c82a68bf6fb2
Reviewed-on: https://go-review.googlesource.com/c/go/+/205418
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-05 18:37:06 +00:00
Bryan C. Mills
a9c0cc6f67 cmd/go: make 'go mod why' require a main module
Updates #32027

Change-Id: Ifc9427f35188c3fd356917d8510f3e01866ebca8
Reviewed-on: https://go-review.googlesource.com/c/go/+/205065
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-11-05 18:03:54 +00:00
Bryan C. Mills
95149147d8 cmd/compile: do not skip TestFormats on short builders
TestFormats adds ~3s of running time to the test, which may be
slightly annoying in an edit/compile/test cycle but is negligible in a
TryBot run.

The test keeps regressing in the longtest builders, requiring a manual
fix. Instead, run it even in short mode on the builders, so that
TryBot runs will detect regressions ahead of time.

Updates #34907
Updates #33915
Updates #28621

Change-Id: I6f9bf0f2ca929a743438310b86d85d8673c720bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/205440
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-05 18:02:45 +00:00
Pantelis Sampaziotis
0994cc1af6 testing/iotest: add missing tests
Adds missing tests for all the types:
* OneByteReader
* HalfReader
* DataErrReader
* TimeoutReader
* TruncateWriter
* writeLogger
* readLogger

Fixes #33650

Change-Id: I1c773f9f1625ff33a1d0b5a045c72a73a9eca9ce
GitHub-Last-Rev: 2ab650677b
GitHub-Pull-Request: golang/go#33651
Reviewed-on: https://go-review.googlesource.com/c/go/+/190259
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-05 17:57:10 +00:00
Dan Scales
1b3a1db19f cmd/compile: fix liveness for open-coded defer args for infinite loops
Once defined, a stack slot holding an open-coded defer arg should always be marked
live, since it may be used at any time if there is a panic. These stack slots are
typically kept live naturally by the open-defer code inlined at each return/exit point.
However, we need to do extra work to make sure that they are kept live if a
function has an infinite loop or a panic exit.

For this fix, only in the case of a function that is using open-coded defers, we
compute the set of blocks (most often empty) that cannot reach a return or a
BlockExit (panic) because of an infinite loop. Then, for each block b which
cannot reach a return or BlockExit or is a BlockExit block, we mark each defer arg
slot as live, as long as the definition of the defer arg slot dominates block b.

For this change, had to export (*Func).sdom (-> Sdom) and SparseTree.isAncestorEq
(-> IsAncestorEq)

Updates #35277

Change-Id: I7b53c9bd38ba384a3794386dd0eb94e4cbde4eb1
Reviewed-on: https://go-review.googlesource.com/c/go/+/204802
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-05 17:19:16 +00:00
Bryan C. Mills
414c1d454e cmd/go: derive TestExecutableGOROOT environment from tg.env instead of os.Environ()
TestExecutableGOROOT, unlike most other tests in go_test.go, was
running subcommands in a process with an environment derived directly
from os.Environ(), rather than using tg.env on its testgoData object.

Since tg.env is what sets GO111MODULE=off for GOPATH-mode tests, that
caused TestExecutableGOROOT to unexpectedly run in module mode instead
of GOPATH mode. If the user's environment included 'GOFLAGS=-mod=mod',
that would cause the test to spuriously fail due to the inability to
download modules to $HOME (which in this test binary is hard-coded to
"/test-go-home-does-not-exist").

Updates #33848

Change-Id: I2f343008dd9e38cd76b9919eafd5a3181d0cbd6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/205064
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-11-05 16:31:48 +00:00
Bryan C. Mills
f0390ffc9d cmd/go/internal/modfetch: remove non-hermetic test
The test for gopkg.in/yaml.v2@v2 assumes that there are
no future upstream releases. That assumption empirically
does not hold. Backporting fixes to this test is annoying,
and other gopkg.in cases are already reasonably covered,
so remove the problematic test.

Updates #28856

Change-Id: I6455baa1816ac69e02d1ad5d03b82a93e1481a17
Reviewed-on: https://go-review.googlesource.com/c/go/+/205437
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-05 16:29:31 +00:00
Bryan C. Mills
8550a58402 cmd/go.mod: upgrade x/mod to pull in CL 204821
Updates #34822

Change-Id: I189d93ebd3ce6cd1b8f1e29336876fd82a7cfff7
Reviewed-on: https://go-review.googlesource.com/c/go/+/204877
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-05 15:40:02 +00:00
Elias Naur
e457cc38af misc/ios: bump -mios-version-min
Recent Xcode versions started to complain about the current min
version:

ld: warning: OS version (6.0.0) too small, changing to 7.0.0

Change-Id: Ieb525dd3e57429fe226b9d30d584b073c5e4768c
Reviewed-on: https://go-review.googlesource.com/c/go/+/204663
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-05 14:44:56 +00:00
Marcel van Lohuizen
79a00a3fe8 unicode: upgrade to Unicode 12
This does not include an upgrade of golang.org/x/net.
This is optional and best done as a separate CL.

Change-Id: Ifecc3fb6e3b7fe026b4ddefbe637186a3445b0bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/204658
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2019-11-05 14:34:56 +00:00
Ian Lance Taylor
3c0fbeea7d runtime: disable preemption during test calls to futexsleep
Fixes #35347

Change-Id: If7380f29e97a5abe86cdd5e2853323de7997ccfc
Reviewed-on: https://go-review.googlesource.com/c/go/+/205378
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-05 05:22:07 +00:00
Ian Lance Taylor
d51f7f3970 cmd/compile: update TestFormats for CL 196959
CL 196959 uses %v to print *EscLocation values. This happens at least at
    Fatalf("path inconsistency: %v != %v", edge.src, src)
in (*Escape).explainPath.

Change-Id: I1c761406af6a1025403dfefa5ec40aee75e72944
Reviewed-on: https://go-review.googlesource.com/c/go/+/205377
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-05 05:19:08 +00:00
Lars Lehtonen
979d65dbc7 net/smtp: fix dropped test error
Pick up a dropped error in TestSendMailWithAuth() and simplify goroutine
to use an error channel instead of a sync.WaitGroup and an empty struct
doneCh.

Change-Id: Ie70d0f7c4c85835eb682e81d086ce4d9900269e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/205247
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-05 03:50:54 +00:00
Cherry Zhang
9a0a824456 cmd/internal/obj/mips: add a flag field to Optab
The flag field will be used for marking unsafe points. This CL
just adds the field, not doing anything with it. The next CL will
make use of it. This is for making the diff simpler.

Change-Id: I6ff5406ba2e53ae8a882184733d88482a2ca8e2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/203938
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-11-05 03:42:29 +00:00
Cherry Zhang
11db7e4469 runtime: test a frameless function for async preemption
Frameless function is an interesting case for call injection
espcially for LR architectures. Extend the test for this case.

Change-Id: I074090d09eeaf642e71e3f44fea216f66d39b817
Reviewed-on: https://go-review.googlesource.com/c/go/+/202339
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-11-05 03:42:00 +00:00
Lars Lehtonen
a866b48e4a io/ioutil: Fix dropped test error
Change-Id: I9cfaba4f1af23ab67627bf496739311e4d1984c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/205245
Reviewed-by: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-05 03:27:26 +00:00
Cherry Zhang
4a7ed1fab7 cmd/compile: mark architecture-specific unsafe points
Introduce a mechanism for marking architecture-specific Ops
unsafe. And mark ones that use REGTMP on ARM64, as for async
preemption we will be using REGTMP as a temporary register in the
injected call.

Change-Id: I8ff22e87d8f9cb10d02a2f0af7c12ad6d7d58f54
Reviewed-on: https://go-review.googlesource.com/c/go/+/203459
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2019-11-05 02:55:11 +00:00
Cherry Zhang
0f992b9948 cmd/compile: not use REGTMP in ZeroRange on ARM64
For async preemption, we will be using REGTMP as a temporary
register in injected call on ARM64, which will clobber it. So any
code that uses REGTMP is not safe for async preemption.

For ZeroRange, which is inserted at the function entry where
there is no register live, we could just use a different register
and avoid REGTMP.

Change-Id: I3db763828df6846908c9843a9912597efb9efcdf
Reviewed-on: https://go-review.googlesource.com/c/go/+/203458
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-11-05 02:54:42 +00:00
Cherry Zhang
2ff746d7dc runtime: add async preemption support on ARM
This CL adds support of call injection and async preemption on
ARM.

Injected call, like sigpanic, has special frame layout. Teach
traceback to handle it.

Change-Id: I887e90134fbf8a676b73c26321c50b3c4762dba4
Reviewed-on: https://go-review.googlesource.com/c/go/+/202338
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2019-11-05 02:49:48 +00:00
Keith Randall
376c4ceab7 cmd/vet: add test for loading complex values with a single instruction
Actual fix will be submitted to x/tools and vendored.
This is just an end-to-end test for vet after that is done.

Update #35264

Change-Id: I1a63f607e7cfa7aafee23c2c081086c276d3c38c
Reviewed-on: https://go-review.googlesource.com/c/go/+/204538
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
2019-11-05 01:01:31 +00:00
Matthew Dempsky
b3bd7ab3d7 cmd/compile: fix //go:uintptrescapes for basic method calls
The logic for keeping arguments alive for calls to //go:uintptrescapes
functions was only applying to direct function calls. This CL changes
it to also apply to direct method calls, which should address most
uses of Proc.Call and LazyProc.Call.

It's still an open question (#34684) whether other call forms (e.g.,
method expressions, or indirect calls via function values, method
values, or interfaces).

Fixes #34474.

Change-Id: I874f97145972b0e237a4c9e8926156298f4d6ce0
Reviewed-on: https://go-review.googlesource.com/c/go/+/198043
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-05 00:26:30 +00:00
Ian Lance Taylor
6cbd737c8e runtime/pprof: update example comment to check error on Close
Programs should always check the error return of Close for a file opened
for writing. Update the example code in the comment to mention this.

Change-Id: I2ff6866ff1fe23b47c54268ac8e182210cc876c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/202137
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-05 00:19:10 +00:00
Ian Lance Taylor
04e80fb2c9 runtime: don't return from netpollGenericInit until init is complete
As a side-effect ensure that netpollinited only reports true when
netpoll initialization is complete.

Fixes #35282
Updates #35353

Change-Id: I21f08a04fcf229e0de5e6b5ad89c990426ae9b89
Reviewed-on: https://go-review.googlesource.com/c/go/+/204937
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-11-05 00:17:34 +00:00
Keith Randall
4af639a568 cmd: vendor in new version of x/tools
Fixes #35264

Change-Id: Id540a48f593d8ac1b414551255c5eff24666aa0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/205240
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-05 00:16:41 +00:00
Matthew Dempsky
ea0b4e7c7d cmd/compile, runtime: add comparison tracing for libFuzzer
This CL extends cmd/compile's experimental libFuzzer support with
calls to __sanitizer_cov_trace_{,const_}cmp{1,2,4,8}. This allows much
more efficient fuzzing of comparisons.

Only supports amd64 and arm64 for now.

Updates #14565.

Change-Id: Ibf82a8d9658f2bc50d955bdb1ae26723a3f0584d
Reviewed-on: https://go-review.googlesource.com/c/go/+/203887
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-05 00:00:43 +00:00
Matthew Dempsky
e341e93c51 cmd/compile, cmd/link: add coverage instrumentation for libfuzzer
This CL adds experimental coverage instrumentation similar to what
github.com/dvyukov/go-fuzz produces in its -libfuzzer mode. The
coverage can be enabled by compiling with -d=libfuzzer. It's intended
to be used in conjunction with -buildmode=c-archive to produce an ELF
archive (.a) file that can be linked with libFuzzer. See #14565 for
example usage.

The coverage generates a unique 8-bit counter for each basic block in
the original source code, and emits an increment operation. These
counters are then collected into the __libfuzzer_extra_counters ELF
section for use by libFuzzer.

Updates #14565.

Change-Id: I239758cc0ceb9ca1220f2d9d3d23b9e761db9bf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/202117
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-05 00:00:36 +00:00
Michael Anthony Knyszek
383b447e0d runtime: clean up power-of-two rounding code with align functions
This change renames the "round" function to the more appropriately named
"alignUp" which rounds an integer up to the next multiple of a power of
two.

This change also adds the alignDown function, which is almost like
alignUp but rounds down to the previous multiple of a power of two.

With these two functions, we also go and replace manual rounding code
with it where we can.

Change-Id: Ie1487366280484dcb2662972b01b4f7135f72fec
Reviewed-on: https://go-review.googlesource.com/c/go/+/190618
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-04 23:41:34 +00:00
Brad Fitzpatrick
2566e21f24 net/http: support disabling built-in HTTP/2 with a new build tag
Fixes #35082
Updates #6853

Change-Id: I4eeb0e15f534cff57fefb6039cd33fadf15b946e
Reviewed-on: https://go-review.googlesource.com/c/go/+/205139
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2019-11-04 23:16:09 +00:00
Michael Knyszek
74af7fc603 runtime: place lower limit on trigger ratio
This change makes it so that the GC pacer's trigger ratio can never fall
below 0.6. Upcoming changes to the allocator make it significantly more
scalable and thus much faster in certain cases, creating a large gap
between the performance of allocation and scanning. The consequence of
this is that the trigger ratio can drop very low (0.07 was observed) in
order to drop GC utilization. A low trigger ratio like this results in a
high amount of black allocations, which causes the live heap to appear
larger, and thus the heap, and RSS, grows to a much higher stable point.

This change alleviates the problem by placing a lower bound on the
trigger ratio. The expected (and confirmed) effect of this is that
utilization in certain scenarios will no longer converge to the expected
25%, and may go higher. As a result of this artificially high trigger
ratio, more time will also be spent doing GC assists compared to
dedicated mark workers, since the GC will be on for an artifically short
fraction of time (artificial with respect to the pacer). The biggest
concern of this change is that allocation latency will suffer as a
result, since there will now be more assists. But, upcoming changes to
the allocator reduce the latency enough to outweigh the expected
increase in latency from this change, without the blowup in RSS observed
from the changes to the allocator.

Updates #35112.

Change-Id: Idd7c94fa974d0de673304c4397e716e89bfbf09b
Reviewed-on: https://go-review.googlesource.com/c/go/+/200439
Reviewed-by: Austin Clements <austin@google.com>
2019-11-04 22:52:25 +00:00
Richard Musiol
54e6ba6724 syscall/js: garbage collect references to JavaScript values
The js.Value struct now contains a pointer, so a finalizer can
determine if the value is not referenced by Go any more.

Unfortunately this breaks Go's == operator with js.Value. This change
adds a new Equal method to check for the equality of two Values.
This is a breaking change. The == operator is now disallowed to
not silently break code.

Additionally the helper methods IsUndefined, IsNull and IsNaN got added.

Fixes #35111

Change-Id: I58a50ca18f477bf51a259c668a8ba15bfa76c955
Reviewed-on: https://go-review.googlesource.com/c/go/+/203600
Run-TryBot: Richard Musiol <neelance@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-04 22:50:43 +00:00
Matthew Dempsky
063d0f11e5 cmd/compile: restore -m=2 diagnostics
This is a rough attempt at restoring -m=2 escape analysis diagnostics
on par with those that were available with esc.go. It's meant to be
simple and non-invasive.

For example, given this random example from bytes/reader.go:

138	func (r *Reader) WriteTo(w io.Writer) (n int64, err error) {
...
143	        b := r.s[r.i:]
144	        m, err := w.Write(b)

esc.go used to report:

bytes/reader.go:138:7: leaking param content: r
bytes/reader.go:138:7:       from r.s (dot of pointer) at bytes/reader.go:143:8
bytes/reader.go:138:7:       from b (assigned) at bytes/reader.go:143:4
bytes/reader.go:138:7:       from w.Write(b) (parameter to indirect call) at bytes/reader.go:144:19

With this CL, escape.go now reports:

bytes/reader.go:138:7: parameter r leaks to {heap} with derefs=1:
bytes/reader.go:138:7:   flow: b = *r:
bytes/reader.go:138:7:     from r.s (dot of pointer) at bytes/reader.go:143:8
bytes/reader.go:138:7:     from r.s[r.i:] (slice) at bytes/reader.go:143:10
bytes/reader.go:138:7:     from b := r.s[r.i:] (assign) at bytes/reader.go:143:4
bytes/reader.go:138:7:   flow: {heap} = b:
bytes/reader.go:138:7:     from w.Write(b) (call parameter) at bytes/reader.go:144:19

Updates #31489.

Change-Id: I0c2b943a0f9ce6345bfff61e1c635172a9290cbb
Reviewed-on: https://go-review.googlesource.com/c/go/+/196959
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
2019-11-04 22:37:49 +00:00
Michael Munday
b6245cef3c internal/bytealg: add SIMD byte count implementation for s390x
Add a 'single lane' SIMD implemementation of the single byte count
function for use on machines that support the vector facility. This
allows up to 16 bytes to be counted per loop iteration.

We can probably improve performance further by adding more 'lanes'
(i.e. counting more bytes in parallel) however this will increase
the complexity of the function so I'm not sure it is worth doing
yet.

name                old speed      new speed       delta
pkg:strings goos:linux goarch:s390x
CountByte/10         789MB/s ± 0%   1131MB/s ± 0%    +43.44%  (p=0.000 n=9+9)
CountByte/32         936MB/s ± 0%   3236MB/s ± 0%   +245.87%  (p=0.000 n=8+9)
CountByte/4096      1.06GB/s ± 0%  21.26GB/s ± 0%  +1907.07%  (p=0.000 n=10+10)
CountByte/4194304   1.06GB/s ± 0%  20.54GB/s ± 0%  +1838.50%  (p=0.000 n=10+10)
CountByte/67108864  1.06GB/s ± 0%  18.31GB/s ± 0%  +1629.51%  (p=0.000 n=10+10)
pkg:bytes goos:linux goarch:s390x
CountSingle/10       800MB/s ± 0%    986MB/s ± 0%    +23.21%  (p=0.000 n=9+10)
CountSingle/32       925MB/s ± 0%   2744MB/s ± 0%   +196.55%  (p=0.000 n=9+10)
CountSingle/4K      1.26GB/s ± 0%  19.44GB/s ± 0%  +1445.59%  (p=0.000 n=10+10)
CountSingle/4M      1.26GB/s ± 0%  20.28GB/s ± 0%  +1510.26%  (p=0.000 n=8+10)
CountSingle/64M     1.23GB/s ± 0%  17.78GB/s ± 0%  +1350.67%  (p=0.000 n=9+10)

Change-Id: I230d57905db92a8fdfc50b1d5be338941ae3a7a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/199979
Run-TryBot: Michael Munday <mike.munday@ibm.com>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-04 22:06:16 +00:00
Cherry Zhang
d9ee4b2859 cmd/link/internal/ld: fix TestArchiveBuildInvokeWithExec
TestArchiveBuildInvokeWithExec is failing on darwin due to
duplicated symbols, because the C definition (int fortytwo;) is
copied to two generated cgo sources. In fact, this test is about
building c-archive, but doesn't need to import "C". Removed the
"C" import.

Change-Id: I3a17546e01272a7ae37e6417791ab949fb44597e
Reviewed-on: https://go-review.googlesource.com/c/go/+/205278
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-11-04 21:38:36 +00:00
Ian Lance Taylor
d80ab3e85a runtime: wake netpoller when dropping P, don't sleep too long in sysmon
When dropping a P, if it has any timers, and if some thread is
sleeping in the netpoller, wake the netpoller to run the P's timers.
This mitigates races between the netpoller deciding how long to sleep
and a new timer being added.

In sysmon, if all P's are idle, check the timers to decide how long to sleep.
This avoids oversleeping if no thread is using the netpoller.
This can happen in particular if some threads use runtime.LockOSThread,
as those threads do not block in the netpoller.

Also, print the number of timers per P for GODEBUG=scheddetail=1.

Before this CL, TestLockedDeadlock2 would fail about 1% of the time.
With this CL, I ran it 150,000 times with no failures.

Updates #6239
Updates #27707
Fixes #35274
Fixes #35288

Change-Id: I7e5193e6c885e567f0b1ee023664aa3e2902fcd1
Reviewed-on: https://go-review.googlesource.com/c/go/+/204800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-11-04 21:37:08 +00:00
Russ Cox
5a7c571ea1 hash/maphash: revise API to be more idiomatic
This CL makes these changes to the hash/maphash API to make it fit a bit
more into the standard library:

 - Move some of the package doc onto type Hash, so that `go doc maphash.Hash` shows it.

 - Instead of having identical AddBytes and Write methods,
   standardize on Write, the usual name for this function.
   Similarly, AddString -> WriteString, AddByte -> WriteByte.

 - Instead of having identical Hash and Sum64 methods,
   standardize on Sum64 (for hash.Hash64). Dropping the "Hash" method
   also helps because Hash is usually reserved to mean the state of a
   hash function (hash.Hash etc), not the hash value itself.

 - Make an uninitialized hash.Hash auto-seed with a random seed.
   It is critical that users not use the same seed for all hash functions
   in their program, at least not accidentally. So the Hash implementation
   must either panic if uninitialized or initialize itself.
   Initializing itself is less work for users and can be done lazily.

 - Now that the zero hash.Hash is useful, drop maphash.New in favor of
   new(maphash.Hash) or simply declaring a maphash.Hash.

 - Add a [0]func()-typed field to the Hash so that Hashes cannot be compared.
   (I considered doing the same for Seed but comparing seeds seems OK.)

 - Drop the integer argument from MakeSeed, to match the original design
   in golang.org/issue/28322. There is no point to giving users control
   over the specific seed bits, since we want the interpretation of those
   bits to be different in every different process. The only thing users
   need is to be able to create a new random seed at each call.
   (Fixes a TODO in MakeSeed's public doc comment.)

This API is new in Go 1.14, so these changes do not violate the compatibility promise.

Fixes #35060.
Fixes #35348.

Change-Id: Ie6fecc441f3f5ef66388c6ead92e875c0871f805
Reviewed-on: https://go-review.googlesource.com/c/go/+/205069
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-04 21:30:29 +00:00
Filippo Valsorda
03aca99f47 crypto/tls: make -update only regenerate transcripts for failing tests
Change-Id: Ie68fd4fe2879e6b5417a1a4240971e3d837bf115
Reviewed-on: https://go-review.googlesource.com/c/go/+/204377
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-04 20:39:53 +00:00
Michael Munday
bb7890b85a cmd/compile: absorb more Not ops into Neq* and Eq* ops
We absorbed Not into most integer comparisons but not into pointer
and floating point equality checks.

The new cases trigger more than 300 times during make.bash.

Change-Id: I77c6b31fcacde10da5470b73fc001a19521ce78d
Reviewed-on: https://go-review.googlesource.com/c/go/+/200618
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-04 20:35:52 +00:00
Cuong Manh Le
26d5f032e9 cmd/compile: add test for skipping empty init functions
CL 200958 adds skipping empty init function feature without any tests
for it. A codegen test sounds ideal, but it's unlikely that we can make
one for now, so use a program to manipulate runtime/proc.go:initTask
directly.

Updates #34869

Change-Id: I2683b9a1ace36af6861af02a3a9fb18b3110b282
Reviewed-on: https://go-review.googlesource.com/c/go/+/204217
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-11-04 20:19:15 +00:00
Wang Xuerui
210e3677f9 runtime: use vDSO clock_gettime on linux/mips64x
Speed up nanotime1 and walltime1 on MIPS64 with vDSO, just like the
other vDSO-enabled targets.

Benchmark numbers on Loongson 3A3000 (GOARCH=mips64le, 1.4GHz) against
current master:

benchmark                old ns/op     new ns/op     delta
BenchmarkNow             868           293           -66.24%
BenchmarkNowUnixNano     851           296           -65.22%

Performance hit on fallback case, tested by using a wrong vDSO symbol name:

benchmark                old ns/op     new ns/op     delta
BenchmarkNow             868           889           +2.42%
BenchmarkNowUnixNano     851           893           +4.94%

Change-Id: Ibfb48893cd060536359863ffee7624c00def646b
GitHub-Last-Rev: 03a58ac2e4
GitHub-Pull-Request: golang/go#35181
Reviewed-on: https://go-review.googlesource.com/c/go/+/203578
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-04 18:53:43 +00:00
Than McIntosh
bf7e55b618 cmd/link: directly exec archive command if external tmpdir
When linking a Go archive, if the archiver invocation is the very last
thing that needs to happen in the link (no "atexit" cleanups required
remove the locally created tmpdir) then call syscall.Exec to invoke
the archiver command instead of the usual exec.Command. This has the
effect of reducing peak memory use for the linker overall, since we
don't be holding onto all of the linker's live memory while the
archiver is running.

Change-Id: Ibbe22d8d67a70cc2a4f91c68aab56d19fb77c393
Reviewed-on: https://go-review.googlesource.com/c/go/+/203821
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-11-04 18:51:13 +00:00
Cherry Zhang
1da575a7bc cmd/internal/obj/s390x: add support of SPM instruction
For restoring condition code (we already support IPM instruction
for saving condition code).

Change-Id: I56d376df44a5f831134a130d052521cec6b5b781
Reviewed-on: https://go-review.googlesource.com/c/go/+/204104
Reviewed-by: Michael Munday <mike.munday@ibm.com>
2019-11-04 17:19:36 +00:00
Dan Scales
7dcd343ed6 runtime: ensure that Goexit cannot be aborted by a recursive panic/recover
When we do a successful recover of a panic, we resume normal execution by
returning from the frame that had the deferred call that did the recover (after
executing any remaining deferred calls in that frame).

However, suppose we have called runtime.Goexit and there is a panic during one of the
deferred calls run by the Goexit. Further assume that there is a deferred call in
the frame of the Goexit or a parent frame that does a recover. Then the recovery
process will actually resume normal execution above the Goexit frame and hence
abort the Goexit.  We will not terminate the thread as expected, but continue
running in the frame above the Goexit.

To fix this, we explicitly create a _panic object for a Goexit call. We then
change the "abort" behavior for Goexits, but not panics. After a recovery, if the
top-level panic is actually a Goexit that is marked to be aborted, then we return
to the Goexit defer-processing loop, so that the Goexit is not actually aborted.

Actual code changes are just panic.go, runtime2.go, and funcid.go. Adjusted the
test related to the new Goexit behavior (TestRecoverBeforePanicAfterGoexit) and
added several new tests of aborted panics (whose behavior has not changed).

Fixes #29226

Change-Id: Ib13cb0074f5acc2567a28db7ca6912cfc47eecb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/200081
Run-TryBot: Dan Scales <danscales@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2019-11-04 16:32:38 +00:00