1
0
mirror of https://github.com/golang/go synced 2024-09-30 16:18:35 -06:00
Commit Graph

41652 Commits

Author SHA1 Message Date
Giovanni Bajo
c70e5475e6 cmd/compile: in poset, implement path collapsing
Sometimes, poset needs to collapse a path making all nodes in
the path aliases. For instance, we know that A<=N1<=B and we
learn that B<=A, we can deduce A==N1==B, and thus we can
collapse all paths from A to B into a single aliased node.

Currently, this is a TODO. This CL implements the path-collapsing
primitive by doing a DFS walk to build a bitset of all nodes
across all paths, and then calling the new aliasnodes that allow
to mark multiple nodes as aliases of a single master node.

This helps only 4 times in std+cmd, but it will be fundamental
when we will rely on poset to calculate numerical limits, to
calculate the correct values.

This also fixes #35157, a bug uncovered by a previous CL in this
serie. A testcase will be added soon.

Change-Id: I5fc54259711769d7bd7c2d166a5abc1cddc26350
Reviewed-on: https://go-review.googlesource.com/c/go/+/200861
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2019-10-26 07:52:35 +00:00
Giovanni Bajo
18d57bc892 cmd/compile: in poset, allow multiple aliases in a single pass
Change aliasnode into aliasnodes, to allow for recording
multiple aliases in a single pass. The nodes being aliased
are passed as bitset for performance reason (O(1) lookups).
It does look worse in the existing case of SetEqual where
we now need to allocate a bitset just for a single node,
but the new API will allow to fully implement a path-collapsing
primitive in next CL.

No functional changes, passes toolstash -cmp.

Change-Id: I06259610e8ef478106b36852464ed2caacd29ab5
Reviewed-on: https://go-review.googlesource.com/c/go/+/200860
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-26 07:16:46 +00:00
Giovanni Bajo
7e68f81dd8 cmd/compile: in poset, refactor aliasnode
In preparation for allowing to make multiple nodes as aliases
in a single pass, refactor aliasnode splitting out the case
in which one of the nodes is not in the post into a new
funciton (aliasnewnode).

No functional changes, passes toolstash -cmp

Change-Id: I19ca6ef8426f8aec9f2622b6151c5c617dbb25b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/200859
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-26 07:06:49 +00:00
nikita-vanyasin
c3e8a20a65 cmd/link/internal: eliminate all ld.Cputime() usages
Also a similar 'elapsed' function and its usages were deleted.

Fixes #19865.

Change-Id: Ib125365e69cf2eda60de64fa74290c8c7d1fd65a
Reviewed-on: https://go-review.googlesource.com/c/go/+/171730
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2019-10-26 05:40:14 +00:00
Ben Shi
65a1e24209 runtime: save/restore callee-save registers in arm's sigtramp
ARM's R4-R8 & R10-R11 are callee-save registers, and R9
may be callee-save or not. This CL saves them at the beginning
of sigtramp and restores them in the end.

fixes #32738

Change-Id: Ib7eb80836bc074e2e6a46ae4602ba8a3b96c5456
Reviewed-on: https://go-review.googlesource.com/c/go/+/183777
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-26 04:00:03 +00:00
Austin Clements
8dc1a158e4 runtime: add test for signalM
For #10958, #24543.

Change-Id: Ib009a83fe02bc623894f4908fe8f6b266382ba95
Reviewed-on: https://go-review.googlesource.com/c/go/+/201404
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-26 02:52:32 +00:00
Austin Clements
42aab4b0af runtime: M-targeted signals for libc-based OSes
For #10958, #24543.

Change-Id: I82bee63b49e15bd5a53228eb85179814c80437ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/201403
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-26 02:52:30 +00:00
Austin Clements
8714e39497 runtime: M-targeted signals for BSDs
For these, we split up the existing runtime.raise assembly
implementation into its constituent "get thread ID" and "signal
thread" parts. This lets us implement signalM and reimplement raise in
pure Go. (NetBSD conveniently already had lwp_self.)

We also change minit to store the procid directly, rather than
depending on newosproc to do so. This is because newosproc isn't
called for the bootstrap M, but we need a procid for every M. This is
also simpler overall.

For #10958, #24543.

Change-Id: Ie5f1fcada6a33046375066bcbe054d1f784d39c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/201402
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-26 02:52:28 +00:00
Austin Clements
334291d1f6 runtime: M-targeted signals for Linux
We'll add a test once all of the POSIX platforms are done.

For #10958, #24543.

Change-Id: If7e3f14e8391791364877629bf415d9f8e788b0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/201401
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-26 02:52:25 +00:00
Vivek V
3706cd85d3 debug/dwarf: call strings.EqualFold instead of calling Lower twice
Change-Id: I8dcb425c37a067277549ba3bda6a21206459890a
GitHub-Last-Rev: dc51b9b425
GitHub-Pull-Request: golang/go#35132
Reviewed-on: https://go-review.googlesource.com/c/go/+/203097
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-26 02:36:26 +00:00
Bryan C. Mills
06bdd52f75 os: use an actual RemoveAll failure in TestRemoveAllWithMoreErrorThanReqSize
Previously we injected an error, and the injection points were
(empirically) not realistic on some platforms.

Instead, we now make the directory read-only, which (on most
platforms) suffices to prevent the removal of its files.

Fixes #35117
Updates #29921

Change-Id: Ica4e2818566f8c14df3eed7c3b8de5c0abeb6963
Reviewed-on: https://go-review.googlesource.com/c/go/+/203502
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-26 01:46:02 +00:00
Cuong Manh Le
66f78e9d88 runtime: mark findObject nosplit
findObject takes the pointer argument as uintptr. If the pointer is to
the local stack and calling findObject happens to require the stack to
be reallocated, then spanOf is called for the old pointer.

Marking findObject as nosplit fixes the issue.

Fixes #35068

Change-Id: I029d36f9c23f91812f18f98839edf02e0ba4082e
Reviewed-on: https://go-review.googlesource.com/c/go/+/202798
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-10-26 00:05:49 +00:00
Cuong Manh Le
813d8e8862 runtime: factor out debug.invalidptr case in findObject
This helps keeping findObject's frame small.

Updates #35068

Change-Id: I1b8c1fcc5831944c86f1a30ed2f2d867a5f2b242
Reviewed-on: https://go-review.googlesource.com/c/go/+/202797
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2019-10-26 00:05:43 +00:00
Cuong Manh Le
80315322f3 runtime: simplify findObject bad pointer checking condition
Factor out case s == nil, make the code cleaner and easier to read.

Change-Id: I63f52e14351c0a0d20a611b1fe10fdc0d4947d96
Reviewed-on: https://go-review.googlesource.com/c/go/+/202498
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2019-10-26 00:05:37 +00:00
Austin Clements
d1969015b4 runtime: abstract M preemption check into a function
We check whether an M is preemptible in a surprising number of places.
Put it in one function.

For #10958, #24543.

Change-Id: I305090fdb1ea7f7a55ffe25851c1e35012d0d06c
Reviewed-on: https://go-review.googlesource.com/c/go/+/201439
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 23:25:36 +00:00
Austin Clements
6058603471 runtime: only shrink stacks at synchronous safe points
We're about to introduce asynchronous safe points, where we won't have
precise pointer maps for all stack frames. That's okay for scanning
the stack (conservatively), but not for shrinking the stack.

Hence, this CL prepares for this by only shrinking the stack as part
of the stack scan if the goroutine is stopped at a synchronous safe
point. Otherwise, it queues up the stack shrink for the next
synchronous safe point.

We already have one condition under which we can't shrink the stack
for very similar reasons: syscalls. Currently, we just give up on
shrinking the stack if it's in a syscall. But with this mechanism, we
defer that stack shrink until the next synchronous safe point.

For #10958, #24543.

Change-Id: Ifa1dec6f33fdf30f9067be2ce3f7ab8a7f62ce38
Reviewed-on: https://go-review.googlesource.com/c/go/+/201438
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 23:25:35 +00:00
Austin Clements
36a432f27b runtime: make copystack/sudog synchronization more explicit
When we copy a stack of a goroutine blocked in a channel operation, we
have to be very careful because other goroutines may be writing to
that goroutine's stack. To handle this, stack copying acquires the
locks for the channels a goroutine is waiting on.

One complication is that stack growth may happen while a goroutine
holds these locks, in which case stack copying must *not* acquire
these locks because that would self-deadlock.

Currently, stack growth never acquires these locks because stack
growth only happens when a goroutine is running, which means it's
either not blocking on a channel or it's holding the channel locks
already. Stack shrinking always acquires these locks because shrinking
happens asynchronously, so the goroutine is never running, so there
are either no locks or they've been released by the goroutine.

However, we're about to change when stack shrinking can happen, which
is going to break the current rules. Rather than find a new way to
derive whether to acquire these locks or not, this CL simply adds a
flag to the g struct that indicates that stack copying should acquire
channel locks. This flag is set while the goroutine is blocked on a
channel op.

For #10958, #24543.

Change-Id: Ia2ac8831b1bfda98d39bb30285e144c4f7eaf9ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/172982
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2019-10-25 23:25:33 +00:00
Austin Clements
8c5861576a runtime: remove g.gcscanvalid
Currently, gcscanvalid is used to resolve a race between attempts to
scan a stack. Now that there's a clear owner of the stack scan
operation, there's no longer any danger of racing or attempting to
scan a stack more than once, so this CL eliminates gcscanvalid.

I double-checked my reasoning by first adding a throw if gcscanvalid
was set in scanstack and verifying that all.bash still passed.

For #10958, #24543.
Fixes #24363.

Change-Id: I76794a5fcda325ed7cfc2b545e2a839b8b3bc713
Reviewed-on: https://go-review.googlesource.com/c/go/+/201139
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 23:25:32 +00:00
Austin Clements
1b79afe460 runtime: remove old stack scanning code
This removes scang and preemptscan, since the stack scanning code now
uses suspendG.

For #10958, #24543.

Change-Id: Ic868bf5d6dcce40662a82cb27bb996cb74d0720e
Reviewed-on: https://go-review.googlesource.com/c/go/+/201138
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 23:25:31 +00:00
Austin Clements
3f834114ab runtime: add general suspendG/resumeG
Currently, the process of suspending a goroutine is tied to stack
scanning. In preparation for non-cooperative preemption, this CL
abstracts this into general purpose suspendG/resumeG functions.

suspendG and resumeG closely follow the existing scang and restartg
functions with one exception: the addition of a _Gpreempted status.
Currently, preemption tasks (stack scanning) are carried out by the
target goroutine if it's in _Grunning. In this new approach, the task
is always carried out by the goroutine that called suspendG. Thus, we
need a reliable way to drive the target goroutine out of _Grunning
until the requesting goroutine is ready to resume it. The new
_Gpreempted state provides the handshake: when a runnable goroutine
responds to a preemption request, it now parks itself and enters
_Gpreempted. The requesting goroutine races to put it in _Gwaiting,
which gives it ownership, but also the responsibility to start it
again.

This CL adds several TODOs about improving the synchronization on the
G status. The existing code already has these problems; we're just
taking note of them.

The next CL will remove the now-dead scang and preemptscan.

For #10958, #24543.

Change-Id: I16dbf87bea9d50399cc86719c156f48e67198f16
Reviewed-on: https://go-review.googlesource.com/c/go/+/201137
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 23:25:28 +00:00
pokutuna
46e0d724b3 net/http: fix comment TimeoutHandler no longer supports Flusher
Fixes #35161
Updates #34439

Change-Id: I978534cbb8b9fb32c115dba0066cf099c61d8ee9
GitHub-Last-Rev: d60581635e
GitHub-Pull-Request: golang/go#35162
Reviewed-on: https://go-review.googlesource.com/c/go/+/203478
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-25 20:46:57 +00:00
Bryan C. Mills
dcad830621 cmd/go: implement svn support in module mode
mod_get_svn passes, and I also tested this manually on a real-world svn-hosted package:

	example.com$ go mod init example.com
	go: creating new go.mod: module example.com

	example.com$ GOPROXY=direct GONOSUMDB=llvm.org go get -d llvm.org/llvm/bindings/go/llvm
	go: finding llvm.org/llvm latest
	go: finding llvm.org/llvm/bindings/go/llvm latest
	go: downloading llvm.org/llvm v0.0.0-20191022153947-000000375505
	go: extracting llvm.org/llvm v0.0.0-20191022153947-000000375505

	example.com$ go list llvm.org/llvm/bindings/...
	llvm.org/llvm/bindings/go
	llvm.org/llvm/bindings/go/llvm

Fixes #26092

Change-Id: Iefe2151b82a0225c73bb6f8dd7cd8a352897d4c0
Reviewed-on: https://go-review.googlesource.com/c/go/+/203497
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-10-25 20:21:24 +00:00
Tobias Klauser
316fb95f4f runtime: define emptyfunc as static function in assembly for freebsd/arm64
CL 198544 broke the linux/arm64 build because it declares emptyfunc for
GOARCH=arm64, but only freebsd/arm64 defines it. Make it a static
assembly function specific for freebsd/arm64 and remove the stub.

Fixes #35160

Change-Id: I5fd94249b60c6fd259c251407b6eccc8fa512934
Reviewed-on: https://go-review.googlesource.com/c/go/+/203418
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-25 20:14:52 +00:00
Bryan C. Mills
8bb47a5eec net/http: skip failing test on windows-amd64-longtest builder
bradfitz is actively thinking about a proper fix.
In the meantime, skip the test to suss out any other failures in the builder.

Updates #35122

Change-Id: I9bf0640222e3d385c1a3e2be5ab52b80d3e8c21a
Reviewed-on: https://go-review.googlesource.com/c/go/+/203500
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-25 18:45:21 +00:00
Bryan C. Mills
096126de6b os/signal: derive TestAtomicStop timeout from overall test timeout
Previously, TestAtomicStop used a hard-coded 2-second timeout.

That empirically is not long enough on certain builders. Rather than
adjusting it to a different arbitrary value, use a slice of the
overall timeout for the test binary. If everything is working, we
won't block nearly that long anyway.

Updates #35085

Change-Id: I7b789388e3152413395088088fc497419976cf5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/203499
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-25 18:44:06 +00:00
Tobias Klauser
3d457f1a36 cmd/dist: add support for freebsd/arm64
Updates #24715

Change-Id: I110a10a5d5ed4a471f67f35cbcdcbea296c5dcaf
Reviewed-on: https://go-review.googlesource.com/c/go/+/198542
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-10-25 17:59:32 +00:00
Tobias Klauser
6b6e67f9b9 runtime: add support for freebsd/arm64
Based on work by Mikaël Urankar (@MikaelUrankar),
Shigeru YAMAMOTO (@bsd-hacker) and @myfreeweb.

Updates #24715

Change-Id: If3189a693ca0aa627029e22b0f91534bcf322bc0
Reviewed-on: https://go-review.googlesource.com/c/go/+/198544
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-25 17:59:05 +00:00
Austin Clements
fc8eb264bb runtime: ensure _Grunning Gs have a valid g.m and g.m.p
We already claim on the documentation for _Grunning that this is case,
but execute transitions to _Grunning before assigning g.m. Fix this
and make the documentation even more explicit.

For #10958, #24543, but also a good cleanup.

Change-Id: I1eb0108e7762f55cfb0282aca624af1c0a15fe56
Reviewed-on: https://go-review.googlesource.com/c/go/+/201440
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 16:59:39 +00:00
Austin Clements
f82956b85b runtime: make m.libcallsp check in shrinkstack panic
Currently, shrinkstack will not shrink a stack on Windows if
gp.m.libcallsp != 0. In general, we can't shrink stacks in syscalls
because the syscall may hold pointers into the stack, and in principle
this is supposed to be preventing that for libcall-based syscalls
(which are direct syscalls from the runtime). But this test is
actually broken and has been for a long time. That turns out to be
okay because it also appears it's not necessary.

This test is racy. g.m points to whatever M the G was last running on,
even if the G is in a blocked state, and that M could be doing
anything, including making libcalls. Hence, observing that libcallsp
== 0 at one moment in shrinkstack is no guarantee that it won't become
non-zero while we're shrinking the stack, and vice-versa.

It's also weird that this check is only performed on Windows, given
that we now use libcalls on macOS, Solaris, and AIX.

This check was added when stack shrinking was first implemented in CL
69580044. The history of that CL (though not the final version)
suggests this was necessary for libcalls that happened on Go user
stacks, which we never do now because of the limited stack space.

It could also be defending against user stack pointers passed to
libcall system calls from blocked Gs. But the runtime isn't allowed to
keep pointers into the user stack for blocked Gs on any OS, so it's
not clear this would be of any value.

Hence, this checks seems to be simply unnecessary.

Rather than simply remove it, this CL makes it defensive. We can't do
anything about blocked Gs, since it doesn't even make sense to look at
their M, but if a G tries to shrink its own stack while in a libcall,
that indicates a bug in the libcall code. This CL makes shrinkstack
panic in this case.

For #10958, #24543, since those are going to rearrange how we decide
that it's safe to shrink a stack.

Change-Id: Ia865e1f6340cff26637f8d513970f9ebb4735c6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/173724
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-10-25 16:59:36 +00:00
Austin Clements
9d032026d6 cmd/internal/obj/x86: correct pcsp for ADJSP
The x86 assembler supports an "ADJSP" pseudo-op that compiles to an
ADD/SUB from SP. Unfortunately, while this seems perfect for an
instruction that would allow obj to continue to track the SP/FP delta,
obj currently doesn't do that. As a result, FP-relative references
won't work and, perhaps worse, the pcsp table will have the wrong
frame size.

We don't currently use this instruction in any assembly or generate it
in the compiler, but this is a perfect instruction for solving a
problem in #24543.

This CL makes ADJSP useful by incorporating it into the SP delta
logic.

One subtlety is that we do generate ADJSP in obj itself to open a
function's stack frame. Currently, when preprocess enters the loop to
compute the SP delta, it may or may not start at this ADJSP
instruction depending on various factors. We clean this up by instead
always starting the SP delta at 0 and always starting this loop at the
entry to the function.

Why not just recognize ADD/SUB of SP? The danger is that could change
the meaning of existing code. For example, walltime1 in
sys_linux_amd64.s saves SP, SUBs from it, and aligns it. Later, it
restores the saved copy and then does a few FP-relative references.
Currently obj doesn't know any of this is happening, but that's fine
once it gets to the FP-relative references. If we taught obj to
recognize the SUB, it would start to miscompile this code. An
alternative would be to recognize unknown instructions that write to
SP and refuse subsequent FP-relative references, but that's kind of
annoying.

This passes toolstash -cmp for std on both amd64 and 386.

Change-Id: Ic6c6a7cbf980bca904576676c07b44c0aaa9c82d
Reviewed-on: https://go-review.googlesource.com/c/go/+/200877
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2019-10-25 16:34:10 +00:00
ZYunH
00d6b28d4e internal/singleflight: format someErr
Error string should not be capitalized.

Change-Id: I8e1d148c6b999450bcd702f420c2a240f82aadc7
GitHub-Last-Rev: 6ca1b3edb4
GitHub-Pull-Request: golang/go#35147
Reviewed-on: https://go-review.googlesource.com/c/go/+/203339
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-25 16:31:41 +00:00
Meng Zhuo
dad512f6df cmd/dist: fix wrong goarch on mips64le
Change-Id: I625f0bc533a7d14010c0344f36e8f157a19c13f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/203437
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2019-10-25 14:31:54 +00:00
Meng Zhuo
9a701017e6 runtime: fix typo of MADV_NOHUGEPAGE
Change-Id: I60a1ca606fe7492c05697c4d58afc7f19fcc63fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/203340
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-25 13:49:01 +00:00
Jason A. Donenfeld
3c25e5ec38 internal/syscall/windows/registry: allow for non-null terminated strings
According to MSDN, "If the data has the REG_SZ, REG_MULTI_SZ or
REG_EXPAND_SZ type, this size includes any terminating null character or
characters unless the data was stored without them. [...] If the data
has the REG_SZ, REG_MULTI_SZ or REG_EXPAND_SZ type, the string may not
have been stored with the proper terminating null characters. Therefore,
even if the function returns ERROR_SUCCESS, the application should
ensure that the string is properly terminated before using it;
otherwise, it may overwrite a buffer."

It's therefore dangerous to pass it off unbounded as we do, and in fact
this led to crashes on real systems.

Change-Id: I6d786211814656f036b87fd78631466634cd764a
Reviewed-on: https://go-review.googlesource.com/c/go/+/202937
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2019-10-25 09:01:11 +00:00
Rob Pike
4f70c151db fmt: fix handling of %% verb in Scanf
There were a couple of bugs, including not requiring a percent and
returning the wrong error for a bad format containing %%.

Both are addressed by fixing the first.

Fixes #34180.

Change-Id: If96c0c0258bcb95eec49871437d719cb9d399d9b
Reviewed-on: https://go-review.googlesource.com/c/go/+/202879
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-10-25 04:15:59 +00:00
Rémy Oudompheng
8f30d25168 math/big: use nat pool to reduce allocations in mul and sqr
This notably allows to reuse temporaries across
the karatsubaSqr recursion.

benchmark                    old ns/op     new ns/op     delta
BenchmarkNatMul/10-4         227           228           +0.44%
BenchmarkNatMul/100-4        8339          8589          +3.00%
BenchmarkNatMul/1000-4       313796        312272        -0.49%
BenchmarkNatMul/10000-4      11924720      11873589      -0.43%
BenchmarkNatMul/100000-4     503813354     503839058     +0.01%
BenchmarkNatSqr/20-4         549           513           -6.56%
BenchmarkNatSqr/30-4         945           874           -7.51%
BenchmarkNatSqr/50-4         1993          1832          -8.08%
BenchmarkNatSqr/80-4         4096          3874          -5.42%
BenchmarkNatSqr/100-4        6192          5712          -7.75%
BenchmarkNatSqr/200-4        20388         19543         -4.14%
BenchmarkNatSqr/300-4        38735         36715         -5.21%
BenchmarkNatSqr/500-4        99562         93542         -6.05%
BenchmarkNatSqr/800-4        195554        184907        -5.44%
BenchmarkNatSqr/1000-4       286302        275053        -3.93%
BenchmarkNatSqr/10000-4      9817057       9441641       -3.82%
BenchmarkNatSqr/100000-4     390713416     379696789     -2.82%

benchmark                    old allocs     new allocs     delta
BenchmarkNatMul/10-4         1              1              +0.00%
BenchmarkNatMul/100-4        1              1              +0.00%
BenchmarkNatMul/1000-4       2              1              -50.00%
BenchmarkNatMul/10000-4      2              1              -50.00%
BenchmarkNatMul/100000-4     9              11             +22.22%
BenchmarkNatSqr/20-4         2              1              -50.00%
BenchmarkNatSqr/30-4         2              1              -50.00%
BenchmarkNatSqr/50-4         2              1              -50.00%
BenchmarkNatSqr/80-4         2              1              -50.00%
BenchmarkNatSqr/100-4        2              1              -50.00%
BenchmarkNatSqr/200-4        2              1              -50.00%
BenchmarkNatSqr/300-4        4              1              -75.00%
BenchmarkNatSqr/500-4        4              1              -75.00%
BenchmarkNatSqr/800-4        10             1              -90.00%
BenchmarkNatSqr/1000-4       10             1              -90.00%
BenchmarkNatSqr/10000-4      731            1              -99.86%
BenchmarkNatSqr/100000-4     19687          6              -99.97%

benchmark                    old bytes     new bytes     delta
BenchmarkNatMul/10-4         192           192           +0.00%
BenchmarkNatMul/100-4        4864          4864          +0.00%
BenchmarkNatMul/1000-4       57344         49224         -14.16%
BenchmarkNatMul/10000-4      565248        498772        -11.76%
BenchmarkNatMul/100000-4     5749504       7263720       +26.34%
BenchmarkNatSqr/20-4         672           352           -47.62%
BenchmarkNatSqr/30-4         992           512           -48.39%
BenchmarkNatSqr/50-4         1792          896           -50.00%
BenchmarkNatSqr/80-4         2688          1408          -47.62%
BenchmarkNatSqr/100-4        3584          1792          -50.00%
BenchmarkNatSqr/200-4        6656          3456          -48.08%
BenchmarkNatSqr/300-4        24448         16387         -32.97%
BenchmarkNatSqr/500-4        36864         24591         -33.29%
BenchmarkNatSqr/800-4        69760         40981         -41.25%
BenchmarkNatSqr/1000-4       86016         49180         -42.82%
BenchmarkNatSqr/10000-4      2524800       487368        -80.70%
BenchmarkNatSqr/100000-4     68599808      5876581       -91.43%

Change-Id: I8e6e409ae1cb48be9d5aa9b5f428d6cbe487673a
Reviewed-on: https://go-review.googlesource.com/c/go/+/172017
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2019-10-25 03:14:39 +00:00
Bryan C. Mills
680ed10c91 cmd/go/internal/modfetch/codehost: remove invariantly-empty return value from Repo.ReadZip
Previously, codehost.Repo.ReadZip returned an 'actualSubdir' value
that was the empty string in all current implementations.

Updates #26092

Change-Id: I6708dd0f13ba88bcf1a1fb405e9d818fd6f9197e
Reviewed-on: https://go-review.googlesource.com/c/go/+/203277
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-10-24 20:19:06 +00:00
Jay Conrod
c357b363cf cmd/go: add -modfile flag that sets go.mod file to read/write
This change adds the -modfile flag to module aware build commands and
to 'go mod' subcommands. -modfile may be set to a path to an alternate
go.mod file to be read and written. A real go.mod file must still
exist and is used to set the module root directory. However, it is not
opened.

When -modfile is set, the effective location of the go.sum file is
also changed to the -modfile with the ".mod" suffix trimmed (if
present) and ".sum" added.

Updates #34506

Change-Id: I2d1e044e18af55505a4f24bbff09b73bb9c908b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/202564
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-24 20:15:50 +00:00
Jay Conrod
c4c37547b1 cmd/go/internal/modload: remove cwd global
base.Cwd should be used instead.

Change-Id: I3dbdecf745b0823160984cc942c883dc04c91d7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/203037
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-24 20:15:14 +00:00
Bryan C. Mills
f922cc6446 cmd/go: correct GoMod field in 'go list' for replacements that lack an explicit go.mod file
Change-Id: I241a3bbaf9c4b779b74146232d2f144bb46a0dc7
Reviewed-on: https://go-review.googlesource.com/c/go/+/203178
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-10-24 17:40:35 +00:00
Jay Conrod
bd622872f4 doc: add skeleton module documentation with headings
Sections will be filled in with individual CLs before Go 1.14.

NOTE: This document is currently in Markdown for ease of writing /
reviewing. Before Go 1.14, we will either ensure that x/website
can render Markdown (flavor TBD) or check in a rendered HTML file that
can be displayed directly.

Updates #33637

Change-Id: Icd43fa2bdb7d256b28a56b93214b70343f43492e
Reviewed-on: https://go-review.googlesource.com/c/go/+/202081
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-10-24 16:58:49 +00:00
Bryan C. Mills
fc33506864 cmd/go: re-enable 'go list -m' with -mod=vendor for limited patterns
I had prohibited 'go list -m' with -mod=vendor because the module
graph is incomplete, but I've realized that many queries do not
actually require the full graph — and may, in fact, be driven using
modules previously reported by 'go list' for specific, vendored
packages. Queries for those modules should succeed.

Updates #33848

Change-Id: I1000b4cf586a830bb78faf620ebf62d73a3cb300
Reviewed-on: https://go-review.googlesource.com/c/go/+/203138
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-10-24 16:18:09 +00:00
Bryan C. Mills
b36f22bff0 cmd/go: populate available module information for packages in vendor mode
Updates #33848

Change-Id: I10b4c79faef8bc3dee2ceba14d496fa049e84fb2
Reviewed-on: https://go-review.googlesource.com/c/go/+/202977
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-10-24 16:17:59 +00:00
Cuong Manh Le
05ee5065d0 syscall: correct comment in testGetdirentries
Correct comment about allocating big enough slice to copy result of
Getdirentries.

While at it, also convert from Dirent directly to slice of byte.

Updates #35092

Change-Id: I892de7953120622882e1561728e1e56b009a2351
Reviewed-on: https://go-review.googlesource.com/c/go/+/202880
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-24 13:58:59 +00:00
Dan Scales
be64a19d99 cmd/compile, cmd/link, runtime: make defers low-cost through inline code and extra funcdata
Generate inline code at defer time to save the args of defer calls to unique
(autotmp) stack slots, and generate inline code at exit time to check which defer
calls were made and make the associated function/method/interface calls. We
remember that a particular defer statement was reached by storing in the deferBits
variable (always stored on the stack). At exit time, we check the bits of the
deferBits variable to determine which defer function calls to make (in reverse
order). These low-cost defers are only used for functions where no defers
appear in loops. In addition, we don't do these low-cost defers if there are too
many defer statements or too many exits in a function (to limit code increase).

When a function uses open-coded defers, we produce extra
FUNCDATA_OpenCodedDeferInfo information that specifies the number of defers, and
for each defer, the stack slots where the closure and associated args have been
stored. The funcdata also includes the location of the deferBits variable.
Therefore, for panics, we can use this funcdata to determine exactly which defers
are active, and call the appropriate functions/methods/closures with the correct
arguments for each active defer.

In order to unwind the stack correctly after a recover(), we need to add an extra
code segment to functions with open-coded defers that simply calls deferreturn()
and returns. This segment is not reachable by the normal function, but is returned
to by the runtime during recovery. We set the liveness information of this
deferreturn() to be the same as the liveness at the first function call during the
last defer exit code (so all return values and all stack slots needed by the defer
calls will be live).

I needed to increase the stackguard constant from 880 to 896, because of a small
amount of new code in deferreturn().

The -N flag disables open-coded defers. '-d defer' prints out the kind of defer
being used at each defer statement (heap-allocated, stack-allocated, or
open-coded).

Cost of defer statement  [ go test -run NONE -bench BenchmarkDefer$ runtime ]
  With normal (stack-allocated) defers only:         35.4  ns/op
  With open-coded defers:                             5.6  ns/op
  Cost of function call alone (remove defer keyword): 4.4  ns/op

Text size increase (including funcdata) for go binary without/with open-coded defers:  0.09%

The average size increase (including funcdata) for only the functions that use
open-coded defers is 1.1%.

The cost of a panic followed by a recover got noticeably slower, since panic
processing now requires a scan of the stack for open-coded defer frames. This scan
is required, even if no frames are using open-coded defers:

Cost of panic and recover [ go test -run NONE -bench BenchmarkPanicRecover runtime ]
  Without open-coded defers:        62.0 ns/op
  With open-coded defers:           255  ns/op

A CGO Go-to-C-to-Go benchmark got noticeably faster because of open-coded defers:

CGO Go-to-C-to-Go benchmark [cd misc/cgo/test; go test -run NONE -bench BenchmarkCGoCallback ]
  Without open-coded defers:        443 ns/op
  With open-coded defers:           347 ns/op

Updates #14939 (defer performance)
Updates #34481 (design doc)

Change-Id: I63b1a60d1ebf28126f55ee9fd7ecffe9cb23d1ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/202340
Reviewed-by: Austin Clements <austin@google.com>
2019-10-24 13:54:11 +00:00
Bryan C. Mills
dc77dc2b9b cmd/go/internal/list: ensure that cfg.BuildMod is initialized before reading it in 'go list -m'
The default value of cfg.BuildMod depends on the 'go' version in the
go.mod file. The go.mod file is read and parsed, and its settings are
applied, in modload.InitMod.

As it turns out, modload.Enabled does not invoke InitMod, so
cfg.BuildMod is not necessarily set even if modload.Enabled returns
true.

Updates #33848

Change-Id: I13a4dd80730528e6f1a5acc492fcfe07cb59d94e
Reviewed-on: https://go-review.googlesource.com/c/go/+/202917
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-10-24 13:21:45 +00:00
Cuong Manh Le
a42a396846 syscall: remove un-used const ptrSize
Change-Id: Ic809a533f9c4042373bdad3ba1cd237d203bacff
Reviewed-on: https://go-review.googlesource.com/c/go/+/202881
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-24 12:09:29 +00:00
Cuong Manh Le
722b0e32ba syscall: make convertFromDirents11 checkptr safe
Fixes #35092

Change-Id: I8f1ee2b79d42b2291548fd5645940a61f6d67582
Reviewed-on: https://go-review.googlesource.com/c/go/+/202878
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2019-10-24 08:47:14 +00:00
Brad Fitzpatrick
68981bf309 syscall: make TestGetdirentries checkptr safe
Fixes Darwin.

Updates #35092

Change-Id: I045f070c8549d00610b459e3a82cac870d9ddb54
Reviewed-on: https://go-review.googlesource.com/c/go/+/203077
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-10-24 04:36:53 +00:00
Robert Griesemer
a52c0a1992 math/big: make Rat.Denom side-effect free
A Rat is represented via a quotient a/b where a and b are Int values.
To make it possible to use an uninitialized Rat value (with a and b
uninitialized and thus == 0), the implementation treats a 0 denominator
as 1.

Rat.Num and Rat.Denom return pointers to these values a and b. Because
b may be 0, Rat.Denom used to first initialize it to 1 and thus produce
an undesirable side-effect (by changing the Rat's denominator).

This CL changes Denom to return a new (not shared) *Int with value 1
in the rare case where the Rat was not initialized. This eliminates
the side effect and returns the correct denominator value.

While this is changing behavior of the API, the impact should now be
minor because together with (prior) CL https://golang.org/cl/202997,
which initializes Rats ASAP, Denom is unlikely used to access the
denominator of an uninitialized (and thus 0) Rat. Any operation that
will somehow set a Rat value will ensure that the denominator is not 0.

Fixes #33792.
Updates #3521.

Change-Id: I0bf15ac60513cf52162bfb62440817ba36f0c3fc
Reviewed-on: https://go-review.googlesource.com/c/go/+/203059
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-10-24 03:34:24 +00:00