1
0
mirror of https://github.com/golang/go synced 2024-11-08 04:36:11 -07:00
Commit Graph

326 Commits

Author SHA1 Message Date
Clément Chigot
e256afff51 all: move cmd/internal/xcoff to internal/xcoff
This commit moves cmd/internal/xcoff package to internal/xcoff because
it will be needed to add XCOFF support in go/internal/gccgoimporter.

Change-Id: Id12df0c438fb7db4a6a458fc1478480851bf7771
Reviewed-on: https://go-review.googlesource.com/c/152719
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-10 18:52:31 +00:00
bill_ofarrell
897e0807c3 crypto/elliptic: utilize faster z14 multiply/square instructions (when available)
In the s390x assembly implementation of NIST P-256 curve, utilize faster multiply/square
instructions introduced in the z14. These new instructions are designed for crypto
and are constant time. The algorithm is unchanged except for faster
multiplication when run on a z14 or later. On z13, the original mutiplication
(also constant time) is used.

P-256 performance is critical in many applications, such as Blockchain.

name            old time      new time     delta
BaseMultP256    24396 ns/op   21564 ns/op  1.13x
ScalarMultP256  87546 ns/op   72813 ns/op. 1.20x

Change-Id: I7e6d8b420fac56d5f9cc13c9423e2080df854bac
Reviewed-on: https://go-review.googlesource.com/c/146022
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Michael Munday <mike.munday@ibm.com>
2018-12-05 10:58:44 +00:00
Tobias Klauser
9e277f7d55 all: use "reports whether" consistently instead of "returns whether"
Follow-up for CL 147037 and after Brad noticed the "returns whether"
pattern during the review of CL 150621.

Go documentation style for boolean funcs is to say:

    // Foo reports whether ...
    func Foo() bool

(rather than "returns whether")

Created with:

    $ perl -i -npe 's/returns whether/reports whether/' $(git grep -l "returns whether" | grep -v vendor)

Change-Id: I15fe9ff99180ad97750cd05a10eceafdb12dc0b4
Reviewed-on: https://go-review.googlesource.com/c/150918
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-12-02 15:12:26 +00:00
Bryan C. Mills
2012227b01 vendor/golang_org/x: move to internal/x
Packages in vendor/ directories have a "vendor/" path prefix in GOPATH
mode, but intentionally do not in module mode. Since the import path
is embedded in the compiled output, changing that path invalidates
cache entries and causes cmd/go to try to rebuild (and reinstall) the
vendored libraries, which will fail if the directory containing those
libraries is read-only.

If I understood correctly, this is the approach Russ suggested as an
alternative to https://golang.org/cl/136138.

Fixes #27285
Fixes #26988

Change-Id: I8a2507fa892b84cde0a803aaa79e460723da572b
Reviewed-on: https://go-review.googlesource.com/c/147443
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2018-11-29 15:42:16 +00:00
Keith Randall
ca3749230b cmd/compile: allow bodyless function if it is linkname'd
In assembly free packages (aka "complete" or "pure go"), allow
bodyless functions if they are linkname'd to something else.

Presumably the thing the function is linkname'd to has a definition.
If not, the linker will complain. And linkname is unsafe, so we expect
users to know what they are doing.

Note this handles only one direction, where the linkname directive
is in the local package. If the linkname directive is in the remote
package, this CL won't help. (See os/signal/sig.s for an example.)

Fixes #23311

Change-Id: I824361b4b582ee05976d94812e5b0e8b0f7a18a6
Reviewed-on: https://go-review.googlesource.com/c/151318
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-11-26 20:00:59 +00:00
Hana Kim
6d5caf38e3 cmd/trace: revert internal/traceparser
The performance improvement is not as big as we hoped.
Until the API is feature complete, we postpone the release
and avoid added complexity.

This change was prepared by reverting all the changes affected
src/cmd/trace and src/internal/traceparser packages after
golang.org/cl/137635, and then bringing back MMU computation
APIs (originally in src/internal/traceparser) to the
src/internal/trace package.

Revert "cmd/trace: use new traceparser to parse the raw trace files"
This reverts https://golang.org/cl/145457
  (commit 08816cb8d7).

Revert "internal/traceparser: provide parser that uses less space and parses segments of runtime trace files"
This reverts https://golang.org/cl/137635
  (commit daaf361f74).

Change-Id: Ic2a068a7dbaf4053cd9674ca7bde9c58e74385b4
Reviewed-on: https://go-review.googlesource.com/c/150517
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2018-11-22 02:59:55 +00:00
Martin Möhrmann
2de53906e1 internal/cpu: move GODEBUGCPU options into GODEBUG
Change internal/cpu feature configuration to use
GODEBUG=cpu.feature1=value,cpu.feature2=value...
instead of GODEBUGCPU=feature1=value,feature2=value... .

This is not a backwards compatibility breaking change
since GODEBUGCPU was introduced in go1.11 as an
undocumented compiler experiment.

Fixes #28757

Change-Id: Ib21b3fed2334baeeb061a722ab1eb513d1137e87
Reviewed-on: https://go-review.googlesource.com/c/149578
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-14 21:47:50 +00:00
Austin Clements
ef7ce57ac2 internal/bytealg, runtime: provide linknames for pushed symbols
The internal/bytealg package defines several symbols in the runtime,
bytes, and strings packages in assembly, and the runtime package
defines symbols in reflect and sync/atomic. Currently, there's no
corresponding Go prototype for these symbols in the defining package.

We're going to start depending on Go prototypes in the same package as
their assembly definitions in order to provide ABI wrappers. Plus,
these are good documentation and colocate type information with
definitions, which could be useful for vet if it learned a little
about linkname.

This CL adds linknamed Go prototypes for all pushed symbols in
internal/bytealg and runtime.

For #27539.

Change-Id: I9b0c12d935a75bb6af46b6761180d451c00f11b8
Reviewed-on: https://go-review.googlesource.com/c/146820
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2018-11-12 20:27:16 +00:00
Keith Randall
ad4a58e315 strings,bytes: use inlineable function trampolines instead of linkname
Cleans things up quite a bit.

There's still a few more, like runtime.cmpstring, which might also
be worth fixing.

Change-Id: Ide18dd621efc129cc686db223f47fa0b044b5580
Reviewed-on: https://go-review.googlesource.com/c/148578
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2018-11-08 20:52:47 +00:00
Keith Randall
be5f646dab internal/syscall/unix: use libc calls on Darwin
Add unexported unlinkat, openat, and fstatat calls, so that
the internal/syscall/unix package can use them.

Change-Id: I1df81ecae6427211dd392ec68c9f020fe131a526
Reviewed-on: https://go-review.googlesource.com/c/148457
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-08 17:26:22 +00:00
Keith Randall
a3b01440fe syscall: implement syscalls on Darwin using libSystem
There are still some references to the bare Syscall functions
in the stdlib. I will root those out in a following CL.
(This CL is big enough as it is.)
Most are in vendor directories:

cmd/vendor/golang.org/x/sys/unix/
vendor/golang_org/x/net/route/syscall.go
syscall/bpf_bsd.go
syscall/exec_unix.go
syscall/flock.go

Update #17490

Change-Id: I69ab707811530c26b652b291cadee92f5bf5c1a4
Reviewed-on: https://go-review.googlesource.com/c/141639
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Elias Naur <elias.naur@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-07 20:27:01 +00:00
Elias Naur
ac277d9234 internal/traceparser: skip test on iOS
Change-Id: Ifc9581ba82a13f507c288282b517ebf8a5f93b4e
Reviewed-on: https://go-review.googlesource.com/c/148058
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-07 13:04:50 +00:00
Austin Clements
33563d1cfc internal/trace: support for mutator utilization distributions
This adds support for computing the quantiles of a mutator utilization
distribution.

Change-Id: Ia8b3ed14bf415c234e2f567360fd1b361d28bd40
Reviewed-on: https://go-review.googlesource.com/c/60799
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-05 19:10:27 +00:00
Austin Clements
bef4efc822 internal/trace: add "per-P" MMU analysis
The current MMU analysis considers all Ps together, so if, for
example, one of four Ps is blocked, mutator utilization is 75%.
However, this is less useful for understanding the impact on
individual goroutines because that one blocked goroutine could be
blocked for a very long time, but we still appear to have good
utilization.

Hence, this introduces a new flag that does a "per-P" analysis where
the utilization of each P is considered independently. The MMU is then
the combination of the MMU for each P's utilization function.

Change-Id: Id67b980d4d82b511d28300cdf92ccbb5ae8f0c78
Reviewed-on: https://go-review.googlesource.com/c/60797
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-05 19:10:23 +00:00
Austin Clements
27920c8ddc internal/trace: flags for what to include in GC utilization
Change-Id: I4ba963b003cb25b39d7575d423f17930d84f3f69
Reviewed-on: https://go-review.googlesource.com/c/60796
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-05 19:10:20 +00:00
Austin Clements
cbe04e8d70 internal/trace: track worst N mutator utilization windows
This will let the trace viewer show specifically when poor utilization
happened and link to specific instances in the trace.

Change-Id: I1f03a0f9d9a7570009bb15762e7b8b6f215e9423
Reviewed-on: https://go-review.googlesource.com/c/60793
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-05 19:10:16 +00:00
Austin Clements
52ee654b25 internal/trace: use banding to optimize MMU computation
This further optimizes MMU construction by first computing a
low-resolution summary of the utilization curve. This "band" summary
lets us compute the worst-possible window starting in each of these
low-resolution bands (even without knowing where in the band the
window falls). This in turn lets us compute precise minimum mutator
utilization only in the worst low-resolution bands until we can show
that any remaining bands can't possibly contain a worse window.

This slows down MMU construction for small traces, but these are
reasonably fast to compute either way. For large traces (e.g.,
150,000+ utilization changes) it's significantly faster.

Change-Id: Ie66454e71f3fb06be3f6173b6d91ad75c61bda48
Reviewed-on: https://go-review.googlesource.com/c/60792
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-05 19:10:09 +00:00
Austin Clements
c6c602a926 internal/trace: use MU slope to optimize MMU
This commit speeds up MMU construction by ~10X (and reduces the number
of windows considered by ~20X) by using an observation about the
maximum slope of the windowed mutator utilization function to advance
the window time in jumps if the window's current mean mutator
utilization is much larger than the current minimum.

Change-Id: If3cba5da0c4adc37b568740f940793e491e96a51
Reviewed-on: https://go-review.googlesource.com/c/60791
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-05 19:10:06 +00:00
Austin Clements
3053788cac cmd/trace: add minimum mutator utilization (MMU) plot
This adds an endpoint to the trace tool that plots the minimum mutator
utilization curve using information on mark assists and GC pauses from
the trace.

This commit implements a fairly straightforward O(nm) algorithm for
computing the MMU (and tests against an even more direct but slower
algorithm). Future commits will extend and optimize this algorithm.

This should be useful for debugging and understanding mutator
utilization issues like #14951, #14812, #18155. #18534, #21107,
particularly once follow-up CLs add trace cross-referencing.

Change-Id: Ic2866869e7da1e6c56ba3e809abbcb2eb9c4923a
Reviewed-on: https://go-review.googlesource.com/c/60790
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-11-05 19:10:04 +00:00
Brad Fitzpatrick
3813edf26e all: use "reports whether" consistently in the few places that didn't
Go documentation style for boolean funcs is to say:

    // Foo reports whether ...
    func Foo() bool

(rather than "returns true if")

This CL also replaces 4 uses of "iff" with the same "reports whether"
wording, which doesn't lose any meaning, and will prevent people from
sending typo fixes when they don't realize it's "if and only if". In
the past I think we've had the typo CLs updated to just say "reports
whether". So do them all at once.

(Inspired by the addition of another "returns true if" in CL 146938
in fd_plan9.go)

Created with:

$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true iff" | grep -v vendor)
$ perl -i -npe 's/returns true if/reports whether/' $(git grep -l "returns true if" | grep -v vendor)

Change-Id: Ided502237f5ab0d25cb625dbab12529c361a8b9f
Reviewed-on: https://go-review.googlesource.com/c/147037
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-02 22:47:58 +00:00
Bryan C. Mills
3a0d6091d6 internal/syscall/windows: add LockFileEx and UnlockFileEx for use in cmd/go
Updates #26794

Change-Id: Ic1d3078176721f3d2e5d8188c234383037babbaf
Reviewed-on: https://go-review.googlesource.com/c/145177
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2018-11-02 15:56:35 +00:00
Clément Chigot
5ee06f5471 internal/poll, os/exec, runtime: replace PollDescriptor by IsPollDescriptor
This commit changes poll.PollDescriptor by poll.IsPollDescriptor. This
is needed for OS like AIX which have more than one FD using inside their
netpoll implementation.

Change-Id: I49e12a8d74045c501e19fdd8527cf166a3c64850
Reviewed-on: https://go-review.googlesource.com/c/146938
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-02 13:43:23 +00:00
Tobias Klauser
cdad408069 os: add support for long path names on aix RemoveAll
Follow CL 146020 and enable RemoveAll based on Unlinkat and Openat on
aix.

Updates #27029

Change-Id: I78b34ed671166ee6fa651d5f2025b88548ee6c68
Reviewed-on: https://go-review.googlesource.com/c/146937
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Clément Chigot <clement.chigot@atos.net>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-02 13:21:12 +00:00
Dmitry Vyukov
21f7f01289 runtime: avoid runtimeNano call on a common netpoll path
runtimeNano is slower than nanotime, so pass the duration
to runtime_pollSetDeadline as is. netpoll can add nanotime itself.
Arguably a bit simpler because, say, a negative duration
clearly represents already expired timer, no need to compare to
nanotime again.
This may also fix an obscure corner case when a deadline in past
which happens to be nanotime 0 is confused with no deadline at all,
which are radically different things.
Also don't compute any durations and times if Time is zero
(currently we first compute everything and then reset d back to 0,
which is wasteful).

name                  old time/op  new time/op  delta
TCP4OneShotTimeout-6  17.1µs ± 0%  17.0µs ± 0%     ~     (p=0.421 n=5+5)
SetReadDeadline-6      230ns ± 0%   205ns ± 1%  -10.63%  (p=0.008 n=5+5)

Change-Id: I2aad699270289a5b9ead68f5e44ec4ec6d96baa0
Reviewed-on: https://go-review.googlesource.com/c/146344
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
2018-11-02 12:55:58 +00:00
Dmitry Vyukov
ff51353c38 runtime: move nanotime wrappers to time and poll packages
The nanotime wrappers in runtime introduce a bunch
of unnecessary code onto hot paths, e.g.:

0000000000449d70 <time.runtimeNano>:
  449d70:       64 48 8b 0c 25 f8 ff    mov    %fs:0xfffffffffffffff8,%rcx
  449d77:       ff ff
  449d79:       48 3b 61 10             cmp    0x10(%rcx),%rsp
  449d7d:       76 26                   jbe    449da5 <time.runtimeNano+0x35>
  449d7f:       48 83 ec 10             sub    $0x10,%rsp
  449d83:       48 89 6c 24 08          mov    %rbp,0x8(%rsp)
  449d88:       48 8d 6c 24 08          lea    0x8(%rsp),%rbp
  449d8d:       e8 ae 18 01 00          callq  45b640 <runtime.nanotime>
  449d92:       48 8b 04 24             mov    (%rsp),%rax
  449d96:       48 89 44 24 18          mov    %rax,0x18(%rsp)
  449d9b:       48 8b 6c 24 08          mov    0x8(%rsp),%rbp
  449da0:       48 83 c4 10             add    $0x10,%rsp
  449da4:       c3                      retq
  449da5:       e8 56 e0 00 00          callq  457e00 <runtime.morestack_noctxt>
  449daa:       eb c4                   jmp    449d70 <time.runtimeNano>

Move them to the corresponding packages which eliminates all of this.

name                  old time/op  new time/op  delta
TCP4OneShotTimeout-6  17.1µs ± 1%  17.0µs ± 0%  -0.66%  (p=0.032 n=5+5)
SetReadDeadline-6      234ns ± 1%   232ns ± 0%  -0.77%  (p=0.016 n=5+4)

Update #25729

Change-Id: Iee05027adcdc289ba895c5f5a37f154e451bc862
Reviewed-on: https://go-review.googlesource.com/c/146342
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-02 12:52:30 +00:00
Alex Brainman
f10815898c os: use CreateFile for Stat of symlinks
Stat uses Windows FindFirstFile + CreateFile to gather symlink
information - FindFirstFile determines if file is a symlink,
and then CreateFile follows symlink to capture target details.

Lstat only uses FindFirstFile.

This CL replaces current approach with just a call to CreateFile.
Lstat uses FILE_FLAG_OPEN_REPARSE_POINT flag, that instructs
CreateFile not to follow symlink. Other than that both Stat and
Lstat look the same now. New code is simpler.

CreateFile + GetFileInformationByHandle (unlike FindFirstFile)
does not report reparse tag of a file. I tried to ignore reparse
tag altogether. And it works for symlinks and mount points.
Unfortunately (see https://github.com/moby/moby/issues/37026),
files on deduped disk volumes are reported with
FILE_ATTRIBUTE_REPARSE_POINT attribute set and reparse tag set
to IO_REPARSE_TAG_DEDUP. So, if we ignore reparse tag, Lstat
interprets deduped volume files as symlinks. That is incorrect.
So I had to add GetFileInformationByHandleEx call to gather
reparse tag after calling CreateFile and GetFileInformationByHandle.

Fixes #27225
Fixes #27515

Change-Id: If60233bcf18836c147597cc17450d82f3f88c623
Reviewed-on: https://go-review.googlesource.com/c/143578
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Kirill Kolyshkin <kolyshkin@gmail.com>
2018-11-02 07:30:03 +00:00
Tobias Klauser
fa466a4f5a os: add support for long path names on freebsd RemoveAll
Follow CL 146020 and enable RemoveAll based on Unlinkat and Openat on
freebsd.

Since the layout of syscall.Stat_t changes in FreeBSD 12, Fstatat needs
a compatibility wrapper akin to Fstatat in x/sys/unix. See CL 138595 and
CL 136816 for details.

Updates #27029

Change-Id: I8851a5b7fa658eaa6e69a1693150b16d9a68f36a
Reviewed-on: https://go-review.googlesource.com/c/146597
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Yuval Pavel Zholkover <paulzhol@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-01 17:08:32 +00:00
Tobias Klauser
4d1e95bb63 os: add support for long path names on solaris RemoveAll
Follow CL 146020 and enable RemoveAll based on Unlinkat and Openat on
solaris.

Updates #27029

Change-Id: I0b0e92f4422fa960a13dcd3e9adb57cd23f09ed4
Reviewed-on: https://go-review.googlesource.com/c/145839
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-31 16:52:45 +00:00
Clément Chigot
cb07f492db internal/cpu, runtime: add CPU feature detection support for AIX
AIX doesn't have HWCAP/HWCAP2 variables like Linux. Therefore, it relies on
getsystemcfg syscall which can provide some information about the CPU.

Change-Id: Ic0dc927e80890d4bf8f0bdfb43fad1e2b890d7a0
Reviewed-on: https://go-review.googlesource.com/c/144959
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
2018-10-31 16:11:20 +00:00
Oliver Stenbom
e8ffb8a74c os: add support for long path names on unix RemoveAll
On unix systems, long enough path names will fail when performing syscalls
like `Lstat`. The current RemoveAll uses several of these syscalls, and so
will fail for long paths. This can be risky, as it can let users "hide"
files from the system or otherwise make long enough paths for programs
to fail. By using `Unlinkat` and `Openat` syscalls instead, RemoveAll is
safer on unix systems. Initially implemented for linux, darwin, dragonfly,
netbsd and openbsd. Not yet implemented on freebsd due to fstatat 64-bit
inode compatibility issues.

Fixes #27029

Co-authored-by: Giuseppe Capizzi <gcapizzi@pivotal.io>
Co-authored-by: Julia Nedialkova <yulia.nedyalkova@sap.com>

Change-Id: I978a6a4986878fe076d3c7af86e7927675624a96
GitHub-Last-Rev: 9235489c81
GitHub-Pull-Request: golang/go#28494
Reviewed-on: https://go-review.googlesource.com/c/146020
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-31 13:55:01 +00:00
Katie Hockman
0e0f798624 Revert "os: add support for long path names on unix RemoveAll"
This reverts commit 85143d3554.

Reason for revert: Breaking all Darwin and FreeBSD builds. Trybots did not pass for this.

Change-Id: I5494e14ad5ab9cf6e1e225a25b2e8b38f3359d13
Reviewed-on: https://go-review.googlesource.com/c/145897
Reviewed-by: Katie Hockman <katie@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-30 16:54:49 +00:00
Oliver Stenbom
85143d3554 os: add support for long path names on unix RemoveAll
On unix systems, long enough path names will fail when performing syscalls
like `Lstat`. The current RemoveAll uses several of these syscalls, and so
will fail for long paths. This can be risky, as it can let users "hide"
files from the system or otherwise make long enough paths for programs
to fail. By using `Unlinkat` and `Openat` syscalls instead, RemoveAll is
safer on unix systems. Initially implemented for linux, darwin, and several bsds.

Fixes #27029

Co-authored-by: Giuseppe Capizzi <gcapizzi@pivotal.io>
Co-authored-by: Julia Nedialkova <yulia.nedyalkova@sap.com>

Change-Id: Id9fcdf4775962b021b7ff438dc51ee6d16bb5f56
GitHub-Last-Rev: b30a621fe3
GitHub-Pull-Request: golang/go#27871
Reviewed-on: https://go-review.googlesource.com/c/137442
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-30 15:18:02 +00:00
Martin Möhrmann
8fc99d20f3 internal/cpu: remove unused and not required ppc64(le) feature detection
Minimum Go requirement for ppc64(le) architecture support is POWER8.
https://github.com/golang/go/wiki/MinimumRequirements#ppc64-big-endian

Reduce CPU features supported in internal/cpu to those needed to
test minimum requirements and cpu feature kernel support for ppc64(le).
Currently no internal/cpu feature variables are used to guard code
from using unsupported instructions. The IsPower9 feature variable
and detection is kept as it will soon be used to guard code execution.

Reducing the set of detected CPU features for ppc64(le) makes
implementing Go support for new operating systems easier as
CPU feature detection for ppc64(le) needs operating system support
(e.g. hwcap on Linux and getsystemcfg syscall on AIX).

Change-Id: Ic4c17b31610970e481cd139c657da46507391d1d
Reviewed-on: https://go-review.googlesource.com/c/145117
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-29 19:23:48 +00:00
Martin Möhrmann
3c9ad7cb41 internal/cpu: replace arch dependent with generic minimal feature test
Use information about required CPU features stored in the CPU feature
options slice to test if minimal CPU requirements are met instead
of hard coding this information in the tests directly.

Change-Id: I72d89b1cff305b8e751995d4230a2217e32f4236
Reviewed-on: https://go-review.googlesource.com/c/145118
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-28 14:03:47 +00:00
Elias Naur
536a7d6712 internal/traceparser: skip test on iOS
The iOS test harness only include files from the tested package or
below. Skip a test on iOS that required files outside the package.

Change-Id: Iaee7e488eb783b443f2b2b84d8be2de01227ab62
Reviewed-on: https://go-review.googlesource.com/c/144110
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Peter Weinberger <pjw@google.com>
2018-10-24 15:33:16 +00:00
Tobias Klauser
6155091042 internal/syscall/unix: omit unnecessary randomTrap check in GetRandom
The randomTrap const is initialized to a non-zero value for linux in
getrandom_linux_$GOARCH.go and for freebsd in getrandom_freebsd.go
directly since CL 16662. Thus, omit the unnecessary check.

Change-Id: Id20cd628dfe6fab9908fa5258c3132e3b422a6b4
Reviewed-on: https://go-review.googlesource.com/c/144108
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-24 13:43:00 +00:00
Martin Möhrmann
980340ade7 internal/cpu: add options and warnings for required cpu features
Updates #27218

Change-Id: I8603f3a639cdd9ee201c4f1566692e5b88877fc4
Reviewed-on: https://go-review.googlesource.com/c/144107
Run-TryBot: Martin Möhrmann <martisch@uos.de>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-24 06:27:53 +00:00
Lynn Boger
6994731ec2 internal/bytealg: improve asm for memequal on ppc64x
This includes two changes to the memequal function.

Previously the asm implementation on ppc64x for Equal called the internal
function memequal using a BL, whereas the other asm implementations for
bytes functions on ppc64x used BR. The BR is preferred because the BL
causes the calling function to stack a frame. This changes Equal so it
uses BR and is consistent with the others.

This also uses vsx instructions where possible to improve performance
of the compares for sizes over 32.

Here are results from the sizes affected:

Equal/32             8.40ns ± 0%     7.66ns ± 0%    -8.81%  (p=0.029 n=4+4)
Equal/4K              193ns ± 0%      144ns ± 0%   -25.39%  (p=0.029 n=4+4)
Equal/4M              346µs ± 0%      277µs ± 0%   -20.08%  (p=0.029 n=4+4)
Equal/64M            7.66ms ± 1%     7.27ms ± 0%    -5.10%  (p=0.029 n=4+4)

Change-Id: Ib6ee2cdc3e5d146e2705e3338858b8e965d25420
Reviewed-on: https://go-review.googlesource.com/c/143060
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
Reviewed-by: David Chase <drchase@google.com>
2018-10-23 19:22:44 +00:00
Peter Weinberger
daaf361f74 internal/traceparser: provide parser that uses less space and parses segments of runtime trace files
Traceparser generally takes 20-30% less space than internal/trace. The only
user of these pakcages is cmd/trace, and the new package lets it handle some
trace files that were too large. The new parser will also convert segments
of the raw trace file (e.g. the last 10 seconds) to Events. Trace files from
go 1.8 and before are not supported.

Change-Id: If83fa183246db8f75182ccd3ba8df07673c0ebd0
Reviewed-on: https://go-review.googlesource.com/c/137635
Run-TryBot: Peter Weinberger <pjw@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
2018-10-23 14:00:14 +00:00
Clément Chigot
93eded0297 internal/syscall/unix, net: improve interface_aix.go
This commit improves the interface_aix.go file, based on feedbacks about
CL 138724.

To retrieve MTU, ioctl is needed. It's implemented inside
internal/syscall/unix.

Change-Id: Ic583d26b93935a32a5f1eb5a2170b86e80a4a85e
Reviewed-on: https://go-review.googlesource.com/c/142157
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-23 02:38:52 +00:00
Rob Pike
a440cc0d70 fmt: print maps in key-sorted order
For easier testing, change the way maps are printed so they
appear in a consistent order between runs. Do this by printing
them in key-sorted order.

To do this, we add a package at the root, internal/fmtsort,
that implements a general mechanism for sorting map keys
regardless of their type. This is a little messy and probably
slow, but formatted printing of maps has never been fast and
is already always reflection-driven.

The rules are:

The ordering rules are more general than with Go's < operator:

 - when applicable, nil compares low
 - ints, floats, and strings order by <
 - NaN compares less than non-NaN floats
 - bool compares false before true
 - complex compares real, then imag
 - pointers compare by machine address
 - channel values compare by machine address
 - structs compare each field in turn
 - arrays compare each element in turn.
 - interface values compare first by reflect.Type describing the concrete type
   and then by concrete value as described in the previous rules.

The new package is internal because we really do not want
everyone using this to sort things. It is slow, not general, and
only suitable for the subset of types that can be map keys.

Also use the package in text/template, which already had a
weaker version of this mechanism.

This change requires adding a dependency on sort to the fmt
package, but that isn't disruptive to the dependency tree.

Fixes #21095

Change-Id: Ia602115c7de5d95993dbd609611d8bd96e054157
Reviewed-on: https://go-review.googlesource.com/c/142737
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2018-10-18 21:12:24 +00:00
Martin Möhrmann
3e0227f6a0 internal/cpu: add invalid option warnings and support to enable cpu features
This CL adds the ability to enable the cpu feature FEATURE by specifying
FEATURE=on in GODEBUGCPU. Syntax support to enable cpu features is useful
in combination with a preceeding all=off to disable all but some specific
cpu features. Example:

GODEBUGCPU=all=off,sse3=on

This CL implements printing of warnings for invalid GODEBUGCPU settings:
- requests enabling features that are not supported with the current CPU
- specifying values different than 'on' or 'off' for a feature
- settings for unkown cpu feature names

Updates #27218

Change-Id: Ic13e5c4c35426a390c50eaa4bd2a408ef2ee21be
Reviewed-on: https://go-review.googlesource.com/c/141800
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-10-15 21:46:44 +00:00
Akhil Indurti
bb3bf5bb53 internal/cpu: expose ARM feature flags for FMA
This change exposes feature flags needed to implement an FMA intrinsic
on ARM CPUs via auxv's HWCAP bits. Specifically, it exposes HasVFPv4 to
detect if an ARM processor has the fourth version of the vector floating
point unit. The relevant instruction for this CL is VFMA, emitted in Go
as FMULAD.

Updates #26630.

Change-Id: Ibbc04fb24c2b4d994f93762360f1a37bc6d83ff7
Reviewed-on: https://go-review.googlesource.com/c/126315
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Martin Möhrmann <moehrmann@google.com>
2018-10-15 10:57:04 +00:00
Martin Möhrmann
4fb8b1de3c internal/cpu: use 'off' for disabling cpu capabilities instead of '0'
Updates #27218

Change-Id: I4ce20376fd601b5f958d79014af7eaf89e9de613
Reviewed-on: https://go-review.googlesource.com/c/141818
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-12 16:44:26 +00:00
Tobias Klauser
d82e51a119 internal/poll: add FD.Fsync on aix
Follow-up for CL 138717. This fixes the build of the os package on
aix.

Change-Id: I879b9360e71837ab622ae3a7b6144782cf5a9ce7
Reviewed-on: https://go-review.googlesource.com/c/141797
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-12 16:10:54 +00:00
Martin Möhrmann
a5248acd91 internal/cpu: enable support for GODEBUGCPU in non-experimental builds
Enabling GODEBUGCPU without the need to set GOEXPERIMENT=debugcpu  enables
trybots and builders to run tests for GODEBUGCPU features in upcoming CLs
that will implement the new syntax and features for non-experimental
GODEBUGCPU support from proposal golang.org/issue/27218.

Updates #27218

Change-Id: Icc69e51e736711a86b02b46bd441ffc28423beba
Reviewed-on: https://go-review.googlesource.com/c/141817
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-12 15:40:45 +00:00
Clément Chigot
86e251c938 internal/syscall: add AIX operating system
This commit adds AIX operating system to internal/syscall package for ppc64
architecture.

Updates: #25893

Change-Id: I5c3a9d4403ca170a7e894e06e68b83387d09b816
Reviewed-on: https://go-review.googlesource.com/c/138718
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-03 21:28:26 +00:00
Clément Chigot
816e493495 internal/poll: add AIX operating system
This commit adds AIX operating system to internal/poll package for ppc64
architecture.

Updates: #25893

Change-Id: I9b1da9255012de58f16547c1b18f8840485da170
Reviewed-on: https://go-review.googlesource.com/c/138717
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-10-03 21:15:18 +00:00
Zhou Peng
b8ac64a581 all: this big patch remove whitespace from assembly files
Don't worry, this patch just remove trailing whitespace from
assembly files, and does not touch any logical changes.

Change-Id: Ia724ac0b1abf8bc1e41454bdc79289ef317c165d
Reviewed-on: https://go-review.googlesource.com/c/113595
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-10-03 15:28:51 +00:00
Ian Lance Taylor
b83ef36d6a go/build: move isStandardPackage to new internal/goroot package
The module code in cmd/go sometimes needs to know whether it is
looking at a standard package, and currently uses gc-specific code for
that. This CL moves the existing isStandardPackage code in the
go/build package, which works for both gc and gccgo, into a new
internal/goroot package so that cmd/go can call it. The changes to
cmd/go will be in a subsequent CL.

Change-Id: Ic1ce4c022a932c6b3e99fa062631577085cc6ecb
Reviewed-on: https://go-review.googlesource.com/137435
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-09-25 22:07:43 +00:00