1
0
mirror of https://github.com/golang/go synced 2024-11-23 02:00:03 -07:00
Commit Graph

45674 Commits

Author SHA1 Message Date
Matthew Dempsky
5736eb0013 cmd/compile: support inlining of type switches
This CL adds support for inlining type switches, including exporting
and importing them.

Type switches are represented mostly the same as expression switches.
However, if the type switch guard includes a short variable
declaration, then there are two differences: (1) there's an ONONAME
(in the OTYPESW's Left) to represent the overall pseudo declaration;
and (2) there's an ONAME (in each OCASE's Rlist) to represent the
per-case variables.

For simplicity, this CL simply writes out each variable separately
using iimport/iiexport's normal Vargen mechanism for disambiguating
identically named variables within a function. This could be improved
somewhat, but inlinable type switches are probably too uncommon to
merit the complexity.

While here, remove "case OCASE" from typecheck1. We only type check
"case" clauses as part of a "select" or "switch" statement, never as
standalone statements.

Fixes #37837

Change-Id: I8f42f6c9afdd821d6202af4a6bf1dbcbba0ef424
Reviewed-on: https://go-review.googlesource.com/c/go/+/266203
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
2020-11-06 20:49:11 +00:00
Russ Cox
362d25f2c8 io/fs: add WalkDir
This commit is a copy of filepath.WalkDir adapted to use fs.FS
instead of the native OS file system. It is the last implementation
piece of the io/fs proposal.

The original io/fs proposal was to adopt filepath.Walk, but we
have since introduced the more efficient filepath.WalkDir (#42027),
so this CL adopts that more efficient option instead.

(The changes in path/filepath bring the two copies more in line
with each other. The main change is unembedding the field
in statDirEntry, so that the fs.DirEntry passed to the WalkDirFunc
for the root of the tree does not have any extra methods.)

For #41190.

Change-Id: I9359dfcc110338c0ec64535f22cafb38d0b613a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/243916
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-11-06 19:42:05 +00:00
Russ Cox
d21af00dd2 path/filepath: add WalkDir
WalkDir is like Walk but can use ReadDir to read directories,
instead of Readdirnames + Lstat on every entry,
which is usually a significant performance improvement.
(The Lstat can still happen if the walk function calls d.Info.)

Fixes #42027.

[Replay of CL 266240 after it was reverted due to accidentally
enabling on Windows a test that does not work on Windows.
The original code only ran the test on os.Getuid() > 0.
The rolled-back CL skipped the test on os.Getuid() == 0.
But on Windows, os.Getuid(), it turns out, always returns -1.
So what looked like a test for root was also excluding Windows.
This CL revises the test to skip Windows explicitly.]

Change-Id: I9b3661013d6449b7486532445d934ae91e5393ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/267887
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-06 15:33:23 +00:00
Michael Munday
854e892ce1 cmd/compile: optimize shift pairs and masks on s390x
Optimize combinations of left and right shifts by a constant value
into a 'rotate then insert selected bits [into zero]' instruction.
Use the same instruction for contiguous masks since it has some
benefits over 'and immediate' (not restricted to 32-bits, does not
overwrite source register).

To keep the complexity of this change under control I've only
implemented 64 bit operations for now.

There are a lot more optimizations that can be done with this
instruction family. However, since their function overlaps with other
instructions we need to be somewhat careful not to break existing
optimization rules by creating optimization dead ends. This is
particularly true of the load/store merging rules which contain lots
of zero extensions and shifts.

This CL does interfere with the store merging rules when an operand
is shifted left before it is stored:

  binary.BigEndian.PutUint64(b, x << 1)

This is unfortunate but it's not critical and somewhat complex so
I plan to fix that in a follow up CL.

file      before    after     Δ       %
addr2line 4117446   4117282   -164    -0.004%
api       4945184   4942752   -2432   -0.049%
asm       4998079   4991891   -6188   -0.124%
buildid   2685158   2684074   -1084   -0.040%
cgo       4553732   4553394   -338    -0.007%
compile   19294446  19245070  -49376  -0.256%
cover     4897105   4891319   -5786   -0.118%
dist      3544389   3542785   -1604   -0.045%
doc       3926795   3927617   +822    +0.021%
fix       3302958   3293868   -9090   -0.275%
link      6546274   6543456   -2818   -0.043%
nm        4102021   4100825   -1196   -0.029%
objdump   4542431   4548483   +6052   +0.133%
pack      2482465   2416389   -66076  -2.662%
pprof     13366541  13363915  -2626   -0.020%
test2json 2829007   2761515   -67492  -2.386%
trace     10216164  10219684  +3520   +0.034%
vet       6773956   6773572   -384    -0.006%
total     107124151 106917891 -206260 -0.193%

Change-Id: I7591cce41e06867ba10a745daae9333513062746
Reviewed-on: https://go-review.googlesource.com/c/go/+/233317
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Michael Munday <mike.munday@ibm.com>
2020-11-06 10:45:31 +00:00
Joel Sing
b7e0adfee2 cmd/dist: remove openbsd/mips64 from incomplete ports
Remove openbsd/mips64 from incomplete ports lists - all of the necessary code
has landed and we want to run tests so we can see/deal with remaining failures.

Update #40995

Change-Id: I5d4f89af82ff3abe57570a9a8abf889498093d32
Reviewed-on: https://go-review.googlesource.com/c/go/+/267606
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-06 07:55:52 +00:00
Jonathan Swinney
ecc3f5112e cmd/compile: improve atomic swap intrinsics on arm64
ARMv8.1 has added new instructions for atomic memory operations. This
change builds on the previous change which added support for atomic add,
0a7ac93c27, to include similar support for
atomic-compare-and-swap, atomic-swap, atomic-or, and atomic-and
intrinsics. Since the new instructions are not guaranteed to be present,
we guard their usages with a branch on a CPU feature.

Peformance on an ARMv8.1 machine:
name                 old time/op  new time/op  delta
CompareAndSwap-16    37.9ns ±16%  24.1ns ± 4%  -36.44%  (p=0.000 n=10+9)
CompareAndSwap64-16  38.6ns ±15%  24.1ns ± 3%  -37.47%  (p=0.000 n=10+10)

name       old time/op  new time/op  delta
Swap-16    46.9ns ±32%  12.5ns ± 6%  -73.40%  (p=0.000 n=10+10)
Swap64-16  53.4ns ± 1%  12.5ns ± 6%  -76.56%  (p=0.000 n=10+10)

name            old time/op  new time/op  delta
Or8-16          8.81ns ± 0%  5.61ns ± 0%  -36.32%  (p=0.000 n=10+10)
Or-16           7.21ns ± 0%  5.61ns ± 0%  -22.19%  (p=0.000 n=10+10)
Or8Parallel-16  59.8ns ± 3%  12.5ns ± 2%  -79.10%  (p=0.000 n=10+10)
OrParallel-16   51.7ns ± 3%  12.5ns ± 2%  -75.84%  (p=0.000 n=10+10)

name             old time/op  new time/op  delta
And8-16          8.81ns ± 0%  5.61ns ± 0%  -36.32%  (p=0.000 n=10+10)
And-16           7.21ns ± 0%  5.61ns ± 0%  -22.19%  (p=0.000 n=10+10)
And8Parallel-16  59.1ns ± 6%  12.8ns ± 3%  -78.33%  (p=0.000 n=10+10)
AndParallel-16   51.4ns ± 7%  12.8ns ± 3%  -75.03%  (p=0.000 n=10+10)

Performance on an ARMv8.0 machine (no atomics instructions):
name                 old time/op  new time/op  delta
CompareAndSwap-16    61.3ns ± 0%  62.4ns ± 0%  +1.70%  (p=0.000 n=8+9)
CompareAndSwap64-16  62.0ns ± 3%  61.3ns ± 2%    ~     (p=0.093 n=10+10)

name       old time/op  new time/op  delta
Swap-16     127ns ± 2%   131ns ± 2%  +2.91%  (p=0.001 n=10+10)
Swap64-16   128ns ± 1%   131ns ± 2%  +2.43%  (p=0.001 n=10+10)

name            old time/op  new time/op  delta
Or8-16          14.9ns ± 0%  15.3ns ± 0%  +2.68%  (p=0.000 n=10+10)
Or-16           11.8ns ± 0%  12.3ns ± 0%  +4.24%  (p=0.000 n=10+10)
Or8Parallel-16   137ns ± 1%   144ns ± 1%  +4.97%  (p=0.000 n=10+10)
OrParallel-16    128ns ± 1%   136ns ± 1%  +6.34%  (p=0.000 n=10+10)

name             old time/op  new time/op  delta
And8-16          14.9ns ± 0%  15.3ns ± 0%  +2.68%  (p=0.000 n=10+10)
And-16           11.8ns ± 0%  12.3ns ± 0%  +4.24%  (p=0.000 n=10+10)
And8Parallel-16   134ns ± 2%   141ns ± 1%  +5.29%  (p=0.000 n=10+10)
AndParallel-16    125ns ± 2%   134ns ± 1%  +7.10%  (p=0.000 n=10+10)

Fixes #39304

Change-Id: Idaca68701d4751650be6b4bedca3d57f51571712
Reviewed-on: https://go-review.googlesource.com/c/go/+/234217
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: fannie zhang <Fannie.Zhang@arm.com>
2020-11-05 23:21:33 +00:00
Than McIntosh
8e5778ed70 cmd/link: report error if builtin referenced but not defined
When the compiler refers to a runtime builtin, it emits an indexed
symbol reference in the object file via predetermined/preassigned ID
within the PkgIdxBuiltin pseudo-package. At link time when the loader
encounters these references, it redirects them to the corresponding
defined symbol in the runtime package. This redirection process
currently assumes that if a runtime builtin is referenced, we'll
always have a definition for it. This assumption holds in most cases,
however for the builtins "runtime.racefuncenter" and
"runtime.racefuncexit", we'll only see definitions if the runtime
package we're linking against was built with "-race".

In the bug in question, build passes "-gcflags=-race" during
compilation of the main package, but doesn't pass "-race" directly to
'go build', and as a result the final link combines a
race-instrumented main with a non-race runtime; this results in R_CALL
relocations with zero-valued target symbols, resulting in a panic
during stack checking.

This patch changes the loader's resolve method to detect situations
where we're asking for builtin "runtime.X", but the runtime package
read in doesn't contain a definition for X.

Fixes #42396.

Change-Id: Iafd38bd3b0f7f462868d120ccd4d7d1b88b27436
Reviewed-on: https://go-review.googlesource.com/c/go/+/267881
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-05 22:14:40 +00:00
Rob Findley
2822bae168 go/types: add unexported start and end positions to type checker errors
Tools often need to associate errors not with a single position, but
with a span of source code. For example, gopls currently estimates
diagnostic spans using heuristics to expand the positions reported by
the type checker to surrounding source code. Unfortunately this is often
inaccurate.

This CL lays the groundwork to solve this within go/types by adding a
start and end position to type checker errors. This is an experimental
API, both because we are uncertain of the ideal representation for these
spans and because their initial positioning is naive. In most cases this
CL simply expands errors to the surrounding ast.Node being typechecked,
if available. This might not be the best error span to present to the
user. For these reasons the API is unexported -- gopls can read these
positions using reflection, allowing us to gain experience and improve
them during the next development cycle.

For golang/go#42290

Change-Id: I39a04d70ea2bb2134b4d4c937f32b2ddb4456430
Reviewed-on: https://go-review.googlesource.com/c/go/+/265250
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
2020-11-05 20:35:00 +00:00
Michael Matloob
d508d86cf1 cmd/go: account for flags when parsing regexps in TestScript
Test script expects the regexp argument for stdout, stderr, and cmp
to be the first argument after the command, but that might not be the
case if the -q or -count flags are provided. Treat the first argument
after a flag as a regexp instead.

For #39958

Change-Id: I369926109ec10cca8b2c3baca27e7a3f7baf364b
Reviewed-on: https://go-review.googlesource.com/c/go/+/267877
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-05 19:33:55 +00:00
Michael Pratt
370682ae98 runtime: disable preemption in startm
startm contains a critical section from when it takes ownership of a P
(either on function entry or call to pidleput) until it wakes the M
receiving the P. If preempted in this critical section, the owned P is
left in limbo. If preempted for a GC stop, there will be nothing to stop
the owned P and STW will wait forever.

golang.org/cl/232298 introduced the first call to startm that is not on
the system stack (via a wakep call), introducing the possibility of
preemption. Disable preemption in startm to ensure this remains
non-preemptible.

Since we're not always on the system stack anymore, we also need to be
careful in allocm.

Updates #42237

Change-Id: Icb95eef9eb262121856485316098331beea045da
Reviewed-on: https://go-review.googlesource.com/c/go/+/267257
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Trust: Michael Pratt <mpratt@google.com>
2020-11-05 19:22:02 +00:00
Bryan C. Mills
06538fa723 cmd/go/internal/modget: resolve paths at the requested versions
Previously, we resolved each argument to 'go get' to a package path or
module path based on what was in the build list at existing versions,
even if the argument specified a different version explicitly. That
resulted in bugs like #37438, in which we variously resolved the wrong
version or guessed the wrong argument type for what is unambiguously a
package argument at the requested version.

We were also using a two-step upgrade/downgrade algorithm, which could
not only upgrade more that is strictly necessary, but could also
unintentionally upgrade *above* the requested versions during the
downgrade step.

This change instead uses an iterative approach, with an explicit
disambiguation step for the (rare) cases where an argument could match
the same package path in multiple modules. We use a hook in the
package loader to halt package loading as soon as an incorrect version
is found — preventing over-resolving — and verify that the result
after applying downgrades successfully obtained the requested versions
of all modules.

Making 'go get' be correct and usable is especially important now that
we are defaulting to read-only mode (#40728), for which we are
recommending 'go get' more heavily.

While I'm in here refactoring, I'm also reworking the API boundary
between the modget and modload packages. Previously, the modget
package edited the build list directly, and the modload package
accepted the edited build list without validation. For lazy loading
(#36460), the modload package will need to maintain additional
metadata about the requirement graph, so it needs tighter control over
the changes to the build list.

As of this change, modget no longer invokes MVS directly, but instead
goes through the modload package. The resulting API gives clearer
reasons in case of updates, which we can use to emit more useful
errors.

Fixes #37438
Updates #36460
Updates #40728

Change-Id: I596f0020f3795870dec258147e6fc26a3292c93a
Reviewed-on: https://go-review.googlesource.com/c/go/+/263267
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2020-11-05 17:52:17 +00:00
Bryan C. Mills
67bf1c9979 cmd/go/internal/modload: fix (*mvsReqs).Max when the second argument is the empty string
As far as I can tell, this bug had gone unnoticed because everything
that uses Max so far happened to only ever present the empty string as
the first argument.

For #37438

Change-Id: Ie8c42313157d2c2c17e4058c53b5bb026b95a1c1
Reviewed-on: https://go-review.googlesource.com/c/go/+/266860
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-11-05 16:47:53 +00:00
Bryan C. Mills
a19a4dcb98 cmd/go/internal/mvs: in Upgrade, pass upgrades to buildList as upgrades
This has no impact on the resulting build list, but provides clearer
diagnostics if reqs.Required returns an error for one of the upgraded
modules.

For #37438

Change-Id: I5cd8f72a9b7b9a0b185e1a728f46fefbd2f09b4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/266897
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-11-05 16:47:45 +00:00
Bryan C. Mills
4a3339223c cmd/go/internal/mvs: test a downgrade where the target explicitly requires itself
Also clean up the test assertions, and add a check for assertions
missing function invocations (there was one).

For #37438

Change-Id: Iafbfeae2c25217eac894181e01480b25b7cffbd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/266859
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-11-05 16:47:29 +00:00
Bryan C. Mills
0c861724a0 cmd/go/internal/modload: add structured errors for queries matching the main module
For #37438

Change-Id: I7df80ae0917b0b4ecad98947da39ddf8554b07c7
Reviewed-on: https://go-review.googlesource.com/c/go/+/266717
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-05 16:47:18 +00:00
Bryan C. Mills
04b5b4f740 cmd/go/internal/modload: return a module-only result from QueryPattern
This allows a single QueryPattern call to resolve a path that could be
either a package or a module. It is important to be able to make a
single QueryPattern call — rather than a QueryPattern followed by a
Query for the specific module path — to provide appropriate fallback
behavior: if the proxy returns package results but does not contain a
module result, we don't want to fall back to the next proxy to look
for the (probably-nonexistent) module.

For #37438

Change-Id: I419b8bb3ab4565f443bb5cee9a8b206f453b9801
Reviewed-on: https://go-review.googlesource.com/c/go/+/266657
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-05 16:46:56 +00:00
Brad Fitzpatrick
40f0359d52 runtime: avoid a bit of unneeded work when MemProfileRate==1
Change-Id: I1dc355bcaeb0e5fb06a7fddc4cf5db596d22e0b3
Reviewed-on: https://go-review.googlesource.com/c/go/+/236148
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-11-05 16:43:34 +00:00
Vee Zhang
db8142fb86 runtime: fix file references in hiter's comments
The file "cmd/internal/gc/range.go" does not exist, but should be
"cmd/compile/internal/gc/range.go".

Change-Id: I26e5560b9d0b7eea8502c6b375e45fc87aed1276
GitHub-Last-Rev: 5f19dca7e9
GitHub-Pull-Request: golang/go#42391
Reviewed-on: https://go-review.googlesource.com/c/go/+/267837
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2020-11-05 16:32:15 +00:00
Brad Fitzpatrick
8ab8125fbd os: remove unused variable in unix implementation of File.readdir
Change-Id: I0dd8a325bce6ed12d1ec1dc206ded62398925aef
Reviewed-on: https://go-review.googlesource.com/c/go/+/267758
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2020-11-05 16:20:01 +00:00
Dmitri Shuralyov
34c09695db cmd/go: revert "add GOVCS setting to control version control usage"
This reverts CL 266420.

Reason for revert: tests aren't passing on linux-{386,amd64}-longtest.

Change-Id: Icec47cded795a51ef7569dfb2d93d9211b4fb578
Reviewed-on: https://go-review.googlesource.com/c/go/+/267799
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-05 15:16:57 +00:00
Dmitri Shuralyov
74ec40fc8a path/filepath: revert "add WalkDir"
This reverts CL 266240.

Reason for revert: tests aren't passing on windows-amd64-longtest.

Change-Id: If323c6254a42aff0418e2c0a9531f3d4c829a242
Reviewed-on: https://go-review.googlesource.com/c/go/+/267798
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-05 14:54:35 +00:00
Rob Findley
05568315f2 go/types: simplify error messages for untyped value assignability
CL 242083 corrected an inaccurate error message related to the
assignability of untyped constant values. Previously the error message
was of the form "cannot convert ... to ...", which is misleading when
there is no explicit conversion in the syntax. The new error message
corrected this to "cannot use ... as ... in ...", but also appended an
inner error message that can be quite verbose. For example:

  cannot use "123" (untyped string constant) as int value in assignment:
  cannot convert "123" (untyped string constant) to int"

This might be more accurate, but is a regression in readability. Correct
this by only including the inner error message in the rare cases where
it is helpful: if the constant value overflows or is truncated.

For golang/go#22070

Change-Id: I8b8ee6ef713f64facc319894be09398b0b5ea500
Reviewed-on: https://go-review.googlesource.com/c/go/+/267717
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-05 12:57:20 +00:00
Martin Möhrmann
3510a1e32c internal/cpu: fix and cleanup ARM64 cpu feature fields and options
Remove all cpu features from the ARM64 struct that are not initialized
to reduce cache lines used and to avoid those features being
accidentially used without actual detection if they are present.

Add missing option to mask the CPUID feature.

Change-Id: I94bf90c0655de1af2218ac72117ac6c52adfc289
Reviewed-on: https://go-review.googlesource.com/c/go/+/267658
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Martin Möhrmann <moehrmann@google.com>
2020-11-05 10:46:08 +00:00
Emmanuel T Odeke
3ef8562c9c net/mail: avoid ParseDate confusion if day name includes "T"
Fixes the check for RFC 5322 "obsolete time zone" to ensure
that we correctly extract the entire date from the "T" of the
implied time zone.

Obsolete Time zones come in the form:
* GMT
* PST
* MDT
etc, as per Section 4.3 of RFC 5322,
https://tools.ietf.org/html/rfc5322#section-4.3.

The prior check from CL 117596 erronenously used strings.Index
which selects the first "T", and that meant that dates containing
days "Tue" or "Thu" could not be parsed.

We also now deal with "T" in the CFWS "Comment Folding White Space".

Thus we'll now accept dates:
* Thu, 20 Nov 1997 09:55:06 MDT
* Thu, 20 Nov 1997 09:55:06 MDT (MDT)
* Fri, 21 Nov 1997 09:55:06 MDT (This comment)
* Fri, 21 Nov 1997 09:55:06 MDT (MDT

Fixes #39260

Change-Id: I6d59d99bc4f05a82582c826b5c5a080a25fd999b
Reviewed-on: https://go-review.googlesource.com/c/go/+/235200
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2020-11-05 02:48:05 +00:00
Johan Knutzen
1e3b535b6e syscall: expose bInheritHandles of CreateProcess
Certain use cases require this parameter to be false. This includes
spawning a child process in a different windows session than session 0.

Docs regarding the behavior of this parameter to CreateProcess:
https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessa

Fixes #42098

Change-Id: If998f57d6f2962824aacbee75e1b508b255ab293
GitHub-Last-Rev: 584eb13e36
GitHub-Pull-Request: golang/go#41957
Reviewed-on: https://go-review.googlesource.com/c/go/+/261917
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
2020-11-05 02:28:14 +00:00
Russ Cox
c018eec1f3 cmd/go: add GOVCS setting to control version control usage
The go command runs commands like git and hg to download modules.
In the past, we have had problems with security bugs in version
control systems becoming security bugs in “go get”.

The original modules draft design removed use of these commands
entirely, saying:

> We want to move away from invoking version control tools such as bzr,
> fossil, git, hg, and svn to download source code. These fragment the
> ecosystem: packages developed using Bazaar or Fossil, for example, are
> effectively unavailable to users who cannot or choose not to install
> these tools. The version control tools have also been a source of
> exciting security problems. It would be good to move them outside the
> security perimeter.

The removal of these commands was not possible in the end: being able
to fetch directly from Git repos is too important, especially for
closed source. But the security exposure has not gone away.
We remain vulnerable to problems in VCS systems, especially the less
scrutinized ones.

This change adds a GOVCS setting to let users control which version
control systems are allowed by default.

It also changes the default allowed version control systems to git and hg
for public code and any version control system for private code
(import path or module path matched by the GOPRIVATE setting).

See the changes in alldocs.go for detailed documentation.
See #41730 for proposal and discussion.

Fixes #41730.

Change-Id: I1999ddf7445b36a7572965be5897c7a1ff7f4265
Reviewed-on: https://go-review.googlesource.com/c/go/+/266420
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-05 00:21:39 +00:00
Derek Parker
63fd764502 cmd/internal/obj: add prologue_end DWARF stmt for ppc64
This patch adds a prologue_end statement to the DWARF information for
the ppc64 arch.

Prologue end is used by the Delve debugger in order to determine where
to set a breakpoint to avoid the stacksplit prologue.

Updates #36612

Change-Id: Ifb16c1476fe716a0bf493c5486d1d88ebe8d0253
GitHub-Last-Rev: 77a217206d
GitHub-Pull-Request: golang/go#42261
Reviewed-on: https://go-review.googlesource.com/c/go/+/266019
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Alessandro Arzilli <alessandro.arzilli@gmail.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-04 23:24:22 +00:00
Russ Cox
fd841f6536 path/filepath: add WalkDir
WalkDir is like Walk but can use ReadDir to read directories,
instead of Readdirnames + Lstat on every entry,
which is usually a significant performance improvement.
(The Lstat can still happen if the walk function calls d.Info.)

Fixes #42027.

Change-Id: Ie11024b23be2656e320d41fd81ff0d8810aa729e
Reviewed-on: https://go-review.googlesource.com/c/go/+/266240
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-11-04 21:45:25 +00:00
Paul E. Murphy
5ed81a3d14 cmd/asm: fix rlwnm reg,reg,const,reg encoding on ppc64
The wrong value for the first reg parameter was selected.
Likewise the wrong opcode was selected.  This should match
rlwnm (rrr type), not rlwinm (irr type).

Similarly, fix the optab matching rules so clrlslwi does
not match reg,reg,const,reg arguments.  This is not a valid
operand combination for clrlslwi.

Fixes #42368

Change-Id: I4eb16d45a760b9fd3f497ef9863f82465351d39f
Reviewed-on: https://go-review.googlesource.com/c/go/+/267421
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
2020-11-04 21:26:23 +00:00
Russ Cox
f532f19d94 os: avoid nil returns from Readdirnames, Readdir, ReadDir
The refactoring of this code while adding ReadDir stopped
pre-allocating a 100-entry slice for the results.
That seemed like a good idea in general, since many
directories have nowhere near 100 entries, but it had the
side effect of returning a nil slice for an empty directory.

Some “golden” tests that are too sensitive about nil vs not
inside Google broke because Readdirnames(-1) was now
returning nil instead of []string{} on an empty directory.
It seems likely there are other such tests in the wild, and
it doesn't seem worth breaking them.

This commit restores the non-nil-ness of the old result,
without restoring the excessive preallocation.

Fixes #42367.

Change-Id: I2be72030ac703346e859a97c2d4e456fadfce9b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/267637
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-11-04 21:17:01 +00:00
Cherry Zhang
e4b4e4b733 race.bash: add darwin/arm64
Race detector support was added in previous CLs.

Updates #38485.

Change-Id: Ie9ae69406ff4770f573c2287bfbbb0421769b3ce
Reviewed-on: https://go-review.googlesource.com/c/go/+/267098
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-11-04 21:02:29 +00:00
Curtis La Graff
3a1d84b393 cmd/go/internal/modfetch/codehost: add support for new fossil info hash prefix
A recent update of the Fossil SCM application changes
the line prefix when the fossil info command is used.
Instead of the revision hash starting with "uuid:", it has been
changed to "hash:".

Fossil check-in introducing this change:

https://fossil-scm.org/home/info/8ad5e4690854a81a

To support older and new versions, fossilParseStat will
now check for either version of the prefix when attempting
to find the line containing the hash of the desired revision.

Fixes #42323

Change-Id: I6eff49f9989b37b295322a8569e222a1fd02f6e3
GitHub-Last-Rev: f4e6652307
GitHub-Pull-Request: golang/go#42324
Reviewed-on: https://go-review.googlesource.com/c/go/+/267080
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-04 20:58:06 +00:00
Joel Sing
594b4a3bfe cmd/dist: enable additional cgo tests on openbsd architectures
OpenBSD gained __thread support quite some time ago.

Change-Id: I7de0a5c0c4de1a7ce59e48ac939fc2daf56be8f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/267318
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-04 16:54:48 +00:00
Jonathan Swinney
5f0fca1475 cmd/asm: rename arm64 instructions LDANDx to LDCLRx
The LDANDx instructions were misleading because they correspond to the
mnemonic LDCLRx as defined in the Arm Architecture Reference Manual for
Armv8. This changes the assembler to use the same mnemonic as the GNU
assembler and the manual.

The instruction has the form:

LDCLRx Rs, (Rb), Rt: *Rb -> Rt, Rs AND NOT(*Rb) -> *Rb

Change-Id: I94ae003e99e817209bba1afe960e612bf3a0b410
Reviewed-on: https://go-review.googlesource.com/c/go/+/267138
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: fannie zhang <Fannie.Zhang@arm.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: fannie zhang <Fannie.Zhang@arm.com>
2020-11-04 15:53:19 +00:00
Tobias Klauser
633f9e2060 internal/poll: treat copy_file_range EIO as not-handled
Fixes #42334

Change-Id: Ife51df4e7d2539a04393abfdec45e3f902975fca
Reviewed-on: https://go-review.googlesource.com/c/go/+/266940
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-04 06:42:33 +00:00
Joel Sing
5ca43acdb3 runtime: allow physical page aligned stacks to be allocated
Add a physPageAlignedStack boolean which if set, results in over allocation
by a physical page, the allocation being rounded to physical page alignment
and the unused memory surrounding the allocation being freed again.

OpenBSD/octeon has 16KB physical pages and requires stacks to be physical page
aligned in order for them to be remapped as MAP_STACK. This change allows Go
to work on this platform.

Based on a suggestion from mknyszek in issue #41008.

Updates #40995
Fixes #41008

Change-Id: Ia5d652292b515916db473043b41f6030094461d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/266919
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-04 06:14:02 +00:00
Joel Sing
8eb846fd37 cmd/compile,cmd/dist,cmd/go: enable pie buildmode for linux/riscv64
Enable pie as a buildmode for linux/riscv64, along with associated tests.

Change-Id: I3fb0234d534dbeb96aa6cee6ae872304fbe02cf4
Reviewed-on: https://go-review.googlesource.com/c/go/+/267317
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-04 06:12:33 +00:00
Ben Hoyt
e1b305af02 strconv: revert ParseFloat/ParseComplex error on incorrect bitSize
This is a partial revert of https://go-review.googlesource.com/c/go/+/248219
because we found that a non-trivial amount of code erroneously calls
ParseFloat(s, 10) or even ParseFloat(s, 0) and expects it to work --
before that change was merged, ParseFloat accepted a bitSize of
anything other than 32 or 64 to mean 64 (and ParseComplex was similar).

So revert that behavior to avoid breaking people's code, and add tests
for this.

I may add a vet check to flag ParseFloat(s, not_32_or_64) in a later
change.

See #42297 for more details.

Change-Id: I4bc0156bd74f67a39d5561b6e5fde3f2d20bd622
Reviewed-on: https://go-review.googlesource.com/c/go/+/267319
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-03 23:05:51 +00:00
Ian Lance Taylor
da7aa86917 cmd/go: in cgoflags, permit -DX1, prohibit -Wp,-D,opt
Restrict -D and -U to ASCII C identifiers, but do permit trailing digits.
When using -Wp, prohibit commas in -D values.

Change-Id: Ibfc4dfdd6e6c258e131448e7682610c44eee9492
Reviewed-on: https://go-review.googlesource.com/c/go/+/267277
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-03 21:43:30 +00:00
Bryan C. Mills
ecd7b7e991 Revert "cmd/go: support cgo files in overlays"
This reverts CL 267197.

Reason for revert: tests failing on windows-amd64-longtest builder (https://build.golang.org/log/83afde1aac3ea49debddb8996d089a741c3b1cf1).

Change-Id: I0d7c706144e34b1715316ca2ee9662c901b7f8f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/267357
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-03 15:13:25 +00:00
Joel Sing
393f2bb067 cmd/dist,cmd/go,runtime: add support for cgo on linux/riscv64
Fixes #36641

Change-Id: I51868d83ce341d78d33b221d184c5a5110c60d14
Reviewed-on: https://go-review.googlesource.com/c/go/+/263598
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-03 12:59:51 +00:00
Heisenberg
974def803e go/ast: add test for Filter
Change-Id: Ia14659a9c44f9e1504eb88b5693932b9dd4bb286
Reviewed-on: https://go-review.googlesource.com/c/go/+/252939
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-11-03 04:11:02 +00:00
kemalelmizan
cc0930cd1d cmd/doc: adding validation before adding comment marker
Previous fix in issue #20929 for adding comment marker does
not check whether string field have // prefix or not.
This commit ensures string field does not contain // before adding
prefix to the line. Test also included in this commit.

Fixes #40992

Change-Id: Ibc5e8ef147eeb2ed732fb9e19815c8b21fcfb2ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/251237
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-03 01:27:45 +00:00
Alwin Doss
45205bc47b os: export ErrProcessDone variable in windows and plan9
Exposes ErrProcessDone variable in windows and plan9
also returns this error code instead of
errors.New("os: process already finished")

Fixes #42311

Change-Id: Ie807b6526e7b6c27636e6bffe5ff0c904b319be4
GitHub-Last-Rev: 2153e0d702
GitHub-Pull-Request: golang/go#42313
Reviewed-on: https://go-review.googlesource.com/c/go/+/266997
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-03 00:56:12 +00:00
Cuong Manh Le
ebc1b8ef28 reflect: update NumMethod doc for interface type
Updates #42123

Change-Id: Ieb43b65c88d15b2475b6f3dd9672c44e7831cc34
Reviewed-on: https://go-review.googlesource.com/c/go/+/264357
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2020-11-03 00:50:57 +00:00
Cherry Zhang
3a76627df4 cmd/link: use internal linking for -race mode on darwin/arm64
The code I wrote in ldmacho.go in CL 266373 was plainly wrong. It
didn't carry rAdd over correctly. Fixed. Also added sign extension
(as ld64 does).

Internal linking with -race mode now works. Enable it.

Updates #38485.

Change-Id: I78aa949687bf6a0987913059059160b018c7560e
Reviewed-on: https://go-review.googlesource.com/c/go/+/267097
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-11-02 23:58:08 +00:00
Michael Anthony Knyszek
39a5ee52b9 runtime: decouple consistent stats from mcache and allow P-less update
This change modifies the consistent stats implementation to keep the
per-P sequence counter on each P instead of each mcache. A valid mcache
is not available everywhere that we want to call e.g. allocSpan, as per
issue #42339. By decoupling these two, we can add a mechanism to allow
contexts without a P to update stats consistently.

In this CL, we achieve that with a mutex. In practice, it will be very
rare for an M to update these stats without a P. Furthermore, the stats
reader also only needs to hold the mutex across the update to "gen"
since once that changes, writers are free to continue updating the new
stats generation. Contention could thus only arise between writers
without a P, and as mentioned earlier, those should be rare.

A nice side-effect of this change is that the consistent stats acquire
and release API becomes simpler.

Fixes #42339.

Change-Id: Ied74ab256f69abd54b550394c8ad7c4c40a5fe34
Reviewed-on: https://go-review.googlesource.com/c/go/+/267158
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2020-11-02 21:21:46 +00:00
Michael Anthony Knyszek
ac766e3718 runtime: make getMCache inlineable
This change moves the responsibility of throwing if an mcache is not
available to the caller, because the inlining cost of throw is set very
high in the compiler. Even if it was reduced down to the cost of a usual
function call, it would still be too expensive, so just move it out.

This choice also makes sense in the context of #42339 since we're going
to have to handle the case where we don't have an mcache to update stats
in a few contexts anyhow.

Also, add getMCache to the list of functions that should be inlined to
prevent future regressions.

getMCache is called on the allocation fast path and because its not
inlined actually causes a significant regression (~10%) in some
microbenchmarks.

Fixes #42305.

Change-Id: I64ac5e4f26b730bd4435ea1069a4a50f55411ced
Reviewed-on: https://go-review.googlesource.com/c/go/+/267157
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-02 21:10:41 +00:00
Austin Clements
4fcb5068f6 doc/go1.16: document switch to MADV_DONTNEED
Updates #42330.

Change-Id: Ifda10a5c3dca30acf1258e9e0af202e9beffc68e
Reviewed-on: https://go-review.googlesource.com/c/go/+/267137
Trust: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-11-02 21:08:14 +00:00
Michael Matloob
d1efaed17a cmd/go: support cgo files in overlays
This is a roll-forward of golang.org/cl/262618, which was reverted in
golang.org/cl/267037. The only differences between this CL and the
original are the three calls to fflush from the C files in
build_overlay.txt, to guarantee that the string we're expecting is
actually written out.

This requires rewriting the paths of the files passed to the cgo tool
toolchain to use the overlaid paths instead of the disk paths of
files. Because the directories of the overlaid paths don't exist in
general, the cgo tool have been updated to run in base.Cwd instead of
the package directory.

For #39958

Change-Id: If7e5e057c62c0c22ddb724f9fe650902fc5f4832
Reviewed-on: https://go-review.googlesource.com/c/go/+/267197
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Michael Matloob <matloob@golang.org>
2020-11-02 21:00:57 +00:00