1
0
mirror of https://github.com/golang/go synced 2024-11-13 19:50:21 -07:00
Commit Graph

45434 Commits

Author SHA1 Message Date
Kevin Parsons
94887410d4 path/filepath: allow EvalSymlinks to work on UNC share roots on Windows
Fixes #42079

Previously, EvalSymlinks returned an error when called with the root of
a UNC share (e.g. \\server\share). This was due to Windows's
FindFirstFile function not supporting a share root path.

To resolve this, now return early from toNorm in the case where the path
after the volume name is empty. Skipping the later path component
resolution shouldn't have any negative impact in this case, as if the
path is empty, there aren't any path components to resolve anyways.

The test case uses the localhost admin share (c$), as it should be
present in most situations. This allows testing without setting up an
external file share. However, this fix applies to all UNC share root
paths.

Change-Id: I05035bd86be93662d7bea34fab4b75fc8e918206
GitHub-Last-Rev: bd3db2cda6
GitHub-Pull-Request: golang/go#42096
Reviewed-on: https://go-review.googlesource.com/c/go/+/263917
Trust: Alex Brainman <alex.brainman@gmail.com>
Trust: Giovanni Bajo <rasky@develer.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2020-10-25 07:51:58 +00:00
imxyb
7930d39f58 log/syslog: set local to true if network is any of "unix", or "unixgram"
Fixes #41960

Change-Id: I0e0f0e11610dd2658a8f6b7e345a4aae2c19c85d
GitHub-Last-Rev: 8cac718e48
GitHub-Pull-Request: golang/go#42135
Reviewed-on: https://go-review.googlesource.com/c/go/+/264297
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-25 05:19:20 +00:00
Tobias Klauser
d03437a7c2 cmd/dist: document why test fails on incomplete ports
It might not be obvious from reading the code why we consider the test
as failed on incomplete ports even though it passed. Add a comment
documenting this behavior, as suggested by Dmitri in CL 155839.

Change-Id: I3eb7db27d01d63db277172381e5fa51577dad941
Reviewed-on: https://go-review.googlesource.com/c/go/+/264682
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-24 22:08:35 +00:00
Tobias Klauser
fe5f95eecc lib/time, time/tzdata: update tz data to 2020d
See http://mm.icann.org/pipermail/tz-announce/2020-October/000060.html
and http://mm.icann.org/pipermail/tz-announce/2020-October/000062.html
for a description of the changes.

Updates #22487

Change-Id: I4b2717d2d642284889345a0125eb6c614575c0ea
Reviewed-on: https://go-review.googlesource.com/c/go/+/264681
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-24 21:42:44 +00:00
Joel Sing
d75b7f9648 cmd/dist: add openbsd/mips64 as incomplete port
Update #40995

Change-Id: Id497f7688b00658b50feb7338157e0411b861910
Reviewed-on: https://go-review.googlesource.com/c/go/+/250578
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-10-24 15:08:50 +00:00
Constantin Konstantinidis
9ec0a46a15 cmd/compile: enforce strongly typed rules for ARM (read)
Add type casting to offset.
L246-L247
L1473-L1475

toolstash-check successful.

Change-Id: I816c7556609ca6dd67bff8007c2d006cab89ee2b
Reviewed-on: https://go-review.googlesource.com/c/go/+/257639
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-10-24 13:20:21 +00:00
Joel Sing
4a67825628 cmd/internal/obj/riscv: support additional register to register moves
Add support for signed and unsigned register to register moves of various
sizes. This makes it easier to handle zero and sign extension and will allow
for further changes that improve the compiler optimisations for riscv64.

While here, change the existing register to register moves from obj.Prog
rewriting to instruction generation.

Change-Id: Id21911019b76922367a134da13c3449a84a1fb08
Reviewed-on: https://go-review.googlesource.com/c/go/+/264657
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-24 08:12:11 +00:00
Alessandro Arzilli
05b6118139 debug/dwarf: add support for DWARFv5 to (*Data).Ranges
Updates the (*Data).Ranges method to work with DWARFv5 which uses the
new debug_rnglists section instead of debug_ranges.

This does not include supporting DW_FORM_rnglistx.

General support for DWARFv5 was added by CL 175138.

Change-Id: I01f919a865616a3ff12f5bf649c2c9abf89fcf52
Reviewed-on: https://go-review.googlesource.com/c/go/+/236657
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-24 04:11:41 +00:00
Tiwei Bie
bc0b198bd7 runtime: dump the status of lockedg on error
The dumpgstatus() will dump current g's status anyway. When lockedg's
status is bad, it's more helpful to dump lockedg's status as well than
dumping current g's status twice.

Change-Id: If5248cb94b9cdcbf4ceea07562237e1d6ee28489
GitHub-Last-Rev: da814c51ff
GitHub-Pull-Request: golang/go#40248
Reviewed-on: https://go-review.googlesource.com/c/go/+/243097
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-24 03:15:23 +00:00
Andrew G. Morgan
94eda9ae37 syscall: disable TestAllThreadsSyscall on linux-ppc64
For some reason, currently unknown, this test case fails exclusively
on the linux-ppc64 platform. Until such time as it can be made to
work, we'll disable this test case on that platform.

The same issue causes TestSetuidEtc to fail too, so disable that
on this platform.

Updates #42178

Change-Id: Idd3f6c2ee9f2fba2eb8ce4de69de7f316858bb15
Reviewed-on: https://go-review.googlesource.com/c/go/+/264719
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-24 03:06:05 +00:00
Keiichi Hirobe
d05c035365 net/http: fix typo in TestTransportReadToEndReusesConn
The test sets a Content-Type where it looks like it wants a Content-Length. The test passes because the Content-Length header is automatically added anyway, but fix the typo and set Content-Length as intended.

Change-Id: Ic2af778f82c3e9d58e164892f6ac6ef5745f884f
Reviewed-on: https://go-review.googlesource.com/c/go/+/246977
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Damien Neil <dneil@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-24 02:36:08 +00:00
Filippo Valsorda
57af9745bf crypto/x509: deprecate legacy PEM encryption
It's unfortunate that we don't implement PKCS#8 encryption (#8860)
so we can't recommend an alternative but PEM encryption is so broken
that it's worth deprecating outright.

Fixes #41949
Fixes #32777

Change-Id: Ieb46444662adec108d0de3550b693a50545c2344
Reviewed-on: https://go-review.googlesource.com/c/go/+/264159
Trust: Filippo Valsorda <filippo@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2020-10-24 01:16:48 +00:00
Thom Wiggers
a36493eb3c crypto/tls: add no-shared to openssl build instructions
This prevents the custom-built version of openssl prefering the system
libraries over the ones compiled with the specified (weak crypto)
options necessary to generate the updates. This difference can lead to
confusing failures when updating the tests.

Fixes #31809

Change-Id: I2dd257f3121d6c6c62c6aeba52e1c74046b3c584
GitHub-Last-Rev: 6d4eeafadf
GitHub-Pull-Request: golang/go#41630
Reviewed-on: https://go-review.googlesource.com/c/go/+/257517
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-10-24 00:32:11 +00:00
Michael Anthony Knyszek
64dc25b2db runtime: add tests for addrRanges.add
Change-Id: I249deb482df74068b0538e9d773b9a87bc5a6df3
Reviewed-on: https://go-review.googlesource.com/c/go/+/242681
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-23 23:01:52 +00:00
Michael Anthony Knyszek
fe70866d1d runtime: throw on zero-sized range passed to addrRanges.add
addrRanges represents a set of addresses. Currently, passing in a
zero-sized range will cause that range to be added to the list, even
though it doesn't represent any address (addrRanges.contains will still
always return false, and findSucc will give surprising results).

We could ignore this input, but it's almost always a bug for the calling
code to pass in a zero-sized range, so just throw.

Change-Id: I8ed09e15b79a3a33e2d0cf5ed55f9e497388e7a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/242817
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-10-23 23:01:26 +00:00
Michael Anthony Knyszek
e01a1c01f8 runtime: add tests for addrRanges.findSucc
This change adds a test suite for addrRanges.findSucc so we can change
the implementation more safely.

For #40191.

Change-Id: I14a834b6d54836cbc676eb0edb292ba6176705cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/242678
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2020-10-23 22:33:17 +00:00
Colin Arnott
9db7db54b0 log: expose std via new Default function
To allow passing around the package level *Logger, it is now exposed to
callers of the Default function. We considered exposing std, however at
this time there is no need to allow callers to replace std only pass and
call methods directly.

Fixes #39057

Change-Id: I710b16a3aa5e4e878870561dbf59560f98d8d09a
Reviewed-on: https://go-review.googlesource.com/c/go/+/264460
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-10-23 22:08:37 +00:00
subham sarkar
a972e8ccb9 all: fix quoting for compress/bzip2 and time's godoc
The existing usage of grave accent (`) and apostrophe (')
at some places made godoc to ignore them and show it as it is.
So, use both of the characters twice (consecutively) so that
godoc can convert it to {left,right} double quotation mark.

Fixes #41958

Change-Id: I64fd9b5fa34f416ad595009d09f5482e10bd8b4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/262397
Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-10-23 21:57:39 +00:00
Michael Pratt
db185e543f runtime: drop redundant gcBlackenEnabled reset
This reset of gcBlackenEnabled is a no-op because it was already reset
almost immediately before in gcMarkDone, which is the only caller of
gcMarkTermination.

Adjust the comment to clarify setGCPhase a bit more. We are coming from
_GCmark, so write barriers are already enabled.

Change-Id: Ieac2dadf33c3c5a44e8a25a499dea8cfe03b8d73
Reviewed-on: https://go-review.googlesource.com/c/go/+/241357
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-10-23 21:26:40 +00:00
Jay Conrod
f24ff3856a cmd/go: change error message for missing import with unused replacement
In readonly mode, if a package is not provided by any module in the
build list, and there is an unused replacement that contains the
package, we now recommend a 'go get' command to add a requirement on
the highest replaced version.

Fixes #41416

Change-Id: Iedf3539292c70ea6ba6857433fd184454d9325da
Reviewed-on: https://go-review.googlesource.com/c/go/+/263146
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
2020-10-23 20:54:41 +00:00
Jay Conrod
4fdb98dcb1 cmd/go: save sums for zips needed to diagnose ambiguous imports
Previously, we would retain entries in go.sum for .mod files in the
module graph (reachable from the main module) and for .zip files
of modules providing packages.

This isn't quite enough: when we load a package, we need the content
of each module in the build list that *could* provide the package
(that is, each module whose path is a prefix of the package's path) so
we can diagnose ambiguous imports.

For #33008

Change-Id: I0b4d9d68c1f4ca382f0983a3a7e537764f35c3aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/262781
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
2020-10-23 20:54:35 +00:00
Jay Conrod
5cd4390f38 cmd/go: don't fetch files missing sums in readonly mode
If the go command needs a .mod or .zip file in -mod=readonly mode
(now the default), and that file doesn't have a hash in the main
module's go.sum file, the go command will now report an error before
fetching the file, rather than at the end when failing to update
go.sum. The error says specifically which entry is missing.

If this error is encountered when loading the build list, it will
suggest 'go mod tidy'.

If this error is encountered when loading a specific package (an
import or command line argument), the error will mention that package
and will suggest 'go mod tidy' or 'go get -d'.

Fixes #41934
Fixes #41935

Change-Id: I96ec2ef9258bd4bade9915c43d47e6243c376a81
Reviewed-on: https://go-review.googlesource.com/c/go/+/262341
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
2020-10-23 20:54:30 +00:00
Jay Conrod
5f616a6fe7 cmd/go: in 'go mod init', suggest running 'go mod tidy'
When 'go mod init' is run in an existing project, it may import
requirements from a vendor configuration file, but the requirements
may not be complete, and go.sum won't contain sums for module
zips. With -mod=readonly, the next build command is likely to fail.

'go mod init' will now suggest running 'go mod tidy' if there are .go
files or subdirectories in the current directory.

We could potentially run 'go mod tidy' automatically within
'go mod init', but it seems better to guide users to using 'go mod tidy'
as a separate command to fix missing dependencies.

For #41712
Updates #40278

Change-Id: Iaece607f291244588a732ef4c5d576108965ca91
Reviewed-on: https://go-review.googlesource.com/c/go/+/264622
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-10-23 20:54:25 +00:00
Jay Conrod
d05e89a8fd cmd/go: refactor modload.InitMod
InitMod is split into two functions. LoadModFile parses an existing
go.mod file and loads the build list (or checks vendor/modules.txt for
consistency in vendor mode). CreateModFile creates a new go.mod file,
possibly inferring the module path and importing a vendor
configuration file.

Some logic is moved from runInit to CreateModFile. init-specific logic
is removed from other functions.

This CL shouldn't cause substantial differences in behavior, though
some error messages are slightly different.

For #41712

Change-Id: Ia684945cfcf5beca30bbb81e7144fc246c4f27ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/264621
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-10-23 20:54:19 +00:00
Jay Conrod
c8c3c29daa cmd/go: don't import requirements into existing go.mod files
Previously, if a go.mod file was present, and it only contained a
module directive, any module-aware command would attempt to import
requirements from a vendor configuration file like Gopkg.lock.

This CL removes that functionality. It was undocumented and untested,
and it can cause problems with -mod=readonly. It should never come up
for go.mod files created with 'go mod init', since they have a "go"
directive.

For #40278

Change-Id: I64c0d67d204560aa5c775d29553883d094fd3b72
Reviewed-on: https://go-review.googlesource.com/c/go/+/264620
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-23 20:54:12 +00:00
Andrew G. Morgan
d1b1145cac syscall: support POSIX semantics for Linux syscalls
This change adds two new methods for invoking system calls
under Linux: syscall.AllThreadsSyscall() and
syscall.AllThreadsSyscall6().

These system call wrappers ensure that all OSThreads mirror
a common system call. The wrappers serialize execution of the
runtime to ensure no race conditions where any Go code observes
a non-atomic OS state change. As such, the syscalls have
higher runtime overhead than regular system calls, and only
need to be used where such thread (or 'm' in the parlance
of the runtime sources) consistency is required.

The new support is used to enable these functions under Linux:

  syscall.Setegid(), syscall.Seteuid(), syscall.Setgroups(),
  syscall.Setgid(), syscall.Setregid(), syscall.Setreuid(),
  syscall.Setresgid(), syscall.Setresuid() and syscall.Setuid().

They work identically to their glibc counterparts.

Extensive discussion of the background issue addressed in this
patch can be found here:

   https://github.com/golang/go/issues/1435

In the case where cgo is used, the C runtime can launch pthreads that
are not managed by the Go runtime. As such, the added
syscall.AllThreadsSyscall*() return ENOTSUP when cgo is enabled.
However, for the 9 syscall.Set*() functions listed above, when cgo is
active, these functions redirect to invoke their C.set*() equivalents
in glibc, which wraps the raw system calls with a nptl:setxid fixup
mechanism. This achieves POSIX semantics for these functions in the
combined Go and C runtime.

As a side note, the glibc/nptl:setxid support (2019-11-30) does not
extend to all security related system calls under Linux so using
native Go (CGO_ENABLED=0) and these AllThreadsSyscall*()s, where
needed, will yield more well defined/consistent behavior over all
threads of a Go program. That is, using the
syscall.AllThreadsSyscall*() wrappers for things like setting state
through SYS_PRCTL and SYS_CAPSET etc.

Fixes #1435

Change-Id: Ib1a3e16b9180f64223196a32fc0f9dce14d9105c
Reviewed-on: https://go-review.googlesource.com/c/go/+/210639
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Ian Lance Taylor <iant@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-10-23 20:53:14 +00:00
Mark Rushakoff
75032ad8cf cmd/go: break after terminal loop condition
After the first time needCostly is set to true, there is no need to
continue checking the remaining args.

Change-Id: I07171ce50d20e2a917117a0f84c442fe978cb274
GitHub-Last-Rev: 6d0c19341b
GitHub-Pull-Request: golang/go#41859
Reviewed-on: https://go-review.googlesource.com/c/go/+/260638
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-23 20:38:27 +00:00
Dmitri Shuralyov
f8376a55b0 sync: document that Once must not be copied
Fixes #42160.

Change-Id: I9bf8b6f0bf1eccd3ab32cbd94c812f768746d291
Reviewed-on: https://go-review.googlesource.com/c/go/+/264557
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-10-23 19:57:55 +00:00
Michael Matloob
3931cc113f cmd/go: replace some more stats with fsys.Stat
To support overlays

For #39958

Change-Id: I5ffd72aeb7f5f30f6c60f6334a01a0a1383c7945
Reviewed-on: https://go-review.googlesource.com/c/go/+/264478
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-23 19:36:38 +00:00
David Chase
c3fe874f25 cmd/compile: avoid generating CSEs; do all aggregates; maintain debug names
This adds a pass to detect common selection operations,
to avoid generating duplicates.  Duplicate offsets are
also detected.

All aggregate types are now handled; there is some freedom in where
expand_calls is run, though it must run before softfloat.

Debug-name-maintenance is now incremental both in decompose builtin
and in expand_calls; it might be good to push this into all the
decompose passes.

(this is a smash of 5 CLs that rewrote some of the same code several
times to deal with phase-ordering problems, and included an abandoned
attempt.)

For #40724.

Change-Id: I2a0c32f20660bf8b99e2bcecd33545d97d2bd3c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/249458
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 18:02:16 +00:00
Michael Pratt
ad64272724 runtime: rename pageAlloc receiver
The history of pageAlloc using 's' as a receiver are lost to the depths
of time (perhaps it used to be called summary?), but it doesn't make
much sense anymore. Rename it to 'p'.

Generated with:

$ cd src/runtime
$ grep -R -b "func (s \*pageAlloc" . | awk -F : '{ print $1 ":#" $2+6 }' | xargs -n 1 -I {} env GOROOT=$(pwd)/../../ gorename -offset {} -to p -v
$ grep -R -b "func (s \*pageAlloc" . | awk -F : '{ print $1 ":#" $2+6 }' | xargs -n 1 -I {} env GOROOT=$(pwd)/../../ GOARCH=386 gorename -offset {} -to p -v
$ GOROOT=$(pwd)/../../ gorename -offset mpagecache.go:#2397 -to p -v

($2+6 to advance past "func (".)

Plus manual comment fixups.

Change-Id: I2d521a1cbf6ebe2ef6aae92e654bfc33c63d1aa9
Reviewed-on: https://go-review.googlesource.com/c/go/+/250517
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-10-23 16:54:35 +00:00
Michael Pratt
e5ad73508e cmd/compile: intrinsify runtime/internal/atomic.{And,Or} on MIPS
This one is trivial, as there are already 32-bit AND and OR ops used to
implement the more complex 8-bit versions.

Change-Id: Ic48a53ea291d0067ebeab8e96c82e054daf20ae7
Reviewed-on: https://go-review.googlesource.com/c/go/+/263149
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 16:25:07 +00:00
Michael Pratt
e313fd7448 runtime: drop unused work.ndone field
This field is unused since golang.org/cl/134785 and thus can be
trivially removed.

Change-Id: I1a87f8e78ffdf662440409404f0251c40bc56a4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/241741
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2020-10-23 16:12:52 +00:00
Meng Zhuo
7e25bdba5e cmd/link: use xcode strip for macho combine dwarf
The GNU strip will shrink text section while xcodetool strip don't.
We have to use xcodetool strip from system explicitly.

Fixes #41967

Change-Id: Ida372869e0ebc9e93f883640b1614836cea3672f
Reviewed-on: https://go-review.googlesource.com/c/go/+/262398
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Meng Zhuo <mzh@golangcn.org>
2020-10-23 15:54:44 +00:00
Meng Zhuo
4f597abe77 internal/bytealg: improve mips64x equal on large size
name               old time/op    new time/op    delta
Equal/0              9.94ns ± 4%    9.12ns ± 5%     -8.26%  (p=0.000 n=10+10)
Equal/1              24.5ns ± 0%    27.2ns ± 1%    +11.22%  (p=0.000 n=9+10)
Equal/6              28.1ns ± 0%    32.1ns ± 1%    +14.20%  (p=0.000 n=8+10)
Equal/9              37.1ns ± 0%    37.8ns ± 1%     +1.95%  (p=0.000 n=8+9)
Equal/15             47.3ns ± 0%    44.3ns ± 0%     -6.34%  (p=0.000 n=9+10)
Equal/16             42.9ns ± 0%    24.6ns ± 0%    -42.66%  (p=0.000 n=10+7)
Equal/20             44.3ns ± 0%    57.4ns ± 0%    +29.57%  (p=0.000 n=9+10)
Equal/32             63.2ns ± 0%    35.8ns ± 0%    -43.35%  (p=0.000 n=10+10)
Equal/4K             6.49µs ± 0%    0.50µs ± 0%    -92.27%  (p=0.000 n=10+8)
Equal/4M             6.70ms ± 0%    0.48ms ± 0%    -92.78%  (p=0.000 n=8+10)
Equal/64M             110ms ± 0%       8ms ± 0%    -92.65%  (p=0.000 n=9+9)
CompareBytesEqual    36.6ns ± 0%    35.9ns ± 0%     -1.83%  (p=0.000 n=10+9)

name               old speed      new speed      delta
Equal/1            40.8MB/s ± 0%  36.7MB/s ± 0%    -10.16%  (p=0.000 n=10+10)
Equal/6             213MB/s ± 0%   187MB/s ± 1%    -12.32%  (p=0.000 n=10+10)
Equal/9             243MB/s ± 0%   238MB/s ± 1%     -1.94%  (p=0.000 n=9+10)
Equal/15            317MB/s ± 0%   339MB/s ± 0%     +6.86%  (p=0.000 n=9+9)
Equal/16            373MB/s ± 0%   651MB/s ± 0%    +74.70%  (p=0.000 n=8+10)
Equal/20            452MB/s ± 0%   348MB/s ± 0%    -22.90%  (p=0.000 n=8+10)
Equal/32            506MB/s ± 0%   893MB/s ± 0%    +76.53%  (p=0.000 n=10+9)
Equal/4K            631MB/s ± 0%  8166MB/s ± 0%  +1194.73%  (p=0.000 n=10+10)
Equal/4M            626MB/s ± 0%  8673MB/s ± 0%  +1284.94%  (p=0.000 n=8+10)
Equal/64M           608MB/s ± 0%  8277MB/s ± 0%  +1260.83%  (p=0.000 n=9+9)

Change-Id: I1cd14ade16390a5097a8d4e9721d5e822fa6218f
Reviewed-on: https://go-review.googlesource.com/c/go/+/199597
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Meng Zhuo <mzh@golangcn.org>
2020-10-23 15:53:01 +00:00
Michael Pratt
646531c52a cmd/compile: intrinsify runtime/internal/atomic.{And,Or} on S390X
This is a simplification of LANfloor/LAOfloor since we have a whole
word.

Change-Id: I791641fb4068cad3f73660ce51699ed4653ae0e6
Reviewed-on: https://go-review.googlesource.com/c/go/+/263151
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 15:11:15 +00:00
Michael Pratt
e223c6cf07 cmd/compile: intrinsify runtime/internal/atomic.{And,Or} on PPC64
This is a simple case of changing the operand size of the existing 8-bit
And/Or.

I've also updated a few operand descriptions that were out-of-sync with
the implementation.

Change-Id: I95ac4445d08f7958768aec9a233698a2d652a39a
Reviewed-on: https://go-review.googlesource.com/c/go/+/263150
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 15:11:03 +00:00
Michael Pratt
cdb19b4dba cmd/compile: intrinsify runtime/internal/atomic.{And,Or} on ARM64
These are identical to And8 and Or8, just using LDAXRW/STLXRW instead of
LDAXRB/STLXRB.

Change-Id: I5308832ae165064550bee4bb245809ab952f4cc8
Reviewed-on: https://go-review.googlesource.com/c/go/+/263148
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 15:06:35 +00:00
Russ Cox
b5ddc42b46 io/fs, path, path/filepath, testing/fstest: validate patterns in Match, Glob
According to #28614, proposal review agreed in December 2018 that
Match should return an error for failed matches where the unmatched
part of the pattern has a syntax error. (The failed match has to date
caused the scan of the pattern to stop early.)

This change implements that behavior: the match loop continues
scanning to the end of the pattern, even after a confirmed mismatch,
to check whether the pattern is even well-formed.

The change applies to both path.Match and filepath.Match.
Then filepath.Glob and fs.Glob make a single validity-checking
call to Match before beginning their usual processing.

Also update fstest.TestFS to check for correct validation in custom
Glob implementations.

Fixes #28614.

Change-Id: Ic1d35a4bb9c3565184ae83dbefc425c5c96318e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/264397
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-23 14:59:03 +00:00
Michael Pratt
4a2cc73f87 runtime: don't attempt to steal from idle Ps
Work stealing is a scalability bottleneck in the scheduler. Since each P
has a work queue, work stealing must look at every P to determine if
there is any work. The number of Ps scales linearly with GOMAXPROCS
(i.e., the number of Ps _is_ GOMAXPROCS), thus this work scales linearly
with GOMAXPROCS.

Work stealing is a later attempt by a P to find work before it goes
idle. Since the P has no work of its own, extra costs here tend not to
directly affect application-level benchmarks. Where they show up is
extra CPU usage by the process as a whole. These costs get particularly
expensive for applications that transition between blocked and running
frequently.

Long term, we need a more scalable approach in general, but for now we
can make a simple observation: idle Ps ([1]) cannot possibly have
anything in their runq, so we need not bother checking at all.

We track idle Ps via a new global bitmap, updated in pidleput/pidleget.
This is already a slow path (requires sched.lock), so we don't expect
high contention there.

Using a single bitmap avoids the need to touch every P to read p.status.
Currently, the bitmap approach is not significantly better than reading
p.status. However, in a future CL I'd like to apply a similiar
optimization to timers. Once done, findrunnable would not touch most Ps
at all (in mostly idle programs), which will avoid memory latency to
pull those Ps into cache.

When reading this bitmap, we are racing with Ps going in and out of
idle, so there are a few cases to consider:

1. _Prunning -> _Pidle: Running P goes idle after we check the bitmap.
In this case, we will try to steal (and find nothing) so there is no
harm.

2. _Pidle -> _Prunning while spinning: A P that starts running may queue
new work that we miss. This is OK: (a) that P cannot go back to sleep
without completing its work, and (b) more fundamentally, we will recheck
after we drop our P.

3. _Pidle -> _Prunning after spinning: After spinning, we really can
miss work from a newly woken P. (a) above still applies here as well,
but this is also the same delicate dance case described in findrunnable:
if nothing is spinning anymore, the other P will unpark a thread to run
the work it submits.

Benchmark results from WakeupParallel/syscall/pair/race/1ms (see
golang.org/cl/228577):

name                            old msec          new msec   delta
Perf-task-clock-8               250 ± 1%          247 ± 4%     ~     (p=0.690 n=5+5)
Perf-task-clock-16              258 ± 2%          259 ± 2%     ~     (p=0.841 n=5+5)
Perf-task-clock-32              284 ± 2%          270 ± 4%   -4.94%  (p=0.032 n=5+5)
Perf-task-clock-64              326 ± 3%          303 ± 2%   -6.92%  (p=0.008 n=5+5)
Perf-task-clock-128             407 ± 2%          363 ± 5%  -10.69%  (p=0.008 n=5+5)
Perf-task-clock-256             561 ± 1%          481 ± 1%  -14.20%  (p=0.016 n=4+5)
Perf-task-clock-512             840 ± 5%          683 ± 2%  -18.70%  (p=0.008 n=5+5)
Perf-task-clock-1024          1.38k ±14%        1.07k ± 2%  -21.85%  (p=0.008 n=5+5)

[1] "Idle Ps" here refers to _Pidle Ps in the sched.pidle list. In other
contexts, Ps may temporarily transition through _Pidle (e.g., in
handoffp); those Ps may have work.

Updates #28808
Updates #18237

Change-Id: Ieeb958bd72e7d8fb375b0b1f414e8d7378b14e29
Reviewed-on: https://go-review.googlesource.com/c/go/+/259578
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Trust: Michael Pratt <mpratt@google.com>
2020-10-23 14:18:27 +00:00
Michael Pratt
44dbeaf356 cmd/compile: intrinsify runtime/internal/atomic.{And,Or} on AMD64
These are identical to And8 and Or8, just using ANDL/ORL instead of
ANDB/ORB.

Change-Id: I99cf90a8b0b5f211fb23325dddd55821875f0c8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/263140
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 14:18:13 +00:00
Michael Pratt
ad61343f88 runtime/internal/atomic: add 32-bit And/Or
These will be used in a following CL to perform larger bit clear and bit
set than And8/Or8.

Change-Id: I60f7b1099e29b69eb64add77564faee862880a8d
Reviewed-on: https://go-review.googlesource.com/c/go/+/260977
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Michael Pratt <mpratt@google.com>
2020-10-23 14:17:57 +00:00
Nigel Tao
7e01b3b387 strconv: add eiselLemire32
This does for ParseFloat(etc, 32) what commit a2eb53c571 did for
ParseFloat(etc, 64).

name              old time/op  new time/op  delta
Atof32Decimal-4   48.3ns ± 4%  48.8ns ± 2%     ~     (p=0.548 n=5+5)
Atof32Float-4     56.2ns ± 5%  54.7ns ± 3%     ~     (p=0.246 n=5+5)
Atof32FloatExp-4   104ns ± 0%    76ns ± 2%  -27.19%  (p=0.008 n=5+5)
Atof32Random-4     142ns ± 2%   109ns ± 1%  -23.07%  (p=0.008 n=5+5)

Change-Id: I6ee5a2f2d791d4fe3028f1d40aca96400120fda0
Reviewed-on: https://go-review.googlesource.com/c/go/+/264517
Trust: Nigel Tao <nigeltao@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-10-23 00:39:08 +00:00
Russ Cox
8bde9b320e cmd/compile: add //go:embed support
This commit contains the compiler support for //go:embed lines.
The go command passes to the compiler an "embed config"
that maps literal patterns like *.txt to the set of files to embed.
The compiler then lays out the content of those files as static data
in the form of an embed.Files or string or []byte in the final object file.

The test for this code is the end-to-end test hooking up the
embed, cmd/compile, and cmd/go changes, in the next CL.

For #41191.

Change-Id: I916e57f8cc65871dc0044c13d3f90c252a3fe1bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/243944
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-23 00:29:10 +00:00
Russ Cox
400581b8b0 embed: implement FS
embed.FS is the implementation of embedded file trees, providing
an fs.FS for each embed.FS variable.

Tests are in a follow-up CL, in the package embed/internal/embedtest.
(They can only be written once the toolchain can initialize one of these,
which requires changes to cmd/compile and cmd/go.)

For #41191.

Change-Id: Ieb0ead1d305cdac3d5d4e11772dca75740a72730
Reviewed-on: https://go-review.googlesource.com/c/go/+/243942
Trust: Russ Cox <rsc@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-23 00:22:00 +00:00
Hana
67bce7c1cf cmd/vendor: sync pprof@v0.0.0-20201007051231-1066cbb265c7
This is a belated early sync for 1.16 dev cycle

For #36905

Change-Id: I387528ae897794841c0c78b0f0910fc5ce8599ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/260538
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Hyang-Ah Hana Kim <hyangah@gmail.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-22 23:33:06 +00:00
Nigel Tao
ad36f87151 strconv: increase the Eisel-Lemire exp10 range
This grows the exp10 range for which the Eisel-Lemire algorithm applies
from [-307, +288] to [-348, +347], roughly equivalent to the existing
powersOfTen table in extfloat.go (which uses a different algorithm).

name                  old time/op  new time/op  delta
Atof64Decimal-4       48.4ns ± 1%  48.7ns ± 3%   ~     (p=0.698 n=5+5)
Atof64Float-4         57.9ns ± 1%  58.1ns ± 2%   ~     (p=0.873 n=5+5)
Atof64FloatExp-4      71.8ns ± 2%  72.2ns ± 2%   ~     (p=0.730 n=5+5)
Atof64Big-4            165ns ± 1%   164ns ± 1%   ~     (p=0.635 n=5+5)
Atof64RandomBits-4     165ns ± 1%   165ns ± 6%   ~     (p=0.143 n=5+5)
Atof64RandomFloats-4   147ns ± 2%   147ns ± 1%   ~     (p=0.857 n=5+5)

Change-Id: Idf7dc5297db6db2bd9e0bd4cb0e55e021916fa43
Reviewed-on: https://go-review.googlesource.com/c/go/+/264139
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Trust: Nigel Tao <nigeltao@golang.org>
2020-10-22 23:20:22 +00:00
qmuntal
f1aa0b081e doc/go1.16: document ASLR support for -buildmode=c-shared on windows
Change-Id: I89c61e444b3ab36f0081a5252d210cb265344122
Reviewed-on: https://go-review.googlesource.com/c/go/+/264157
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
2020-10-22 23:00:42 +00:00
Quim Muntal
6f7b553c82 cmd/cgo: avoid exporting all symbols on windows buildmode=c-shared
Disable default symbol auto-export behaviour by marking exported
function with the __declspec(dllexport) attribute. Old behaviour can
still be used by setting -extldflags=-Wl,--export-all-symbols.

See https://sourceware.org/binutils/docs/ld/WIN32.html for more info.

This change cuts 50kb of a "hello world" dll.

Updates #6853
Fixes #30674

Change-Id: I9c7fb09c677cc760f24d0f7d199740ae73981413
Reviewed-on: https://go-review.googlesource.com/c/go/+/262797
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
2020-10-22 22:40:17 +00:00
Nigel Tao
4ce9ea52c9 strconv: fix Eisel-Lemire for negative zero
This is somewhat academic (and no tests failed before this commit),
since func atof64 only calls func eiselLemire when func atof64exact
fails, and func atof64exact doesn't fail when parsing positive or
negative zeroes. But it's still worth fixing.

Change-Id: Ibe6ef4c8fd96827673b711d5456003fbc447e39c
Reviewed-on: https://go-review.googlesource.com/c/go/+/264140
Trust: Nigel Tao <nigeltao@golang.org>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2020-10-22 21:12:43 +00:00