1
0
mirror of https://github.com/golang/go synced 2024-11-24 03:50:18 -07:00
Commit Graph

45688 Commits

Author SHA1 Message Date
Michael Matloob
60b1253293 cmd/go: pass in overlaid file paths to C compiler
This change moves the code in work.(*Builder).cgo that, when there is
an overlay, copies non-Go files to objdir into work.(*Builder).Build,
and creates an overlay structure mapping from the nominal file paths
into the copies in objdir. That's propagated through to
work.(*Builder).ccompile, which will use it to pass in the path to the
overlaid contents in objdir when calling the compiler.

This allows for overlays of C/C++/Fortran files.

For #39958

Change-Id: I9a2e3d3ba6afdf7ce19be1dbf4eee34805cdc05f
Reviewed-on: https://go-review.googlesource.com/c/go/+/266376
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-12 21:21:18 +00:00
Ian Lance Taylor
062e0e5ce6 cmd/go, cmd/cgo: don't let bogus symbol set cgo_ldflag
A hand-edited object file can have a symbol name that uses newline and
other normally invalid characters. The cgo tool will generate Go files
containing symbol names, unquoted. That can permit those symbol names
to inject Go code into a cgo-generated file. If that Go code uses the
//go:cgo_ldflag pragma, it can cause the C linker to run arbitrary
code when building a package. If you build an imported package we
permit arbitrary code at run time, but we don't want to permit it at
package build time. This CL prevents this in two ways.

In cgo, reject invalid symbols that contain non-printable or space
characters, or that contain anything that looks like a Go comment.

In the go tool, double check all //go:cgo_ldflag directives in
generated code, to make sure they follow the existing LDFLAG restrictions.

Thanks to Imre Rad / https://www.linkedin.com/in/imre-rad-2358749b for
reporting this.

Fixes CVE-2020-28367

Change-Id: Ia1ad8f3791ea79612690fa7d26ac451d0f6df7c1
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/895832
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/269658
Trust: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2020-11-12 20:58:06 +00:00
Katie Hockman
1e1fa5903b math/big: fix shift for recursive division
The previous s value could cause a crash
for certain inputs.

Will check in tests and documentation improvements later.

Thanks to the Go Ethereum team and the OSS-Fuzz project for reporting this.
Thanks to Rémy Oudompheng and Robert Griesemer for their help
developing and validating the fix.

Fixes CVE-2020-28362

Change-Id: Ibbf455c4436bcdb07c84a34fa6551fb3422356d3
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/899974
Reviewed-by: Roland Shoemaker <bracewell@google.com>
Reviewed-by: Filippo Valsorda <valsorda@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/269657
Trust: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-12 20:42:40 +00:00
xiaodong liu
b34b0aaf69 cmd/go: skip TestScript/build_plugin_non_main on platforms that do not support -buildmode=plugin
Fixes #42474

Change-Id: I1550b44b92cd272854e2f17493245a14e3d39f41
GitHub-Last-Rev: 948d01716e
GitHub-Pull-Request: golang/go#42475
Reviewed-on: https://go-review.googlesource.com/c/go/+/268737
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Trust: Jay Conrod <jayconrod@google.com>
2020-11-12 15:28:43 +00:00
Cherry Zhang
c167635a6e cmd/compile: gofmt
Change-Id: I2aa44b9976bdac577db44ef76e18fdf5748df8ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/269477
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-12 14:38:38 +00:00
Jay Conrod
e75aef80ca cmd/go: migrate away from semver.Max
For #32700

Change-Id: Ib5cd7004e4558bebebc5f9e7c9263d720c590845
Reviewed-on: https://go-review.googlesource.com/c/go/+/269338
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-12 14:17:24 +00:00
Alessandro Arzilli
9ef65ff137 cmd/compile: do not emit an extra debug_line entry for the end of seq addr
Uses DW_LNS_advance_pc directly, instead of calling putpclcdelta
because the latter will create a new debug_line entry for the end of
sequence address.

Fixes #42484

Change-Id: Ib6355605cac101b9bf37a3b4961ab0cee678a839
Reviewed-on: https://go-review.googlesource.com/c/go/+/268937
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-12 13:47:13 +00:00
Cherry Zhang
4bc5f6f45f cmd/link: put DYLD_INFO at beginning of LINKEDIT segment on darwin
Apparently, code signing requires DYLD_INFO tables are at the
beginning of the LINKEDIT segment. Put it there.

May fix #42507.

Change-Id: I1836e0f495719cf75f66d0831fe1544bbe3ff1a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/269377
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-11-12 13:40:55 +00:00
Tobias Klauser
d7974c31d0 os: gofmt
As reported by John Papandriopoulos, some parts of CL 216622 weren't
properly formatted.

Change-Id: I3a76abb6213bb17ef440036295c86d930703b456
Reviewed-on: https://go-review.googlesource.com/c/go/+/269218
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
2020-11-12 10:22:50 +00:00
Federico Guerinoni
141fa337ad bytes: add example for (*Buffer).Bytes
Change-Id: I49ac604530fff7928fa15de07563418b104da5e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/268260
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Giovanni Bajo <rasky@develer.com>
2020-11-11 20:51:00 +00:00
Andrew G. Morgan
f2e58c6d42 syscall: improve TestSetuidEtc() /proc/ parsing against races
TestSetuidEtc() was failing sporadically on linux-ppc64. From the
three https://build.golang.org/ logs, it looked like the logged
errors could be associated with threads dying, but proc reads
were, in some way, racing with their demise.

Exploring ways to increase thread demise, revealed that races
of this type can happen on non-ppc64 systems, and that
os.IsNotExist(err) was not a sufficient error condition test
for a thread's status file disappearing. This change includes a
fix for that to.

The actual issue on linux-ppc64 appears to be tied to PID reaping
and reuse latency on whatever the build test environment is for
linux-ppc64-buildlet. I suspect this can happen on any linux
system, however, especially where the container has a limited PID
range.

The fix for this, limited to the test (the runtime syscall support
is unchanged), is to confirm that the Pid for the interrogated
thread's /proc/<TID>/status file confirms that it is still
associated with the test-process' PID.

linux-ppc64-buildlet:
  go/bin/go test syscall -run=TestSetuidEtc -count=10000
  ok      syscall 104.285s

Fixes #42462

Change-Id: I55c84ab8361003570a405fa52ffec4949bf91113
Reviewed-on: https://go-review.googlesource.com/c/go/+/268717
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
2020-11-11 20:49:53 +00:00
Bryan Boreham
4c174a7ba6 testing: reduce memory allocation in Helper
Store the PC instead of the string name of the function, and defer
that conversion until we need it.

Helper is still relatively expensive in CPU time (few hundred ns),
but memory allocation is now constant for a test rather than linear in
the number of times Helper is called.

benchstat:
name        old time/op    new time/op    delta
TBHelper-4    1.30µs ±27%    0.53µs ± 1%   -59.03%  (p=0.008 n=5+5)

name        old alloc/op   new alloc/op   delta
TBHelper-4      216B ± 0%        0B       -100.00%  (p=0.008 n=5+5)

name        old allocs/op  new allocs/op  delta
TBHelper-4      2.00 ± 0%      0.00       -100.00%  (p=0.008 n=5+5)

Change-Id: I6565feb491513815e1058637d086b0374fa94e19
GitHub-Last-Rev: c2329cf225
GitHub-Pull-Request: golang/go#38834
Reviewed-on: https://go-review.googlesource.com/c/go/+/231717
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2020-11-11 20:48:31 +00:00
Ian Lance Taylor
b641f0dcf4 os: clarify that IsExist and friends do not use errors.Is
Fixes #41122

Change-Id: Ie5cb0b19ac461d321520b1ebfc493a0ca22232a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/268897
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-11-11 20:26:44 +00:00
Roland Shoemaker
26a860706a doc/go1.16: add crypto/x509 CSR release note
Change-Id: If74d49c3be9299d8c136003673e0fee2a563389d
Reviewed-on: https://go-review.googlesource.com/c/go/+/268957
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2020-11-11 17:12:34 +00:00
Michael Matloob
28437546f4 cmd/go: don't copy cgo files to objdir when overlay is present
This cl is a roll-forward of golang.org/cl/265758, which was rolled back
in golang.org/cl/268900. The changes made are removing cgofiles
from the list of files that are copied to objdir (because the cgofiles
themselves aren't actually provided to the compiler) and fixing test
cases to properly provide the overlay flag and to allow for paths with
backslashes (as in Windows).

The previous cl (golang.org/cl/262618) copied non-overlaid cgo files
to objdir, mostly to get around the issue that otherwise cgo-generated
files were written out with the wrong names (they'd get the base path
of the overlay file containing the replaced contents, instead of the
base path of the path whose contents are being replaced). So that CL
it would copy the files to objdir with the base path of the file
being replaced to circumvent that.

This CL changes cmd/go and cmd/cgo so that instead of copying
files, it passes the actual path of the file on disk either of
the original file (if it is not overlaid) or its replacement
file (if it is) as well as a flag --path_rewrite, newly added to
cmd/cgo, that specifies the actual original file path that corresponds
to the replaced files.

Updates #39958

Change-Id: Ia45b022f9d27cfce0f9ec6da5f3a9f53654c67b8
Reviewed-on: https://go-review.googlesource.com/c/go/+/269017
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-11 16:22:54 +00:00
Russ Cox
c906608406 io/fs: fix reference to WalkFunc
The comment explains differences between WalkDirFunc and WalkFunc,
but when this code moved out of path/filepath, we forgot to change
the reference to be filepath.WalkFunc. Fix that.

(The text should not be deleted, because path/filepath does not
contain this type - WalkDirFunc - nor this text anymore.)

Pointed out by Carl Johnson on CL 243916 post-submit.

For #41190.

Change-Id: I44c64d0b7e60cd6d3694cfd6d0b95468ec4612fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/268417
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-11 14:29:34 +00:00
Dmitri Shuralyov
f2e186b877 all: update vendored dependencies for Go 1.16 release
The Go 1.16 code freeze has recently started. This is a time to update
all golang.org/x/... module versions that contribute packages to the
std and cmd modules in the standard library to latest master versions.

Those versions have already gone through code review, and now they
will undergo additional testing during the upcoming freeze period.
If new issues in these dependencies are discovered, we have the freeze
period to address them. By the end of the freeze period, we will have
confidence that the Go 1.16 release and the dependency versions it has
selected are robust.

The dependency module versions that are selected in this commit are:

	github.com/google/pprof v0.0.0-20201007051231-1066cbb265c7
	github.com/ianlancetaylor/demangle v0.0.0-20200414190113-039b1ae3a340
	golang.org/x/arch v0.0.0-20201008161808-52c3e6f60cff
	golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897
	golang.org/x/mod v0.3.1-0.20200828183125-ce943fd02449
	golang.org/x/net v0.0.0-20201029221708-28c70e62bb1d
	golang.org/x/sys v0.0.0-20201110211018-35f3e6cf4a65
	golang.org/x/text v0.3.4
	golang.org/x/tools v0.0.0-20201110201400-7099162a900a
	golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1

This change was created with a program from CL 256357 patch set 3
(which updates golang.org/x modules only) and the latest bundle tool,
but replacing golang.org/x/net version with a slightly older commit
golang/net@28c70e62bb due to #42498:

	$ updatestd -goroot=$HOME/gotip -branch=master
	> go version
	go version devel +ecc3f5112e Thu Nov 5 23:21:33 2020 +0000 darwin/amd64
	> go env GOROOT
	/Users/dmitshur/gotip
	> go version -m /Users/dmitshur/go/bin/bundle
	/Users/dmitshur/go/bin/bundle: go1.15.4
		path	golang.org/x/tools/cmd/bundle
		mod	golang.org/x/tools	v0.0.0-20201110201400-7099162a900a	h1:5E6TPwSBG74zT8xSrVc8W59K4ch4NFobVTnh2BYzHyU=
		dep	golang.org/x/mod	v0.3.0	h1:RM4zey1++hCTbCVQfnWeKs9/IEsaBLA8vTkd0WVtmH4=
		dep	golang.org/x/xerrors	v0.0.0-20200804184101-5ec99f83aff1	h1:go1bK/D/BFZV2I8cIQd1NKEZ+0owSTG1fDTci4IqFcE=

	updating module cmd in /Users/dmitshur/gotip/src/cmd
	skipping github.com/chzyer/logex (out of scope, it's not a golang.org/x dependency)
	skipping github.com/chzyer/readline (out of scope, it's not a golang.org/x dependency)
	skipping github.com/chzyer/test (out of scope, it's not a golang.org/x dependency)
	skipping github.com/google/pprof (out of scope, it's not a golang.org/x dependency)
	skipping github.com/ianlancetaylor/demangle (out of scope, it's not a golang.org/x dependency)
	skipping github.com/yuin/goldmark (out of scope, it's not a golang.org/x dependency)
	skipping rsc.io/pdf (out of scope, it's not a golang.org/x dependency)
	> go mod edit -go=1.16
	> go get -d golang.org/x/arch@52c3e6f60cffa0133a3f9b2fc7f6862504a6cba0 golang.org/x/crypto@9e8e0b390897c84cad53ebe9ed2d1d331a5394d9 golang.org/x/mod@ce943fd02449f621243c9ea6e64098e84752b92b golang.org/x/net@28c70e62bb1d140c3f2579fb7bb5095134d9cb1e golang.org/x/sync@67f06af15bc961c363a7260195bcd53487529a21 golang.org/x/sys@35f3e6cf4a65a85bc280e5fe63faed8ac8b25721 golang.org/x/text@22f1617af38ed4cd65b3b96e02bab267e560155c golang.org/x/tools@7099162a900ae8260c5b97cfaf5f374243dfa742 golang.org/x/xerrors@5ec99f83aff198f5fbd629d6c8d8eb38a04218ca
	> go mod tidy
	> go mod vendor

	updating module std in /Users/dmitshur/gotip/src
	> go mod edit -go=1.16
	> go get -d golang.org/x/crypto@9e8e0b390897c84cad53ebe9ed2d1d331a5394d9 golang.org/x/net@28c70e62bb1d140c3f2579fb7bb5095134d9cb1e golang.org/x/sys@35f3e6cf4a65a85bc280e5fe63faed8ac8b25721 golang.org/x/text@22f1617af38ed4cd65b3b96e02bab267e560155c golang.org/x/tools@7099162a900ae8260c5b97cfaf5f374243dfa742
	> go mod tidy
	> go mod vendor

	updating bundles in /Users/dmitshur/gotip/src
	> go generate -run=bundle std cmd

golang.org/x/net will be updated further later, after #42498 is fixed.

github.com/google/pprof and github.com/ianlancetaylor/demangle
contribute packages but are out of scope for this generated CL.

Also rename http2configureTransport in net/http to follow the internal
rename that happened in CL 264017 to fix the build.

For #36905.
Updates #41721.
Updates #42498.

Change-Id: Ifcd2e76f0406e389b6db88041ca51cd0a2115152
Reviewed-on: https://go-review.googlesource.com/c/go/+/266898
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-11 06:26:05 +00:00
Jay Conrod
8f2db14cd3 cmd/go: release note for -mod=readonly by default
For #40728
Fixes #42466

Change-Id: If2b21b37a590c243828c4fd278ab10b2705450f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/268859
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-10 18:42:47 +00:00
Daniel Martí
b2ef159db2 cmd/go: introduce the GOVERSION env variable
This is an extra variable available via 'go env', but not read from the
user's environment. It corresponds to the same string that
runtime.Version returns, assuming a program is built by the same version
of Go.

It's similar to the output of 'go version', but without the "go version"
prefix nor the "$GOOS/$GOARCH" suffix.

The main use case here is tools, which often use 'go env' to query basic
information about the installed Go tree. Its version was one missing
piece of information, which required an extra call to 'go version'
before this change.

Fixes #41116.

Change-Id: I5c9d8c2ba856c816c9f4c462ba73c907b3441445
Reviewed-on: https://go-review.googlesource.com/c/go/+/265637
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Trust: Daniel Martí <mvdan@mvdan.cc>
2020-11-10 18:33:37 +00:00
Cherry Zhang
1948c00b6e doc/go1.16: add release notes for darwin ports
Updates #38485, #42100.
For #40700.

Change-Id: I2caaa8482f13f9b79d4c2d2fdd242543981060cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/267718
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-10 18:03:53 +00:00
Michael Matloob
da3957ad0d Revert "cmd/go: don't copy cgo files to objdir when overlay is present"
This reverts CL 265758.

Reason for revert: longtest builders were failing

Change-Id: Ic6c3f3759399e45c1625c7c57f7aa67a1d90c601
Reviewed-on: https://go-review.googlesource.com/c/go/+/268900
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
2020-11-10 17:59:22 +00:00
Michael Anthony Knyszek
0e0a872a76 runtime: add lock rank partial order edge pollDesc -> spanSetSpine
This change adds a missing partial order edge. This edge captures of
the case of `wakep` getting called in `wakeNetPoller` which may then
allocate.

Fixes #42461.

Change-Id: Ie67d868e9cd24ed3cc94381dbf8a691dd13f068d
Reviewed-on: https://go-review.googlesource.com/c/go/+/268858
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2020-11-10 16:25:38 +00:00
Michael Anthony Knyszek
c68745b130 runtime: add lock rank partial order edge sweep -> mspanSpecial
This change adds a missing partial order edge. The edge captures the
case where the background sweeper handles some specials (i.e. finalizers
or memory profile sampling) and is otherwise correct.

Fixes #42472.

Change-Id: Ic45f6cc1635fd3d6bc6c91ff6f64d436088cef33
Reviewed-on: https://go-review.googlesource.com/c/go/+/268857
Trust: Michael Knyszek <mknyszek@google.com>
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2020-11-10 16:25:27 +00:00
Michael Matloob
e3de852f3e cmd/go: don't copy cgo files to objdir when overlay is present
The previous cl (golang.org/cl/262618) copied non-overlaid cgo files
to objdir, mostly to get around the issue that otherwise cgo-generated
files were written out with the wrong names (they'd get the base path
of the overlay file containing the replaced contents, instead of the
base path of the path whose contents are being replaced). So that CL
it would copy the files to objdir with the base path of the file
being replaced to circumvent that.

This CL changes cmd/go and cmd/cgo so that instead of copying
files, it passes the actual path of the file on disk either of
the original file (if it is not overlaid) or its replacement
file (if it is) as well as a flag --path_rewrite, newly added to
cmd/cgo, that specifies the actual original file path that corresponds
to the replaced files.

Updates #39958

Change-Id: Ic4aae5ef77fe405011fcdce7f6c162488d13daa2
Reviewed-on: https://go-review.googlesource.com/c/go/+/265758
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-10 16:09:36 +00:00
Michael Munday
189931296f cmd/internal/obj/s390x: fix SYNC instruction encoding
SYNC is supposed to correspond to 'fast-BCR-serialization' which is
encoded as 'bcr 14,0'. In CL 197178 I accidentally modified the
encoding to 'bcr 7,0' which is a no-op. This CL reverses that change.

Fixes #42479.

Change-Id: I9918d93d720f5e12acc3014cde20d2d32cc87ee5
Reviewed-on: https://go-review.googlesource.com/c/go/+/268797
Run-TryBot: Michael Munday <mike.munday@ibm.com>
Trust: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-10 15:48:04 +00:00
Mark Pulford
81322b9191 runtime/race: remove race from TestNoRaceAfterFunc2
For #14119

Change-Id: I2a9ae43da228cf5c3e38d1f0d1b0768145b6548f
Reviewed-on: https://go-review.googlesource.com/c/go/+/267998
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
2020-11-10 15:05:17 +00:00
Tobias Klauser
1c7650aa93 internal/poll: use copy_file_range only on Linux kernel >= 5.3
https://man7.org/linux/man-pages/man2/copy_file_range.2.html#VERSIONS states:

  A major rework of the kernel implementation occurred in 5.3.  Areas
  of the API that weren't clearly defined were clarified and the API
  bounds are much more strictly checked than on earlier kernels.
  Applications should target the behaviour and requirements of 5.3
  kernels.

Rather than attempting to detect the file system for source and
destination files (which means two additional statfs syscalls) and skip
copy_file_range in case of known defects (e.g. CIFS -> CIFS), just
assume copy_file_range to be broken on kernels < 5.3.

Fixes #42400

Change-Id: I3a531296182c1d6e341772cc9d2be5bf83e52575
Reviewed-on: https://go-review.googlesource.com/c/go/+/268338
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-10 09:01:41 +00:00
Bryan C. Mills
1642cd78b5 cmd/go: update test_race_install expected output for CL 266368
test_race_install checks that 'go test -i -race …' does not rebuild
already installed packages, by also passing '-v' and verifying that no
package names are printed to stderr.

CL 266368 added a deprecation message for the '-i' flag that caused
the stderr output to be non-empty, although it still does not print
any package names.

Updates #41696

Change-Id: I13e10e49b7c33139be9b13f24cb393c9f58fd85d
Reviewed-on: https://go-review.googlesource.com/c/go/+/268581
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-10 04:11:42 +00:00
Roland Shoemaker
9f39a43e0d crypto/tls: de-prioritize AES-GCM ciphers when lacking hardware support
When either the server or client are lacking hardware support for
AES-GCM ciphers, indicated by the server lacking the relevant
instructions and by the client not putting AES-GCM ciphers at the top
of its preference list, reorder the preference list to de-prioritize
AES-GCM based ciphers when they are adjacent to other AEAD ciphers.

Also updates a number of recorded openssl TLS tests which previously
only specified TLS 1.2 cipher preferences (using -cipher), but not
TLS 1.3 cipher preferences (using -ciphersuites), to specify both
preferences, making these tests more predictable.

Fixes #41181.

Change-Id: Ied896c96c095481e755aaff9ff0746fb4cb9568e
Reviewed-on: https://go-review.googlesource.com/c/go/+/262857
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Trust: Katie Hockman <katie@golang.org>
2020-11-10 01:40:27 +00:00
Dmitrii Okunev
d361691201 encoding/asn1: optimize asn1.Unmarshal
Used type-switch instead of switch by reflect.Type and added
BenchmarkUnmarshal.

name         old time/op    new time/op    delta
Marshal-8      28.1µs ± 2%    27.9µs ± 1%     ~     (p=0.094 n=9+9)
Unmarshal-8    6.45µs ± 1%    5.83µs ± 4%   -9.59%  (p=0.000 n=10+10)

name         old alloc/op   new alloc/op   delta
Marshal-8      8.26kB ± 0%    8.26kB ± 0%     ~     (all equal)
Unmarshal-8      840B ± 0%      488B ± 0%  -41.90%  (p=0.000 n=10+10)

name         old allocs/op  new allocs/op  delta
Marshal-8         363 ± 0%       363 ± 0%     ~     (all equal)
Unmarshal-8      50.0 ± 0%      43.0 ± 0%  -14.00%  (p=0.000 n=10+10)

Change-Id: I6b53833c7a3e2524f025453311841d03c1256a45
GitHub-Pull-Request: golang/go#36341
Reviewed-on: https://go-review.googlesource.com/c/go/+/268557
Trust: Filippo Valsorda <filippo@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2020-11-09 21:36:24 +00:00
Emmanuel Odeke
d4957122ee Revert "runtime: make stack traces of endless recursion print only top and bottom 50"
This reverts commit 3a81338622.

Reason for revert: Some edge cases not properly covered due to changes within runtime traceback generation since 2017, that need to be examined. This change landed very late in the Go1.16 cycle.

Change-Id: I8cf6f46ea0ef6161d878e79943e6c7cdac94bccf
Reviewed-on: https://go-review.googlesource.com/c/go/+/268577
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Emmanuel Odeke <emmanuel@orijtech.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2020-11-09 21:03:36 +00:00
Cherry Zhang
8badef4cf6 doc/articles/race_detector.html: add darwin/arm64
Updates #38485.

Change-Id: I46f515973c0a31d7c3e0e05ce006121c60c4041e
Reviewed-on: https://go-review.googlesource.com/c/go/+/268497
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-11-09 20:41:26 +00:00
Filippo Valsorda
5e181357c7 crypto/x509: drop the cgo implementation of root_darwin_amd64.go
This code was preserved just to do side-by-side testing while
transitioning to the Go implementation. There haven't been mismatch
issues, so drop the cgo code, which was making it hard to improve the Go
code without diverging.

Change-Id: I2a23039c31a46e88b94250aafbc98d4ea8daf22f
Reviewed-on: https://go-review.googlesource.com/c/go/+/232397
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-11-09 20:02:56 +00:00
Filippo Valsorda
d7fff1f2cf crypto/tls: ensure the server picked an advertised ALPN protocol
This is a SHALL in RFC 7301, Section 3.2.

Also some more cleanup after NPN, which worked the other way around
(with the possibility that the client could pick a protocol the server
did not suggest).

Change-Id: I83cc43ca1b3c686dfece8315436441c077065d82
Reviewed-on: https://go-review.googlesource.com/c/go/+/239748
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2020-11-09 19:48:28 +00:00
Michael Matloob
a2d01473ae cmd/go: support cgo files in overlays
This is a roll-forward of golang.org/cl/267197, which was reverted in
golang.org/cl/267357. It makes the following changes in addition to
the ones in the next paragraph: It avoids outputting trimpath
arguments for an overlay unless the overlay affects the package being
compiled (to avoid hitting windows command line argument limits), and
it fixes processing of regexps in the script test framework to treat
the first *non flag* argument to grep, stdout, and stderr as a regexp,
not just the first argument.

golang.org/cl/267917 was a roll-forward of golang.org/cl/262618, which
was reverted in golang.org/cl/267037. The only differences between
this CL and the original were the three calls to fflush from the C
files in build_overlay.txt, to guarantee that the string we were
expecting was
actually written out.

The CL requires rewriting the paths of the files passed to the cgo
tool toolchain to use the overlaid paths instead of the disk paths of
files. Because the directories of the overlaid paths don't exist in
general, the cgo tool have been updated to run in base.Cwd instead of
the package directory.

For #39958

Change-Id: I1bd96db257564bcfd95b3502aeca14d04bd28618
Reviewed-on: https://go-review.googlesource.com/c/go/+/267797
Trust: Michael Matloob <matloob@golang.org>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Michael Matloob <matloob@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-09 19:46:24 +00:00
Filippo Valsorda
01cdd365a9 crypto/tls: drop macFunction abstraction
Since we dropped SSLv3, there is only one MAC scheme, and it doesn't
need any state beyond a keyed HMAC, so we can replace the macFunction
with the hash.Hash it wraps.

Pointed out by mtp@.

Change-Id: I5545be0e6ccb34a3055fad7f6cb5f628ff748e9f
Reviewed-on: https://go-review.googlesource.com/c/go/+/251859
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
2020-11-09 19:00:00 +00:00
Johan Brandhorst
fdecb5c5b4 crypto/tls: add HandshakeContext method to Conn
Adds the (*tls.Conn).HandshakeContext method. This allows
us to pass the context provided down the call stack to
eventually reach the tls.ClientHelloInfo and
tls.CertificateRequestInfo structs.
These contexts are exposed to the user as read-only via Context()
methods.

This allows users of (*tls.Config).GetCertificate and
(*tls.Config).GetClientCertificate to use the context for
request scoped parameters and cancellation.

Replace uses of (*tls.Conn).Handshake with (*tls.Conn).HandshakeContext
where appropriate, to propagate existing contexts.

Fixes #32406

Change-Id: I33c228904fe82dcf57683b63627497d3eb841ff2
Reviewed-on: https://go-review.googlesource.com/c/go/+/246338
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-11-09 18:34:47 +00:00
Koen
858fa061ae crypto/x509: return additional chains from Verify on Windows
Previously windows only returned the certificate-chain with the highest quality.
This change makes it so chains with a potentially lower quality
originating from other root certificates are also returned by verify.

Tests in verify_test flagged with systemLax are now allowed to pass if the system returns additional chains

Fixes #40604

Change-Id: I66edc233219f581039d47a15f2200ff627154691
Reviewed-on: https://go-review.googlesource.com/c/go/+/257257
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-09 18:32:41 +00:00
Jay Conrod
8a368c63ec cmd/go: print deprecation messages for -i
build, install, and test will now print deprecation messages when the
-i flag is used. clean will continue to support -i.

For #41696

Change-Id: I956c235c487a872c5e6c1395388b4d6cd5ef817a
Reviewed-on: https://go-review.googlesource.com/c/go/+/266368
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-09 18:32:40 +00:00
Jay Conrod
65607683f5 cmd/go: print deprecation messages for 'go get' installing executables
For #40276

Change-Id: I5e631a4c9ce07f23640fb56eb455457bc55072c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/266360
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-11-09 18:32:36 +00:00
Bryan C. Mills
a6462a608d cmd/go: set FOSSIL_HOME in TestScript/mod_get_fossil
Without HOME or FOSSIL_HOME set, this test fails for me when run with
fossil 2.12.1.

Also verify that the 'go get' command produces an executable, which
helps to verify that the files extracted by fossil are not corrupted.

Updates #42323

Change-Id: Ie6f5d2eab6a6338e997a4f85591195e5bd9a0d37
Reviewed-on: https://go-review.googlesource.com/c/go/+/267884
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-09 17:38:50 +00:00
Bryan C. Mills
a07e4360a8 cmd/go: prefer 'go get -d' instead of 'go get' in script tests
'get -d' has somewhat narrower semantics and is generally faster.
We're deprecating the non-'-d' mode in CL 266360.

For #26472

Change-Id: Id4a324771f77b83e5f47043fd50b74e1c062390b
Reviewed-on: https://go-review.googlesource.com/c/go/+/267883
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-09 17:38:46 +00:00
Roland Shoemaker
51f4168812 crypto/x509: add additional convenience fields to CertificateRequest
Adds the following additional convenience fields to CertificateRequest:
* KeyUsage
* ExtKeyUsage
* UnknownExtKeyUsage
* IsCA
* MaxPathLen
* BasicConstraintsValid
* MaxPathLenZero
* SubjectKeyId
* PolicyIdentifier

These fields are parsed during ParseCertificateRequest and marshalled
during CreateCertificateRequest. The parsing/marshalling code is
factored out of parseCertificate and buildExtensions (which is renamed
buildCertExtensions). This has the side effect of making these methods
somewhat easier to read.

Documentation for the fields is copied from Certificate.

Example CSR created with all of these fields parsed with openssl:
$ openssl req -in ~/test-csr.pem -noout -text
Certificate Request:
    Data:
        Version: 0 (0x0)
        Subject:
        Subject Public Key Info:
            Public Key Algorithm: id-ecPublicKey
                Public-Key: (256 bit)
                pub:
                    04:a4:cb:64:35:8e:dd:8c:2b:a6:f1:aa:39:d1:be:
                    d0:b9:95:1e:59:19:82:76:28:d3:85:1b:c6:88:62:
                    e1:15:33:be:26:18:80:14:fe:f4:d4:91:66:4e:a4:
                    a4:47:bd:53:db:f7:2e:e3:31:ce:5f:86:cb:92:59:
                    93:bb:d0:7f:a2
                ASN1 OID: prime256v1
                NIST CURVE: P-256
        Attributes:
        Requested Extensions:
            X509v3 Key Usage: critical
                Certificate Sign
            X509v3 Extended Key Usage:
                Any Extended Key Usage, 1.2.3
            X509v3 Basic Constraints: critical
                CA:TRUE, pathlen:0
            X509v3 Subject Key Identifier:
                01:02:03
            X509v3 Certificate Policies:
                Policy: 1.2.3

    Signature Algorithm: ecdsa-with-SHA256
         30:45:02:21:00:a7:88:e5:96:d4:ad:ae:24:26:ab:5f:15:6a:
         3f:22:6d:0e:a6:ba:15:64:8d:78:34:f4:c4:7d:ac:37:b0:2a:
         84:02:20:68:44:f0:8e:8a:1b:c1:68:be:14:a6:e3:83:41:fd:
         2d:cc:00:aa:bc:50:f6:50:56:12:9e:a4:09:84:5c:bf:c1

Fixes #37172

Change-Id: Ife79d01e203827ef0ac3c787aa13c00d0751a1ec
Reviewed-on: https://go-review.googlesource.com/c/go/+/233163
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Trust: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
2020-11-09 17:24:38 +00:00
Joel Sing
cfea52b04c runtime: disable TestCrashDumpsAllThreads on openbsd/mips64
This test fails consistently on openbsd/mips64 - disable it until we can investigate
and resolve the issue.

Updates #42464

Change-Id: Ie640f776823137a967a12817ff18953207f558a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/268438
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-11-09 16:56:37 +00:00
Cherry Zhang
a444458112 cmd/compile: make sure linkname'd symbol is non-package
When a variable symbol is both imported (possibly through
inlining) and linkname'd, make sure its LSym is marked as
non-package for symbol indexing in the object file, so it is
resolved by name and dedup'd with the original definition.

Fixes #42401.

Change-Id: I8e90c0418c6f46a048945c5fdc06c022b77ed68d
Reviewed-on: https://go-review.googlesource.com/c/go/+/268178
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
2020-11-09 16:09:16 +00:00
Russ Cox
979e137609 cmd/go: add GOVCS setting to control version control usage
The go command runs commands like git and hg to download modules.
In the past, we have had problems with security bugs in version
control systems becoming security bugs in “go get”.

The original modules draft design removed use of these commands
entirely, saying:

> We want to move away from invoking version control tools such as bzr,
> fossil, git, hg, and svn to download source code. These fragment the
> ecosystem: packages developed using Bazaar or Fossil, for example, are
> effectively unavailable to users who cannot or choose not to install
> these tools. The version control tools have also been a source of
> exciting security problems. It would be good to move them outside the
> security perimeter.

The removal of these commands was not possible in the end: being able
to fetch directly from Git repos is too important, especially for
closed source. But the security exposure has not gone away.
We remain vulnerable to problems in VCS systems, especially the less
scrutinized ones.

This change adds a GOVCS setting to let users control which version
control systems are allowed by default.

It also changes the default allowed version control systems to git and hg
for public code and any version control system for private code
(import path or module path matched by the GOPRIVATE setting).

See the changes in alldocs.go for detailed documentation.
See #41730 for proposal and discussion.

Fixes #41730.

[Replay of CL 266420. See changes from Patch Set 1 for updates to fix
a few long tests.]

Change-Id: I4fe93804548956c42aea985368b4571bdb220f48
Reviewed-on: https://go-review.googlesource.com/c/go/+/267888
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-09 15:46:56 +00:00
Dmitri Shuralyov
cb4df98334 cmd/go: add /v2 to another require example
This is the same change as in CL 144917, but applied to a nearby line.

For #28374.

Change-Id: I6e1693d3a14e2517d863d1052a06c1156fc1edd4
Reviewed-on: https://go-review.googlesource.com/c/go/+/263437
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-11-09 15:20:26 +00:00
Filippo Valsorda
3fad58f12e crypto/x509: update iOS bundled roots to version 55161.140.3
Extended the sorting logic to be stable even when there are two roots
with the same name and notBefore timestamp, like the GlobalSign ones.

Updates #38843

Change-Id: Ie4db0bb8b6a8b5ffbb7390b6bd527fc0c3badaca
Reviewed-on: https://go-review.googlesource.com/c/go/+/266677
Reviewed-by: Katie Hockman <katie@golang.org>
Trust: Filippo Valsorda <filippo@golang.org>
2020-11-09 15:09:58 +00:00
Filippo Valsorda
564ec4867b crypto/tls: don't use CN in BuildNameToCertificate if SANs are present
Change-Id: I18d5b9fc392a6a52fbdd240254d6d9db838073a4
Reviewed-on: https://go-review.googlesource.com/c/go/+/266540
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2020-11-09 15:09:37 +00:00
Filippo Valsorda
feccfb8ada crypto/x509: use fingerprint map for (*CertPool).contains
This fell through the cracks from the CL 229917 comments.

Change-Id: I22584107f1e8111f9c523f45307dd50e1e5f4b8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/268339
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-11-09 14:41:38 +00:00