1
0
mirror of https://github.com/golang/go synced 2024-09-30 04:34:33 -06:00
Commit Graph

50308 Commits

Author SHA1 Message Date
Austin Clements
6113dacf32 cmd/compile: make -memprofilerate work
There are multiple things preventing the -memprofilerate flag from
working right now:

- CmdFlags.MemProfileRate has type int64, which is currently not
  supported by the compiler's reflection-based registerFlags.
  Unfortunately, rather than letting you know this, registerFlags
  simply ignores this field.

- Nothing consumes CmdFlags.MemProfileRate anyway. startProfile
  instead uses a package-local memprofilerate variable that is never
  set to anything.

Fix this by making CmdFlags.MemProfileRate an int (that's what
runtime.MemProfileRate is anyway) and using it in startProfile. While
we're here, prevent similar flag parsing bugs in the future by making
registerFlags panic if it encounters a flag field of unsupported type.

Change-Id: Ib9a1fcd8f4c5e9d7175a4fabc375f31e79774f9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/359955
Trust: Austin Clements <austin@google.com>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
2021-10-30 18:30:34 +00:00
Austin Clements
f582778ee9 cmd/compile: emit sensible go_asm.h consts for big ints
Currently, the compiler will emit any const that doesn't fit in an
int64 to go_asm.h like

    #define const_stackPreempt constant.intVal{val:(*big.Int)(0xc000c06c40)}

This happens because dumpasmhdr formats the constant.Value using the
verb "%#v". Since constant.Value doesn't implement the GoString()
method, this just prints the Go-syntax representation of the value.
This happens to work for small integer constants, which go/constant
represents directly as an int64, but not for integer constants that
don't fit in an int64, which go/constant represents as a big.Int.

Make these constants usable by changing the formatting verb to "%v",
which will call the String() method, giving a reasonable result in all
cases.

Change-Id: I365eeb88c8acfc43ff377cc873432269bde3f541
Reviewed-on: https://go-review.googlesource.com/c/go/+/359954
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2021-10-30 18:30:05 +00:00
Filippo Valsorda
30b5d6385e crypto/elliptic: move P-521 group logic to internal/nistec
This abstracts the clunky and not constant time math/big elliptic.Curve
compatibility layer away from the pure fiat-backed group logic.

Change-Id: I3b7a7495034d0c569b21c442ae36958763b8b2d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/320074
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-30 16:47:17 +00:00
Filippo Valsorda
d1dceafc29 crypto/elliptic: use a 4-bit sliding window for P-521 ScalarMult
name                    old time/op    new time/op    delta
pkg:crypto/elliptic goos:darwin goarch:amd64
ScalarBaseMult/P521-16    1.63ms ± 4%    1.00ms ± 1%  -38.69%  (p=0.000 n=10+8)
ScalarMult/P521-16        1.65ms ± 4%    0.99ms ± 2%  -40.15%  (p=0.000 n=10+10)
pkg:crypto/ecdsa goos:darwin goarch:amd64
Sign/P521-16              1.67ms ± 1%    1.12ms ± 2%  -32.82%  (p=0.000 n=8+10)
Verify/P521-16            3.10ms ± 2%    2.00ms ± 2%  -35.54%  (p=0.000 n=9+10)
GenerateKey/P521-16       1.53ms ± 1%    0.98ms ± 2%  -35.81%  (p=0.000 n=9+10)

Change-Id: I109e821399d71330a77d105496e227746cc3ea0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/320072
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-30 16:46:47 +00:00
Filippo Valsorda
e39b854a67 crypto/elliptic: use complete addition formulas for P-521
Complete formulas don't have exceptions for P = Q or P = 0, which makes
them significantly simpler and safer to implement. Notice how the
constant time IsZero checks are gone.

It's not free, but still well within the performance gains of CL 315271.

name                    old time/op    new time/op    delta
pkg:crypto/elliptic goos:darwin goarch:amd64
ScalarBaseMult/P521-16    1.34ms ± 3%    1.63ms ± 4%  +21.78%  (p=0.000 n=10+10)
ScalarMult/P521-16        1.35ms ± 3%    1.65ms ± 4%  +22.58%  (p=0.000 n=10+10)
pkg:crypto/ecdsa goos:darwin goarch:amd64
Sign/P521-16              1.45ms ± 2%    1.67ms ± 1%  +15.00%  (p=0.000 n=10+8)
Verify/P521-16            2.68ms ± 1%    3.10ms ± 2%  +16.02%  (p=0.000 n=10+9)
GenerateKey/P521-16       1.31ms ± 4%    1.53ms ± 1%  +16.89%  (p=0.000 n=10+9)

Change-Id: Ibd9a961e9865df68a1250aba739c190caf9a54de
Reviewed-on: https://go-review.googlesource.com/c/go/+/320071
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-30 16:45:25 +00:00
Dan Scales
5d6d9f5610 cmd/compile: use Structure() to get single underlying type of typeparam.
Use types2.Structure() to get single underlying type of typeparams, to
handle some unusual cases where a type param is constrained to a single
underlying struct or map type.

Fixes #48538

Change-Id: I289fb7b31d489f7586f2b04aeb1df74e15a9f965
Reviewed-on: https://go-review.googlesource.com/c/go/+/359335
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 23:25:18 +00:00
Dan Scales
8dfb447231 runtime: do not add open defer entry above a started open defer entry
Fix two defer bugs related to adding/removing open defer entries.
The bugs relate to the way that we add and remove open defer entries
from the defer chain. At the point of a panic, when we want to start
processing defer entries in order during the panic process, we need to
add entries to the defer chain for stack frames with open defers, since
the normal fast-defer code does not add these entries. We do this by
calling addOneOpenDeferFrame() at the beginning of each time around the
defer loop in gopanic(). Those defer entries get sorted with other open
and non-open-coded defer frames.

However, the tricky part is that we also need to remove defer entries if
they end not being needed because of a recover (which means we are back
to executing the defer code inline at function exits). But we need
to deal with multiple panics and in-process defers on the stack, so we
can't just remove all open-coded defers from the the defer chain during
a recover.

The fix (and new invariant) is that we should not add any open-coded
defers to the defer chain that are higher up the stack than an open-coded
defer that is in progress. We know that open-coded defer will still be
run until completed, and when it is completed, then a more outer frame
will be added (if there is one). This fits with existing code in gopanic
that only removes open-coded defer entries up to any defer in progress.

These bugs were because of the previous inconsistency between adding and
removing open defer entries, which meant that stale defer entries could
be left on the list, in these unusual cases with both recursive
panics plus multiple independent (non-nested) cases of panic & recover.

The test for #48898 was difficult to add to defer_test.go (while keeping
the failure mode), so I added as a go/test/fixedbug test instead.

Fixes #43920
Updates #43941
Fixes #48898

Change-Id: I593b77033e08c33094315abf8089fbc4cab07376
Reviewed-on: https://go-review.googlesource.com/c/go/+/356011
Trust: Dan Scales <danscales@google.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
2021-10-29 23:15:00 +00:00
Robert Findley
c812b97ec6 go/types: comp literals to accept type sets with single underlying types
This is a clean port of CL 357915 to go/types.

Change-Id: Idf5850a8bdcf3596c561c97bcd60539945139743
Reviewed-on: https://go-review.googlesource.com/c/go/+/359877
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:29:31 +00:00
Robert Findley
71e6ab8f95 go/types: slice exprs to accept type sets with single underlying types
This is a port of CL 357779 to go/types. A test error message was
repositioned on the sliced operand.

Change-Id: Ie775c128f70d9cb08a2eba54b8bc082134ec3200
Reviewed-on: https://go-review.googlesource.com/c/go/+/359876
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:27:54 +00:00
Robert Findley
02bd226b8a go/types: range clause to accept type sets with single underlying types
This is a port of CL 357778 to go/types, adjusted to include error codes
and to use the different range statement syntax in go/ast.

Change-Id: Id537c195cd33a8b422a366269ca8730c2a5bccf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/359875
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:27:40 +00:00
Robert Findley
3571ab58b8 go/types: make built-in to accept type sets with single underlying types
This is a straightforward port of CL 357776 to go/types.

Change-Id: I64220840a01f57cd7955f7d956b9aa8227473b46
Reviewed-on: https://go-review.googlesource.com/c/go/+/359874
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:27:26 +00:00
Filippo Valsorda
c76893d0da crypto/elliptic: refactor P-224 field implementation
Improved readability, replaced constant time bit masked operations with
named functions, added comments. The behavior of every function should
be unchanged.

The largest change is the logic that in p224Contract checks if the value
is greater than or equal to p. Instead of a lot of error-prone masking,
we run a throwaway subtraction chain and look at the final borrow bit.

We could also not throw away the subtraction chain output and do a
constant time select instead of another masked subtraction, but we'd
still have to fix any underflows (because these are unsaturated limbs
and they underflow at 2^32 instead of 2^28). That's similar but
different from the carry-down chain we do elsewhere in that function
(which does undeflow fixing and borrow at the same time). I thought
having both variations in the same function would be confusing. Here's
how it would look like.

	var b uint32
	var outMinusP p224FieldElement
	for i := 0; i < len(out); i++ {
		outMinusP[i], b = bits.Sub32(out[i], p224P[i], b)
	}
	for i := 0; i < 3; i++ {
		mask := maskIfNegative(outMinusP[i])
		outMinusP[i] += (1 << 28) & mask
		// Note we DON'T borrow here, because it happened above.
	}
	for i := 0; i < len(out); i++ {
		out[i] = select32(b, out[i], outMinusP[i])
	}

Change-Id: I00932e8f171eff7f441b45666dccfd219ecbbc50
Reviewed-on: https://go-review.googlesource.com/c/go/+/326311
Trust: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-29 22:01:56 +00:00
Michael Knyszek
994049a9ad runtime: add testing framework and basic tests for GC pacer
This change creates a formal exported interface for the GC pacer and
creates tests for it that simulate some series of GC cycles. The tests
are completely driven by the real pacer implementation, except for
assists, which are idealized (though revise is called repeatedly).

For #44167.

Change-Id: I0112242b07e7702595ca71001d781ad6c1fddd2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/353354
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 19:51:20 +00:00
Cherry Mui
5ec139fa78 runtime: remove unnecessary argument stores for panicIndex etc. on ARM64
If register ABI is used, no need to store the arguments to stack.
I forgot them in CL 323937.

Change-Id: I888af2b547a8fc97d13716bc8e8f3acd5c5bc127
Reviewed-on: https://go-review.googlesource.com/c/go/+/351609
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2021-10-29 19:35:47 +00:00
Bryan C. Mills
459293388c cmd/go: document that tests must not write to their source modules
Fixes #28386
Updates #48495

Change-Id: I76186077c7bbe3f8f608026ee1865de83fe169b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/359574
Trust: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 19:30:03 +00:00
Bryan C. Mills
6afdf01e53 cmd/go: make assertions more precise in TestScript/gcflags_patterns
This fixes a rare test flake observed in
https://build.golang.org/log/18e531785e7f8b3e6a6d5231787a9d8bdd4a60ea.

Change-Id: I0986752f5c1abc626070b9f8ecf34c629a15cd2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/359735
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 18:56:29 +00:00
Michael Knyszek
5ec2427357 runtime: pass nanotime and gomaxprocs into startCycle and endCycle explicitly
This is to facilitate testing of the pacer, since otherwise this is
accessing global state, which is impossible to stub out properly.

For #44167.

Change-Id: I52c3b51fc0ffff38e3bbe534bd66e5761c0003a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/353353
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:59 +00:00
Michael Anthony Knyszek
9da64156a6 runtime: track scannable globals space
For #44167.

Change-Id: I2cd13229d88f630451fabd113b0e5a04841e9e79
Reviewed-on: https://go-review.googlesource.com/c/go/+/309590
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:29 +00:00
Michael Anthony Knyszek
9ac1ee2d46 runtime: track the amount of scannable allocated stack for the GC pacer
This change adds two fields to gcControllerState: stackScan, used for
pacing decisions, and scannableStackSize, which directly tracks the
amount of space allocated for inuse stacks that will be scanned.

scannableStackSize is not updated directly, but is instead flushed from
each P when at an least 8 KiB delta has accumulated. This helps reduce
issues with atomics contention for newly created goroutines. Stack
growth paths are largely unaffected.

StackGrowth-48			51.4ns ± 0%	51.4ns ± 0%	~	(p=0.927 n=10+10)
StackGrowthDeep-48		6.14µs ± 3%	6.25µs ± 4%	~	(p=0.090 n=10+9)
CreateGoroutines-48		273ns ± 1%	273ns ± 1%	~	(p=0.676 n=9+10)
CreateGoroutinesParallel-48	65.5ns ± 5%	66.6ns ± 7%	~	(p=0.340 n=9+9)
CreateGoroutinesCapture-48	2.06µs ± 1%	2.07µs ± 4%	~	(p=0.217 n=10+10)
CreateGoroutinesSingle-48	550ns ± 3%	563ns ± 4%	+2.41%	(p=0.034 n=8+10)

For #44167.

Change-Id: Id1800d41d3a6c211b43aeb5681c57c0dc8880daf
Reviewed-on: https://go-review.googlesource.com/c/go/+/309589
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:20 +00:00
Michael Anthony Knyszek
8e112a7c2a runtime: move heapLive and heapScan updates into a method
This change moves heapLive and heapScan updates on gcController into a
method for better testability. It's also less error-prone because code
that updates these fields needs to remember to emit traces and/or call
gcController.revise; this method now handles those cases.

For #44167.

Change-Id: I3d6f2e7abb22def27c93feacff50162b0b074da2
Reviewed-on: https://go-review.googlesource.com/c/go/+/309275
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:01 +00:00
Michael Anthony Knyszek
4a5d78ff8b runtime: move pacer time updates and state resets into methods
Currently GC pacer updates are applied somewhat haphazardly via direct
field access. To facilitate ease of testing, move these field updates
into methods. Further CLs will move more of these updates into methods.

For #44167.

Change-Id: I25b10d2219ae27b356b5f236d44827546c86578d
Reviewed-on: https://go-review.googlesource.com/c/go/+/309274
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:34:43 +00:00
Michael Anthony Knyszek
903f313d8b runtime: fix unclosed GCSweepStart trace event
CL 333389 erroneously moved traceGCSweepDone inside the sl.valid block
that it introduced in mcentral.cacheSpan, when it should have left it
outside that scope, because the trace event is created unconditionally
at the top of the method.

Fixes #49231.

Change-Id: If719c05063d4f4d330a320da6dc00d1e9d8102e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/359775
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 18:34:24 +00:00
Michael Anthony Knyszek
413672fc84 runtime: detangle sweeper pacing from GC pacing
The sweeper's pacing state is global, so detangle it from the GC pacer's
state updates so that the GC pacer can be tested.

For #44167.

Change-Id: Ibcea989cd435b73c5891f777d9f95f9604e03bd1
Reviewed-on: https://go-review.googlesource.com/c/go/+/309273
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:23:03 +00:00
Zhi Zheng
353d5b6c53 runtime: enable crash dump creation on Windows
This change provides ability to create dumps on Windows that can be used by
"dlv core" command. Currently only full dumps can be correctly read by Delve.
Below are the steps to create and use the dumps.

1. Configure Windows OS to collect dumps before running the program.
Instructions on how to do the configuration are here:
https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps.
In order for Delve to read the dump, set the DumpType to full dump, i.e. DumpType=2.

2. Go program only generates dumps when the environment variable GOTRACEBACK
is set to crash. Run command "set GOTRACEBACK=crash" before running the program.

3. Dump files will be generated in %LOCALAPPDATA%\CrashDumps

4. Use Delve command "dlv core" to open the dump, e.g.: "dlv core a.exe a.exe.3840.dmp".

Fixes #20498

Change-Id: Ib9aa82e7aea9da19594dc49348876997b24e9600
Reviewed-on: https://go-review.googlesource.com/c/go/+/307372
Run-TryBot: Alessandro Arzilli <alessandro.arzilli@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Patrik Nyblom <pnyb@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
2021-10-29 17:44:52 +00:00
Michael Anthony Knyszek
d8fc7f785e runtime: clean up allocation zeroing
Currently, the runtime zeroes allocations in several ways. First, small
object spans are always zeroed if they come from mheap, and their slots
are zeroed later in mallocgc if needed. Second, large object spans
(objects that have their own spans) plumb the need for zeroing down into
mheap. Thirdly, large objects that have no pointers have their zeroing
delayed until after preemption is reenabled, but before returning in
mallocgc.

All of this has two consequences:
1. Spans for small objects that come from mheap are sometimes
   unnecessarily zeroed, even if the mallocgc call that created them
   doesn't need the object slot to be zeroed.
2. This is all messy and difficult to reason about.

This CL simplifies this code, resolving both (1) and (2). First, it
recognizes that zeroing in mheap is unnecessary for small object spans;
mallocgc and its callees in mcache and mcentral, by design, are *always*
able to deal with non-zeroed spans. They must, for they deal with
recycled spans all the time. Once this fact is made clear, the only
remaining use of zeroing in mheap is for large objects.

As a result, this CL lifts mheap zeroing for large objects into
mallocgc, to parallel all the other codepaths in mallocgc. This is makes
the large object allocation code less surprising.

Next, this CL sets the flag for the delayed zeroing explicitly in the one
case where it matters, and inverts and renames the flag from isZeroed to
delayZeroing.

Finally, it adds a check to make sure that only pointer-free allocations
take the delayed zeroing codepath, as an extra safety measure.

Benchmark results: https://perf.golang.org/search?q=upload:20211028.8

Inspired by tapir.liu@gmail.com's CL 343470.

Change-Id: I7e1296adc19ce8a02c8d93a0a5082aefb2673e8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/359477
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2021-10-29 17:44:15 +00:00
Bryan C. Mills
a88575d662 Revert "cmd/go: remove support for -buildmode=shared"
This reverts CL 359096.

Updates #47788.

Reason for revert: -buildmode=shared may have actually been working in a few very specific cases. We should not remove -buildmode=shared until we have implemented an alternative to support those few cases.

Change-Id: Ia962b06abacc11f6f29fc29d092773be175e32f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/359575
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2021-10-29 17:32:24 +00:00
Russ Cox
645d07819b sync: add Mutex.TryLock, RWMutex.TryLock, RWMutex.TryRLock
Use of these functions is almost (but not) always a bad idea.

Very rarely they are necessary, and third-party implementations
(using a mutex and an atomic word, say) cannot integrate as well
with the race detector as implmentations in package sync itself.

Fixes #45435.

Change-Id: I0128ca48ef5e0a3b09c913f0f3a7ee5c56388000
Reviewed-on: https://go-review.googlesource.com/c/go/+/319769
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 17:13:13 +00:00
Michael Anthony Knyszek
3aecb3a8f7 runtime: fix sweep termination condition
Currently, there is a chance that the sweep termination condition could
flap, causing e.g. runtime.GC to return before all sweep work has not
only been drained, but also completed. CL 307915 and CL 307916 attempted
to fix this problem, but it is still possible that mheap_.sweepDrained is
marked before any outstanding sweepers are accounted for in
mheap_.sweepers, leaving a window in which a thread could observe
isSweepDone as true before it actually was (and after some time it would
revert to false, then true again, depending on the number of outstanding
sweepers at that point).

This change fixes the sweep termination condition by merging
mheap_.sweepers and mheap_.sweepDrained into a single atomic value.

This value is updated such that a new potential sweeper will increment
the oustanding sweeper count iff there are still outstanding spans to be
swept without an outstanding sweeper to pick them up. This design
simplifies the sweep termination condition into a single atomic load and
comparison and ensures the condition never flaps.

Updates #46500.
Fixes #45315.

Change-Id: I6d69aff156b8d48428c4cc8cfdbf28be346dbf04
Reviewed-on: https://go-review.googlesource.com/c/go/+/333389
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2021-10-29 17:12:47 +00:00
Bryan C. Mills
f288526374 cmd/go: test that the reserved paths "example" and "test" are not used
Proposal #37641 reserved the paths "example" and "test" for end users,
so no path beginning with either of those elements may be added to the
standard library.

(We are unlikely to add them by accident, but this test codifies the
policy accepted in the proposal.)

Fixes #37641

Change-Id: I999064180336c923d4918a1176c371b8f5666f8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/359576
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-29 04:21:07 +00:00
Meng Zhuo
2c7cdec2a7 io: remove UTF8 specified document for RuneReader
RuneReader is fine with UTF16 or any other character encoding

Fixes #49178

Change-Id: I08a5ac205e095349d451d3b60411eaeebc3aa563
Reviewed-on: https://go-review.googlesource.com/c/go/+/359334
Trust: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 02:16:47 +00:00
Rob Pike
d3d8852d89 text/template: use reflect.Value.FieldByIndexErr
to avoid a crash on a nil pointer as an embedded field.

Fixes #48215

Change-Id: I214faa6e3cf08cdec1c01035e4bbca0900c6a408
Reviewed-on: https://go-review.googlesource.com/c/go/+/357963
Trust: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 02:12:05 +00:00
Rob Pike
33c392f72d reflect: add FieldByIndexErr
This new function, although different in signature from other
reflect functions, allows the caller to avoid the panic caused by
nil embedded fields in calls to FieldByIndex.

Fixes #48218

Change-Id: I447f135bb789148c27ae3f2f23dcf43094f4c1de
Reviewed-on: https://go-review.googlesource.com/c/go/+/357962
Trust: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 02:11:42 +00:00
Russ Cox
af05d8be3d all: manual fixups for //go:build vs // +build
Update many generators, also handle files that were not part of the
standard build during 'go fix' in CL 344955.

Fixes #41184.

Change-Id: I1edc684e8101882dcd11f75c6745c266fccfe9e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/359476
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 22:38:00 +00:00
Bryan C. Mills
d0751b1ad8 cmd/go: test with SIGTERM in test_fuzz_non_crash_signal
The test currently uses only SIGKILL (which cannot be trapped by the
worker process) and SIGILL (which is handled by the Go runtime, and
does pretty unambiguously indicate a crash). It is missing a third
(and perhaps more likely) case: a signal like SIGTERM that is handled
by the Go runtime and causes the test to terminate, but nonetheless
does not indicate a crash.

Change-Id: I0595a0917e977e8eb2c24ae8b706d734ce3b2f4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/359482
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-10-28 20:50:34 +00:00
Bryan C. Mills
b9540246ba cmd/go: disallow the -fuzz flag for tests outside the main module
Normally, when fuzzing identifies a failure it saves the failing input
to the package's testdata directory. However, the testdata directory
for packages outside the main module is normally not writable — and
when it is, writing to a testdata directory inside the module cache
would corrupt the checksum for that module (and permanently alter the
behavior of that version of the module globally).

In the future we could consider a flag to allow failures to be saved
to an alternate location, or perhaps in the build cache; or, we could
suppress writes entirely and rely on the user to identify and copy the
failing input from the test log. However, it's a bit late in the cycle
for that big a design decision right now. For Go 1.18, we will just
enforce that the package to be fuzzed resides in the main module,
which is typically a writable VCS checkout.

Fixes #48495

Change-Id: I8d3d56372394b1aaa94fa920399c659363fa17fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/359414
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-10-28 20:43:39 +00:00
Bryan C. Mills
e741e2fe0e cmd/go: consolidate fuzz-support checks
We had been repeating conditions for specific platforms and
architectures to gate fuzzing tests, but the more of those tests we
add the more we will have to update if the set of supported platforms
and archictures expands over time.

We also ought to provide a friendlier error message when
'go test -fuzz' is used on non-supported platforms.

This change adds predicates in cmd/internal/sys, which already
contains similar predicates for related functionality (such as the
race detector), and uses those predicates in 'go test' and TestScript.

For #48495

Change-Id: If24c3997aeb4d201258e21e5b6cf4f7c08fbadd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/359481
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-10-28 20:43:02 +00:00
Bryan C. Mills
834e36ec77 net/http: restore generated // +build comment
The upstream cmd/bundle tool does not yet omit it,
and the longtest builders test that the generated
file is not modified.

For #41184.

Change-Id: Ib68f532139ed1436cbaf3a756f300fe60f520cab
Reviewed-on: https://go-review.googlesource.com/c/go/+/359484
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-28 20:39:36 +00:00
Cherry Mui
5e217284dc cmd/internal/goobj: bump version number
There have a few updates to the Go object file in this cycle (e.g.
FuncInfo format change, some changes in constant values), and it
is not compatible with the old tools. Bump up the version number.

Change-Id: Id176979b139c76ded2c50f2678eb313934326d6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/359483
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-28 20:27:44 +00:00
Russ Cox
f229e7031a all: go fix -fix=buildtag std cmd (except for bootstrap deps, vendor)
When these packages are released as part of Go 1.18,
Go 1.16 will no longer be supported, so we can remove
the +build tags in these files.

Ran go fix -fix=buildtag std cmd and then reverted the bootstrapDirs
as defined in src/cmd/dist/buildtool.go, which need to continue
to build with Go 1.4 for now.

Also reverted src/vendor and src/cmd/vendor, which will need
to be updated in their own repos first.

Manual changes in runtime/pprof/mprof_test.go to adjust line numbers.

For #41184.

Change-Id: Ic0f93f7091295b6abc76ed5cd6e6746e1280861e
Reviewed-on: https://go-review.googlesource.com/c/go/+/344955
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 18:17:57 +00:00
Russ Cox
ed475fe53b cmd/go: update for //go:build lines
cmd/go has its own //go:build evaluator, which is needed for
patterns like 'all'. The code is a modified copy of some unexported
routines from the go/build package. Update it by copying those
again and re-modifying them. The modifications are primarily the new
func eval and also ignoring errors.

This CL will need to be backported to Go 1.17, or else Go 1.17
will break when faced with certain //go:build-only repos during
'go list all' or 'go mod tidy'.

For #41184.
Fixes #49198.

Change-Id: Ie0fe3caa8d49004935ecd76d7977f767fe50e317
Reviewed-on: https://go-review.googlesource.com/c/go/+/359355
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 18:10:59 +00:00
Michael Pratt
2bc8ed8e9c runtime: normalize sigprof traceback flags
Each gentraceback call uses a different set of flags. Combine these into
a common variable, only adjusted as necessary.

The effective changes here are:

* cgo traceback now has _TraceJumpStack. This is a no-op since it
  already passes curg.
* libcall traceback now has _TraceJumpStack. This is a behavior change
  and will allow following stack transitions if a libcall is performed on
  g0.
* VDSO traceback drops _TraceTrap. vdsoPC is a return address, so
  _TraceTrap was not necessary.

Change-Id: I351b3cb8dc77df7466795d5fbf2bd8f30bba2d37
Reviewed-on: https://go-review.googlesource.com/c/go/+/358900
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-10-28 18:10:50 +00:00
Neil Alexander
266626211e net/http: guarantee that the Transport dial functions are respected in js/wasm
The net/http package has a documented contract that if DialContext, DialDLSContext, Dial or DialTLS are specified in an instance of Transport, that they will be used to set up the connection. If they are not specified, then a reasonable fallback is made (e.g. using the net package).

This is ordinarily true, except for when compiling for the js/wasm target, where the browser's Fetch API is preferred in all cases (except for when it is undefined/unavailable) and therefore the dial functions are all ignored. As a result, the http.Transport implementation under js/wasm doesn't meet that contract.

This PR updates the RoundTrip behaviour of http.Transport so that if DialContext, DialTLSContext, Dial or DialTLS are specified, they are used as expected. The Fetch API will be used as a fallback if they are not specified.

Fixes #27495

Change-Id: I88c6eb6ffdd077827b421d606f3e60ebdafd538f
GitHub-Last-Rev: 948a0ed97e
GitHub-Pull-Request: golang/go#46923
Reviewed-on: https://go-review.googlesource.com/c/go/+/330852
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-28 17:57:57 +00:00
Russ Cox
3124968848 cmd/fix: add buildtag fix
Now that Go 1.17 is out and Go 1.15 is unsupported,
removing // +build lines can be done safely: in the worst case,
if code is compiled using Go 1.16 the toolchain will detect
the presence of a //go:build without // +build and fail the build.
(It will not silently choose the wrong files.)

Note that +build lines will continue to work in Go sources forever.
This just provides a mechanism for users who are done with
Go 1.16 to remove them easily, by running "go fix".

Also update for new generics AST.

For #41184.
Fixes #48978.

Change-Id: I11a432c319e5abd05ad68dda9ccd7a7fdcc8bbb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/240611
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 17:52:59 +00:00
Michael Pratt
6bd0e7fa8a runtime: fix backward error message
This was added in CL 339990.

Change-Id: I4b0f97bf1a3926e37a42f77e149dcab3b7b75a63
Reviewed-on: https://go-review.googlesource.com/c/go/+/359255
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-10-28 16:54:58 +00:00
Michael Pratt
5fce1d9613 runtime: disable TestSegv panic check on darwin, illumos, solaris
CL 339990 made this test more strict, exposing pre-existing issues on
these OSes. Skip for now until they can be resolved.

Updates #49182

Change-Id: I3ac400dcd21b801bf4ab4eeb630e23b5c66ba563
Reviewed-on: https://go-review.googlesource.com/c/go/+/359254
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-28 16:54:29 +00:00
Robert Findley
6f0185bf38 go/types: factor out slice elem computation for copy built-in
This is a port of CL 357413 to go/types. Some test constraints are also
updated to remove 'interface', to coincide with the corresponding test
data file in types2.

Change-Id: I5248190501c2e4381eb7625f8d4fb269301d6e16
Reviewed-on: https://go-review.googlesource.com/c/go/+/359138
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-28 16:08:36 +00:00
Roland Shoemaker
61536ec030 debug/macho: fail on invalid dynamic symbol table command
Fail out when loading a file that contains a dynamic symbol table
command that indicates a larger number of symbols than exist in the
loaded symbol table.

Thanks to Burak Çarıkçı - Yunus Yıldırım (CT-Zer0 Crypttech) for
reporting this issue.

Fixes #48990
Fixes CVE-2021-41771

Change-Id: Ic3d6e6529241afcc959544b326b21b663262bad5
Reviewed-on: https://go-review.googlesource.com/c/go/+/355990
Reviewed-by: Julie Qiu <julie@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Roland Shoemaker <roland@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Katie Hockman <katie@golang.org>
2021-10-28 15:55:26 +00:00
Robert Findley
278b9b3a4c go/types: implement generic conversions
This is a port of 4 CLs from types2: CL 356010, CL 357333, CL 357410,
and CL 357249. These 4 CLs are all related to implementing conversions,
and porting them together saved time (particularly because go/types was
already threading a *reason argument in some places).

Change-Id: Ic89b608d7096b61bfb9f7d71fdae2cc50b0ed70e
Reviewed-on: https://go-review.googlesource.com/c/go/+/359137
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-28 15:35:25 +00:00
Robert Findley
4f1c9aace0 go/types: add debugging support for delayed actions
This is a port of CL 355871 to go/types.

Change-Id: I2dbc3c625c16b545a271a19606ef34ce04a4a6df
Reviewed-on: https://go-review.googlesource.com/c/go/+/359136
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-28 15:34:49 +00:00
Robert Findley
925ea2dfc4 go/types, types2, go/ast, go/parser: remove support for type lists
This is a rough port of CL 354131 to go/* libraries, though in practice
I just tried to reconcile any places where the phrase "type list"
occurred in the source. This resulted in adjusting quite a bit more code
than initially expected, including a few lingering cases in the
compiler.

Change-Id: Ie62a9e1aeb831b73931bc4c78bbb6ccb24f53fb0
Reviewed-on: https://go-review.googlesource.com/c/go/+/359135
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-28 15:34:22 +00:00