1
0
mirror of https://github.com/golang/go synced 2024-11-24 08:40:14 -07:00
Commit Graph

50320 Commits

Author SHA1 Message Date
Robert Griesemer
e2e910ef30 cmd/compile/internal/types2: remove subscripts from type parameter names
This is a port of CL 357814 from go/types to types2 with minor
adjustments due to small differences in error handling code.

Change-Id: I72ecc4532e8349f569cabb38006f3d8ff517bf30
Reviewed-on: https://go-review.googlesource.com/c/go/+/360276
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-11-01 15:55:25 +00:00
Robert Griesemer
d2b512160e cmd/compile/internal/types2: differently named types are not assignable
When checking assignability, a value of a named type (incl. a type parameter)
can never be assigned to a variable of a differently named type. Specifically,
if the types are two differently named type parameters, then values of one are
never assignable to variables of the other.

This CL clarifies the assignabiliy rules and simplifies the implementation.

Fixes #49242.

Change-Id: Id72a2c9bed5cdb726855e7a707137db1009e7953
Reviewed-on: https://go-review.googlesource.com/c/go/+/360274
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-11-01 15:55:20 +00:00
Martin Möhrmann
82f902ae8e strings: align Clone output with string([]byte(""))
Add a comment how strings of length 0 are treated and
that they always will result in the return of a string
equal to the constant string "".

The previous implementation would return a string header that uses
runtime.zerobase as the backing array pointer while the string constant
"" has 0 as pointer value.

Using 0 has the backing array pointer is also the behaviour of
string([]byte(input)) which makes the new behaviour a better drop in
replacement.

Change-Id: Ic5460e9494b6708edbdfa4361e878d50db54ba10
Reviewed-on: https://go-review.googlesource.com/c/go/+/360255
Trust: Martin Möhrmann <martin@golang.org>
Run-TryBot: Martin Möhrmann <martin@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2021-11-01 14:45:28 +00:00
Archana R
4056934e48 test/codegen: updated arithmetic tests to verify on ppc64,ppc64le
Updated multiple tests in test/codegen/arithmetic.go to verify
on ppc64/ppc64le as well

Change-Id: I79ca9f87017ea31147a4ba16f5d42ba0fcae64e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/358546
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-11-01 13:12:37 +00:00
Paul E. Murphy
732db407d3 cmd/link: make ppc64le __glink_PLTresolve position-independent
This code is only generated when linking cgo internally with fixed
position code. This feature of the internal linker is only supported
on ppc64le/linux targets. This moves ppc64le/linux a little closer
to supporting PIE when internal linking.

This is more similar to the implementation suggested in the power
architecture elfv2 supplement, and works with both PIE and static
code.

Change-Id: I0b64e1c22b9e07b5151378d2ab19ee0e50405fc5
Reviewed-on: https://go-review.googlesource.com/c/go/+/357332
Run-TryBot: Paul Murphy <murp@ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2021-11-01 13:08:16 +00:00
Bryan C. Mills
fde4cc2a31 testing: reduce memory used by subtest names
This is heavily based on CL 341336 by Joe Tsai and CL 351452 by
Jay Conrod.

T.Run and T.Name use a map[string]int64 to hold the next suffix to use
when duplicate names are passed to T.Run. This map necessarily retains
one entry per unique name. However, it's a waste of memory to retain
one entry per duplicate name: when we encounter the Nth duplicate, we
know that names 00 through N-1 have been used just by looking at N.

We do still need to store (and check for collisions againsts) explicit
names provided by the caller. For example, if the user passes in "a",
then "a#01", then "a" again, we cannot deduplicate the second "a" to
"a#01" — we need to instead skip ahead to "a#02". We can do so by
checking the count of "a", then generating a proposed deduplicated
name, then double-checking that proposed name against only the
explicit names so far.

This somewhat reduces memory usage for tests that spawn large numbers
of duplicate subtests, but doesn't solve the problem of memory growth
for fuzzing — we still have to track all of the explicit,
user-provided subtest names, and in a long-running fuzz test that set
alone may be unbounded.

This fixes memory growth for the example described in
https://golang.org/issue/44517#issuecomment-897104060,
but not the one in
https://golang.org/issue/44517#issuecomment-933825661.

For #44517

Change-Id: Ia159ecfcf44561ba67508d3af6377c27856df31d
Reviewed-on: https://go-review.googlesource.com/c/go/+/354749
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2021-11-01 02:47:30 +00:00
Martin Möhrmann
89c527007f reflect: avoid stack copies of hiter
Use a pointer reciever to avoid copying the hiter struct when
checking if it is intialized.

Found through profiling that showed reflect map iteration spending
a good amount of time in duffcopy.

This change will also help other MapIter methods checking hiter struct
initialization like Value() and Key().

name            old time/op  new time/op  delta
MapIterNext-12  97.9ns ± 4%  83.8ns ± 2%  -14.37%  (p=0.000 n=10+10)

Change-Id: I73ab964fa28061ee7e6d5c663a85048bd2e0274e
Reviewed-on: https://go-review.googlesource.com/c/go/+/360254
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Trust: Martin Möhrmann <martin@golang.org>
2021-10-31 18:39:05 +00:00
Alexander Melentyev
fd09e88722 testing: remove redundant type conversion
Change-Id: Ib6415bb09af02e9ebc0099f50bf0fd7b572f2516
GitHub-Last-Rev: ebaa849d66
GitHub-Pull-Request: golang/go#49244
Reviewed-on: https://go-review.googlesource.com/c/go/+/360214
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
2021-10-31 18:13:09 +00:00
Alexander Melentyev
3fa9ee49a0 bytes: remove redundant type conversion
Change-Id: Icafb2d9fd11b23d3acaf9df07e7f90d8bcd829fa
GitHub-Last-Rev: 2b76646055
GitHub-Pull-Request: golang/go#49243
Reviewed-on: https://go-review.googlesource.com/c/go/+/360096
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Martin Möhrmann <martin@golang.org>
2021-10-31 17:52:41 +00:00
Robert Findley
243c5ae019 go/parser: simplify parsing of array or slice constraint types
Simplify the parsing of array or slice constraint types added in CL
359134, following the port in CL 360135.

Change-Id: Ia86d4b0149a222423d3b19623dd39d4aeb23857d
Reviewed-on: https://go-review.googlesource.com/c/go/+/360115
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-31 15:36:50 +00:00
Nigel Tao
8e3d5f0bb3 image/draw: rename and reorder some benchmarks
The intention is for BenchmarkFooBar functions to map 1:1 to drawFooBar
functions. Recent draw.go changes have added more drawFooBar functions
and have further modified the mapping, as fallback drawFooBar functions
aren't invoked as often as they used to.

This commit restores the 1:1 mapping and reorganizes the BenchmarkFooBar
functions in the same order as the matching drawFooBar functions appear.

Also modify a TestDraw test case from vgradGreen(255) = {0, 136, 0, 255}
to vgradGreen(90) = {0, 48, 0, 90}. Doing so matches the existing "The
source pixel is {0, 48, 0, 90}" comment but also makes for a more
interesting test case, as the source pixel is no longer fully opaque.
Fully opaque is already covered by the vgradGray() test case on the next
line.

Also fix a "variable source" comment copy-pasto when the source image is
actually uniform, not variable.

Also add a func DrawMask type switch comment about interface types.

Change-Id: I828e71f2ee8ec617f523c8aafb118fb7ba166876
Reviewed-on: https://go-review.googlesource.com/c/go/+/358974
Trust: Nigel Tao <nigeltao@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Nigel Tao <nigeltao@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2021-10-31 08:29:02 +00:00
Robert Griesemer
0bef30d28a cmd/compile/internal/syntax: fix parsing of array or slice constraint types
This is a port of the idea used in CL 359134 from go/parser to syntax,
with adjustments due to the slightly different structure of the two
parsers, and some refactoring to simplify the logic.

Fixes #49175.

Change-Id: Ib4955bde708f2b08345f35523e6094c03ab3076c
Reviewed-on: https://go-review.googlesource.com/c/go/+/360135
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
2021-10-31 02:24:29 +00:00
Austin Clements
6113dacf32 cmd/compile: make -memprofilerate work
There are multiple things preventing the -memprofilerate flag from
working right now:

- CmdFlags.MemProfileRate has type int64, which is currently not
  supported by the compiler's reflection-based registerFlags.
  Unfortunately, rather than letting you know this, registerFlags
  simply ignores this field.

- Nothing consumes CmdFlags.MemProfileRate anyway. startProfile
  instead uses a package-local memprofilerate variable that is never
  set to anything.

Fix this by making CmdFlags.MemProfileRate an int (that's what
runtime.MemProfileRate is anyway) and using it in startProfile. While
we're here, prevent similar flag parsing bugs in the future by making
registerFlags panic if it encounters a flag field of unsupported type.

Change-Id: Ib9a1fcd8f4c5e9d7175a4fabc375f31e79774f9a
Reviewed-on: https://go-review.googlesource.com/c/go/+/359955
Trust: Austin Clements <austin@google.com>
Trust: Josh Bleecher Snyder <josharian@gmail.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
2021-10-30 18:30:34 +00:00
Austin Clements
f582778ee9 cmd/compile: emit sensible go_asm.h consts for big ints
Currently, the compiler will emit any const that doesn't fit in an
int64 to go_asm.h like

    #define const_stackPreempt constant.intVal{val:(*big.Int)(0xc000c06c40)}

This happens because dumpasmhdr formats the constant.Value using the
verb "%#v". Since constant.Value doesn't implement the GoString()
method, this just prints the Go-syntax representation of the value.
This happens to work for small integer constants, which go/constant
represents directly as an int64, but not for integer constants that
don't fit in an int64, which go/constant represents as a big.Int.

Make these constants usable by changing the formatting verb to "%v",
which will call the String() method, giving a reasonable result in all
cases.

Change-Id: I365eeb88c8acfc43ff377cc873432269bde3f541
Reviewed-on: https://go-review.googlesource.com/c/go/+/359954
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2021-10-30 18:30:05 +00:00
Filippo Valsorda
30b5d6385e crypto/elliptic: move P-521 group logic to internal/nistec
This abstracts the clunky and not constant time math/big elliptic.Curve
compatibility layer away from the pure fiat-backed group logic.

Change-Id: I3b7a7495034d0c569b21c442ae36958763b8b2d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/320074
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-30 16:47:17 +00:00
Filippo Valsorda
d1dceafc29 crypto/elliptic: use a 4-bit sliding window for P-521 ScalarMult
name                    old time/op    new time/op    delta
pkg:crypto/elliptic goos:darwin goarch:amd64
ScalarBaseMult/P521-16    1.63ms ± 4%    1.00ms ± 1%  -38.69%  (p=0.000 n=10+8)
ScalarMult/P521-16        1.65ms ± 4%    0.99ms ± 2%  -40.15%  (p=0.000 n=10+10)
pkg:crypto/ecdsa goos:darwin goarch:amd64
Sign/P521-16              1.67ms ± 1%    1.12ms ± 2%  -32.82%  (p=0.000 n=8+10)
Verify/P521-16            3.10ms ± 2%    2.00ms ± 2%  -35.54%  (p=0.000 n=9+10)
GenerateKey/P521-16       1.53ms ± 1%    0.98ms ± 2%  -35.81%  (p=0.000 n=9+10)

Change-Id: I109e821399d71330a77d105496e227746cc3ea0d
Reviewed-on: https://go-review.googlesource.com/c/go/+/320072
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-30 16:46:47 +00:00
Filippo Valsorda
e39b854a67 crypto/elliptic: use complete addition formulas for P-521
Complete formulas don't have exceptions for P = Q or P = 0, which makes
them significantly simpler and safer to implement. Notice how the
constant time IsZero checks are gone.

It's not free, but still well within the performance gains of CL 315271.

name                    old time/op    new time/op    delta
pkg:crypto/elliptic goos:darwin goarch:amd64
ScalarBaseMult/P521-16    1.34ms ± 3%    1.63ms ± 4%  +21.78%  (p=0.000 n=10+10)
ScalarMult/P521-16        1.35ms ± 3%    1.65ms ± 4%  +22.58%  (p=0.000 n=10+10)
pkg:crypto/ecdsa goos:darwin goarch:amd64
Sign/P521-16              1.45ms ± 2%    1.67ms ± 1%  +15.00%  (p=0.000 n=10+8)
Verify/P521-16            2.68ms ± 1%    3.10ms ± 2%  +16.02%  (p=0.000 n=10+9)
GenerateKey/P521-16       1.31ms ± 4%    1.53ms ± 1%  +16.89%  (p=0.000 n=10+9)

Change-Id: Ibd9a961e9865df68a1250aba739c190caf9a54de
Reviewed-on: https://go-review.googlesource.com/c/go/+/320071
Trust: Filippo Valsorda <filippo@golang.org>
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-30 16:45:25 +00:00
Dan Scales
5d6d9f5610 cmd/compile: use Structure() to get single underlying type of typeparam.
Use types2.Structure() to get single underlying type of typeparams, to
handle some unusual cases where a type param is constrained to a single
underlying struct or map type.

Fixes #48538

Change-Id: I289fb7b31d489f7586f2b04aeb1df74e15a9f965
Reviewed-on: https://go-review.googlesource.com/c/go/+/359335
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 23:25:18 +00:00
Dan Scales
8dfb447231 runtime: do not add open defer entry above a started open defer entry
Fix two defer bugs related to adding/removing open defer entries.
The bugs relate to the way that we add and remove open defer entries
from the defer chain. At the point of a panic, when we want to start
processing defer entries in order during the panic process, we need to
add entries to the defer chain for stack frames with open defers, since
the normal fast-defer code does not add these entries. We do this by
calling addOneOpenDeferFrame() at the beginning of each time around the
defer loop in gopanic(). Those defer entries get sorted with other open
and non-open-coded defer frames.

However, the tricky part is that we also need to remove defer entries if
they end not being needed because of a recover (which means we are back
to executing the defer code inline at function exits). But we need
to deal with multiple panics and in-process defers on the stack, so we
can't just remove all open-coded defers from the the defer chain during
a recover.

The fix (and new invariant) is that we should not add any open-coded
defers to the defer chain that are higher up the stack than an open-coded
defer that is in progress. We know that open-coded defer will still be
run until completed, and when it is completed, then a more outer frame
will be added (if there is one). This fits with existing code in gopanic
that only removes open-coded defer entries up to any defer in progress.

These bugs were because of the previous inconsistency between adding and
removing open defer entries, which meant that stale defer entries could
be left on the list, in these unusual cases with both recursive
panics plus multiple independent (non-nested) cases of panic & recover.

The test for #48898 was difficult to add to defer_test.go (while keeping
the failure mode), so I added as a go/test/fixedbug test instead.

Fixes #43920
Updates #43941
Fixes #48898

Change-Id: I593b77033e08c33094315abf8089fbc4cab07376
Reviewed-on: https://go-review.googlesource.com/c/go/+/356011
Trust: Dan Scales <danscales@google.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
2021-10-29 23:15:00 +00:00
Robert Findley
c812b97ec6 go/types: comp literals to accept type sets with single underlying types
This is a clean port of CL 357915 to go/types.

Change-Id: Idf5850a8bdcf3596c561c97bcd60539945139743
Reviewed-on: https://go-review.googlesource.com/c/go/+/359877
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:29:31 +00:00
Robert Findley
71e6ab8f95 go/types: slice exprs to accept type sets with single underlying types
This is a port of CL 357779 to go/types. A test error message was
repositioned on the sliced operand.

Change-Id: Ie775c128f70d9cb08a2eba54b8bc082134ec3200
Reviewed-on: https://go-review.googlesource.com/c/go/+/359876
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:27:54 +00:00
Robert Findley
02bd226b8a go/types: range clause to accept type sets with single underlying types
This is a port of CL 357778 to go/types, adjusted to include error codes
and to use the different range statement syntax in go/ast.

Change-Id: Id537c195cd33a8b422a366269ca8730c2a5bccf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/359875
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:27:40 +00:00
Robert Findley
3571ab58b8 go/types: make built-in to accept type sets with single underlying types
This is a straightforward port of CL 357776 to go/types.

Change-Id: I64220840a01f57cd7955f7d956b9aa8227473b46
Reviewed-on: https://go-review.googlesource.com/c/go/+/359874
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2021-10-29 22:27:26 +00:00
Filippo Valsorda
c76893d0da crypto/elliptic: refactor P-224 field implementation
Improved readability, replaced constant time bit masked operations with
named functions, added comments. The behavior of every function should
be unchanged.

The largest change is the logic that in p224Contract checks if the value
is greater than or equal to p. Instead of a lot of error-prone masking,
we run a throwaway subtraction chain and look at the final borrow bit.

We could also not throw away the subtraction chain output and do a
constant time select instead of another masked subtraction, but we'd
still have to fix any underflows (because these are unsaturated limbs
and they underflow at 2^32 instead of 2^28). That's similar but
different from the carry-down chain we do elsewhere in that function
(which does undeflow fixing and borrow at the same time). I thought
having both variations in the same function would be confusing. Here's
how it would look like.

	var b uint32
	var outMinusP p224FieldElement
	for i := 0; i < len(out); i++ {
		outMinusP[i], b = bits.Sub32(out[i], p224P[i], b)
	}
	for i := 0; i < 3; i++ {
		mask := maskIfNegative(outMinusP[i])
		outMinusP[i] += (1 << 28) & mask
		// Note we DON'T borrow here, because it happened above.
	}
	for i := 0; i < len(out); i++ {
		out[i] = select32(b, out[i], outMinusP[i])
	}

Change-Id: I00932e8f171eff7f441b45666dccfd219ecbbc50
Reviewed-on: https://go-review.googlesource.com/c/go/+/326311
Trust: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Julie Qiu <julie@golang.org>
2021-10-29 22:01:56 +00:00
Michael Knyszek
994049a9ad runtime: add testing framework and basic tests for GC pacer
This change creates a formal exported interface for the GC pacer and
creates tests for it that simulate some series of GC cycles. The tests
are completely driven by the real pacer implementation, except for
assists, which are idealized (though revise is called repeatedly).

For #44167.

Change-Id: I0112242b07e7702595ca71001d781ad6c1fddd2d
Reviewed-on: https://go-review.googlesource.com/c/go/+/353354
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 19:51:20 +00:00
Cherry Mui
5ec139fa78 runtime: remove unnecessary argument stores for panicIndex etc. on ARM64
If register ABI is used, no need to store the arguments to stack.
I forgot them in CL 323937.

Change-Id: I888af2b547a8fc97d13716bc8e8f3acd5c5bc127
Reviewed-on: https://go-review.googlesource.com/c/go/+/351609
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2021-10-29 19:35:47 +00:00
Bryan C. Mills
459293388c cmd/go: document that tests must not write to their source modules
Fixes #28386
Updates #48495

Change-Id: I76186077c7bbe3f8f608026ee1865de83fe169b4
Reviewed-on: https://go-review.googlesource.com/c/go/+/359574
Trust: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 19:30:03 +00:00
Bryan C. Mills
6afdf01e53 cmd/go: make assertions more precise in TestScript/gcflags_patterns
This fixes a rare test flake observed in
https://build.golang.org/log/18e531785e7f8b3e6a6d5231787a9d8bdd4a60ea.

Change-Id: I0986752f5c1abc626070b9f8ecf34c629a15cd2a
Reviewed-on: https://go-review.googlesource.com/c/go/+/359735
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 18:56:29 +00:00
Michael Knyszek
5ec2427357 runtime: pass nanotime and gomaxprocs into startCycle and endCycle explicitly
This is to facilitate testing of the pacer, since otherwise this is
accessing global state, which is impossible to stub out properly.

For #44167.

Change-Id: I52c3b51fc0ffff38e3bbe534bd66e5761c0003a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/353353
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:59 +00:00
Michael Anthony Knyszek
9da64156a6 runtime: track scannable globals space
For #44167.

Change-Id: I2cd13229d88f630451fabd113b0e5a04841e9e79
Reviewed-on: https://go-review.googlesource.com/c/go/+/309590
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:29 +00:00
Michael Anthony Knyszek
9ac1ee2d46 runtime: track the amount of scannable allocated stack for the GC pacer
This change adds two fields to gcControllerState: stackScan, used for
pacing decisions, and scannableStackSize, which directly tracks the
amount of space allocated for inuse stacks that will be scanned.

scannableStackSize is not updated directly, but is instead flushed from
each P when at an least 8 KiB delta has accumulated. This helps reduce
issues with atomics contention for newly created goroutines. Stack
growth paths are largely unaffected.

StackGrowth-48			51.4ns ± 0%	51.4ns ± 0%	~	(p=0.927 n=10+10)
StackGrowthDeep-48		6.14µs ± 3%	6.25µs ± 4%	~	(p=0.090 n=10+9)
CreateGoroutines-48		273ns ± 1%	273ns ± 1%	~	(p=0.676 n=9+10)
CreateGoroutinesParallel-48	65.5ns ± 5%	66.6ns ± 7%	~	(p=0.340 n=9+9)
CreateGoroutinesCapture-48	2.06µs ± 1%	2.07µs ± 4%	~	(p=0.217 n=10+10)
CreateGoroutinesSingle-48	550ns ± 3%	563ns ± 4%	+2.41%	(p=0.034 n=8+10)

For #44167.

Change-Id: Id1800d41d3a6c211b43aeb5681c57c0dc8880daf
Reviewed-on: https://go-review.googlesource.com/c/go/+/309589
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:20 +00:00
Michael Anthony Knyszek
8e112a7c2a runtime: move heapLive and heapScan updates into a method
This change moves heapLive and heapScan updates on gcController into a
method for better testability. It's also less error-prone because code
that updates these fields needs to remember to emit traces and/or call
gcController.revise; this method now handles those cases.

For #44167.

Change-Id: I3d6f2e7abb22def27c93feacff50162b0b074da2
Reviewed-on: https://go-review.googlesource.com/c/go/+/309275
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:35:01 +00:00
Michael Anthony Knyszek
4a5d78ff8b runtime: move pacer time updates and state resets into methods
Currently GC pacer updates are applied somewhat haphazardly via direct
field access. To facilitate ease of testing, move these field updates
into methods. Further CLs will move more of these updates into methods.

For #44167.

Change-Id: I25b10d2219ae27b356b5f236d44827546c86578d
Reviewed-on: https://go-review.googlesource.com/c/go/+/309274
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:34:43 +00:00
Michael Anthony Knyszek
903f313d8b runtime: fix unclosed GCSweepStart trace event
CL 333389 erroneously moved traceGCSweepDone inside the sl.valid block
that it introduced in mcentral.cacheSpan, when it should have left it
outside that scope, because the trace event is created unconditionally
at the top of the method.

Fixes #49231.

Change-Id: If719c05063d4f4d330a320da6dc00d1e9d8102e4
Reviewed-on: https://go-review.googlesource.com/c/go/+/359775
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 18:34:24 +00:00
Michael Anthony Knyszek
413672fc84 runtime: detangle sweeper pacing from GC pacing
The sweeper's pacing state is global, so detangle it from the GC pacer's
state updates so that the GC pacer can be tested.

For #44167.

Change-Id: Ibcea989cd435b73c5891f777d9f95f9604e03bd1
Reviewed-on: https://go-review.googlesource.com/c/go/+/309273
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
2021-10-29 18:23:03 +00:00
Zhi Zheng
353d5b6c53 runtime: enable crash dump creation on Windows
This change provides ability to create dumps on Windows that can be used by
"dlv core" command. Currently only full dumps can be correctly read by Delve.
Below are the steps to create and use the dumps.

1. Configure Windows OS to collect dumps before running the program.
Instructions on how to do the configuration are here:
https://docs.microsoft.com/en-us/windows/win32/wer/collecting-user-mode-dumps.
In order for Delve to read the dump, set the DumpType to full dump, i.e. DumpType=2.

2. Go program only generates dumps when the environment variable GOTRACEBACK
is set to crash. Run command "set GOTRACEBACK=crash" before running the program.

3. Dump files will be generated in %LOCALAPPDATA%\CrashDumps

4. Use Delve command "dlv core" to open the dump, e.g.: "dlv core a.exe a.exe.3840.dmp".

Fixes #20498

Change-Id: Ib9aa82e7aea9da19594dc49348876997b24e9600
Reviewed-on: https://go-review.googlesource.com/c/go/+/307372
Run-TryBot: Alessandro Arzilli <alessandro.arzilli@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Patrik Nyblom <pnyb@google.com>
Trust: Alex Brainman <alex.brainman@gmail.com>
2021-10-29 17:44:52 +00:00
Michael Anthony Knyszek
d8fc7f785e runtime: clean up allocation zeroing
Currently, the runtime zeroes allocations in several ways. First, small
object spans are always zeroed if they come from mheap, and their slots
are zeroed later in mallocgc if needed. Second, large object spans
(objects that have their own spans) plumb the need for zeroing down into
mheap. Thirdly, large objects that have no pointers have their zeroing
delayed until after preemption is reenabled, but before returning in
mallocgc.

All of this has two consequences:
1. Spans for small objects that come from mheap are sometimes
   unnecessarily zeroed, even if the mallocgc call that created them
   doesn't need the object slot to be zeroed.
2. This is all messy and difficult to reason about.

This CL simplifies this code, resolving both (1) and (2). First, it
recognizes that zeroing in mheap is unnecessary for small object spans;
mallocgc and its callees in mcache and mcentral, by design, are *always*
able to deal with non-zeroed spans. They must, for they deal with
recycled spans all the time. Once this fact is made clear, the only
remaining use of zeroing in mheap is for large objects.

As a result, this CL lifts mheap zeroing for large objects into
mallocgc, to parallel all the other codepaths in mallocgc. This is makes
the large object allocation code less surprising.

Next, this CL sets the flag for the delayed zeroing explicitly in the one
case where it matters, and inverts and renames the flag from isZeroed to
delayZeroing.

Finally, it adds a check to make sure that only pointer-free allocations
take the delayed zeroing codepath, as an extra safety measure.

Benchmark results: https://perf.golang.org/search?q=upload:20211028.8

Inspired by tapir.liu@gmail.com's CL 343470.

Change-Id: I7e1296adc19ce8a02c8d93a0a5082aefb2673e8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/359477
Trust: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2021-10-29 17:44:15 +00:00
Bryan C. Mills
a88575d662 Revert "cmd/go: remove support for -buildmode=shared"
This reverts CL 359096.

Updates #47788.

Reason for revert: -buildmode=shared may have actually been working in a few very specific cases. We should not remove -buildmode=shared until we have implemented an alternative to support those few cases.

Change-Id: Ia962b06abacc11f6f29fc29d092773be175e32f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/359575
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2021-10-29 17:32:24 +00:00
Russ Cox
645d07819b sync: add Mutex.TryLock, RWMutex.TryLock, RWMutex.TryRLock
Use of these functions is almost (but not) always a bad idea.

Very rarely they are necessary, and third-party implementations
(using a mutex and an atomic word, say) cannot integrate as well
with the race detector as implmentations in package sync itself.

Fixes #45435.

Change-Id: I0128ca48ef5e0a3b09c913f0f3a7ee5c56388000
Reviewed-on: https://go-review.googlesource.com/c/go/+/319769
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 17:13:13 +00:00
Michael Anthony Knyszek
3aecb3a8f7 runtime: fix sweep termination condition
Currently, there is a chance that the sweep termination condition could
flap, causing e.g. runtime.GC to return before all sweep work has not
only been drained, but also completed. CL 307915 and CL 307916 attempted
to fix this problem, but it is still possible that mheap_.sweepDrained is
marked before any outstanding sweepers are accounted for in
mheap_.sweepers, leaving a window in which a thread could observe
isSweepDone as true before it actually was (and after some time it would
revert to false, then true again, depending on the number of outstanding
sweepers at that point).

This change fixes the sweep termination condition by merging
mheap_.sweepers and mheap_.sweepDrained into a single atomic value.

This value is updated such that a new potential sweeper will increment
the oustanding sweeper count iff there are still outstanding spans to be
swept without an outstanding sweeper to pick them up. This design
simplifies the sweep termination condition into a single atomic load and
comparison and ensures the condition never flaps.

Updates #46500.
Fixes #45315.

Change-Id: I6d69aff156b8d48428c4cc8cfdbf28be346dbf04
Reviewed-on: https://go-review.googlesource.com/c/go/+/333389
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2021-10-29 17:12:47 +00:00
Bryan C. Mills
f288526374 cmd/go: test that the reserved paths "example" and "test" are not used
Proposal #37641 reserved the paths "example" and "test" for end users,
so no path beginning with either of those elements may be added to the
standard library.

(We are unlikely to add them by accident, but this test codifies the
policy accepted in the proposal.)

Fixes #37641

Change-Id: I999064180336c923d4918a1176c371b8f5666f8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/359576
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2021-10-29 04:21:07 +00:00
Meng Zhuo
2c7cdec2a7 io: remove UTF8 specified document for RuneReader
RuneReader is fine with UTF16 or any other character encoding

Fixes #49178

Change-Id: I08a5ac205e095349d451d3b60411eaeebc3aa563
Reviewed-on: https://go-review.googlesource.com/c/go/+/359334
Trust: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Meng Zhuo <mzh@golangcn.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Meng Zhuo <mzh@golangcn.org>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-29 02:16:47 +00:00
Rob Pike
d3d8852d89 text/template: use reflect.Value.FieldByIndexErr
to avoid a crash on a nil pointer as an embedded field.

Fixes #48215

Change-Id: I214faa6e3cf08cdec1c01035e4bbca0900c6a408
Reviewed-on: https://go-review.googlesource.com/c/go/+/357963
Trust: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 02:12:05 +00:00
Rob Pike
33c392f72d reflect: add FieldByIndexErr
This new function, although different in signature from other
reflect functions, allows the caller to avoid the panic caused by
nil embedded fields in calls to FieldByIndex.

Fixes #48218

Change-Id: I447f135bb789148c27ae3f2f23dcf43094f4c1de
Reviewed-on: https://go-review.googlesource.com/c/go/+/357962
Trust: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2021-10-29 02:11:42 +00:00
Russ Cox
af05d8be3d all: manual fixups for //go:build vs // +build
Update many generators, also handle files that were not part of the
standard build during 'go fix' in CL 344955.

Fixes #41184.

Change-Id: I1edc684e8101882dcd11f75c6745c266fccfe9e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/359476
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2021-10-28 22:38:00 +00:00
Bryan C. Mills
d0751b1ad8 cmd/go: test with SIGTERM in test_fuzz_non_crash_signal
The test currently uses only SIGKILL (which cannot be trapped by the
worker process) and SIGILL (which is handled by the Go runtime, and
does pretty unambiguously indicate a crash). It is missing a third
(and perhaps more likely) case: a signal like SIGTERM that is handled
by the Go runtime and causes the test to terminate, but nonetheless
does not indicate a crash.

Change-Id: I0595a0917e977e8eb2c24ae8b706d734ce3b2f4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/359482
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-10-28 20:50:34 +00:00
Bryan C. Mills
b9540246ba cmd/go: disallow the -fuzz flag for tests outside the main module
Normally, when fuzzing identifies a failure it saves the failing input
to the package's testdata directory. However, the testdata directory
for packages outside the main module is normally not writable — and
when it is, writing to a testdata directory inside the module cache
would corrupt the checksum for that module (and permanently alter the
behavior of that version of the module globally).

In the future we could consider a flag to allow failures to be saved
to an alternate location, or perhaps in the build cache; or, we could
suppress writes entirely and rely on the user to identify and copy the
failing input from the test log. However, it's a bit late in the cycle
for that big a design decision right now. For Go 1.18, we will just
enforce that the package to be fuzzed resides in the main module,
which is typically a writable VCS checkout.

Fixes #48495

Change-Id: I8d3d56372394b1aaa94fa920399c659363fa17fa
Reviewed-on: https://go-review.googlesource.com/c/go/+/359414
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-10-28 20:43:39 +00:00
Bryan C. Mills
e741e2fe0e cmd/go: consolidate fuzz-support checks
We had been repeating conditions for specific platforms and
architectures to gate fuzzing tests, but the more of those tests we
add the more we will have to update if the set of supported platforms
and archictures expands over time.

We also ought to provide a friendlier error message when
'go test -fuzz' is used on non-supported platforms.

This change adds predicates in cmd/internal/sys, which already
contains similar predicates for related functionality (such as the
race detector), and uses those predicates in 'go test' and TestScript.

For #48495

Change-Id: If24c3997aeb4d201258e21e5b6cf4f7c08fbadd7
Reviewed-on: https://go-review.googlesource.com/c/go/+/359481
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Katie Hockman <katie@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2021-10-28 20:43:02 +00:00
Bryan C. Mills
834e36ec77 net/http: restore generated // +build comment
The upstream cmd/bundle tool does not yet omit it,
and the longtest builders test that the generated
file is not modified.

For #41184.

Change-Id: Ib68f532139ed1436cbaf3a756f300fe60f520cab
Reviewed-on: https://go-review.googlesource.com/c/go/+/359484
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-28 20:39:36 +00:00
Cherry Mui
5e217284dc cmd/internal/goobj: bump version number
There have a few updates to the Go object file in this cycle (e.g.
FuncInfo format change, some changes in constant values), and it
is not compatible with the old tools. Bump up the version number.

Change-Id: Id176979b139c76ded2c50f2678eb313934326d6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/359483
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2021-10-28 20:27:44 +00:00