For struct like { { a int64; b int16 }; c int32 }, on 64-bit
machines the offset of c is 16, as the inner struct takes 16
bytes because we round up type size to its alignment. Update the
abi package's offset calculation to include this.
We only need to do this for struct type, because for all other
types its size is naturally aligned.
TODO: add a test.
Change-Id: I0c661768cb1ed3cb409b20a88b7e23e059f8e3e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/306449
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
The covers three kinds of uses:
1. Calls of closures from assembly. These are always ABIInternal calls
without wrappers. I went through every indirect call in the runtime
and I think mcall is the only case of assembly calling a Go closure in
a way that's affected by ABIInternal. systemstack also calls a
closure, but it takes no arguments.
2. Calls of Go functions that expect raw ABIInternal pointers. I also
only found one of these: callbackasm1 -> cgocallback on Windows. These
are trickier to find, though.
3. Finally, I found one case on NetBSD where new OS threads were
directly calling the Go runtime entry-point from assembly via a PC,
rather than going through a wrapper. This meant new threads may not
have special registers set up. In this case, a change on all other
OSes had already forced new thread entry to go through an ABI wrapper,
so I just caught NetBSD up with that change.
With this change, I'm able to run a "hello world" with
GOEXPERIMENT=regabi,regabiargs.
For #40724.
Change-Id: I2a6d0e530c4fd4edf13484d923891c6160d683aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/305669
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
The logic for constructing calls in (*state).call is based around
targeted experiments with register-based calls. However, when the
register ABI is turned on everywhere, it currently doesn't account for
direct calls to non-ABIInternal functions. This CL adds a much simpler
path to (*state).call when regabiargs is turned on that looks at the
ABI of the target function.
For #40724.
Change-Id: I7f4f5fed8a5ec131bcf1ce5b9d94d45672a304cb
Reviewed-on: https://go-review.googlesource.com/c/go/+/306410
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When regabiargs is enabled, a function's incoming ABI should be
determined solely by the function's own definition ABI (which is
usually ABIInternal, but can be ABI0 for ABI wrappers).
For example, the current code miscompiles ABI0 -> ABIInternal wrappers
when the experiment is enabled because it treats the wrapper itself as
being called as ABIInternal. This causes it to assume the incoming
arguments are already in registers, so usually the wrapper doesn't do
anything with the arguments because it thinks they're already in the
right place. With this fix, these wrappers now correctly load the
arguments from the stack and put them in registers.
For #40724.
Change-Id: Iec784e88ebc55d9e95e830ed7533aa336f3b1ca2
Reviewed-on: https://go-review.googlesource.com/c/go/+/306409
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Currently the prologue generated for WRAPPER assembly functions uses BX
and DI, but these are argument registers in the register-based calling
convention. Thus, these end up being clobbered when we want to have an
ABIInternal assembly function.
Define REGENTRYTMP0 and REGENTRYTMP1, aliases for the dedicated function
entry scratch registers R12 and R13, and use those instead.
For #40724.
Change-Id: Ica78c4ccc67a757359900a66b56ef28c83d88b3e
Reviewed-on: https://go-review.googlesource.com/c/go/+/303314
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
For in-register arguments, it must have only a single copy of it
present in the function. If there are multiple copies, it confuses
the register allocator, as they are in the same register.
Change-Id: I55cb06746f08aa7c9168026d0f411bce0a9f93f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/306330
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Coupling object resolution to parsing complicates the parsing code, and
is a barrier to improvement. It requires passing around context such as
'lhs' or 'keyOk', and even then sometimes requires guess-work, such as
whether to resolve the key in a composite literal.
In this CL we delay object resolution to a separate pass after the file
parse completes. This makes it easier to see logic of scoping, and
removes state from the parsing code. This can enable subsequent
improvements such as optionally skipping object resolution, aligning the
parser with cmd/compile/internal/syntax, and allowing alternative
parsers to reuse object resolution.
The additional AST traversal appears to slow down parsing by around 4%.
That seems small enough not to worry about, especially since performance
sensitive users may eventually be able to disable object resolution
entirely, saving around 18% off the previous baseline. I'll also mail a
speculative CL showing how we can significantly mitigate the cost of
object resolution by transposing scopes.
For #45104
Change-Id: I98d9143fd77ae29e84ec7c3ae2fdb1139510da37
Reviewed-on: https://go-review.googlesource.com/c/go/+/304455
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This change modifies the system that allows Go functions to be set as
callbacks in various Windows systems to support the new register ABI.
For #40724.
Change-Id: Ie067f9e8a76c96d56177d7aa88f89cbe7223e12e
Reviewed-on: https://go-review.googlesource.com/c/go/+/300113
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
OpArgXXXReg values must be scheduled at the very top, as their
registers need to be live at the beginning before any other use
of the register.
Change-Id: Ic76768bb74da402adbe61db3b2d174ecd3f9fffc
Reviewed-on: https://go-review.googlesource.com/c/go/+/306329
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The existing implementation implicitly reads from the filesystem
instead of using the overlay file data (due to src == nil), so
pass in the overlaid source if we have an overlay for this file.
Fixes#44946
Change-Id: I61ce09d10c5edac1b47332583efdcd3c1241f58a
Reviewed-on: https://go-review.googlesource.com/c/go/+/305071
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
This is accomplished by checking for simple stores of pointer types
and leaving them alone. The failure case was when a *mspan
(not in heap) stored type was replaced by unsafe.Pointer.
Updates #40724.
Change-Id: I529e1705bf58fb0e64e60d48fd550b3a407e57e7
Reviewed-on: https://go-review.googlesource.com/c/go/+/305672
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
With GOEXPERIMENT=regabidefer, all deferred functions take no
arguments and have no results (their signature is always func()).
Since the signature is fixed, we can replace all of the reflectcalls
in the defer code with direct closure calls.
For #40724.
Change-Id: I3acd6742fe665610608a004c675f473b9d0e65ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/306010
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
The changes between (equivalent, and reviewed) go/types/operand.go
and operand.go can be seen by comparing patchset 1 and 2. The actual
changes are removing the "// UNREVIEWED" marker.
The primary differences compared to go/types are:
- use of syntax rather than go/ast package
- explicit mode for untyped nil (rather than relying on the type)
Change-Id: I0c9c1c6153c55cb0550096bd966c9f0f1c766734
Reviewed-on: https://go-review.googlesource.com/c/go/+/305571
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
The changes between (equivalent, and reviewed) go/types/decl.go
and decl.go can be seen by comparing patchset 1 and 2. The actual
changes are removing the "// UNREVIEWED" marker and a minor comment
update.
The primary differences to go/types/decl.go are:
- use of syntax rather than go/ast package
- use of error_ objects to collect follow-on error info
- use of check.conf.Trace rather than global trace flag
- more aggressively marking variables as used in the presence errors
- not using a walkDecl abstraction for const/var/type declarations
Change-Id: I5cf26779c9939b686a3dbaa4d38fdd0c154a92ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/305570
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Findley <rfindley@google.com>
Change-Id: I78c5744bb01988f1f599569703d83fd21542ac7a
Reviewed-on: https://go-review.googlesource.com/c/go/+/305911
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Change-Id: Iff5074713e1a4484c04b8628fb2611b6d4e154c7
GitHub-Last-Rev: bb0861bbbe
GitHub-Pull-Request: golang/go#45294
Reviewed-on: https://go-review.googlesource.com/c/go/+/305772
Reviewed-by: Ben Shi <powerman1st@163.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ben Shi <powerman1st@163.com>
TryBot-Result: Go Bot <gobot@golang.org>
Change-Id: I75d089a7c1c3cdd50e5d2dafdb3386620efff4c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/304454
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
For #45104
For #45221
Change-Id: I8966555f4e8844d5b6766d00d48f7a81868ccf40
Reviewed-on: https://go-review.googlesource.com/c/go/+/304453
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
The setting of n.Use for a call node in transformCall() (and previously
in Call()), was not corrrect, since it was trying to use the number of
results of the call, rather than whether the call result was actually
used. We are already setting n.Use to ir.CallUseStmt if the call node is
directly a statement, so we just need to initialize n.Use to
ir.CallExprStmt in Call(), which will get changed to ir.CallUseStmt at
the statement level if it's used as a statement.
Enable inlining of stenciled functions (just disabled for testing,
easier debugging). The above n.Use fix was required for the inlining
to work for two cases.
Change-Id: Ie4ef6cd53fd4b20a4f3be31e629280909a545b7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/305913
Trust: Dan Scales <danscales@google.com>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
During substitution of the function type during stenciling, we must set
the Name nodes of the param/result fields of the func type. We get those
name nodes from the substituted Dcl nodes of the PPARAMS and PPARAMOUTs.
But we must check that the names match with the Dcl nodes, so that we
skip any param fields that correspond to unnamed (in) parameters.
Added a few tests to typelist.go by removing a variety of unneeded
function parameter names.
Change-Id: If786961b64549da6f18eeeb5060ea58fab874eb9
Reviewed-on: https://go-review.googlesource.com/c/go/+/305912
Trust: Dan Scales <danscales@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Change-Id: I96170f1ca481b5fe21c85f99e877fa090e4dccc2
Reviewed-on: https://go-review.googlesource.com/c/go/+/304452
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
parsePrimaryExpr has to be careful to resolve identifiers used in
composite expressions when parsing in LHS mode. It missed the literal
type name.
Fixes#45136
Change-Id: I3e12f91e3ef5fdb43faa436cdf1240eb3293fe1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/304451
Trust: Robert Findley <rfindley@google.com>
Run-TryBot: Robert Findley <rfindley@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Add new tests for object resolution driven by source files with
declarations and uses marked via special comments. This made it easier
to add test coverage while refactoring object resolution for #45104.
Tests are added to a new resolver_test.go file. In a subsequent CL the
resolver.go file will be added, making this choice of file name more
sensible.
For #45104
For #45136
For #45160
Change-Id: I240fccc0de95aa8f2d03e39c77146d4c61f1ef9e
Reviewed-on: https://go-review.googlesource.com/c/go/+/304450
Trust: Robert Findley <rfindley@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Findley <rfindley@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
In expand_calls, when rewriting OpArg to OpArgIntReg/OpArgFloatReg,
avoid generating duplicates. Otherwise it will confuse the
register allocator: it would think the second occurance clobbers
the first's register, causing it to generate copies, which may
clobber other args.
Change-Id: I4f1dc0519afb77500eae1c0e6ac8745e51f7aa4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/306029
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
exitsyscall0 contains two G variables: _g_ and gp. _g_ is the active G,
g0, while gp is the G to run (which just exited from a syscall).
It is passing _g_ to schedEnabled, which is incorrect; we are about to
execute gp, so that is what we should be checking the schedulability of.
While this is incorrect and should be fixed, I don't think it has ever
caused a problem in practice:
* g0 does not have g.startpc set, so schedEnabled simplifies to
just !sched.disable.user.
* This is correct provided gp is never a system goroutine.
* As far as I know, system goroutines never use entersyscall /
exitsyscall.
As far I can tell, this was a simple copy/paste error from exitsyscall,
where variable _g_ is the G to run.
While we are here, eliminate _g_ entirely, as the one other use is
identical to using gp.
Change-Id: I5df98a34569238b89ab13ff7012cd756fefb10dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/291329
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Handle the case where types can be partially inferred for an
instantiated function that is not immediately called. The key for the
Inferred map is the CallExpr (if inferring types required the function
arguments) or the IndexExpr (if types could be inferred without the
function arguments).
Added new tests for the case where the function isn't immediately called
to typelist.go.
Change-Id: I60f503ad67cd192da2f2002060229efd4930dc39
Reviewed-on: https://go-review.googlesource.com/c/go/+/305909
Trust: Dan Scales <danscales@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Dan Scales <danscales@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Change the amd64 version of 'zerorange' to avoid using RAX/RDI, since
it can be called in a context when one of these registers is live
(contains an incoming parameter).
Updates #40724.
Change-Id: Ibfa2b4e156b876354d4f8bd04eb8773c7056d948
Reviewed-on: https://go-review.googlesource.com/c/go/+/305829
Trust: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
This change guards a call to ast.Inspect with a nil check on the first
argument. This avoids a panic when inspecting a reference to a function
with a nil body. This can only happen when a function body is defined outside Go.
Fixes#42706
Change-Id: I91bc607b24b6224920c24cfd07e76ce7737a98d4
GitHub-Last-Rev: 08072b9ce5
GitHub-Pull-Request: golang/go#43011
Reviewed-on: https://go-review.googlesource.com/c/go/+/275516
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Daniel Martí <mvdan@mvdan.cc>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Go Bot <gobot@golang.org>
The correct command to create new go.mod file should be 'go mod init',
not 'go help init'.
Change-Id: I1150621987d989997f8b75e6a13fe96423a11cf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/305289
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Trust: Jay Conrod <jayconrod@google.com>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
old code was always ABI0, new code tracks the default
this may cause some write barrier removals to fail to fire
Updates #40724.
Change-Id: I656bdd5511c5bd6ee6e021999e30d842a6b9f0a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/305671
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When using cgo, some of the frames can be provided by cgoTraceback, a
cgo-provided function to generate C tracebacks. Unlike Go tracebacks,
cgoTraceback has no particular guarantees that it produces valid
tracebacks.
If one of the (invalid) frames happens to put the PC in the alignment
region at the end of a function (filled with int 3's on amd64), then
Frames.Next will find a valid funcInfo for the PC, but pcdatavalue will
panic because PCDATA doesn't cover this PC.
Tolerate this case by doing a non-strict PCDATA lookup. We'll still show
a bogus frame, but at least avoid throwing.
Fixes#44971
Change-Id: I9eed728470d6f264179a7615bd19845c941db78c
Reviewed-on: https://go-review.googlesource.com/c/go/+/301369
Trust: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Pratt <mpratt@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The correct setting of t.nod is needed when exporting types. Make sure
we create instantiated named types correctly so t.nod is set.
New test file interfacearg.go that tests this (by instantiating a type
with an interface). Also has tests for various kinds of method
expressions.
Change-Id: Ia7fd9debd495336b73788af9e35d72331bb7d2b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/305730
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dan Scales <danscales@google.com>
Trust: Robert Griesemer <gri@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
After recent discussion about bodyless functions, their wrappers,
their stack maps, nosplit, and callbacks, I was inspired to go and
be sure that more defaults were sensible. This may not be all --
currently rtcall is "ABIDefault" which I think is correct, but I
am not 100% certain.
Updates #40724.
Change-Id: I95b14ee0e5952fa53e7fea9f6f5192358aa24f23
Reviewed-on: https://go-review.googlesource.com/c/go/+/304549
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Fix various small bugs related to delaying transformations due to type
params. Most of these relate to the need to delay a transformation when
an argument of an expression or statement has a type parameter that has
a structural constraint. The structural constraint implies the operation
should work, but the transformation can't happen until the actual value
of the type parameter is known.
- delay transformations for send statements and return statements if
any args/values have type params.
- similarly, delay transformation of a call where the function arg has
type parameters. This is mainly important for the case where the
function arg is a pure type parameter, but has a structural
constraint that requires it to be a function. Move the setting of
n.Use to transformCall(), since we may not know how many return
values there are until then, if the function arg is a type parameter.
- set the type of unary expressions from the type2 type (as we do with
most other expressions), since that works better with expressions
with type params.
- deal with these delayed transformations in subster.node() and convert
the CALL checks to a switch statement.
- make sure ir.CurFunc is set properly during stenciling, including
closures (needed for transforming return statements during
stenciling).
New test file typelist.go with tests for these cases.
Change-Id: I1b82f949d8cec47d906429209e846f4ebc8ec85e
Reviewed-on: https://go-review.googlesource.com/c/go/+/305729
Trust: Dan Scales <danscales@google.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
CL 298669 implemented wrapping for defer/go calls so the function
being called with defer or go statement has no arguments. This
simplifies the compiler and the runtime, especially with the
new ABI.
Currently, it does not wrap functions that has no arguments but
only results. For defer/go calls, the results are not used. But
the runtime needs to allocate stack space for the callee to store
the results. Wrapping functions with results makes the runtime
simpler.
TODO: maybe not wrap if all results are in registers.
Updates #40724.
Change-Id: I74d2f4db1cbf9979afbcd846facb30d11d72ab23
Reviewed-on: https://go-review.googlesource.com/c/go/+/305550
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
CL 298669 implemented wrapping for defer/go calls so the function
being called with defer or go statement has no arguments. This
simplifies the compiler and the runtime, especially with the
new ABI.
If the called function does not have any argument, we don't need
to wrap. But the code missed the cases of method receiver, as
well as some apparent argumentless builtin calls which may later
be rewritten to having arguments (e.g. recover). This CL makes
sure to wrap those cases. Also add a check to ensure that go and
defer calls are indeed argumentless.
Handle "defer recover()" specially, as recover() is lowered to
runtime.gorecover(FP) where FP is the frame's FP. FP needs to be
evaluated before wrapping.
Updates #40724.
Change-Id: I2758b6c69ab6aa02dd588441a457fe28ddd0d5a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/304771
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Currently, for "defer i.M()" if i is nil it panics at the point of
defer statement, not when deferred function is called. We need to
do the nil check before wrapping it.
Updates #40724.
Change-Id: I62c669264668991f71999e2cf4610a9066247f9d
Reviewed-on: https://go-review.googlesource.com/c/go/+/305549
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
This CL adds a set of helper functions for testing GC interactions.
These are intended for use in the regabi signature fuzzer, but are
generally useful for GC tests, so we make them generally available to
runtime tests.
These provide:
1. An easy way to force stack movement, for testing stack copying.
2. A simple and robust way to check the reachability of a set of
pointers.
3. A way to check what general category of memory a pointer points to,
mostly so tests can make sure they're testing what they mean to.
For #40724, but generally useful.
Change-Id: I15d33ccb3f5a792c0472a19c2cc9a8b4a9356a66
Reviewed-on: https://go-review.googlesource.com/c/go/+/305330
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
The specials processing loop in mspan.sweep is about to get more
complicated and I'm too allergic to list manipulation to open code
more of it there.
Change-Id: I767a0889739da85fb2878fc06a5c55b73bf2ba7d
Reviewed-on: https://go-review.googlesource.com/c/go/+/305551
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Pratt <mpratt@google.com>
This was missed in https://golang.org/cl/303329 . It is another
impossible usage of MOVBU as a load like "MOVBU 0(rX), rY, rZ" or
"MOVBU rX(rB), rY, rZ".
Change-Id: Ib3dd984b6424907498ed65b798649f0b990d50a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/304471
Reviewed-by: Lynn Boger <laboger@linux.vnet.ibm.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This CL restructures how we track function ABIs and generate ABI
wrappers in the compiler and adds import/export of ABIs across package
boundaries.
Currently, we start by tracking definition and referencing ABIs in two
global maps and eventually move some of this information into the
LSyms for functions. This complicates a lot of the existing code for
handling wrappers and makes it particularly hard to export ABI
information across packages. This change is built around instead
recording this information on the ir.Func.
First, this change replaces the global ABI def/ref maps with a type,
which makes the data flow and lifetime of this information clear in
gc.Main. These are populated during flag parsing.
Then, early in the front-end, we loop over all ir.Funcs to 1. attach
ABI def/ref information to the ir.Funcs and 2. create new ir.Funcs for
ABI wrappers. Step 1 is slightly subtle because the information is
keyed by linker symbol names, so we can't simply look things up in the
compiler's regular symbol table.
By generating ABI wrappers early in the front-end, we decouple this
step from LSym creation, which makes LSym creation much simpler (like
it was before ABI wrappers). In particular, LSyms for wrappers are now
created at the same time as all other functions instead of by
makeABIWrapper, which means we're back to the simpler, old situation
where InitLSym was the only thing responsible for constructing
function LSyms. Hence, we can restore the check that InitLSym is
called exactly once per function.
Attaching the ABI information to the ir.Func has several follow-on
benefits:
1. It's now easy to include in the export info. This enables direct
cross-package cross-ABI calls, which are important for the performance
of calling various hot assembly functions (e.g., internal/bytealg.*).
This was really the point of this whole change.
2. Since all Funcs, including wrappers, now record their definition
ABI, callTargetLSym no longer needs to distinguish wrappers from
non-wrappers, so it's now nearly trivial (it would be completely
trivial except that it has to work around a handful of cases where
ir.Name.Func is nil).
The simplification of callTargetLSym has one desirable but potentially
surprising side-effect: the compiler will now generate direct calls to
the definition ABI even when ABI wrappers are turned off. This is
almost completely unnoticeable except that cmd/internal/obj/wasm looks
for the call from runtime.deferreturn (defined in Go) to
runtime.jmpdefer (defined in assembly) to compile is specially. That
now looks like a direct call to ABI0 rather than going through the
ABIInternal alias.
While we're in here, we also set up the structures to support more
than just ABI0 and ABIInternal and add various additional consistency
checks all around.
Performance-wise, this reduces the overhead induced by wrappers from
1.24% geomean (on Sweet) to 0.52% geomean, and reduces the number of
benchmarks impacts >2% from 5 to 3. It has no impact on compiler speed.
Impact of wrappers before this change:
name old time/op new time/op delta
BiogoIgor 15.8s ± 2% 15.8s ± 1% ~ (p=0.863 n=25+25)
BiogoKrishna 18.3s ± 6% 18.1s ± 7% -1.39% (p=0.015 n=25+25)
BleveIndexBatch100 5.88s ± 3% 6.04s ± 6% +2.72% (p=0.000 n=25+25)
BleveQuery 6.42s ± 1% 6.76s ± 1% +5.31% (p=0.000 n=24+24)
CompileTemplate 245ms ± 3% 250ms ± 6% ~ (p=0.068 n=22+25)
CompileUnicode 93.6ms ± 2% 93.9ms ± 5% ~ (p=0.958 n=22+25)
CompileGoTypes 1.60s ± 2% 1.59s ± 2% ~ (p=0.115 n=24+24)
CompileCompiler 104ms ± 4% 104ms ± 3% ~ (p=0.453 n=22+25)
CompileSSA 11.0s ± 2% 11.0s ± 1% ~ (p=0.789 n=24+25)
CompileFlate 153ms ± 2% 153ms ± 1% ~ (p=0.055 n=21+20)
CompileGoParser 229ms ± 2% 230ms ± 2% ~ (p=0.305 n=21+22)
CompileReflect 585ms ± 5% 582ms ± 3% ~ (p=0.365 n=25+25)
CompileTar 211ms ± 1% 211ms ± 3% ~ (p=0.592 n=20+22)
CompileXML 282ms ± 3% 281ms ± 2% ~ (p=0.937 n=22+23)
CompileStdCmd 13.7s ± 3% 13.6s ± 2% ~ (p=0.700 n=25+25)
FoglemanFauxGLRenderRotateBoat 8.67s ± 1% 8.78s ± 1% +1.30% (p=0.000 n=25+25)
FoglemanPathTraceRenderGopherIter1 20.5s ± 2% 20.9s ± 2% +1.85% (p=0.000 n=25+25)
GopherLuaKNucleotide 30.1s ± 2% 31.1s ± 2% +3.38% (p=0.000 n=25+25)
MarkdownRenderXHTML 246ms ± 5% 250ms ± 1% +1.42% (p=0.002 n=25+23)
Tile38WithinCircle100kmRequest 828µs ± 6% 885µs ± 6% +6.85% (p=0.000 n=23+25)
Tile38IntersectsCircle100kmRequest 1.04ms ± 5% 1.10ms ± 7% +5.63% (p=0.000 n=25+25)
Tile38KNearestLimit100Request 974µs ± 4% 972µs ± 4% ~ (p=0.356 n=25+24)
[Geo mean] 588ms 595ms +1.24%
(https://perf.golang.org/search?q=upload:20210328.5)
And after this change:
name old time/op new time/op delta
BiogoIgor 15.9s ± 1% 15.8s ± 1% -0.48% (p=0.008 n=22+25)
BiogoKrishna 18.4s ± 6% 17.8s ± 6% -3.55% (p=0.008 n=25+25)
BleveIndexBatch100 5.86s ± 3% 5.97s ± 4% +1.88% (p=0.001 n=25+25)
BleveQuery 6.42s ± 1% 6.75s ± 1% +5.14% (p=0.000 n=25+25)
CompileTemplate 246ms ± 5% 245ms ± 2% ~ (p=0.472 n=23+23)
CompileUnicode 93.7ms ± 3% 93.5ms ± 2% ~ (p=0.813 n=22+23)
CompileGoTypes 1.60s ± 2% 1.60s ± 2% ~ (p=0.108 n=25+23)
CompileCompiler 104ms ± 3% 104ms ± 2% ~ (p=0.845 n=23+23)
CompileSSA 11.0s ± 2% 11.0s ± 2% ~ (p=0.525 n=25+25)
CompileFlate 152ms ± 1% 153ms ± 2% ~ (p=0.408 n=22+22)
CompileGoParser 230ms ± 1% 230ms ± 1% ~ (p=0.363 n=21+23)
CompileReflect 582ms ± 3% 584ms ± 4% ~ (p=0.658 n=25+25)
CompileTar 212ms ± 2% 211ms ± 2% ~ (p=0.315 n=23+24)
CompileXML 282ms ± 1% 282ms ± 1% ~ (p=0.991 n=23+22)
CompileStdCmd 13.6s ± 2% 13.6s ± 2% ~ (p=0.699 n=25+24)
FoglemanFauxGLRenderRotateBoat 8.66s ± 1% 8.69s ± 1% +0.28% (p=0.002 n=25+24)
FoglemanPathTraceRenderGopherIter1 20.5s ± 3% 20.5s ± 2% ~ (p=0.407 n=25+25)
GopherLuaKNucleotide 30.1s ± 2% 31.2s ± 2% +3.82% (p=0.000 n=25+25)
MarkdownRenderXHTML 246ms ± 3% 245ms ± 1% ~ (p=0.478 n=23+22)
Tile38WithinCircle100kmRequest 820µs ± 4% 856µs ± 5% +4.39% (p=0.000 n=24+25)
Tile38IntersectsCircle100kmRequest 1.05ms ± 6% 1.07ms ± 6% +1.91% (p=0.014 n=25+25)
Tile38KNearestLimit100Request 970µs ± 4% 970µs ± 3% ~ (p=0.819 n=22+24)
[Geo mean] 588ms 591ms +0.52%
(https://perf.golang.org/search?q=upload:20210328.6)
For #40724.
Change-Id: I1c374e32d4bbc88efed062a1b360017d3642140d
Reviewed-on: https://go-review.googlesource.com/c/go/+/305274
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
We haven't needed this debugging flag in a while and it's going to
complicate a change to how to generate wrappers. Eliminate it in favor
of just using the objabi.Experiment.RegabiWrappers global toggle.
Updates #40724.
Change-Id: Ieda660ea7a0167ae4e881b396ef556d7c962fe4c
Reviewed-on: https://go-review.googlesource.com/c/go/+/305273
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
