For certain type of method wrappers we used to generate a tail
call. That was disabled in CL 307234 when register ABI is used,
because with the current IR it was difficult to generate a tail
call with the arguments in the right places. The problem was that
the IR does not contain a CALL-like node with arguments; instead,
it contains an OAS node that adjusts the receiver, than an
OTAILCALL node that just contains the target, but no argument
(with the assumption that the OAS node will put the adjusted
receiver in the right place). With register ABI, putting
arguments in registers are done in SSA. The assignment (OAS)
doesn't put the receiver in register.
This CL changes the IR of a tail call to take an actual OCALL
node. Specifically, a tail call is represented as
OTAILCALL (OCALL target args...)
This way, the call target and args are connected through the OCALL
node. So the call can be analyzed in SSA and the args can be passed
in the right places.
(Alternatively, we could have OTAILCALL node directly take the
target and the args, without the OCALL node. Using an OCALL node is
convenient as there are existing code that processes OCALL nodes
which do not need to be changed. Also, a tail call is similar to
ORETURN (OCALL target args...), except it doesn't preserve the
frame. I did the former but I'm open to change.)
The SSA representation is similar. Previously, the IR lowers to
a Store the receiver then a BlockRetJmp which jumps to the target
(without putting the arg in register). Now we use a TailCall op,
which takes the target and the args. The call expansion pass and
the register allocator handles TailCall pretty much like a
StaticCall, and it will do the right ABI analysis and put the args
in the right places. (Args other than the receiver are already in
the right places. For register args it generates no code for them.
For stack args currently it generates a self copy. I'll work on
optimize that out.) BlockRetJmp is still used, signaling it is a
tail call. The actual call is made in the TailCall op so
BlockRetJmp generates no code (we could use BlockExit if we like).
This slightly reduces binary size:
old new
cmd/go 14003088 13953936
cmd/link 6275552 6271456
Change-Id: I2d16d8d419fe1f17554916d317427383e17e27f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/350145
Trust: Cherry Mui <cherryyz@google.com>
Run-TryBot: Cherry Mui <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
This disables the "testing names" for method names and
trailing input types passed to closure/interface/other calls.
The logic using the names remains, so that editing the change
to enable local testing is not too hard.
Also fixes broken build tag in reflect/abi_test.go
Updates #44816.
Change-Id: I3d222d2473c98d04ab6f1122ede9fea70c994af1
Reviewed-on: https://go-review.googlesource.com/c/go/+/300150
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
In the register allocator, if possible, we allocate a value to its
desired register (the ideal register for its next use). In some
cases the desired register does not satisfies the value's output
register mask. We should not use the register in this case.
In the following example, v33 is going to be returned as a
function result, so it is allocated to its desired register AX.
However, its Op cannot use AX as output, causing miscompilation.
v33 = CMOVQEQF <int> v24 v28 v29 : AX (~R0[int])
v35 = MakeResult <int,int,mem> v33 v26 v18
Ret v35
Change-Id: Id0f4f27c4b233ee297e83077e3c8494fe193e664
Reviewed-on: https://go-review.googlesource.com/c/go/+/314630
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The defer wrapping feature added to the compiler's "order" phase
creates temporaries into which it copies defer arguments. If one of
these temps is large enough that we place it into the defer closure by
address (as opposed to by value), then the temp in question can't be
reused later on in the order phase, nor do we want a VARKILL
annotation for it at the end of the current block scope.
Test written by Cherry.
Updates #40724.
Change-Id: Iec7efd87ec5a3e3d7de41cdcc7f39c093ed1e815
Reviewed-on: https://go-review.googlesource.com/c/go/+/312869
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When creating the temporary for map functions, if the key
contains pointer, we need to create pointer-typed temporary. So
if the temporary is live across a function call, the pointer is
live.
Change-Id: Id6e14ec9def8bc7987f0f8ce8423caf1e3754fcb
Reviewed-on: https://go-review.googlesource.com/c/go/+/311379
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
A quick check of the source to run.go suggests that it does not
look for the new-style build tags.
Updates #45465.
Change-Id: Ib4be040935d71e732f81d52c4a22c2b514195f40
Reviewed-on: https://go-review.googlesource.com/c/go/+/308934
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: David Chase <drchase@google.com>
This tickles some other bug, do this to clear builders.
Updates #40724.
Updates #45465.
Change-Id: Id51efbcf474865da231fcbc6216e5d604f99c296
Reviewed-on: https://go-review.googlesource.com/c/go/+/308889
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
The helper function used by the compiler's walk phase to determine
whether a param can be passed in a single float register wasn't quite
correct (didn't allow for the possibility of struct with two fields,
first zero size and second float). Fix up the helper to take this
case into account.
Updates #40724.
Change-Id: I55b42a1b17ea86de1d696788f029ad3aae4a179c
Reviewed-on: https://go-review.googlesource.com/c/go/+/308689
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The function runtime.convT64 accepts a single uint64 argument, but the
compiler's rules in the walk phase for determining whether is it ok to
pass a value of type T to a call to runtime.convT64 were slightly off.
In particular the test was allowing a type T with size less than eight
bytes but with more than one internal element (e.g. a struct). This
patch tightens up the rules somewhat to prevent this from happening.
Updates #40724.
Change-Id: I3b909267534db59429b0aa73a3d73333e1bd6432
Reviewed-on: https://go-review.googlesource.com/c/go/+/308069
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
go.mod file was not tidy, made builders sad.
Updates #40724.
Change-Id: I28371a1093108f9ec473eb20bb4d185e35dee67d
Reviewed-on: https://go-review.googlesource.com/c/go/+/308590
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
In expand_calls, OpSelectN occurs both before and after the rewriting.
Attempting to rewrite a post-expansion OpSelectN is bad.
(The only ones rewritten in place are the ones returning mem;
others are synthesized to replace other selection chains with
register references.)
Updates #40724.
Updates #44816#issuecomment-815258897.
Change-Id: I7b6022cfb47f808d3ce6cc796c067245f36047f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/308309
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
There's a problem in liveness, where liveness of any
part of an aggregate keeps the whole aggregate alive,
but the not-live parts don't get spilled. The GC
can observe those live-but-not-spilled slots, which
can contain junk.
A better fix is to change liveness to work
pointer-by-pointer, but that is also a riskier,
trickier fix.
To avoid this, in the case of
(1) an aggregate input parameter
(2) containing pointers
(3) passed in registers
pre-spill the pointers.
Updates #40724.
Change-Id: I6beb8e0a353b1ae3c68c16072f56698061922c04
Reviewed-on: https://go-review.googlesource.com/c/go/+/307909
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When a function panics then recovers, it needs to return to the
caller with named results having the correct values. For
in-register results, we need to load them into registers at the
defer return path.
For non-open-coded defers, we already generate correct code, as
the defer return path is part of the SSA CFG and contains the
instructions that are the same as an ordinary return statement,
including putting the results to the right places.
For open-coded defers, we have a special code generation that
emits a disconnected block that currently contains only the
deferreturn call and a RET instruction. It leaves the result
registers unset. This CL adds instructions that load the result
registers on that path.
Updates #40724.
Change-Id: I1f60514da644fd5fb4b4871a1153c62f42927282
Reviewed-on: https://go-review.googlesource.com/c/go/+/307231
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Austin Clements <austin@google.com>
This fixes a compile crash for
GOEXPERIMENT=regabi,regabiargs go test -c go/constant
Updates #40724.
Change-Id: I238cef436e045647815326fc8fdb025c30ba1f5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/307309
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Leftover values that have been replaced can cause problems in later
passes (within expandCalls). For example, a struct select that
itself yields a struct will have a problematic rewrite, if the chance
is presented.
Updates #40724.
Change-Id: I1b445c47c301c3705f7fc0a9d39f1f5c84f4e190
Reviewed-on: https://go-review.googlesource.com/c/go/+/306869
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Includes test.
Long term, need to make the offending code be more in terms
of official types package offsets, instead of duplicating that
logic.
For #40724.
Change-Id: Id33a153f10aed3289cc48d1f99a8e0f6ece9474d
Reviewed-on: https://go-review.googlesource.com/c/go/+/306469
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
For in-register arguments, it must have only a single copy of it
present in the function. If there are multiple copies, it confuses
the register allocator, as they are in the same register.
Change-Id: I55cb06746f08aa7c9168026d0f411bce0a9f93f4
Reviewed-on: https://go-review.googlesource.com/c/go/+/306330
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
In expand_calls, when rewriting OpArg to OpArgIntReg/OpArgFloatReg,
avoid generating duplicates. Otherwise it will confuse the
register allocator: it would think the second occurance clobbers
the first's register, causing it to generate copies, which may
clobber other args.
Change-Id: I4f1dc0519afb77500eae1c0e6ac8745e51f7aa4e
Reviewed-on: https://go-review.googlesource.com/c/go/+/306029
Trust: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: David Chase <drchase@google.com>
Code generation for open defers failed to account for
presence of method receiver and thus was OFF BY ONE.
Fixes#45062.
Updates #44816.
Updates #40724.
Change-Id: Ia90ea8fd0f7d823e1f757c406f9127136c2ffdd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/302249
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Repair of CL 300749.
ALSO:
found evidence that stack maps for bodyless methods are wrong.
gofmt in test/abi
removed never-executed code in types/size.go
Updates #44816.
Updates #40724.
Change-Id: Ifeb5fee60f60e7c7b58ee0457f58a3265d6cf3f6
Reviewed-on: https://go-review.googlesource.com/c/go/+/302071
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This reverts commit 8ed438c077, CL 300749.
Reason for revert: Looks like it crashes on link-register architectures
Change-Id: I0c261df58900008cada3359889d2a87508158447
Reviewed-on: https://go-review.googlesource.com/c/go/+/302053
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
ALSO:
found evidence that stack maps for bodyless methods are wrong.
gofmt in test/abi
removed never-executed code in types/size.go
Updates #44816.
Change-Id: I658c33f049337fb6f1e625f0c55430d25bfa877e
Reviewed-on: https://go-review.googlesource.com/c/go/+/300749
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This is enabled with a ridiculous magic name for method,
or for last input type passed, that needs to be changed
to something inutterable before actual release.
Ridiculous method name: MagicMethodNameForTestingRegisterABI
Ridiculous last (input) type name: MagicLastTypeNameForTestingRegisterABI
RLTN is tested with strings.Contains, so you can have
MagicLastTypeNameForTestingRegisterABI1
and
MagicLastTypeNameForTestingRegisterABI2
if that is helpful
Includes test test/abi/fibish2.go
Updates #44816.
Change-Id: I592a6edc71ca9bebdd1d00e24edee1ceebb3e43f
Reviewed-on: https://go-review.googlesource.com/c/go/+/299410
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
ABI info producer and consumer had different ideas for register
order for parameters.
Includes a test, includes improvements to debugging output.
Updates #44816.
Change-Id: I4812976f7a6c08d6fc02aac1ec0544b1f141cca6
Reviewed-on: https://go-review.googlesource.com/c/go/+/299570
Trust: David Chase <drchase@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Includes more enhancements to debugging output.
Updates #44816.
Change-Id: I5b21815cf37ed21e7dec6c06f538090f32260203
Reviewed-on: https://go-review.googlesource.com/c/go/+/299409
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
includes three tests
Change-Id: I33ac0cfe35085d4b6ad2775abcaa3d7d6527b49f
Reviewed-on: https://go-review.googlesource.com/c/go/+/297031
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
It is not multithreaded-compilation-safe, and also seems
to cause problems on the noopt-builder.
Change-Id: I52dbcd507d256990f1ec7c8040ec7b76595aae4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/298850
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Added a test that exercises named results
Change-Id: Ie228b68f4f846266595a95e0f65a6e4b8bf79635
Reviewed-on: https://go-review.googlesource.com/c/go/+/297029
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
at least for ints and strings
includes simple test
For #40724.
Change-Id: Ib8484e5b957b08f961574a67cfd93d3d26551558
Reviewed-on: https://go-review.googlesource.com/c/go/+/295309
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Verified with test and with single step watching changes to register
values across morestack calls, after reload.
Also added stack-growth test with pointer parameters of varying lifetime.
For #40724.
Change-Id: Idb5fe27786ac5c6665a734d41e68d3d39de2f4da
Reviewed-on: https://go-review.googlesource.com/c/go/+/294429
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Morestack works for non-pointer register parameters
Within a function body, pointer-typed parameters are correctly
tracked.
Results still not hooked up.
For #40724.
Change-Id: Icaee0b51d0da54af983662d945d939b756088746
Reviewed-on: https://go-review.googlesource.com/c/go/+/294410
Trust: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
A function returning multiple results, some of them zero-width,
will have more than one result present at an offset. Be sure
that offset AND type match.
Includes test.
Change-Id: I3eb1f56116d989b4e73f533fefabb1bf554c901b
Reviewed-on: https://go-review.googlesource.com/c/go/+/297169
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Plumb abi information into ssa/ssagen for plain calls
and plain functions (not methods). Does not extend all the
way through the compiler (yet).
One test disabled because it extends far enough to break the test.
Normalized all the compiler's register args TODOs to
// TODO(register args) ...
For #40724.
Change-Id: I0173a4579f032ac3c9db3aef1749d40da5ea01ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/293389
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Currently, we call Warnl in SSA backend when we see a function
(defined or called) with regparams pragma. Calling Warnl in
concurrent environment is racy. As the debugging output is
temporary, for testing purposes we just pass -c=1. We'll remove
the pragma and the debugging print some time soon.
Change-Id: I6f925a665b953259453fc458490c5ff91f67c91a
Reviewed-on: https://go-review.googlesource.com/c/go/+/291710
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Now that TailCallStmt carries an *ir.Name instead of a *types.Sym,
callTargetLSym can be similarly updated to take the target function as
an *ir.Name.
This inches us closer towards being able to move Linksym and other
properties from *types.Sym to *ir.Name, where they belong.
Passes toolstash -cmp w/ -gcflags=all=-abiwrap.
Change-Id: I091da290751970eba8ed0438f66d6cca88b665a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/284228
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The feature being tested is insensitive to the OS anyway.
Change-Id: Ieac9bfaafc6a54c00017afcc0b87bd8bbe80af7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/284032
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
This only works for functions; if you try it with a method, it will
fail. It does work for both local package and imports. For now,
it tells you when it thinks it sees either a declaration or a call of
such a function (this will normally be silent since no existing
code uses this pragma).
Note: it appears to be really darn hard to figure out if this
pragma was set for a method, and the method's call site. Better
ir.Node wranglers than I might be able to make headway, but it
seemed unnecessary for this experiment.
Change-Id: I601c2ddd124457bf6d62f714d7ac871705743c0a
Reviewed-on: https://go-review.googlesource.com/c/go/+/279521
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
