getg reads from memory, so it should really have a
memory arg. It is critical in functions which call setg
to make sure getg gets ordered correctly with setg.
Change-Id: Ief4875421f741fc49c07b0e1f065ce2535232341
Reviewed-on: https://go-review.googlesource.com/16100
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
It isn't safe, the place where we're moving the value to
might have a different live memory. Moving will introduce
two simultaneously live memories.
Change-Id: I07e61a6db8ef285088c530dc2e5d5768d27871ff
Reviewed-on: https://go-review.googlesource.com/16099
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
It isn't safe in functions that also call setg.
Change-Id: I76a7bf0401b4b6c8a129c245b15a2d6f06080e94
Reviewed-on: https://go-review.googlesource.com/16095
Reviewed-by: Todd Neal <todd@tneal.org>
Rematerialize constants instead of spilling and loading them.
"Constants" includes constant offsets from SP and SB.
Should help somewhat with stack frame sizes. I'm not sure
exactly how much yet.
Change-Id: I44dbad97aae870cf31cb6e89c92fe4f6a2b9586f
Reviewed-on: https://go-review.googlesource.com/16029
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reuse the ArgSymbol for nodes so that the Aux values will be equal for cse.
Change-Id: Iaae80bd19ff2d3f51b6c9049fd860e04baa6f175
Reviewed-on: https://go-review.googlesource.com/15930
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: Todd Neal <todd@tneal.org>
For each type, maintain a list of stack slots used to spill
SSA values to the stack. Reuse those stack slots for noninterfering
spills.
Lowers frame sizes. As an example, runtime.mSpan_Sweep goes from
584 bytes to 392 bytes. heapBitsSetType goes from 576 bytes to 152 bytes.
Change-Id: I0e9afe80c2fd84aff9eb368318685de293c363d0
Reviewed-on: https://go-review.googlesource.com/16022
Reviewed-by: David Chase <drchase@google.com>
Changed racewalk/race detector to use FP in a more
sensible way.
Relaxed checks for CONVNOP when race detecting.
Modified tighten to ensure that GetClosurePtr cannot float
out of entry block (turns out this cannot be relaxed, DX is
sometimes stomped by other code accompanying race detection).
Added case for addr(CONVNOP)
Modified addr to take "bounded" flag to suppress nilchecks
where it is set (usually, by race detector).
Cannot leave unimplemented-complainer enabled because it
turns out we are optimistically running SSA on every platform.
Change-Id: Ife021654ee4065b3ffac62326d09b4b317b9f2e0
Reviewed-on: https://go-review.googlesource.com/15710
Reviewed-by: Keith Randall <khr@golang.org>
Some rewrite rules need to make sure the rewrite target ends up
in a specific block. For example:
(MOVBQSX (MOVBload [off] {sym} ptr mem)) ->
@v.Args[0].Block (MOVBQSXload <v.Type> [off] {sym} ptr mem)
The MOVBQSXload op needs to be in the same block as the MOVBload
(to ensure exactly one memory is live at basic block boundaries).
Change-Id: Ibe49a4183ca91f6c859cba8135927f01d176e064
Reviewed-on: https://go-review.googlesource.com/15804
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
This can lead to multiple stores being live at once.
Do OINDEX and ODOT using addresses & loads instead of specific ops.
This keeps SSA values from containing unSSAable types.
Change-Id: I79567e9d43cdee09084eb89ea0bd7aa3aad48ada
Reviewed-on: https://go-review.googlesource.com/15654
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Turns out that these do occur after all, so did the obvious
refactoring into the addr method.
Also added better debugging for the case of unhandled
closure args.
Change-Id: I1cd8ac58f78848bae0b995736f1c744fd20a6c95
Reviewed-on: https://go-review.googlesource.com/15640
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
Changed tree generation to correctly use PARAMOUT instead
of PARAM.
Emit Func.Exit before any returns.
Change-Id: I2fa53cc7fad05fb4eea21081ba33d1f66db4ed49
Reviewed-on: https://go-review.googlesource.com/15610
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
One was OAPPEND of large types. We need to mem-mem copy them
instead of storing them.
Another was pointer-like struct and array types being put in the
data field of an eface. We need to use the underlying pointer
type for the load that fills in the eface.data field.
Change-Id: Id8278c0381904e52d59011a66ce46386b41b5521
Reviewed-on: https://go-review.googlesource.com/15552
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
For appending large types, we want to evaluate the
values being appended after the growslice call, not before.
Evaluating them before leads to phi operations on large types
which confuses the lowering pass.
The order pass has already removed any side-effects from the
values being appended, so it doesn't matter if we do this
last eval before or after the growslice call.
This change fixes a bunch (but not all) of our failed lowerings.
Change-Id: I7c697d4d5275d71b7ef4677b830fd86c52ba03a4
Reviewed-on: https://go-review.googlesource.com/15430
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
Cleaned up first-block-in-function code.
Added cases for |PHEAP for PPARAM and PAUTO.
Made PPARAMOUT act more like PAUTO for purposes
of address generation and vardef placement.
Added cases for OCLOSUREVAR and Ops for getting closure
pointer. Closure ops are scheduled at top of entry block
to capture DX.
Wrote test that seems to show proper behavior for addressed
parameters, locals, and returns.
Change-Id: Iee93ebf9e3d9f74cfb4d1c1da8038eb278d8a857
Reviewed-on: https://go-review.googlesource.com/14650
Reviewed-by: Keith Randall <khr@golang.org>
Run-TryBot: David Chase <drchase@google.com>
There's no need for special ops for panicindex and panicslice.
Just use regular runtime calls.
Change-Id: I71b9b73f4f1ebce1220fdc1e7b7f65cfcf4b7bae
Reviewed-on: https://go-review.googlesource.com/14726
Reviewed-by: David Chase <drchase@google.com>
For variables which get SSA'd, SSA keeps track of all the def/kill.
It is only for on-stack variables that we need them.
This reduces stack frame sizes significantly because often the
only use of a variable was a varkill, and without that last use
the variable doesn't get allocated in the frame at all.
Fixes#12602
Change-Id: I3f00a768aa5ddd8d7772f375b25f846086a3e689
Reviewed-on: https://go-review.googlesource.com/14758
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We need to move any objects whose types are not SSA-able.
Fixes the "not lowered: Load ARRAY PTR64 mem" errors.
Change-Id: I7a0b609f917d7fb34bc9215fee4da15f9961cf6c
Reviewed-on: https://go-review.googlesource.com/14753
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The frontend rewrites most literals, so we see only zero
ones during SSA construction. We can implement those
using the existing zeroing behavior.
Change-Id: I390ad1be0a4b6729baf0c8936c7610aae2aef049
Reviewed-on: https://go-review.googlesource.com/14754
Reviewed-by: David Chase <drchase@google.com>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Generate AUNDEF for every exit block, not just for certain
control values.
Change-Id: Ife500ac5159ee790bc1e70c0e9b0b1f854bc4c47
Reviewed-on: https://go-review.googlesource.com/14721
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
OCALLINTER, as well as ODEFER/OPROC with OCALLMETH/OCALLINTER.
Move all the call logic to its own routine, a lot of the
code is shared.
Change-Id: Ieac59596165e434cc6d1d7b5e46b78957e9c5ed3
Reviewed-on: https://go-review.googlesource.com/14464
Reviewed-by: Todd Neal <todd@tneal.org>
Reviewed-by: David Chase <drchase@google.com>
A simpler way to do iface/slice comparisons. Fixes some
cases of failed lowerings.
Change-Id: Ia252bc8648293a2d460f63c41f1591785543a1e9
Reviewed-on: https://go-review.googlesource.com/14493
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Fixes build. Not great, but it will let others make progress.
Change-Id: If9cf2bbb5016e40aa91cf1c8bb62982ae2aed5e4
Reviewed-on: https://go-review.googlesource.com/14621
Reviewed-by: Keith Randall <khr@golang.org>
Load-and-sign-extend opcodes were being generated in the
wrong block, leading to having more than one memory variable
live at once. Fix the rules + add a test.
Change-Id: Iadf80e55ea901549c15c628ae295c2d0f1f64525
Reviewed-on: https://go-review.googlesource.com/14591
Reviewed-by: Todd Neal <todd@tneal.org>
Run-TryBot: Todd Neal <todd@tneal.org>
For now, we only use typedmemmove. This can be optimized
in future CLs.
Also add a feature to help with binary searching bad compilations.
Together with GOSSAPKG, GOSSAHASH specifies the last few binary digits
of the hash of function names that should be compiled. So
GOSSAHASH=0110 means compile only those functions whose last 4 bits
of hash are 0110. By adding digits to the front we can binary search
for the function whose SSA-generated code is causing a test to fail.
Change-Id: I5a8b6b70c6f034f59e5753965234cd42ea36d524
Reviewed-on: https://go-review.googlesource.com/14530
Reviewed-by: Keith Randall <khr@golang.org>
This makes deadstore elimination work reasonably again.
Change-Id: I3a8caced71f12dfb6c1d0c68b7a7d8d7a736ea23
Reviewed-on: https://go-review.googlesource.com/14536
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Move the AST to SSA conversion to the caller.
This enables it to be used in contexts in which
the RHS is already an *ssa.Value.
Change-Id: Ibb87210fb9fda095a9b7c7f4ad1264a7cbd269bf
Reviewed-on: https://go-review.googlesource.com/14521
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
Move to implicit (mostly) instead of explicit exit blocks.
RET and RETJMP have no outgoing edges - they implicitly exit.
CALL only has one outgoing edge, as its exception edge is
implicit as well.
Exit blocks are only used for unconditionally panicking code,
like the failed branches of nil and bounds checks.
There may now be more than one exit block. No merges happen
at exit blocks.
The only downside is it is harder to find all the places code
can exit the method. See the reverse dominator code for an
example.
Change-Id: I42e2fd809a4bf81301ab993e29ad9f203ce48eb0
Reviewed-on: https://go-review.googlesource.com/14462
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
It was using 64-bit float comparison ops for complex64.
It should use 32-bit float comparison.
Fixes build.
Change-Id: I6452b227257fecc09e04cd092ccf328d1fc9917f
Reviewed-on: https://go-review.googlesource.com/14497
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
They were using the result type to look up the op, not the arg type.
Change-Id: I0641cba363fa6e7a66ad0860aa340106c10c2cea
Reviewed-on: https://go-review.googlesource.com/14469
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Add line that was inadvertently removed.
Change-Id: I99ebc1041e984e408ae5825836c28b9891d6043b
Reviewed-on: https://go-review.googlesource.com/14470
Run-TryBot: Todd Neal <todd@tneal.org>
Reviewed-by: Keith Randall <khr@golang.org>
This matches existing behavior, see issue #2196
Change-Id: Ifa9359b7c821115389f337a57de355c5ec23be8f
Reviewed-on: https://go-review.googlesource.com/14261
Reviewed-by: Keith Randall <khr@golang.org>
Added tree numbering data structure.
Changed dominator query in CSE.
Removed skip-for-too-big patch in CSE.
Passes all.bash.
Change-Id: I98d7c61b6015c81f5edab553615db17bc7a58d68
Reviewed-on: https://go-review.googlesource.com/14326
Reviewed-by: Keith Randall <khr@golang.org>
Evaluating args can overwrite arg area, so we can't write argsize and func
until args are evaluated.
Fixes test/recover.go, test/recover1.go, and test/fixedbugs/issue4066.go
Change-Id: I862e4934ccdb8661431bcc3e1e93817ea834ea3f
Reviewed-on: https://go-review.googlesource.com/14405
Reviewed-by: David Chase <drchase@google.com>
Make sure that return blocks take a store as their control. Without
this, code was getting inserted between the return and exit blocks.
Use AEND to mark the end of code. The live variable analysis gets
confused when routines end like:
JMP earlier
RET
because the RET is unreachable. The RET was incorrectly added to the
last basic block, rendering the JMP invisible to the CFG builder.
Change-Id: I91b32c8b37075347243ff039b4e4385856fba7cd
Reviewed-on: https://go-review.googlesource.com/14398
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
