Motivation:
Previously, we assumed that the set of types for which a
complete method set (containing all synthesized wrapper
functions) is required at runtime was the set of types
used as operands to some *ssa.MakeInterface instruction.
In fact, this is an underapproximation because types can
be derived from other ones via reflection, and some of
these may need methods. The reflect.Type API allows *T to
be derived from T, and these may have different method
sets. Reflection also allows almost any subcomponent of a
type to be accessed (with one exception: given T, defined
'type T struct{S}', you can reach S but not struct{S}).
As a result, the pointer analysis was unable to generate
all necessary constraints before running the solver,
causing a crash when reflection derives types whose
methods are unavailable. (A similar problem would afflict
an ahead-of-time compiler based on ssa. The ssa/interp
interpreter was immune only because it does not require
all wrapper methods to be created before execution
begins.)
Description:
This change causes the SSA builder to record, for each
package, the set of all types with non-empty method sets that
are referenced within that package. This set is accessed via
Packages.TypesWithMethodSets(). Program.TypesWithMethodSets()
returns its union across all packages.
The set of references that matter are:
- types of operands to some MakeInterface instruction (as before)
- types of all exported package members
- all subcomponents of the above, recursively.
This is a conservative approximation to the set of types
whose methods may be called dynamically.
We define the owning package of a type as follows:
- the owner of a named type is the package in which it is defined;
- the owner of a pointer-to-named type is the owner of that named type;
- the owner of all other types is nil.
A package must include the method sets for all types that it
owns, and all subcomponents of that type that are not owned by
another package, recursively. Types with an owner appear in
exactly one package; types with no owner (such as struct{T})
may appear within multiple packages.
(A typical Go compiler would emit multiple copies of these
methods as weak symbols; a typical linker would eliminate
duplicates.)
Also:
- go/types/typemap: implement hash function for *Tuple.
- pointer: generate nodes/constraints for all of
ssa.Program.TypesWithMethodSets().
Add rtti.go regression test.
- Add API test of Package.TypesWithMethodSets().
- Set Function.Pkg to nil (again) for wrapper functions,
since these may be shared by many packages.
- Remove a redundant logging statement.
- Document that ssa CREATE phase is in fact sequential.
Fixesgolang/go#6605
R=gri
CC=golang-dev
https://golang.org/cl/14920056
A function such as this:
func one() (x int) {
defer func() { recover() }()
x = 1
panic("return")
}
that combines named return parameters (NRPs) with deferred calls
that call recover, may return non-zero values despite the
fact it doesn't even contain a return statement. (!)
This requires a change to the SSA API: all functions'
control-flow graphs now have a second entry point, called
Recover, which is the block at which control flow resumes
after a recovered panic. The Recover block simply loads the
NRPs and returns them.
As an optimization, most functions don't need a Recover block,
so it is omitted. In fact it is only needed for functions that
have NRPs and defer a call to another function that _may_ call
recover.
Dataflow analysis of SSA now requires extra work, since every
may-panic instruction has an implicit control-flow edge to
the Recover block. The only dataflow analysis so far implemented
is SSA renaming, for which we make the following simplifying
assumption: the Recover block only loads the NRPs and returns.
This means we don't really need to analyze it, we can just
skip the "lifting" of such NRPs. We also special-case the Recover
block in the dominance computation.
Rejected alternative approaches:
- Specifying a Recover block for every defer instruction (like a
traditional exception handler).
This seemed like excessive generality, since Go programs
only need the same degenerate form of Recover block.
- Adding an instruction to set the Recover block immediately
after the named return values are set up, so that dominance
can be computed without special-casing.
This didn't seem worth the effort.
Interpreter:
- This CL completely reimplements the panic/recover/
defer logic in the interpreter. It's clearer and simpler
and closer to the model in the spec.
- Some runtime panic messages have been changed to be closer
to gc's, since tests depend on it.
- The interpreter now requires that the runtime.runtimeError
type be part of the SSA program. This requires that clients
import this package prior to invoking the interpreter.
This in turn requires (Importer).ImportPackage(path string),
which this CL adds.
- All $GOROOT/test/recover{,1,2,3}.go tests are now passing.
NB, the bug described in coverage.go (defer/recover in a concatenated
init function) remains. Will be fixed in a follow-up.
Fixesgolang/go#6381
R=gri
CC=crawshaw, golang-dev
https://golang.org/cl/13844043
Before, we would concatenate all the init() blocks together,
resulting in incorrect treatment of a recovered panic in one
init block: the implicit return would cause the subsequent ones
to be skipped.
The result is simpler, and closer to what gc does.
The additional functions are visible in the call graph,
so some tests required updating.
R=gri
CC=crawshaw, golang-dev
https://golang.org/cl/14671044
Motivation: pointer analysis tools (like the oracle) want the
user to specify a set of initial packages, like 'go test'.
This change enables the user to specify a set of packages on
the command line using importer.LoadInitialPackages(args).
Each argument is interpreted as either:
- a comma-separated list of *.go source files together
comprising one non-importable ad-hoc package.
e.g. "src/pkg/net/http/triv.go" gives us [main].
- an import path, denoting both the imported package
and its non-importable external test package, if any.
e.g. "fmt" gives us [fmt, fmt_test].
Current type-checker limitations mean that only the first
import path may contribute tests: multiple packages augmented
by *_test.go files could create import cycles, which 'go test'
avoids by building a separate executable for each one.
That approach is less attractive for static analysis.
Details: (many files touched, but importer.go is the crux)
importer:
- PackageInfo.Importable boolean indicates whether
package is importable.
- un-expose Importer.Packages; expose AllPackages() instead.
- CreatePackageFromArgs has become LoadInitialPackages.
- imports() moved to util.go, renamed importsOf().
- InitialPackagesUsage usage message exported to clients.
- the package name for ad-hoc packages now comes from the
'package' decl, not "main".
ssa.Program:
- added CreatePackages() method
- PackagesByPath un-exposed, renamed 'imported'.
- expose AllPackages and ImportedPackage accessors.
oracle:
- describe: explain and workaround a go/types bug.
Misc:
- Removed various unnecessary error.Error() calls in Printf args.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13579043
CanonicalPos was inadequate since many pairs of instruction share the same pos (e.g. Allocs and Phis). Instead, we generalize the DebugRef instruction to associate not just Idents but Exprs with ssa.Values.
We no longer store any DebugRefs for constant expressions, to save space. (The type and value of such expressions can be obtained by other means, at a cost in complexity.)
Function.ValueForExpr queries the DebugRef info to return the ssa.Value of a given Expr.
Added tests.
Also:
- the DebugInfo flag is now per package, not global.
It must be set between Create and Build phases if desired.
- {Value,Instruction}.Pos() documentation updated: we still maintain
this information in the instruction stream even in non-debug mode,
but we make fewer claims about its invariants.
- Go and Defer instructions can now use their respective go/defer
token positions (not the call's lparen), so they do.
- SelectState:
Posn token.Pos indicates the <- position
DebugNode ast.Expr is the send stmt or receive expr.
- In building SelectStmt, we introduce extra temporaries in debug
mode to hold the result of the receive in 'case <-ch' even though
this value isn't ordinarily needed.
- Use *SelectState (indirectly) since the struct is getting bigger.
- Document some missing instructions in doc.go.
R=gri
CC=golang-dev
https://golang.org/cl/12147043
buildDecl was visiting all decls in source order, but the spec
calls for visiting all vars and init() funcs in order, then
all remaining functions. These two passes are now called
buildInit(), buildFuncDecl().
+ Test.
Also:
- Added workaround to gcimporter for Func with pkg==nil.
- Prog.concreteMethods has been merged into Pkg.values.
- Prog.concreteMethod() renamed declaredFunc().
- s/mfunc/obj/ (name cleanup from recent gri CL)
R=gri
CC=golang-dev
https://golang.org/cl/12030044
We now use LookupFieldOrMethod for all SelectorExprs, and
simplify the logic to discriminate the various cases.
We inline static calls to promoted/indirected functions,
dramatically reducing the number of functions created.
More tests are needed, but I'd like to submit this as-is.
In this CL, we:
- rely less on Id strings. Internally we now use
*types.Method (and its components) almost everywhere.
- stop thinking of types.Methods as objects. They don't
have stable identities. (Hopefully they will become
plain-old structs soon.)
- eliminate receiver indirection wrappers:
indirection and promotion are handled together by makeWrapper.
- Handle the interactions of promotion, indirection and
abstract methods much more cleanly.
- support receiver-bound interface method closures.
- break up builder.selectField so we can re-use parts
(emitFieldSelection).
- add importer.PackageInfo.classifySelector utility.
- delete interfaceMethodIndex()
- delete namedTypeMethodIndex()
- delete isSuperInterface() (replaced by types.IsAssignable)
- call memberFromObject on each declared concrete method's
*types.Func, not on every Method frem each method set, in the
CREATE phase for packages loaded by gcimporter.
go/types:
- document Func, Signature.Recv() better.
- use fmt in {Package,Label}.String
- reimplement Func.String to be prettier and to include method
receivers.
API changes:
- Function.method now holds the types.Method (soon to be
not-an-object) for synthetic wrappers.
- CallCommon.Method now contains an abstract (interface)
method object; was an abstract method index.
- CallCommon.MethodId() gone.
- Program.LookupMethod now takes a *Method not an Id string.
R=gri
CC=golang-dev
https://golang.org/cl/11674043
Details:
- emitImplicitSelections now emits common code for implicit
field selections in both method and field lookups.
The last iteration over the LookupFieldOrMethod indices---the explicit,
final index---is handled by the caller.
- anonFieldPath, candidate and the BFS algo in buildMethodSet are all gone.
R=gri
CC=golang-dev
https://golang.org/cl/11576045
ssa:
- Prog.CreatePackages inlined into all callers.
- Prog.CreatePackage is now exposed; idempotent; and checks for errors.
- '*address' not 'address' now implements lvalue (since it's 6 words).
- removed types.Method case from createMemberFromObject.
importer:
- added importer.PackageInfo.String method.
- simplifed importer.PackageInfo by putting types.Info in it.
- removed obsolete precondition from IsType.
R=gri
CC=golang-dev
https://golang.org/cl/11408045
- fixed method set computation
- lazily compute it for a type on demand
- lazy allocation of various temporary maps
Also:
- removed Interface.IsEmpty, Scope.IsEmpty
(these convenience functions didn't carry their weight)
- cosmetic changes
Next:
- consolidate various lookup APIs
- use lazily computed method sets for various checks
instead of individual lookups
R=adonovan
CC=golang-dev
https://golang.org/cl/11385044
(Motivation: "Literal" is a syntactic property, not a semantic one.)
Also: delete a "TODO: opt" that the lifting pass already does for us.
R=gri
CC=golang-dev
https://golang.org/cl/11351043
- removed a number of obsolete TODO(gri) comments.
- bring ssa.DefaultType back into sync with types.defaultType.
- re-enable types.Package.Path()!="" assertion.
- use Path() (not reflect pointer) in sort routine.
- make interp.checkInterface use types.MissingMethod.
- un-export ssa.MakeId function.
- inline pointer() into all callers, and delete.
- enable two more interp_tests: $GOROOT/test/{method3,cmp}.go
- add links to bugs to other interp_tests.
- add runtime.NumCPU to ssa/interp/externals.go
R=gri
CC=golang-dev
https://golang.org/cl/11353043
This CL adds three new functions to determine the SSA Value
for a given syntactic var, func or const object:
Program.{Const,Func,Var}Value.
Since constants and functions are immutable, the first
two only need a types.Object; but each distinct
reference to a var may return a distinct Value, so the third
requires an ast.Ident parameter too.
Debug information for local vars is encoded in the
instruction stream in the form of DebugRef instructions,
which are a no-op but relate their operand to a particular
ident in the AST. The beauty of this approach is that it
naturally stays consistent during optimisation passes
(e.g. lifting) without additional bookkeeping.
DebugRef instructions are only generated if the DebugMode
builder flag is set; I plan to make the policy more fine-
grained (per function).
DebugRef instructions are inserted for:
- expr(Ident) for rvalue idents
- address.store() for idents that update an lvalue
- address.address() for idents that take address of lvalue
(this new method replaces all uses of lval.(address).addr)
- expr() for all constant expressions
- local ValueSpecs with implicit zero initialization (no RHS)
(this case doesn't call store() or address())
To ensure we don't forget to emit debug info for uses of Idents,
we must use the lvalue mechanism consistently. (Previously,
many simple cases had effectively inlined these functions.)
Similarly setCallFunc no longer inlines expr(Ident).
Also:
- Program.Value() has been inlined & specialized.
- Program.Package() has moved nearer the new lookup functions.
- refactoring: funcSyntax has lost paramFields, resultFields;
gained funcType, which provides access to both.
- add package-level constants to Package.values map.
- opt: don't call localValueSpec for constants.
(The resulting code is always optimised away.)
There are a number of comments asking whether Literals
should have positions. Will address in a follow-up.
Added tests of all interesting cases.
R=gri
CC=golang-dev
https://golang.org/cl/11259044
Objects:
- provide IsExported, SameName, uniqueName methods
- clean up a lot of dependent code
Scopes:
- don't add children to Universe scope (!)
- document Node, WriteTo
Types:
- remove Deref in favor of internal function deref
ssa, ssa/interp:
- introduced local deref, adjusted code
- fixed some "Underlying" bugs (pun intended)
R=adonovan
CC=golang-dev
https://golang.org/cl/11232043
This exposed a bug: we weren't creating Functions for methods
of imported named types.
Also:
- simplification: 'candidate' no longer contains 'concrete *Function'.
We look this up on demand.
NB: this CL contains a copy of CL 10935047 (use of typemap);
will submit/sync/resolve soon.
R=gri
CC=golang-dev
https://golang.org/cl/11051043
Yields a ~20% improvement in SSA construction time.
Also: better names for promotion wrapper functions.
R=gri
CC=golang-dev
https://golang.org/cl/11050043
We use the new field to determine whether or not a function is
synthetic, not Pos() == 0, so synthetic functions can have
positions too.
R=gri
CC=golang-dev
https://golang.org/cl/10916044
Before, all values received on some channel by Select would
flow to an empty interface, creating a spurious confluence for
flow analyses. Now, the tuple returned by Select has one
component for each 'receive' case.
Also, fixes:
- Removed workarounds for now-fixed typechecker bug in FuncLit+TypeAssert.
- sanity check that all Value Instructions have non-nil Type().
- Convert: document and sanity-check that at least one of the types is basic.
Also, other things to help clients:
- Define CallInstruction interface: common parts of Call, Go, Defer.
- Add CallCommon.Signature() method.
- Literal.Pos() is now populated.
R=gri
CC=golang-dev
https://golang.org/cl/10505043
methodIndex() utility was split and specialized to its two
cases, *Interface vs *Named, which are logically quite
different.
We can't memoize promotion wrappers yet; we need typemap.
Terminology:
- "thunks" are now "wrappers"
- "bridge methods" are now "promotion wrappers"
Where the diff is messy it's just because of indentation.
R=gri
CC=golang-dev
https://golang.org/cl/10282043
Method sets:
- Simplify CallCommon.
Avoid the implicit copy when calling a T method on a *T
receiver. This simplifies clients. Instead we generate
"indirection wrapper" functions that do this (like gc does).
New invariant:
m's receiver type is exactly T for all m in MethodSet(T)
- MakeInterface no longer holds the concrete type's MethodSet.
We can defer its computation this way.
- ssa.Type now just wraps a types.TypeName object.
MethodSets are computed as needed, not eagerly.
Position info:
- new CanonicalPos utility maps ast.Expr to canonical
token.Pos, as returned by {Instruction,Value}.Pos() methods.
- Don't set posn for implicit operations (e.g. varargs array alloc)
- Set position info for ChangeInterface and Slice instructions.
Cosmetic:
- add Member.Token() method
- simplify isPointer
- Omit words "interface", "slice" when printing MakeInterface,
MakeSlice; the type is enough.
- Comments on PathEnclosingInterval.
- Remove Function.FullName() where implicit String() suffices.
Also:
- Exposed NewLiteral to clients.
- Added ssa.Instruction.Parent() *Function
Added ssa.BasicBlock.Parent() *Function.
Added Sanity checks for above.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/10166045
Details:
- builder is now un-exported and is now a per-package entity.
- Package.nTo1Vars is now part of builder, where it belongs.
- CREATE phase code split out into its own file, create.go
- Context type is gone; it had become trivial after the
Importer refactoring.
- importer.PackageInfo.Imports() now encapsulates iteration
over imports.
Typical usage is now:
prog := ssa.NewProgram(imp.Fset, mode)
prog.CreatePackages(imp)
prog.BuildAll()
Builder.BuildPackage(Package) is now Package.Build()
Builder.BuildAllPackages() is now Program.BuildAll()
R=iant, gri
CC=golang-dev
https://golang.org/cl/9970044
