1) We remove context sensitivity from API. The pointer analysis is
not sufficiently context-sensitive for the context information to
be worth exposing. (The actual analysis precision still benefits
from being context-sensitive, though.) Since all clients would
discard the context info, we now do that for them.
2) Make the graph doubly-linked. Edges are now shared by the Nodes
at both ends of the edge so it's possible to navigate more easily
(e.g. to the callers).
3) Graph and Node are now concrete, not interfaces.
Less code in every file!
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/66460043
CL 49530047 made the (over-)simplifying assumption that the
Path of an ad-hoc (Created) package should default to its
Name, i.e. package declaration.
With this change, the Name is still always computed from the
package declaration (by go/types) but the Path may be
specified by the loader.Config. If "", the value of the Name
is used, which is not globally unique.
R=gri, axwalk
CC=golang-codereviews
https://golang.org/cl/55180043
The Importer type has been replaced with Config and Program.
Clients populate a Config, directly or more usually via
convenience functions. They then call its Load() method to do
all of the typechecking and transitive-closure computation.
ssa.NewProgram and ssa.CreatePackages have been fused into
ssa.Create, which now cannot fail, since (*Config).Load()
reports all type errors.
Also:
- The addition of an ssa.GlobalDebug builder mode flag
eliminates a loop-over-packages repeated in many clients.
- PackageInfo.Err flag unexported. Clients never see bad infos now.
- cmd/ssadump: now only looks for func "main" in package "main".
- importsOf deleted, was dead code.
STILL TODO:
- ParseFile seems like API creep (though it's convenient)
and CreateFromFiles is dangerous (w.r.t. FileSet identity).
Need to think more...
- the need for clients to rely on elementwise correspondence
of Config.CreatePkgs and Program.Created is a little sad.
- The command-line interface has not changed.
That will happen in a follow-up.
r recommends using a repeated flag: -package p -package q ...
R=gri
CC=axwalk, frederik.zipp, golang-codereviews
https://golang.org/cl/49530047
Command set:
- what: an extremely fast query that parses a single
file and returns the AST stack, package name and the
set of query modes that apply to the current selection.
Intended for GUI tools that need to grey out UI elements.
- definition: shows the definition of an identifier.
- pointsto: the PTA features of 'describe' have been split
out into their own command.
- describe: with PTA stripped out, the cost is now bounded by
type checking.
Performance:
- The importer.Config.TypeCheckFuncBodies predicate supports
setting the 'IgnoreFuncBodies' typechecker flag on a
per-package basis. This means we can load dependencies from
source more quickly if we only need exported types.
(We avoid gcimport data because it may be absent or stale.)
This also means we can run type-based queries on packages
that aren't part of the pointer analysis scope. (Yay.)
- Modes that require only type analysis of the query package
run a "what" query first, and restrict their analysis scope
to just that package and its dependencies (sans func
bodies), making them much faster.
- We call newOracle not oracle.New in Query, so that the
'needs' bitset isn't ignored (oops!). This makes the
non-PTA queries faster.
Also:
- removed vestigial timers junk.
- pos.go: existing position utilties split out into own file.
Added parsePosFlag utility.
- numerous cosmetic tweaks.
+ very basic tests.
To do in follow-ups:
- sophisticated editor integration of "what".
- better tests.
- refactoring of control flow as described in comment.
- changes to "implements", "describe" commands.
- update design doc + user manual.
R=crawshaw, dominik.honnef
CC=golang-dev, gri
https://golang.org/cl/40630043
Motivation:
Previously, we assumed that the set of types for which a
complete method set (containing all synthesized wrapper
functions) is required at runtime was the set of types
used as operands to some *ssa.MakeInterface instruction.
In fact, this is an underapproximation because types can
be derived from other ones via reflection, and some of
these may need methods. The reflect.Type API allows *T to
be derived from T, and these may have different method
sets. Reflection also allows almost any subcomponent of a
type to be accessed (with one exception: given T, defined
'type T struct{S}', you can reach S but not struct{S}).
As a result, the pointer analysis was unable to generate
all necessary constraints before running the solver,
causing a crash when reflection derives types whose
methods are unavailable. (A similar problem would afflict
an ahead-of-time compiler based on ssa. The ssa/interp
interpreter was immune only because it does not require
all wrapper methods to be created before execution
begins.)
Description:
This change causes the SSA builder to record, for each
package, the set of all types with non-empty method sets that
are referenced within that package. This set is accessed via
Packages.TypesWithMethodSets(). Program.TypesWithMethodSets()
returns its union across all packages.
The set of references that matter are:
- types of operands to some MakeInterface instruction (as before)
- types of all exported package members
- all subcomponents of the above, recursively.
This is a conservative approximation to the set of types
whose methods may be called dynamically.
We define the owning package of a type as follows:
- the owner of a named type is the package in which it is defined;
- the owner of a pointer-to-named type is the owner of that named type;
- the owner of all other types is nil.
A package must include the method sets for all types that it
owns, and all subcomponents of that type that are not owned by
another package, recursively. Types with an owner appear in
exactly one package; types with no owner (such as struct{T})
may appear within multiple packages.
(A typical Go compiler would emit multiple copies of these
methods as weak symbols; a typical linker would eliminate
duplicates.)
Also:
- go/types/typemap: implement hash function for *Tuple.
- pointer: generate nodes/constraints for all of
ssa.Program.TypesWithMethodSets().
Add rtti.go regression test.
- Add API test of Package.TypesWithMethodSets().
- Set Function.Pkg to nil (again) for wrapper functions,
since these may be shared by many packages.
- Remove a redundant logging statement.
- Document that ssa CREATE phase is in fact sequential.
Fixesgolang/go#6605
R=gri
CC=golang-dev
https://golang.org/cl/14920056
Since rev 4c5f46cc7b9d, error messages no longer contain
"file:line:col: " prefixes, so applying stripLocation to them
is incorrect.
Also: add rationale comment to callgraph2.go test.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13888044
The previous CL made the assumption that Root is the first
node, which is false for programs that import "reflect".
Reverted to the previous way: an explicit root field.
Added regression test (callgraph2) via oracle.
R=crawshaw
TBR=crawshaw
CC=golang-dev
https://golang.org/cl/13967043
This CL is mostly a renaming s/json/serial/, abstracting the
oracle package away from any particular data syntax. (The
encoding/* machinery is very clean; clearly I should have
structured it this way from the outset.)
Supporting XML then becomes a one-liner in cmd/oracle/main.go.
Also: call MarshalIndent(), not Marshall() then Indent().
R=crawshaw
CC=golang-dev
https://golang.org/cl/13858046
(reflect.Value).Send
(reflect.Value).TrySend
(reflect.Value).Recv
(reflect.Value).TryRecv
(reflect.Type).ChanOf
(reflect.Type).In
(reflect.Type).Out
reflect.Indirect
reflect.MakeChan
Also:
- specialize genInvoke when the receiver is a reflect.Type under the
assumption that there's only one possible concrete type. This
makes all reflect.Type operations context-sensitive since the calls
are no longer dynamic.
- Rename all variables to match the actual parameter names used in
the reflect API.
- Add pointer.Config.Reflection flag
(exposed in oracle as --reflect, default false) to enable reflection.
It currently adds about 20% running time. I'll make it true after
the presolver is implemented.
- Simplified worklist datatype and solver main loop slightly
(~10% speed improvement).
- Use addLabel() utility to add a label to a PTS.
(Working on my 3 yr old 2x2GHz+4GB Mac vs 8x4GHz+24GB workstation,
one really notices the cost of pointer analysis.
Note to self: time to implement presolver.)
R=crawshaw
CC=golang-dev
https://golang.org/cl/13242062
The existing standalone Query function builds an importer, ssa.Program, oracle,
and query position, executes the query and returns the result.
For clients (such as Frederik Zipp's web-based github.com/fzipp/pythia tool)
that wish to load the program once and make several queries, we now expose
these as separate operations too. Here's a client, in pseudocode:
o := oracle.New(...)
for ... {
qpos := o.ParseQueryPos(...)
res := o.Query(mode, qpos)
print result
}
NB: this is a slight deoptimisation in the one-shot case since we have to
build the entire SSA program with debug info, not just the query package,
since we now don't know the query package at that time.
The 'exact' param to ParseQueryPos needs more thought since its
ideal value is a function of the query mode. This will do for now.
Details:
- expose Oracle type, New() func and Query() method.
- expose QueryPos type and ParseQueryPos func.
- improved package doc comment.
- un-exposed the "needs" bits.
- added test.
R=crawshaw
CC=frederik.zipp, golang-dev
https://golang.org/cl/13810043
Core:
reflect.TypeOf
reflect.ValueOf
reflect.Zero
reflect.Value.Interface
Maps:
(reflect.Value).MapIndex
(reflect.Value).MapKeys
(reflect.Value).SetMapIndex
(*reflect.rtype).Elem
(*reflect.rtype).Key
+ tests:
pointer/testdata/mapreflect.go.
oracle/testdata/src/main/reflection.go.
Interface objects (T, V...) have been renamed "tagged objects".
Abstraction: we model reflect.Value similar to
interface{}---as a pointer that points only to tagged
objects---but a reflect.Value may also point to an "indirect
tagged object", one in which the payload V is of type *T not T.
These are required because reflect.Values can hold lvalues,
e.g. when derived via Field() or Elem(), though we won't use
them till we get to structs and pointers.
Solving: each reflection intrinsic defines a new constraint
and resolution rule. Because of the nature of reflection,
generalizing across types, the resolution rules dynamically
create additional complex constraints during solving, where
previously only simple (copy) constraints were created.
This requires some solver changes:
The work done before the main solver loop (to attach new
constraints to the graph) is now done before each iteration,
in processNewConstraints.
Its loop over constraints is broken into two passes:
the first handles base (addr-of) constraints,
the second handles simple and complex constraints.
constraint.init() has been inlined. The only behaviour that
varies across constraints is ptr()
Sadly this will pessimize presolver optimisations, when we get
there; such is the price of reflection.
Objects: reflection intrinsics create objects (i.e. cause
memory allocations) with no SSA operation. We will represent
them as the cgnode of the instrinsic (e.g. reflect.New), so we
extend Labels and node.data to represent objects as a product
(not sum) of ssa.Value and cgnode and pull this out into its
own type, struct object. This simplifies a number of
invariants and saves space. The ntObject flag is now
represented by obj!=nil; the other flags are moved into
object.
cgnodes are now always recorded in objects/Labels for which it
is appropriate (all but those for globals, constants and the
shared contours for functions).
Also:
- Prepopulate the flattenMemo cache to consider reflect.Value
a fake pointer, not a struct.
- Improve accessors and documentation on type Label.
- @conctypes assertions renamed @types (since dyn. types needn't be concrete).
- add oracle 'describe' test on an interface (missing, an oversight).
R=crawshaw
CC=golang-dev
https://golang.org/cl/13418048
+ test.
Also:
- provide non-nil map to Importer.doImport0() to avoid a crash.
- reorganize oracle "needs" bits.
- reduce "needs" of 'freevars' and 'implements' queries by avoiding
ssa.Packages when types.Package suffices.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13421046
The previous notation (sans '#') now yields an error but is
"reserved for future use", e.g. to denote line/column offsets.
Will implement as needed.
R=r, crawshaw
CC=golang-dev
https://golang.org/cl/13526043
See json.go for interface specification.
Example usage:
% oracle -format=json -mode=callgraph code.google.com/p/go.tools/cmd/oracle
+ Tests, based on (small) golden files.
Overview:
Each <query>Result structure has been "lowered" so that all
but the most trivial logic in each display() function has
been moved to the main query.
Each one now has a toJSON method that populates a json.Result
struct. Though the <query>Result structs are similar to the
correponding JSON protocol, they're not close enough to be
used directly; for example, the former contain richer
semantic entities (token.Pos, ast.Expr, ssa.Value,
pointer.Pointer, etc) whereas JSON contains only their
printed forms using Go basic types.
The choices of what levels of abstractions the two sets of
structs should have is somewhat arbitrary. We may want
richer information in the JSON output in future.
Details:
- oracle.Main has been split into oracle.Query() and the
printing of the oracle.Result.
- the display() method no longer needs an *oracle param, only
a print function.
- callees: sort the result for determinism.
- callees: compute the union across all contexts.
- callers: sort the results for determinism.
- describe(package): fixed a bug in the predicate for method
accessibility: an unexported method defined in pkg A may
belong to a type defined in package B (via
embedding/promotion) and may thus be accessible to A. New
accessibleMethods() utility fixes this.
- describe(type): filter methods by accessibility.
- added tests of 'callgraph'.
- pointer: eliminated the 'caller CallGraphNode' parameter from
pointer.Context.Call callback since it was redundant w.r.t
site.Caller().
- added warning if CGO_ENABLED is unset.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13270045
Pro: no shell quotation needed.
Con: can't be parsed by (the perpetually useless) Scanf.
R=crawshaw, dgryski
CC=golang-dev
https://golang.org/cl/13441043
diff -u3: the 3 is redundant and an error on darwin; redundant and unnecessary on linux.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/13231044
+ Tests.
+ Emacs integration.
+ Emacs integration test.
+ very rudimentary Vim integration. Needs some love from a Vim user.
TODO (in follow-ups):
- More tests would be good.
We'll need to make the output order deterministic in more places.
- Documentation.
R=gri, crawshaw, dominik.honnef
CC=golang-dev
https://golang.org/cl/9502043
