This change sets ParserMode=AllErrors so that the parser is never
allowed to discard the AST and use a dummy one just because it saw too
many errors.
Also, change (*loader.Program).PathEnclosingInterval so that other
clients that forget to set this flag don't panic while calling
fset.File(f.Pos()).Base() on an ast.File f with no position info.
Change-Id: Ie544f169d367d2aa85426212b27063dc72e36fb1
Reviewed-on: https://go-review.googlesource.com/10290
Reviewed-by: Robert Griesemer <gri@golang.org>
This makes type-based queries faster and more robust because they
needn't invoke a C compiler, at the cost of worse results in/near .go
files that import "C". It's particularly important for "referrers"
since a refererrs query on a name from the standard library can load
every package in the workspace.
Fixes issue 10347
Change-Id: I2f65474ce963de5c0897ba67eeb26290dd449cf1
Reviewed-on: https://go-review.googlesource.com/8493
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The check for len(argv)==0 now only applies to these modes.
Also, more consistent variable naming.
Change-Id: I9adb6bebc819eb43d54ddf63c42d952671ce9236
Reviewed-on: https://go-review.googlesource.com/8244
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Features:
More robust: silently ignore type errors in modes that don't need
SSA form: describe, referrers, implements, freevars, description.
This makes the tool much more robust for everyday queries.
Less configuration: don't require a scope argument for all queries.
Only queries that do pointer analysis need it.
For the rest, the initial position is enough for
importQueryPackage to deduce the scope.
It now works for queries in GoFiles, TestGoFiles, or XTestGoFiles.
(It no longer works for ad-hoc main packages like
$GOROOT/src/net/http/triv.go)
More complete: "referrers" computes the scope automatically by
scanning the import graph of the entire workspace, using gorename's
refactor/importgraph package. This requires two passes at loading.
Faster: simplified start-up logic avoids unnecessary package loading
and SSA construction (a consequence of bad abstraction) in many
cases.
"callgraph": remove it. Unlike all the other commands it isn't
related to the current selection, and we have
golang.org/x/tools/cmdcallgraph now.
Internals:
Drop support for long-running clients (i.e., Pythia), since
godoc -analysis supports all the same features except "pointsto",
and precomputes all the results so latency is much lower.
Get rid of various unhelpful abstractions introduced to support
long-running clients. Expand out the set-up logic for each
subcommand. This is simpler, easier to read, and gives us more
control, at a small cost in duplication---the familiar story of
abstractions.
Discard PTA warnings. We weren't showing them (nor should we).
Split tests into separate directories (so that importgraph works).
Change-Id: I55d46b3ab33cdf7ac22436fcc2148fe04c901237
Reviewed-on: https://go-review.googlesource.com/8243
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Emacs integration:
- eliminate oracle minor mode
- in go-mode, bind F5, F6 to "describe", "referrers".
This reverts a previous policy decision but convenience matters too.
- don't insist on an analysis scope for modes that don't do PTA.
- don't hide the filename as "▶"; show the last 20 chars.
(Especially useful for "referrers" mode.)
- output postprocessing: don't get stuck in a loop if the output
is not as expected (e.g. when it includes a panic log).
referrers:
- show the matching lines (like grep does).
We do the I/O in parallel.
Change-Id: I86b18c1d3a4d9fa4242984cba62b314796669d8e
Reviewed-on: https://go-review.googlesource.com/8120
Reviewed-by: David Crawshaw <crawshaw@golang.org>
...since the zero value is more useful by far.
This is a breaking API change, obviously. (One or two tests in this
CL have intentional been left using the zero value, i.e., they now
load source.)
Change-Id: I42287bfcdb1afef8ee84e5eac12534dd0a1fd5d2
Reviewed-on: https://go-review.googlesource.com/5653
Reviewed-by: Robert Griesemer <gri@golang.org>
Before this change, many kinds of error would cause the loader to stop.
making it brittle when analyzing large codebases, as in "godoc -analysis".
This change moves operations that used to occur during
configuration---(*build.Context).Import, loading, and parsing of
initial packages---into the Load call, and ensures that all failures
during Loading are reported at the end so that the maximum amount of
progress is made.
Also: redesign the tests and add many new cases.
Change-Id: Ia8cd99416af7c5d4a5fe133908adfa83676d401f
Reviewed-on: https://go-review.googlesource.com/3626
Reviewed-by: Robert Griesemer <gri@golang.org>
The whicherrs query mode takes the position of an error and returns the set of constants, globals and types visible from within the scope of the error being queried.
It is meant to be used as a shortcut to find out which errors should be handled for a given functions call.
LGTM=adonovan
R=golang-codereviews, dominik.honnef, adonovan
CC=golang-codereviews
https://golang.org/cl/167420043
Rewrite performed with this command:
sed -i '' 's_code.google.com/p/go\._golang.org/x/_g' \
$(grep -lr 'code.google.com/p/go.' *)
LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/170920043
(godoc is excluded from this CL since it will continue to use
/src/pkg in its URL namespace, making the necessary cleanup
more subtle.)
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/141770043
This is to avoid an internal error in pointer analysis from
bringing down a long-lived application such as godoc.
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/68930046
If a -pos argument is specified, a 'callgraph' query reports only the
functions within the query package. This produces a far more manageable
amount of information, and because we don't need to package-qualify the
names, the result is easier to read.
Added tests:
- callgraph query with/without -pos
(The test driver was extended to allow "nopos" queries.)
- callers and callees queries don't return wrappers
Also, in go/callgraph:
- (*Node).String, (*Edge).String
- (*Graph).DeleteSyntheticNodes eliminates synthetic wrapper functions,
preserving topology. Used in all four oracle "call*" queries.
- (*Graph).DeleteNode
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/66240044
Previously, each {Indirect,}Query would return a set of Pointers, one per context; now it returns (at most) one Pointer combining information from all contexts.
The old API was more faithful to the implementation concepts, but the analysis is not sufficiently context-sensitive that it makes sense: all existing clients simply throw away the context information---so now we do that for them.
(I may remove the context-sensitivity from the callgraph too, but I'll benchmark that first to see if it reduces precision.)
LGTM=crawshaw
R=crawshaw
CC=golang-codereviews
https://golang.org/cl/66130044
Previously, each word could be a package import path or a
comma-separated list of *.go file names. Now, if the
first word ends with ".go", all words are assumed to be
Go source files. This makes it impossible to specify
two ad-hoc packages from source files, but no-one needs that.
FromArgs also takes a boolean indicating whether tests
are wanted or not.
Also: ssadump: add -test flag to set that boolean.
For the oracle it's always true.
LGTM=gri
R=gri
CC=golang-codereviews
https://golang.org/cl/61470047
Was: Now:
call.Graph callgraph.Graph
call.GraphNode callgraph.Node
call.Edge callgraph.Edge
Though call.Graph was cute, the original naming was a mistake:
'call' is too useful a var name to waste on a package.
R=gri, crawshaw
CC=golang-codereviews
https://golang.org/cl/53190043
The Importer type has been replaced with Config and Program.
Clients populate a Config, directly or more usually via
convenience functions. They then call its Load() method to do
all of the typechecking and transitive-closure computation.
ssa.NewProgram and ssa.CreatePackages have been fused into
ssa.Create, which now cannot fail, since (*Config).Load()
reports all type errors.
Also:
- The addition of an ssa.GlobalDebug builder mode flag
eliminates a loop-over-packages repeated in many clients.
- PackageInfo.Err flag unexported. Clients never see bad infos now.
- cmd/ssadump: now only looks for func "main" in package "main".
- importsOf deleted, was dead code.
STILL TODO:
- ParseFile seems like API creep (though it's convenient)
and CreateFromFiles is dangerous (w.r.t. FileSet identity).
Need to think more...
- the need for clients to rely on elementwise correspondence
of Config.CreatePkgs and Program.Created is a little sad.
- The command-line interface has not changed.
That will happen in a follow-up.
r recommends using a repeated flag: -package p -package q ...
R=gri
CC=axwalk, frederik.zipp, golang-codereviews
https://golang.org/cl/49530047
Packages were not being created for all types.Packages,
specifically, indirectly imported packages were missing.
(*Program).CreatePackages now iterates over the type-checker's
package map too.
Also: removed all concurrency from importer. I think it was
incorrect (and hard to fix).
Also: change LoadInitialPackages so that all named packages
are loaded from source. This happens regardless of whether
GCImporter is used to satisfy imports.
Details:
- importer.Config.SourceImports flag determines whether to
load all packages from *.go source.
(Before, this was indicated by Config.Build != nil.)
- importer.Config.Build field effectively defaults to
&go/build.Default. A zero importer.Config is now usable.
- importer.Importer.Config field is now exported.
- LoadPackage renamed to ImportPackage since the resulting
packages may come from GCImporter (and be incomplete).
- doImport and ImportPackage fused.
Fixesgolang/go#7028
R=gri, axwalk
CC=golang-codereviews
https://golang.org/cl/48770043
Command set:
- what: an extremely fast query that parses a single
file and returns the AST stack, package name and the
set of query modes that apply to the current selection.
Intended for GUI tools that need to grey out UI elements.
- definition: shows the definition of an identifier.
- pointsto: the PTA features of 'describe' have been split
out into their own command.
- describe: with PTA stripped out, the cost is now bounded by
type checking.
Performance:
- The importer.Config.TypeCheckFuncBodies predicate supports
setting the 'IgnoreFuncBodies' typechecker flag on a
per-package basis. This means we can load dependencies from
source more quickly if we only need exported types.
(We avoid gcimport data because it may be absent or stale.)
This also means we can run type-based queries on packages
that aren't part of the pointer analysis scope. (Yay.)
- Modes that require only type analysis of the query package
run a "what" query first, and restrict their analysis scope
to just that package and its dependencies (sans func
bodies), making them much faster.
- We call newOracle not oracle.New in Query, so that the
'needs' bitset isn't ignored (oops!). This makes the
non-PTA queries faster.
Also:
- removed vestigial timers junk.
- pos.go: existing position utilties split out into own file.
Added parsePosFlag utility.
- numerous cosmetic tweaks.
+ very basic tests.
To do in follow-ups:
- sophisticated editor integration of "what".
- better tests.
- refactoring of control flow as described in comment.
- changes to "implements", "describe" commands.
- update design doc + user manual.
R=crawshaw, dominik.honnef
CC=golang-dev, gri
https://golang.org/cl/40630043
This allows us to run/analyze multiple tests.
Also it causes the production code packages to be properly initialized.
Also:
- cmd/ssadump: improved usage message (add example;
incorporate LoadInitialPackages usage; explain how -run
finds main).
- pointer, oracle, ssa/interp: use CreateTestMainPackage.
- ssa/builder.go: remove 'rundefers' instruction from package init,
which no longer uses 'defer'.
R=gri
CC=golang-dev
https://golang.org/cl/15920047
Details:
- Warnings are reported as values in Result, not a callback in Config.
- remove TODO to eliminate Print callback. It's better than the alternative.
- remove unused Config.root field.
- hang Result off analysis object (impl. detail)
- reword TODO.
R=crawshaw
CC=golang-dev
https://golang.org/cl/14128043
Also: pointer.Analyze now returns a pointer.Result object,
containing the callgraph and the results of ssa.Value queries.
The oracle has been updated to use the new call and pointer APIs.
R=crawshaw, gri
CC=golang-dev
https://golang.org/cl/13915043
e.g. "oracle callgraph <package>"
Also: simplified error handling.
Eliminated oracle.errorf because it prepends "file:line:col: "
to the error message so the main function can't safely prepend "Error: ".
The position wasn't interesting though: it was just -pos, more or less.
R=crawshaw, dominik.honnef, r
CC=golang-dev
https://golang.org/cl/13864044
This CL is mostly a renaming s/json/serial/, abstracting the
oracle package away from any particular data syntax. (The
encoding/* machinery is very clean; clearly I should have
structured it this way from the outset.)
Supporting XML then becomes a one-liner in cmd/oracle/main.go.
Also: call MarshalIndent(), not Marshall() then Indent().
R=crawshaw
CC=golang-dev
https://golang.org/cl/13858046
(reflect.Value).Send
(reflect.Value).TrySend
(reflect.Value).Recv
(reflect.Value).TryRecv
(reflect.Type).ChanOf
(reflect.Type).In
(reflect.Type).Out
reflect.Indirect
reflect.MakeChan
Also:
- specialize genInvoke when the receiver is a reflect.Type under the
assumption that there's only one possible concrete type. This
makes all reflect.Type operations context-sensitive since the calls
are no longer dynamic.
- Rename all variables to match the actual parameter names used in
the reflect API.
- Add pointer.Config.Reflection flag
(exposed in oracle as --reflect, default false) to enable reflection.
It currently adds about 20% running time. I'll make it true after
the presolver is implemented.
- Simplified worklist datatype and solver main loop slightly
(~10% speed improvement).
- Use addLabel() utility to add a label to a PTS.
(Working on my 3 yr old 2x2GHz+4GB Mac vs 8x4GHz+24GB workstation,
one really notices the cost of pointer analysis.
Note to self: time to implement presolver.)
R=crawshaw
CC=golang-dev
https://golang.org/cl/13242062
The existing standalone Query function builds an importer, ssa.Program, oracle,
and query position, executes the query and returns the result.
For clients (such as Frederik Zipp's web-based github.com/fzipp/pythia tool)
that wish to load the program once and make several queries, we now expose
these as separate operations too. Here's a client, in pseudocode:
o := oracle.New(...)
for ... {
qpos := o.ParseQueryPos(...)
res := o.Query(mode, qpos)
print result
}
NB: this is a slight deoptimisation in the one-shot case since we have to
build the entire SSA program with debug info, not just the query package,
since we now don't know the query package at that time.
The 'exact' param to ParseQueryPos needs more thought since its
ideal value is a function of the query mode. This will do for now.
Details:
- expose Oracle type, New() func and Query() method.
- expose QueryPos type and ParseQueryPos func.
- improved package doc comment.
- un-exposed the "needs" bits.
- added test.
R=crawshaw
CC=frederik.zipp, golang-dev
https://golang.org/cl/13810043
Full help is only displayed when -help is requested;
CLI usage errors just remind the the user of this flag.
R=r, crawshaw
CC=golang-dev
https://golang.org/cl/13523048
Every one of the oracle's query modes needs to have typed ASTs
available, at least transiently, so that the -pos flag can be
interpreted. (The only mode that doesn't need the -pos flag
is callgraph, but that needs PTA.) So we hard-code it to true.
This change fixes a bug in the 'implements' query that causes
-pos parsing to fail. (This wasn't exposed by the tests
because they are degenerate in that the query always occurs in
the main package, which is specified ad-hoc, i.e. as a source
file not an import path. That's unfortunate, but this
change renders the distinction uninteresting in future.)
R=crawshaw, dominik.honnef
CC=golang-dev
https://golang.org/cl/13334050
+ test.
Also:
- provide non-nil map to Importer.doImport0() to avoid a crash.
- reorganize oracle "needs" bits.
- reduce "needs" of 'freevars' and 'implements' queries by avoiding
ssa.Packages when types.Package suffices.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13421046
Motivation: pointer analysis tools (like the oracle) want the
user to specify a set of initial packages, like 'go test'.
This change enables the user to specify a set of packages on
the command line using importer.LoadInitialPackages(args).
Each argument is interpreted as either:
- a comma-separated list of *.go source files together
comprising one non-importable ad-hoc package.
e.g. "src/pkg/net/http/triv.go" gives us [main].
- an import path, denoting both the imported package
and its non-importable external test package, if any.
e.g. "fmt" gives us [fmt, fmt_test].
Current type-checker limitations mean that only the first
import path may contribute tests: multiple packages augmented
by *_test.go files could create import cycles, which 'go test'
avoids by building a separate executable for each one.
That approach is less attractive for static analysis.
Details: (many files touched, but importer.go is the crux)
importer:
- PackageInfo.Importable boolean indicates whether
package is importable.
- un-expose Importer.Packages; expose AllPackages() instead.
- CreatePackageFromArgs has become LoadInitialPackages.
- imports() moved to util.go, renamed importsOf().
- InitialPackagesUsage usage message exported to clients.
- the package name for ad-hoc packages now comes from the
'package' decl, not "main".
ssa.Program:
- added CreatePackages() method
- PackagesByPath un-exposed, renamed 'imported'.
- expose AllPackages and ImportedPackage accessors.
oracle:
- describe: explain and workaround a go/types bug.
Misc:
- Removed various unnecessary error.Error() calls in Printf args.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13579043
The previous notation (sans '#') now yields an error but is
"reserved for future use", e.g. to denote line/column offsets.
Will implement as needed.
R=r, crawshaw
CC=golang-dev
https://golang.org/cl/13526043
1. ParseFiles (in util.go) parses each file in its own goroutine.
2. (*Importer).LoadPackage asynchronously prefetches the
import graph by scanning the imports of each loaded package
and calling LoadPackage on each one.
LoadPackage is now thread-safe and idempotent: it uses a
condition variable per package; the first goroutine to
request a package becomes responsible for loading it and
broadcasts to the others (waiting) when it becomes ready.
ssadump runs 34% faster when loading the oracle.
Also, refactorings:
- delete SourceLoader mechanism; just expose go/build.Context directly.
- CreateSourcePackage now also returns an error directly,
rather than via PackageInfo.Err, since every client wants that.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13509045
See json.go for interface specification.
Example usage:
% oracle -format=json -mode=callgraph code.google.com/p/go.tools/cmd/oracle
+ Tests, based on (small) golden files.
Overview:
Each <query>Result structure has been "lowered" so that all
but the most trivial logic in each display() function has
been moved to the main query.
Each one now has a toJSON method that populates a json.Result
struct. Though the <query>Result structs are similar to the
correponding JSON protocol, they're not close enough to be
used directly; for example, the former contain richer
semantic entities (token.Pos, ast.Expr, ssa.Value,
pointer.Pointer, etc) whereas JSON contains only their
printed forms using Go basic types.
The choices of what levels of abstractions the two sets of
structs should have is somewhat arbitrary. We may want
richer information in the JSON output in future.
Details:
- oracle.Main has been split into oracle.Query() and the
printing of the oracle.Result.
- the display() method no longer needs an *oracle param, only
a print function.
- callees: sort the result for determinism.
- callees: compute the union across all contexts.
- callers: sort the results for determinism.
- describe(package): fixed a bug in the predicate for method
accessibility: an unexported method defined in pkg A may
belong to a type defined in package B (via
embedding/promotion) and may thus be accessible to A. New
accessibleMethods() utility fixes this.
- describe(type): filter methods by accessibility.
- added tests of 'callgraph'.
- pointer: eliminated the 'caller CallGraphNode' parameter from
pointer.Context.Call callback since it was redundant w.r.t
site.Caller().
- added warning if CGO_ENABLED is unset.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13270045
Pro: no shell quotation needed.
Con: can't be parsed by (the perpetually useless) Scanf.
R=crawshaw, dgryski
CC=golang-dev
https://golang.org/cl/13441043
+ Tests.
+ Emacs integration.
+ Emacs integration test.
+ very rudimentary Vim integration. Needs some love from a Vim user.
TODO (in follow-ups):
- More tests would be good.
We'll need to make the output order deterministic in more places.
- Documentation.
R=gri, crawshaw, dominik.honnef
CC=golang-dev
https://golang.org/cl/9502043
