1
0
mirror of https://github.com/golang/go synced 2024-11-19 00:24:41 -07:00
Commit Graph

25 Commits

Author SHA1 Message Date
Alan Donovan
8b9d1fd507 go.tools/oracle: implements: now shows whole-program implements relation for selected type.
(Previously it showed the implements relation for all types within the query package.)

R=crawshaw
CC=golang-dev
https://golang.org/cl/42000043
2013-12-13 18:00:55 -05:00
Alan Donovan
f119874203 go.tools/oracle: improvements to command set and performance.
Command set:
- what: an extremely fast query that parses a single
  file and returns the AST stack, package name and the
  set of query modes that apply to the current selection.
  Intended for GUI tools that need to grey out UI elements.
- definition: shows the definition of an identifier.
- pointsto: the PTA features of 'describe' have been split
  out into their own command.
- describe: with PTA stripped out, the cost is now bounded by
  type checking.

Performance:
- The importer.Config.TypeCheckFuncBodies predicate supports
  setting the 'IgnoreFuncBodies' typechecker flag on a
  per-package basis.  This means we can load dependencies from
  source more quickly if we only need exported types.
  (We avoid gcimport data because it may be absent or stale.)
  This also means we can run type-based queries on packages
  that aren't part of the pointer analysis scope. (Yay.)
- Modes that require only type analysis of the query package
  run a "what" query first, and restrict their analysis scope
  to just that package and its dependencies (sans func
  bodies), making them much faster.
- We call newOracle not oracle.New in Query, so that the
  'needs' bitset isn't ignored (oops!).  This makes the
  non-PTA queries faster.

Also:
- removed vestigial timers junk.
- pos.go: existing position utilties split out into own file.
  Added parsePosFlag utility.
- numerous cosmetic tweaks.

+ very basic tests.

To do in follow-ups:
- sophisticated editor integration of "what".
- better tests.
- refactoring of control flow as described in comment.
- changes to "implements", "describe" commands.
- update design doc + user manual.

R=crawshaw, dominik.honnef
CC=golang-dev, gri
https://golang.org/cl/40630043
2013-12-13 10:04:55 -05:00
Alan Donovan
fb3d862cae go.tools/importer: move PathEnclosingInterval to package astutil.
R=crawshaw
CC=golang-dev
https://golang.org/cl/38650044
2013-12-09 09:36:29 -05:00
Alan Donovan
9c112540f6 go.tools/oracle: use SelectionString when printing methods.
R=gri, crawshaw, gri
CC=golang-dev
https://golang.org/cl/26900043
2013-11-15 12:35:11 -05:00
Alan Donovan
f488a2c4f5 go.tools/oracle: use TypeString, ObjectString to print relative (unqualified) names when a package is implied by the context.
+ a couple more tests.

R=gri, crawshaw
CC=golang-dev
https://golang.org/cl/26370046
2013-11-15 09:22:16 -05:00
Kamil Kisiel
e59751ce56 oracle: Fix documentation references to importer package functions.
R=golang-dev, adonovan
CC=golang-dev
https://golang.org/cl/21230043
2013-11-07 16:47:35 -05:00
Alan Donovan
8636f40baf go.tools/ssa: CreateTestMainPackage: synthesize test driver as a package ("testmain") not 'main' function.
This allows us to run/analyze multiple tests.
Also it causes the production code packages to be properly initialized.

Also:
- cmd/ssadump: improved usage message (add example;
  incorporate LoadInitialPackages usage; explain how -run
  finds main).
- pointer, oracle, ssa/interp: use CreateTestMainPackage.
- ssa/builder.go: remove 'rundefers' instruction from package init,
  which no longer uses 'defer'.

R=gri
CC=golang-dev
https://golang.org/cl/15920047
2013-10-23 18:07:53 -04:00
Alan Donovan
06c4192423 go.tools/pointer: minor API simplifications.
Details:
- Warnings are reported as values in Result, not a callback in Config.
- remove TODO to eliminate Print callback.  It's better than the alternative.
- remove unused Config.root field.
- hang Result off analysis object (impl. detail)
- reword TODO.

R=crawshaw
CC=golang-dev
https://golang.org/cl/14128043
2013-09-30 12:39:54 -04:00
Alan Donovan
785cfaa938 go.tools/pointer: use new callgraph API.
Also: pointer.Analyze now returns a pointer.Result object,
containing the callgraph and the results of ssa.Value queries.

The oracle has been updated to use the new call and pointer APIs.

R=crawshaw, gri
CC=golang-dev
https://golang.org/cl/13915043
2013-09-25 17:17:42 -04:00
Alan Donovan
3a4c0462d8 go.tools/oracle: change -mode argument into subcommand.
e.g. "oracle callgraph <package>"

Also: simplified error handling.
Eliminated oracle.errorf because it prepends "file:line:col: "
to the error message so the main function can't safely prepend "Error: ".
The position wasn't interesting though: it was just -pos, more or less.

R=crawshaw, dominik.honnef, r
CC=golang-dev
https://golang.org/cl/13864044
2013-09-25 14:34:39 -04:00
Alan Donovan
37f76edde8 go.tools/oracle: support -format=xml (for Eclipse)
This CL is mostly a renaming s/json/serial/, abstracting the
oracle package away from any particular data syntax.  (The
encoding/* machinery is very clean; clearly I should have
structured it this way from the outset.)

Supporting XML then becomes a one-liner in cmd/oracle/main.go.

Also: call MarshalIndent(), not Marshall() then Indent().

R=crawshaw
CC=golang-dev
https://golang.org/cl/13858046
2013-09-24 15:08:14 -04:00
Alan Donovan
3371b79a96 go.tools/pointer: reflect, part 2: channels.
(reflect.Value).Send
        (reflect.Value).TrySend
        (reflect.Value).Recv
        (reflect.Value).TryRecv
        (reflect.Type).ChanOf
        (reflect.Type).In
        (reflect.Type).Out
        reflect.Indirect
        reflect.MakeChan

Also:
- specialize genInvoke when the receiver is a reflect.Type under the
  assumption that there's only one possible concrete type.  This
  makes all reflect.Type operations context-sensitive since the calls
  are no longer dynamic.
- Rename all variables to match the actual parameter names used in
  the reflect API.
- Add pointer.Config.Reflection flag
  (exposed in oracle as --reflect, default false) to enable reflection.
  It currently adds about 20% running time.  I'll make it true after
  the presolver is implemented.
- Simplified worklist datatype and solver main loop slightly
  (~10% speed improvement).
- Use addLabel() utility to add a label to a PTS.

(Working on my 3 yr old 2x2GHz+4GB Mac vs 8x4GHz+24GB workstation,
one really notices the cost of pointer analysis.
Note to self: time to implement presolver.)

R=crawshaw
CC=golang-dev
https://golang.org/cl/13242062
2013-09-23 16:13:01 -04:00
Alan Donovan
25a0cc4bfd go.tools/oracle: refactor Oracle API to allow repeated queries on same scope.
The existing standalone Query function builds an importer, ssa.Program, oracle,
and query position, executes the query and returns the result.
For clients (such as Frederik Zipp's web-based github.com/fzipp/pythia tool)
that wish to load the program once and make several queries, we now expose
these as separate operations too.  Here's a client, in pseudocode:

        o := oracle.New(...)
        for ... {
                qpos := o.ParseQueryPos(...)
                res := o.Query(mode, qpos)
                print result
        }

NB: this is a slight deoptimisation in the one-shot case since we have to
build the entire SSA program with debug info, not just the query package,
since we now don't know the query package at that time.

The 'exact' param to ParseQueryPos needs more thought since its
ideal value is a function of the query mode.  This will do for now.

Details:
- expose Oracle type, New() func and Query() method.
- expose QueryPos type and ParseQueryPos func.
- improved package doc comment.
- un-exposed the "needs" bits.
- added test.

R=crawshaw
CC=frederik.zipp, golang-dev
https://golang.org/cl/13810043
2013-09-23 15:02:18 -04:00
Alan Donovan
0c4a02a8c4 go.tools/oracle: improve usage messages.
Full help is only displayed when -help is requested;
CLI usage errors just remind the the user of this flag.

R=r, crawshaw
CC=golang-dev
https://golang.org/cl/13523048
2013-09-20 11:35:00 -04:00
Alan Donovan
d38c09ed22 go.tools/oracle: set AllASTs=true always, and simplify.
Every one of the oracle's query modes needs to have typed ASTs
available, at least transiently, so that the -pos flag can be
interpreted.  (The only mode that doesn't need the -pos flag
is callgraph, but that needs PTA.)  So we hard-code it to true.

This change fixes a bug in the 'implements' query that causes
-pos parsing to fail.  (This wasn't exposed by the tests
because they are degenerate in that the query always occurs in
the main package, which is specified ad-hoc, i.e. as a source
file not an import path.  That's unfortunate, but this
change renders the distinction uninteresting in future.)

R=crawshaw, dominik.honnef
CC=golang-dev
https://golang.org/cl/13334050
2013-09-12 10:55:24 -04:00
Alan Donovan
0725e5a5b3 go.tools/oracle: new query 'referrers' returns all references to an identifier.
+ test.

Also:
- provide non-nil map to Importer.doImport0() to avoid a crash.
- reorganize oracle "needs" bits.
- reduce "needs" of 'freevars' and 'implements' queries by avoiding
  ssa.Packages when types.Package suffices.

R=crawshaw
CC=golang-dev
https://golang.org/cl/13421046
2013-09-10 14:11:42 -04:00
Alan Donovan
c6e88b1b2a go.tools/cmd/oracle: use -pos=file:#start,#end syntax to indicate half-open [start,end) extent of byte offsets.
Also: improve help message.

R=r, crawshaw, rsc
CC=golang-dev
https://golang.org/cl/13593043
2013-09-08 22:10:11 -04:00
Alan Donovan
3f2f9a7e70 go.tools/importer: generalize command-line syntax.
Motivation: pointer analysis tools (like the oracle) want the
user to specify a set of initial packages, like 'go test'.
This change enables the user to specify a set of packages on
the command line using importer.LoadInitialPackages(args).

Each argument is interpreted as either:
- a comma-separated list of *.go source files together
  comprising one non-importable ad-hoc package.
  e.g. "src/pkg/net/http/triv.go" gives us [main].
- an import path, denoting both the imported package
  and its non-importable external test package, if any.
  e.g. "fmt" gives us [fmt, fmt_test].

Current type-checker limitations mean that only the first
import path may contribute tests: multiple packages augmented
by *_test.go files could create import cycles, which 'go test'
avoids by building a separate executable for each one.
That approach is less attractive for static analysis.

Details:  (many files touched, but importer.go is the crux)

importer:
- PackageInfo.Importable boolean indicates whether
  package is importable.
- un-expose Importer.Packages; expose AllPackages() instead.
- CreatePackageFromArgs has become LoadInitialPackages.
- imports() moved to util.go, renamed importsOf().
- InitialPackagesUsage usage message exported to clients.
- the package name for ad-hoc packages now comes from the
  'package' decl, not "main".

ssa.Program:
- added CreatePackages() method
- PackagesByPath un-exposed, renamed 'imported'.
- expose AllPackages and ImportedPackage accessors.

oracle:
- describe: explain and workaround a go/types bug.

Misc:
- Removed various unnecessary error.Error() calls in Printf args.

R=crawshaw
CC=golang-dev
https://golang.org/cl/13579043
2013-09-06 18:13:57 -04:00
Alan Donovan
f9e325b575 go.tools/oracle: change notation for byte offsets to "-pos=file.go:#123-#456"
The previous notation (sans '#') now yields an error but is
"reserved for future use", e.g. to denote line/column offsets.
Will implement as needed.

R=r, crawshaw
CC=golang-dev
https://golang.org/cl/13526043
2013-09-04 14:35:24 -04:00
Alan Donovan
e2921e188a go.tools/importer: make loading/parsing concurrent.
1. ParseFiles (in util.go) parses each file in its own goroutine.

2. (*Importer).LoadPackage asynchronously prefetches the
   import graph by scanning the imports of each loaded package
   and calling LoadPackage on each one.

   LoadPackage is now thread-safe and idempotent: it uses a
   condition variable per package; the first goroutine to
   request a package becomes responsible for loading it and
   broadcasts to the others (waiting) when it becomes ready.

ssadump runs 34% faster when loading the oracle.

Also, refactorings:
- delete SourceLoader mechanism; just expose go/build.Context directly.
- CreateSourcePackage now also returns an error directly,
  rather than via PackageInfo.Err, since every client wants that.

R=crawshaw
CC=golang-dev
https://golang.org/cl/13509045
2013-09-04 13:15:49 -04:00
Alan Donovan
d2cdbefbfc go.tools/oracle: add option to output results in JSON syntax.
See json.go for interface specification.

Example usage:
% oracle -format=json -mode=callgraph code.google.com/p/go.tools/cmd/oracle

+ Tests, based on (small) golden files.

Overview:
  Each <query>Result structure has been "lowered" so that all
  but the most trivial logic in each display() function has
  been moved to the main query.

  Each one now has a toJSON method that populates a json.Result
  struct.  Though the <query>Result structs are similar to the
  correponding JSON protocol, they're not close enough to be
  used directly; for example, the former contain richer
  semantic entities (token.Pos, ast.Expr, ssa.Value,
  pointer.Pointer, etc) whereas JSON contains only their
  printed forms using Go basic types.

  The choices of what levels of abstractions the two sets of
  structs should have is somewhat arbitrary.  We may want
  richer information in the JSON output in future.

Details:
- oracle.Main has been split into oracle.Query() and the
  printing of the oracle.Result.
- the display() method no longer needs an *oracle param, only
  a print function.
- callees: sort the result for determinism.
- callees: compute the union across all contexts.
- callers: sort the results for determinism.
- describe(package): fixed a bug in the predicate for method
  accessibility: an unexported method defined in pkg A may
  belong to a type defined in package B (via
  embedding/promotion) and may thus be accessible to A.  New
  accessibleMethods() utility fixes this.
- describe(type): filter methods by accessibility.
- added tests of 'callgraph'.
- pointer: eliminated the 'caller CallGraphNode' parameter from
  pointer.Context.Call callback since it was redundant w.r.t
  site.Caller().
- added warning if CGO_ENABLED is unset.

R=crawshaw
CC=golang-dev
https://golang.org/cl/13270045
2013-09-03 15:29:02 -04:00
Alan Donovan
0126405cad go.tools/oracle: change -pos flag syntax from "file pos-pos" to file:pos-pos.
Pro: no shell quotation needed.
Con: can't be parsed by (the perpetually useless) Scanf.

R=crawshaw, dgryski
CC=golang-dev
https://golang.org/cl/13441043
2013-09-03 10:58:58 -04:00
Alan Donovan
a483497cf1 go.tools/oracle: address reviewer suggestions from CL 9502043
(I made these changes in the wrong workspace.)

R=crawshaw
CC=golang-dev
https://golang.org/cl/13328044
2013-08-29 21:36:59 -04:00
Alan Donovan
713699d8ad go.tools: add copyright messages to source files.
R=r
CC=golang-dev
https://golang.org/cl/13305043
2013-08-27 18:49:13 -04:00
Alan Donovan
e08d89f3ed go.tools/oracle: an oracle that answers questions about Go source code.
+ Tests.
+ Emacs integration.
+ Emacs integration test.
+ very rudimentary Vim integration.  Needs some love from a Vim user.

TODO (in follow-ups):
- More tests would be good.
  We'll need to make the output order deterministic in more places.
- Documentation.

R=gri, crawshaw, dominik.honnef
CC=golang-dev
https://golang.org/cl/9502043
2013-08-27 17:58:26 -04:00