Command set:
- what: an extremely fast query that parses a single
file and returns the AST stack, package name and the
set of query modes that apply to the current selection.
Intended for GUI tools that need to grey out UI elements.
- definition: shows the definition of an identifier.
- pointsto: the PTA features of 'describe' have been split
out into their own command.
- describe: with PTA stripped out, the cost is now bounded by
type checking.
Performance:
- The importer.Config.TypeCheckFuncBodies predicate supports
setting the 'IgnoreFuncBodies' typechecker flag on a
per-package basis. This means we can load dependencies from
source more quickly if we only need exported types.
(We avoid gcimport data because it may be absent or stale.)
This also means we can run type-based queries on packages
that aren't part of the pointer analysis scope. (Yay.)
- Modes that require only type analysis of the query package
run a "what" query first, and restrict their analysis scope
to just that package and its dependencies (sans func
bodies), making them much faster.
- We call newOracle not oracle.New in Query, so that the
'needs' bitset isn't ignored (oops!). This makes the
non-PTA queries faster.
Also:
- removed vestigial timers junk.
- pos.go: existing position utilties split out into own file.
Added parsePosFlag utility.
- numerous cosmetic tweaks.
+ very basic tests.
To do in follow-ups:
- sophisticated editor integration of "what".
- better tests.
- refactoring of control flow as described in comment.
- changes to "implements", "describe" commands.
- update design doc + user manual.
R=crawshaw, dominik.honnef
CC=golang-dev, gri
https://golang.org/cl/40630043
Using a type containing a sync type directly
in a function call (whether as a receiver,
a param, or a return value) is an easy way
to accidentally copy a lock or other sync primitive.
Check for it.
The test as implemented does not provide 100%
coverage; see the discussion near the bottom of
testdata/copylock.go for shortcomings.
Fixesgolang/go#6729.
R=adg, r, dsymonds
CC=golang-dev
https://golang.org/cl/23420043
Split the profile parsing code out of cmd/cover into
a reusable package, to support third-party coverage
tools.
R=golang-dev, r, adg
CC=golang-dev
https://golang.org/cl/36050045
Add explicit options to Corpus to control search indexing of documentation, Go source code, and full-text.
R=bradfitz, r
CC=golang-dev
https://golang.org/cl/24190043
- fixed a couple of TODOs
- various cleanups along the way
- adjusted clients
Once submitted, clients of go/types that don't explicitly
specify Config.Import will need to add the extra import:
import _ "code.google.com/p/go.tools/go/gcimporter"
to install the default (gc) importer in go/types.
R=adonovan, gri
CC=golang-dev
https://golang.org/cl/26390043
Performance increase is much smaller than expected;
need to investigate. Possibly, repeatedly (for each
line) accessing file sets from multiple goroutines
(in the scanner) is a bottle neck; or perhaps i/o.
R=adonovan
CC=golang-dev
https://golang.org/cl/23090043
The symbolic names such as NOSPLIT for annotations on the TEXT
directive appeared after vet started checking .s files. This CL tweaks
the regular expression to allow CAPITALS and the symbols | and +
as well as digits in that field, and interprets NOSPLIT as equivalent
to 7 in that field. All magic.
Fixesgolang/go#6695
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/18700044
This CL makes gotype usable again. Removed -r
(recursive) mode; use go/build to determine
the correct set of Go files when processing
a directory. The -v (verbose) mode now prints
some basic stats (duration, number of files,
lines, and lines/s).
Thoroughly restructured the code.
Applying gotype -v -a . to the go/types directory:
128.94141ms (40 files, 12008 lines, 93127 lines/s)
On a 2.8 GHz Quad-Core Intel Xeon, 800 MHz DDR2 FB-DIMM,
with go/types built with the (interal) debug flag set to
false. There's still quite a bit of room for performance
improvement in all parts of the code since no tuning has
been done.
R=golang-dev, adonovan
CC=golang-dev
https://golang.org/cl/19930043
This allows us to run/analyze multiple tests.
Also it causes the production code packages to be properly initialized.
Also:
- cmd/ssadump: improved usage message (add example;
incorporate LoadInitialPackages usage; explain how -run
finds main).
- pointer, oracle, ssa/interp: use CreateTestMainPackage.
- ssa/builder.go: remove 'rundefers' instruction from package init,
which no longer uses 'defer'.
R=gri
CC=golang-dev
https://golang.org/cl/15920047
A function such as this:
func one() (x int) {
defer func() { recover() }()
x = 1
panic("return")
}
that combines named return parameters (NRPs) with deferred calls
that call recover, may return non-zero values despite the
fact it doesn't even contain a return statement. (!)
This requires a change to the SSA API: all functions'
control-flow graphs now have a second entry point, called
Recover, which is the block at which control flow resumes
after a recovered panic. The Recover block simply loads the
NRPs and returns them.
As an optimization, most functions don't need a Recover block,
so it is omitted. In fact it is only needed for functions that
have NRPs and defer a call to another function that _may_ call
recover.
Dataflow analysis of SSA now requires extra work, since every
may-panic instruction has an implicit control-flow edge to
the Recover block. The only dataflow analysis so far implemented
is SSA renaming, for which we make the following simplifying
assumption: the Recover block only loads the NRPs and returns.
This means we don't really need to analyze it, we can just
skip the "lifting" of such NRPs. We also special-case the Recover
block in the dominance computation.
Rejected alternative approaches:
- Specifying a Recover block for every defer instruction (like a
traditional exception handler).
This seemed like excessive generality, since Go programs
only need the same degenerate form of Recover block.
- Adding an instruction to set the Recover block immediately
after the named return values are set up, so that dominance
can be computed without special-casing.
This didn't seem worth the effort.
Interpreter:
- This CL completely reimplements the panic/recover/
defer logic in the interpreter. It's clearer and simpler
and closer to the model in the spec.
- Some runtime panic messages have been changed to be closer
to gc's, since tests depend on it.
- The interpreter now requires that the runtime.runtimeError
type be part of the SSA program. This requires that clients
import this package prior to invoking the interpreter.
This in turn requires (Importer).ImportPackage(path string),
which this CL adds.
- All $GOROOT/test/recover{,1,2,3}.go tests are now passing.
NB, the bug described in coverage.go (defer/recover in a concatenated
init function) remains. Will be fixed in a follow-up.
Fixesgolang/go#6381
R=gri
CC=crawshaw, golang-dev
https://golang.org/cl/13844043
Break the basic block at the function literal. The code to do this analysis
was already there; this CL just factors it out more nicely and uses it in
one new place. Also adds a test.
Fixesgolang/go#6555.
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/14601043
- removed support for nil constants from go/exact
- instead define a singleton Nil Object (the nil _value_)
- in assignments, follow more closely spec wording
(pending spec CL 14415043)
- removed use of goto in checker.unary
- cleanup around handling of isRepresentable for
constants, with better error messages
- fix missing checks in checker.convertUntyped
- added isTyped (== !isUntyped) and isInterface predicates
- fixed hasNil predicate: unsafe.Pointer also has nil
- adjusted ssa per adonovan
- implememted types.Implements (wrapper arounfd types.MissingMethod)
- use types.Implements in vet (and fix a bug)
R=adonovan, r
CC=golang-dev
https://golang.org/cl/14438052
Present the files in lexical order so the output is reproducible
and easier to navigate. Do a little type rearrangement to simplify
things while we're there.
R=adg
CC=golang-dev
https://golang.org/cl/14357043
The redirect.PrefixHandler redirects "/foo/" to "/foo", which is what
we want in most cases ("/cl/", "/change/", etc) but not here.
So wrap it with a handler that handles "/blog/" explictly.
R=dsymonds
CC=golang-dev
https://golang.org/cl/14326043
e.g. "oracle callgraph <package>"
Also: simplified error handling.
Eliminated oracle.errorf because it prepends "file:line:col: "
to the error message so the main function can't safely prepend "Error: ".
The position wasn't interesting though: it was just -pos, more or less.
R=crawshaw, dominik.honnef, r
CC=golang-dev
https://golang.org/cl/13864044
This CL is mostly a renaming s/json/serial/, abstracting the
oracle package away from any particular data syntax. (The
encoding/* machinery is very clean; clearly I should have
structured it this way from the outset.)
Supporting XML then becomes a one-liner in cmd/oracle/main.go.
Also: call MarshalIndent(), not Marshall() then Indent().
R=crawshaw
CC=golang-dev
https://golang.org/cl/13858046
(reflect.Value).Send
(reflect.Value).TrySend
(reflect.Value).Recv
(reflect.Value).TryRecv
(reflect.Type).ChanOf
(reflect.Type).In
(reflect.Type).Out
reflect.Indirect
reflect.MakeChan
Also:
- specialize genInvoke when the receiver is a reflect.Type under the
assumption that there's only one possible concrete type. This
makes all reflect.Type operations context-sensitive since the calls
are no longer dynamic.
- Rename all variables to match the actual parameter names used in
the reflect API.
- Add pointer.Config.Reflection flag
(exposed in oracle as --reflect, default false) to enable reflection.
It currently adds about 20% running time. I'll make it true after
the presolver is implemented.
- Simplified worklist datatype and solver main loop slightly
(~10% speed improvement).
- Use addLabel() utility to add a label to a PTS.
(Working on my 3 yr old 2x2GHz+4GB Mac vs 8x4GHz+24GB workstation,
one really notices the cost of pointer analysis.
Note to self: time to implement presolver.)
R=crawshaw
CC=golang-dev
https://golang.org/cl/13242062
Full help is only displayed when -help is requested;
CLI usage errors just remind the the user of this flag.
R=r, crawshaw
CC=golang-dev
https://golang.org/cl/13523048
line2byte doesn't handle non utf-8 fileencoding. So added s:getpos().
And also, changing errorformat is not right way on go filetype.
Added range operations.
R=golang-dev, kamil.kisiel, dsymonds, dominik.honnef
CC=golang-dev
https://golang.org/cl/13656045
Defer parsing of blog content until accessed for faster startup.
Fall back on redirect if blog content unavailable locally.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/13335052
Remove References section heading.
Add redirects from old paths to new content.
Add a link to the SubRepositories wiki page from package list.
Add styles for "pop-out" link.
R=r
CC=golang-dev
https://golang.org/cl/13356047
Previously these helpers were added by a private deployment script.
There's no reason why they shouldn't be part of godoc proper now
that it's in the go.tools repository.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/13722043
Define a minor mode called go-oracle-mode. Right now its sole
purpose is to define a keymap but it might later be used to add
hooks or add other features to go-mode.
R=adonovan
CC=golang-dev
https://golang.org/cl/13412048
+ test.
Also:
- provide non-nil map to Importer.doImport0() to avoid a crash.
- reorganize oracle "needs" bits.
- reduce "needs" of 'freevars' and 'implements' queries by avoiding
ssa.Packages when types.Package suffices.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13421046
Motivation: pointer analysis tools (like the oracle) want the
user to specify a set of initial packages, like 'go test'.
This change enables the user to specify a set of packages on
the command line using importer.LoadInitialPackages(args).
Each argument is interpreted as either:
- a comma-separated list of *.go source files together
comprising one non-importable ad-hoc package.
e.g. "src/pkg/net/http/triv.go" gives us [main].
- an import path, denoting both the imported package
and its non-importable external test package, if any.
e.g. "fmt" gives us [fmt, fmt_test].
Current type-checker limitations mean that only the first
import path may contribute tests: multiple packages augmented
by *_test.go files could create import cycles, which 'go test'
avoids by building a separate executable for each one.
That approach is less attractive for static analysis.
Details: (many files touched, but importer.go is the crux)
importer:
- PackageInfo.Importable boolean indicates whether
package is importable.
- un-expose Importer.Packages; expose AllPackages() instead.
- CreatePackageFromArgs has become LoadInitialPackages.
- imports() moved to util.go, renamed importsOf().
- InitialPackagesUsage usage message exported to clients.
- the package name for ad-hoc packages now comes from the
'package' decl, not "main".
ssa.Program:
- added CreatePackages() method
- PackagesByPath un-exposed, renamed 'imported'.
- expose AllPackages and ImportedPackage accessors.
oracle:
- describe: explain and workaround a go/types bug.
Misc:
- Removed various unnecessary error.Error() calls in Printf args.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13579043
The previous notation (sans '#') now yields an error but is
"reserved for future use", e.g. to denote line/column offsets.
Will implement as needed.
R=r, crawshaw
CC=golang-dev
https://golang.org/cl/13526043
1. ParseFiles (in util.go) parses each file in its own goroutine.
2. (*Importer).LoadPackage asynchronously prefetches the
import graph by scanning the imports of each loaded package
and calling LoadPackage on each one.
LoadPackage is now thread-safe and idempotent: it uses a
condition variable per package; the first goroutine to
request a package becomes responsible for loading it and
broadcasts to the others (waiting) when it becomes ready.
ssadump runs 34% faster when loading the oracle.
Also, refactorings:
- delete SourceLoader mechanism; just expose go/build.Context directly.
- CreateSourcePackage now also returns an error directly,
rather than via PackageInfo.Err, since every client wants that.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13509045
1. call display-buffer after the postprocessing step to avoid display glitch.
2. suppress the postprocessing progress message---it's too verbose.
(instead I should just make the postprocessing loop faster)
Also: rename channel-peers to just peers for consistency with other commands and documentation.
R=dominik.honnef
CC=golang-dev
https://golang.org/cl/13388044
See json.go for interface specification.
Example usage:
% oracle -format=json -mode=callgraph code.google.com/p/go.tools/cmd/oracle
+ Tests, based on (small) golden files.
Overview:
Each <query>Result structure has been "lowered" so that all
but the most trivial logic in each display() function has
been moved to the main query.
Each one now has a toJSON method that populates a json.Result
struct. Though the <query>Result structs are similar to the
correponding JSON protocol, they're not close enough to be
used directly; for example, the former contain richer
semantic entities (token.Pos, ast.Expr, ssa.Value,
pointer.Pointer, etc) whereas JSON contains only their
printed forms using Go basic types.
The choices of what levels of abstractions the two sets of
structs should have is somewhat arbitrary. We may want
richer information in the JSON output in future.
Details:
- oracle.Main has been split into oracle.Query() and the
printing of the oracle.Result.
- the display() method no longer needs an *oracle param, only
a print function.
- callees: sort the result for determinism.
- callees: compute the union across all contexts.
- callers: sort the results for determinism.
- describe(package): fixed a bug in the predicate for method
accessibility: an unexported method defined in pkg A may
belong to a type defined in package B (via
embedding/promotion) and may thus be accessible to A. New
accessibleMethods() utility fixes this.
- describe(type): filter methods by accessibility.
- added tests of 'callgraph'.
- pointer: eliminated the 'caller CallGraphNode' parameter from
pointer.Context.Call callback since it was redundant w.r.t
site.Caller().
- added warning if CGO_ENABLED is unset.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13270045
Pro: no shell quotation needed.
Con: can't be parsed by (the perpetually useless) Scanf.
R=crawshaw, dgryski
CC=golang-dev
https://golang.org/cl/13441043
(Its former location was based on a misunderstanding of 'go build'.)
Also: set GOMAXPROCS to NumCPU by default.
R=crawshaw
CC=golang-dev
https://golang.org/cl/13354043
Update golang/go#6212
See issue 6259.
When that is resolved, we can do a better job. Until then, we just see if the
type has a method called Format and, if so, assume it's a Formatter and so
there's nothing to check.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/13267043
+ Tests.
+ Emacs integration.
+ Emacs integration test.
+ very rudimentary Vim integration. Needs some love from a Vim user.
TODO (in follow-ups):
- More tests would be good.
We'll need to make the output order deterministic in more places.
- Documentation.
R=gri, crawshaw, dominik.honnef
CC=golang-dev
https://golang.org/cl/9502043
App Engine needs the whitelist file and it's cleaner if it's
in its own package where it can be imported directly, without
pushing composite.go through a pile of sed.
R=dsymonds, r
CC=golang-dev
https://golang.org/cl/12746045
TBR: gri
I cannot create an issue on the tracker for some reason, so here it is:
go vet contains this snippet:
if types.IsAssignableTo(typ, errorType) || types.IsAssignableTo(typ, stringerType) {
It's getting the wrong answer: It claims
interface {
f()
}
or even
interface {
f() float64
}
matches the Error and Stringer interfaces. Both of them. This causes a test failure:
$ go test code.google.com/p/go.tools/cmd/vet
BUG: errchk: testdata/print.go:124: missing expected error: '"for printf verb %s of wrong type"'
$
This worked until very recently.
R=gri
CC=golang-dev
https://golang.org/cl/12398043
The old code only got it right for Stringers (etc.) and a few other simple cases.
But the rule used by fmt.Printf for non-Stringers is that pointers to structs
print as pointers, the rest must satisfy the format verb element-wise.
Thus for example
struct {a int, b []byte}
prints with %d and %q (sic) but not %g.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/12340043
Also move some common code from main.go to handlers.go, and
exclude remotesearch.go from the appengine build (it is only
relevant to the command line interface).
R=bradfitz, dsymonds
CC=golang-dev
https://golang.org/cl/12001049
The analysis for types.Array was just missing. It's the same as a slice,
but we can't share code easily because the types differ, so we just dup it.
R=dsymonds
CC=golang-dev
https://golang.org/cl/12041045
First, %v and %T accept any arguments, so they should never warn.
Second, pointer types were not handled in matchArgType.
Third, the default response for matchArgType should be false.
R=r
CC=adonovan, golang-dev
https://golang.org/cl/12038050
Phrases like "returns whether or not the image is opaque" could be
describing what the function does (it always returns, regardless of
the opacity) or what it returns (a boolean indicating the opacity).
Even when the "or not" is missing, the phrasing is bizarre.
Go with "reports whether", which is still clunky but at least makes
it clear we're talking about the return value.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/11458046
I'd like to make vet work as a filter, but passing /dev/stdin as a
command line argument doesn't work. This at least makes it not panic.
R=r
CC=golang-dev
https://golang.org/cl/11521045
The exported Server becomes handlerServer, and part of Presentation
now. Presentation is also now an http.Handler with its own
internal mux (a detail, which might go away).
main.go becomes ever simpler.
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/11505043
Allmost all uses of go/types that wanted the type
information computed, installed callback functions
that stored the information in maps. Most of the
time this is the only thing that could be done because
there is no guarantee that types are completely set
up before the end of type-checking.
This CL removes the respective Context callbacks in favor
of corresponding maps that collect the desired information
on demand, grouped together in an optional Info struct.
R=adonovan
CC=golang-dev
https://golang.org/cl/11530044
cmd/godoc/godoc.go is now merged into main.go, which is now
only 530 lines.
App Engine mode is still broken, but should be easy to fix up.
(just needs a global *godoc.Presentation created in init)
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/11498044
The plan for godoc:
- Copy godoc source from the core repo to go.tools (this CL).
- Break godoc into several packages inside go.tools, leaving a package
main that merely sets up a local file system, interprets the command
line, and otherwise delegates the heavy-lifting to the new packages.
- Remove godoc from the core repo.
- Update cmd/go to install this godoc binary in $GOROOT/bin.
- Update misc/dist to include godoc when building binary distributions.
R=bradfitz
CC=golang-dev
https://golang.org/cl/11408043
Details:
- added support for complex numbers as distinct from floats:
%[efg] allows complex; %b does not.
- %p: only Signature, Map, Chan, Slice, unsafe.Pointer allowed.
- %s: allow []byte.
- allow a verb to match map[K]V and []T if it matches K/V/T,
e.g. %d now matches []int. i.e. matching is recursive.
- use go/types' constant folding. literal() is gone.
- group cases together.
Added tests.
R=gri, r
CC=golang-dev
https://golang.org/cl/10895043
This requires a little more tree rewriting to put a block around the if of an "else if".
More tests too.
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/11042045
Need the ./ for the path. Quick fix to get the build green, but what really is the right answer for Windows?
R=golang-dev
CC=golang-dev
https://golang.org/cl/10359044
Test the statistics work as expected for a simple program, which can be extended as needed. This is all a bit meta.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10392050
Experimental feature. It's too noisy yet to be enabled by default,
so it must be enabled explicitly by
go tool vet -shadow *.go
or
go tool vet -shadow directory
(The go command does not know about the -shadow flag.)
Fixesgolang/go#5634.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/10409047
Before:
math/big/nat_test.go:688: arg r for printf verb %s of wrong type: ..Word
After:
math/big/nat_test.go:688: arg r for printf verb %s of wrong type: big.Word
R=gri
CC=golang-dev
https://golang.org/cl/10400044
A trailing % resulted in a bad error message.
Also clean up a couple of dregs left over from the
refactoring to add indexed formats.
R=dsymonds
CC=golang-dev
https://golang.org/cl/10336044
This number will allow us to give a conventional meaning to "coverage":
the percentage of executable statements visited by the test.
R=adonovan
CC=golang-dev
https://golang.org/cl/10271045
We are going to need one more piece of data, so rather than create
a third variable let's just put it all in one struct. The interface gets
easier too.
R=adonovan
CC=golang-dev
https://golang.org/cl/10271044
This is just the tool proper; stitching into "go test" will be a separate CL.
Tests are missing - they'll come once it's integrated - but it can handle,
perhaps correctly, all of src/pkg/...
The basic approach is to rewrite the source to add annotations that will
track coverage; the rewritten source must of course be compiled and
run after this tool has done its job.
R=adonovan
CC=golang-dev
https://golang.org/cl/10102043
Rewrite the checker to be more flexible and better documented, being
more explicit about parsed format vs. checked arguments.
No attempt yet to do check indexed formats; this just paves the way.
All tests still pass.
R=gri
CC=golang-dev
https://golang.org/cl/9881044
Can't reproduce the failure outside the builder, but attempt a fix
by changing the criterion for failure: FAIL iff the output contains "BUG".
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/9798050
Simplify the code a bit, get it working after recent go/types changes,
and handle "%*%" just in case. Preparation for handling argument indexing.
R=gri
CC=golang-dev
https://golang.org/cl/9747045
- remove Makefile
- move test data into a subdirectory
- encapsulate the invocation of errchk into a standard Test using os.exec
R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/9509045
Now that it's in the same repository as the go/types package we can delete
the mechanism to build it without type checking.
Add a make rule to install it where the go tool can find it.
Why doesn't "go install" take a -o flag?
R=gri
CC=gobot, golang-dev
https://golang.org/cl/9526043
They will be deleted from their current homes once this has landed.
Changes made to import paths to make the code compile, and to find
errchk in the right place in cmd/vet's Makefile.
TODO in a later CL: tidy up vet.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/9495043
