go/godoc/analysis

Type and Pointer Analysis to-do list
====================================

Alan Donovan <adonovan@google.com>


Overall design
--------------

We should re-run the type and pointer analyses periodically,
as we do with the indexer.

Version skew: how to mitigate the bad effects of stale URLs in old pages?
We could record the file's length/CRC32/mtime in the go/loader, and
refuse to decorate it with links unless they match at serving time.

Use the VFS mechanism when (a) enumerating packages and (b) loading
them.  (Requires planned changes to go/loader.)

Future work: shard this using map/reduce for larger corpora.

Testing: how does one test that a web page "looks right"?


Bugs
----

(*go/loader.Program).Load fails if it encounters a single parse error.
Make this more robust.

(*ssa.Program).Create requires transitively error-free packages.  We
can make this more robust by making the requirement transitively free
of "hard" errors; soft errors are fine.

Markup of compiler errors is slightly buggy because they overlap with
other selections (e.g. Idents).  Fix.


User Interface
--------------

CALLGRAPH:
- Add a search box: given a search node, expand path from each entry
  point to it.
- Cause hovering over a given node to highlight that node, and all
  nodes that are logically identical to it.
- Initially expand the callgraph trees (but not their toggle divs).

CALLEES:
- The '(' links are not very discoverable.  Highlight them?

Type info:
- In the source viewer's lower pane, use a toggle div around the
  IMPLEMENTS and METHODSETS lists, like we do in the pacakge view.
  Only expand them initially if short.
- Include IMPLEMENTS and METHOD SETS information in search index.
- URLs in IMPLEMENTS/METHOD SETS always link to source, even from the
  package docs view.  This makes sense for links to non-exported
  types, but links to exported types and funcs should probably go to
  other package docs.
- Suppress toggle divs for empty method sets.

Misc:
- Add an "analysis help" page explaining the features and UI in more detail.
- The [X] button in the lower pane is subject to scrolling.
- Should the lower pane be floating?  An iframe?
  When we change document.location by clicking on a link, it will go away.
  How do we prevent that (a la Gmail's chat windows)?
- Progress/status: for each file, display its analysis status, one of:
   - not in analysis scope
   - type analysis running...
   - type analysis complete
     (+ optionally: there were type errors in this file)
   And if PTA requested:
   - type analysis complete; PTA not attempted due to type errors
   - PTA running...
   - PTA complete
- Scroll the selection into view, e.g. the vertical center, or better
  still, under the pointer (assuming we have a mouse).


More features
-------------

Display the REFERRERS relation?  (Useful but potentially large.)

Display the INSTANTIATIONS relation? i.e. given a type T, show the set of
syntactic constructs that can instantiate it:
    var x T
    x := T{...}
    x = new(T)
    x = make([]T, n)
    etc
    + all INSTANTIATIONS of all S defined as struct{t T} or [n]T
(Potentially a lot of information.)
(Add this to oracle too.)


Optimisations
-------------

Each call to addLink takes a (per-file) lock.  The locking is
fine-grained so server latency isn't terrible, but overall it makes
the link computation quite slow.  Batch update might be better.

Memory usage is now about 1.5GB for GOROOT + go.tools.  It used to be 700MB.

Optimize for time and space.  The main slowdown is the network I/O
time caused by an increase in page size of about 3x: about 2x from
HTML, and 0.7--2.1x from JSON (unindented vs indented).  The JSON
contains a lot of filenames (e.g. 820 copies of 16 distinct
filenames).  20% of the HTML is L%d spans (now disabled).  The HTML
also contains lots of tooltips for long struct/interface types.
De-dup or just abbreviate?  The actual formatting is very fast.

The pointer analysis constraint solver is way too slow.  We really
need to implement the constraint optimizer.  Also: performance has
been quite unpredictable; I recently optimized it fourfold with
type-based label tracking, but the benefit seems to have evaporated.
TODO(adonovan): take a look at recent CLs for regressions.