In line with other functions such as Fprintf, put the
thing to be written first.
Apologies for the breakages this is sure to cause.
R=rsc, gri, adg, eds, r2, aam
CC=golang-dev
https://golang.org/cl/4169042
Neither gofmt nor godoc are making use of a Styler (for
token-specific formatting) anymore. Stylers interacted in complicated
ways with HTML-escaping which was why the printer needed an HTML mode
in the first place.
godoc now uses a more powerful and general text formatting
function that does HTML escaping, text selection, and can
handle token-specific formatting if so desired (currently
used only for comments).
As a consequence, cleaned up uses of go/printer in godoc;
simplified the various write utility functions, and also
removed the need for the "html" template format (in favor of
html-esc which now does the same and is used more pervasively).
Applied gofmt -w src misc to verify no changes occured,
and tested godoc manually.
There should be no visible changes except that (type) code
snippets presented for godoc package documentation now
uses the same formatting as for general source code and
thus comments get the comment-specific color here as well
(not the case at the moment).
(TODO: godoc needs a good automatic test suite).
R=rsc
CC=golang-dev
https://golang.org/cl/4152042
This aligns the naming scheme with the testing package and
also lets govet work on more logging calls.
R=rsc
CC=golang-dev
https://golang.org/cl/4001048
This bug prevented files such as READMEs etc. from being included in the index.
For instance, now author names recorded in the AUTHORS file can be found with
a godoc query.
R=rsc, r2
CC=golang-dev
https://golang.org/cl/4005047
- added flag -maxresults (default: 10000) to limit the max.
number of full text results shown
- removed flag -fulltext; use -maxresults=0 to disable fulltext
index
- better indication on result page if not all results are shown
(... after line list)
R=rsc, gri1
CC=golang-dev
https://golang.org/cl/4049042
A query string of the form ident.ident will be used both as a qualified
identifier for identifier search and as a regular expression.
Qualified identifier lookup got broken accidentally when introducing
regexp full text search. Cleaned up surrounding logic a bit.
R=rsc
CC=golang-dev
https://golang.org/cl/3984042
Until now, each scan of a file added a new file to the file set.
With this change, a file can be re-scanned using the same *token.File
w/o changing the file set. Eventually this will enable the re-use of
cached source code in godoc (for the fulltext index). At the moment,
source files are read over and over again from disk.
This is the first step in that direction.
R=r, rsc, r2
CC=golang-dev
https://golang.org/cl/4001041
When searching for regular expressions such as
".", there are many consecutive matches.
In the generated HTML, combine them instead of
generating a new <span> for each adjacent text
segment highlighting a match.
Massively reduces the size of the generated
HTML in those cases.
R=r, rsc
CC=golang-dev
https://golang.org/cl/3971041
Regular expressions may now be used in conjuction with full text
search. Godoc will show the first 10000 occurences in the source
code and highlight the respective text segments.
- added new flag -testDir to specify a small directory for testing
(fast index creation; default = "")
- use new FormatText function to format text and Go source
code in HTML, supporting multiple kinds of text selections
simulatenously); this replaces the uses of go/printer
Stylers
- for now removed currently unused mechanism for identifier-
specific JS popups (will come back in some form once we
have type or other useful information)
- various typo fixes and minor cleanups throughout
Missing:
- indexing of non-.go files
R=r, r2
CC=golang-dev, rsc
https://golang.org/cl/3699041
- improved search result page
- clicking on files shows highlighted search phrase
(.go files loose their godoc-specific formatting and
highlighting in this mode - a better solution is in
the works)
- support for textual results
- fixed bug with non-URL escaped URL parameter (Query)
R=rsc, adg
CC=golang-dev
https://golang.org/cl/3585041
To enable use -fulltext flag; e.g.: godoc -v -fulltext -http=:7777
Enabling the fulltext index will use significantly more memory as
the text of all source code, the respective suffixarray, and the
file set data structure is kept in memory. At the moment there is
about 6Mb of source code (~1400 files) indexed under GOROOT.
Source code + suffix array together consume 5*(size of source) or
about 30Mb. The file set data structure consumes about 4b/src line.
By default only up to 5000 results are shown for now.
The presentation of the results needs tuning. In particular,
if a string is found, clicking on the respective file does not
highlight them, yet.
At the moment, only Go source files are indexed. Eventually,
the full text index should encompass other files as well.
R=rsc, adg
CC=golang-dev
https://golang.org/cl/3182043
- change Walk signature to use an ast.Node instead of interface{}
- add Pos functions to a couple of ast types to make them proper nodes
- explicit nil checks where a node can be nil; incorrect ASTs cause Walk to crash
For now ast.Walk is exercised extensively as part of godoc's indexer;
so we have some confidence in its correctness. But this needs a test,
eventually.
Fixes#1326.
R=rsc, r
CC=golang-dev
https://golang.org/cl/3481043
Specifically:
* lib/godoc:
- provide file set (FSet) argument to formatters where needed
* src/cmd:
- cgo, ebnflint, godoc, gofmt, goinstall: provide file set (fset) where needed
- godoc: remove local binary search with sort.Search (change by rsc),
extract file set for formatters
* src/pkg:
- exp/eval: remove embedded token.Position fields from nodes and replace
with named token.Pos fields; add corresponding Pos() accessor methods
- go/token: added file.Line(), changed signature of File.Position()
* test/fixedbugs/:
- bug206.go: change test to not rely on token.Pos details
* added various extra comments
* Runs all.bash
* gofmt formats all of src, misc w/o changes
* godoc runs
* performance:
- The new version of godoc consumes about the same space after indexing
has completed, but indexing is half the speed. Significant space savings
are expected from smaller ASTs, but since they are thrown away after a
file has been indexed, this is not visible anymore. The slower indexing
time is due to the much more expensive computation of line information.
However, with the new compressed position information, indexing can be
rewritten and simplified. Furthermore, computing the line info can be
done more efficiently.
New godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
44381 godoc 0.0% 0:38.00 4 19 149 145M 184K 148M 176M
2010/12/03 17:58:35 index updated (39.231s, 18505 unique words, 386387 spots)
2010/12/03 17:58:35 bytes=90858456 footprint=199182584
2010/12/03 17:58:36 bytes=47858568 footprint=167295224
Old godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
23167 godoc 0.0% 0:22.02 4 17 132 129M 184K 132M 173M
2010/12/03 14:51:32 index updated (24.892s, 18765 unique words, 393830 spots)
2010/12/03 14:51:32 bytes=66404528 footprint=163907832
2010/12/03 14:51:32 bytes=46282224 footprint=163907832
The different numbers for unique words/spots stem from the fact the the
two workspaces are not exactly identical. The new godoc maintains a large
file set data structure during indexing which (probably) is the reason
for the larger heap (90858456 vs 66404528) before garbage collection.
R=rsc, r
CC=golang-dev
https://golang.org/cl/3050041
Make them more like Printf, with a ... final argument. This breaks
code with existing formatters but not the templates that use them.
R=rsc, gri
CC=golang-dev
https://golang.org/cl/3378041
- compare against fsModified to check if index is out of date
- don't change fsModified if there are no user-mapped file systems
R=rsc, iant
CC=golang-dev
https://golang.org/cl/3213041
This fixes a problem with relativePath, where
a prefix was not recognized because it ended
in "//" as opposed to just "/".
Also: Minor unrelated cleanup of a declaration.
R=rsc
CC=golang-dev
https://golang.org/cl/3146041
This is in preparation for a different position representation.
It also resolves situations where a node would be printed as
it's node position simply because the embedded token.Position
has a String method.
R=r
CC=golang-dev
https://golang.org/cl/2991041
New logging interface simplifies and generalizes.
1) Loggers now have only one output.
2) log.Stdout, Stderr, Crash and friends are gone.
Logging is now always to standard error by default.
3) log.Panic* replaces log.Crash*.
4) Exiting and panicking are not part of the logger's state; instead
the functions Exit* and Panic* simply call Exit or panic after
printing.
5) There is now one 'standard logger'. Instead of calling Stderr,
use Print etc. There are now triples, by analogy with fmt:
Print, Println, Printf
What was log.Stderr is now best represented by log.Println,
since there are now separate Print and Println functions
(and methods).
6) New functions SetOutput, SetFlags, and SetPrefix allow global
editing of the standard logger's properties. This is new
functionality. For instance, one can call
log.SetFlags(log.Lshortfile|log.Ltime|log.Lmicroseconds)
to get all logging output to show file name, line number, and
time stamp.
In short, for most purposes
log.Stderr -> log.Println or log.Print
log.Stderrf -> log.Printf
log.Crash -> log.Panicln or log.Panic
log.Crashf -> log.Panicf
log.Exit -> log.Exitln or log.Exit
log.Exitf -> log.Exitf (no change)
This has a slight breakage: since loggers now write only to one
output, existing calls to log.New() need to delete the second argument.
Also, custom loggers with exit or panic properties will need to be
reworked.
All package code updated to new interface.
The test has been reworked somewhat.
The old interface will be removed after the new release.
For now, its elements are marked 'deprecated' in their comments.
Fixes#1184.
R=rsc
CC=golang-dev
https://golang.org/cl/2419042
Use -timestamps=false flag to disable.
(This used to be shown on the front-page below the
build information with the old godoc. However, the
time stamps are directory-specific and should be
shown with the directory.)
R=rsc
CC=golang-dev
https://golang.org/cl/2233044
- in prep. for some restructuring to be able to better deal
with very large file systems
- moved a utility function into index.go
- no functionality changes, only code reorg.
R=r, adg1
CC=golang-dev
https://golang.org/cl/2098044
- refine/define Scope, Object, and Type structures
(note: scope.go has the addition of types, the rest is only re-organized
for better readability)
- implemented top-level of type checker:
resolve global type declarations (deal with double decls, cycles, etc.)
- temporary hooks for checking of const/var declarations, function/method bodies
- test harness for fine-grained testing (exact error locations)
with initial set of tests
This is a subset of the code for easier review.
R=rsc
CC=golang-dev
https://golang.org/cl/1967049
- change ast.Ident back to contain the name and adjust all dependent code
- identifier object information will be added again through an optional
typechecker phase (in the works).
- remove tracking of scopes in parser - it's easier to do this in a separate
phase (in the works)
- in godoc, generate popup info table directly instead of through a formatter
for simpler data flow (at the expense of a little bit more code)
Runs all tests.
As a result of this change, the currently shown popup information
(const, var, type, func, followed by identifier name) will not be
shown anymore temporarily.
R=rsc
CC=golang-dev
https://golang.org/cl/1994041
$ godoc xml | grep Copy\(\)
func (c CharData) Copy() CharData
func (c Comment) Copy() Comment
func (d Directive) Copy() Directive
func (p ProcInst) Copy() ProcInst
func (e StartElement) Copy() StartElement
--------------------------------------------
$ godoc -src xml | grep Copy\(\)
func (c CharData) Copy() CharData
--------------------------------------------
$ godoc -src xml Copy
func (c CharData) Copy() CharData { return CharData(makeCopy(c)) }
--------------------------------------------
The command "godoc -src pkg_name" should output the interface of the named package, but it excludes all duplicate entries. Also the command "godoc -src pkg_name method_name" will output the source code only for one method even if there are more of them with the same name in the same package. This patch set fixes this issue.
R=gri
CC=golang-dev
https://golang.org/cl/883051
note that sortmain.go has been run through hg gofmt;
only the formatting of the day initializers changed.
i'm happy to revert that formatting if you'd prefer.
stop on error in doc/progs/run
R=r
CC=golang-dev
https://golang.org/cl/850041
- change the various url-xxx formatters to return a relative URL path
- make the leading '/' for URLs explicit in the template
- on the way change some |html formatters to |html-esc
(html should only be used for formatting AST nodes)
R=rsc, r
CC=golang-dev
https://golang.org/cl/740041
Instead of returning the index lookup result via
RPC which has to be corrected for the client,
simply render it on the server and return the
final output.
R=rsc, r
CC=golang-dev
https://golang.org/cl/669041
and highlight special words, if provided. Also:
- related cleanups in src/pkg/go/doc/comment.go
- fix typos in src/cmd/goinstall/doc.go
Fixes#672.
R=rsc
CC=adg, golang-dev
https://golang.org/cl/601042
The command-line search is using a running webserver
as index server; i.e., the search result is reflecting
the index at the server. See the documentation for
details.
Usage: godoc -q query1 query2 ...
Known issue: Results don't show the all-important
line numbers yet due to the way the index is organized.
Next CL.
R=rsc, r
CC=golang-dev
https://golang.org/cl/648041
for identifiers in Go source code
- at the moment just show identifier kind (var, func, etc.) and name
(eventually should show declaration, type, etc.)
- JavaScript parts by adg
R=rsc
CC=adg, golang-dev
https://golang.org/cl/578042
- on the commandline: godoc -x big
- in a webpage: provide form parameter ?m=src
Known issues:
- Positioning of comments incorrect in several cases. Separate CL.
- Need a link/menu to switch between different modes of presentation
in the web view.
R=rsc
CC=golang-dev
https://golang.org/cl/376041
- gofmt -w src misc
- improves several lists and fixes minor degradation introduced
with the fix for issue 628
- removed some dead code (stringList)
R=rsc
CC=golang-dev
https://golang.org/cl/223058
- go/ast: removed StringList (not needed anymore)
- go/ast: changed import path and field list tag to a single string
- updated all dependencies
R=rsc
CC=golang-dev
https://golang.org/cl/217056
Example use: godoc -path=/home/user1:/home/build/foo -http=:6666
will start a local godoc that maps urls starting with /pkg/user1 or
/pkg/foo to the respective roots specified in the path.
Missing: Handling of overlapping package directories, multiple
packages per directory.
R=rsc
CC=golang-dev
https://golang.org/cl/206078
Go source outside $GOROOT.
If the argument is a path starting with / or ., disregard $GOROOT.
Also, disable the check for package name matching the directory,
which is counterproductive in this case.
Apologies for the violence to the code but we need some help documenting
Go code outside the standard repository.
R=gri
CC=golang-dev
https://golang.org/cl/201064
- provide scope to parse functions; if non-nil, parser uses the scope
to declare and lookup identifiers
- resolve forward references where possible
R=rsc
CC=golang-dev
https://golang.org/cl/194098
- Identifiers refer now to the language entity (Object)
that they denote. At the moment this is at best an
approximation.
- Initial data structures for language entities (Objects)
and expression types (Type) independent of the actual
type notations.
- Initial support for declaring and looking up identifiers.
- Updated various dependent files and added support functions.
- Extensively tested to avoid breakage. This is an AST change.
R=rsc
CC=golang-dev, rog
https://golang.org/cl/189080
now printed using the semicolon-free style.
Removed NoSemis flag and mechanism dealing with
optional semicolons from go/printer.
Updated all go/printer output files using the
semi-colon free style.
Changes have no formatting impact on checked in
go code under src and misc.
R=rsc, r
CC=golang-dev
https://golang.org/cl/184068
tabs after an empty line where not converted.
Also, made it more robust in the presence of
(unexpected) ' ' and '\v' chars in indentation
mode.
R=r
CC=golang-dev
https://golang.org/cl/181085
parsing and printing to new syntax.
Use -oldparser to parse the old syntax,
use -oldprinter to print the old syntax.
2) Change default gofmt formatting settings
to use tabs for indentation only and to use
spaces for alignment. This will make the code
alignment insensitive to an editor's tabwidth.
Use -spaces=false to use tabs for alignment.
3) Manually changed src/exp/parser/parser_test.go
so that it doesn't try to parse the parser's
source files using the old syntax (they have
new syntax now).
4) gofmt -w src misc test/bench
1st set of files.
R=rsc
CC=agl, golang-dev, iant, ken2, r
https://golang.org/cl/180047
nodes in the tree are nested with respect to one another.
a simple change to the Visitor interface makes it possible
to do this (for example to maintain a current node-depth, or a
knowledge of the name of the current function).
Visit(nil) is called at the end of a node's children;
this make possible the channel-based interface below,
amongst other possibilities.
It is still just as simple to get the original behaviour - just
return the same Visitor from Visit.
Here are a couple of possible Visitor types.
// closure-based
type FVisitor func(n interface{}) FVisitor
func (f FVisitor) Visit(n interface{}) Visitor {
return f(n);
}
// channel-based
type CVisitor chan Visit;
type Visit struct {
node interface{};
reply chan CVisitor;
};
func (v CVisitor) Visit(n interface{}) Visitor
{
if n == nil {
close(v);
} else {
reply := make(chan CVisitor);
v <- Visit{n, reply};
r := <-reply;
if r == nil {
return nil;
}
return r;
}
return nil;
}
R=gri
CC=rsc
https://golang.org/cl/166047
- simplified dealing with parse errors: no need to intersperse them in the source
- improve visibility of highlighted identifiers by showing them in bold
R=rsc
https://golang.org/cl/163051
the bash scripts and makefiles for building go didn't take into account
the fact $GOROOT / $GOBIN could both be directories containing whitespaces,
and was not possible to build it in such a situation.
this commit adjusts the various makefiles/scripts to make it aware of that
possibility, and now it builds successfully when using a path with whitespaces
as well.
Fixes#115.
R=rsc, dsymonds1
https://golang.org/cl/157067
package documentation using templates. The line tag interferes
with the anchor tag introduces by the template.
This fixes an an issue where some headers lost their ability
to link to the respective source code.
R=rsc
https://golang.org/cl/154166
- When providing alternative spellings to a query, do not
prefix it with a package qualifier as the suggestion may
not have any results. Correctly filtering is quite a bit
of work, and clicking the alternative spelling will always
also show the qualified hits if they exist (but also others).
Seems good enough for now.
- Give user feedback when the query syntax was wrong.
- Package names in search results are now links to the respective
package documentation.
- Experimented with excluding main packages and test files
from index with inconclusive results. Code is present and
can be enabled by changing a flag in the source. This needs
some more work.
R=rsc
CC=r, dsymonds
http://go/go-review/1026033
HTML vs Html, URL vs Url, HTTP vs Http,
current source is 6:1 in favor of the former,
so change instances of the latter.
R=r
CC=go-dev
http://go/go-review/1024026
- introduced a new run per file containing all spots belonging
to the same kind (e.g. var decl, const decl, etc.)
- more comments, better index.go file organization
R=rsc
http://go/go-review/1026028
(this was surprisingly hard to get right in HTML)
- show modification times in source directory listings
- various tweaks
R=rsc
http://go/go-review/1024024
- formatting in dirs.html is crude, needs better html (open to suggestions),
but shows the synopsis
- many package comments should probably be adjusted such that the first
sentence is more concise
R=rsc, iant
http://go/go-review/1025014
- enabled for function declarations (not just function literals)
- applied gofmt -w $GOROOT/src
(look for instance at src/pkg/debug/elf/elf.go)
R=r, rsc
CC=go-dev
http://go/go-review/1026006
(No visual change yet since the comments for
exported code snippets - as shown on the pkg
page - are not yet formatted using the standard
comment intersperse mechanism in the printer.
This is a TODO in the printer.)
R=rsc
http://go/go-review/1016031
just a single directory
- all pieces present but not well integrated
- directory tree served at the moment under /tree
R=rsc
http://go/go-review/1018016
- replaced deprecated use of </font> with </span> tag
- added html escaping to godoc formatters where missing
- enabled text format for package documentation
R=rsc
http://go/go-review/1017001
- sort by package name (instead of package path) for results with snippets
- sort line numbers in results without snippets
- properly characterize package clauses
- experiment with a leaner look: no underlines for top-level godoc links in the left side bar
Still using colors to distinguish results. Next step.
R=rsc
http://go/go-review/1015016
- added goroutine to automatically index in the background
- added handler for search requests
- added search box to top-level godoc template
- added search.html template for the display of search results
- changes to spec.go because of name conflicts
- added extra styles to style.css (for shorter .html files)
R=rsc
http://go/go-review/1014011
- handle HTML tagging via (client-installable) Stylers
go/doc:
- basic styler support
- some factoring
- ready to contain the search code (but for now excluded)
doc/style.css:
- updated
doc/go_spec.css:
- cleanup: replace deprecated uses of <font> tag with <span> tag
R=rsc
DELTA=302 (160 added, 62 deleted, 80 changed)
OCL=35973
CL=35996
- the first HTML comment in those files is extracted as page
title when serving them
- lib/godoc.html is top-level template for all pages served
- experimented a bit with package documentation layout
(feedback welcome)
- various related cleanups
TODO:
- The / page (doc/root.html) content repeats links that are
in the navigation bar. It needs to be cleaned up.
R=rsc
DELTA=826 (86 added, 692 deleted, 48 changed)
OCL=35230
CL=35245
- emit line tag id's in html mode
- support for general html tags
- better names for a few identifiers
godoc.go:
- emit links from exported names to source code
(actual placement needs fine-tuning)
R=rsc
DELTA=108 (68 added, 4 deleted, 36 changed)
OCL=32639
CL=32654
- prepare for generation of HTML id tags and links
- do HTML-escaping in central print routine
- move tabwriter setup into printer
- fixed various TODOs
godoc:
- removed tabwriter setup, need for various HTML-escaping
R=rsc
DELTA=210 (107 added, 36 deleted, 67 changed)
OCL=32612
CL=32616
- more orthogonal functionality of filter functions for better re-use
go/doc/doc.go:
- simplified interface
- collect filenames of packages so that they can be shown
godoc:
- removed TODO, show list of package (linked) files used to create documentation
R=rsc
DELTA=130 (68 added, 24 deleted, 38 changed)
OCL=32549
CL=32552
- Changed filter function for parser.ParsePackage to
take an *os.Dir instead of a filename for more
powerful filters
- Removed TODO in ast.PackageInterface: Now collect
package comments from all package files
- Cleanups in godoc: Use the new ParsePackage and
PackageInterface functions; as a result computing
package information is much simpler now.
R=rsc
DELTA=285 (80 added, 110 deleted, 95 changed)
OCL=32473
CL=32486
gofmt formatted source code looks pretty good already;
with a bit more fine-tuning it should be great.
printer.go:
- Implemented comment intersperse algorithm.
The approach is a result of many trial-and-error
experiments but at this point reasonably simple
and open to arbitrary fine-tuning.
parser.go:
- Simplified handling of lead and line comments
(formerly called leading and trailing comments).
- Use a comments list instead of an array (I may
change this back - this is not obviously better
and uses more space).
doc.go:
- Remove comments from AST nodes that have been
'consumed' in the documentation to avoid duplicate
printing of them. Allows for better control of
what is printed w/o use of printing control flags
(which are hard to use and not fine-grained enough).
Corresponding adjustments to various clients of these
files.
R=rsc
DELTA=478 (275 added, 108 deleted, 95 changed)
OCL=32185
CL=32380
- renamed Program -> SourceFile
- added Package node representing the AST for an entire package
- added filter function to create a source file mimicking the
interface of an entire package
parser:
- parser entry to parse entire packages
- unified naming of parser entry points
- factored out entry points into new file (interface.go)
gofmt:
- extended to accept single .go files, and package paths:
gofmt file.go // formatting of a single file
gofmt -x file.go // interface of a single file
gofmt -x ./MyPackage // interface of a local package
gofmt -x math // interface of a $GOROOT relative package
Various adjustments in dependent files, documentation.
R=rsc
DELTA=634 (369 added, 153 deleted, 112 changed)
OCL=31743
CL=31748
- use FilterExports instead of the various predicates in printer.go and doc.go
which simplifies a lot of code and makes it easier to deal with complex cases
R=rsc
DELTA=445 (197 added, 190 deleted, 58 changed)
OCL=31110
CL=31196
