Example:
PACKAGE
package utf8
import "unicode/utf8"
Package utf8 implements functions and constants to support text
encoded in UTF-8. This package calls a Unicode character a rune for
brevity.
CONSTANTS
const (
RuneError = unicode.ReplacementChar // the "error" Rune or "replacement character".
RuneSelf = 0x80 // characters below Runeself are represented as themselves in a single byte.
UTFMax = 4 // maximum number of bytes of a UTF-8 encoded Unicode character.
)
Numbers fundamental to the encoding.
FUNCTIONS
func DecodeLastRune(p []byte) (r rune, size int)
DecodeLastRune unpacks the last UTF-8 encoding in p and returns the
rune and its width in bytes.
func DecodeLastRuneInString(s string) (r rune, size int)
DecodeLastRuneInString is like DecodeLastRune but its input is a
string.
func DecodeRune(p []byte) (r rune, size int)
DecodeRune unpacks the first UTF-8 encoding in p and returns the rune
and its width in bytes.
func DecodeRuneInString(s string) (r rune, size int)
DecodeRuneInString is like DecodeRune but its input is a string.
func EncodeRune(p []byte, r rune) int
EncodeRune writes into p (which must be large enough) the UTF-8
encoding of the rune. It returns the number of bytes written.
func FullRune(p []byte) bool
FullRune reports whether the bytes in p begin with a full UTF-8
encoding of a rune. An invalid encoding is considered a full Rune
since it will convert as a width-1 error rune.
func FullRuneInString(s string) bool
FullRuneInString is like FullRune but its input is a string.
func RuneCount(p []byte) int
RuneCount returns the number of runes in p. Erroneous and short
encodings are treated as single runes of width 1 byte.
func RuneCountInString(s string) (n int)
RuneCountInString is like RuneCount but its input is a string.
func RuneLen(r rune) int
RuneLen returns the number of bytes required to encode the rune.
func RuneStart(b byte) bool
RuneStart reports whether the byte could be the first byte of an
encoded rune. Second and subsequent bytes always have the top two
bits set to 10.
func Valid(p []byte) bool
Valid reports whether p consists entirely of valid UTF-8-encoded
runes.
func ValidString(s string) bool
ValidString reports whether s consists entirely of valid UTF-8-encoded
runes.
TYPES
type String struct {
// contains filtered or unexported fields
}
String wraps a regular string with a small structure that provides
more efficient indexing by code point index, as opposed to byte index.
Scanning incrementally forwards or backwards is O(1) per index
operation (although not as fast a range clause going forwards).
Random access is O(N) in the length of the string, but the overhead is
less than always scanning from the beginning. If the string is ASCII,
random access is O(1). Unlike the built-in string type, String has
internal mutable state and is not thread-safe.
func NewString(contents string) *String
NewString returns a new UTF-8 string with the provided contents.
func (s *String) At(i int) rune
At returns the rune with index i in the String. The sequence of runes
is the same as iterating over the contents with a "for range" clause.
func (s *String) Init(contents string) *String
Init initializes an existing String to hold the provided contents.
It returns a pointer to the initialized String.
func (s *String) IsASCII() bool
IsASCII returns a boolean indicating whether the String contains only
ASCII bytes.
func (s *String) RuneCount() int
RuneCount returns the number of runes (Unicode code points) in the
String.
func (s *String) Slice(i, j int) string
Slice returns the string sliced at rune positions [i:j].
func (s *String) String() string
String returns the contents of the String. This method also means the
String is directly printable by fmt.Print.
Fixes#2479.
R=golang-dev, dsymonds, mattn.jp, r, gri, r
CC=golang-dev
https://golang.org/cl/5472051
Gives other projects the benefit of the gofmt presubmit.
Results in minor changes in the doc/ directory:
find doc -name \*.go -exec gofmt -d {} \;
R=rsc, gri
CC=golang-dev
https://golang.org/cl/5158042
This feature should make it easier to look at very large
directory trees.
- a new mode (URL: /pkg/?m=flat) shows directory listings w/o
indentation and entries with full path (html and text mode)
- in text mode, hierarchical (non-flat) directory listings are
now presented with indentation (/pkg/?m=text)
- in html mode, hierarchical (non-flat) directory listings are
presented with slightly less indentation
- there is an internal hook for programmatic control of the
display mode (for specialized versions of godoc).
R=bradfitz
CC=golang-dev, rsc
https://golang.org/cl/5410043
Shows first line of any response that the codereview server
has identified as an LGTM. Example output below.
5305046:
big: update for fmt interface changes
Nothing terribly interesting here.
Reviewer: gri@golang.org
gri: LGTM
CC: golang-dev@googlegroups.com
Files:
src/pkg/big/int.go
src/pkg/big/nat.go
src/pkg/big/nat_test.go
src/pkg/big/rat.go
5307044:
exp/template/html: use rune
Nothing terribly interesting here.
Reviewer: mikesamuel@gmail.com, nigeltao@golang.org
mikesamuel: I don't see a type def for rune. Assuming that's a new intrinsic, LGTM.
CC: golang-dev@googlegroups.com
Files:
src/pkg/exp/template/html/css.go
src/pkg/exp/template/html/css_test.go
src/pkg/exp/template/html/html.go
src/pkg/exp/template/html/js.go
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5297045
1. Generate TOC for package pages using template,
instead of using JavaScript magic. This makes the
pages generated by godoc -html easier to export
to other systems.
2. Make TOC one column. It's hard to do two columns
portably without invoking JavaScript.
3. Since the TOC is only one column, show full type
signatures for functions and methods. Many times
that's all you need to see anyway.
4. Name the section after the TOC "Overview".
Naming it something is important, to set it off
from the TOC and so that there's a quick link to
it in the TOC.
For now, some illustrative examples:
http://swtch.com:6060/pkg/io/http://swtch.com:6060/pkg/strings/http://swtch.com:6060/pkg/tabwriter/http://swtch.com:6060/pkg/unicode/Fixes#1982.
R=gri, bradfitz, r
CC=golang-dev
https://golang.org/cl/5303044
This CL introduces the go.Example type and go.Examples functions that
are used to represent and extract code samples from Go source.
They should be of the form:
// Output of this function.
func ExampleFoo() {
fmt.Println("Output of this function.")
}
It also modifies godoc to read example code from _test.go files,
and include them in the HTML output with JavaScript-driven toggles.
It also implements testing of example functions with gotest.
The stdout/stderr is compared against the output comment on the
function.
This CL includes examples for the sort.Ints function and the
sort.SortInts type. After patching this CL in and re-building go/doc
and godoc, try
godoc -http=localhost:6060
and visit http://localhost:6060/pkg/sort/
R=gri, r, rsc
CC=golang-dev
https://golang.org/cl/5137041
Also work around Mercurial issue 3023.
If anyone has local changes in their repo (due to
patch queues or whatever) stop them from leaking
into the main repository.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5144043
Also: fix layout of textual search results and
fix a field reference in the respective template.
Fixes#1987.
R=rsc, r
CC=golang-dev
https://golang.org/cl/4962061
- KindRuns don't need to repeat SpotKind,
it is stored in each Spot
- removed extra indirection from FileRuns
to KindRuns
- slight reduction of written index size
(~500KB)
R=rsc
CC=golang-dev
https://golang.org/cl/4969052
Use naming convention for template variables
to indicate "escaped-ness" for easier reviewing.
(per suggestion from bradfitz)
R=bradfitz
CC=golang-dev
https://golang.org/cl/4914041
Since the posLink_url also adds a non-URL attribute, the quoting and URL-escaping
must happen inside posLink_url (otherwise the non-URL attribute becomes part or the
URL portion of the tag.
R=r
CC=golang-dev
https://golang.org/cl/4888041
- rename template funcs for better consistency and
sort them into groups of related functionality
- try to be more consistent with html vs url escaping
R=r
CC=golang-dev
https://golang.org/cl/4887041
- simplified pipelines
- simplified templates by using template variables
- converted most old-style formatters into new-style funcs
- fixed some escaping bugs (use of url escaping where it was missing)
R=r, dsymonds
CC=golang-dev
https://golang.org/cl/4868044
- first step; rough conversion of all template files
- there is plenty of opportunity for cleanups/simplifications (next CLs)
- html and text output as before
R=r, dsymonds
CC=golang-dev
https://golang.org/cl/4852048
Note that if you are working on the upcoming release
branch you have to point your extension path to a
copy of lib/codereview/codereview.py that won't change
as the repository flips between release-branch and default branch.
This warning should only apply to this one branch and only to rsc,
but you never know.
R=adg
CC=golang-dev
https://golang.org/cl/4446076
Set mailed bit correctly for self-clpatch.
Use repo.rollback correctly.
Allow leading spaces in some C code.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4438064
Uses placeholder attribute instead of changing the value of search
field on browsers that support it. On other browsers, the fake
placeholder text is restored when the empty box loses focus.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/4441041
In the current codereview, if a patch was written against
a version of a file that had subsequently been edited,
hg clpatch would fail, even if the patch and the edits were
in different parts of the file. In this situation the reviewer
typically wrote back saying "please hg sync and hg mail
to update the patch".
This change rewrites the patch automatically, using the
same transformation that hg sync + hg mail would.
If the interim changes (since the patch was created)
affect the same line ranges as the patch, clpatch will
still refuse to apply it. But this CL should make
of the trivial conflicts we see just go away.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4377046
Right now if a Go developer makes a patch on one machine
and then clpatches it onto another machine, changes
subsequently made to the description are kept only
locally, under the assumption that you are running
clpatch because someone else wrote the CL, so you
don't have permission to update the web.
This change makes clpatch discard the "this was a
clpatch" information from the metadata when you
clpatch your own CLs from one machine to another.
This should eliminate some confusion (for example
in CL 4314054) but will no doubt introduce other
confusion.
R=golang-dev, r2
CC=golang-dev
https://golang.org/cl/4387041
Avoid passing the placeholder diff to hgpatch, so that
clpatch-ing an empty diff grabs the metadata and warns
about it being empty, rather than failing with a
hard-to-debug problem ("mkdir: no such file or dir",
no metadata, etc).
R=rsc
CC=golang-dev
https://golang.org/cl/4172060
Neither gofmt nor godoc are making use of a Styler (for
token-specific formatting) anymore. Stylers interacted in complicated
ways with HTML-escaping which was why the printer needed an HTML mode
in the first place.
godoc now uses a more powerful and general text formatting
function that does HTML escaping, text selection, and can
handle token-specific formatting if so desired (currently
used only for comments).
As a consequence, cleaned up uses of go/printer in godoc;
simplified the various write utility functions, and also
removed the need for the "html" template format (in favor of
html-esc which now does the same and is used more pervasively).
Applied gofmt -w src misc to verify no changes occured,
and tested godoc manually.
There should be no visible changes except that (type) code
snippets presented for godoc package documentation now
uses the same formatting as for general source code and
thus comments get the comment-specific color here as well
(not the case at the moment).
(TODO: godoc needs a good automatic test suite).
R=rsc
CC=golang-dev
https://golang.org/cl/4152042
Uploading go files on Windows aborts with gofmt: exceptions.ValueError:
close_fds is not supported on Windows platforms if you redirect stdin/stdout/stderr
R=rsc, mattn, Joe Poirier
CC=golang-dev
https://golang.org/cl/4025046
If a file pattern is given and matches files that look
like they need to be hg added or hg removed, offer to do so.
If a file pattern is given and matches files in another CL, warn.
If a file pattern doesn't match anything, point that out.
Vet first line of CL description.
Fixes#972.
R=adg, niemeyer
CC=bradfitzgo, golang-dev
https://golang.org/cl/4099042
- added flag -maxresults (default: 10000) to limit the max.
number of full text results shown
- removed flag -fulltext; use -maxresults=0 to disable fulltext
index
- better indication on result page if not all results are shown
(... after line list)
R=rsc, gri1
CC=golang-dev
https://golang.org/cl/4049042
Regular expressions may now be used in conjuction with full text
search. Godoc will show the first 10000 occurences in the source
code and highlight the respective text segments.
- added new flag -testDir to specify a small directory for testing
(fast index creation; default = "")
- use new FormatText function to format text and Go source
code in HTML, supporting multiple kinds of text selections
simulatenously); this replaces the uses of go/printer
Stylers
- for now removed currently unused mechanism for identifier-
specific JS popups (will come back in some form once we
have type or other useful information)
- various typo fixes and minor cleanups throughout
Missing:
- indexing of non-.go files
R=r, r2
CC=golang-dev, rsc
https://golang.org/cl/3699041
changeset: 6839:545c9926d61a
user: Robert Griesemer <gri@golang.org>
date: Mon Dec 06 14:23:18 2010 -0800
summary: go/ast: use token.Pos instead of token.Position; adjust all dependent code
broke 'godoc -src fmt', one of the example uses of godoc. It gives a stack backtrace essentially
caused by dereferencing a NULL pointer (in C terms). This change fixes the particular issue, but
the code probably should be made more robust.
Fixes#3818044.
R=gri
CC=golang-dev
https://golang.org/cl/3818044
- improved search result page
- clicking on files shows highlighted search phrase
(.go files loose their godoc-specific formatting and
highlighting in this mode - a better solution is in
the works)
- support for textual results
- fixed bug with non-URL escaped URL parameter (Query)
R=rsc, adg
CC=golang-dev
https://golang.org/cl/3585041
To enable use -fulltext flag; e.g.: godoc -v -fulltext -http=:7777
Enabling the fulltext index will use significantly more memory as
the text of all source code, the respective suffixarray, and the
file set data structure is kept in memory. At the moment there is
about 6Mb of source code (~1400 files) indexed under GOROOT.
Source code + suffix array together consume 5*(size of source) or
about 30Mb. The file set data structure consumes about 4b/src line.
By default only up to 5000 results are shown for now.
The presentation of the results needs tuning. In particular,
if a string is found, clicking on the respective file does not
highlight them, yet.
At the moment, only Go source files are indexed. Eventually,
the full text index should encompass other files as well.
R=rsc, adg
CC=golang-dev
https://golang.org/cl/3182043
Specifically:
* lib/godoc:
- provide file set (FSet) argument to formatters where needed
* src/cmd:
- cgo, ebnflint, godoc, gofmt, goinstall: provide file set (fset) where needed
- godoc: remove local binary search with sort.Search (change by rsc),
extract file set for formatters
* src/pkg:
- exp/eval: remove embedded token.Position fields from nodes and replace
with named token.Pos fields; add corresponding Pos() accessor methods
- go/token: added file.Line(), changed signature of File.Position()
* test/fixedbugs/:
- bug206.go: change test to not rely on token.Pos details
* added various extra comments
* Runs all.bash
* gofmt formats all of src, misc w/o changes
* godoc runs
* performance:
- The new version of godoc consumes about the same space after indexing
has completed, but indexing is half the speed. Significant space savings
are expected from smaller ASTs, but since they are thrown away after a
file has been indexed, this is not visible anymore. The slower indexing
time is due to the much more expensive computation of line information.
However, with the new compressed position information, indexing can be
rewritten and simplified. Furthermore, computing the line info can be
done more efficiently.
New godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
44381 godoc 0.0% 0:38.00 4 19 149 145M 184K 148M 176M
2010/12/03 17:58:35 index updated (39.231s, 18505 unique words, 386387 spots)
2010/12/03 17:58:35 bytes=90858456 footprint=199182584
2010/12/03 17:58:36 bytes=47858568 footprint=167295224
Old godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
23167 godoc 0.0% 0:22.02 4 17 132 129M 184K 132M 173M
2010/12/03 14:51:32 index updated (24.892s, 18765 unique words, 393830 spots)
2010/12/03 14:51:32 bytes=66404528 footprint=163907832
2010/12/03 14:51:32 bytes=46282224 footprint=163907832
The different numbers for unique words/spots stem from the fact the the
two workspaces are not exactly identical. The new godoc maintains a large
file set data structure during indexing which (probably) is the reason
for the larger heap (90858456 vs 66404528) before garbage collection.
R=rsc, r
CC=golang-dev
https://golang.org/cl/3050041
The front page remains fixed-width.
All other pages should look good in windows >=500px wide.
Includes additional styles for search result highlighting,
code comments, and general CSS clean-ups.
Tested with Chrome 6, Firefox 3.6, IE 7, and IE 8.
R=gri, r
CC=golang-dev
https://golang.org/cl/2229041
End the charade (farce?) that we are using upload.py unaltered.
Cut all the unused stuff.
Indent using tabs to match the rest of the file.
Next: rewrite MercurialVCS to use mercurial package,
to avoid overhead of forking off a new hg command
multiple times for every file. And parallelize upload.
R=gri
CC=golang-dev
https://golang.org/cl/2009046
$ hg p
codereview disabled: cannot open /Users/rsc/g/go/src/pkg/goplan9.googlecode.com/hg/lib/codereview/codereview.cfg
$
R=dsymonds, r
CC=golang-dev
https://golang.org/cl/1998046
- change ast.Ident back to contain the name and adjust all dependent code
- identifier object information will be added again through an optional
typechecker phase (in the works).
- remove tracking of scopes in parser - it's easier to do this in a separate
phase (in the works)
- in godoc, generate popup info table directly instead of through a formatter
for simpler data flow (at the expense of a little bit more code)
Runs all tests.
As a result of this change, the currently shown popup information
(const, var, type, func, followed by identifier name) will not be
shown anymore temporarily.
R=rsc
CC=golang-dev
https://golang.org/cl/1994041
This is a stop-gap change to give more current information visibility
before a more thorough reorganization.
R=r, rsc, gri
CC=golang-dev
https://golang.org/cl/1902042
- change the various url-xxx formatters to return a relative URL path
- make the leading '/' for URLs explicit in the template
- on the way change some |html formatters to |html-esc
(html should only be used for formatting AST nodes)
R=rsc, r
CC=golang-dev
https://golang.org/cl/740041
Instead of returning the index lookup result via
RPC which has to be corrected for the client,
simply render it on the server and return the
final output.
R=rsc, r
CC=golang-dev
https://golang.org/cl/669041
The command-line search is using a running webserver
as index server; i.e., the search result is reflecting
the index at the server. See the documentation for
details.
Usage: godoc -q query1 query2 ...
Known issue: Results don't show the all-important
line numbers yet due to the way the index is organized.
Next CL.
R=rsc, r
CC=golang-dev
https://golang.org/cl/648041
for identifiers in Go source code
- at the moment just show identifier kind (var, func, etc.) and name
(eventually should show declaration, type, etc.)
- JavaScript parts by adg
R=rsc
CC=adg, golang-dev
https://golang.org/cl/578042
- on the commandline: godoc -x big
- in a webpage: provide form parameter ?m=src
Known issues:
- Positioning of comments incorrect in several cases. Separate CL.
- Need a link/menu to switch between different modes of presentation
in the web view.
R=rsc
CC=golang-dev
https://golang.org/cl/376041
hg.parseurl now returns a two-value tuple; codereview.py expected 3.
Changed to merely take the first return value.
R=rsc, iant
CC=golang-dev
https://golang.org/cl/223087
Example use: godoc -path=/home/user1:/home/build/foo -http=:6666
will start a local godoc that maps urls starting with /pkg/user1 or
/pkg/foo to the respective roots specified in the path.
Missing: Handling of overlapping package directories, multiple
packages per directory.
R=rsc
CC=golang-dev
https://golang.org/cl/206078
on my linux machine this is the correct one. lxml.etree
exists with an ElementTree class, but does not contain
an .XML method.
R=rsc
https://golang.org/cl/163082
- simplified dealing with parse errors: no need to intersperse them in the source
- improve visibility of highlighted identifiers by showing them in bold
R=rsc
https://golang.org/cl/163051
of limited utility but good for creating the metadata
for an AUTHORS/CONTRIBUTORS change even if
the patch doesn't apply cleanly.
R=r
https://golang.org/cl/154140
- When providing alternative spellings to a query, do not
prefix it with a package qualifier as the suggestion may
not have any results. Correctly filtering is quite a bit
of work, and clicking the alternative spelling will always
also show the qualified hits if they exist (but also others).
Seems good enough for now.
- Give user feedback when the query syntax was wrong.
- Package names in search results are now links to the respective
package documentation.
- Experimented with excluding main packages and test files
from index with inconclusive results. Code is present and
can be enabled by changing a flag in the source. This needs
some more work.
R=rsc
CC=r, dsymonds
http://go/go-review/1026033
- introduced a new run per file containing all spots belonging
to the same kind (e.g. var decl, const decl, etc.)
- more comments, better index.go file organization
R=rsc
http://go/go-review/1026028
(this was surprisingly hard to get right in HTML)
- show modification times in source directory listings
- various tweaks
R=rsc
http://go/go-review/1024024
- formatting in dirs.html is crude, needs better html (open to suggestions),
but shows the synopsis
- many package comments should probably be adjusted such that the first
sentence is more concise
R=rsc, iant
http://go/go-review/1025014
add check before Upload, for now disabled by default,
that files do not change when run through gofmt.
enable by adding
[codereview]
force_gofmt = True
to .hgrc or .hg/hgrc.
add doc strings for a few more commands.
rename codereview-login to code-login
to make module doc visible.
R=r
CC=gri
http://go/go-review/1018056
after sync (or sync --local), clean up repository:
* look for and close CLs submitted on our behalf
* remove unmodified files from CLs
* warn about empty CLs
R=r
http://go/go-review/1017029
main one is to check at submit time that
user name being used in checkin message
is listed in the CONTRIBUTORS file.
this should catch misconfigurations.
another is to cut the @domain part
from the R= and CC= lines on checkin
messages, so that cc'ing someone on
a change does not mean their email
address is recorded for all time.
R=r
CC=go-dev
http://go/go-review/1016036
just a single directory
- all pieces present but not well integrated
- directory tree served at the moment under /tree
R=rsc
http://go/go-review/1018016
