Example:
PACKAGE
package utf8
import "unicode/utf8"
Package utf8 implements functions and constants to support text
encoded in UTF-8. This package calls a Unicode character a rune for
brevity.
CONSTANTS
const (
RuneError = unicode.ReplacementChar // the "error" Rune or "replacement character".
RuneSelf = 0x80 // characters below Runeself are represented as themselves in a single byte.
UTFMax = 4 // maximum number of bytes of a UTF-8 encoded Unicode character.
)
Numbers fundamental to the encoding.
FUNCTIONS
func DecodeLastRune(p []byte) (r rune, size int)
DecodeLastRune unpacks the last UTF-8 encoding in p and returns the
rune and its width in bytes.
func DecodeLastRuneInString(s string) (r rune, size int)
DecodeLastRuneInString is like DecodeLastRune but its input is a
string.
func DecodeRune(p []byte) (r rune, size int)
DecodeRune unpacks the first UTF-8 encoding in p and returns the rune
and its width in bytes.
func DecodeRuneInString(s string) (r rune, size int)
DecodeRuneInString is like DecodeRune but its input is a string.
func EncodeRune(p []byte, r rune) int
EncodeRune writes into p (which must be large enough) the UTF-8
encoding of the rune. It returns the number of bytes written.
func FullRune(p []byte) bool
FullRune reports whether the bytes in p begin with a full UTF-8
encoding of a rune. An invalid encoding is considered a full Rune
since it will convert as a width-1 error rune.
func FullRuneInString(s string) bool
FullRuneInString is like FullRune but its input is a string.
func RuneCount(p []byte) int
RuneCount returns the number of runes in p. Erroneous and short
encodings are treated as single runes of width 1 byte.
func RuneCountInString(s string) (n int)
RuneCountInString is like RuneCount but its input is a string.
func RuneLen(r rune) int
RuneLen returns the number of bytes required to encode the rune.
func RuneStart(b byte) bool
RuneStart reports whether the byte could be the first byte of an
encoded rune. Second and subsequent bytes always have the top two
bits set to 10.
func Valid(p []byte) bool
Valid reports whether p consists entirely of valid UTF-8-encoded
runes.
func ValidString(s string) bool
ValidString reports whether s consists entirely of valid UTF-8-encoded
runes.
TYPES
type String struct {
// contains filtered or unexported fields
}
String wraps a regular string with a small structure that provides
more efficient indexing by code point index, as opposed to byte index.
Scanning incrementally forwards or backwards is O(1) per index
operation (although not as fast a range clause going forwards).
Random access is O(N) in the length of the string, but the overhead is
less than always scanning from the beginning. If the string is ASCII,
random access is O(1). Unlike the built-in string type, String has
internal mutable state and is not thread-safe.
func NewString(contents string) *String
NewString returns a new UTF-8 string with the provided contents.
func (s *String) At(i int) rune
At returns the rune with index i in the String. The sequence of runes
is the same as iterating over the contents with a "for range" clause.
func (s *String) Init(contents string) *String
Init initializes an existing String to hold the provided contents.
It returns a pointer to the initialized String.
func (s *String) IsASCII() bool
IsASCII returns a boolean indicating whether the String contains only
ASCII bytes.
func (s *String) RuneCount() int
RuneCount returns the number of runes (Unicode code points) in the
String.
func (s *String) Slice(i, j int) string
Slice returns the string sliced at rune positions [i:j].
func (s *String) String() string
String returns the contents of the String. This method also means the
String is directly printable by fmt.Print.
Fixes#2479.
R=golang-dev, dsymonds, mattn.jp, r, gri, r
CC=golang-dev
https://golang.org/cl/5472051
This feature should make it easier to look at very large
directory trees.
- a new mode (URL: /pkg/?m=flat) shows directory listings w/o
indentation and entries with full path (html and text mode)
- in text mode, hierarchical (non-flat) directory listings are
now presented with indentation (/pkg/?m=text)
- in html mode, hierarchical (non-flat) directory listings are
presented with slightly less indentation
- there is an internal hook for programmatic control of the
display mode (for specialized versions of godoc).
R=bradfitz
CC=golang-dev, rsc
https://golang.org/cl/5410043
- rename template funcs for better consistency and
sort them into groups of related functionality
- try to be more consistent with html vs url escaping
R=r
CC=golang-dev
https://golang.org/cl/4887041
- simplified pipelines
- simplified templates by using template variables
- converted most old-style formatters into new-style funcs
- fixed some escaping bugs (use of url escaping where it was missing)
R=r, dsymonds
CC=golang-dev
https://golang.org/cl/4868044
- first step; rough conversion of all template files
- there is plenty of opportunity for cleanups/simplifications (next CLs)
- html and text output as before
R=r, dsymonds
CC=golang-dev
https://golang.org/cl/4852048
changeset: 6839:545c9926d61a
user: Robert Griesemer <gri@golang.org>
date: Mon Dec 06 14:23:18 2010 -0800
summary: go/ast: use token.Pos instead of token.Position; adjust all dependent code
broke 'godoc -src fmt', one of the example uses of godoc. It gives a stack backtrace essentially
caused by dereferencing a NULL pointer (in C terms). This change fixes the particular issue, but
the code probably should be made more robust.
Fixes#3818044.
R=gri
CC=golang-dev
https://golang.org/cl/3818044
Specifically:
* lib/godoc:
- provide file set (FSet) argument to formatters where needed
* src/cmd:
- cgo, ebnflint, godoc, gofmt, goinstall: provide file set (fset) where needed
- godoc: remove local binary search with sort.Search (change by rsc),
extract file set for formatters
* src/pkg:
- exp/eval: remove embedded token.Position fields from nodes and replace
with named token.Pos fields; add corresponding Pos() accessor methods
- go/token: added file.Line(), changed signature of File.Position()
* test/fixedbugs/:
- bug206.go: change test to not rely on token.Pos details
* added various extra comments
* Runs all.bash
* gofmt formats all of src, misc w/o changes
* godoc runs
* performance:
- The new version of godoc consumes about the same space after indexing
has completed, but indexing is half the speed. Significant space savings
are expected from smaller ASTs, but since they are thrown away after a
file has been indexed, this is not visible anymore. The slower indexing
time is due to the much more expensive computation of line information.
However, with the new compressed position information, indexing can be
rewritten and simplified. Furthermore, computing the line info can be
done more efficiently.
New godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
44381 godoc 0.0% 0:38.00 4 19 149 145M 184K 148M 176M
2010/12/03 17:58:35 index updated (39.231s, 18505 unique words, 386387 spots)
2010/12/03 17:58:35 bytes=90858456 footprint=199182584
2010/12/03 17:58:36 bytes=47858568 footprint=167295224
Old godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
23167 godoc 0.0% 0:22.02 4 17 132 129M 184K 132M 173M
2010/12/03 14:51:32 index updated (24.892s, 18765 unique words, 393830 spots)
2010/12/03 14:51:32 bytes=66404528 footprint=163907832
2010/12/03 14:51:32 bytes=46282224 footprint=163907832
The different numbers for unique words/spots stem from the fact the the
two workspaces are not exactly identical. The new godoc maintains a large
file set data structure during indexing which (probably) is the reason
for the larger heap (90858456 vs 66404528) before garbage collection.
R=rsc, r
CC=golang-dev
https://golang.org/cl/3050041
- on the commandline: godoc -x big
- in a webpage: provide form parameter ?m=src
Known issues:
- Positioning of comments incorrect in several cases. Separate CL.
- Need a link/menu to switch between different modes of presentation
in the web view.
R=rsc
CC=golang-dev
https://golang.org/cl/376041
(this was surprisingly hard to get right in HTML)
- show modification times in source directory listings
- various tweaks
R=rsc
http://go/go-review/1024024
- the first HTML comment in those files is extracted as page
title when serving them
- lib/godoc.html is top-level template for all pages served
- experimented a bit with package documentation layout
(feedback welcome)
- various related cleanups
TODO:
- The / page (doc/root.html) content repeats links that are
in the navigation bar. It needs to be cleaned up.
R=rsc
DELTA=826 (86 added, 692 deleted, 48 changed)
OCL=35230
CL=35245
ast.go:
- rename Comments -> CommentGroup (less confusion)
- change all comments/docs to be *CommentGroup
filter.go:
- do not remove unassociated comments from program as part
of export filtering (they are needed by doc.go for BUG comments)
scanner.go:
- exclude '\n' from //-style comments
parser.go:
- rewrote collection of comments: now all collected comments
are *ast.CommentGroups
- clarified distinction between leading and trailing comments
- fixed a bug in comment collection (parseGenDecl);
corresponding test case is in printer/testdata
- extra documentation
doc.go:
- collect BUG comments
- corresponding fix for parser bug in comment collection
comment.go:
- simplified regex
printer.go:
- adjust comment printing to new representation
printer_test.go, testdata/*:
- enable printing of doc comments
- extended tests
package.html, package.txt:
- added Bugs section
gofmt:
- enable printing of doc comments
R=rsc
DELTA=339 (126 added, 38 deleted, 175 changed)
OCL=31403
CL=31416
