Also add exp/regexp to build (forgot before).
At this point I am very confident in exp/regexp's
behavior. It should be usable as a drop-in
replacement for regexp now.
Later CLs could introduce a CompilePOSIX
to get at traditional POSIX ``extended regular expressions''
as in egrep and also an re.MatchLongest method to
change the matching mode to leftmost longest
instead of leftmost first. On the other hand, I expect
very few people to use either.
R=r, r, gustavo
CC=golang-dev
https://golang.org/cl/4990041
Needed to ensure that finding the last boundary does not result in O(n^2)-like behavior.
Now prevents lookbacks beyond 31 characters across the board (starter + 30 non-starters).
composition.go:
- maxCombiningCharacters now means exactly that.
- Bug fix.
- Small performance improvement/ made code consistent with other code.
forminfo.go:
- Bug fix: ccc needs to be 0 for inert runes.
normalize.go:
- A few bug fixes.
- Limit the amount of combining characters considered in FirstBoundary.
- Ditto for LastBoundary.
- Changed semantics of LastBoundary to not consider trailing illegal runes a boundary
as long as adding bytes might still make them legal.
trie.go:
- As utf8.UTFMax is 4, we should treat UTF-8 encodings of size 5 or greater as illegal.
This has no impact on the normalization process, but it prevents buffer overflows
where we expect at most UTFMax bytes.
R=r
CC=golang-dev
https://golang.org/cl/4963041
This transitions into a JS state when entering any attribute whose
name starts with "on".
It does not yet enter a JS on entry into a <script> element as script
element handling is introduced in another CL.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4968052
1. adds a urlPart field to context
2. implements tURL to figure out the URL part
3. modifies joinContext to allow common context mismatches
around branches to be ignored when not material as in
<a href="/foo{{if .HasQuery}}?q={{.Query}}{{/if}}">
4. adds a pipeline function that filters dynamically inserted
protocols to prevent code injection via URLs.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4957041
The following testcases now pass:
`<a href=x` tests that we do not error on partial unquoted attrs.
`<a href=x ` tests that spaces do end unquoted attrs on spaces.
`<a href=''` tests that we recognize the end of single quoted attrs.
`<a href=""` tests that we recognize the end of double quoted attrs.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/4932051
maketables.go/tables.go
- Properly set combinesForward flag for JamoL and JamoV.
- Fixed Printf bug.
composition.go
- Make insertString use the same control flow as insert.
- Better Hangul and non-Hangul mixing.
forminfo.go
- Fixed bug in compBoundaryBefore that affected a few esoteric cases.
- Buffer overflow now tested in normalize_test.go (other CL).
R=r
CC=golang-dev
https://golang.org/cl/4924041
This defines just enough context to distinguish HTML URI attributes
from parsed character data.
It does not affect any public module API as I thought I would get
early comment on style for defining enumerations and tables.
R=rsc, r, nigeltao, r
CC=golang-dev
https://golang.org/cl/4906043
forminfo.go:
- Wrappers for table data.
- Per Form dispatch table.
composition.go:
- reorderBuffer type. Implements decomposition, reordering, and composition.
- Note: decompose and decomposeString fields in formInfo could be replaced by
a pointer to the trie for the respective form. The proposed design makes
testing easier, though.
normalization.go:
- Temporarily added panic("not implemented") methods to make the tests run.
These will be removed again with the next CL, which will introduce the
implementation.
R=r, rogpeppe, mpvl, rsc
CC=golang-dev
https://golang.org/cl/4875043
Replaces the toy func Reverse(*Template) with one that implements
naive autoescaping.
Now Escape(*Template) walks a template parse tree to find all
template actions and adds the |html command to them if it is not
already present.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4867049
This is just moving the URL code from package http into its own package,
which has been planned for a while.
Besides clarity, this also breaks a nascent dependency cycle the new template
package was about to introduce.
Add a gofix module, url, and use it to generate changes outside http and url.
Sadness about the churn, gladness about some of the naming improvements.
R=dsymonds, bradfitz, rsc, gustavo, r
CC=golang-dev
https://golang.org/cl/4893043
While using exp/template in practice, the syntax for
indexing values using the "index" action was found to be
very inconvenient for frequent use when handling dynamic
data from maps such as the ones used with json and yaml,
that use a type like map[string]interface{}.
For these kinds of maps, the default handling of fields as
{{.Field}} makes the task of handling the several references
significantly more pleasant and elegant, and is equivalent
to what's currently done in the "template" package and in
other external packages (e.g. mustache).
Even with this change, the index action is still relevant
as it allows indexing maps in other scenarios where keys
wouldn't be valid field names.
R=golang-dev, r, gustavo
CC=golang-dev
https://golang.org/cl/4898043
This avoids a non-obvious panic when range is used on a
nil interface, and fixes it by behaving as if the range
was empty.
The new behavior is equivalent to the outcome of iterating
on a nil map or slice, and is useful because it allows
generic structures such as used in json (map[string]interface{})
to behave correctly if a key generally set to a list or map
isn't present.
R=golang-dev, r, gustavo
CC=golang-dev
https://golang.org/cl/4876046
func Reverse(*Template) *Template
returns a template that produces the reverse of the original
for any input.
Changes outside exp/template/html include:
- Adding a getter for a template's FuncMap so that derived templates
can inherit function definitions.
- Exported one node factory function, newIdentifier.
Deriving tempaltes requires constructing new nodes, but I didn't
export all of them because I think shallow copy functions might
be more useful for this kind of work.
- Bugfix: Template's Name() method ignores the name field so
template.New("foo") is a nil dereference instead of "foo".
Caveats: Reverse is a toy. It is not UTF-8 safe, and does not
preserve order of calls to funcs in FuncMap.
For context, see http://groups.google.com/group/golang-nuts/browse_thread/thread/e8bc7c771aae3f20/b1ac41dc6f609b6e?lnk=gst
R=rsc, r, nigeltao, r
CC=golang-dev
https://golang.org/cl/4808089
IMPORTANT: Previous usage of *Files will continue to compile
fine but misbehave since the interface is compatible.
The following functions have been renamed:
ParseFiles => ParseGlob
ParseFile => ParseFiles
ParseSetFiles => ParseSetGlob
ParseSetFile => ParseSetFiles
ParseTemplateFiles => ParseTemplateGlob
ParseTemplateFile => ParseTemplateFiles
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4867041
- triegen.go: Factored out trie generation code from maketables.go
(only renamed printTrieTables to printTables and made it a method).
- maketesttables.go: new tool to generate data for the trie unit test.
- Makefile: changed accordingly.
- trie.go: trie lookup code.
- trietest_data.go: generated by maketesttables.go.
- trie_test.go: unit test for trie.go.
R=r
CC=golang-dev
https://golang.org/cl/4844053
- triegen.go: Factored out trie generation code from maketables.go
(only renamed printTrieTables to printTables and made it a method).
- maketesttables.go: new tool to generate data for the trie unit test.
- Makefile: changed accordingly.
- trie.go: trie lookup code.
- trietest_data.go: generated by maketesttables.go.
- trie_test.go: unit test for trie.go.
R=r
CC=golang-dev
https://golang.org/cl/4844053
Mostly a mechanical change, with a few cleanups to make the split easier.
The external interface to exp/template is unaffected.
In another round I will play with the function map setup to see if I can
avoid exposing reflect across the boundary, but that will require some
structural changes I did not want to mix into this CL.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4849049
Before this change, syscall package would load
all dlls used anywhere in the go tree on startup.
For example, this program:
package main
import "fmt"
func main() {
fmt.Printf("Hello world\n")
}
would load these dlls
kernel32.dll
advapi32.dll
shell32.dll
wsock32.dll
ws2_32.dll
dnsapi.dll
iphlpapi.dll
Most of these dlls are network related and are not used
in this program. Now the same program loads only
kernel32.dll
shell32.dll
This decreases start times somewhat.
This also relaxes the rules of which dlls can be included
in the standard library. We could now include system calls
that are not available on all versions of Windows, because
we could decide if we should call them during runtime.
R=rsc, vcc.163
CC=golang-dev
https://golang.org/cl/4815046
- create a pair of "Must" functions, one for templates, one for sets
- regularize the return values of the parsers so they can be wrapped by Must
- delete all the old Must functions and methods.
R=golang-dev, gri, r
CC=golang-dev
https://golang.org/cl/4826052
This brings them into line with the Template.MustParse* methods
and makes it possible to use them in global variable initializations.
R=r
CC=golang-dev
https://golang.org/cl/4798059
Still need to write tests for new syntax
and fix bugs that the tests find, but this
is a good check point.
All tests pass.
Compared against existing regexp:
benchmark old ns/op new ns/op delta
regexp.BenchmarkLiteral 1869 620 -66.83%
regexp.BenchmarkNotLiteral 9489 7823 -17.56%
regexp.BenchmarkMatchClass 10372 8386 -19.15%
regexp.BenchmarkMatchClass_InRange 10800 7750 -28.24%
regexp.BenchmarkReplaceAll 13492 8519 -36.86%
regexp.BenchmarkAnchoredLiteralShortNonMatch 747 339 -54.62%
regexp.BenchmarkAnchoredLiteralLongNonMatch 599 335 -44.07%
regexp.BenchmarkAnchoredShortMatch 2137 917 -57.09%
regexp.BenchmarkAnchoredLongMatch 2029 917 -54.81%
R=r, r
CC=golang-dev, sam.thorogood
https://golang.org/cl/4820046
This is the last piece (I hope) of the set creation code.
These helpers create sets from files containing individual
template definitions, free of {{define}} clauses. This
design is helpful if the templates live one per file,
undecorated.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4801052
Also make the Set.Parse methods variadic so you can parse static lists of files without loops.
R=rsc, dsymonds, r
CC=golang-dev
https://golang.org/cl/4802051
This simplifies the API and makes it easier to make the template
invocation statically secure, at the cost of some minor flexibility.
R=golang-dev, dsymonds, r
CC=golang-dev
https://golang.org/cl/4794045
interface field can be unpacked. We don't have type assertions here so we
must be forthright.
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/4757047
The previous CL doicumented and diagnosed the old situation.
This one changes it to something more traditional: any action
may declare a variable, and the block structure of scopes
applies only to control seequences.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4748047
{{range $key, $element := pipeline}}
This CL is smaller than it looks due to some rearrangement and renaming.
R=rsc, r
CC=golang-dev
https://golang.org/cl/4709047
Angle brackets can trigger some browser sniffers,
causing some output to be interpreted as HTML.
Escaping angle brackets closes that security hole.
R=r
CC=golang-dev
https://golang.org/cl/4714044
Russ suggested this technique, making the "and" and "or" functions handier.
But it's hacky, and I can be talked out of it.
R=dsymonds, rsc
CC=golang-dev
https://golang.org/cl/4698044
- template invocation is by string constant only.
- NewSet is gone.
- no global Funcs
- writer is now first arg to Execute
R=rsc, r
CC=golang-dev
https://golang.org/cl/4700043
Must keep dot and the receiver separate - variables broke that symmetry.
Also clean up function invocation and rename "data" to "dot" for clarity.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4678048
Not strictly necessary (you could achieve the same, clumsily,
via with blocks) but great to have: $x.Field, $y.Method.
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/4678047
We can avoid the check against empty constants (''),
which UnquoteChar doesn't handle well, by leaving on
the trailing quote and seeing that's all we have left at the end.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4657090
of the origin.
image/png and image/jpeg benchmarks show no significant changes.
The image/draw changes suggest to me that making a gofix for this is not
feasible. People are just going to have to make manual fixes.
R=r
CC=golang-dev
https://golang.org/cl/4681044
1) Make the value optional ({{template "foo"}})
2) Allow the template identifier to be a thing of type *Template.
The second makes it easy to drop templates in to a set dynamically
during invocation.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4671056
It was an ill-advised carryover from the previous template package.
Also clean up function evaluation.
Also add a Name method to Template.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/4657088
- fix line numbers - forgot to update state.line during execution
- add a comment convention {{/* comment */}}
- set.Template returns the named template in the set
- set.Execute executes the named template in the set
- use a local methodByName so this package can be used with earlier release of reflect.
- use initial cap to detect exported names
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/4668054
Add the ability to attach functions to template and template sets.
Make variadic functions and methods work.
Still to come: static checking of function names during parse.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4643068
Also reuse of *Regexp nodes.
I believe this is the end of the parser.
The only non-execution code that remains is
the code to expand x{3,5} into simpler operations.
R=sam.thorogood, r
CC=golang-dev
https://golang.org/cl/4629078
including evaluation up the data tree (in this code all fields must be
in dot itself), plus more control structure, but the basics are in place.
R=rsc, r
CC=golang-dev
https://golang.org/cl/4665041
Change the signature of Split to have no count,
assuming a full split, and rename the existing
Split with a count to SplitN.
Do the same to package bytes.
Add a gofix module.
R=adg, dsymonds, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4661051
Still TODO: parsing optimizations
make_perl_groups.pl is copied with minimal modifications
(just to generate Go syntax instead of C++) from RE2.
Google Inc is "The RE2 Author" of that file and is one of
the Go Authors, so copyright changed to the Go Authors instead.
R=sam.thorogood, r, fvbommel, robert.hencke
CC=golang-dev
https://golang.org/cl/4612041
Parser is a work in progress but can populate most of the
interesting parts of the data structure, so a good checkpoint.
All the complicated Perl syntax is missing, as are various
important optimizations made during parsing to the
syntax tree.
The plan is that exp/regexp's API will mimic regexp,
and exp/regexp/syntax provides the parser directly
for programs that need it (and for implementing exp/regexp).
Once finished, exp/regexp will replace regexp.
R=r, sam.thorogood, kevlar, edsrzf
CC=golang-dev
https://golang.org/cl/4538123
An unusual design using slice and a goroutine makes for a
compact scanner with very little allocation.
R=rsc, r
CC=golang-dev, kevlar
https://golang.org/cl/4610041
On my laptop, I had an 800x600 jpeg and an 800x600 png (with
transparency). I timed how long it took to draw each image onto an
equivalently sized, zeroed RGBA image.
Previously, the jpeg took 75ms and the png took 70ms, going through
the medium-fast path, i.e. func drawRGBA in draw.go.
After this CL, the jpeg took 14ms, and the png took 21ms with the
Over operator and 12ms with the Src operator.
It's only a rough estimate basd on one image file, but it should
give an idea of the order of magnitude of improvement.
R=rsc, r
CC=adg, golang-dev
https://golang.org/cl/4468044
We replace the current Open with:
OpenFile(name, flag, perm) // same as old Open
Open(name) // same as old Open(name, O_RDONLY, 0)
Create(name) // same as old Open(name, O_RDWR|O_TRUNC|O_CREAT, 0666)
This CL includes a gofix module and full code updates: all.bash passes.
(There may be a few comments I missed.)
The interesting packages are:
gofix
os
Everything else is automatically generated except for hand tweaks to:
src/pkg/io/ioutil/ioutil.go
src/pkg/io/ioutil/tempfile.go
src/pkg/crypto/tls/generate_cert.go
src/cmd/goyacc/goyacc.go
src/cmd/goyacc/units.y
R=golang-dev, bradfitzwork, rsc, r2
CC=golang-dev
https://golang.org/cl/4357052
The parser populates all scopes for a given file (except
type-related scopes for structs, interfaces, and methods
of types) at parse time.
A new parser flag, DeclarationErrors, enables error messages
related to declaration errors (as far as it is possible to
provide them).
The resulting AST has all (non-field, non-method) identifiers
resolved that can be resolved w/o doing imports or parsing
missing package files.
The ast.File node contains the (partially complete)
package scope and a list of unresolved global identifiers.
All type-specific data structures have been removed from the AST.
The existing typechecker is functional but needs to be adjusted
(simplified) accordingly. Utility functions to resolve all
identifiers for a package (after handling imports and parsing
all package files) are missing.
Unrelated changes:
- Rename typechecker/testdata files to that they are not considered
by gofmt.
- Minor cleanups/simplifications.
Parses all .go files in src and misc without declaration errors.
Runs all tests. Changes do not affect gofmt output.
R=rsc
CC=golang-dev
https://golang.org/cl/4244049
There are some minor irregularities in the printer
output (some paren's are present that should be
removed), but these are unrelated issues.
Will review in a 2nd step.
R=rsc
CC=golang-dev
https://golang.org/cl/4188068
In line with other functions such as Fprintf, put the
thing to be written first.
Apologies for the breakages this is sure to cause.
R=rsc, gri, adg, eds, r2, aam
CC=golang-dev
https://golang.org/cl/4169042
The callback mechanism has been made more flexible.
Eliminated one round of argument copying in Syscall.
Faster Get/SetLastError implemented.
Added gettime for gc perf profiling.
R=rsc, brainman, mattn, rog
CC=golang-dev
https://golang.org/cl/4058046
Until now, each scan of a file added a new file to the file set.
With this change, a file can be re-scanned using the same *token.File
w/o changing the file set. Eventually this will enable the re-use of
cached source code in godoc (for the fulltext index). At the moment,
source files are read over and over again from disk.
This is the first step in that direction.
R=r, rsc, r2
CC=golang-dev
https://golang.org/cl/4001041
- add End() method to all nodes; the text range of a node n is [n.Pos(), n.End())
- various small bug fixes in the process
- fixed several comments
R=r, rsc
CC=golang-dev
https://golang.org/cl/3769042
It was only used by exp/4s, and even if it is general purpose, I think
it belongs in a graphics library atop exp/draw, not in exp/draw itself.
R=rsc
CC=golang-dev
https://golang.org/cl/3705041
The recent linker changes broke NaCl support
a month ago, and there are no known users of it.
The NaCl code can always be recovered from the
repository history.
R=adg, r
CC=golang-dev
https://golang.org/cl/3671042
Specifically:
* lib/godoc:
- provide file set (FSet) argument to formatters where needed
* src/cmd:
- cgo, ebnflint, godoc, gofmt, goinstall: provide file set (fset) where needed
- godoc: remove local binary search with sort.Search (change by rsc),
extract file set for formatters
* src/pkg:
- exp/eval: remove embedded token.Position fields from nodes and replace
with named token.Pos fields; add corresponding Pos() accessor methods
- go/token: added file.Line(), changed signature of File.Position()
* test/fixedbugs/:
- bug206.go: change test to not rely on token.Pos details
* added various extra comments
* Runs all.bash
* gofmt formats all of src, misc w/o changes
* godoc runs
* performance:
- The new version of godoc consumes about the same space after indexing
has completed, but indexing is half the speed. Significant space savings
are expected from smaller ASTs, but since they are thrown away after a
file has been indexed, this is not visible anymore. The slower indexing
time is due to the much more expensive computation of line information.
However, with the new compressed position information, indexing can be
rewritten and simplified. Furthermore, computing the line info can be
done more efficiently.
New godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
44381 godoc 0.0% 0:38.00 4 19 149 145M 184K 148M 176M
2010/12/03 17:58:35 index updated (39.231s, 18505 unique words, 386387 spots)
2010/12/03 17:58:35 bytes=90858456 footprint=199182584
2010/12/03 17:58:36 bytes=47858568 footprint=167295224
Old godoc, immediately after indexing completed (best of three runs):
PID COMMAND %CPU TIME #TH #PRTS #MREGS RPRVT RSHRD RSIZE VSIZE
23167 godoc 0.0% 0:22.02 4 17 132 129M 184K 132M 173M
2010/12/03 14:51:32 index updated (24.892s, 18765 unique words, 393830 spots)
2010/12/03 14:51:32 bytes=66404528 footprint=163907832
2010/12/03 14:51:32 bytes=46282224 footprint=163907832
The different numbers for unique words/spots stem from the fact the the
two workspaces are not exactly identical. The new godoc maintains a large
file set data structure during indexing which (probably) is the reason
for the larger heap (90858456 vs 66404528) before garbage collection.
R=rsc, r
CC=golang-dev
https://golang.org/cl/3050041
Rather than updating the stripped-down regexp implementation embedded
in testing, delete it by passing the one function we need from the package
main file created by gotest.
R=rsc
CC=golang-dev
https://golang.org/cl/2761043
New logging interface simplifies and generalizes.
1) Loggers now have only one output.
2) log.Stdout, Stderr, Crash and friends are gone.
Logging is now always to standard error by default.
3) log.Panic* replaces log.Crash*.
4) Exiting and panicking are not part of the logger's state; instead
the functions Exit* and Panic* simply call Exit or panic after
printing.
5) There is now one 'standard logger'. Instead of calling Stderr,
use Print etc. There are now triples, by analogy with fmt:
Print, Println, Printf
What was log.Stderr is now best represented by log.Println,
since there are now separate Print and Println functions
(and methods).
6) New functions SetOutput, SetFlags, and SetPrefix allow global
editing of the standard logger's properties. This is new
functionality. For instance, one can call
log.SetFlags(log.Lshortfile|log.Ltime|log.Lmicroseconds)
to get all logging output to show file name, line number, and
time stamp.
In short, for most purposes
log.Stderr -> log.Println or log.Print
log.Stderrf -> log.Printf
log.Crash -> log.Panicln or log.Panic
log.Crashf -> log.Panicf
log.Exit -> log.Exitln or log.Exit
log.Exitf -> log.Exitf (no change)
This has a slight breakage: since loggers now write only to one
output, existing calls to log.New() need to delete the second argument.
Also, custom loggers with exit or panic properties will need to be
reworked.
All package code updated to new interface.
The test has been reworked somewhat.
The old interface will be removed after the new release.
For now, its elements are marked 'deprecated' in their comments.
Fixes#1184.
R=rsc
CC=golang-dev
https://golang.org/cl/2419042
Package iterable has outlived its utility.
It is an interesting demonstration, but it encourages
people to use iteration over channels where simple
iteration over array indices or a linked list would be
cheaper, simpler, and have fewer races.
R=dsymonds, r
CC=golang-dev
https://golang.org/cl/2436041