This makes sure that all JS newlines are encoded in JSON.
It also moots a TODO about possibly escaping supplemental codepoints.
I served:
Content-Type: text/javascript;charset=UTF-8
var s = "%s";
document.write("<p>", s, "</p><ol>");
for (var i = 0; i < s.length; i++) {
document.write("<li>", s.charCodeAt(i).toString(16), "</li>");
}
document.write("</l>");
where %s was replaced with bytes "\xf0\x9d\x84\x9e" to test
straight UTF-8 instead of encoding surrogates separately.
Recent Firefox, Chrome, and Safari all decoded it properly.
I have yet to try it on IE or older versions.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5129042
The template
"<a="
caused an infinite loop in escape text.
The change to tTag fixes that and the change to escape.go causes
escapeText to panic on any infinite loop that does not involve
a state cycle.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5115041
HTML parsers may differ on whether
<input id= onchange=f( ends in id's or onchange's value,
<a class=`foo ends inside a value,
<input style=font:'Arial' needs open-quote fixup.
Per
http://www.w3.org/TR/html5/tokenization.html#attribute-value-unquoted-state
this treats the error cases in 8.2.4.40 Attribute value (unquoted) state
as fatal errors.
\> U+0022 QUOTATION MARK (")
\> U+0027 APOSTROPHE (')
\> U+003C LESS-THAN SIGN (<)
\> U+003D EQUALS SIGN (=)
\> U+0060 GRAVE ACCENT (`)
Parse error. Treat it as per the "anything else" entry below.
and emits ErrBadHTML.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5085050
The Dwarf info has the full typenames, the go *struct runtime.commonType
has the short name. A more permanent fix would link the two together
but this way the user gets useable stack traces for now.
R=rsc
CC=golang-dev
https://golang.org/cl/5097046
One benefit of websocket is that it is full-duplex so that it could
send and receive at the same time.
This CL makes websocket goroutine safe, so user could use websocket
both on goroutine for read and on goroutine for write.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5058043
When templates are stored in external files, developers often embed
comments to explain&|disable code.
<!-- Oblique reference to project code name here -->
{{if .C}}...{{else}}<!-- commented out default -->{{end}}
This unnecessarily increases the size of shipped HTML and can leak
information.
This change elides all comments of the following types:
1. <!-- ... --> comments found in source.
2. /*...*/ and // comments found in <script> elements.
3. /*...*/ and // comments found in <style> elements.
It does not elide /*...*/ or // comments found in HTML attributes:
4. <button onclick="/*...*/">
5. <div style="/*...*/">
I can find no examples of comments in attributes in Closure Templates
code and doing so would require keeping track of character positions
post decode in
<button onclick="/*...*/">
To prevent token joining, /*comments*/ are JS and CSS comments are
replaced with a whitespace char.
HTML comments are not, but to prevent token joining we could try to
detect cases like
<<!---->b>
</<!---->b>
which has a well defined meaning in HTML but will cause a validator
to barf. This is difficult, and this is a very minor case.
I have punted for now, but if we need to address this case, the best
way would be to normalize '<' in stateText to '<' consistently.
The whitespace to replace a JS /*comment*/ with depends on whether
there is an embedded line terminator since
break/*
*/foo
...
is equivalent to
break;
foo
...
while
break/**/foo
...
is equivalent to
break foo;
...
Comment eliding can interfere with IE conditional comments.
http://en.wikipedia.org/wiki/Conditional_comment
<!--[if IE 6]>
<p>You are using Internet Explorer 6.</p>
<![endif]-->
/*@cc_on
document.write("You are using IE4 or higher");
@*/
I have not encountered these in production template code, and
the typed content change in CL 4962067 provides an escape-hatch
if conditional comments are needed.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4999042
This simplifies transition functions to make it easier to reliably
elide comments in a later CL.
Before:
- transition functions are responsible for detecting special end tags.
After:
- the code to detect special end tags is done in one place.
We were relying on end tags being skipped which meant we were
not noticing comments inside script/style elements that contain no
substitutions.
This change means we will notice all such comments where necessary,
but stripTags will notice none since it does not need to. This speeds
up stripTags.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5074041
The documentation for bytes.Replace says it copies
the slice but it won't necessarily copy them. Since
the data is mutable, breaking the contract is an issue.
We either have to fix this by making the copy at all
times, as suggested in this CL, or we should change the
documentation and perhaps make better use of the fact
it's fine to mutate the slice in place otherwise.
R=golang-dev, bradfitz, adg, rsc
CC=golang-dev
https://golang.org/cl/5081043
Use gobs to serialize indexes instead of encoding/binary.
Even with gobs, serialize data in slices instead of
applying gob to the entire data structure at once,
to reduce the amount of extra buffer memory needed
inside gob.
7x faster Write/Read for new BenchmarkSaveRestore
compared to old code; possibly because encoding/binary
is more expensive for int32 slice elements (interface
call to get little/big endian encoding), while gob's
encoding is fixed (unconfirmed).
new (using gobs):
suffixarray.BenchmarkSaveRestore 1 2153604000 ns/op
old (using encoding/binary):
suffixarray.BenchmarkSaveRestore 1 15118322000 ns/op
The actual serialized data is slightly larger then using
the old code for very large indices because full 32bit indices
require 5bytes using gobs instead of 4bytes (encoding/binary)
in serialized form.
R=r
CC=golang-dev
https://golang.org/cl/5087041
This CL generalises the pair of halfConnection members that the
serverConn holds into a single transport struct that is shared by
both Server and Client, see also CL 5037047.
This CL is a replacement for 5040046 which I closed by accident.
R=agl, bradfitz
CC=golang-dev
https://golang.org/cl/5075042
Formulaic changes to transition functions in preparation for CL 5074041.
This should be completely semantics preserving.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5091041
Instead of erroring on actions inside comments, use existing escaping
pipeline to quash the output of actions inside comments.
If a template maintainer uses a comment to disable template code:
{{if .}}Hello, {{.}}!{{end}}
->
<!--{{if true}}Hello, {{.}}!{{end}}-->
will result in
<!--Hello, !-->
regardless of the value of {{.}}.
In a later CL, comment elision will result in the entire commented-out
section being dropped from the template output.
Any side-effects in pipelines, such as panics, will still be realized.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5078041
MFENCE was introduced only on the Pentium4 (SSE2),
while XADD was introduced on the 486.
Fixes#2268.
R=golang-dev, rsc
CC=fshahriar, golang-dev
https://golang.org/cl/5056045
Previously /etc/hosts would be ignored altogether, this change returns matching results
from that file without talking to a DNS server.
R=rsc
CC=golang-dev
https://golang.org/cl/5061042
This addresses several use cases:
(1) <h{{.HeaderLevel}}> used to build hierarchical documents.
(2) <input on{{.EventType}}=...> used in widgets.
(3) <div {{" dir=ltr"}}> used to embed bidi-hints.
It also makes sure that we treat the two templates below the same:
<img src={{if .Avatar}}"{{.Avatar}}"{{else}}"anonymous.png"{{end}}>
<img src="{{if .Avatar}}{{.Avatar}}{{else}}anonymous.png{{end}}">
This splits up tTag into a number of sub-states and adds testcases.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5043042
The template
<{{.}}
would violate the structure preservation property if allowed and not
normalized, because when {{.}} emitted "", the "<" would be part of
a text node, but if {{.}} emitted "a", the "<" would not be part of
a text node.
This change rewrites '<' in text nodes and RCDATA text nodes to
'<' allowing template authors to write the common, and arguably more
readable:
Your price: {{.P1}} < list price {{.P2}}
while preserving the structure preservation property.
It also lays the groundwork for comment elision, rewriting
Foo <!-- comment with secret project details --> Bar
to
Foo Bar
R=nigeltao
CC=golang-dev
https://golang.org/cl/5043043
The typical UNIX method for controlling long running process is to
send the process signals. Since this doesn't get you very far, various
ad-hoc, remote-control protocols have been used over time by programs
like Apache and BIND.
Implementing an SSH server means that Go code will have a standard,
secure way to do this in the future.
R=bradfitz, borman, dave, gustavo, dsymonds, r, adg, rsc, rogpeppe, lvd, kevlar, raul.san
CC=golang-dev
https://golang.org/cl/4962064
gotest src/pkg/exp/template/html was crashing because the exception handler overflowed the goroutine stack.
R=alex.brainman, golang-dev
CC=golang-dev
https://golang.org/cl/5031049
filepath.Glob is documented to return nil if no files match
and an error only if the pattern is invalid. This change
fixes it to work as documented and adds a regression test.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5040045
clean up duplicate environment for CGI.
overriding former by latter.
On windows, When there are duplicated environments like following,
SCRIPT_FILENAME=c:/progra~1/php/php-cgi.exe
SCRIPT_FILENAME=/foo.php
CreateProcess use first entry.
If make cgi.Handle like following,
cgih = cgi.Handler{
Path: "c:/strawberry/perl/bin/perl.exe",
Dir: "c:/path/to/webroot",
Root: "c:/path/to/webroot",
Args: []string{"foo.php"},
Env: []string{"SCRIPT_FILENAME=foo.php"},
}
http/cgi should behave "SCRIPT_FILENAME is foo.php".
But currently, http/cgi is set duplicate environment entries.
So, browser show binary dump of "php-cgi.exe" that is specified indented
SCRIPT_FILENAME in first entry.
This change clean up duplicates, and use latters.
R=golang-dev, bradfitz, bradfitz
CC=golang-dev
https://golang.org/cl/5010044
I found a simple test case that does require doing the fixed point TODO
in computeOutCtx.
I found a way though to do this and simplify away the escapeRange
hackiness that was added in https://golang.org/cl/5012044/
R=nigeltao
CC=golang-dev
https://golang.org/cl/5015052
This replaces the errStr & errLine members of context with a single err
*Error, and introduces a number of const error codes, one per
escape-time failure mode, that can be separately documented.
The changes to the error documentation moved from doc.go to error.go
are cosmetic.
R=r, nigeltao
CC=golang-dev
https://golang.org/cl/5026041
The go/build package already recognizes
system-specific file names like
mycode_darwin.go
mycode_darwin_386.go
mycode_386.s
However, it is also common to write files that
apply to multiple architectures, so a recent CL added
to go/build the ability to process comments
listing a set of conditions for building. For example:
// +build darwin freebsd openbsd/386
says that this file should be compiled only on
OS X, FreeBSD, or 32-bit x86 OpenBSD systems.
These conventions are not yet documented
(hence this long CL description).
This CL adds build comments to the multi-system
files in the core library, a step toward making it
possible to use go/build to build them.
With this change go/build can handle crypto/rand,
exec, net, path/filepath, os/user, and time.
os and syscall need additional adjustments.
R=golang-dev, r, gri, r, gustavo
CC=golang-dev
https://golang.org/cl/5011046
Not all content is plain text. Sometimes content comes from a trusted
source, such as another template invocation, an HTML tag whitelister,
etc.
Template authors can deal with over-escaping in two ways.
1) They can encapsulate known-safe content via
type HTML, type CSS, type URL, and friends in content.go.
2) If they know that the for a particular action never needs escaping
then they can add |noescape to the pipeline.
{{.KnownSafeContent | noescape}}
which will prevent any escaping directives from being added.
This CL defines string type aliases: HTML, CSS, JS, URI, ...
It then modifies stringify to unpack the content type.
Finally it modifies the escaping functions to use the content type and
decline to escape content that does not require it.
There are minor changes to escapeAction and helpers to treat as
equivalent explicit escaping directives such as "html" and "urlquery"
and the escaping directives defined in the contextual autoescape module
and to recognize the special "noescape" directive.
The html escaping functions are rearranged. Instead of having one
escaping function used in each {{.}} in
{{.}} : <textarea title="{{.}}">{{.}}</textarea>
a slightly different escaping function is used for each.
When {{.}} binds to a pre-sanitized string of HTML
`one < <i>two</i> & two < "3"`
we produces something like
one < <i>two</i> & two < "3" :
<textarea title="one < two & two < "3"">
one < <i>two</i> & two < "3"
</textarea>
Although escaping is not required in <textarea> normally, if the
substring </textarea> is injected, then it breaks, so we normalize
special characters in RCDATA and do the same to preserve attribute
boundaries. We also strip tags since developers never intend
typed HTML injected in an attribute to contain tags escaped, but
do occasionally confuse pre-escaped HTML with HTML from a
tag-whitelister.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/4962067
This moots a caveat in the proposed package documentation by
rendering useless any template that could not be escaped.
From https://golang.org/cl/4969078/
> If EscapeSet returns an error, do not Execute the set; it is not
> safe against injection.
r: [but isn't the returned set nil? i guess you don't overwrite the
r: original if there's a problem, but i think you're in your rights to
r: do so]
R=r
CC=golang-dev
https://golang.org/cl/5020043
The Windows implementation of the net package churns through a couple of channels for every read/write operation. This translates into a lot of time spent in the kernel creating and deleting event objects.
R=rsc, dvyukov, alex.brainman, jp
CC=golang-dev
https://golang.org/cl/4997044
It would be nice not to have to support this since all the clients
that we care about support TLSv1 by now. However, due to buggy
implementations of SSLv3 on the Internet which can't do version
negotiation correctly, browsers will sometimes switch to SSLv3. Since
there's no good way for a browser tell a network problem from a buggy
server, this downgrade can occur even if the server in question is
actually working correctly.
So we need to support SSLv3 for robustness :(
Fixes#1703.
R=bradfitz
CC=golang-dev
https://golang.org/cl/5018045
The template package is the only one that has a doc.go not mentioned
in its Makefile.
This doesn't seem to bother godoc, but seems like a bug to me.
$ for d in $(find pkg -name doc.go); do echo $d; grep doc.go $(dirname $d)/Makefile; done
pkg/fmt/doc.go
doc.go\
pkg/go/doc/doc.go
doc.go\
pkg/gob/doc.go
doc.go\
pkg/html/doc.go
doc.go\
pkg/old/template/doc.go
doc.go\
pkg/sync/atomic/doc.go
doc.go\
pkg/template/doc.go
R=r
CC=golang-dev
https://golang.org/cl/5003047
This one uses a closure than an interface, and is much simpler to use.
It also enables a called function to return an error and (possibly)
halt processing.
Fixes#2237.
R=golang-dev, gri, rsc, r, cw, n13m3y3r
CC=golang-dev
https://golang.org/cl/5014043
This adds support for {{template "callee"}} calls.
It recognizes that calls can appear in many contexts.
{{if .ImageURL}}
<img src="{{.ImageURL}}" alt="{{template "description"}}">
{{else}}
<p>{{template "description"}}</p>
{{end}}
calls a template in two different contexts, first in an HTML attribute
context, and second in an HTML text context.
Those two contexts aren't very different, but when linking text
to search terms, the escaping context can be materially different:
<a href="/search?q={{template "tags"}}">{{template "tags"}}</a>
This adds API:
EscapeSet(*template.Set, names ...string) os.Error
takes a set of templates and the names of those which might be called
in the default context as starting points.
It changes the escape* functions to be methods of an object which
maintains a conceptual mapping of
(template names*input context) -> output context.
The actual mapping uses as key a mangled name which combines the
template name with the input context.
The mangled name when the input context is the default context is the
same as the unmangled name.
When a template is called in multiple contexts, we clone the template.
{{define "tagLink"}}
<a href="/search?q={{template "tags"}}">{{template "tags"}}</a>
{{end}}
{{define "tags"}}
{{range .Tags}}{{.}},{{end}}
{{end}}
given []string{ "foo", "O'Reilly", "bar" } produces
<a href="/search?q=foo,O%27Reilly,bar">foo,O'Reilly,bar</a>
This involves rewriting the above to something like
{{define "tagLink"}}
<a href="/search?q={{template "tags$1"}}">{{template "tags"}}</a>
{{end}}
{{define "tags"}}
{{range .Tags}}{{. | html}},{{end}}
{{end}}
{{define "tags$1"}}
{{range .Tags}}{{. | urlquery}},{{end}}
{{end}}
clone.go provides a mechanism for cloning template "tags" to produce
"tags$1".
changes to escape.go implement the new API and context propagation
around the call graph.
context.go includes minor changes to support name mangling and
context_test.go tests those.
js.go contains a bug-fix.
R=nigeltao, r
CC=golang-dev
https://golang.org/cl/4969072
Often, division/regexp ambiguity doesn't matter in JS because the next
token is not a slash.
For example, in
<script>var global{{if .InitVal}} = {{.InitVal}}{{end}}</script>
When there is an initial value, the {{if}} ends with jsCtxDivOp
since a '/' following {{.InitVal}} would be a division operator.
When there is none, the empty {{else}} branch ends with jsCtxRegexp
since a '/' would start a regular expression. A '/' could result
in a valid program if it were on a new line to allow semicolon
insertion to terminate the VarDeclaration.
There is no '/' though, so we can ignore the ambiguity.
There are cases where a missing semi can result in ambiguity that
we should report.
<script>
{{if .X}}var x = {{.X}}{{end}}
/...{{.Y}}
</script>
where ... could be /foo/.test(bar) or /divisor. Disambiguating in
this case is hard and is required to sanitize {{.Y}}.
Note, that in the case where there is a '/' in the script tail but it
is not followed by any interpolation, we already don't care. So we
are already tolerant of
<script>{{if .X}}var x = {{.X}}{{end}}/a-bunch-of-text</script>
because tJS checks for </script> before looking in /a-bunch-of-text.
This CL
- Adds a jsCtx value: jsCtxUnknown
- Changes joinContext to join contexts that only differ by jsCtx.
- Changes tJS to return an error when a '/' is seen in jsCtxUnknown.
- Adds tests for both the happy and sad cases.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4956077
Weekday is redundant information for a Time structure.
When parsing a time with a weekday specified, it can create an
incorrect Time value.
When parsing a time without a weekday specified, people
expect the weekday to be set.
Fix all three problems by computing the weekday on demand.
This is hard to gofix, since we must change the type of the node.
Since uses are rare and existing code will be caught by the compiler,
there is no gofix module here.
Fixes#2245.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4974077
Augments type context and adds grammatical rules to handle special HTML constructs:
<!-- comments -->
<script>raw text</script>
<textarea>no tags here</textarea>
This CL does not elide comment content. I recommend we do that but
have not done it in this CL.
I used a codesearch tool over a codebase in another template language.
Based on the below I think we should definitely recognize
<script>, <style>, <textarea>, and <title>
as each of these appears frequently enough that there are few
template using apps that do not use most of them.
Of the other special tags,
<xmp>, <noscript>
are used but infrequently, and
<noframe> and friend, <listing>
do not appear at all.
We could support <xmp> even though it is obsolete in HTML5
because we already have the machinery, but I suggest we do not
support noscript since it is a normal tag in some browser
configurations.
I suggest recognizing and eliding <!-- comments -->
(but not escaping text spans) as they are widely used to
embed comments in template source. Not eliding them increases
the size of content sent over the network, and risks leaking
code and project internal details.
The template language I tested elides them so there are
no instance of IE conditional compilation directives in the
codebase but that could be a source of confusion.
The codesearch does the equivalent of
$ find . -name \*.file-extension \
| perl -ne 'print "\L$1\n" while s@<([a-z][a-z0-9])@@i' \
| sort | uniq -c | sort
The 5 uses of <plaintext> seem to be in tricky code and can be ignored.
The 2 uses of <xmp> appear in the same tricky code and can be ignored.
I also ignored end tags to avoid biasing against unary
elements and threw out some nonsense names since since the
long tail is dominated by uses of < as a comparison operator
in the template languages expression language.
I have added asterisks next to abnormal elements.
26765 div
7432 span
7414 td
4233 a
3730 tr
3238 input
2102 br
1756 li
1755 img
1674 table
1388 p
1311 th
1064 option
992 b
891 label
714 script *
519 ul
446 tbody
412 button
381 form
377 h2
358 select
353 strong
318 h3
314 body
303 html
266 link
262 textarea *
261 head
258 meta
225 title *
189 h1
176 col
156 style *
151 hr
119 iframe
103 h4
101 pre
100 dt
98 thead
90 dd
83 map
80 i
69 object
66 ol
65 em
60 param
60 font
57 fieldset
51 string
51 field
51 center
44 bidi
37 kbd
35 legend
30 nobr
29 dl
28 var
26 small
21 cite
21 base
20 embed
19 colgroup
12 u
12 canvas
10 sup
10 rect
10 optgroup
10 noscript *
9 wbr
9 blockquote
8 tfoot
8 code
8 caption
8 abbr
7 msg
6 tt
6 text
6 h5
5 svg
5 plaintext *
5 article
4 shortquote
4 number
4 menu
4 ins
3 progress
3 header
3 content
3 bool
3 audio
3 attribute
3 acronym
2 xmp *
2 overwrite
2 objects
2 nobreak
2 metadata
2 description
2 datasource
2 category
2 action
R=nigeltao
CC=golang-dev
https://golang.org/cl/4964045
This does not wire up <style> elements as that is pending support
for raw text content in CL https://golang.org/cl/4964045/
This CL allows actions to appear in contexts like
selectors: {{.Tag}}{{.Class}}{{.Id}}
property names: border-{{.BidiLeadingEdge}}
property values: color: {{.Color}}
strings: font-family: "{{font-name}}"
URL strings: background: "/foo?image={{.ImgQuery}}"
URL literals: background: url("{{.Image}}")
but disallows actions inside CSS comments and disallows
embedding of JS in CSS entirely.
It is based on the CSS3 lexical grammar with affordances for
common browser extensions including line comments.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4968058
I don't know the protocol regarding the zsyscall files which appear to
be hand-generated, so I've re-done them and added them to the change.
R=rsc, alex.brainman, nigeltao
CC=golang-dev
https://golang.org/cl/4975060
API change. Needs further reflection.
««« original CL description
path/filepath: Simplify Walk interface
The last argument of filepath.Walk was removed, and the Visitor
interface now contains an Error method that is called on errors.
Fixes#2237.
R=golang-dev, gri, r
CC=golang-dev
https://golang.org/cl/4964067
»»»
R=r
CC=golang-dev
https://golang.org/cl/4974065
The last argument of filepath.Walk was removed, and the Visitor
interface now contains an Error method that is called on errors.
Fixes#2237.
R=golang-dev, gri, r
CC=golang-dev
https://golang.org/cl/4964067
Also add exp/regexp to build (forgot before).
At this point I am very confident in exp/regexp's
behavior. It should be usable as a drop-in
replacement for regexp now.
Later CLs could introduce a CompilePOSIX
to get at traditional POSIX ``extended regular expressions''
as in egrep and also an re.MatchLongest method to
change the matching mode to leftmost longest
instead of leftmost first. On the other hand, I expect
very few people to use either.
R=r, r, gustavo
CC=golang-dev
https://golang.org/cl/4990041
Note that this CL will break your existing code which uses
ParseCIDR.
This CL changes ParseCIDR("172.16.253.121/28") to return
the IP address "172.16.253.121", the network implied by the
network number "172.16.253.112" and mask "255.255.255.240".
R=rsc, borman
CC=golang-dev
https://golang.org/cl/4749043
- func f(int,) is a legal signature
- func f(...int,) is a legal signature
Defer checking for correct use of "..." with last
paremeter type to type checker instead of parser.
R=rsc
CC=golang-dev
https://golang.org/cl/4973059
There may be more fine-tuning down the line,
but this CL fixes the most pressing issue at
hand.
Also: gofmt -w src misc
Fixes#1524.
R=rsc, bradfitz
CC=golang-dev
https://golang.org/cl/4975053
Most web frameworks allow ; as a synonym for &,
following a recommendation in some versions of
the HTML specification. Do the same.
Remove overuse of Split.
Move ParseQuery tests from package http to package url.
Fixes#2210.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4973062
Needed to ensure that finding the last boundary does not result in O(n^2)-like behavior.
Now prevents lookbacks beyond 31 characters across the board (starter + 30 non-starters).
composition.go:
- maxCombiningCharacters now means exactly that.
- Bug fix.
- Small performance improvement/ made code consistent with other code.
forminfo.go:
- Bug fix: ccc needs to be 0 for inert runes.
normalize.go:
- A few bug fixes.
- Limit the amount of combining characters considered in FirstBoundary.
- Ditto for LastBoundary.
- Changed semantics of LastBoundary to not consider trailing illegal runes a boundary
as long as adding bytes might still make them legal.
trie.go:
- As utf8.UTFMax is 4, we should treat UTF-8 encodings of size 5 or greater as illegal.
This has no impact on the normalization process, but it prevents buffer overflows
where we expect at most UTFMax bytes.
R=r
CC=golang-dev
https://golang.org/cl/4963041
Interesting that Fprintf can do zero mallocs.
(Sprintf must allocate the returned string.)
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/4977049
This transitions into a JS state when entering any attribute whose
name starts with "on".
It does not yet enter a JS on entry into a <script> element as script
element handling is introduced in another CL.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4968052
My string literal was being rewritten from
"runtime.SysReserve(%p, %D) = error %d"
to
"runtime.SysReserve ( %p , %D ) = error %d"
R=iant
CC=golang-dev
https://golang.org/cl/4972051
1. adds a urlPart field to context
2. implements tURL to figure out the URL part
3. modifies joinContext to allow common context mismatches
around branches to be ignored when not material as in
<a href="/foo{{if .HasQuery}}?q={{.Query}}{{/if}}">
4. adds a pipeline function that filters dynamically inserted
protocols to prevent code injection via URLs.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4957041
release.r50 looks for newest tag <= go.r50
weekly.2010-10-10 looks for newest tag <= go.2010-10-10
Implements behavior for hg, git, and bzr.
R=dsymonds, rsc, n13m3y3r
CC=golang-dev
https://golang.org/cl/4873057
Since JavaScript doesn't have [u]int64 types, some JSON APIs
encode such types as strings to avoid losing precision.
This adds a new struct tag option ",string" to cause
fields to be wrapped in JSON strings on encoding
and unwrapped from strings when decoding.
R=rsc, gustavo
CC=golang-dev
https://golang.org/cl/4918051
This should allow symlinks in tar files. Where previously
as far as I can see they were skipped completely.
R=golang-dev, dsymonds, rsc, bradfitz, bradfitz
CC=golang-dev, mike.rosset
https://golang.org/cl/4973044
This CL changes the internal form of IPMask for IPv4
from 16-byte to 4-byte, also adds Size method to IPMask
struct and changes output string format of IPMask.String
method.
R=rsc
CC=golang-dev
https://golang.org/cl/4950046
Add openbsd 386 syscall support, partially based on the existing
freebsd 386 syscall implementation.
FTR zerrors_openbsd_386.go cannot currently be completely built on
openbsd/i386 due to what appears to be a gcc bug. The constants can be
successfully generated with -m32 on openbsd/amd64 and the error
table can then be generated on openbsd/i386.
R=rsc
CC=golang-dev
https://golang.org/cl/4969045
- Rename sys_sched_yield() to osyield() as this is now defined in asm.h.
- Only print kern.rtheads message if rfork_thread() failed with ENOTSUP.
- Remove unused variables.
R=rsc
CC=golang-dev
https://golang.org/cl/4973043
cdecl calbacks have been implemented in C/ASM code, just Go function is missing
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/4969047
-s now means *disable* escape analysis.
Fix escape leaks for struct/slice/map literals.
Add ... tracking.
Rewrite new(T) and slice literal into stack allocation when safe.
Add annotations to reflect.
Reflect is too chummy with the compiler,
so changes like these affect it more than they should.
R=lvd, dave, gustavo
CC=golang-dev
https://golang.org/cl/4954043
The fix is to add ' ' after ';' so that we match
what we used to generate.
Packages like http look for the string with
the space in it, and I don't see a reason to
be so terse.
Also s/buffer/b/
TBR=bradfitz
CC=golang-dev
https://golang.org/cl/4959044
Doing a socket/listen on an unspecified address with an unspecified
address family is likely to result in an AF_INET6 socket on an IPv6
capable system, which under OpenBSD means IPv6 only - not IPv4 *and*
IPv6. In this case trying to connect to this socket from an IPv4
loopback address is not going to end well.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4807057
When the encoder was updated to respect the ",omitempty"
struct tag options, the decoder half was never updated to know
about the new struct tag format. (the format is now an optional
name, followed by zero or more ",option" strings)
This only affected people who used ",omitempty" along with
a field name. In that case, the serialized JSON wouldn't
decode to the original value.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/4965049
This introduces new APIs.
- DialConfig can open client connection using Config, so user can specify protocol version, tls.Config if necessary.
- Message can be used to send/receive text/binary data in a frame.
- JSON can be used to send/receive JSON data in a frame.
R=golang-dev, adg, rsc, m, tarmigan, raul.san, yohcop
CC=golang-dev
https://golang.org/cl/4635084
Write to image.*.Pix directly in the case of RGB, RGBA and NRGBA
images. For the latter two, the file format matches the memory layout
so a simple copy can be used.
RGB image before/after:
tiff.BenchmarkDecoder 748137 ns/op (62.39 MB/s) 251256 ns/op (185.76 MB/s) x3.0
NRGBA image before/after:
tiff.BenchmarkDecoder 775540 ns/op (80.12 MB/s) 116721 ns/op (532.34 MB/s) x6.6
R=nigeltao
CC=golang-dev
https://golang.org/cl/4929046
This fixes goinstall so it doesn't try to install unneeded
packages or get confused with non-existent loops.
R=golang-dev, adg, gustavo
CC=golang-dev
https://golang.org/cl/4958046
This makes decimal a good test
case for the escape analysis.
With escape analysis:
benchmark old ns/op new ns/op delta
BenchmarkAtof64Decimal 1954 243 -87.56%
BenchmarkAtof64Float 2008 293 -85.41%
BenchmarkAtof64FloatExp 10106 8814 -12.78%
BenchmarkAtof64Big 5113 3486 -31.82%
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/4861042
The following testcases now pass:
`<a href=x` tests that we do not error on partial unquoted attrs.
`<a href=x ` tests that spaces do end unquoted attrs on spaces.
`<a href=''` tests that we recognize the end of single quoted attrs.
`<a href=""` tests that we recognize the end of double quoted attrs.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/4932051
Pow10 failed for MinInt32 (endless loop until out of
memory). Fix by returning 0 and +Inf for all arguments
where the result is not representable in a float64.
Fixes#2159.
R=rsc
CC=golang-dev
https://golang.org/cl/4930041
My theory is that suse has larger TCP buffer sizes
by default. I now check over 100MB, rather than over 2MB.
100MB is ~halfway between the 1MB limit and the 200MB
request that's attempted.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4926048
This allows code that wants to handle
[]byte separately to get at the actual slice
instead of just at individual bytes.
It seems to come up often enough.
R=r
CC=golang-dev
https://golang.org/cl/4942051
A side-effect is that, just like foo_386.go
is only built on 386, foo_386_test.go is only
built for testing on 386.
R=adg, r, mattn.jp
CC=golang-dev
https://golang.org/cl/4942050
The kludge is targeted at broken web browsers
like Chrome and IE, but it gets in the way of
sending 400 or 500-series error results with
formatted bodies in response to AJAX requests
made by pages executing in those browsers.
Now the AJAX cases will work and Chrome
and IE will be as broken with Go servers as
they are with all the other servers.
Fixes#2169.
R=bradfitz, dsymonds
CC=golang-dev
https://golang.org/cl/4930047
CL 4938041 made some incorrect changes to the filter
function which caused a different doc/codelab/wiki/index.html
file to be generated.
Added FilterFileExports and FilterPackageExports function.
Same as the existing FileExpors/PackageExports functions
but using shared code. The old functions will be removed
in the next CL.
R=r, rsc
CC=golang-dev
https://golang.org/cl/4932048
Introduce extra scope for the variable declared by a
TypeSwitchGuard so that it doesn't conflict with vars
declared by the initial SimpleStmt of a type switch.
This is a replacement for CL 4896053 which caused
a build breakage.
Also:
- explicitly detect type switches (as opposed to detecting
expression switches and then do extra testing for type switches)
- fix all outstanding TODOs in parser.go
- ran all tests
R=rsc
CC=golang-dev
https://golang.org/cl/4914044
This adds http.MaxBytesReader, similar to io.LimitReader,
but specific to http, and for preventing a class of DoS
attacks.
This also makes the 10MB ParseForm limit optional (if
not already set by a MaxBytesReader), documents it,
and also adds "PUT" as a valid verb for parsing forms
in the request body.
Improves issue 2093 (DoS protection)
Fixes#2165 (PUT form parsing)
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/4921049
Make the stack traces more readable for new
Go programmers while preserving their utility for old hands.
- Change status number [4] to string.
- Elide frames in runtime package (internal details).
- Swap file:line and arguments.
- Drop 'created by' for main goroutine.
- Show goroutines in order of allocation:
implies main goroutine first if nothing else.
There is no option to get the extra frames back.
Uncomment 'return 1' at the bottom of symtab.c.
$ 6.out
throw: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:22 +0x8a
goroutine 2 [select (no cases)]:
main.sel()
/Users/rsc/g/go/src/pkg/runtime/x.go:11 +0x18
created by main.main
/Users/rsc/g/go/src/pkg/runtime/x.go:19 +0x23
goroutine 3 [chan receive]:
main.recv(0xf8400010a0, 0x0)
/Users/rsc/g/go/src/pkg/runtime/x.go:15 +0x2e
created by main.main
/Users/rsc/g/go/src/pkg/runtime/x.go:20 +0x50
goroutine 4 [chan receive (nil chan)]:
main.recv(0x0, 0x0)
/Users/rsc/g/go/src/pkg/runtime/x.go:15 +0x2e
created by main.main
/Users/rsc/g/go/src/pkg/runtime/x.go:21 +0x66
$
$ 6.out index
panic: runtime error: index out of range
goroutine 1 [running]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:25 +0xb9
$
$ 6.out nil
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x22ca]
goroutine 1 [running]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:28 +0x211
$
$ 6.out panic
panic: panic
goroutine 1 [running]:
main.main()
/Users/rsc/g/go/src/pkg/runtime/x.go:30 +0x101
$
R=golang-dev, qyzhai, n13m3y3r, r
CC=golang-dev
https://golang.org/cl/4907048
Add support for syscalls on openbsd amd64. This is based on the
existing freebsd amd64 implementation.
R=mikioh.mikioh, rsc, yourcomputerpal
CC=golang-dev
https://golang.org/cl/4798060
ast.FilterFile(src, ast.IsExported) has the same
effect as ast.FileExports(src) with this change.
1st step towards removing FileExports - it is
just a special case of FilterFile with this CL.
Added corresponding test.
R=r
CC=golang-dev
https://golang.org/cl/4938041
maketables.go/tables.go
- Properly set combinesForward flag for JamoL and JamoV.
- Fixed Printf bug.
composition.go
- Make insertString use the same control flow as insert.
- Better Hangul and non-Hangul mixing.
forminfo.go
- Fixed bug in compBoundaryBefore that affected a few esoteric cases.
- Buffer overflow now tested in normalize_test.go (other CL).
R=r
CC=golang-dev
https://golang.org/cl/4924041
ParseMediaType previously documented that it always returned
a non-nil map, but also documented that it returned a nil map
to signal an error.
That is confusing, contradictory and not Go-like.
Now it returns (mediatype string, params map, os.Error).
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4867054
The subtle AST changes introduced with CL 4896053
broke type checking of type switches in gofix.
Coming up with a correct fix will take some time.
Undoing this change for now.
««« original CL description
go/parser: fix type switch scoping
The variable declared by a TypeSwitchGuard must be
visible in each TypeCaseClause and must not conflict
with other variables declared by the initial SimpleStmt
of a type switch.
Also:
- explicitly detect type switches (as opposed to detecting
regular (expression switches) and then do extra testing
for type switches
- fix all outstanding TODOs in parser.go
R=rsc
CC=golang-dev
https://golang.org/cl/4896053
»»»
R=rsc
CC=golang-dev
https://golang.org/cl/4902052
The variable declared by a TypeSwitchGuard must be
visible in each TypeCaseClause and must not conflict
with other variables declared by the initial SimpleStmt
of a type switch.
Also:
- explicitly detect type switches (as opposed to detecting
regular (expression switches) and then do extra testing
for type switches
- fix all outstanding TODOs in parser.go
R=rsc
CC=golang-dev
https://golang.org/cl/4896053
This CL changes both JoinGroup and LeaveGroup methods
to take an interface as an argument for enabling IPv6
group address join/leave, join a group address on a
specific interface.
R=rsc, dave
CC=golang-dev
https://golang.org/cl/4815074
Allocate Defer on stack during cgo calls, as suggested
by dvyukov. Also includes some comment corrections.
benchmark old,ns/op new,ns/op
BenchmarkCgoCall 669 330
(Intel Xeon CPU 1.80GHz * 4, Linux 386)
R=dvyukov, rsc
CC=golang-dev
https://golang.org/cl/4910041
This defines just enough context to distinguish HTML URI attributes
from parsed character data.
It does not affect any public module API as I thought I would get
early comment on style for defining enumerations and tables.
R=rsc, r, nigeltao, r
CC=golang-dev
https://golang.org/cl/4906043
All tests enabled by default passes except those in timeout_test.go.
For TestLookupPort, add an entry for "bootps" in /lib/ndb/common
(Plan 9 calls it "bootp"). I've sent out a patch to fix this.
R=paulzhol, rsc, mikioh.mikioh
CC=ality, golang-dev
https://golang.org/cl/4779041
forminfo.go:
- Wrappers for table data.
- Per Form dispatch table.
composition.go:
- reorderBuffer type. Implements decomposition, reordering, and composition.
- Note: decompose and decomposeString fields in formInfo could be replaced by
a pointer to the trie for the respective form. The proposed design makes
testing easier, though.
normalization.go:
- Temporarily added panic("not implemented") methods to make the tests run.
These will be removed again with the next CL, which will introduce the
implementation.
R=r, rogpeppe, mpvl, rsc
CC=golang-dev
https://golang.org/cl/4875043
Replaces the toy func Reverse(*Template) with one that implements
naive autoescaping.
Now Escape(*Template) walks a template parse tree to find all
template actions and adds the |html command to them if it is not
already present.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4867049
It's already in old/template; make that build.
Update a couple of references to point to the old template.
They can be updated later.
Update goplay to use exp/template.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4902046
This is just moving the URL code from package http into its own package,
which has been planned for a while.
Besides clarity, this also breaks a nascent dependency cycle the new template
package was about to introduce.
Add a gofix module, url, and use it to generate changes outside http and url.
Sadness about the churn, gladness about some of the naming improvements.
R=dsymonds, bradfitz, rsc, gustavo, r
CC=golang-dev
https://golang.org/cl/4893043
Multicast address handling is not consistent across all BSDs. Move
the multicast address handling code into OS dependent files. This
will be needed for OpenBSD support.
R=mikioh.mikioh, golang-dev
CC=golang-dev
https://golang.org/cl/4809074
The corruption can occur when GOMAXPROCS
is changed from >1 to 1, since GOMAXPROCS=1
does not imply there is only 1 goroutine running,
other goroutines can still be not parked after
the change.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4873050
Currently it's possible to write:
var s rpc.Server
...
// reuse for my own purposes
s.Lock()
...
s.Unlock()
which is seemingly not intended.
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/4888049
This is useful for applications that want to micromanage the rpc service.
Moved part of ServeCodec into a new readRequest function.
Renamed existing readRequest to readRequestHeader, and reordered
its parameters to align with the new readRequest and service.call.
R=golang-dev, r, rsc, sougou
CC=golang-dev, msolomon
https://golang.org/cl/4889043
Do this by adding a fake package called builtin. At the moment, godoc will
not present this documentation because the function names are all lower case,
but there are plans to address this.
The print and println functions are undocumented here. I could be talked
into doing them but I'd rather not promote their use.
R=dsymonds, n13m3y3r, r, gri, rsc
CC=golang-dev
https://golang.org/cl/4907041
This was initially pushed as part of CL 4876046, found
when logic in exp/template was using the method on
an Invalid value.
R=rsc
CC=golang-dev
https://golang.org/cl/4890043
First step of moving exp/template into template: save the old code.
Code is unedited except for target name in Makefile.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4893042
While using exp/template in practice, the syntax for
indexing values using the "index" action was found to be
very inconvenient for frequent use when handling dynamic
data from maps such as the ones used with json and yaml,
that use a type like map[string]interface{}.
For these kinds of maps, the default handling of fields as
{{.Field}} makes the task of handling the several references
significantly more pleasant and elegant, and is equivalent
to what's currently done in the "template" package and in
other external packages (e.g. mustache).
Even with this change, the index action is still relevant
as it allows indexing maps in other scenarios where keys
wouldn't be valid field names.
R=golang-dev, r, gustavo
CC=golang-dev
https://golang.org/cl/4898043
This avoids a non-obvious panic when range is used on a
nil interface, and fixes it by behaving as if the range
was empty.
The new behavior is equivalent to the outcome of iterating
on a nil map or slice, and is useful because it allows
generic structures such as used in json (map[string]interface{})
to behave correctly if a key generally set to a list or map
isn't present.
R=golang-dev, r, gustavo
CC=golang-dev
https://golang.org/cl/4876046
func Reverse(*Template) *Template
returns a template that produces the reverse of the original
for any input.
Changes outside exp/template/html include:
- Adding a getter for a template's FuncMap so that derived templates
can inherit function definitions.
- Exported one node factory function, newIdentifier.
Deriving tempaltes requires constructing new nodes, but I didn't
export all of them because I think shallow copy functions might
be more useful for this kind of work.
- Bugfix: Template's Name() method ignores the name field so
template.New("foo") is a nil dereference instead of "foo".
Caveats: Reverse is a toy. It is not UTF-8 safe, and does not
preserve order of calls to funcs in FuncMap.
For context, see http://groups.google.com/group/golang-nuts/browse_thread/thread/e8bc7c771aae3f20/b1ac41dc6f609b6e?lnk=gst
R=rsc, r, nigeltao, r
CC=golang-dev
https://golang.org/cl/4808089
This continues the work in revision 914a659b44ff, now passing more test
cases. As before, the new tokenization tests match html5lib's behavior.
Fixes#2124.
R=dsymonds, r
CC=golang-dev
https://golang.org/cl/4867042
IMPORTANT: Previous usage of *Files will continue to compile
fine but misbehave since the interface is compatible.
The following functions have been renamed:
ParseFiles => ParseGlob
ParseFile => ParseFiles
ParseSetFiles => ParseSetGlob
ParseSetFile => ParseSetFiles
ParseTemplateFiles => ParseTemplateGlob
ParseTemplateFile => ParseTemplateFiles
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/4867041
Fixes#2140.
The http.DefaultTransport's RoundTrip method leaves the http.Request object
in an altered state after performing the round trip. This patch removes
the header from the Request before returning to the client.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/4857041
- triegen.go: Factored out trie generation code from maketables.go
(only renamed printTrieTables to printTables and made it a method).
- maketesttables.go: new tool to generate data for the trie unit test.
- Makefile: changed accordingly.
- trie.go: trie lookup code.
- trietest_data.go: generated by maketesttables.go.
- trie_test.go: unit test for trie.go.
R=r
CC=golang-dev
https://golang.org/cl/4844053
- triegen.go: Factored out trie generation code from maketables.go
(only renamed printTrieTables to printTables and made it a method).
- maketesttables.go: new tool to generate data for the trie unit test.
- Makefile: changed accordingly.
- trie.go: trie lookup code.
- trietest_data.go: generated by maketesttables.go.
- trie_test.go: unit test for trie.go.
R=r
CC=golang-dev
https://golang.org/cl/4844053
Every time we enter callback from Windows, it is
possible that go exception handler is not at the top
of per-thread exception handlers chain. So it needs
to be installed again. At this moment this is done
by replacing top SEH frame with SEH frame as at time
of syscall for the time of callback. This is incorrect,
because, if exception strike, we won't be able to call
any exception handlers installed inside syscall,
because they are not in the chain. This changes
procedure to add new SEH frame on top of existing
chain instead.
I also removed m sehframe field, because I don't
think it is needed. We use single global exception
handler everywhere.
R=golang-dev, r
CC=golang-dev, hectorchu
https://golang.org/cl/4832060
Mostly a mechanical change, with a few cleanups to make the split easier.
The external interface to exp/template is unaffected.
In another round I will play with the function map setup to see if I can
avoid exposing reflect across the boundary, but that will require some
structural changes I did not want to mix into this CL.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4849049
The mmap system call varies across BSDs. Move mmap and munmap into
the operating system dependent files. This will be needed to add
syscall support for OpenBSD.
R=rsc
CC=golang-dev
https://golang.org/cl/4822056
Add support for the go runtime on openbsd/amd64. This is based on
the existing freebsd runtime.
Threads are implemented using OpenBSD's rthreads, which are currently
disabled by default, however can be enabled via the kern.rthreads
sysctl.
For now, cgo is disabled.
R=rsc
CC=golang-dev
https://golang.org/cl/4815067
The change adds specialized type algorithms
for slices and types of size 8/16/32/64/128.
It significantly accelerates chan and map operations
for most builtin types as well as user structs.
benchmark old,ns/op new,ns/op
BenchmarkChanUncontended 226 94
(on Intel Xeon E5620, 2.4GHz, Linux 64 bit)
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4815087
calling filepath.FromSlash(x) make invalid character to serve file.
Fixes#2128
R=golang-dev, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4810064
This CL enables to list the multicast, joined group addresses
for a specific interface by using Interface.MulticastAddrs
method.
R=rsc
CC=golang-dev
https://golang.org/cl/4808062
The test case input is "<!DOCTYPE html><span><button>foo</span>bar".
The correct parse is:
| <!DOCTYPE html>
| <html>
| <head>
| <body>
| <span>
| <button>
| "foobar"
R=gri
CC=golang-dev
https://golang.org/cl/4794063
Before this change, syscall package would load
all dlls used anywhere in the go tree on startup.
For example, this program:
package main
import "fmt"
func main() {
fmt.Printf("Hello world\n")
}
would load these dlls
kernel32.dll
advapi32.dll
shell32.dll
wsock32.dll
ws2_32.dll
dnsapi.dll
iphlpapi.dll
Most of these dlls are network related and are not used
in this program. Now the same program loads only
kernel32.dll
shell32.dll
This decreases start times somewhat.
This also relaxes the rules of which dlls can be included
in the standard library. We could now include system calls
that are not available on all versions of Windows, because
we could decide if we should call them during runtime.
R=rsc, vcc.163
CC=golang-dev
https://golang.org/cl/4815046