Not all content is plain text. Sometimes content comes from a trusted
source, such as another template invocation, an HTML tag whitelister,
etc.
Template authors can deal with over-escaping in two ways.
1) They can encapsulate known-safe content via
type HTML, type CSS, type URL, and friends in content.go.
2) If they know that the for a particular action never needs escaping
then they can add |noescape to the pipeline.
{{.KnownSafeContent | noescape}}
which will prevent any escaping directives from being added.
This CL defines string type aliases: HTML, CSS, JS, URI, ...
It then modifies stringify to unpack the content type.
Finally it modifies the escaping functions to use the content type and
decline to escape content that does not require it.
There are minor changes to escapeAction and helpers to treat as
equivalent explicit escaping directives such as "html" and "urlquery"
and the escaping directives defined in the contextual autoescape module
and to recognize the special "noescape" directive.
The html escaping functions are rearranged. Instead of having one
escaping function used in each {{.}} in
{{.}} : <textarea title="{{.}}">{{.}}</textarea>
a slightly different escaping function is used for each.
When {{.}} binds to a pre-sanitized string of HTML
`one < <i>two</i> & two < "3"`
we produces something like
one < <i>two</i> & two < "3" :
<textarea title="one < two & two < "3"">
one < <i>two</i> & two < "3"
</textarea>
Although escaping is not required in <textarea> normally, if the
substring </textarea> is injected, then it breaks, so we normalize
special characters in RCDATA and do the same to preserve attribute
boundaries. We also strip tags since developers never intend
typed HTML injected in an attribute to contain tags escaped, but
do occasionally confuse pre-escaped HTML with HTML from a
tag-whitelister.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/4962067
This moots a caveat in the proposed package documentation by
rendering useless any template that could not be escaped.
From https://golang.org/cl/4969078/
> If EscapeSet returns an error, do not Execute the set; it is not
> safe against injection.
r: [but isn't the returned set nil? i guess you don't overwrite the
r: original if there's a problem, but i think you're in your rights to
r: do so]
R=r
CC=golang-dev
https://golang.org/cl/5020043
The Windows implementation of the net package churns through a couple of channels for every read/write operation. This translates into a lot of time spent in the kernel creating and deleting event objects.
R=rsc, dvyukov, alex.brainman, jp
CC=golang-dev
https://golang.org/cl/4997044
It would be nice not to have to support this since all the clients
that we care about support TLSv1 by now. However, due to buggy
implementations of SSLv3 on the Internet which can't do version
negotiation correctly, browsers will sometimes switch to SSLv3. Since
there's no good way for a browser tell a network problem from a buggy
server, this downgrade can occur even if the server in question is
actually working correctly.
So we need to support SSLv3 for robustness :(
Fixes#1703.
R=bradfitz
CC=golang-dev
https://golang.org/cl/5018045
The spec says that all methods are inherited from an anonymous
field. There is no exception for non-exported methods.
This is related to issue 1536.
R=rsc
CC=golang-dev
https://golang.org/cl/5012043
The template package is the only one that has a doc.go not mentioned
in its Makefile.
This doesn't seem to bother godoc, but seems like a bug to me.
$ for d in $(find pkg -name doc.go); do echo $d; grep doc.go $(dirname $d)/Makefile; done
pkg/fmt/doc.go
doc.go\
pkg/go/doc/doc.go
doc.go\
pkg/gob/doc.go
doc.go\
pkg/html/doc.go
doc.go\
pkg/old/template/doc.go
doc.go\
pkg/sync/atomic/doc.go
doc.go\
pkg/template/doc.go
R=r
CC=golang-dev
https://golang.org/cl/5003047
This one uses a closure than an interface, and is much simpler to use.
It also enables a called function to return an error and (possibly)
halt processing.
Fixes#2237.
R=golang-dev, gri, rsc, r, cw, n13m3y3r
CC=golang-dev
https://golang.org/cl/5014043
This adds support for {{template "callee"}} calls.
It recognizes that calls can appear in many contexts.
{{if .ImageURL}}
<img src="{{.ImageURL}}" alt="{{template "description"}}">
{{else}}
<p>{{template "description"}}</p>
{{end}}
calls a template in two different contexts, first in an HTML attribute
context, and second in an HTML text context.
Those two contexts aren't very different, but when linking text
to search terms, the escaping context can be materially different:
<a href="/search?q={{template "tags"}}">{{template "tags"}}</a>
This adds API:
EscapeSet(*template.Set, names ...string) os.Error
takes a set of templates and the names of those which might be called
in the default context as starting points.
It changes the escape* functions to be methods of an object which
maintains a conceptual mapping of
(template names*input context) -> output context.
The actual mapping uses as key a mangled name which combines the
template name with the input context.
The mangled name when the input context is the default context is the
same as the unmangled name.
When a template is called in multiple contexts, we clone the template.
{{define "tagLink"}}
<a href="/search?q={{template "tags"}}">{{template "tags"}}</a>
{{end}}
{{define "tags"}}
{{range .Tags}}{{.}},{{end}}
{{end}}
given []string{ "foo", "O'Reilly", "bar" } produces
<a href="/search?q=foo,O%27Reilly,bar">foo,O'Reilly,bar</a>
This involves rewriting the above to something like
{{define "tagLink"}}
<a href="/search?q={{template "tags$1"}}">{{template "tags"}}</a>
{{end}}
{{define "tags"}}
{{range .Tags}}{{. | html}},{{end}}
{{end}}
{{define "tags$1"}}
{{range .Tags}}{{. | urlquery}},{{end}}
{{end}}
clone.go provides a mechanism for cloning template "tags" to produce
"tags$1".
changes to escape.go implement the new API and context propagation
around the call graph.
context.go includes minor changes to support name mangling and
context_test.go tests those.
js.go contains a bug-fix.
R=nigeltao, r
CC=golang-dev
https://golang.org/cl/4969072
Often, division/regexp ambiguity doesn't matter in JS because the next
token is not a slash.
For example, in
<script>var global{{if .InitVal}} = {{.InitVal}}{{end}}</script>
When there is an initial value, the {{if}} ends with jsCtxDivOp
since a '/' following {{.InitVal}} would be a division operator.
When there is none, the empty {{else}} branch ends with jsCtxRegexp
since a '/' would start a regular expression. A '/' could result
in a valid program if it were on a new line to allow semicolon
insertion to terminate the VarDeclaration.
There is no '/' though, so we can ignore the ambiguity.
There are cases where a missing semi can result in ambiguity that
we should report.
<script>
{{if .X}}var x = {{.X}}{{end}}
/...{{.Y}}
</script>
where ... could be /foo/.test(bar) or /divisor. Disambiguating in
this case is hard and is required to sanitize {{.Y}}.
Note, that in the case where there is a '/' in the script tail but it
is not followed by any interpolation, we already don't care. So we
are already tolerant of
<script>{{if .X}}var x = {{.X}}{{end}}/a-bunch-of-text</script>
because tJS checks for </script> before looking in /a-bunch-of-text.
This CL
- Adds a jsCtx value: jsCtxUnknown
- Changes joinContext to join contexts that only differ by jsCtx.
- Changes tJS to return an error when a '/' is seen in jsCtxUnknown.
- Adds tests for both the happy and sad cases.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4956077
This makes it possible to grab a block of code
in an editor and pipe it through gofmt, instead of
having to pipe in the entire file.
R=gri
CC=golang-dev
https://golang.org/cl/4973074
Weekday is redundant information for a Time structure.
When parsing a time with a weekday specified, it can create an
incorrect Time value.
When parsing a time without a weekday specified, people
expect the weekday to be set.
Fix all three problems by computing the weekday on demand.
This is hard to gofix, since we must change the type of the node.
Since uses are rare and existing code will be caught by the compiler,
there is no gofix module here.
Fixes#2245.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/4974077
Augments type context and adds grammatical rules to handle special HTML constructs:
<!-- comments -->
<script>raw text</script>
<textarea>no tags here</textarea>
This CL does not elide comment content. I recommend we do that but
have not done it in this CL.
I used a codesearch tool over a codebase in another template language.
Based on the below I think we should definitely recognize
<script>, <style>, <textarea>, and <title>
as each of these appears frequently enough that there are few
template using apps that do not use most of them.
Of the other special tags,
<xmp>, <noscript>
are used but infrequently, and
<noframe> and friend, <listing>
do not appear at all.
We could support <xmp> even though it is obsolete in HTML5
because we already have the machinery, but I suggest we do not
support noscript since it is a normal tag in some browser
configurations.
I suggest recognizing and eliding <!-- comments -->
(but not escaping text spans) as they are widely used to
embed comments in template source. Not eliding them increases
the size of content sent over the network, and risks leaking
code and project internal details.
The template language I tested elides them so there are
no instance of IE conditional compilation directives in the
codebase but that could be a source of confusion.
The codesearch does the equivalent of
$ find . -name \*.file-extension \
| perl -ne 'print "\L$1\n" while s@<([a-z][a-z0-9])@@i' \
| sort | uniq -c | sort
The 5 uses of <plaintext> seem to be in tricky code and can be ignored.
The 2 uses of <xmp> appear in the same tricky code and can be ignored.
I also ignored end tags to avoid biasing against unary
elements and threw out some nonsense names since since the
long tail is dominated by uses of < as a comparison operator
in the template languages expression language.
I have added asterisks next to abnormal elements.
26765 div
7432 span
7414 td
4233 a
3730 tr
3238 input
2102 br
1756 li
1755 img
1674 table
1388 p
1311 th
1064 option
992 b
891 label
714 script *
519 ul
446 tbody
412 button
381 form
377 h2
358 select
353 strong
318 h3
314 body
303 html
266 link
262 textarea *
261 head
258 meta
225 title *
189 h1
176 col
156 style *
151 hr
119 iframe
103 h4
101 pre
100 dt
98 thead
90 dd
83 map
80 i
69 object
66 ol
65 em
60 param
60 font
57 fieldset
51 string
51 field
51 center
44 bidi
37 kbd
35 legend
30 nobr
29 dl
28 var
26 small
21 cite
21 base
20 embed
19 colgroup
12 u
12 canvas
10 sup
10 rect
10 optgroup
10 noscript *
9 wbr
9 blockquote
8 tfoot
8 code
8 caption
8 abbr
7 msg
6 tt
6 text
6 h5
5 svg
5 plaintext *
5 article
4 shortquote
4 number
4 menu
4 ins
3 progress
3 header
3 content
3 bool
3 audio
3 attribute
3 acronym
2 xmp *
2 overwrite
2 objects
2 nobreak
2 metadata
2 description
2 datasource
2 category
2 action
R=nigeltao
CC=golang-dev
https://golang.org/cl/4964045
