Clean up flow of control.
Ignore </table>, </tbody>, </tfoot>, </thead>, </tr> if there is not
an appropriate element in table scope.
Pass 3 more tests.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/6206093
Delete cases that just fall down to "anything else" action.
Handle </tbody>, </tfoot>, and </thead>.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/6203061
key and simple comparisson. Search is not yet implemented in this CL.
Changed some of the types of table_test.go to allow reuse in the new test.
Also reduced number of primary values for illegal runes to 1 (both map to
the same).
R=r
CC=golang-dev
https://golang.org/cl/6202062
Also set maxContractLen automatically.
Note that the table size is much bigger than it needs to be.
Optimization is best done, though, when the language specific
tables are added.
R=r
CC=golang-dev
https://golang.org/cl/6167044
dictates a CJK rune is only part of a certain specified range if it
is explicitly defined in the Unicode Codepoint Database.
Fixed the code and some of the tests accordingly.
R=r
CC=golang-dev
https://golang.org/cl/6160044
The first bug was that tertiary ignorables had the same colElem as
implicit colElems, yielding unexpected results. The current encoding
ensures that a non-implicit colElem is never 0. This fix uncovered
another bug of the trie that indexed incorrectly into the null block.
This was caused by an unfinished optimization that would avoid the
need to max out the most-significant bits of continuation bytes.
This bug was also present in the trie used in exp/norm and has been
fixed there as well. The appearence of the bug was rare, as the lower
blocks happened to be nearly nil.
R=r
CC=golang-dev
https://golang.org/cl/6127070
context for change lists of lower-level types. The public APIs are defined
in builder.go and collate.go. Type table is the glue between the lower and
higher level code and might be a good starting point for understanding the
collation code.
R=r, r
CC=golang-dev
https://golang.org/cl/5999053
The trie code looks a lot like the trie in exp/norm. It uses different
types, however. Also, there is only a lookup for []byte and the unsafe
lookup methods have been dropped, as well as sparse mode.
There is now a method for generating a trie. To output Go code, one now needs
to first generate a trie and then call print() on it.
R=r, r, mpvl
CC=golang-dev
https://golang.org/cl/5966064
Don't foster-parent text nodes that consist only of whitespace.
(I implemented this entirely in inTableIM instead of creating an
inTableTextIM, because the sole purpose of inTableTextIM seems to be
to combine character tokens into a string, which our tokenizer does
already.)
Use parseImpliedToken to clarify a couple of cases.
Handle <style>, <script>, <input>, and <form>.
Ignore doctype tokens.
Pass 20 additional tests.
R=nigeltao
CC=golang-dev
https://golang.org/cl/6117048
This CL corrects the remaining differences that I could find between the
implementation of inBodyIM and the spec:
Handle <rp> and <rt>.
Adjust SVG and MathML attributes.
Reconstruct active formatting elements in the "any other start tag" case.
Pass 7 additional tests.
R=nigeltao
CC=golang-dev
https://golang.org/cl/6101055
Clean up the flow of control.
Fix the TODO for handling <html> tags.
Add a case to ignore doctype declarations.
Pass one additional test.
R=nigeltao
CC=golang-dev
https://golang.org/cl/6072047
This method will allow us to be explicit about what we're doing when
we insert an implied token, and avoid repeating the logic involved in
multiple places.
R=nigeltao
CC=golang-dev
https://golang.org/cl/6060048
Add a case to ignore doctype tokens.
Clean up the flow of control to more clearly match the spec.
Pass one more test.
R=nigeltao
CC=golang-dev
https://golang.org/cl/6062047
Add case for doctype tokens (which are ignored).
This CL does not change the status of any tests.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/6061047
Currently, the html package only runs a limited subset of the tests
in the testdata directory. This tends to limit development of the
parser to fixing the bug that causes the first failing test.
This CL gives it the ability to run all the tests and produce a
log showing the status of each test. (It does it when tests are run with
'go test --update-logs') The status is listed as PASS, FAIL, or PARSE
(PARSE means that parsing produced the correct tree, but rendering and
re-parsing does not produce the same tree).
When 'go test' is run without --update-logs, it runs the tests marked
'PASS' in the logs (and the parsing portion of the tests marked 'PARSE').
Thus it will fail if there has been a regression since the last
time the logs were updated.
My goal for this CL is to allow develoment of the html package to
be less test-driven, while still having the advantages of regression
tests. In other words, one can work on any portion of the parser
and quickly see whether he is breaking things or improving them.
Current statistics of the tests:
$ grep ^PASS *.log|wc -l
1017
$ grep ^PARSE *.log|wc -l
46
$ grep ^FAIL *.log|wc -l
181
R=nigeltao
CC=golang-dev
https://golang.org/cl/6031049
below do not support '.
This makes package html consistent with package text/template's
HTMLEscape function.
Fixes#3489.
R=rsc, mikesamuel, dsymonds
CC=golang-dev
https://golang.org/cl/5992071
For completeness, we also expose the Canonical Combining Class of a rune.
This does not increase the data size.
R=r
CC=golang-dev
https://golang.org/cl/5931043
- Renamed ExportData -> FindGcExportData
and base it on an a bufio.Reader rather
than a filename so it can be used in
environments where object files are
stored elsewhere.
- Factor former GcImporter into GcImportData
and GcImport. Implementations with different
storage locations for object files can build
a customized GcImport using GcImportData.
This is pkg/exp only - no impact on Go 1.
R=golang-dev, lvd, rsc
CC=golang-dev
https://golang.org/cl/5574069
They have been deleted from package unsafe.
Also delete their appearance in exp/types.
Fixes#3338.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5847056
This is an API change, but one I have been promising would
happen when it was clear what the go command needed.
This is basically a complete replacement of what used to be here.
build.Tree is gone.
build.DirInfo is expanded and now called build.Package.
build.FindTree is now build.Import(package, srcDir, build.FindOnly).
The returned *Package contains information that FindTree returned,
but applicable only to a single package.
build.ScanDir is now build.ImportDir.
build.FindTree+build.ScanDir is now build.Import.
The new Import API allows specifying the source directory,
in order to resolve local imports (import "./foo") and also allows
scanning of packages outside of $GOPATH. They will come back
with less information in the Package, but they will still work.
The old go/build API exposed both too much and too little.
This API is much closer to what the go command needs,
and it works well enough in the other places where it is
used. Path is gone, so it can no longer be misused. (Fixes issue 2749.)
This CL updates clients of go/build other than the go command.
The go command changes are in a separate CL, to be submitted
at the same time.
R=golang-dev, r, alex.brainman, adg
CC=golang-dev
https://golang.org/cl/5713043
When debugging ASTs, it's useful to also
see the comments on occasion. Usage:
gotype -ast -comments file.go
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5703043
Not a Go 1 issue, but appeared to be fairly easy to fix.
- Note that a few existing test cases look slightly worse but
those cases were not representative for real code. All real
code looks better now.
- Manual move of the comment in go/scanner/example_test.go
before applying gofmt.
- gofmt -w $GOROOT/src $GOROOT/misc
Fixes#3062.
R=rsc
CC=golang-dev
https://golang.org/cl/5674093
by other low-level libraries, like collate. Extra care has been given to optimize the performance
of normalizing to NFD, as this is what will be used by the collator. The overhead of checking
whether a string is normalized vs simply decomposing a string is neglible. Assuming that most
strings are in the FCD form, this iterator can be used to decompose strings and normalize with
minimal overhead.
R=r
CC=golang-dev
https://golang.org/cl/5676057
* add -work option to save temporary files (Fixes issue 2980)
* fix go test -i to work with cgo packages (Fixes issue 2936)
* do not overwrite/remove empty directories or non-object
files during build (Fixes issue 2829)
* remove package main vs package non-main heuristic:
a directory must contain only one package (Fixes issue 2864)
* to make last item workable, ignore +build tags for files
named on command line: go build x.go builds x.go even
if it says // +build ignore.
* add // +build ignore tags to helper programs
R=golang-dev, r, r
CC=golang-dev
https://golang.org/cl/5674043
This CL makes it possible to run make.bash with
GOOS and GOARCH set to something other than
the native host GOOS and GOARCH.
As part of the CL, the tool directory moves from bin/tool/
to pkg/tool/goos_goarch where goos and goarch are
the values for the host system (running the build), not
the target. pkg/ is not technically appropriate, but C objects
are there now tool (pkg/obj/) so this puts all the generated
binaries in one place (rm -rf $GOROOT/pkg cleans everything).
Including goos_goarch in the name allows different systems
to share a single $GOROOT on a shared file system.
Fixes#2920.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5645093
Restore package os/signal, with new API:
Notify replaces Incoming, allowing clients
to ask for certain signals only. Also, signals
go to everyone who asks, not just one client.
This could plausibly move into package os now
that there are no magic side effects as a result
of the import.
Update runtime for new API: move common Unix
signal handling code into signal_unix.c.
(It's so easy to do this now that we don't have
to edit Makefiles!)
Tested on darwin,linux 386,amd64.
Fixes#1266.
R=r, dsymonds, bradfitz, iant, borman
CC=golang-dev
https://golang.org/cl/3749041
one trie lookup per rune is needed. See forminfo.go for a description
of the new format. Also included leading and trailing canonical
combining class in decomposition information. This will often avoid
additional trie lookups.
R=r, r
CC=golang-dev
https://golang.org/cl/5616071
Issue 2856 asks for a rename of a few methods to a
more idiomatic Go style. This is a very early API
that evolved organically throughout the years.
Together with the fact that ErrorVectors were embedded
in other data structures (e.g. Parser), just renaming
methods (e.g. GetError -> Error) would lead to undesired
behavior (e.g., Parser would act like an Error). Instead,
cleaned up API a bit more:
- removed ErrorVector in favor of ErrorList (already
present)
- simplified Scanner.Init by making the error handler a
function instead of requiring an ErrorHandler implementation
- adjusted helper functions accordingly
- updated Go 1 doc
Fixes#2856.
R=rsc
CC=golang-dev
https://golang.org/cl/5624047
The practice encourages people to think this is the way to
create a bytes.Buffer when new(bytes.Buffer) or
just var buf bytes.Buffer work fine.
(html/token.go was missing the point altogether.)
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5637043
The go- is redundant now that the directory is required
to be inside $GOROOT. Rob LGTMed the idea.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5618044
- Unified bounary conditions for NFC and NFD and removed some indirections.
This enforces boundaries at the character level, which is typically what
the user expects. (NFD allows a boundary between 'a' and '`', for example,
which may give unexpected results for collation. The current implementation
is already stricter than the standard, so nothing much changes. This change
just formalizes it.
- Moved methods of qcflags to runeInfo.
- Swapped YesC and YesMaybe bits in qcFlags. This is to aid future changes.
- runeInfo return values use named fields in preperation for struct change.
- Replaced some left-over uint32s with rune.
R=r
CC=golang-dev
https://golang.org/cl/5607050
Consequently, remove many package Makefiles,
and shorten the few that remain.
gomake becomes 'go tool make'.
Turn off test phases of run.bash that do not work,
flagged with $BROKEN. Future CLs will restore these,
but this seemed like a big enough CL already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
Also delete gotest, since it's messy to fix and slated for deletion anyway.
A couple of things outside src can't be tested any more. "go test" will be
fixed and these tests will be re-enabled. They're noisy for now.
Fixes#284.
R=rsc
CC=golang-dev
https://golang.org/cl/5598049
work in progress, and we are not ready to freeze its API for Go 1.
Package html still exists, containing just two functions: EscapeString
and UnescapeString.
Both the packages at exp/html and html are "package html". The former
is a superset of the latter.
At some point in the future, the exp/html code will move back into
html, once we have finalized the parser API.
R=rsc, dsymonds
CC=golang-dev
https://golang.org/cl/5571059
Remove NewMD5, NewSHA1 and NewSHA256 in favor of using New and
explicitly importing the used hash-function. This way when using, for
example, HMAC with RIPEMD there's no md5, sha1 and sha256 linked in
through the hmac package.
A gofix rule is included, and applied to the standard library (3 files
altered).
This change is the result of a discussion at
https://golang.org/cl/5550043/ to pull the discussion about
deprecating these functions out of that issue.
R=golang-dev, agl
CC=golang-dev, r, rsc
https://golang.org/cl/5556058
Previously, a timeout (in int64 nanoseconds) applied to a granularity
even smaller than one operation: a 100 byte read with a 1 second timeout
could take 100 seconds, if the bytes all arrived on the network 1 second
apart. This was confusing.
Rather than making the timeout granularity be per-Read/Write,
this CL makes callers set an absolute deadline (in time.Time)
after which operations will fail. This makes it possible to
set deadlines at higher levels, without knowing exactly how
many read/write operations will happen in e.g. reading an HTTP
request.
Fixes#2723
R=r, rsc, dave
CC=golang-dev
https://golang.org/cl/5555048
- Use a faster closure-based iterator rather than a channel-based one.
- Otherwise: better code organization, but no other API changes.
R=r, r
CC=golang-dev
https://golang.org/cl/5557051
First was, apart from adding tests, a single line of code (to add the
constructor function). Adding SHA512-based hashing to crypto/hmac
required minor rework of the package because of a previously hardcoded
block-size in it's implementation. Instead of using a hash.Hash
generator function the constructor function now uses a crypto.Hash
type, which was extended to expose information about block size.
The only standard library package impacted by the change is
crypto/tls, for which the fix is included in this patch. It might be
useful to extend gofix to include this API change too.
R=agl, r, rsc, r
CC=golang-dev
https://golang.org/cl/5550043
The terminal code in exp/terminal was forked from the code in exp/ssh.
This change removes the duplicated code from exp/ssh in favour of
using exp/terminal.
R=rsc
CC=golang-dev
https://golang.org/cl/5375064
(This was missing in the last change because I uploaded it from the
wrong machine.)
Large pastes previously misbehaved because the code tried reading from
the terminal before checking whether an line was already buffered.
Large pastes can cause multiples lines to be read at once from the
terminal.
R=bradfitz
CC=golang-dev
https://golang.org/cl/5542049
exp/proxy provides client support for tunneling connections through
various proxies.
This is an initial, incomplete sketch of the code to lay down an
API.
R=golang-dev, r, r, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/5490062
Also verified in external test suite that this fixes MySQL
resource exhaustion problems, and also exposed a double-free
bug in the gosqlite3 driver (where gosqlite3 either got lucky
before, or was working around this bug)
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5544057
This is a relic from the times when we switched
to automatic semicolon insertion. It's still use-
ful to have a non-exported switch for testing.
R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/5528077
This CL fixes two issues sending data to the remote peer.
The first bug occurs when the size of the buffer passed to
Write is larger than the current window, in this case, w.rwin
can become negative.
The second issue is more problematic than the first as the
amount of data passed to writePacket was not limited to w.rwin.
In this case the remote peer could silently drop the additional
data, or drop the connection.
Credit to Jacek Masiulaniec for the bug report.
R=agl, jacek.masiulaniec
CC=golang-dev
https://golang.org/cl/5511043
transport.go:
* remove unused nil check.
doc.go:
* improve documentation about supported auth
methods and update Run example.
Thanks Jacek Masiulaniec for both reports.
R=jacek.masiulaniec, agl
CC=golang-dev
https://golang.org/cl/5501075
1) incorrect length given for out buffer in String.
2) patchTail bug that could cause characters to be lost
when crossing into the out-buffer boundary.
Added tests to expose these bugs. Also slightly improved
performance of Bytes() and String() by sharing the reorderBuffer
across operations.
Fixes#2567.
R=r
CC=golang-dev
https://golang.org/cl/5502069
1) Add EscapeCodes to the terminal so that applications don't wire
them in.
2) Add a callback for auto-complete
3) Fix an issue with input lines longer than the width of the
terminal.
4) Have Write() not stomp the current line. It now erases the current
input, writes the output and reprints the prompt and partial input.
5) Support prompting without local echo in Terminal.
6) Add GetSize to report the size of terminal.
R=bradfitz
CC=golang-dev
https://golang.org/cl/5479043
- no empty lines inside empty structs and interfaces
- top-level declarations are separated by a blank line if
a) they are of different kind (e.g. const vs type); or
b) there are documentation comments associated with a
declaration (this is new)
- applied gofmt -w misc src
The actual changes are in go/printer/nodes.go:397-400 (empty structs/interfaces),
and go/printer/printer.go:307-309 (extra line break). The remaining
changes are cleanups w/o changing the existing functionality.
Fixes issue 2570.
R=rsc
CC=golang-dev
https://golang.org/cl/5493057
If a Pipe method is called, return the underlying
reader/writer from session.clientChan, bypassing the
io.Copy and io.Pipe harness.
StdoutPipe and StderrPipe now return an io.Reader not
an io.ReadCloser as SSH cannot signal the close of the
local reader to the remote process.
R=rsc, agl, gustav.paul, cw
CC=golang-dev
https://golang.org/cl/5493047
I added the clientChan's msg channel to the list of channels that are closed in mainloop when the server sends a channelCloseMsg.
I added an ExitError type that wraps a Waitmsg similar to that of os/exec. I fill ExitStatus with the data returned in the 'exit-status' channel message and Msg with the data returned in the 'exit-signal' channel message.
Instead of having Wait() return on the first 'exit-status'/'exit-signal' I have it return an ExitError containing the status and signal when the clientChan's msg channel is closed.
I added two tests cases to session_test.go that test for exit status 0 (in which case Wait() returns nil) and exit status 1 (in which case Wait() returns an ExitError with ExitStatus 1)
R=dave, agl, rsc, golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5452051
This CL cleans up the client auth tests, making the
individual test body more manageable.
Also, adds tests for rsa and dsa key negotiation.
Finally, remove the package level use of the variable
strings, which avoids conflicting with the strings pkg.
R=gustav.paul, agl, n13m3y3r, rsc
CC=golang-dev
https://golang.org/cl/5447049
This is part one of a small set of CL's that aim to resolve
the outstanding TODOs relating to channel close and blocking
behavior.
Firstly, the hairy handling of assigning the peersId is now
done in one place. The cost of this change is the slightly
paradoxical construction of the partially created clientChan.
Secondly, by creating clientChan.stdin/out/err when the channel
is opened, the creation of consumers like tcpchan and Session
is simplified; they just have to wire themselves up to the
relevant readers/writers.
R=agl, gustav.paul, rsc
CC=golang-dev
https://golang.org/cl/5448073
All but 3 cases (in gcimporter.go and hixie.go)
are automatic conversions using gofix.
No attempt is made to use the new Append functions
even though there are definitely opportunities.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5447069
The SSH spec allows for the server to send a banner message to the client at any point during the authentication process. Currently the ssh client auth types all assume that the first response from the server after issuing a userAuthRequestMsg will be one of a couple of possible authentication success/failure messages. This means that client authentication breaks if the ssh server being connected to has a banner message configured.
This changeset refactors the noneAuth, passwordAuth and publickeyAuth types' auth() function and allows for msgUserAuthBanner during authentication.
R=golang-dev, rsc, dave, agl
CC=golang-dev
https://golang.org/cl/5432065
This is the result of running `gofix -r hashsum` over the tree, changing
the hash function implementations by hand and then fixing a couple of
instances where gofix didn't catch something.
The changed implementations are as simple as possible while still
working: I'm not trying to optimise in this CL.
R=rsc, cw, rogpeppe
CC=golang-dev
https://golang.org/cl/5448065
Exec() has been renamed to Run() in keeping with the os/exec API.
Added func (*Session) Start(cmd string) which starts a remote process but unlike Run() doesn't wait for it to finish before returning.
Run() has been refactored to use Start internally. Its really just a refactoring, no new code but some extra functionality was won.
Also added func (*Session) Signal(sig signal) which sends a UNIX signal to a remote process. This is espcially useful in conjunction with Start() as the two allow you to start a remote process, monitor its stdout/stderr, and send it a TERM/HUP/etc signal when you want it to close.
R=dave, rsc, agl, bradfitz, n13m3y3r, gustavo
CC=golang-dev
https://golang.org/cl/5437058
According to http://www.ietf.org/rfc/rfc4254.txt most channel messages contain the channel id of the recipient channel, not the sender id. This allows the recipient connection multiplexer to route the message to the correct channel.
This changeset fixes several messages that incorrectly send the local channel id instead of the remote channel's id.
While sessions were being created and closed in sequence channels in the channel pool were freed and reused on the server side of the connection at the same rate as was done on the client, so the channel local and remote channel ids always corresponded. As soon as I had concurrent sessions on the same clientConn the server started to complain of 'uknown channel id N' where N is the local channel id, which is actually paired with server channel id K.
R=golang-dev, dave, rsc, agl
CC=golang-dev
https://golang.org/cl/5433063
IIRC, package sql used to be called db. There was one occurrence
of the old name in a comment.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5431075
This CL inverts the direction of the Stdin/out/err members of the
Session struct so they reflect the API of the exec.Cmd. In doing so
it borrows heavily from the exec package.
Additionally Shell now returns immediately, wait for completion using
Wait. Exec calls Wait internally and so blocks until the remote
command is complete.
Credit to Gustavo Niemeyer for the impetus for this CL.
R=rsc, agl, n13m3y3r, huin, bradfitz
CC=cw, golang-dev
https://golang.org/cl/5322055
deps.bash does not understand current format,
so make.bash fails (with MAKEFLAGS=-j20).
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5410046
Fixes use of c after Dial failure (causes crash).
May fix Dial failure by listening to 127.0.0.1:0
instead of 0.0.0.0:0 (tests should only listen on
localhost).
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5395052
Ensure that empty NameLists always return
a zero length []string, not nil.
In practice NameLists are only used in a few
message types and always consumed by a for
range function so the difference between nil
and []string{} is not significant.
Also, add exp/ssh to pkg/Makefile as suggested
by rsc.
R=rsc, agl
CC=golang-dev
https://golang.org/cl/5400042
- syscall (not os) now defines the Errno type.
- the low-level assembly functions Syscall, Syscall6, and so on
return Errno, not uintptr
- syscall wrappers all return error, not uintptr.
R=golang-dev, mikioh.mikioh, r, alex.brainman
CC=golang-dev
https://golang.org/cl/5372080
This CL adds experimental support for making proxied
net.Conn connections via the remote server.
nb. Functional tests exist for this feature but CL
5320053 or similar needs to be committed to support them.
R=rsc, agl, n13m3y3r
CC=cw, golang-dev, huin
https://golang.org/cl/5371081
client.go/client_auth.go:
* add support for publickey key auth using the interface
outlined by rsc in the previous auth CL
client_auth_test.go:
* password and publickey tests against server.go
common.go/server.go:
* move some helper methods from server.go into common.go
* generalise serializeRSASignature
R=rsc, agl, huin
CC=cw, golang-dev, n13m3y3r
https://golang.org/cl/5373055
Some remote servers send a 0 window size in the channel
open confirm msg, others send a non zero window size. Make
sure this initial advertisement is not lost.
R=agl, rsc, cw
CC=golang-dev
https://golang.org/cl/5372083
It was never really a shell, but the name carried
over from SSH's ServerShell.
Two small functional changes:
Add SetSize, which allows the size of the terminal
to be changed in response, say, to an SSH message.
Don't write the prompt if there's already something
on the current line.
R=rsc
CC=golang-dev
https://golang.org/cl/5376066
(This is part of removing the duplicate code between exp/terminal and
exp/ssh, but hg is having a very hard time keeping up so I'm doing it
in small steps.)
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/5373061
This CL adds an API for handling the various SSH
authenticaton methods. None and password continue
to be the only supported methods.
R=bradfitz, agl, n13m3y3r, rsc, cw
CC=golang-dev
https://golang.org/cl/5328045
There are three classes of methods/functions called Error:
a) The Error method in the just introduced error interface
b) Error methods that create or report errors (http.Error, etc)
c) Error methods that return errors previously associated with
the receiver (Tokenizer.Error, rows.Error, etc).
This CL introduces the convention that methods in case (c)
should be named Err.
The reasoning for the change is:
- The change differentiates the two kinds of APIs based on
names rather than just on signature, unloading Error a bit
- Err is closer to the err variable name that is so commonly
used with the intent of verifying an error
- Err is shorter and thus more convenient to be used often
on error verifications, such as in iterators following the
convention of the sql package.
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/5327064
This is Go 1 package renaming CL #2.
This one merely moves the source; the import strings will be
changed after the next weekly release.
exp/template/html -> html/template
big -> math/big
cmath -> math/cmplx
rand -> math/rand
syslog -> log/syslog
The only edits are in Makefiles and deps.bash.
Note that this CL moves exp/template/html out of exp. I decided
to do that so all the renamings can be done together, even though
the API (and that of template, for that matter) is still fluid.
R=r, rsc
CC=golang-dev
https://golang.org/cl/5332053
(The definition of ErrorList is in another file, so gofix
has no hope of getting this right.)
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/5330043
This CL grew the archive file name length from 16 to 64:
changeset: 909:58574851d792
user: Russ Cox <rsc@golang.org>
date: Mon Oct 20 13:53:56 2008 -0700
Back then, every x.go file in a package became an x.6 file
in the archive. It was important to be able to allow the
use of long Go source file names, hence the increase in size.
Today, all Go source files compile into a single _go_.6 file
regardless of their names, so the archive file name length
no longer needs to be long. The longer name causes some
problems on Plan 9, where the native archive format is the
same but with 16-byte names, so revert back to 16.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5333050
Fixed error checking in exec.go to give a sensible error message when
execution is attempted before a successful parse (rather than an
outright panic).
R=r
CC=golang-dev
https://golang.org/cl/5306065
The payload of a data message is defined as an SSH string type,
which uses the first four bytes to encode its length. When channelData
and channelExtendedData were added I defined Payload as []byte to
be able to use it directly without a string to []byte conversion. This
resulted in the length data leaking into the payload data.
This CL fixes the bug, and restores agl's original fast path code.
Additionally, a bug whereby s.lock was not released if a packet arrived
for an invalid channel has been fixed.
Finally, as they were no longer used, I have removed
the channelData and channelExtedendData structs.
R=agl, rsc
CC=golang-dev
https://golang.org/cl/5330053
I found these by adding a check to govet, but the check
produces far too many false positives to be useful.
Even so, these few seem worth cleaning up.
R=golang-dev, bradfitz, iant
CC=golang-dev
https://golang.org/cl/5311067
Nothing terribly interesting here. (!)
Since the public APIs are all in terms of UTF-8,
the changes are all internal only.
R=mpvl, gri, r
CC=golang-dev
https://golang.org/cl/5309042
API question: is a scanner token an int or a rune?
Since the rune is the common case and the token values
are the special (negative) case, I chose rune. But it could
easily go the other way.
R=gri
CC=golang-dev
https://golang.org/cl/5301049
This CL replaces the Cmd type with a Session type representing
interactive channels. This lays the foundation for supporting
other kinds of channels like direct-tcpip or x11.
client.go:
* replace chanlist map with slice.
* generalize stdout and stderr into a single type.
* unexport ClientChan to clientChan.
doc.go:
* update ServerConfig/ServerConn documentation.
* update Client example for Session.
message.go:
* make channelExtendedData more like channelData.
session.go:
* added Session which replaces Cmd.
R=agl, rsc, n13m3y3r, gustavo
CC=golang-dev
https://golang.org/cl/5302054
server.go/channel.go:
* rename Server to ServerConfig to match Client.
* rename ServerConnection to ServeConn to match Client.
* add Listen/Listener.
* ServerConn.Handshake(), general cleanups.
client.go:
* fix bug where fmt.Error was not assigned to err
R=rsc, agl
CC=golang-dev
https://golang.org/cl/5265049
cssEscaper escapes using the CSS convention: `\` + hex + optional-space
It outputs the space when the escape could be followed by
a hex digit to distinguish a "\na" from "\u00aa".
It did not output a space when the escape is followed by a space
character so did not distinguish "\n " from "\n".
Currently when doing lookahead, it does not distinguish spaces that
will be escaped later by the same function from ones that will not.
This is correct but suboptimal.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5306042
This change splits terminal handling from exp/ssh, as suggested
several times in the ssh code review.
shell.go and shell_test.go are copies from exp/ssh with minimal
changes, so don't need another full review. A future CL will remove
that code from exp/ssh.
R=bradfitz, r, dave, rsc
CC=golang-dev
https://golang.org/cl/5278049
(more are possible but omitted for now as they are part of
specific tests where rather than changing what is there we
should probably expand the tests to cover the new case)
R=rsc, dvyukov
CC=golang-dev
https://golang.org/cl/5247058
therefore unlikely that there is a good use for its string version
LastBoundaryInString. Yet, the implemenation of this method would complicate
things a bit as it would require the introduction for another interface and
some duplication of code. Removing it seems a better choice.
R=r
CC=golang-dev
https://golang.org/cl/5182044
This is a possible optimization. I'm not sure the complexity is worth it.
The new benchmark in escape_test is 46us without and 35us with the optimization.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5168041
This removes a few cases from escapeAction and clarifies the
responsibilities of urlFilter which no longer does any
escaping or normalization. It is now solely a filter.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5162043
HTML5 allows embedded SVG and MathML.
Code searches show SVG is used for graphing.
This changes transition to deal with constructs like
<svg xmlns:xlink="http://www.w3.org/1999/xlink">
It changes attr and clients to call a single function that combines
the name lookup and "on" prefix check to determine an attribute
value type given an attribute name.
That function uses heuristics to recognize that
xlink:href and svg:href
have URL content, and that data-url is likely contains URL content,
since "javascript:" injection is such a problem.
I did a code search over a closure templates codebase to determine
patterns of custom attribute usage. I did something like
$ find . -name \*.soy | \
xargs egrep perl -ne 'while (s/\b((data-|\w+:)\w+)\s*=//) { print "$1\n"; }' | \
sort | uniq
to produce the list at the bottom.
Filtering that by egrep -i 'src|url|uri' produces
data-docConsumptionUri
data-docIconUrl
data-launchUrl
data-lazySrc
data-pageUrl
data-shareurl
data-suggestServerUrl
data-tweetUrl
g:secondaryurls
g:url
which seem to match all the ones that are likely URL content.
There are some short words that match that heuristic, but I still think it decent since
any custom attribute that has a numeric or enumerated keyword value will be unaffected by
the URL assumption.
Counterexamples from /usr/share/dict:
during, hourly, maturity, nourish, purloin, security, surly
Custom attributes present in existing closure templates codebase:
buzz:aid
data-a
data-action
data-actor
data-allowEqualityOps
data-analyticsId
data-bid
data-c
data-cartId
data-categoryId
data-cid
data-command
data-count
data-country
data-creativeId
data-cssToken
data-dest
data-docAttribution
data-docConsumptionUri
data-docCurrencyCode
data-docIconUrl
data-docId
data-docPrice
data-docPriceMicros
data-docTitle
data-docType
data-docid
data-email
data-entityid
data-errorindex
data-f
data-feature
data-fgid
data-filter
data-fireEvent
data-followable
data-followed
data-hashChange
data-height
data-hover
data-href
data-id
data-index
data-invitable
data-isFree
data-isPurchased
data-jid
data-jumpid
data-launchUrl
data-lazySrc
data-listType
data-maxVisiblePages
data-name
data-nid
data-nodeid
data-numItems
data-numPerPage
data-offerType
data-oid
data-opUsesEquality
data-overflowclass
data-packageName
data-pageId
data-pageUrl
data-pos
data-priceBrief
data-profileIds
data-query
data-rating
data-ref
data-rentalGrantPeriodDays
data-rentalactivePeriodHours
data-reviewId
data-role
data-score
data-shareurl
data-showGeLe
data-showLineInclude
data-size
data-sortval
data-suggestServerType
data-suggestServerUrl
data-suggestionIndex
data-tabBarId
data-tabBarIndex
data-tags
data-target
data-textColor
data-theme
data-title
data-toggletarget
data-tooltip
data-trailerId
data-transactionId
data-transition
data-ts
data-tweetContent
data-tweetUrl
data-type
data-useAjax
data-value
data-width
data-x
dm:index
dm:type
g:aspects
g:decorateusingsecondary
g:em
g:entity
g:groups
g:id
g:istoplevel
g:li
g:numresults
g:oid
g:parentId
g:pl
g:pt
g:rating_override
g:secondaryurls
g:sortby
g:startindex
g:target
g:type
g:url
g:value
ga:barsize
ga:css
ga:expandAfterCharsExceed
ga:initialNumRows
ga:nocancelicon
ga:numRowsToExpandTo
ga:type
ga:unlockwhenrated
gw:address
gw:businessname
gw:comment
gw:phone
gw:source
ng:controller
xlink:href
xml:lang
xmlns:atom
xmlns:dc
xmlns:jstd
xmlns:ng
xmlns:og
xmlns:webstore
xmlns:xlink
R=nigeltao
CC=golang-dev
https://golang.org/cl/5119041
The normalization that prevents element name and comment injection in
<{{.}}
by converting it to
<{{.}}
breaks
<!DOCTYPE html>
Instead of splitting states to have a start of document state and a text
state, I whitelist <!DOCTYPE.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5131051
In
{{$x := . | foo}}
{{$x}}
the first action is a variable assignment that contributes
nothing to the output while the first is a use that needs
to be escaped.
This CL fixes escapeAction to distinguish assignments from
interpolations and to only modify interpolations.
R=nigeltao, r
CC=golang-dev
https://golang.org/cl/5143048
Does some TODOs and changes the term "div" in an error message
to "division" to avoid confusion with "<div>".
R=nigeltao, r
CC=golang-dev
https://golang.org/cl/5141047
This makes sure that all JS newlines are encoded in JSON.
It also moots a TODO about possibly escaping supplemental codepoints.
I served:
Content-Type: text/javascript;charset=UTF-8
var s = "%s";
document.write("<p>", s, "</p><ol>");
for (var i = 0; i < s.length; i++) {
document.write("<li>", s.charCodeAt(i).toString(16), "</li>");
}
document.write("</l>");
where %s was replaced with bytes "\xf0\x9d\x84\x9e" to test
straight UTF-8 instead of encoding surrogates separately.
Recent Firefox, Chrome, and Safari all decoded it properly.
I have yet to try it on IE or older versions.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5129042
The template
"<a="
caused an infinite loop in escape text.
The change to tTag fixes that and the change to escape.go causes
escapeText to panic on any infinite loop that does not involve
a state cycle.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5115041
HTML parsers may differ on whether
<input id= onchange=f( ends in id's or onchange's value,
<a class=`foo ends inside a value,
<input style=font:'Arial' needs open-quote fixup.
Per
http://www.w3.org/TR/html5/tokenization.html#attribute-value-unquoted-state
this treats the error cases in 8.2.4.40 Attribute value (unquoted) state
as fatal errors.
\> U+0022 QUOTATION MARK (")
\> U+0027 APOSTROPHE (')
\> U+003C LESS-THAN SIGN (<)
\> U+003D EQUALS SIGN (=)
\> U+0060 GRAVE ACCENT (`)
Parse error. Treat it as per the "anything else" entry below.
and emits ErrBadHTML.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5085050
When templates are stored in external files, developers often embed
comments to explain&|disable code.
<!-- Oblique reference to project code name here -->
{{if .C}}...{{else}}<!-- commented out default -->{{end}}
This unnecessarily increases the size of shipped HTML and can leak
information.
This change elides all comments of the following types:
1. <!-- ... --> comments found in source.
2. /*...*/ and // comments found in <script> elements.
3. /*...*/ and // comments found in <style> elements.
It does not elide /*...*/ or // comments found in HTML attributes:
4. <button onclick="/*...*/">
5. <div style="/*...*/">
I can find no examples of comments in attributes in Closure Templates
code and doing so would require keeping track of character positions
post decode in
<button onclick="/*...*/">
To prevent token joining, /*comments*/ are JS and CSS comments are
replaced with a whitespace char.
HTML comments are not, but to prevent token joining we could try to
detect cases like
<<!---->b>
</<!---->b>
which has a well defined meaning in HTML but will cause a validator
to barf. This is difficult, and this is a very minor case.
I have punted for now, but if we need to address this case, the best
way would be to normalize '<' in stateText to '<' consistently.
The whitespace to replace a JS /*comment*/ with depends on whether
there is an embedded line terminator since
break/*
*/foo
...
is equivalent to
break;
foo
...
while
break/**/foo
...
is equivalent to
break foo;
...
Comment eliding can interfere with IE conditional comments.
http://en.wikipedia.org/wiki/Conditional_comment
<!--[if IE 6]>
<p>You are using Internet Explorer 6.</p>
<![endif]-->
/*@cc_on
document.write("You are using IE4 or higher");
@*/
I have not encountered these in production template code, and
the typed content change in CL 4962067 provides an escape-hatch
if conditional comments are needed.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4999042
This simplifies transition functions to make it easier to reliably
elide comments in a later CL.
Before:
- transition functions are responsible for detecting special end tags.
After:
- the code to detect special end tags is done in one place.
We were relying on end tags being skipped which meant we were
not noticing comments inside script/style elements that contain no
substitutions.
This change means we will notice all such comments where necessary,
but stripTags will notice none since it does not need to. This speeds
up stripTags.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5074041
This CL generalises the pair of halfConnection members that the
serverConn holds into a single transport struct that is shared by
both Server and Client, see also CL 5037047.
This CL is a replacement for 5040046 which I closed by accident.
R=agl, bradfitz
CC=golang-dev
https://golang.org/cl/5075042
Formulaic changes to transition functions in preparation for CL 5074041.
This should be completely semantics preserving.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5091041
Instead of erroring on actions inside comments, use existing escaping
pipeline to quash the output of actions inside comments.
If a template maintainer uses a comment to disable template code:
{{if .}}Hello, {{.}}!{{end}}
->
<!--{{if true}}Hello, {{.}}!{{end}}-->
will result in
<!--Hello, !-->
regardless of the value of {{.}}.
In a later CL, comment elision will result in the entire commented-out
section being dropped from the template output.
Any side-effects in pipelines, such as panics, will still be realized.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5078041
This addresses several use cases:
(1) <h{{.HeaderLevel}}> used to build hierarchical documents.
(2) <input on{{.EventType}}=...> used in widgets.
(3) <div {{" dir=ltr"}}> used to embed bidi-hints.
It also makes sure that we treat the two templates below the same:
<img src={{if .Avatar}}"{{.Avatar}}"{{else}}"anonymous.png"{{end}}>
<img src="{{if .Avatar}}{{.Avatar}}{{else}}anonymous.png{{end}}">
This splits up tTag into a number of sub-states and adds testcases.
R=nigeltao
CC=golang-dev
https://golang.org/cl/5043042
The template
<{{.}}
would violate the structure preservation property if allowed and not
normalized, because when {{.}} emitted "", the "<" would be part of
a text node, but if {{.}} emitted "a", the "<" would not be part of
a text node.
This change rewrites '<' in text nodes and RCDATA text nodes to
'<' allowing template authors to write the common, and arguably more
readable:
Your price: {{.P1}} < list price {{.P2}}
while preserving the structure preservation property.
It also lays the groundwork for comment elision, rewriting
Foo <!-- comment with secret project details --> Bar
to
Foo Bar
R=nigeltao
CC=golang-dev
https://golang.org/cl/5043043
The typical UNIX method for controlling long running process is to
send the process signals. Since this doesn't get you very far, various
ad-hoc, remote-control protocols have been used over time by programs
like Apache and BIND.
Implementing an SSH server means that Go code will have a standard,
secure way to do this in the future.
R=bradfitz, borman, dave, gustavo, dsymonds, r, adg, rsc, rogpeppe, lvd, kevlar, raul.san
CC=golang-dev
https://golang.org/cl/4962064
I found a simple test case that does require doing the fixed point TODO
in computeOutCtx.
I found a way though to do this and simplify away the escapeRange
hackiness that was added in https://golang.org/cl/5012044/
R=nigeltao
CC=golang-dev
https://golang.org/cl/5015052
This replaces the errStr & errLine members of context with a single err
*Error, and introduces a number of const error codes, one per
escape-time failure mode, that can be separately documented.
The changes to the error documentation moved from doc.go to error.go
are cosmetic.
R=r, nigeltao
CC=golang-dev
https://golang.org/cl/5026041
Not all content is plain text. Sometimes content comes from a trusted
source, such as another template invocation, an HTML tag whitelister,
etc.
Template authors can deal with over-escaping in two ways.
1) They can encapsulate known-safe content via
type HTML, type CSS, type URL, and friends in content.go.
2) If they know that the for a particular action never needs escaping
then they can add |noescape to the pipeline.
{{.KnownSafeContent | noescape}}
which will prevent any escaping directives from being added.
This CL defines string type aliases: HTML, CSS, JS, URI, ...
It then modifies stringify to unpack the content type.
Finally it modifies the escaping functions to use the content type and
decline to escape content that does not require it.
There are minor changes to escapeAction and helpers to treat as
equivalent explicit escaping directives such as "html" and "urlquery"
and the escaping directives defined in the contextual autoescape module
and to recognize the special "noescape" directive.
The html escaping functions are rearranged. Instead of having one
escaping function used in each {{.}} in
{{.}} : <textarea title="{{.}}">{{.}}</textarea>
a slightly different escaping function is used for each.
When {{.}} binds to a pre-sanitized string of HTML
`one < <i>two</i> & two < "3"`
we produces something like
one < <i>two</i> & two < "3" :
<textarea title="one < two & two < "3"">
one < <i>two</i> & two < "3"
</textarea>
Although escaping is not required in <textarea> normally, if the
substring </textarea> is injected, then it breaks, so we normalize
special characters in RCDATA and do the same to preserve attribute
boundaries. We also strip tags since developers never intend
typed HTML injected in an attribute to contain tags escaped, but
do occasionally confuse pre-escaped HTML with HTML from a
tag-whitelister.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/4962067
This moots a caveat in the proposed package documentation by
rendering useless any template that could not be escaped.
From https://golang.org/cl/4969078/
> If EscapeSet returns an error, do not Execute the set; it is not
> safe against injection.
r: [but isn't the returned set nil? i guess you don't overwrite the
r: original if there's a problem, but i think you're in your rights to
r: do so]
R=r
CC=golang-dev
https://golang.org/cl/5020043
This adds support for {{template "callee"}} calls.
It recognizes that calls can appear in many contexts.
{{if .ImageURL}}
<img src="{{.ImageURL}}" alt="{{template "description"}}">
{{else}}
<p>{{template "description"}}</p>
{{end}}
calls a template in two different contexts, first in an HTML attribute
context, and second in an HTML text context.
Those two contexts aren't very different, but when linking text
to search terms, the escaping context can be materially different:
<a href="/search?q={{template "tags"}}">{{template "tags"}}</a>
This adds API:
EscapeSet(*template.Set, names ...string) os.Error
takes a set of templates and the names of those which might be called
in the default context as starting points.
It changes the escape* functions to be methods of an object which
maintains a conceptual mapping of
(template names*input context) -> output context.
The actual mapping uses as key a mangled name which combines the
template name with the input context.
The mangled name when the input context is the default context is the
same as the unmangled name.
When a template is called in multiple contexts, we clone the template.
{{define "tagLink"}}
<a href="/search?q={{template "tags"}}">{{template "tags"}}</a>
{{end}}
{{define "tags"}}
{{range .Tags}}{{.}},{{end}}
{{end}}
given []string{ "foo", "O'Reilly", "bar" } produces
<a href="/search?q=foo,O%27Reilly,bar">foo,O'Reilly,bar</a>
This involves rewriting the above to something like
{{define "tagLink"}}
<a href="/search?q={{template "tags$1"}}">{{template "tags"}}</a>
{{end}}
{{define "tags"}}
{{range .Tags}}{{. | html}},{{end}}
{{end}}
{{define "tags$1"}}
{{range .Tags}}{{. | urlquery}},{{end}}
{{end}}
clone.go provides a mechanism for cloning template "tags" to produce
"tags$1".
changes to escape.go implement the new API and context propagation
around the call graph.
context.go includes minor changes to support name mangling and
context_test.go tests those.
js.go contains a bug-fix.
R=nigeltao, r
CC=golang-dev
https://golang.org/cl/4969072
Often, division/regexp ambiguity doesn't matter in JS because the next
token is not a slash.
For example, in
<script>var global{{if .InitVal}} = {{.InitVal}}{{end}}</script>
When there is an initial value, the {{if}} ends with jsCtxDivOp
since a '/' following {{.InitVal}} would be a division operator.
When there is none, the empty {{else}} branch ends with jsCtxRegexp
since a '/' would start a regular expression. A '/' could result
in a valid program if it were on a new line to allow semicolon
insertion to terminate the VarDeclaration.
There is no '/' though, so we can ignore the ambiguity.
There are cases where a missing semi can result in ambiguity that
we should report.
<script>
{{if .X}}var x = {{.X}}{{end}}
/...{{.Y}}
</script>
where ... could be /foo/.test(bar) or /divisor. Disambiguating in
this case is hard and is required to sanitize {{.Y}}.
Note, that in the case where there is a '/' in the script tail but it
is not followed by any interpolation, we already don't care. So we
are already tolerant of
<script>{{if .X}}var x = {{.X}}{{end}}/a-bunch-of-text</script>
because tJS checks for </script> before looking in /a-bunch-of-text.
This CL
- Adds a jsCtx value: jsCtxUnknown
- Changes joinContext to join contexts that only differ by jsCtx.
- Changes tJS to return an error when a '/' is seen in jsCtxUnknown.
- Adds tests for both the happy and sad cases.
R=nigeltao
CC=golang-dev
https://golang.org/cl/4956077