When writing custom scanners, I found that
Token itself was rarely useful, as I did not always
want to stop at white space. This change makes
it possible to stop at any class of characters
while reusing the buffer within State.
(also fix a bug in Token)
R=r, r2
CC=golang-dev
https://golang.org/cl/4243055
Detect when scan is being called recursively and
re-use the same scan state.
On my machine, for a recursion-heavy benchmark, this
results in 44x speed up. This does impose a 4% penalty
on the non-recursive case, which can be removed by
heap-allocating the saved state, at 40% performance penalty
on the recursive case. Either way is fine with me.
R=r
CC=golang-dev
https://golang.org/cl/4253049
Add a new Read method to ScanState so that it
satisfies the io.Reader interface; rename
Getrune and Ungetrune to ReadRune and UnreadRune.
Make sure ReadRune does not read past width restrictions;
remove now-unnecessary Width method from ScanState.
Also make the documentation a little clearer as to
how ReadRune and UnreadRune are used.
R=r, r2
CC=golang-dev
https://golang.org/cl/4240056
signed or unsigned integers using %v or the formatless scanner.
That is, Sscan("0x11", &i) or Sscanf("0x11", "%v", &i) will now
set i to 17. If a format other than %v is presented, the behavior
is as before.
Fixes#1469.
R=rsc
CC=golang-dev
https://golang.org/cl/4131042
fmt.Printf("%U", 1) yields "U+0001"
It's essentially "U+%.4x" but lets you override the precision works in scan, too.
R=rsc
CC=golang-dev
https://golang.org/cl/3423043
While we're at it, clean up and test the code to guarantee we see every byte when
the text is erroneous UTF-8.
Fixes#866.
R=rsc
CC=golang-dev
https://golang.org/cl/1712042
Also fix an interface bug: white-space-delimited doesn't work well for cases like "%d, %d" on "23, 23")
R=rsc
CC=golang-dev
https://golang.org/cl/1502041
- implement scanning for all renamed types
(compiler bug stops complex from being renamable,
so it can't be tested but the code is there)
- %q %x for strings
- error handling now done with panic/recover
R=rsc
CC=golang-dev
https://golang.org/cl/1458041
- implement scanning for all renamed types
(compiler bug stops complex from being renamable,
so it can't be tested but the code is there)
- %q %x for strings
- error handling now done with panic/recover
R=rsc
CC=golang-dev
https://golang.org/cl/1458041
- reimplement integer scanning to handle renamed basic integer types
- start implementation of Fscanf and Scanf; not yet ready for general use.
This intermediate CL is a useful checkpoint. A similar change needs to be
done for float and complex, but it seemed worth getting the approach
reviewed before making those changes.
R=rsc
CC=golang-dev
https://golang.org/cl/1418041
Still to do:
- composite types
- user-defined scanners
- format-driven scanning
The package comment will be updated when more of the functionality is in place.
R=rsc
CC=golang-dev
https://golang.org/cl/1252045