BER allows the sender to choose either short form or long form where
both are legal, but DER requires the minimal one be used. Enforce this
and add a test. Fix one test which was not minimally-encoded and another
which would not distinguish rejecting the input because the long form
length wasn't minimally-encoded from rejecting it because long form was
chosen when short form was allowed.
Change-Id: I1b56fcca594dcdeddea9378b4fab427cbe7cd26d
Reviewed-on: https://go-review.googlesource.com/16517
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
This allows slices of custom types with byte as underlying type to be
decoded, fixing a regression introduced in CL 9371.
Fixes#12921.
Change-Id: I62a715eaeaaa912b6bc599e94f9981a9ba5cb242
Reviewed-on: https://go-review.googlesource.com/16303
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I1da1d718609eb6a7b78d29b173ec780bde22c687
Reviewed-on: https://go-review.googlesource.com/16422
Reviewed-by: Ralph Corderoy <ralph@inputplus.co.uk>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Addresses issue #12367.
Must be checked in before CL 14010.
Change-Id: I4523a1de112ed02371504e27882659bce8028a9f
Reviewed-on: https://go-review.googlesource.com/14012
Reviewed-by: Russ Cox <rsc@golang.org>
Addresses issue #12367.
Must be checked in before CL 14010.
Change-Id: I7233c3a62d4f55d0ac7e8a87df5fc4ee7beb7207
Reviewed-on: https://go-review.googlesource.com/14011
Reviewed-by: Russ Cox <rsc@golang.org>
As correctly mentioned in #11883, encodeState.string and
encodeState.stringBytes never return an error.
This CL removes the error from the function signatures and somewhat
simplifies call sites.
Fixes#11883
Change-Id: I1d1853d09631c545b68b5eea86ff7daa2e0ca10b
Reviewed-on: https://go-review.googlesource.com/15836
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The documentation listing err == EOF can be confusing to newcomers
to the language who are looking for the relevant documentation for
that error.
Change-Id: I301885950d0e1d0fbdf3a1892fca86eac7a0c616
Reviewed-on: https://go-review.googlesource.com/15806
Reviewed-by: Andrew Gerrand <adg@golang.org>
Also add a unit test to lock this behavior into the API.
Fixes#12016
Change-Id: Ib6ec6e7948f0705f3504ede9143b5dc4e790fc44
Reviewed-on: https://go-review.googlesource.com/15171
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The fields step and redoState of struct scanner are now defined as
`func(s *scanner, c byte) int` instead of
`func(s *scanner, c int) int`, since bytes are sufficient.
Further changes improve the consistency in the scanner.go file.
Change-Id: Ifb85f2130d728d2b936d79914d87a1f0b5c6ee7d
Reviewed-on: https://go-review.googlesource.com/14801
Reviewed-by: Andrew Gerrand <adg@golang.org>
Run-TryBot: Andrew Gerrand <adg@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is understood, obvious (to me), and well known but has not been clearly documented.
Fixes#11117.
Change-Id: Ib2b1e318924748d1eac0d735ad6286533be7fd39
Reviewed-on: https://go-review.googlesource.com/14693
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Currently, the xml.Decoder's Token routine returns successfully for
XML input that does not properly close root start elements (and any
unclosed descendants). For example, all the following inputs
<root>
<root><foo>
<root><foo></foo>
cause Token to return with nil and io.EOF, indicating a successful
parse.
This change fixes that. It leaves the semantics of RawToken intact.
Fixes#11405
Change-Id: I6f1328c410cf41e17de0a93cf357a69f12c2a9f7
Reviewed-on: https://go-review.googlesource.com/14315
Reviewed-by: Nigel Tao <nigeltao@golang.org>
This is not a functional change. nr is a uint64 and can never be less
than zero, remove the no-op comparison.
Fixes#11279
Change-Id: Iebb36cc8fe97428b503e65d01b5e67d2b2bc7369
Reviewed-on: https://go-review.googlesource.com/13876
Run-TryBot: Todd Neal <todd@tneal.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Could go in 1.5, although not critical.
See also #12107
Change-Id: I7f1608b58581d21df4db58f0db654fef79e33a90
Reviewed-on: https://go-review.googlesource.com/13481
Reviewed-by: Dave Cheney <dave@cheney.net>
Rename test name from Http to HTTP, and fix some style nits.
Change-Id: I00fe1cecd69ca2f50be86a76ec90031c2f921707
Reviewed-on: https://go-review.googlesource.com/12760
Reviewed-by: Andrew Gerrand <adg@golang.org>
The new Token API is meant to sit on the side of the Decoder,
so that you only get the new code (and any latent bugs in it)
if you are actively using the Token API.
The unconditional use of dec.peek in dec.tokenPrepareForDecode
violates that intention.
Change tokenPrepareForDecode not to call dec.peek unless needed
(because the Token API has advanced the state).
This restores the old code path behavior, no peeking allowed.
I checked by patching in the new tests from CL 12726 that
this change suffices to "fix" the error handling bug in dec.peek.
Obviously that bug should be fixed too, but the point is that
with this CL, bugs in dec.peek do not affect plain use of Decode
or Unmarshal.
I also checked by putting a panic in dec.peek that the only
tests that now invoke peek are:
TestDecodeInStream
ExampleDecoder_Token
ExampleDecoder_Decode_stream
and those tests all invoke dec.Token directly.
Change-Id: I0b242d0cb54a9c830548644670dc5ab5ccef69f2
Reviewed-on: https://go-review.googlesource.com/12740
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Peter Waldschmidt <peter@waldschmidt.com>
This race was identified in #9796, but a sequence of fixes
proposed in golang.org/cl/4152 were rolled into
golang.org/cl/5910 which both fixed the race and
modified the name space behavior.
We rolled back the name space changes and lost the race fix.
Fix the race separate from the name space changes,
following the suggestion made by Roger Peppe in
https://go-review.googlesource.com/#/c/4152/7/src/encoding/xml/marshal.go@897Fixes#9796.
Fixes#11885.
Change-Id: Ib2b68982da83dee9e04db8b8465a8295259bba46
Reviewed-on: https://go-review.googlesource.com/12687
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
This change adds new methods to Decoder.
* Decoder.Token steps through a JSON document, returning a value for each token.
* Decoder.Decode unmarshals the entire value at the token stream's current
position (in addition to its existing function in a stream of JSON values)
Fixes#6050.
Fixes#6499.
Change-Id: Iff283e0e7b537221ae256392aca6529f06ebe211
Reviewed-on: https://go-review.googlesource.com/9073
Reviewed-by: Russ Cox <rsc@golang.org>
There is clearly work to do here with respect to xml name spaces,
but I don't believe the changes in this cycle are clearly correct.
The changes in this cycle have visible impact on the generated xml,
possibly breaking existing programs, and yet it's not clear that they
are the end of the story: there is still significant confusion about how
name spaces work or should work (see #9519, #9775, #8167, #7113).
I would like to wait to make breaking changes until we completely
understand what the behavior should be and can evaluate the benefit
of those breaking changes. My main concern here is that we will break
programs in Go 1.5 for the sake of name space adjustments and then
while trying to fix those other bugs we'll break programs in Go 1.6 too.
Let's wait until we know all the changes we want to make before we
decide whether or how to break existing programs.
This CL reverts:
5ae822b encoding/xml: minor changes
bb7e665 encoding/xml: fix xmlns= behavior
9f9d66d encoding/xml: fix default namespace of tags
b69ea01 encoding/xml: fix namespaces in a>b tags
3be158d encoding/xml: encoding name spaces correctly
and adjusts tests from
a9dddb5 encoding/xml: add more EncodeToken tests.
to expect Go 1.4 behavior.
I have confirmed that the name space parts of the test suite
as of this CL passes against the Go 1.4 encoding/xml package,
indicating that this CL successfully restores the Go 1.4 behavior.
(Other tests do not, but that's because there were some real
bug fixes in this cycle that are being kept. Specifically, the
tests that don't pass in Go 1.4 are TestMarshal's NestedAndComment
case, TestEncodeToken's encoding of newlines, and
TestSimpleUseOfEncodeToken returning an error for invalid
token types.)
I also checked that the Go 1.4 tests pass when run against
this copy of the sources.
Fixes#11841.
Change-Id: I97de06761038b40388ef6e3a55547ff43edee7cb
Reviewed-on: https://go-review.googlesource.com/12570
Reviewed-by: Nigel Tao <nigeltao@golang.org>
EncodeToken takes a Token (i.e. an interface{}) as a parameter,
and expects a value of type StartElement, EndElement, CharData,
Comment, ProcInst, or Directive.
If a pointer is passed instead, or any type which does not match
this list, the token is silently ignored.
Added a default case in the type switch to issue a proper error
when the type is invalid.
The behavior could be later improved by allowing pointers to
token to be accepted as well, but not for go1.5.
Fixes#11719
Change-Id: Ifd13c1563450b474acf66d57669fdccba76c1949
Reviewed-on: https://go-review.googlesource.com/12252
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
JSON decoding currently fails for null values bound to any type
which does implement the JSON Unmarshaler interface without checking
for null values (such as time.Time).
It also fails for types implementing the TextUnmarshaler interface.
The expected behavior of the JSON decoding engine in such case is
to process null by keeping the value unchanged without producing
any error.
Make sure null values are handled by the decoding engine itself,
and never passed to the UnmarshalText or UnmarshalJSON methods.
Fixes#9037
Change-Id: I261d85587ba543ef6f1815555b2af9311034d5bb
Reviewed-on: https://go-review.googlesource.com/9376
Reviewed-by: Russ Cox <rsc@golang.org>
When building a directive, the current sanity check prevents
a '>' to be used, which makes a DOCTYPE directive with an
internal subset be rejected. It is accepted by the parser
though, so what can be parsed cannot be encoded.
Improved the corresponding sanity check to mirror the behavior
of the parser (in the way it handles angle brackets, quotes,
and comments).
Fixes#10158
Change-Id: Ieffea9f870f2694548e12897f8f47babc0ea4414
Reviewed-on: https://go-review.googlesource.com/11630
Reviewed-by: Russ Cox <rsc@golang.org>
The "string" option only applies for strings, floats, integers, and
booleans as per the documentation. So when decoding ignore the "string"
option if the value is not of one of the types mentioned. This matches
the Marshal step which also ignores the "string" option for invalid
types.
Fixes#9812
Change-Id: I0fb2b43d0668bc0e2985886d989abbf2252070e2
Reviewed-on: https://go-review.googlesource.com/10183
Reviewed-by: Russ Cox <rsc@golang.org>
The previous commit (git 2ae77376) just did golang.org. This one
includes golang.org subdomains like blog, play, and build.
Change-Id: I4469f7b307ae2a12ea89323422044e604c5133ae
Reviewed-on: https://go-review.googlesource.com/12071
Reviewed-by: Rob Pike <r@golang.org>
The one in misc/makerelease/makerelease.go is particularly bad and
probably warrants rotating our keys.
I didn't update old weekly notes, and reverted some changes involving
test code for now, since we're late in the Go 1.5 freeze. Otherwise,
the rest are all auto-generated changes, and all manually reviewed.
Change-Id: Ia2753576ab5d64826a167d259f48a2f50508792d
Reviewed-on: https://go-review.googlesource.com/12048
Reviewed-by: Rob Pike <r@golang.org>
When an xmlns="..." attribute was explicitly generated,
it was being ignored because the name space on the
attribute was assumed to have been explicitly set (to the empty
name space) and it's not possible to have an element in the
empty name space when there is a non-empty name space set.
We fix this by recording when a default name space has been
explicitly set and setting the name space of the element to that
so printer.defineNS can do its work correctly.
We do not attempt to add our own xmlns="..." attribute
when one is explicitly set.
We also add tests for EncodeElement, as that's the only way
to attain coverage of some of the changed behaviour.
Some other test coverage is also increased, although
more work remains to be done in this area.
This change was jointly developed with Martin Hilton (mhilton on github).
Fixes#11431.
Change-Id: I7b85e06eea5b18b2c15ec16dcbd92a8e1d6a9a4e
Reviewed-on: https://go-review.googlesource.com/11635
Reviewed-by: Russ Cox <rsc@golang.org>
Invalid UTF-8 triggers an error when marshaling but, previously, not
when unmarshaling. This means that ASN.1 structures were not
round-tripping.
This change makes invalid UTF-8 in a string marked as UTF-8 to be an
error when Unmarshaling.
Fixes#11126.
Change-Id: Ic37be84d21dc5c03983525e244d955a8b1e1ff14
Reviewed-on: https://go-review.googlesource.com/11056
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
The time package does normalisation of times: for example day zero is
converted to the last day of the previous month and the 31st of February
is moved into March etc. This makes the ASN.1 parsing a little
worryingly lax.
This change causes the parser to reserialise parsed times to ensure that
they round-trip correctly and thus were not normalised.
Fixes#11134.
Change-Id: I3988bb95153a7b33d64ab861fbe51b1a34a359e9
Reviewed-on: https://go-review.googlesource.com/11094
Run-TryBot: Adam Langley <agl@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Check that if a version is declared, for example
in '<?xml version="XX" ?>', version must be '1.0'.
Change-Id: I16ba9f78873a5f31977dcf75ac8e671fe6c08280
Reviewed-on: https://go-review.googlesource.com/8961
Reviewed-by: Russ Cox <rsc@golang.org>
When the scanner receives a non-whitespace character in stateEndTop,
it creates an error message and caches it to return on the next
transition. nextValue() uses the scanner to sub-scan for a value
inside a larger JSON structure. Since stateEndTop is triggered
*after* the ending byte, whatever character immediately follows the
sub-value gets pulled into the scanner's state machine as well.
Even though it is not used and doesn't cause an error, it does
cause the state machine to allocate an error that will never be used.
The fix is to probe the state machine with whitespace after
scanEndObject or scanEndArray to see if the next character would
result in a scanEnd state transition. If so, we can return right
away without processing the next character and avoid triggering
an allocation.
benchmark old ns/op new ns/op delta
BenchmarkCodeEncoder 17022194 16611336 -2.41%
BenchmarkCodeMarshal 18443250 18090144 -1.91%
BenchmarkCodeDecoder 61502053 61010936 -0.80%
BenchmarkCodeUnmarshal 61410829 60363605 -1.71%
BenchmarkCodeUnmarshalReuse 59124836 58361772 -1.29%
BenchmarkUnmarshalString 602 603 +0.17%
BenchmarkUnmarshalFloat64 535 537 +0.37%
BenchmarkUnmarshalInt64 482 482 +0.00%
BenchmarkIssue10335 1206 799 -33.75%
BenchmarkSkipValue 17605751 18355391 +4.26%
BenchmarkEncoderEncode 612 604 -1.31%
benchmark old MB/s new MB/s speedup
BenchmarkCodeEncoder 114.00 116.82 1.02x
BenchmarkCodeMarshal 105.21 107.27 1.02x
BenchmarkCodeDecoder 31.55 31.81 1.01x
BenchmarkCodeUnmarshal 31.60 32.15 1.02x
BenchmarkSkipValue 111.63 107.07 0.96x
benchmark old allocs new allocs delta
BenchmarkIssue10335 11 4 -63.64%
BenchmarkEncoderEncode 2 2 +0.00%
benchmark old bytes new bytes delta
BenchmarkIssue10335 376 272 -27.66%
BenchmarkEncoderEncode 40 40 +0.00%
Fixes#10335
Change-Id: I3d4f2b67f7a038adfb33ba48bb6b680f528baf18
Reviewed-on: https://go-review.googlesource.com/9074
Reviewed-by: Russ Cox <rsc@golang.org>
This was found while fuzzing another program, triggering a panic in
x509.ParseECPrivateKey.
Fixes#11154
Change-Id: Ief35ead38adf14caec4d37b9eacf8a92e67cd1e6
Reviewed-on: https://go-review.googlesource.com/10712
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
These were found by grepping the comments from the go code and feeding
the output to aspell.
Change-Id: Id734d6c8d1938ec3c36bd94a4dbbad577e3ad395
Reviewed-on: https://go-review.googlesource.com/10941
Reviewed-by: Aamir Khan <syst3m.w0rm@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The documentation for quick.Value says that it "returns an arbitrary
value of the given type." In spite of this, nil values for pointers were
never generated, which seems more like an oversight than an intentional
choice.
The lack of nil values meant that testing recursive type like
type Node struct {
Next *Node
}
with testing/quick would lead to a stack overflow since the data
structure would never terminate.
This change may break tests that don't check for nil with pointers
returned from quick.Value. Two such instances were found in the standard
library, one of which was in the testing/quick package itself.
Fixes#8818.
Change-Id: Id390dcce649d12fbbaa801ce6f58f5defed77e60
Reviewed-on: https://go-review.googlesource.com/10821
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
This CL resets the parent stack when printing a character or comment field struct.
In the case of XML elements, the previous parents stack must be considered. However,
charadata or comment fields can't be printed in other fields so it seems required to reset
the parent stack each time a chardata or comment field is printed.
Fixes#5072
Change-Id: I84f61c9bfce94133cd0c076c11211b9be5b4b1ac
Reviewed-on: https://go-review.googlesource.com/9910
Reviewed-by: Nigel Tao <nigeltao@golang.org>
Reviewed-by: roger peppe <rogpeppe@gmail.com>
