Found with mvdan.cc/unindent. It skipped the cases where parentheses
would need to be added, where comments would have to be moved elsewhere,
or where actions and simple logic would mix.
One of them was of the form "err != nil && err == io.EOF", so the first
part was removed.
Change-Id: Ie504c2b03a2c87d10ecbca1b9270069be1171b91
Reviewed-on: https://go-review.googlesource.com/57690
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Most of these are return values that were part of a receiving parameter,
so they're still accessible.
A few others are not, but those have never had a use.
Found with github.com/mvdan/unparam, after Kevin Burke's suggestion that
the tool should also warn about unused result parameters.
Change-Id: Id8b5ed89912a99db22027703a88bd94d0b292b8b
Reviewed-on: https://go-review.googlesource.com/55910
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Went mainly for the ones that make no sense, such as the ones
mid-sentence or after commas.
Change-Id: Ie245d2c19cc7428a06295635cf6a9482ade25ff0
Reviewed-on: https://go-review.googlesource.com/57293
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The destination slice does not need to be created at all. The source
slice itself can be used as the destination because the decode loop
increments by one and then the 'seen' byte is not used anymore. Therefore
the decoded byte can be stored in that index of the source slice itself.
This trick cannot be applied to EncodeString() because in that case,
the destination slice is large than the source. And for a single byte
in the source slice, two bytes in the destination slice is written.
func BenchmarkDecodeString(b *testing.B) {
for i := 0; i < b.N; i++ {
DecodeString("0123456789abcdef")
}
}
name old time/op new time/op delta
DecodeString 71.0ns ± 6% 58.0ns ± 0% -18.28% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
DecodeString 16.0B ± 0% 8.0B ± 0% -50.00% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
DecodeString 1.00 ± 0% 1.00 ± 0% ~ (all equal)
Change-Id: Id98db4e712444557a804155457a4dd8d1b8b416d
Reviewed-on: https://go-review.googlesource.com/55611
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The parser mistakenly assumed it could always fold \r\n into \n, which
is not true since a \r\n inside a quoted fields has no special meaning
and should be kept as is.
Fix this by not folding \r\n to \n inside quotes fields.
Fixes#21201
Change-Id: Ifebc302e49cf63e0a027ee90f088dbc050a2b7a6
Reviewed-on: https://go-review.googlesource.com/52810
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The tests for error scenarios were done by manually checking
error strings. Improved them by checking the actual error type
instead of just the string.
Printing the actual error in case of failure instead of a
generic string.
Also added a new scenario with both an invalid byte and an
invalid length string to verify that the length is checked first
before doing any computation.
Change-Id: Ic2a19a6d6058912632d597590186ee2d8348cb45
Reviewed-on: https://go-review.googlesource.com/55256
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Errors returned by Reader contain the line where the Reader originally
encountered the error. This can be suboptimal since that line does not
always correspond with the line the current record/field started at.
This can easily happen with LazyQuotes as seen in #19019, but also
happens for example when a quoted fields has no closing quote and
the parser hits EOF before it finds another quote.
When this happens finding the erroneous field can be somewhat
complicated and time consuming, and in most cases it would be better to
report the line where the record started.
This change updates Reader to keep track of the line on which a record
begins and uses it for errors instead of the current line, making it
easier to find errors.
Although a user-visible change, this should have no impact on existing
code, since most users don't explicitly work with the line in the error
and probably already expect the new behaviour.
Updates #19019
Change-Id: Ic9bc70fad2651c69435d614d537e7a9266819b05
Reviewed-on: https://go-review.googlesource.com/52830
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I27ff99aa7abb070f6ae79c8f964aa9bd6a83b89d
Reviewed-on: https://go-review.googlesource.com/53730
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This change fixes the remaining examples where the raw strings had
suboptimal indentation (one level too many) when viewed in godoc.
Follows CL 48910.
Fixes#21026.
Change-Id: Ifc0dae3fa899a9fff8b1ff958414e2fe6852321d
Reviewed-on: https://go-review.googlesource.com/50990
Run-TryBot: Dmitri Shuralyov <shurcool@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
https://golang.org/cl/33773 fixes the JSON marshaler to avoid serializing
embedded fields on unexported types of non-struct types. However, Go allows
embedding pointer to types, so the check for whether the field is a non-struct
type must first dereference the pointer to get at the underlying type.
Furthermore, due to a edge-case in the behavior of StructField.PkgPath not
being a reliable indicator of whether the field is unexported (see #21122),
we use our own logic to determine whether the field is exported or not.
The logic in this CL may be simplified depending on what happens in #21122.
Fixes#21121
Updates #21122
Change-Id: I8dfd1cdfac8a87950df294a566fb96dfd04fd749
Reviewed-on: https://go-review.googlesource.com/50711
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The existing example for Decoder.Decode (Stream) had excessive
indentation in the godoc interface for the const jsonStream,
making it hard to read. This fixes the indentation in the
example_test.go to improve the readability in godoc.
Helps #21026.
Change-Id: I16f56b82182da1dcc73cca44e535a7f5695e975d
Reviewed-on: https://go-review.googlesource.com/48910
Reviewed-by: Dmitri Shuralyov <shurcool@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
CL 47341 added support for decoding non-padded messages. But DecodedLen
still returned a multiple of 5 for messages without a padding, even
though it is possible to calculate the len exactly when using NoPadding.
This change makes DecodedLen return the exact number of bytes that
will be written. A change to the decoding logic is also made so that it
can handle this case.
DecodedLen now has the same behaviour as DecodedLen in encoding/base64.
Fixes#20854
Change-Id: I729e0b1c0946c866fb675c854f835f366dd4b5a4
Reviewed-on: https://go-review.googlesource.com/47710
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
And some double space after period cleanup while I'm here.
I guess my previous regexps missed these. My next cleaner should
probably use go/ast instead of perl.
Updates #20221
Change-Id: Idb051e7ac3a7fb1fb86e015f709e32139d065d92
Reviewed-on: https://go-review.googlesource.com/47094
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Use 2 slashes, space, then tab. This is more consistent, and removes
inadvertent leading space.
Change-Id: I383770ed4eb8ac17c78c7ae5675b553d4fb70b1e
Reviewed-on: https://go-review.googlesource.com/46726
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
ascii85_test.go contains a variable called bigtest that is used as
test data for TestDecoderBuffering and TestEncoderBuffering. The
variable is initialised to a copy of the last element of the pairs
slice. When the variable was first added the last element of this
slice contained a sizable test case, 342 encoded characters. However,
https://golang.org/cl/5970078 added a new element to the end of the pairs
slice without updating bigtest. As the new element contained only 1 byte
of encoded data bigtest became very small test. This commit fixes the
problem by resetting bigtest to its original value and making its
initialisation independent of the layout of pairs. All the unit tests
still pass.
Change-Id: If7fb609ced9da93a2321dfd8372986b2fa772fd5
Reviewed-on: https://go-review.googlesource.com/46475
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The ascii85, base32 and base64 packages all contain a test called
TestDecoderBuffering. Each of these tests contain a loop that ignores
the error returned from the Read method of their decoders. The result
being that the tests loop for ever if the decoders actually return an
error. This commit fixes the issue by terminating the loops if an error
occurs and failing the tests with a suitable error message.
Change-Id: Idb385673cf9f3f6f8befe4288b4be366ab0985fd
Reviewed-on: https://go-review.googlesource.com/46010
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some of the _test.go files in the encoding packages contain a private
function called testEqual that calls testing.Errorf if the arguments
passed to it are unequal. The line numbers output by such calls to
Errorf identify the failure as being in testEqual itself which is not
very useful. This commit fixes the problem by adding a call to the
new t.Helper method in each of the testEqual functions. The line
numbers output when errors do occur now identify the real source of
the error.
Change-Id: I582d1934f40ef2b788116c3811074c67ea882021
Reviewed-on: https://go-review.googlesource.com/45871
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Marshal must process unexported embedded fields of struct type,
looking for exported fields in those structs. However, it must
not process unexported embedded fields of non-struct type.
For example, consider:
type t1 struct {
X int
}
type t2 int
type T struct {
t1
t2
}
When considering T, Marshal must process t1 to find t1.X.
Marshal must not process t2, but it was. Fix that.
Fixes#18009
Change-Id: I62ba0b65ba30fd927990e101a26405a9998787a3
Reviewed-on: https://go-review.googlesource.com/33773
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
A number of issues in decoder.Read and newlineFilteringReader.Read were
preventing errors from the reader supplying the encoded data from being
propagated to the caller. Fixing these issues revealed some additional
problems in which valid decoded data was not always returned to the user
when errors were actually propagated.
This commit fixes both the error propagation and the lost decoded data
problems. It also adds some new unit tests to ensure errors are handled
correctly by decoder.Read. The new unit tests increase the test coverage
of this package from 96.2% to 97.9%.
Fixes#20044
Change-Id: I1a8632da20135906e2d191c2a8825b10e7ecc4c5
Reviewed-on: https://go-review.googlesource.com/42094
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In many cases the records returned by Reader.Read will only be used between calls
to Read and become garbage once a new record is read. In this case, instead of
allocating a new slice on each call to Read, we can reuse the last allocated slice
for successive calls to avoid unnecessary allocations.
This change adds a new field ReuseRecord to the Reader struct to enable this reuse.
ReuseRecord is false by default to avoid breaking existing code which dependss on
the current behaviour.
I also added 4 new benchmarks, corresponding to the existing Read benchmarks, which
set ReuseRecord to true.
Benchstat on my local machine (old is ReuseRecord = false, new is ReuseRecord = true)
name old time/op new time/op delta
Read-8 2.75µs ± 2% 1.88µs ± 1% -31.52% (p=0.000 n=14+15)
ReadWithFieldsPerRecord-8 2.75µs ± 0% 1.89µs ± 1% -31.43% (p=0.000 n=13+13)
ReadWithoutFieldsPerRecord-8 2.77µs ± 1% 1.88µs ± 1% -32.06% (p=0.000 n=15+15)
ReadLargeFields-8 55.4µs ± 1% 54.2µs ± 0% -2.07% (p=0.000 n=15+14)
name old alloc/op new alloc/op delta
Read-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 664B ± 0% 24B ± 0% -96.39% (p=0.000 n=15+15)
ReadLargeFields-8 3.94kB ± 0% 2.98kB ± 0% -24.39% (p=0.000 n=15+15)
name old allocs/op new allocs/op delta
Read-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadWithoutFieldsPerRecord-8 18.0 ± 0% 8.0 ± 0% -55.56% (p=0.000 n=15+15)
ReadLargeFields-8 24.0 ± 0% 12.0 ± 0% -50.00% (p=0.000 n=15+15)
Fixes#19721
Change-Id: I79b14128bb9bb3465f53f40f93b1b528a9da6f58
Reviewed-on: https://go-review.googlesource.com/41730
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Mostly unnecessary *testing.T arguments.
Found with github.com/mvdan/unparam.
Change-Id: Ifb955cb88f2ce8784ee4172f4f94d860fa36ae9a
Reviewed-on: https://go-review.googlesource.com/41691
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There were a number of places in crypto/x509 that used hardcoded
representations of the ASN.1 NULL type, in both byte slice and
RawValue struct forms. This change adds two new exported vars to
the asn1 package for working with ASN.1 NULL in both its forms, and
converts all usages from the x509 package.
In addition, tests were added to exercise Marshal and Unmarshal on
both vars.
See #19446 for discussion.
Change-Id: I63dbd0835841ccbc810bd6ec794360a84e933f1e
Reviewed-on: https://go-review.googlesource.com/38660
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
Returns at the end of func bodies where the funcs have no return values
are pointless.
Change-Id: I0da5ea78671503e41a9f56dd770df8c919310ce5
Reviewed-on: https://go-review.googlesource.com/41093
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 27254 changed hextable to a byte array for performance.
CL 28219 fixed the compiler so that that is no longer necessary.
As Kirill notes in #15808, a string is preferable
as the linker can easily de-dup it.
So go back. No performance changes.
Change-Id: Ibef7d21d0f2507968a0606602c5dd57ed4a85b1b
Reviewed-on: https://go-review.googlesource.com/40970
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The current implementation uses a max of 28 bits when decoding an
ObjectIdentifier. This change makes it so that an int64 is used to
accumulate up to 35 bits. If the resulting data would not overflow
an int32, it is used as an int. Thus up to 31 bits may be used to
represent each subidentifier of an ObjectIdentifier.
Fixes#19933
Change-Id: I95d74b64b24cdb1339ff13421055bce61c80243c
Reviewed-on: https://go-review.googlesource.com/40436
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
The header is literally
Key: Value
If the value or the key has leading or trailing spaces, those will
be lost by the round trip.
Found because testing/quick returns different values now.
Change-Id: I0f574bdbb5990689509c24309854d8f814b5efa0
Reviewed-on: https://go-review.googlesource.com/39211
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When unmarshaling, if an element is empty, eg. '<tag></tag>', and
destination type is int, uint, float or bool, do not attempt to parse
value (""). Set to its zero value instead.
Fixes#13417
Change-Id: I2d79f6d8f39192bb277b1a9129727d5abbb2dd1f
Reviewed-on: https://go-review.googlesource.com/38386
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I2d155c838935cd8427abd142a462ff4c56829715
Reviewed-on: https://go-review.googlesource.com/37948
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This paragraph has been added, as the notion was missing from the
documentation.
If a value is passed to Encode and the type is not a struct (or pointer to struct,
etc.), for simplicity of processing it is represented as a struct of one field.
The only visible effect of this is to encode a zero byte after the value, just as
after the last field of an encoded struct, so that the decode algorithm knows when
the top-level value is complete.
Fixes#16978
Change-Id: I5f008e792d1b6fe80d2e026a7ff716608889db32
Reviewed-on: https://go-review.googlesource.com/38414
Reviewed-by: Ian Lance Taylor <iant@golang.org>
JSON decoding performs poorly for unmapped and ignored fields. We noticed better
performance when unmarshalling unused fields. The loss comes mostly from calls
to scanner.error as described at #17914.
benchmark old ns/op new ns/op delta
BenchmarkIssue10335-8 431 408 -5.34%
BenchmarkUnmapped-8 1744 1314 -24.66%
benchmark old allocs new allocs delta
BenchmarkIssue10335-8 4 3 -25.00%
BenchmarkUnmapped-8 18 4 -77.78%
benchmark old bytes new bytes delta
BenchmarkIssue10335-8 320 312 -2.50%
BenchmarkUnmapped-8 568 344 -39.44%
Fixes#17914, improves #10335
Change-Id: I7d4258a94eb287c0fe49e7334795209b90434cd0
Reviewed-on: https://go-review.googlesource.com/33276
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Tinkering with the gob package shows that is currently possible to
*completely destroy* Int slices encoding without triggering a single
test failure.
The various encInt{8,16,32,64}Slice methods are only called during the
execution of the GobMapInterfaceEncode test, which only encodes a few
slices of length exactly 1 and then just checks that the error
returned by Encode is nil (without trying to Decode back the data).
This patch adds a few tests for signed integer slices encoding.
Change-Id: Ifaaee2f32132873118b241f79aa8203e4ad31416
Reviewed-on: https://go-review.googlesource.com/38066
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Replace 'does not contains' with 'does not contain' where it appears
in the source code.
Change-Id: Ie7266347c429512c8a41a7e19142afca7ead3922
Reviewed-on: https://go-review.googlesource.com/37887
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Document and check that the alphabet cannot contain '\n' or '\r'.
Document that the alphabet cannot contain the padding character.
Document that the padding character must be equal or bellow '\xff'.
Document that the padding character must not be '\n' or '\r'.
Fixes#19343Fixes#19318
Change-Id: I6de0034d347ffdf317d7ea55d6fe38b01c2c4c03
Reviewed-on: https://go-review.googlesource.com/37838
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fix Decode to return the correct illegal data index from a corrupted
input that contains whitespaces.
Fixes#19406
Change-Id: Ib2b2b6ed7e41f024d0da2bd035caec4317c2869c
Reviewed-on: https://go-review.googlesource.com/37837
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Found by github.com/mvdan/unparam.
Change-Id: Ic97f05a2ecb5b17caa36aafe403e2266abea3e0e
Reviewed-on: https://go-review.googlesource.com/37836
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Found by github.com/mvdan/unparam.
Change-Id: I5a6664cceeba1cf1c2f3236ddf4db5ce7a64b02a
Reviewed-on: https://go-review.googlesource.com/37835
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously the code didn't check for extra data after the final five
dashes of the ending line of a PEM block.
Fixes#19147Fixes#7042
Change-Id: Idaab2390914a2bed8c2c12b14dfb6d68233fdfec
Reviewed-on: https://go-review.googlesource.com/37147
Reviewed-by: Adam Langley <agl@golang.org>
The new tests in this CL have been checked against Go 1.7 as well
and all pass in Go 1.7, with the one exception noted in a comment
(an intentional change to omitempty already present before this CL).
CL 15684 made the intentional change to omitempty.
This CL fixes bugs introduced along the way.
Most of these are corner cases that are arguably not that important,
but they've always worked all the way back to Go 1, and someone
cared enough to file #19063. The most significant problem found
while adding tests is that in the case of a nil *string field with
`xml:",chardata"`, the existing code silently stops processing not just
that field but the entire remainder of the struct.
Even if #19063 were not worth fixing, this chardata bug would be.
Fixes#19063.
Change-Id: I318cf8f9945e1a4615982d9904e109fde577ebf9
Reviewed-on: https://go-review.googlesource.com/36954
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
... and same for stores. This does for binary.BigEndian.Uint16() what
was already done for Uint32 and Uint64 with BSWAP in 10f75748 (CL 32222).
Here is how generated code changes e.g. for the following function
(omitting saying the same prologue/epilogue):
func get16(b [2]byte) uint16 {
return binary.BigEndian.Uint16(b[:])
}
"".get16 t=1 size=21 args=0x10 locals=0x0
// before
0x0000 00000 (x.go:15) MOVBLZX "".b+9(FP), AX
0x0005 00005 (x.go:15) MOVBLZX "".b+8(FP), CX
0x000a 00010 (x.go:15) SHLL $8, CX
0x000d 00013 (x.go:15) ORL CX, AX
// after
0x0000 00000 (x.go:15) MOVWLZX "".b+8(FP), AX
0x0005 00005 (x.go:15) ROLW $8, AX
encoding/binary is speedup overall a bit:
name old time/op new time/op delta
ReadSlice1000Int32s-4 4.83µs ± 0% 4.83µs ± 0% ~ (p=0.206 n=4+5)
ReadStruct-4 1.29µs ± 2% 1.28µs ± 1% -1.27% (p=0.032 n=4+5)
ReadInts-4 384ns ± 1% 385ns ± 1% ~ (p=0.968 n=4+5)
WriteInts-4 534ns ± 3% 526ns ± 0% -1.54% (p=0.048 n=4+5)
WriteSlice1000Int32s-4 5.02µs ± 0% 5.11µs ± 3% ~ (p=0.175 n=4+5)
PutUint16-4 0.59ns ± 0% 0.49ns ± 2% -16.95% (p=0.016 n=4+5)
PutUint32-4 0.52ns ± 0% 0.52ns ± 0% ~ (all equal)
PutUint64-4 0.53ns ± 0% 0.53ns ± 0% ~ (all equal)
PutUvarint32-4 19.9ns ± 0% 19.9ns ± 1% ~ (p=0.556 n=4+5)
PutUvarint64-4 54.5ns ± 1% 54.2ns ± 0% ~ (p=0.333 n=4+5)
name old speed new speed delta
ReadSlice1000Int32s-4 829MB/s ± 0% 828MB/s ± 0% ~ (p=0.190 n=4+5)
ReadStruct-4 58.0MB/s ± 2% 58.7MB/s ± 1% +1.30% (p=0.032 n=4+5)
ReadInts-4 78.0MB/s ± 1% 77.8MB/s ± 1% ~ (p=0.968 n=4+5)
WriteInts-4 56.1MB/s ± 3% 57.0MB/s ± 0% ~ (p=0.063 n=4+5)
WriteSlice1000Int32s-4 797MB/s ± 0% 783MB/s ± 3% ~ (p=0.190 n=4+5)
PutUint16-4 3.37GB/s ± 0% 4.07GB/s ± 2% +20.83% (p=0.016 n=4+5)
PutUint32-4 7.73GB/s ± 0% 7.72GB/s ± 0% ~ (p=0.556 n=4+5)
PutUint64-4 15.1GB/s ± 0% 15.1GB/s ± 0% ~ (p=0.905 n=4+5)
PutUvarint32-4 201MB/s ± 0% 201MB/s ± 0% ~ (p=0.905 n=4+5)
PutUvarint64-4 147MB/s ± 1% 147MB/s ± 0% ~ (p=0.286 n=4+5)
( "a bit" only because most of the time is spent in reflection-like things
there, not actual bytes decoding. Even for direct PutUint16 benchmark the
looping adds overhead and lowers visible benefit. For code-generated encoders /
decoders actual effect is more than 20% )
Adding Uint32 and Uint64 raw benchmarks too for completeness.
NOTE I had to adjust load-combining rule for bswap case to match first 2 bytes
loads as result of "2-bytes load+shift" -> "loadw + rorw 8" rewrite. Reason is:
for loads+shift, even e.g. into uint16 var
var b []byte
var v uin16
v = uint16(b[1]) | uint16(b[0])<<8
the compiler eventually generates L(ong) shift - SHLLconst [8], probably
because it is more straightforward / other reasons to work on the whole
register. This way 2 bytes rewriting rule is using SHLLconst (not SHLWconst) in
its pattern, and then it always gets matched first, even if 2-byte rule comes
syntactically after 4-byte rule in AMD64.rules because 4-bytes rule seemingly
needs more applyRewrite() cycles to trigger. If 2-bytes rule gets matched for
inner half of
var b []byte
var v uin32
v = uint32(b[3]) | uint32(b[2])<<8 | uint32(b[1])<<16 | uint32(b[0])<<24
and we keep 4-byte load rule unchanged, the result will be MOVW + RORW $8 and
then series of byte loads and shifts - not one MOVL + BSWAPL.
There is no such problem for stores: there compiler, since it probably knows
store destination is 2 bytes wide, uses SHRWconst 8 (not SHRLconst 8) and thus
2-byte store rule is not a subset of rule for 4-byte stores.
Fixes#17151 (int16 was last missing piece there)
Change-Id: Idc03ba965bfce2b94fef456b02ff6742194748f6
Reviewed-on: https://go-review.googlesource.com/34636
Reviewed-by: Ilya Tocar <ilya.tocar@intel.com>
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Mention that it specifically returns x / 2, and do the same for
EncodedLen.
Change-Id: Ie334f5abecbc487caf4965abbcd14442591bef2a
Change-Id: Idfa413faad487e534489428451bf736b009293d6
Reviewed-on: https://go-review.googlesource.com/33191
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The tree is inconsistent about single l vs double l in those
words in documentation, test messages, and one error value text.
$ git grep -E '[Mm]arshall(|s|er|ers|ed|ing)' | wc -l
42
$ git grep -E '[Mm]arshal(|s|er|ers|ed|ing)' | wc -l
1694
Make it consistently a single l, per earlier decisions. This means
contributors won't be confused by misleading precedence, and it helps
consistency.
Change the spelling in one error value text in newRawAttributes of
crypto/x509 package to be consistent.
This change was generated with:
perl -i -npe 's,([Mm]arshal)l(|s|er|ers|ed|ing),$1$2,' $(git grep -l -E '[Mm]arshall' | grep -v AUTHORS | grep -v CONTRIBUTORS)
Updates #12431.
Follows https://golang.org/cl/14150.
Change-Id: I85d28a2d7692862ccb02d6a09f5d18538b6049a2
Reviewed-on: https://go-review.googlesource.com/33017
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We only support unmarshaling into a string or a []byte, but we
previously would try (and panic while) setting a slice of a different
type. The docs say ",innerxml" is ignored if the type is not string or
[]byte, so do that for other slices as well.
Fixes#15600.
Change-Id: Ia64815945a14c3d04a0a45ccf413e38b58a69416
Reviewed-on: https://go-review.googlesource.com/32919
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
I used the slowtests.go tool as described in
https://golang.org/cl/32684 on packages that stood out.
go test -short std drops from ~56 to ~52 seconds.
This isn't a huge win, but it was mostly an exercise.
Updates #17751
Change-Id: I9f3402e36a038d71e662d06ce2c1d52f6c4b674d
Reviewed-on: https://go-review.googlesource.com/32751
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL expands upon a change made in (http://golang.org/cl/21811)
to ensure that a nil RawMessage gets serialized as "null" instead of
being a nil slice.
The added check only triggers when the RawMessage is nil. We do not
handle the case when the RawMessage is non-nil, but empty.
Fixes#17704
Updates #14493
Change-Id: I0fbebcdd81f7466c5b78c94953afc897f162ceb4
Reviewed-on: https://go-review.googlesource.com/32472
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
- Like ",any" for elements, add ",any,attr" for attributes to allow
a mop-up field that gets any otherwise unmapped attributes.
- Map attributes to fields of type slice by extending the slice,
just like for elements.
- Allow storing an attribute into an xml.Attr directly, to provide
a way to record the name.
Combined, these three independent features allow
AllAttrs []Attr `xml:",any,attr"`
to collect all attributes not otherwise spoken for in a particular struct.
Tests based on CL 16292 by Charles Weill.
Fixes#3633.
Change-Id: I2d75817f17ca8752d7df188080a407836af92611
Reviewed-on: https://go-review.googlesource.com/30946
Reviewed-by: Quentin Smith <quentin@golang.org>
1. Define behavior for Unmarshal of JSON null into Unmarshaler and
TextUnmarshaler. Specifically, an Unmarshaler will be given the
literal null and can decide what to do (because otherwise
json.RawMessage is impossible to implement), and a TextUnmarshaler
will be skipped over (because there is no text to unmarshal), like
most other inappropriate types. Document this in Unmarshal, with a
reminder in UnmarshalJSON about handling null.
2. Test all this.
3. Fix the TextUnmarshaler case, which was returning an unmarshalling
error, to match the definition.
4. Fix the error that had been used for the TextUnmarshaler, since it
was claiming that there was a JSON string when in fact the problem was
NOT having a string.
5. Adjust time.Time and big.Int's UnmarshalJSON to ignore null, as is
conventional.
Fixes#9037.
Change-Id: If78350414eb8dda712867dc8f4ca35a9db041b0c
Reviewed-on: https://go-review.googlesource.com/30944
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
No functional changes here. Just makes next CL easier to read.
Change-Id: Icf7b2281b4da6cb59ff4edff05943b2ee288576a
Reviewed-on: https://go-review.googlesource.com/30945
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
There was an inconsistency between the (json encoding + documentation)
and the xml encoding implementation. Pointer to an empty value was
not being serialized (i.e simply ignored). Which had the effect of making
impossible to have a struct with a string field for which we wanted to
serialize the value ""
Fixes#5452
Change-Id: Id858701801158409be01e962d2cda843424bd22a
Reviewed-on: https://go-review.googlesource.com/15684
Reviewed-by: Russ Cox <rsc@golang.org>
If strict option is enabled, when decoding, instead of skip the padding
bits, it will do strict check to enforce they are set to zero.
Fixes#15656
Change-Id: I869fb725a39cc9dde44dbc4ff0046446e7abc642
Reviewed-on: https://go-review.googlesource.com/24964
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Struct fields can be suppressed in JSON serialization by "-" tags, but
that doesn't preclude generation of "-" object keys.
Document and verify the mechanism for doing so.
Change-Id: I7f60e1759cfee15cb7b2447cd35fab91c5b004e6
Reviewed-on: https://go-review.googlesource.com/21204
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The UnmarshalTypeError has two new fields Struct and Field,
used when constructing the error message.
Fixes#6716.
Change-Id: I67da171480a9491960b3ae81893770644180f848
Reviewed-on: https://go-review.googlesource.com/18692
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change float32/float64 formatting to use non-exponential form
for a slightly wider range, to more closely match ES6 JSON.stringify
and other JSON generators.
Most notably:
1e20 now formats as 100000000000000000000 (previously 1e+20)
1e-6 now formats as 0.000001 (previously 1e-06)
1e-7 now formats as 1e-7 (previously 1e-07)
This also brings the int64 and float64 formatting in line with each other,
for all shared representable values. For example both int64(1234567)
and float64(1234567) now format as "1234567", where before the
float64 formatted as "1.234567e+06".
The only variation now compared to ES6 JSON.stringify is that
Go continues to encode negative zero as "-0", not "0", so that
the value continues to be preserved during JSON round trips.
Fixes#6384.
Fixes#14135.
Change-Id: Ib0e0e009cd9181d75edc0424a28fe776bcc5bbf8
Reviewed-on: https://go-review.googlesource.com/30371
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit changes parseRecord to allocate a single string per record,
instead of per field, by using indexes into the raw record.
Benchstat (done with f69991c17)
name old time/op new time/op delta
Read-8 3.17µs ± 0% 2.78µs ± 1% -12.35% (p=0.016 n=4+5)
ReadWithFieldsPerRecord-8 3.18µs ± 1% 2.79µs ± 1% -12.23% (p=0.008 n=5+5)
ReadWithoutFieldsPerRecord-8 4.59µs ± 0% 2.77µs ± 0% -39.58% (p=0.016 n=4+5)
ReadLargeFields-8 57.0µs ± 0% 55.7µs ± 0% -2.18% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
Read-8 660B ± 0% 664B ± 0% +0.61% (p=0.008 n=5+5)
ReadWithFieldsPerRecord-8 660B ± 0% 664B ± 0% +0.61% (p=0.008 n=5+5)
ReadWithoutFieldsPerRecord-8 1.14kB ± 0% 0.66kB ± 0% -41.75% (p=0.008 n=5+5)
ReadLargeFields-8 3.86kB ± 0% 3.94kB ± 0% +1.86% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
Read-8 30.0 ± 0% 18.0 ± 0% -40.00% (p=0.008 n=5+5)
ReadWithFieldsPerRecord-8 30.0 ± 0% 18.0 ± 0% -40.00% (p=0.008 n=5+5)
ReadWithoutFieldsPerRecord-8 50.0 ± 0% 18.0 ± 0% -64.00% (p=0.008 n=5+5)
ReadLargeFields-8 66.0 ± 0% 24.0 ± 0% -63.64% (p=0.008 n=5+5)
For a simple application that I wrote, which reads in a CSV file (via
ReadAll) and outputs the number of rows read (15857625 rows), this change
reduces the total time on my notebook from ~58 seconds to ~48 seconds.
This reduces time and allocations (bytes) each by ~6% for a real world
CSV file at work (~230000 rows, 13 colums).
Updates #16791
Change-Id: Ia07177c94624e55cdd3064a7d2751fb69322d3e4
Reviewed-on: https://go-review.googlesource.com/24723
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Benchmarks broken off from https://golang.org/cl/24723 and modified to
allocate less in the places we're not trying to measure.
Updates #16791
Change-Id: I508e4cfeac60322d56f1d71ff1912f6a6f183a63
Reviewed-on: https://go-review.googlesource.com/30357
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I avoided anywhere in the compiler or things which might be used by
the compiler in the future, since they need to build with Go 1.4.
I also avoided anywhere where there was no benefit to changing it.
I probably missed some.
Updates #16721
Change-Id: Ib3c895ff475c6dec2d4322393faaf8cb6a6d4956
Reviewed-on: https://go-review.googlesource.com/30250
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
This change adds support for decoding and encoding the bool type. The
encoding is a single byte, with a zero value for false and a non-zero
value for true.
Closes#16856.
Change-Id: I1d1114b320263691473bb100cad0f380e0204186
Reviewed-on: https://go-review.googlesource.com/28514
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Documentation made reference to an unknown entity "DisableHTMLEscaping,"
but I think it actually meant the method "Encoder.SetEscapeHTML."
Fixes#17255
Change-Id: I18fda76f8066110caef85fd33698de83d632e646
Reviewed-on: https://go-review.googlesource.com/29931
Reviewed-by: Ian Lance Taylor <iant@golang.org>
