Roll back CL making primitive type unmarshal faster,
because it broke the Unmarshal of malformed data.
Add benchmarks for unmarshal of primitive types.
Update #3949.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7228061
Otherwise it's impossible to know how much data from the
json.Decoder's underlying Reader was actually consumed.
The old fix from golang.org/issue/1955 just added docs. This
provides an actual mechanism.
Update #1955
R=golang-dev, adg, rsc
CC=golang-dev
https://golang.org/cl/7181053
Go 1.0 behavior was to create an UnmarshalFieldError when a json value name matched an unexported field name. This error will no longer be created and the field will be skipped instead.
Fixes#4660.
R=adg, rsc, bradfitz
CC=golang-dev
https://golang.org/cl/7139049
bytes.Equal is simpler to read and should also be faster because
of short-circuiting and assembly implementations.
Change generated automatically using:
gofmt -r 'bytes.Compare(a, b) == 0 -> bytes.Equal(a, b)'
gofmt -r 'bytes.Compare(a, b) != 0 -> !bytes.Equal(a, b)'
R=golang-dev, dave, adg, rsc
CC=golang-dev
https://golang.org/cl/7038051
Add a check for this case and don't try to follow the anonymous
type's non-existent fields.
Fixes#4474.
R=rsc
CC=golang-dev
https://golang.org/cl/6945065
Allows encoding and decoding of maps with key of string kind, not just string type.
Fixes#3519.
R=rsc, dave
CC=golang-dev
https://golang.org/cl/6943047
Fixes#3559.
This makes Marshal handle fields marked ",any" instead of ignoring
them. That makes Marshal more symmetrical with Unmarshal, which seems
to have been a design goal.
Note some test cases were changed, because this patch changes
marshalling behavior. I think the previous behavior was buggy, but
there's still a backward-compatibility question to consider.
R=rsc
CC=golang-dev, n13m3y3r
https://golang.org/cl/6938068
reader.Read() can return both 0,nil and len(buf),err.
To be safe, we use io.ReadFull instead of doing reader.Read directly.
Fixes#3472.
R=bradfitz, rsc, ality
CC=golang-dev
https://golang.org/cl/6285050
As discussed in issue 2540, nulls are allowed for any type in JSON so they should not result in an error during Unmarshal.
Fixes#2540.
R=rsc
CC=golang-dev
https://golang.org/cl/6759043
Previously we converted a time to UTC *and* serialized the timezone of
the original time. With this change, we serialize a UTCTime in the
original timezone.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/6817048
Previously, multi-byte characters were not allowed. Also certain single-byte
characters, such as '-', were disallowed.
Fixes#3813.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6641052
Add commentary to explain better what's going on, but the
code change is a simple one-line reversal to the previous
form.
R=rsc
CC=golang-dev
https://golang.org/cl/6428072
They can generate huge amounts of memory, causing failure on
small machines. Also they can be very slow. So slow that one test
was commented out! We uncomment it and use a flag.
Fixes#3742.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/6373044
Gob decoding reads a whole message into memory and then
copies it into a bytes.Buffer. For large messages this wastes
an entire copy of the message. In this CL, we use a staging
buffer to avoid the large temporary.
Update #2539
RSS drops to 775MB from 1GB.
Active memory drops to 858317048 from 1027878136,
essentially the size of one copy of the input file.
R=dsymonds, nigeltao
CC=golang-dev
https://golang.org/cl/6392057
Previously, strings that didn't have an explicit ASN.1 string type
were taken to be ASN.1 PrintableStrings. This resulted in an error if
a unrepresentable charactor was included.
For compatibility reasons, I'm too afraid to switch the default string
type to UTF8String, but this patch causes untyped strings to become
UTF8Strings if they contain a charactor that's not valid in a
PrintableString.
Fixes#3791.
R=golang-dev, bradfitz, r, r
CC=golang-dev
https://golang.org/cl/6348074
Number represents the actual JSON text,
preserving the precision and
formatting of the original input.
R=rsc, iant
CC=golang-dev
https://golang.org/cl/6202068
Removes an incorrect code comment and some superfluous variables.
The comment I removed says that struct fields which implement
Unmarshaler must be pointers, even if they're in an addressable
struct. That's not the case, and there's already a test in decode_test.go
that demonstrates as much.
Encoding/json has quite a few assignments of reflect.Values to extra
variables – things like "iv := v" when there's no need to make a copy. I
think these are left over from a previous version of the reflect API. If they
aren't wanted, I wouldn't mind going through the package and getting
rid of the rest of them.
R=rsc
CC=golang-dev
https://golang.org/cl/6318047
Fixes a situation where a nested bad type would still
permit the outer type to install a working engine, leading
to inconsistent behavior.
Fixes#3273.
R=bsiegert, rsc
CC=golang-dev
https://golang.org/cl/6294067
This CL makes
type T struct { *U }
behave in a similar way to:
type T struct { U }
Fixes#3108.
R=golang-dev, rsc, gustavo
CC=golang-dev
https://golang.org/cl/5694044
Special case for encoding 4 zeros as 'z' didn't
update source slice, causing 'index out of bounds'
panic in destination slice.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5970078
asn1 didn't have an omitempty tag, so the list of additional primes in
an RSA private key was serialised as an empty SEQUENCE, even for
version 1 structures. This tripped up external code that didn't handle
v2.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5729062
UTCTime only has a two digit date field and year values from 50 should
be 1950, not 2050.
R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/5729063
The current package comment doesn't mention varints and
protocol buffers. Also, the first sentence is incomprehensible
without further context as "fixed-size values" is undefined.
R=rsc
CC=golang-dev
https://golang.org/cl/5715048
I've elected to omit escaping the output of Marshalers for now.
I haven't thought through the implications of that;
I suspect that double escaping might be the undoing of that idea.
Fixes#3127.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5694098
This also fixes MarshalIndent's example after the
recent formatting convention changes.
Fixes#2831.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5671062
We should, after Go 1, make them work the same as
package xml, that is, make them appear in the outer
struct. For now turn them off so that people do not
depend on the old behavior.
Fixing them is issue 3069.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5656102
When we build the encode engine for a recursive type, we
mustn't disregard the indirections or we can try to reuse an
engine at the wrong indirection level.
Fixes#3026.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5675087
Fix by setting the element type if we discover it's zero while building.
We could have fixed this better with foresight by doing the id setting in a
different sequence, but doing that now would break binary compatibility.
Fixes#2995.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5675083
It was being skipped due to an oversight.
Also adjust naming parameters for map type construction - makes debugging easier.
Prelude to issue 3026.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5674071
An unindented XML example is hard to follow. MarshalIndent
allows moving the example over to a test file (and fixing it).
R=golang-dev, r, gustavo, r, rsc
CC=golang-dev
https://golang.org/cl/5674050
go/doc: move Examples to go/ast
cmd/go: use go/doc to read examples
src/pkg: update examples to use new convention
This is to make whole file examples more readable. When presented as a
complete function, preceding an Example with its output is confusing.
The new convention is to put the expected output in the final comment
of the example, preceded by the string "output:" (case insensitive).
An idiomatic example looks like this:
// This example demonstrates Foo by doing bar and quux.
func ExampleFoo() {
// example body that does bar and quux
// Output:
// example output
}
R=rsc, gri
CC=golang-dev
https://golang.org/cl/5673053
* add -work option to save temporary files (Fixes issue 2980)
* fix go test -i to work with cgo packages (Fixes issue 2936)
* do not overwrite/remove empty directories or non-object
files during build (Fixes issue 2829)
* remove package main vs package non-main heuristic:
a directory must contain only one package (Fixes issue 2864)
* to make last item workable, ignore +build tags for files
named on command line: go build x.go builds x.go even
if it says // +build ignore.
* add // +build ignore tags to helper programs
R=golang-dev, r, r
CC=golang-dev
https://golang.org/cl/5674043
unsafe: delete Typeof, Reflect, Unreflect, New, NewArray
Part of issue 2955 and issue 2968.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5650069
encoding/xml: handle time.Time as recognized type
The long term plan is to define an interface that time.Time
can implement and that encoding/xml can call, but we are
not going to try to define that interface before Go 1.
Instead, special-case time.Time in package xml, because
it is such a fundamental type, as a stop-gap.
The eventual methods will behave this way.
Fixes#2793.
R=golang-dev, r, r, n13m3y3r
CC=golang-dev
https://golang.org/cl/5634051
This also changes the behavior of attribute marshalling so
that strings and byte slices are marshalled even if empty.
The omitempty flag may be used to obtain the previous behavior.
Fixes#2899.
R=rsc
CC=golang-dev
https://golang.org/cl/5645050
The function has a bizarre signature: it was the only public function there
that exposed the reflect package. Also, its definition is peculiar and hard to
explain. It doesn't merit being exported.
This is an API change but really, it should never have been exported and
it's certain very few programs will depend on it: it's too weird.
Fixes#2846.
R=golang-dev, gri, bradfitz
CC=golang-dev
https://golang.org/cl/5639054
Also bring the names in doc.go in line with the source.
More radical resolutions are possible but require substantial internal
changes for very little benefit. Fixing it this way lets us keep the
embedding, which has a huge simplifying effect, and guarantees
binary compatibility.
Fixes#2848.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5644045
Specifically, remove simply where it is claiming that the
code or the action to be carried out is simple, since the
reader might disagree.
R=golang-dev, bradfitz, gri
CC=golang-dev
https://golang.org/cl/5637048
Unexports runtime.MemStats and rename MemStatsType to MemStats.
The new accessor requires passing a pointer to a user-allocated
MemStats structure.
Fixes#2572.
R=bradfitz, rsc, bradfitz, gustavo
CC=golang-dev, remy
https://golang.org/cl/5616072
The practice encourages people to think this is the way to
create a bytes.Buffer when new(bytes.Buffer) or
just var buf bytes.Buffer work fine.
(html/token.go was missing the point altogether.)
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5637043
Consequently, remove many package Makefiles,
and shorten the few that remain.
gomake becomes 'go tool make'.
Turn off test phases of run.bash that do not work,
flagged with $BROKEN. Future CLs will restore these,
but this seemed like a big enough CL already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
Marshaler has a number of open areas that need
further thought (e.g. it doesn't handle attributes,
it's supposed to handle tag names internally but has
no information to do so, etc).
We're removing it now and will bring it back with an
interface that covers these aspects, after Go 1.
Related to issue 2771, but doesn't fix it.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5574057
Includes gofix module. The only case not covered should be
xml.Unmarshal, since it remains with a similar interface, and
would require introspecting the type of its first argument
better.
Fixes#2626.
R=golang-dev, rsc, gustavo
CC=golang-dev
https://golang.org/cl/5574053
Marshalling of []byte in attributes and the general
marshalling of named []byte types was fixed.
A []byte field also won't be nil if an XML element
was mapped to it, even if the element is empty.
Tests were introduced to make sure that *struct{}
fields works correctly for element presence testing.
No changes to the logic made in that regard.
R=rsc
CC=golang-dev
https://golang.org/cl/5539070
It was 2^31, but that could cause overflow and trouble.
Reduce it to 2^30 and add a TODO.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/5562049
This CL improves the xml package in the following ways:
- makes its interface match established conventions
- brings Marshal and Unmarshal closer together
- fixes a large number of bugs and adds tests
- improves speed significantly
- organizes and simplifies the code
Fixes#2426.
Fixes#2406.
Fixes#1989.
What follows is a detailed list of those changes.
- All matching is case sensitive without special processing
to the field name or xml tag in an attempt to match them.
Customize the field tag as desired to match the correct XML
elements.
- Flags are ",flag" rather than "flag". The names "attr",
"chardata", etc, may be used to name actual XML elements.
- Overriding of attribute names is possible with "name,attr".
- Attribute fields are marshalled properly if they have
non-string types. Previously they were unmarshalled, but were
ignored at marshalling time.
- Comment fields tagged with ",comment" are marshalled properly,
rather than being marshalled as normal fields.
- The handling of the Any field has been replaced by the ",any"
flag to avoid unexpected results when using the field name for
other purposes, and has also been fixed to interact properly
with name paths. Previously the feature would not function
if any field in the type had a name path in its tag.
- Embedded struct support fixed and cleaned so it works when
marshalling and also when using field paths deeper than one level.
- Conflict reporting on field names have been expanded to cover
all fields. Previously it'd catch only conflicts of paths
deeper than one level. Also interacts correctly with embedded
structs now.
- A trailing '>' is disallowed in xml tags. It used to be
supported for removing the ambiguity between "attr" and "attr>",
but the marshalling support for that was broken, and it's now
unnecessary. Use "name" instead of "name>".
- Fixed docs to point out that a XMLName doesn't have to be
an xml.Name (e.g. a struct{} is a good fit too). The code was
already working like that.
- Fixed asymmetry in the precedence of XML element names between
marshalling and unmarshalling. Marshal would consider the XMLName
of the field type before the field tag, while unmarshalling would
do the opposite. Now both respect the tag of the XMLName field
first, and a nice error message is provided in case an attempt
is made to name a field with its tag in a way that would
conflict with the underlying type's XMLName field.
- Do not marshal broken "<???>" tags when in doubt. Use the type
name, and error out if that's not possible.
- Do not break down unmarshalling if there's an interface{} field
in a struct.
- Significant speed boost due to caching of type metadata and
overall allocation clean ups. The following timings reflect
processing of the the atom test data:
Old:
BenchmarkMarshal 50000 48798 ns/op
BenchmarkUnmarshal 5000 357174 ns/op
New:
BenchmarkMarshal 100000 19799 ns/op
BenchmarkUnmarshal 10000 128525 ns/op
R=cw, gustavo, kevlar, adg, rogpeppe, fullung, christoph, rsc
CC=golang-dev
https://golang.org/cl/5503078
