This bug has been introduced in the following revision:
changeset: 11404:26dceba5c610
user: Ivan Krasin <krasin@golang.org>
date: Mon Jan 23 09:19:39 2012 -0500
summary: compress/flate: reduce memory pressure at cost of additional arithmetic operation.
This is the review page for that CL: https://golang.org/cl/5555070/
R=rsc, imkrasin
CC=golang-dev
https://golang.org/cl/6249067
These files change from exactly 10003 bytes long to 100003: a digit,
a '.', 100k digits, and a '\n'.
The magic constants in compress/flate/deflate_test.go change since
deflateInflateStringTests checks that the compressed form of e.txt
is not 'too large'. I'm not exactly sure how these numbers were
originally calculated (they were introduced in codereview 5554066
"make lazy matching work"); perhaps krasin@golang.org can comment.
My change was to increase the first one (no compression) to a tight
bound, and multiply all the others by 10.
Benchcmp numbers for compress/flate and compress/lzw below. LZW's
window size of 4096 is less than 10k, so shows no significant change.
Flate's window size is 32768, between 10k and 100k, and so the .*1e5
and .*1e6 benchmarks show a dramatic drop, since the compressed forms
are no longer a trivial forward copy of 10k digits repeated over and
over, but should now be more representative of real world usage.
compress/flate:
benchmark old MB/s new MB/s speedup
BenchmarkDecodeDigitsSpeed1e4 16.58 16.52 1.00x
BenchmarkDecodeDigitsSpeed1e5 68.09 18.10 0.27x
BenchmarkDecodeDigitsSpeed1e6 124.63 18.35 0.15x
BenchmarkDecodeDigitsDefault1e4 17.21 17.12 0.99x
BenchmarkDecodeDigitsDefault1e5 118.28 19.19 0.16x
BenchmarkDecodeDigitsDefault1e6 295.62 20.52 0.07x
BenchmarkDecodeDigitsCompress1e4 17.22 17.17 1.00x
BenchmarkDecodeDigitsCompress1e5 118.19 19.21 0.16x
BenchmarkDecodeDigitsCompress1e6 295.59 20.55 0.07x
BenchmarkEncodeDigitsSpeed1e4 8.18 8.19 1.00x
BenchmarkEncodeDigitsSpeed1e5 43.22 12.84 0.30x
BenchmarkEncodeDigitsSpeed1e6 80.76 13.48 0.17x
BenchmarkEncodeDigitsDefault1e4 6.29 6.19 0.98x
BenchmarkEncodeDigitsDefault1e5 31.63 3.60 0.11x
BenchmarkEncodeDigitsDefault1e6 52.97 3.24 0.06x
BenchmarkEncodeDigitsCompress1e4 6.20 6.19 1.00x
BenchmarkEncodeDigitsCompress1e5 31.59 3.59 0.11x
BenchmarkEncodeDigitsCompress1e6 53.18 3.25 0.06x
compress/lzw:
benchmark old MB/s new MB/s speedup
BenchmarkDecoder1e4 21.99 22.09 1.00x
BenchmarkDecoder1e5 22.77 22.71 1.00x
BenchmarkDecoder1e6 22.90 22.90 1.00x
BenchmarkEncoder1e4 21.04 21.19 1.01x
BenchmarkEncoder1e5 22.06 22.06 1.00x
BenchmarkEncoder1e6 22.16 22.28 1.01x
R=rsc
CC=golang-dev, krasin
https://golang.org/cl/6207043
I'm not sure where the BOM came from, originally.
http://www.gutenberg.org/files/74/74.txt doesn't have it, although
a fresh download of that URL gives me "\r\n"s instead of plain "\n"s,
and the extra line "Character set encoding: ASCII". Maybe Project
Gutenberg has changed their server configuration since we added that
file to the Go repo.
Anyway, this change is just manually excising the BOM from the start
of the file, leaving pure ASCII.
R=r, bradfitz
CC=golang-dev, krasin, rsc
https://golang.org/cl/6197061
In CL 6127051, nigeltao suggested that further gains
were possible by improving the performance of flate.
This CL adds a set of benchmarks (based on compress/lzw)
that can be used to judge any future improvements.
R=nigeltao
CC=golang-dev
https://golang.org/cl/6128049
(*Writer, error) if they take a compression level, and *Writer otherwise.
Rename gzip's Compressor and Decompressor to Writer and Reader, similar to
flate and zlib.
Clarify commentary when writing gzip metadata that is not representable
as Latin-1, and fix io.EOF comment bug.
Also refactor gzip_test to be more straightforward.
Fixes#2839.
R=rsc, r, rsc, bradfitz
CC=golang-dev
https://golang.org/cl/5639057
The practice encourages people to think this is the way to
create a bytes.Buffer when new(bytes.Buffer) or
just var buf bytes.Buffer work fine.
(html/token.go was missing the point altogether.)
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5637043
Consequently, remove many package Makefiles,
and shorten the few that remain.
gomake becomes 'go tool make'.
Turn off test phases of run.bash that do not work,
flagged with $BROKEN. Future CLs will restore these,
but this seemed like a big enough CL already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
I realize I didn't send the tests in last time. Anyway, I added
a test that knows too much about the package's internal structure,
and I'm not sure whether it's the right thing to do.
Vadik.
R=bradfitz, rsc, go.peter.90
CC=golang-dev
https://golang.org/cl/5450073
All but 3 cases (in gcimporter.go and hixie.go)
are automatic conversions using gofix.
No attempt is made to use the new Append functions
even though there are definitely opportunities.
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5447069
There are three classes of methods/functions called Error:
a) The Error method in the just introduced error interface
b) Error methods that create or report errors (http.Error, etc)
c) Error methods that return errors previously associated with
the receiver (Tokenizer.Error, rows.Error, etc).
This CL introduces the convention that methods in case (c)
should be named Err.
The reasoning for the change is:
- The change differentiates the two kinds of APIs based on
names rather than just on signature, unloading Error a bit
- Err is closer to the err variable name that is so commonly
used with the intent of verifying an error
- Err is shorter and thus more convenient to be used often
on error verifications, such as in iterators following the
convention of the sql package.
R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/5327064
Change the signature of Split to have no count,
assuming a full split, and rename the existing
Split with a count to SplitN.
Do the same to package bytes.
Add a gofix module.
R=adg, dsymonds, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4661051
This is a core API change.
1) gofix misc src
2) Manual adjustments to the following files under src/pkg:
gob/decode.go
rpc/client.go
os/error.go
io/io.go
bufio/bufio.go
http/request.go
websocket/client.go
as well as:
src/cmd/gofix/testdata/*.go.in (reverted)
test/fixedbugs/bug243.go
3) Implemented gofix patch (oserrorstring.go) and test case (oserrorstring_test.go)
Compiles and runs all tests.
R=r, rsc, gri
CC=golang-dev
https://golang.org/cl/4607052
Programs expect that Read and Write are synchronous.
The background goroutines make the implementation
a little easier, but they introduce asynchrony that
trips up calling code. Remove them.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/4548080
Programs expect that Read and Write are synchronous.
The background goroutines make the implementation
a little easier, but they introduce asynchrony that
trips up calling code. Remove them.
R=golang-dev, krasin
CC=golang-dev
https://golang.org/cl/4548079
