1
0
mirror of https://github.com/golang/go synced 2024-10-04 08:31:22 -06:00
Commit Graph

107 Commits

Author SHA1 Message Date
Dave Cheney
d5b299eda2 compress/flate: add Encoder/Decoder benchmarks
In CL 6127051, nigeltao suggested that further gains
were possible by improving the performance of flate.
This CL adds a set of benchmarks (based on compress/lzw)
that can be used to judge any future improvements.

R=nigeltao
CC=golang-dev
https://golang.org/cl/6128049
2012-04-29 20:41:13 +10:00
Quan Yong Zhai
dd963ddf05 compress/zlib: minor fixes
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/6058043
2012-04-21 14:34:42 +10:00
Russ Cox
0e70f2722b all: shorten some of the longer tests
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5675092
2012-02-18 16:24:23 -05:00
Nigel Tao
22636be8b0 flate: delete WrongValueError type.
Fixes #2838.

R=rsc, r
CC=golang-dev
https://golang.org/cl/5651060
2012-02-11 12:09:11 +11:00
Nigel Tao
18f518362b compress: add comments to gzip and zlib.
Fixes #2939.

R=rsc, r
CC=golang-dev
https://golang.org/cl/5655050
2012-02-11 09:42:07 +11:00
Nigel Tao
cc9ed447d0 compress: make flate, gzip and zlib's NewWriterXxx functions all return
(*Writer, error) if they take a compression level, and *Writer otherwise.
Rename gzip's Compressor and Decompressor to Writer and Reader, similar to
flate and zlib.

Clarify commentary when writing gzip metadata that is not representable
as Latin-1, and fix io.EOF comment bug.

Also refactor gzip_test to be more straightforward.

Fixes #2839.

R=rsc, r, rsc, bradfitz
CC=golang-dev
https://golang.org/cl/5639057
2012-02-10 18:49:19 +11:00
Alex Brainman
309863aec4 compress/gzip: remove dead code
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/5645074
2012-02-10 09:33:51 +11:00
Rob Pike
5be24046c7 all: avoid bytes.NewBuffer(nil)
The practice encourages people to think this is the way to
create a bytes.Buffer when new(bytes.Buffer) or
just var buf bytes.Buffer work fine.
(html/token.go was missing the point altogether.)

R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5637043
2012-02-06 14:09:00 +11:00
Russ Cox
2050a9e478 build: remove Make.pkg, Make.tool
Consequently, remove many package Makefiles,
and shorten the few that remain.

gomake becomes 'go tool make'.

Turn off test phases of run.bash that do not work,
flagged with $BROKEN.  Future CLs will restore these,
but this seemed like a big enough CL already.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
2012-01-30 23:43:46 -05:00
Russ Cox
cb34f5c357 compress/flate: undo misuse of skipNever
I assume this was a too aggressive search-and-replace.

R=imkrasin
CC=golang-dev
https://golang.org/cl/5580047
2012-01-30 13:24:20 -05:00
Ivan Krasin
903752f484 compress/flate: remove unused huffmanEncoder.generateChains.
R=rsc, gri
CC=golang-dev
https://golang.org/cl/5577061
2012-01-27 09:52:58 -08:00
Brad Fitzpatrick
37d2f8190d rename FooError vars to ErrFoo
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5574056
2012-01-24 11:48:48 -08:00
Ivan Krasin
565e140a16 compress/flate: increase the length of hash table from 1<<15 to 1<<17. 0%-16% speedup.
R=rsc, imkrasin
CC=golang-dev
https://golang.org/cl/5569048
2012-01-24 13:52:45 -05:00
Ivan Krasin
858f0b4d95 compress/flate: delete unused util functions.
R=rsc
CC=golang-dev
https://golang.org/cl/5555071
2012-01-23 10:31:51 -05:00
Ivan Krasin
b35cef6704 compress/flate: use append instead of slice+counter.
R=rsc, nigeltao
CC=golang-dev
https://golang.org/cl/5561056
2012-01-23 09:26:14 -05:00
Ivan Krasin
d7e34051fc compress/flate: reduce memory pressure at cost of additional arithmetic operation.
R=rsc
CC=golang-dev
https://golang.org/cl/5555070
2012-01-23 09:19:39 -05:00
Ivan Krasin
4d3b9d9757 compress/flate: fix a typo, improve compression rate by 3-4%.
R=rsc
CC=golang-dev
https://golang.org/cl/5556077
2012-01-21 12:18:15 -05:00
Ivan Krasin
c4b16a3864 compress/flate: make lazy matching work.
R=rsc, imkrasin
CC=golang-dev
https://golang.org/cl/5554066
2012-01-20 23:35:18 -05:00
Vadim Vygonets
8fbeb945db gzip: Convert between Latin-1 and Unicode
I realize I didn't send the tests in last time.  Anyway, I added
a test that knows too much about the package's internal structure,
and I'm not sure whether it's the right thing to do.

Vadik.

R=bradfitz, rsc, go.peter.90
CC=golang-dev
https://golang.org/cl/5450073
2011-12-14 17:17:40 -05:00
Ivan Krasin
b1ae728d19 compress/flate: fix out of bounds error
Fixes #2508.

R=rsc, krasin
CC=golang-dev
https://golang.org/cl/5449115
2011-12-12 18:25:32 -05:00
Rob Pike
0a5508c692 various: we don't cast, we convert
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/5437142
2011-12-05 19:40:52 -08:00
Russ Cox
2666b815a3 use new strconv API
All but 3 cases (in gcimporter.go and hixie.go)
are automatic conversions using gofix.

No attempt is made to use the new Append functions
even though there are definitely opportunities.

R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5447069
2011-12-05 15:48:46 -05:00
Russ Cox
dcf1d7bc0e gofmt -s misc src
R=golang-dev, bradfitz, gri
CC=golang-dev
https://golang.org/cl/5451079
2011-12-02 14:14:25 -05:00
Russ Cox
03823b881c use new time API
R=bradfitz, gri, r, dsymonds
CC=golang-dev
https://golang.org/cl/5390042
2011-11-30 12:01:46 -05:00
Russ Cox
558e7fc332 various: avoid func compare
R=gri, r, bradfitz
CC=golang-dev
https://golang.org/cl/5371074
2011-11-13 22:57:19 -05:00
Russ Cox
fd34e78b53 various: reduce overuse of os.EINVAL + others
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5372081
2011-11-13 22:42:42 -05:00
Gustavo Niemeyer
f2dc50b48d html,bzip2,sql: rename Error methods that return error to Err
There are three classes of methods/functions called Error:

a) The Error method in the just introduced error interface
b) Error methods that create or report errors (http.Error, etc)
c) Error methods that return errors previously associated with
   the receiver (Tokenizer.Error, rows.Error, etc).

This CL introduces the convention that methods in case (c)
should be named Err.

The reasoning for the change is:

- The change differentiates the two kinds of APIs based on
  names rather than just on signature, unloading Error a bit
- Err is closer to the err variable name that is so commonly
  used with the intent of verifying an error
- Err is shorter and thus more convenient to be used often
  on error verifications, such as in iterators following the
  convention of the sql package.

R=bradfitz, rsc
CC=golang-dev
https://golang.org/cl/5327064
2011-11-04 09:50:20 -04:00
Russ Cox
492098eb75 all: rename os.Error to error in various non-code contexts
R=adg
CC=golang-dev
https://golang.org/cl/5328062
2011-11-01 22:58:09 -04:00
Russ Cox
c2049d2dfe src/pkg/[a-m]*: gofix -r error -force=error
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/5322051
2011-11-01 22:04:37 -04:00
Robert Griesemer
90564a9256 go/printer: changed max. number of newlines from 3 to 2
manual changes in src/pkg/go/printer, src/cmd/gofix/signal_test.go
(cd src/cmd/gofix/testdata; gofmt -w *.in *.out)
(cd src/pkg/go/printer; gotest -update)
gofmt -w misc src

runs all tests

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/4715041
2011-07-14 14:39:40 -07:00
Rob Pike
ebb1566a46 strings.Split: make the default to split all.
Change the signature of Split to have no count,
assuming a full split, and rename the existing
Split with a count to SplitN.
Do the same to package bytes.
Add a gofix module.

R=adg, dsymonds, alex.brainman, rsc
CC=golang-dev
https://golang.org/cl/4661051
2011-06-28 09:43:14 +10:00
Robert Griesemer
712fb6dcd3 os.Error API: don't export os.ErrorString, use os.NewError consistently
This is a core API change.

1) gofix misc src
2) Manual adjustments to the following files under src/pkg:
   gob/decode.go
   rpc/client.go
   os/error.go
   io/io.go
   bufio/bufio.go
   http/request.go
   websocket/client.go
as well as:
   src/cmd/gofix/testdata/*.go.in (reverted)
   test/fixedbugs/bug243.go
3) Implemented gofix patch (oserrorstring.go) and test case (oserrorstring_test.go)

Compiles and runs all tests.

R=r, rsc, gri
CC=golang-dev
https://golang.org/cl/4607052
2011-06-22 10:52:47 -07:00
Nigel Tao
833529fd6f compress/lzw: reduce decoder buffer size from 3*4096 to 2*4096.
This happens to speed up the decoder benchmarks by 50% on my computer
(GOARCH=amd64 GOOS=linux), but I don't have a good intuition as to why.
For example, just adding an unused [4096]byte field to the decoder
struct doesn't significantly change the numbers.

Before:

lzw.BenchmarkDecoder1e4	    5000	    488057 ns/op	  20.49 MB/s
lzw.BenchmarkDecoder1e5	     500	   4613638 ns/op	  21.67 MB/s
lzw.BenchmarkDecoder1e6	      50	  45672260 ns/op	  21.90 MB/s
lzw.BenchmarkEncoder1e4	    5000	    353563 ns/op	  28.28 MB/s
lzw.BenchmarkEncoder1e5	     500	   3431618 ns/op	  29.14 MB/s
lzw.BenchmarkEncoder1e6	      50	  34009640 ns/op	  29.40 MB/s

After:

lzw.BenchmarkDecoder1e4	    5000	    339725 ns/op	  29.44 MB/s
lzw.BenchmarkDecoder1e5	     500	   3166894 ns/op	  31.58 MB/s
lzw.BenchmarkDecoder1e6	      50	  31317260 ns/op	  31.93 MB/s
lzw.BenchmarkEncoder1e4	    5000	    354909 ns/op	  28.18 MB/s
lzw.BenchmarkEncoder1e5	     500	   3432710 ns/op	  29.13 MB/s
lzw.BenchmarkEncoder1e6	      50	  34010500 ns/op	  29.40 MB/s

R=rsc, r
CC=golang-dev
https://golang.org/cl/4535123
2011-06-09 09:50:38 +10:00
Russ Cox
63dae3c3be compress/lzw: do not use background goroutines
Programs expect that Read and Write are synchronous.
The background goroutines make the implementation
a little easier, but they introduce asynchrony that
trips up calling code.  Remove them.

R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/4548080
2011-06-07 14:37:06 -04:00
Russ Cox
07acc02a29 compress/flate: do not use background goroutines
Programs expect that Read and Write are synchronous.
The background goroutines make the implementation
a little easier, but they introduce asynchrony that
trips up calling code.  Remove them.

R=golang-dev, krasin
CC=golang-dev
https://golang.org/cl/4548079
2011-06-02 09:32:38 -04:00
Ivan Krasin
1b5d04c5ae compress/flate: fix Huffman tree bug
Incorporate refactoring and a regression test from https://golang.org/cl/4538090/

R=rsc, go.peter.90, imkrasin
CC=golang-dev, mirtchovski
https://golang.org/cl/4524070
2011-05-26 17:02:11 -04:00
Robert Hencke
c8727c81bb pkg: spelling tweaks, A-H
R=ality, bradfitz, rsc, dsymonds, adg, qyzhai, dchest
CC=golang-dev
https://golang.org/cl/4536063
2011-05-18 13:14:56 -04:00
Adam Langley
d080a1cf14 compress/zlib: actually use provided dictionary.
R=rsc, bradfitz, bradfitzgoog
CC=golang-dev
https://golang.org/cl/4518056
2011-05-11 17:00:19 -04:00
Nigel Tao
f467803dcd compress/lzw: silently drop implied codes that are too large,
instead of returning an error.

For example, http://www.w3.org/Graphics/GIF/spec-gif89a.txt
explicitly says that GIF encoders can use a full table as is,
without needing to send a clear code.

R=r, dsymonds, nigeltao_gnome, r2
CC=golang-dev
https://golang.org/cl/4518041
2011-05-07 18:57:32 -07:00
Brad Fitzpatrick
437015bbed png: speed up opaque RGBA encoding
With Linux/8g on a 2006 Mac Mini (1.66 GHz Intel Core Duo,
2KB L1, 2MB L2, 2G main memory), GOMAXPROCS unset:

start:
png.BenchmarkEncodePaletted	      50	  44772820 ns/op
png.BenchmarkEncodeRGBOpaque	      10	 208395900 ns/op
png.BenchmarkEncodeRGBA		       5	 331088000 ns/op

remove interface method calls:
png.BenchmarkEncodePaletted	      50	  44722880 ns/op
png.BenchmarkEncodeRGBOpaque	      10	 139042600 ns/op
png.BenchmarkEncodeRGBA		       5	 334033600 ns/op

flate inline min/max():
png.BenchmarkEncodePaletted	      50	  40631180 ns/op
png.BenchmarkEncodeRGBOpaque	      10	 124894900 ns/op
png.BenchmarkEncodeRGBA		       5	 312099000 ns/op

after adler change:
png.BenchmarkEncodePaletted	      50	  40181760 ns/op
png.BenchmarkEncodeRGBOpaque	      20	 121781950 ns/op
png.BenchmarkEncodeRGBA		       5	 313890800 ns/op

In comparison to 121 ms on this 2006 machine, on my
Core2 Duo 2.66 GHz laptop, the final BenchmarkEncodeRGBOpaque
runs in 27 ms. (these are all for 640x480 images)

R=nigeltao, rsc, r
CC=golang-dev
https://golang.org/cl/4432077
2011-05-02 07:25:53 -07:00
Brad Fitzpatrick
9d12307a12 ioutil: add Discard, update tree.
This also removes an unnecessary allocation in
http/transfer.go

R=r, rsc1, r2, adg
CC=golang-dev
https://golang.org/cl/4426066
2011-04-27 15:47:04 -07:00
Nigel Tao
6a186d38d1 src/pkg: make package doc comments consistently start with "Package foo".
R=rsc
CC=golang-dev
https://golang.org/cl/4442064
2011-04-20 09:57:05 +10:00
Ross Light
bfb486875a compress/zlib: add FDICT flag in Reader/Writer
R=bradfitzgo, rsc, bradfitzwork
CC=golang-dev
https://golang.org/cl/4406046
2011-04-15 15:32:03 -07:00
Russ Cox
36713a2a53 compress/flate: dictionary support
R=bradfitzwork
CC=golang-dev
https://golang.org/cl/4397043
2011-04-14 10:41:55 -04:00
Rob Pike
8a90fd3c72 os: New Open API.
We replace the current Open with:
OpenFile(name, flag, perm) // same as old Open
Open(name) // same as old Open(name, O_RDONLY, 0)
Create(name) // same as old Open(name, O_RDWR|O_TRUNC|O_CREAT, 0666)

This CL includes a gofix module and full code updates: all.bash passes.
(There may be a few comments I missed.)

The interesting packages are:
        gofix
        os
Everything else is automatically generated except for hand tweaks to:
        src/pkg/io/ioutil/ioutil.go
        src/pkg/io/ioutil/tempfile.go
        src/pkg/crypto/tls/generate_cert.go
        src/cmd/goyacc/goyacc.go
        src/cmd/goyacc/units.y

R=golang-dev, bradfitzwork, rsc, r2
CC=golang-dev
https://golang.org/cl/4357052
2011-04-04 23:42:14 -07:00
Nigel Tao
78879b3b8b compress/lzw: benchmark a range of input sizes.
R=rsc, nigeltao_gnome
CC=golang-dev
https://golang.org/cl/4240096
2011-03-09 16:32:02 +11:00
Russ Cox
d044674a0d compress/flate: fix test
The test was checking for a buffer to be empty but
actually racing with the background goroutine that
was emptying it.  Left a comment so that the check
is not reintroduced later.

Fixes #1557.

R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/4248063
2011-03-05 14:35:03 -05:00
Adam Langley
e7fa307902 bzip2: speed up decompression.
This borrows a trick from the bzip2 source and effects a decent speed
up when decompressing highly compressed sources. Rather than unshuffle
the BTW block when performing the IBTW, a linked-list is threaded
through the array, in place. This improves cache hit rates.

R=bradfitzgo, bradfitzwork, cw
CC=golang-dev
https://golang.org/cl/4247047
2011-03-03 09:13:06 -05:00
Nigel Tao
fdbbb066ed compress/lzw: don't use a closure in NewReader, which avoids having
to move some variables from the stack to the heap.

Sorted benchmark runs on my 2007-era Mac Mini (GOARCH=amd64, GOOS=linux):

Before:
lzw.BenchmarkDecoder        2000        878176 ns/op
lzw.BenchmarkDecoder        2000        878415 ns/op
lzw.BenchmarkDecoder        2000        880352 ns/op
lzw.BenchmarkDecoder        2000        898445 ns/op
lzw.BenchmarkDecoder        2000        901728 ns/op

After:
lzw.BenchmarkDecoder        2000        859065 ns/op
lzw.BenchmarkDecoder        2000        859402 ns/op
lzw.BenchmarkDecoder        2000        860035 ns/op
lzw.BenchmarkDecoder        2000        860555 ns/op
lzw.BenchmarkDecoder        2000        861109 ns/op

The ratio of before/after median times is 1.024.

The runtime.MemStats.Mallocs delta per loop drops from 109 to 104.

R=r, r2, dfc
CC=golang-dev
https://golang.org/cl/4253043
2011-02-26 16:42:49 +11:00
Nigel Tao
5b1d47d105 compress/lzw: fix the stupidity of allocating and zeroing a new buffer
on each loop iteration, yielding a 20x performance improvement.

R=rsc, r2
CC=golang-dev
https://golang.org/cl/4240044
2011-02-26 09:25:29 +11:00