The GIF89a spec says: "Each image must fit within the
boundaries of the Logical Screen, as defined in the
Logical Screen Descriptor." Also, do not accept
GIFs which have too much data for the image size.
R=nigeltao, jra, r
CC=bradfitz, golang-dev
https://golang.org/cl/7602045
Fixes#4705.
Note that libjpeg will print a warning to stderr if there are many
extraneous bytes, but can be silent if the extraneous bytes can fit
into its int32 bit-buffer for Huffman decoding. I'm guessing that
this is why whatever encoder that produced the image filed for issue
4705 did not realize that they are, strictly speaking, generating an
invalid JPEG. That issue's attached image has two extraneous bytes.
For example, piping the program below into libjpeg's djpeg program
will print an "18 extraneous bytes" warning, even though N == 20.
$ cat main.go
package main
import (
"bytes"
"image"
"image/color"
"image/jpeg"
"os"
)
const N = 20
func main() {
// Encode a 1x1 red image.
m := image.NewRGBA(image.Rect(0, 0, 1, 1))
m.Set(0, 0, color.RGBA{255, 0, 0, 255})
buf := new(bytes.Buffer)
jpeg.Encode(buf, m, nil)
b := buf.Bytes()
// Strip the final "\xff\xd9" EOI marker.
b = b[:len(b)-2]
// Append N dummy 0x80 bytes to the SOS data.
for i := 0; i < N; i++ {
b = append(b, 0x80)
}
// Put back the "\xff\xd9" EOI marker.
b = append(b, 0xff, 0xd9)
os.Stdout.Write(b)
}
$ go run main.go | djpeg /dev/stdin > /tmp/foo.pnm
Corrupt JPEG data: 18 extraneous bytes before marker 0xd9
The resultant /tmp/foo.pnm is a perfectly good 1x1 red image.
R=r
CC=golang-dev
https://golang.org/cl/7750043
The old code would decode the palette only for 8-bit
images during a DecodeConfig.
This CL keeps the behavior for 8-bit images and sets
up the decoded palette also for 1, 2 and 4-bit images.
Fixes#4279.
R=golang-dev, nigeltao
CC=golang-dev
https://golang.org/cl/7421048
ratio isn't 1x1.
Fixes#4259.
The test data was generated by
cjpeg -quality 50 -sample 2x2 video-005.gray.pgm > video-005.gray.q50.2x2.jpeg
cjpeg -quality 50 -sample 2x2 -progressive video-005.gray.pgm > video-005.gray.q50.2x2.progressive.jpeg
similarly to video-005.gray.q50.* from
http://code.google.com/p/go/source/detail?r=51f26e36ba98
the key difference being the "-sample 2x2".
R=rsc
CC=golang-dev
https://golang.org/cl/7069045
defined by the PLTE chunk. Such pixels decode to opaque black,
which matches what libpng does.
Fixes#4319.
On my reading, the PNG spec isn't clear whether palette index values
outside of those defined by the PLTE chunk is an error, and if not,
what to do.
Libpng 1.5.3 falls back to opaque black. png_set_PLTE says:
/* Changed in libpng-1.2.1 to allocate PNG_MAX_PALETTE_LENGTH instead
* of num_palette entries, in case of an invalid PNG file that has
* too-large sample values.
*/
png_ptr->palette = (png_colorp)png_calloc(png_ptr,
PNG_MAX_PALETTE_LENGTH * png_sizeof(png_color));
ImageMagick 6.5.7 returns an error:
$ convert -version
Version: ImageMagick 6.5.7-8 2012-08-17 Q16 http://www.imagemagick.org
Copyright: Copyright (C) 1999-2009 ImageMagick Studio LLC
Features: OpenMP
$ convert packetloss.png x.bmp
convert: Invalid colormap index `packetloss.png' @ image.c/SyncImage/3849.
R=r
CC=golang-dev
https://golang.org/cl/6822065
On 6g/linux:
benchmark old ns/op new ns/op delta
BenchmarkFDCT 4606 4241 -7.92%
BenchmarkIDCT 4187 3923 -6.31%
BenchmarkDecodeBaseline 3154864 3170224 +0.49%
BenchmarkDecodeProgressive 4072812 4017132 -1.37%
BenchmarkEncode 39406920 34596760 -12.21%
Stack requirements before (from 'go tool 6g -S'):
(scan.go:37) TEXT (*decoder).processSOS+0(SB),$1352-32
(writer.go:448) TEXT (*encoder).writeSOS+0(SB),$5344-24
after:
(scan.go:37) TEXT (*decoder).processSOS+0(SB),$1064-32
(writer.go:448) TEXT (*encoder).writeSOS+0(SB),$2520-24
Also, in encoder.writeSOS, re-use the yBlock scratch buffer for Cb and
Cr. This reduces the stack requirement slightly, but also avoids an
unlucky coincidence where a BenchmarkEncode stack split lands between
encoder.writeByte and bufio.Writer.WriteByte, which occurs very often
during Huffman encoding and is otherwise disasterous for the final
benchmark number. FWIW, the yBlock re-use *without* the s/int/int32/
change does not have a noticable effect on the benchmarks.
R=r
CC=golang-dev, rsc
https://golang.org/cl/6823043
The value of cosines are cached in a global array
instead of being recomputed each time.
The test was terribly slow on arm.
R=golang-dev, dave, nigeltao
CC=golang-dev
https://golang.org/cl/6733046
To be clear, this supports decoding the bytes on the wire into an
in-memory image. There is no API change: jpeg.Decode will still not
return until the entire image is decoded.
The code is obviously more complicated, and costs around 10% in
performance on baseline JPEGs. The processSOS code could be cleaned up a
bit, and maybe some of that loss can be reclaimed, but I'll leave that
for follow-up CLs, to keep the diff for this one as small as possible.
Before:
BenchmarkDecode 1000 2855637 ns/op 21.64 MB/s
After:
BenchmarkDecodeBaseline 500 3178960 ns/op 19.44 MB/s
BenchmarkDecodeProgressive 500 4082640 ns/op 15.14 MB/s
Fixes#3976.
The test data was generated by:
# Create intermediate files; cjpeg on Ubuntu 10.04 can't read PNG.
convert video-001.png video-001.bmp
convert video-005.gray.png video-005.gray.pgm
# Create new test files.
cjpeg -quality 100 -sample 1x1,1x1,1x1 -progressive video-001.bmp > video-001.progressive.jpeg
cjpeg -quality 50 -sample 2x2,1x1,1x1 video-001.bmp > video-001.q50.420.jpeg
cjpeg -quality 50 -sample 2x1,1x1,1x1 video-001.bmp > video-001.q50.422.jpeg
cjpeg -quality 50 -sample 1x1,1x1,1x1 video-001.bmp > video-001.q50.444.jpeg
cjpeg -quality 50 -sample 2x2,1x1,1x1 -progressive video-001.bmp > video-001.q50.420.progressive.jpeg
cjpeg -quality 50 -sample 2x1,1x1,1x1 -progressive video-001.bmp > video-001.q50.422.progressive.jpeg
cjpeg -quality 50 -sample 1x1,1x1,1x1 -progressive video-001.bmp > video-001.q50.444.progressive.jpeg
cjpeg -quality 50 video-005.gray.pgm > video-005.gray.q50.jpeg
cjpeg -quality 50 -progressive video-005.gray.pgm > video-005.gray.q50.progressive.jpeg
# Delete intermediate files.
rm video-001.bmp video-005.gray.pgm
R=r
CC=golang-dev
https://golang.org/cl/6684046
decoder struct, inside the unmappedzero limit, to eliminate some
TESTB instructions in the inner decoding loop.
benchmark old ns/op new ns/op delta
BenchmarkDecode 2943204 2746360 -6.69%
R=r, dave
CC=golang-dev
https://golang.org/cl/6625058
to be consistent with the fdct function, and to ease any future
idct rewrites in assembly.
The BenchmarkIDCT delta is obviously just an accounting change and not
a real saving, but it does give an indication of what proportion of
time was spent in the actual IDCT and what proportion was in shift and
clip. The idct time taken is now comparable to fdct.
The BenchmarkFDCT delta is an estimate of benchmark noise.
benchmark old ns/op new ns/op delta
BenchmarkFDCT 3842 3837 -0.13%
BenchmarkIDCT 5611 3478 -38.01%
BenchmarkDecodeRGBOpaque 2932785 2929751 -0.10%
R=r
CC=golang-dev
https://golang.org/cl/6625057
refer to opacity. Those references were copy/pasted from the
image/png encoding benchmarks, which cares whether or not the
source image is opaque, but the JPEG encoder does not care.
R=r
CC=golang-dev
https://golang.org/cl/6623052
elimination) in idct.go.
benchmark old ns/op new ns/op delta
BenchmarkIDCT 5649 5610 -0.69%
BenchmarkDecodeRGBOpaque 2948607 2941051 -0.26%
The "type block" declaration moved so that idct.go is compilable
as a stand-alone file: "go tool 6g -S idct.go" works.
R=r
CC=golang-dev
https://golang.org/cl/6619056
order.
JPEG images generated prior to this CL are still valid JPEGs, as the
quantization tables used are encoded in the wire format. Such JPEGs just
don't use the recommended quantization tables.
R=r, dsymonds, raph, adg
CC=golang-dev, tuom.larsen
https://golang.org/cl/6497083
filterPaeth takes []byte arguments instead of byte arguments,
which avoids some redudant computation of the previous pixel
in the inner loop.
Also eliminate a bounds check in decoding the up filter.
benchmark old ns/op new ns/op delta
BenchmarkDecodeGray 3139636 2812531 -10.42%
BenchmarkDecodeNRGBAGradient 12341520 10971680 -11.10%
BenchmarkDecodeNRGBAOpaque 10740780 9612455 -10.51%
BenchmarkDecodePaletted 1819535 1818913 -0.03%
BenchmarkDecodeRGB 8974695 8178070 -8.88%
R=rsc
CC=golang-dev
https://golang.org/cl/6243061
(*Writer, error) if they take a compression level, and *Writer otherwise.
Rename gzip's Compressor and Decompressor to Writer and Reader, similar to
flate and zlib.
Clarify commentary when writing gzip metadata that is not representable
as Latin-1, and fix io.EOF comment bug.
Also refactor gzip_test to be more straightforward.
Fixes#2839.
R=rsc, r, rsc, bradfitz
CC=golang-dev
https://golang.org/cl/5639057
It complicates the interface unnecessarily.
Document this in go1.html.
Also update the go/doc Makefile.
Fixes#2836.
R=golang-dev, gri, bradfitz
CC=golang-dev
https://golang.org/cl/5642054
The practice encourages people to think this is the way to
create a bytes.Buffer when new(bytes.Buffer) or
just var buf bytes.Buffer work fine.
(html/token.go was missing the point altogether.)
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5637043
Consequently, remove many package Makefiles,
and shorten the few that remain.
gomake becomes 'go tool make'.
Turn off test phases of run.bash that do not work,
flagged with $BROKEN. Future CLs will restore these,
but this seemed like a big enough CL already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
instead of the origin.
This makes YCbCr match the other image types (e.g. RGBA, Gray) in
that an image's bounds is not restricted to the positive quadrant.
Also optimize the YCbCr draw code by hoisting some computation
outside of the loop.
benchmark old ns/op new ns/op delta
draw.BenchmarkYCbCr 2544418 2373558 -6.72%
Like https://golang.org/cl/4681044/ I don't think a gofix is
feasible. People will have to make manual changes. On the other hand,
directly manipulating YCbCr images is relatively rare, compared to
RGBA images, and if other code just uses the jpeg and draw packages
instead of messing directly with a YCbCr's []byte representations,
then things should just continue to work.
R=r
CC=golang-dev
https://golang.org/cl/5558048
This matches the other color models. It seems that this was missed
during the review of 5544073.
R=nigeltao, david.crawshaw
CC=golang-dev
https://golang.org/cl/5536057
