Fixes#4705.
Note that libjpeg will print a warning to stderr if there are many
extraneous bytes, but can be silent if the extraneous bytes can fit
into its int32 bit-buffer for Huffman decoding. I'm guessing that
this is why whatever encoder that produced the image filed for issue
4705 did not realize that they are, strictly speaking, generating an
invalid JPEG. That issue's attached image has two extraneous bytes.
For example, piping the program below into libjpeg's djpeg program
will print an "18 extraneous bytes" warning, even though N == 20.
$ cat main.go
package main
import (
"bytes"
"image"
"image/color"
"image/jpeg"
"os"
)
const N = 20
func main() {
// Encode a 1x1 red image.
m := image.NewRGBA(image.Rect(0, 0, 1, 1))
m.Set(0, 0, color.RGBA{255, 0, 0, 255})
buf := new(bytes.Buffer)
jpeg.Encode(buf, m, nil)
b := buf.Bytes()
// Strip the final "\xff\xd9" EOI marker.
b = b[:len(b)-2]
// Append N dummy 0x80 bytes to the SOS data.
for i := 0; i < N; i++ {
b = append(b, 0x80)
}
// Put back the "\xff\xd9" EOI marker.
b = append(b, 0xff, 0xd9)
os.Stdout.Write(b)
}
$ go run main.go | djpeg /dev/stdin > /tmp/foo.pnm
Corrupt JPEG data: 18 extraneous bytes before marker 0xd9
The resultant /tmp/foo.pnm is a perfectly good 1x1 red image.
R=r
CC=golang-dev
https://golang.org/cl/7750043
ratio isn't 1x1.
Fixes#4259.
The test data was generated by
cjpeg -quality 50 -sample 2x2 video-005.gray.pgm > video-005.gray.q50.2x2.jpeg
cjpeg -quality 50 -sample 2x2 -progressive video-005.gray.pgm > video-005.gray.q50.2x2.progressive.jpeg
similarly to video-005.gray.q50.* from
http://code.google.com/p/go/source/detail?r=51f26e36ba98
the key difference being the "-sample 2x2".
R=rsc
CC=golang-dev
https://golang.org/cl/7069045
On 6g/linux:
benchmark old ns/op new ns/op delta
BenchmarkFDCT 4606 4241 -7.92%
BenchmarkIDCT 4187 3923 -6.31%
BenchmarkDecodeBaseline 3154864 3170224 +0.49%
BenchmarkDecodeProgressive 4072812 4017132 -1.37%
BenchmarkEncode 39406920 34596760 -12.21%
Stack requirements before (from 'go tool 6g -S'):
(scan.go:37) TEXT (*decoder).processSOS+0(SB),$1352-32
(writer.go:448) TEXT (*encoder).writeSOS+0(SB),$5344-24
after:
(scan.go:37) TEXT (*decoder).processSOS+0(SB),$1064-32
(writer.go:448) TEXT (*encoder).writeSOS+0(SB),$2520-24
Also, in encoder.writeSOS, re-use the yBlock scratch buffer for Cb and
Cr. This reduces the stack requirement slightly, but also avoids an
unlucky coincidence where a BenchmarkEncode stack split lands between
encoder.writeByte and bufio.Writer.WriteByte, which occurs very often
during Huffman encoding and is otherwise disasterous for the final
benchmark number. FWIW, the yBlock re-use *without* the s/int/int32/
change does not have a noticable effect on the benchmarks.
R=r
CC=golang-dev, rsc
https://golang.org/cl/6823043
The value of cosines are cached in a global array
instead of being recomputed each time.
The test was terribly slow on arm.
R=golang-dev, dave, nigeltao
CC=golang-dev
https://golang.org/cl/6733046
To be clear, this supports decoding the bytes on the wire into an
in-memory image. There is no API change: jpeg.Decode will still not
return until the entire image is decoded.
The code is obviously more complicated, and costs around 10% in
performance on baseline JPEGs. The processSOS code could be cleaned up a
bit, and maybe some of that loss can be reclaimed, but I'll leave that
for follow-up CLs, to keep the diff for this one as small as possible.
Before:
BenchmarkDecode 1000 2855637 ns/op 21.64 MB/s
After:
BenchmarkDecodeBaseline 500 3178960 ns/op 19.44 MB/s
BenchmarkDecodeProgressive 500 4082640 ns/op 15.14 MB/s
Fixes#3976.
The test data was generated by:
# Create intermediate files; cjpeg on Ubuntu 10.04 can't read PNG.
convert video-001.png video-001.bmp
convert video-005.gray.png video-005.gray.pgm
# Create new test files.
cjpeg -quality 100 -sample 1x1,1x1,1x1 -progressive video-001.bmp > video-001.progressive.jpeg
cjpeg -quality 50 -sample 2x2,1x1,1x1 video-001.bmp > video-001.q50.420.jpeg
cjpeg -quality 50 -sample 2x1,1x1,1x1 video-001.bmp > video-001.q50.422.jpeg
cjpeg -quality 50 -sample 1x1,1x1,1x1 video-001.bmp > video-001.q50.444.jpeg
cjpeg -quality 50 -sample 2x2,1x1,1x1 -progressive video-001.bmp > video-001.q50.420.progressive.jpeg
cjpeg -quality 50 -sample 2x1,1x1,1x1 -progressive video-001.bmp > video-001.q50.422.progressive.jpeg
cjpeg -quality 50 -sample 1x1,1x1,1x1 -progressive video-001.bmp > video-001.q50.444.progressive.jpeg
cjpeg -quality 50 video-005.gray.pgm > video-005.gray.q50.jpeg
cjpeg -quality 50 -progressive video-005.gray.pgm > video-005.gray.q50.progressive.jpeg
# Delete intermediate files.
rm video-001.bmp video-005.gray.pgm
R=r
CC=golang-dev
https://golang.org/cl/6684046
decoder struct, inside the unmappedzero limit, to eliminate some
TESTB instructions in the inner decoding loop.
benchmark old ns/op new ns/op delta
BenchmarkDecode 2943204 2746360 -6.69%
R=r, dave
CC=golang-dev
https://golang.org/cl/6625058
to be consistent with the fdct function, and to ease any future
idct rewrites in assembly.
The BenchmarkIDCT delta is obviously just an accounting change and not
a real saving, but it does give an indication of what proportion of
time was spent in the actual IDCT and what proportion was in shift and
clip. The idct time taken is now comparable to fdct.
The BenchmarkFDCT delta is an estimate of benchmark noise.
benchmark old ns/op new ns/op delta
BenchmarkFDCT 3842 3837 -0.13%
BenchmarkIDCT 5611 3478 -38.01%
BenchmarkDecodeRGBOpaque 2932785 2929751 -0.10%
R=r
CC=golang-dev
https://golang.org/cl/6625057
refer to opacity. Those references were copy/pasted from the
image/png encoding benchmarks, which cares whether or not the
source image is opaque, but the JPEG encoder does not care.
R=r
CC=golang-dev
https://golang.org/cl/6623052
elimination) in idct.go.
benchmark old ns/op new ns/op delta
BenchmarkIDCT 5649 5610 -0.69%
BenchmarkDecodeRGBOpaque 2948607 2941051 -0.26%
The "type block" declaration moved so that idct.go is compilable
as a stand-alone file: "go tool 6g -S idct.go" works.
R=r
CC=golang-dev
https://golang.org/cl/6619056
order.
JPEG images generated prior to this CL are still valid JPEGs, as the
quantization tables used are encoded in the wire format. Such JPEGs just
don't use the recommended quantization tables.
R=r, dsymonds, raph, adg
CC=golang-dev, tuom.larsen
https://golang.org/cl/6497083
The practice encourages people to think this is the way to
create a bytes.Buffer when new(bytes.Buffer) or
just var buf bytes.Buffer work fine.
(html/token.go was missing the point altogether.)
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/5637043
Consequently, remove many package Makefiles,
and shorten the few that remain.
gomake becomes 'go tool make'.
Turn off test phases of run.bash that do not work,
flagged with $BROKEN. Future CLs will restore these,
but this seemed like a big enough CL already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
instead of the origin.
This makes YCbCr match the other image types (e.g. RGBA, Gray) in
that an image's bounds is not restricted to the positive quadrant.
Also optimize the YCbCr draw code by hoisting some computation
outside of the loop.
benchmark old ns/op new ns/op delta
draw.BenchmarkYCbCr 2544418 2373558 -6.72%
Like https://golang.org/cl/4681044/ I don't think a gofix is
feasible. People will have to make manual changes. On the other hand,
directly manipulating YCbCr images is relatively rare, compared to
RGBA images, and if other code just uses the jpeg and draw packages
instead of messing directly with a YCbCr's []byte representations,
then things should just continue to work.
R=r
CC=golang-dev
https://golang.org/cl/5558048
The spin-off renames some types. The new names are simply better:
image.Color -> color.Color
image.ColorModel -> color.Model
image.ColorModelFunc -> color.ModelFunc
image.PalettedColorModel -> color.Palette
image.RGBAColor -> color.RGBA
image.RGBAColorModel -> color.RGBAModel
image.RGBA64Color -> color.RGBA64
image.RGBA64ColorModel -> color.RGBA64Model
(similarly for NRGBAColor, GrayColorModel, etc)
The image.ColorImage type stays in the image package, but is renamed:
image.ColorImage -> image.Uniform
The image.Image implementations (image.RGBA, image.RGBA64, image.NRGBA,
image.Alpha, etc) do not change their name, and gain a nice symmetry:
an image.RGBA is an image of color.RGBA, etc.
The image.Black, image.Opaque uniform images remain unchanged (although
their type is renamed from image.ColorImage to image.Uniform). The
corresponding color types (color.Black, color.Opaque, etc) are new.
Nothing in the image/ycbcr is renamed yet. The ycbcr.YCbCrColor and
ycbcr.YCbCrImage types will eventually migrate to color.YCbCr and
image.YCbCr, but that will be a separate CL.
R=r, bsiegert
CC=golang-dev
https://golang.org/cl/5132048
of the origin.
image/png and image/jpeg benchmarks show no significant changes.
The image/draw changes suggest to me that making a gofix for this is not
feasible. People are just going to have to make manual fixes.
R=r
CC=golang-dev
https://golang.org/cl/4681044
writing the idct result directly to the image buffer instead of
storing it in an intermediate d.blocks field.
Writing to d.blocks was necessary when decoding to an image.RGBA image,
but now that we decode to a ycbcr.YCbCr we can write each component
directly to the image buffer.
Crude "time ./6.out" scores to decode a specific 2592x1944 JPEG 20
times show a 16% speed-up:
BEFORE
user 0m10.410s
user 0m10.400s
user 0m10.480s
user 0m10.480s
user 0m10.460s
AFTER
user 0m9.050s
user 0m9.050s
user 0m9.050s
user 0m9.070s
user 0m9.020s
R=r
CC=golang-dev
https://golang.org/cl/4523052
Avoids image.At(), color.RGBA(), opposing 8 bit shifts,
and min function calls in a loop. Not as pretty as before,
but the pure version is still there to revert back to
later if/when the compiler gets better.
before (best of 5)
jpeg.BenchmarkEncodeRGBOpaque 50 64781360 ns/op 18.97 MB/s
after (best of 5)
jpeg.BenchmarkEncodeRGBOpaque 50 42044300 ns/op 29.23 MB/s
(benchmarked on an HP z600; 16 core Xeon E5520 @ 2.27Ghz)
R=r, r2, nigeltao
CC=golang-dev
https://golang.org/cl/4433088
