In the distant past, Pipe was implemented with channels and a
long running pipe.run goroutine (see CL 994043).
This approach of having all communication serialized through the
run method was error prone giving Pipe a history of deadlocks
and race conditions.
After the introduction of sync.Cond, the implementation was rewritten
(see CL 4252057) to use condition variables and avoid the
long running pipe.run goroutine. While this implementation is superior
to the previous one, this implementation is strange in that the
p.data field is always set immediately prior to signaling the other
goroutine with Cond.Signal, effectively making the combination of the
two a channel-like operation. Inferior to a channel, however, this still
requires explicit locking around the p.data field.
The data+rwait can be effectively be replaced by a "chan []byte" to
inform a reader that there is data available.
The data+wwait can be effectively be replaced by a "chan int" to
inform a writer of how many bytes were read.
This implementation is a simplified from net.Pipe in CL 37402.
Change-Id: Ia5b26320b0525934fd87a3b69a091c787167f5aa
Reviewed-on: https://go-review.googlesource.com/65330
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
When we added a Stat call to determine the initial buffer size in
https://golang.org/cl/163069, we included an arbitrary 1e9-byte limit
"just in case". That interacts badly with power-of-2 resizing in
*bytes.Buffer: it causes buffers reading from very large files to
consume up to twice the necessary space.
The documentation for (os.FileInfo).Size says that it reports "length
in bytes for regular files; system-dependent for others", but the
"system dependent" cases overwhelmingly return either a small number
(e.g., the length of the target path for a symlink) or a non-positive
number (e.g., for a file in /proc under Linux). It should be
appropriate to use the number reported by Size as an approximate lower
bound, even if it is large.
fixes#21455
Change-Id: I609c72519b7b87428c24d0b22db46eede30e0e54
Reviewed-on: https://go-review.googlesource.com/55870
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Change-Id: I24374accf48d43edf4bf27ea6ba2245ddca558ad
Reviewed-on: https://go-review.googlesource.com/50910
Reviewed-by: Giovanni Bajo <rasky@develer.com>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
Run-TryBot: Joe Tsai <thebrokentoaster@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
While there's an example for SectionReader.Seek, if someone is
seeking documentation specifically about Seeker.Seek, they may
not immediately find the SectionReader example. Offset and whence
may not be entirely intuitive to new developers either, so include
examples of both positive/negative offsets and SeekStart/SeekEnd.
Change-Id: I5b7442ccf683d9706e9261c11bc0ea31a1ac21d4
Reviewed-on: https://go-review.googlesource.com/48873
Reviewed-by: Kevin Burke <kev@inburke.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Document that the byte value returned by ReadByte() is meaningless
if its error != nil. Because io.Reader and io.ByteReader are similar in
name, this CL aims to clear up any ambiguity surrounding the returned
values, particularly where io.Reader is allowed to return both a
non-zero number of valid bytes and err == EOF.
Fixes#20825
Change-Id: I3a23c18c80c471c0caae3b4d2f6f8e547da0bed9
Reviewed-on: https://go-review.googlesource.com/46950
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TestMultiReaderFlatten determines the call depth by counting PCs
returned by runtime.Callers. With inlining, this is incorrect because
a PC can represent multiple calls. Furthermore, runtime.Callers might
return an additional "skip" PC, which does not represent a real call.
This modifies the test to use CallersFrames to determine the call depth.
Now the test passes with -l=4.
Change-Id: I284f3b1e0b2d194bd08c230c616914503e5a370d
Reviewed-on: https://go-review.googlesource.com/40990
Run-TryBot: David Lazar <lazard@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The first part of this test tries to confirm that we can't create
a TempFile in a non-existent directory, but does not ensure that
the non-existent directory really does not exist. Instead, let's
create an empty temp directory, and use a non-existent subdir of
that.
Change-Id: I176f14ed5f5a2d7a8c29d8f6949755db69d7dbb6
Reviewed-on: https://go-review.googlesource.com/40914
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This ensures there isn't a live reference to buf1 on our stack
when MultiReader is inlined.
Fixes#18819.
Change-Id: I96a8cdc1ffad8f8a10c0ddcbf0299005f3176b61
Reviewed-on: https://go-review.googlesource.com/35931
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Since commit cc62bed0 (CL 994043) the pipe deadlock when doing
Read+Close or Write+Close on same end was fixed, alas with test for
Read+Close case only.
Then commit 6d6f3381 (CL 4252057) made a thinko: in the writer path
p.werr is checked for != nil and then err is set but there is no break
from waiting loop unlike break is there in similar condition for reader.
Together with having only Read+Close case tested that made it to leave
reintroduced Write+Close deadlock unnoticed.
Fix it.
Implicitly this also fixes net.Pipe to conform to semantic of net.Conn
interface where Close is documented to unblock any blocked Read or Write
operations.
No test added to net/ since net.Pipe tests are "Assuming that the
underlying io.Pipe implementation is solid and we're just testing the
net wrapping". The test added in this patch should be enough to cover
the breakage.
Fixes#18401
Updates #18170
Change-Id: I9e9460b3fd7d220bbe60b726accf86f352aed8d4
Reviewed-on: https://go-review.googlesource.com/34637
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Specify that that LimitedReader returns EOF when the underlying
R returns EOF even if bytes remaining, N > 0.
Fixes#18271
Change-Id: I990a7135f1d31488d535238ae061d42ee96bacb7
Reviewed-on: https://go-review.googlesource.com/34249
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The combination of two prior CLs can cause panics:
* CL/17873: make chained multiReader Read more efficient
* CL/28533: make MultiReader nil exhausted Readers for earlier GC
The first CL allows MultiReader to "inherit" another MultiReader's list of Readers
for efficiency reasons. This is problematic when combined with the
later CL since that can set prior Readers in that list to nil for GC reasons.
This causes panics when two MultiReaders are used together (even synchronously).
To fix this, rather than setting consumed Readers as nil, we set them with
a special eofReader that always returns EOF.
Fixes#18232
Change-Id: I2a9357ab217e3d54d38ea9022d18e4d14f4182d3
Reviewed-on: https://go-review.googlesource.com/34140
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I avoided anywhere in the compiler or things which might be used by
the compiler in the future, since they need to build with Go 1.4.
I also avoided anywhere where there was no benefit to changing it.
I probably missed some.
Updates #16721
Change-Id: Ib3c895ff475c6dec2d4322393faaf8cb6a6d4956
Reviewed-on: https://go-review.googlesource.com/30250
TryBot-Result: Gobot Gobot <gobot@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Fixes#14196
Change-Id: Ife7950289ac6adbcfc4d0f2fce31f20bc2657858
Reviewed-on: https://go-review.googlesource.com/28772
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
No test because the language spec makes no promises in this area.
Fixes#16983
Change-Id: I1a6aa7ff87dd14aa27e8400040a6f6fc908aa1fd
Reviewed-on: https://go-review.googlesource.com/28533
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
If an io.Reader returned (non-zero, EOF), MultiReader would yield
bytes forever.
This bug has existed before Go 1 (!!), introduced in the original
MultiReader implementation in https://golang.org/cl/1764043 and also
survived basically the only update to this code since then
(https://golang.org/cl/17873, git rev ccdca832c), which was added in
Go 1.7.
This just bit me when writing a test for some unrelated code.
Fixes#16795
Change-Id: I36e6a701269793935d19a47ac12f67b07179fbff
Reviewed-on: https://go-review.googlesource.com/27397
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
The documentation previously used C style enumerations: 0, 1, 2.
While this is pretty much universally correct, it does not help a user
become aware of the existence of the SeekStart, SeekCurrent, and SeekEnd
constants. Thus, we should use them in the documentation to direct people's
attention to them.
Updates #6885
Change-Id: I44b5e78d41601c68a0a1c96428c853df53981d52
Reviewed-on: https://go-review.googlesource.com/23551
Reviewed-by: Andrew Gerrand <adg@golang.org>
It's not clear we want to enshrine an io interface in which Size cannot
return an error. Because this requires more thought before committing
to the API, remove from Go 1.7.
Fixes#15818.
Change-Id: Ic4138ffb0e033030145a12d33f78078350a8381f
Reviewed-on: https://go-review.googlesource.com/23392
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Russ Cox <rsc@golang.org>
before this change, when io.MultiReader was called many times but contain few
underlying readers, calls to Read were unnecessarily expensive.
Fixes#13558
Change-Id: I3ec4e88c7b50c075b148331fb1b7348a5840adbe
Reviewed-on: https://go-review.googlesource.com/17873
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL/19862 (f79b50b8d5) recently introduced the constants
SeekStart, SeekCurrent, and SeekEnd to the io package. We should use these constants
consistently throughout the code base.
Updates #15269
Change-Id: If7fcaca7676e4a51f588528f5ced28220d9639a2
Reviewed-on: https://go-review.googlesource.com/22097
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Joe Tsai <joetsai@digital-static.net>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a proposal. The old name is pretty poor. The new one describes
it better and may be easier to remember. It does not start with Read,
though I think that inconsistency is worthwhile.
Reworded the comment a bit for clarity.
Change-Id: Icb4f9c663cc68958e0363d7ff78a0b29cc521f98
Reviewed-on: https://go-review.googlesource.com/21629
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
ReadAtSizer is a common abstraction for a stateless,
concurrently-readable fixed number of bytes.
This interface has existed in various codebases for over 3 years (previously
usually named SizeReaderAt). It is used inside Google in dl.google.com
(mentioned in https://talks.golang.org/2013/oscon-dl.slide) and other
packages. It is used in Camlistore, in Juju, in the Google API Go client, in
github.com/nightlyone/views, and 33 other pages of Github search results.
It is implemented by io.SectionReader, bytes.Reader, strings.Reader, etc.
Time to finally promote this interface to the standard library and give it a
standard name, blessing it as best practice.
Updates #7263
Updates #14889
Change-Id: Id28c0cafa7d2d37e8887c54708b5daf1b11c83ea
Reviewed-on: https://go-review.googlesource.com/21492
Reviewed-by: Rob Pike <r@golang.org>
This change removes a lot of dead code. Some of the code has never been
used, not even when it was first commited. The rest shouldn't have
survived refactors.
This change doesn't remove unused routines helpful for debugging, nor
does it remove code that's used in commented out blocks of code that are
only unused temporarily. Furthermore, unused constants weren't removed
when they were part of a set of constants from specifications.
One noteworthy omission from this CL are about 1000 lines of unused code
in cmd/fix, 700 lines of which are the typechecker, which hasn't been
used ever since the pre-Go 1 fixes have been removed. I wasn't sure if
this code should stick around for future uses of cmd/fix or be culled as
well.
Change-Id: Ib714bc7e487edc11ad23ba1c3222d1fd02e4a549
Reviewed-on: https://go-review.googlesource.com/20926
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The tree's pretty inconsistent about single space vs double space
after a period in documentation. Make it consistently a single space,
per earlier decisions. This means contributors won't be confused by
misleading precedence.
This CL doesn't use go/doc to parse. It only addresses // comments.
It was generated with:
$ perl -i -npe 's,^(\s*// .+[a-z]\.) +([A-Z]),$1 $2,' $(git grep -l -E '^\s*//(.+\.) +([A-Z])')
$ go test go/doc -update
Change-Id: Iccdb99c37c797ef1f804a94b22ba5ee4b500c4f7
Reviewed-on: https://go-review.googlesource.com/20022
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Dave Day <djd@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is a subset of https://golang.org/cl/20022 with only the copyright
header lines, so the next CL will be smaller and more reviewable.
Go policy has been single space after periods in comments for some time.
The copyright header template at:
https://golang.org/doc/contribute.html#copyright
also uses a single space.
Make them all consistent.
Change-Id: Icc26c6b8495c3820da6b171ca96a74701b4a01b0
Reviewed-on: https://go-review.googlesource.com/20111
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Named returned values should only be used on public funcs and methods
when it contributes to the documentation.
Named return values should not be used if they're only saving the
programmer a few lines of code inside the body of the function,
especially if that means there's stutter in the documentation or it
was only there so the programmer could use a naked return
statement. (Naked returns should not be used except in very small
functions)
This change is a manual audit & cleanup of public func signatures.
Signatures were not changed if:
* the func was private (wouldn't be in public godoc)
* the documentation referenced it
* the named return value was an interesting name. (i.e. it wasn't
simply stutter, repeating the name of the type)
There should be no changes in behavior. (At least: none intended)
Change-Id: I3472ef49619678fe786e5e0994bdf2d9de76d109
Reviewed-on: https://go-review.googlesource.com/20024
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Gerrand <adg@golang.org>
Let C be whether c valid
Let E be whether err is non-nil
The old comment explicitly says that (~C → E). However, that does call
into question whether (E → ~C), which causes doubts for users.
Without a comment at all, it is obvious that only (E ↔ ~C) makes sense.
Fixes#11308
Change-Id: I5a7d51ceb509057eccca91f57a7e48c9d1c6d112
Reviewed-on: https://go-review.googlesource.com/15256
Reviewed-by: Andrew Gerrand <adg@golang.org>
The later part of the docstring simply talks about "offset" but does
not disambiguate what it is relative to. For both the return value
and valid offsets to seek to, it only makes sense in the context of
"offset relative to origin of file".
Fixes#11877
Change-Id: Ic238a407cf8e8fdd64991d98a6584cdc8a51cd6b
Reviewed-on: https://go-review.googlesource.com/15257
Reviewed-by: Andrew Gerrand <adg@golang.org>
I think this has the same meaning as before,
but the text is tighter, and it makes some people happy.
Fixes#10182.
Change-Id: I7ee1eae4bcd6ee4a5898ea948648939e6bde5f01
Reviewed-on: https://go-review.googlesource.com/11674
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
This trivial addition to the io package makes it easy to control the
buffer size and allocation properties of io.Copy.
Change-Id: Ica1a6bd015e429d4e655bc0c6f66cea21c454acf
Reviewed-on: https://go-review.googlesource.com/8730
Reviewed-by: Russ Cox <rsc@golang.org>
"returns ... the first error" was misleading or at least confusing:
in case a Read results in an error with non-zero bytes read, and the
subsequent Write also results in an error, the error from Write is
returned, which is the second one (in the temporal dimension).
Fixes#9744
Change-Id: If8925a701e4fae820cd9df7446503403fc0785d4
Reviewed-on: https://go-review.googlesource.com/3686
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The previous wording implied that reads would return no error, rather
than EOF. It's convenient for users to know that Close() is equivalent
to CloseWithError(nil) because it can remove a branch from their error
handling code where they want to close the pipe in the appropriate way.
For example:
6e9a8cec0a/gcs/bucket.go (L637-L643)
Change-Id: I618bffe556eb518011e7ba5cdce1eb0ff536350e
Reviewed-on: https://go-review.googlesource.com/8152
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
