When building the corpus of local packages, a "too many open files"
error would cause some directories inside GOPATH to be skipped.
Further, the error would not be reported because it was masked by a
"file not found" error from the GOROOT VFS layer.
This change adds a rate limit around parsing files when buildling
the directory tree, error reporting when godoc is run with -v, and
fixes the masked error issue in the vfs package.
It's possible that the rate limiting could be put into the
godoc/vfs/gatefs package, but I tried making the gate account for
open files (not just individual open/close/read/write operations)
but then godoc just hard locks (it wasn't designed to only open 20
files at once).
Change-Id: I925d120b53d9a86430b6977cb90eb143785ecc48
Reviewed-on: https://go-review.googlesource.com/24060
Reviewed-by: Dave Day <djd@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The existing implementation of NameSpace implicitly assumes that a
FileSystem with a directory at the top will be mounted at the root
mount point "/" of the NameSpace. If this is not the case, then
Stat("/") will fail even if ReadDir("/") succeedes. This is unexpected
behavior which breaks directory traversal routines (eg. http.FileServer).
This CL adds an unexported implementation of FileSystem called emptyVFS
that emulates an empty directory and adds a NewNameSpace() function that
binds emptyVFS to "/" so that unexpected behavior does not arise even if
the use does not mount anything explicitly at "/".
Latest patch set causes the FileInfo of the empty top level emulated
directory to return "/" for Name() and Zero Time for ModTime() and
removes the related struct state fields being used in the previous
implementation.
Fixesgolang/go#14190
Change-Id: Idce2fc3c9b81206847a33840d76b660059d53d18
Reviewed-on: https://go-review.googlesource.com/19445
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
When a file is not found, it's common to return an error that can
be detected with os.IsNotExist helper. It's possible to use
os.PathError type to satisfy that requirement while still providing
the path information in the error.
Add a test that files that are not found return a non-nil error, and
that the error satisfies os.IsNotFound.
Change-Id: I5f1a26b18f2556af822ede73306541e8575ede28
Reviewed-on: https://go-review.googlesource.com/19503
Reviewed-by: Andrew Gerrand <adg@golang.org>
- add tests for Open, ReadDir, and Stat funcs
- add tests for Seek of Open()
- simplify internal handling of absolute vs. "zip" paths
- fix handling of "/"
The fix special cases this scenario, leaving the codepath for all
other file paths the same. Specifically,
- Exported funcs call stat(), so stat("/") is handled by simply
returning 0 to indicate all entries are (effectively) prefixed
by "/" and zipFI{"", nil} because "/" has no name and nil
indicates it is a directory.
- ReadDir("/") is further handled by seeding the existing lookup
logic with "" instead of what would have been "/". This is
necessary because, per the zipfs spec, the zip file entries MUST
NOT start with "/", so using "/" would incorrectly match nothing.
This works because seeding lookup with "" (correctly) matches
all files and then the following, existing logic (correctly) pares
things down to just the files in the root directory; not in any
subdirectories.
Verified that godoc -zip still works.
Fixesgolang/go#12743
Change-Id: Icb5f01b8a29cefa4e2820135f318894042970301
Reviewed-on: https://go-review.googlesource.com/16925
Reviewed-by: Andrew Gerrand <adg@golang.org>
When Stat() or IsDir() return errors, close the file to avoid a leak.
Change-Id: I46d5e34c3443413ca674f2a703d033d945c21efa
Reviewed-on: https://go-review.googlesource.com/10557
Reviewed-by: Minux Ma <minux@golang.org>
Rewrite performed with this command:
sed -i '' 's_code.google.com/p/go\._golang.org/x/_g' \
$(grep -lr 'code.google.com/p/go.' *)
LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/170920043
Not for Go 1.2. Still needs a flag.
Linux at least (and likely other OSes) don't like you doing
a few hundred readdirs at once and spawing as many threads.
R=golang-dev, adg, jeremyjackins
CC=golang-dev
https://golang.org/cl/30620043
On big corpuses, the indexer was spending most of its time waiting
for filesystem operations (especially with network filesystems)
and not actually indexing. This keeps the filesystem busy and indexer
running in different goroutines.
Also, add a hook to let godoc hosts disable indexing of certain
directories.
And finally, start adding tests for godoc, which required
fleshing out (and testing) the mapfs code.
R=golang-dev, adg, bgarcia
CC=golang-dev
https://golang.org/cl/21520045
cmd/godoc/godoc.go is now merged into main.go, which is now
only 530 lines.
App Engine mode is still broken, but should be easy to fix up.
(just needs a global *godoc.Presentation created in init)
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/11498044
