1
0
mirror of https://github.com/golang/go synced 2024-11-14 15:10:54 -07:00
Commit Graph

641 Commits

Author SHA1 Message Date
Russ Cox
478bde3a43 io/fs: add Sub
Sub provides a convenient way to refer to a subdirectory
automatically in future operations, like Unix's chdir(2).

The CL also includes updates to fstest to check Sub implementations.

As part of updating fstest, I changed the meaning of TestFS's
expected list to introduce a special case: if you list no expected files,
that means the FS must be empty. In general it's OK not to list all
the expected files, but if you list none, that's almost certainly a
mistake - if your FS were broken and empty, you wouldn't find out.
Making no expected files mean "must be empty" makes the mistake
less likely - if your file system ever worked, then your test will keep
it working.

That change found a testing bug: embedtest was making exactly
that mistake.

Fixes #42322.

Change-Id: I63fd4aa866b30061a0e51ca9a1927e576d6ec41e
Reviewed-on: https://go-review.googlesource.com/c/go/+/274856
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-12-04 16:49:30 +00:00
Russ Cox
3d913a9266 os: add ReadFile, WriteFile, CreateTemp (was TempFile), MkdirTemp (was TempDir) from io/ioutil
io/ioutil was a poorly defined collection of helpers.
Proposal #40025 moved out the generic I/O helpers to io.
This CL for proposal #42026 moves the OS-specific helpers to os,
making the entire io/ioutil package deprecated.

For #42026.

Change-Id: I018bcb2115ef2ff1bc7ca36a9247eda429af21ad
Reviewed-on: https://go-review.googlesource.com/c/go/+/266364
Trust: Russ Cox <rsc@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
2020-12-02 17:00:06 +00:00
Chris Waldon
c193279e2c os: return proper user directories on iOS
Separating iOS into its own runtime constant broke the logic
here to derive the correct home, cache, and config directories
on iOS devices.

Fixes #42878

Change-Id: Ie4ff57895fcc34b0a9af45554ea3a346447d2e7a
GitHub-Last-Rev: 5e74e64917
GitHub-Pull-Request: golang/go#42879
Reviewed-on: https://go-review.googlesource.com/c/go/+/273947
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2020-11-30 17:45:06 +00:00
Joshua M. Clulow
e5da18df52 os/exec: constrain thread usage in leaked descriptor test on illumos
On illumos systems, libc can under some conditions make use of files
from /proc.  In the case of this test, the creation of new threads was
(in the target thread) causing libc to open and close
"/proc/self/lwp/5/lwpname" to set the thread name, which raced with the
leaking descriptor check (see detailed analysis in #42431).

This change requests that the Go runtime use less threads in the child
process used to check for leaked descriptors, without just disabling the
test.  After a thousand repeated trials, the test no longer fails on
illumos.

Fixes #42431.

Change-Id: Iefda26134fc91f7cb205754676e9845d9b7205cc
Reviewed-on: https://go-review.googlesource.com/c/go/+/273966
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-11-30 03:18:36 +00:00
Tobias Klauser
d7974c31d0 os: gofmt
As reported by John Papandriopoulos, some parts of CL 216622 weren't
properly formatted.

Change-Id: I3a76abb6213bb17ef440036295c86d930703b456
Reviewed-on: https://go-review.googlesource.com/c/go/+/269218
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
2020-11-12 10:22:50 +00:00
Ian Lance Taylor
b641f0dcf4 os: clarify that IsExist and friends do not use errors.Is
Fixes #41122

Change-Id: Ie5cb0b19ac461d321520b1ebfc493a0ca22232a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/268897
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2020-11-11 20:26:44 +00:00
Joshua M. Clulow
bb9a96d03a os/exec: use "pfiles" for fd debugging on illumos
On illumos (and Solaris) systems, the native "pfiles" tool provides the
best information about open file descriptors for a process:

    https://illumos.org/man/1/pfiles

Use that instead of "lsof" when debugging file descriptor leaks.

Updates #42431.

Change-Id: If1250c4e6c9e8adbd076495a09fb1ce63abcc68b
Reviewed-on: https://go-review.googlesource.com/c/go/+/268019
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-07 14:23:20 +00:00
Brad Fitzpatrick
8ab8125fbd os: remove unused variable in unix implementation of File.readdir
Change-Id: I0dd8a325bce6ed12d1ec1dc206ded62398925aef
Reviewed-on: https://go-review.googlesource.com/c/go/+/267758
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
2020-11-05 16:20:01 +00:00
Russ Cox
f532f19d94 os: avoid nil returns from Readdirnames, Readdir, ReadDir
The refactoring of this code while adding ReadDir stopped
pre-allocating a 100-entry slice for the results.
That seemed like a good idea in general, since many
directories have nowhere near 100 entries, but it had the
side effect of returning a nil slice for an empty directory.

Some “golden” tests that are too sensitive about nil vs not
inside Google broke because Readdirnames(-1) was now
returning nil instead of []string{} on an empty directory.
It seems likely there are other such tests in the wild, and
it doesn't seem worth breaking them.

This commit restores the non-nil-ness of the old result,
without restoring the excessive preallocation.

Fixes #42367.

Change-Id: I2be72030ac703346e859a97c2d4e456fadfce9b2
Reviewed-on: https://go-review.googlesource.com/c/go/+/267637
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-11-04 21:17:01 +00:00
Alwin Doss
45205bc47b os: export ErrProcessDone variable in windows and plan9
Exposes ErrProcessDone variable in windows and plan9
also returns this error code instead of
errors.New("os: process already finished")

Fixes #42311

Change-Id: Ie807b6526e7b6c27636e6bffe5ff0c904b319be4
GitHub-Last-Rev: 2153e0d702
GitHub-Pull-Request: golang/go#42313
Reviewed-on: https://go-review.googlesource.com/c/go/+/266997
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-11-03 00:56:12 +00:00
Colin Arnott
f14119b561 os: export errFinished as ErrProcessDone
(*Process).Signal returns an error sentinel, previously errFinished,
when (*Process).done or syscall.ESRCH. Callers would like the ability to
test for this state, so the value has been exported as ErrProcessDone.

Fixes #39444

Change-Id: I510e7647cc032af290180de5149f35ab7b09a526
Reviewed-on: https://go-review.googlesource.com/c/go/+/242998
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
2020-10-31 08:41:25 +00:00
Antonio Huete Jimenez
e3c58bbeb8 os: do not use procfs for os.Executable in dragonfly
procfs(5) is not always mounted in DragonFly BSD, for example during
  the binary package build with synth. os.Executable() consumers
  will then fail, we've spotted this when trying to build tinygo:

    [...]

    copying source files
    ./build/tinygo build-builtins -target=armv6m-none-eabi [...]
    panic: could not get executable path: readlink /proc/curproc/file:
    no such file or directory

    [...]

  Use KERN_PROC_PATHNAME as FreeBSD does.

Change-Id: Ic65bea02cd0309fb24dec8ba8d2b151d1acde67b
GitHub-Last-Rev: 083120a43b
GitHub-Pull-Request: golang/go#36826
Reviewed-on: https://go-review.googlesource.com/c/go/+/216622
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
2020-10-28 17:08:06 +00:00
Russ Cox
1b09d43067 all: update references to symbols moved from io/ioutil to io
The old ioutil references are still valid, but update our code
to reflect best practices and get used to the new locations.

Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.
Also excluded vendored code.

For #41190.

Change-Id: I6d86f2bf7bc37a9d904b6cee3fe0c7af6d94d5b1
Reviewed-on: https://go-review.googlesource.com/c/go/+/263142
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20 18:41:18 +00:00
Russ Cox
b1f76f7a22 os: add DirFS
It will inevitably be important to be able to pass an operating system
directory to code written to expect an fs.FS.

os.DirFS provides the conversion.

For #41190.

Change-Id: Id1a8fcbe4c7a30de2c47dea0504e9481a88b1b39
Reviewed-on: https://go-review.googlesource.com/c/go/+/243911
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 17:52:48 +00:00
Russ Cox
fcb9d6b5d0 io/fs: add FS, File, ReadDirFile; move DirEntry from os
These are the core interfaces for the io/fs design.
See #41190 and https://golang.org/s/draft-iofs-design for details.

DirEntry was left behind in the previous move from os
but is needed for ReadDirFile, so it moves in this commit.

Also apply a couple comment changes suggested in
the review of CL 261540.

For #41190.

Change-Id: I087741545139ed30b9ba5db728a0bad71129500b
Reviewed-on: https://go-review.googlesource.com/c/go/+/243908
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-20 02:32:46 +00:00
Russ Cox
7bb721b938 all: update references to symbols moved from os to io/fs
The old os references are still valid, but update our code
to reflect best practices and get used to the new locations.

Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.

For #41190.

Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/243907
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 02:32:42 +00:00
Russ Cox
d4da735091 io/fs: move FileInfo, FileMode, PathError, ErrInvalid, ... from os to io/fs
First step of creating the new io/fs package.

For #41190.

Change-Id: I1339b1abdd533b0f1deab283628088b2f706fb5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/243906
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20 02:32:41 +00:00
Russ Cox
2291cae2af os: use keyed literals for PathError
Necessary to move PathError to io/fs.

For #41190.

Change-Id: I05e87675f38a22f0570d4366b751b6169f7a1b13
Reviewed-on: https://go-review.googlesource.com/c/go/+/243900
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-20 00:59:22 +00:00
Russ Cox
a4ede9f9a6 os: add File.ReadDir method and DirEntry type
ReadDir provides a portable, efficient way to read a directory
and discover the type of directory entries.
This enables a more efficient file system walk, yet to be added.

See #41467 for the proposal review for the API.

Fixes #41467.

Change-Id: I461a526793ae46df48821aa448b04f1705546739
Reviewed-on: https://go-review.googlesource.com/c/go/+/261540
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 00:59:20 +00:00
Alberto Donizetti
3853272573 Revert "test: add regression test from #41474"
This reverts CL 263097.

Reason for revert: broke the noopt builder.

Change-Id: Ie36d2c3ed9449b4425732072db624c8e18f965f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/263537
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-10-19 13:49:50 +00:00
Alberto Donizetti
c8f6135d4f test: add regression test from #41474
This issue was fixed with multiple individual compiler optimizations,
each of which had their own respective test cases. This CL just adds
the capstone test case to demonstrate that the issue has been fixed
and doesn't regress again.

Updates #41474.

Change-Id: Iae752d4b0e7b83ee356b946843340a4fbc254058
Reviewed-on: https://go-review.googlesource.com/c/go/+/263097
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-17 07:18:20 +00:00
gzdaijie
e92ce92662 os: call f.Close before t.Fatal to avoid vet unreachable code warnings
Change-Id: Ic3f756ecf9b4fad8fb8c259e7dad6df894863b0e
GitHub-Last-Rev: 303e524029
GitHub-Pull-Request: golang/go#41900
Reviewed-on: https://go-review.googlesource.com/c/go/+/261217
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-12 09:28:40 +00:00
Cherry Zhang
f8df205e74 all: enable more tests on macOS/ARM64
On macOS, we can do "go build", can exec, and have the source
tree available, so we can enable more tests.

Skip ones that don't work. Most of them are due to that it
requires external linking (for now) and some tests don't work
with external linking (e.g. runtime deadlock detection). For
them, helper functions CanInternalLink/MustInternalLink are
introduced. I still want to have internal linking implemented,
but it is still a good idea to identify which tests don't work
with external linking.

Updates #38485.

Change-Id: I6b14697573cf3f371daf54b9ddd792acf232f2f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/260719
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-09 01:09:06 +00:00
Changkun Ou
05b626e490 os: fix SyscallConn typos in the File.Fd comments
This CL fixes two typos introduced in CL 256899.

Change-Id: I47f0a3097deeeec8d6e9bbe7073fcf7a28c5dff9
Reviewed-on: https://go-review.googlesource.com/c/go/+/257997
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-28 08:16:57 +00:00
Tobias Klauser
5755bad42a os: remove ENOTSUP special case in Getwd on darwin
ENOTSUP was used as a signaling error in the custom implementation of
syscall.Getwd to fall back to the slow algorithm. Since CL 257637 Getwd
directly calls the respective function from libSystem.dylib which can no
longer return ENOTSUP.

Change-Id: I8e65e42b3ea069bf78969a29f2af1c55552e2949
Reviewed-on: https://go-review.googlesource.com/c/go/+/257644
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-28 06:28:02 +00:00
Changkun Ou
7bb6fed9b5 os: document and emphasize a potential misuse of File.Fd
This CL revises the document of File.Fd that explicitly points
its user to runtime.SetFinalizer where contains the information
that a file descriptor could be closed in a finalizer and therefore
causes a failure in syscall.Write if runtime.KeepAlive is not invoked.

The CL also suggests an alternative of File.Fd towards File.SyscallConn.

Fixes #41505

Change-Id: I6816f0157add48b649bf1fb793cf19dcea6894b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/256899
Reviewed-by: Rob Pike <r@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
2020-09-27 02:23:55 +00:00
Cherry Zhang
a413908dd0 all: add GOOS=ios
Introduce GOOS=ios for iOS systems. GOOS=ios matches "darwin"
build tag, like GOOS=android matches "linux" and GOOS=illumos
matches "solaris". Only ios/arm64 is supported (ios/amd64 is
not).

GOOS=ios and GOOS=darwin remain essentially the same at this
point. They will diverge at later time, to differentiate macOS
and iOS.

Uses of GOOS=="darwin" are changed to (GOOS=="darwin" || GOOS=="ios"),
except if it clearly means macOS (e.g. GOOS=="darwin" && GOARCH=="amd64"),
it remains GOOS=="darwin".

Updates #38485.

Change-Id: I4faacdc1008f42434599efb3c3ad90763a83b67c
Reviewed-on: https://go-review.googlesource.com/c/go/+/254740
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2020-09-23 18:12:59 +00:00
Tobias Klauser
0d83fe68a8 os: close fd if fd.Stat fails in Getwd
Avoid leaking fd in case fd.Stat() fails in the fall back implementation
of Getwd.

Change-Id: I8656d42e8dbc8893b7159873f173d6bf0d4febe6
Reviewed-on: https://go-review.googlesource.com/c/go/+/256221
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-22 13:25:27 +00:00
Tobias Klauser
e82c9bd816 os, internal/syscall/unix: use pipe2 instead of pipe on illumos
Illumos provides the pipe2 syscall. Add a wrapper to
internal/syscall/unix and use it to implement os.Pipe.

Change-Id: I26ecdbcae1e8d51f80e2bc8a86fb129826387b1f
Reviewed-on: https://go-review.googlesource.com/c/go/+/254981
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-16 08:49:14 +00:00
Henrique Vicente
b6dbaef68f os/signal: add NotifyContext to cancel context using system signals
Fixes #37255

Change-Id: Ic0fde3498afefed6e4447f8476e4da7c1faa7145
Reviewed-on: https://go-review.googlesource.com/c/go/+/219640
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Giovanni Bajo <rasky@develer.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-15 23:14:33 +00:00
Constantin Konstantinidis
d7384f3612 os: implement File.Chmod on Windows
Fixes: #39606

Change-Id: I4def67ef18bd3ff866b140f6e76cdabe5d51a1c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/250077
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2020-09-11 22:10:02 +00:00
Ian Lance Taylor
4f76fe8675 cmd/go, testing, os: fail test that calls os.Exit(0)
This catches cases where a test calls code that calls os.Exit(0),
thereby skipping all subsequent tests.

Fixes #29062

Change-Id: If9478972f40189e27623557e7141469ca4234d89
Reviewed-on: https://go-review.googlesource.com/c/go/+/250977
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-08-27 23:19:15 +00:00
Bryan C. Mills
b3d9cf7a07 os: return a *PathError from Readdirnames and Readdir on POSIX platforms
Previously, Readdirnames returned a *PathError on Windows and Plan 9,
but a *SyscallError on POSIX systems.

In contrast, similar methods (such as Stat) return a *PathError on all platforms.

Fixes #38923

Change-Id: I26395905b1e723933f07b792c7aeee7c335949cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/233917
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-08-25 04:39:40 +00:00
Ian Lance Taylor
6b420169d7 os, internal/poll: loop on EINTR for all file syscalls
When using a FUSE file system, any system call that touches the file
system can return EINTR.

Fixes #40846

Change-Id: I25d32da22cec08dea81ab297291a85ad72db2df7
Reviewed-on: https://go-review.googlesource.com/c/go/+/249178
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-08-19 21:49:56 +00:00
Ian Lance Taylor
9699086043 runtime: mark OpenBSD raise function nosplit
It is called by the signal handler before switching to gsignal
(sigtrampgo -> sigfwdgo -> dieFromSignal -> raise)
which means that it must not split the stack.

All other instances of raise are already marked nosplit.

Fixes #40076

Change-Id: I4794491331af48c46d0d8ebc82d34c6483f0e6cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/241121
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-07-08 19:50:20 +00:00
Joe Tsai
0844ff8eef os: fix regression with handling of nil *File
Use of a nil *File as an argument should not result in a panic,
but result in the ErrInvalid error being returned.
Fix the copy_file_range implementation to preserve this semantic.

Fixes #40115

Change-Id: Iad5ac39664a3efb7964cf55685be636940a8db13
Reviewed-on: https://go-review.googlesource.com/c/go/+/241417
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-07-08 18:29:38 +00:00
Fazlul Shahriar
334752dc82 os: return *PathError error in File.Sync on Plan 9
File.Sync was returning *SyscallError instead of *PathError on Plan 9.
Adjust the error type to match other systems.

Fixes #39800

Change-Id: I844e716eb61c193ef78d29cb0b4a3ef790bb3320
Reviewed-on: https://go-review.googlesource.com/c/go/+/239857
Reviewed-by: David du Colombier <0intro@gmail.com>
2020-06-25 12:02:38 +00:00
Ian Lance Taylor
e64675a79f os: always check for EINTR in calls to open
For #11180
For #20400
For #39237

Change-Id: I8de97517c8f92c08f0c8a51f651a17e31617979b
Reviewed-on: https://go-review.googlesource.com/c/go/+/236997
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-06-08 22:06:39 +00:00
Filippo Valsorda
608cdcaede all: replace usages of whitelist/blacklist and master/slave
There's been plenty of discussion on the usage of these terms in tech.
I'm not trying to have yet another debate. It's clear that there are
people who are hurt by them and who are made to feel unwelcome by their
use due not to technical reasons but to their historical and social
context. That's simply enough reason to replace them.

Anyway, allowlist and blocklist are more self-explanatory than whitelist
and blacklist, so this change has negative cost.

Didn't change vendored, bundled, and minified files. Nearly all changes
are tests or comments, with a couple renames in cmd/link and cmd/oldlink
which are extremely safe. This should be fine to land during the freeze
without even asking for an exception.

Change-Id: I8fc54a3c8f9cc1973b710bbb9558a9e45810b896
Reviewed-on: https://go-review.googlesource.com/c/go/+/236857
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Khosrow Moossavi <khos2ow@gmail.com>
Reviewed-by: Leigh McCulloch <leighmcc@gmail.com>
Reviewed-by: Urban Ishimwe <urbainishimwe@gmail.com>
2020-06-08 01:03:14 +00:00
Bryan C. Mills
86ed0955bf os: in Symlink, stat the correct target path for drive-relative targets on Windows
Previously, when the target (“old”) path passed to os.Symlink was a
“root-relative” Windows path,¹ we would erroneously prepend
destination (“new”) path when determining which path to Stat,
resulting in an invalid path which was then masked by the lack of
error propagation for the Stat call (#39183).

If the link target is a directory (rather than a file), that would
result in the symlink being created without the
SYMBOLIC_LINK_FLAG_DIRECTORY flag, which then fails in os.Open.

¹https://docs.microsoft.com/en-us/windows/win32/fileio/creating-symbolic-links

Updates #39183

Change-Id: I04f179cd2b0c44f984f34ec330acad2408aa3a20
Reviewed-on: https://go-review.googlesource.com/c/go/+/235317
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-05-28 21:41:10 +00:00
matsuyoshi
8194187a2d os: use same link in UserCacheDir/UserConfigDir doc
Change-Id: I94c385243c37589f56aadaa30336b400adf31308
Reviewed-on: https://go-review.googlesource.com/c/go/+/234959
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-05-24 01:19:35 +00:00
Ian Lance Taylor
567556d786 syscall: preserve Windows file permissions for O_CREAT|O_TRUNC
On Windows, calling syscall.Open(file, O_CREAT|O_TRUNC, 0) for a file
that already exists would change the file to be read-only.
That is not how the Unix syscall.Open behaves, so avoid it on
Windows by calling CreateFile twice if necessary.

Fixes #38225

Change-Id: I70097fca8863df427cc8a97b9376a9ffc69c6318
Reviewed-on: https://go-review.googlesource.com/c/go/+/234534
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2020-05-20 21:52:34 +00:00
Bryan C. Mills
13617380ca testing: clean up remaining TempDir issues from CL 231958
Updates #38850

Change-Id: I33f48762f5520eb0c0a841d8ca1ccdd65ecc20c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/234583
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-05-19 19:55:14 +00:00
Ian Lance Taylor
8c1db77a92 internal/poll, os: loop on EINTR
Historically we've assumed that we can install all signal handlers
with the SA_RESTART flag set, and let the system restart slow functions
if a signal is received. Therefore, we don't have to worry about EINTR.

This is only partially true, and we've added EINTR checks already for
connect, and open/read on Darwin, and sendfile on Solaris.

Other cases have turned up in #36644, #38033, and #38836.

Also, #20400 points out that when Go code is included in a C program,
the C program may install its own signal handlers without SA_RESTART.
In that case, Go code will see EINTR no matter what it does.

So, go ahead and check for EINTR. We don't check in the syscall package;
people using syscalls directly may want to check for EINTR themselves.
But we do check for EINTR in the higher level APIs in os and net,
and retry the system call if we see it.

This change looks safe, but of course we may be missing some cases
where we need to check for EINTR. As such cases turn up, we can add
tests to runtime/testdata/testprogcgo/eintr.go, and fix the code.
If there are any such cases, their handling after this change will be
no worse than it is today.

For #22838
Fixes #20400
Fixes #36644
Fixes #38033
Fixes #38836

Change-Id: I7e46ca8cafed0429c7a2386cc9edc9d9d47a6896
Reviewed-on: https://go-review.googlesource.com/c/go/+/232862
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-05-11 22:38:32 +00:00
Alex Brainman
53f27474a4 syscall, internal/syscall/windows: remove utf16PtrToString parameter
CL 208617 introduced syscall.utf16PtrToString and
internal/syscall/windows.UTF16PtrToString functions.

Original version of CL 208617 did not include syscall.utf16PtrToString
and internal/syscall/windows.UTF16PtrToString max parameter. The
parameter was added by Brad at the request of Ian. Ian said:

"In some cases it seems at least possible that the null terminator is
not present. I think it would be safer if we passed a maximum length
here."

The syscall.utf16PtrToString and
internal/syscall/windows.UTF16PtrToString function are designed to work
with only null terminated strings. So max parameter is superfluous.

This change removes max parameter.

Updates #34972

Change-Id: Ifea65dbd86bca8a08353579c6b9636c6f963d165
Reviewed-on: https://go-review.googlesource.com/c/go/+/228858
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-05-03 07:23:32 +00:00
Ian Lance Taylor
be08e10b3b syscall: if Setctty, require that Ctty be a child descriptor
Ctty was always handled as a child descriptor, but in some cases
passing a parent descriptor would also work. This depended on
unpredictable details of the implementation. Reject those cases to
avoid confusion.

Also reject setting both Setctty and Foreground, as they use Ctty
in incompatible ways. It's unlikely that any programs set both fields,
as they don't make sense together.

Fixes #29458

Change-Id: Ieba2d625711fd4b82c8e65e1feed02fd1fb25e6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/231638
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
2020-05-01 21:57:29 +00:00
Andrei Tudor Călin
7be3f09deb os, internal/poll, internal/syscall/unix: use copy_file_range on Linux
Linux 4.5 introduced (and Linux 5.3 refined) the copy_file_range
system call, which allows file systems the opportunity to implement
copy acceleration techniques. This commit adds support for
copy_file_range(2) to the os package.

Introduce a new ReadFrom method on *os.File, which makes *os.File
implement the io.ReaderFrom interface. If dst and src are both files,
this enables io.Copy(dst, src) to call dst.ReadFrom(src), which, in
turn, will call copy_file_range(2) if possible. If copy_file_range(2)
is not supported by the host kernel, or if either of dst or src
refers to a non-regular file, ReadFrom falls back to the regular
io.Copy code path.

Add internal/poll.CopyFileRange, which acquires locks on the
appropriate poll.FDs and performs the actual work, as well as
internal/syscall/unix.CopyFileRange, which wraps the copy_file_range
system call itself at the lowest level.

Rework file layout in internal/syscall/unix to accomodate the
additional system call numbers needed for copy_file_range.
Merge these definitions with the ones used by getrandom(2) into
sysnum_linux_$GOARCH.go files.

A note on additional optimizations: if dst and src both refer to pipes
in the invocation dst.ReadFrom(src), we could, in theory, use the
existing splice(2) code in package internal/poll to splice directly
from src to dst. Attempting this runs into trouble with the poller,
however. If we call splice(src, dst) and see EAGAIN, we cannot know
if it came from src not being ready for reading or dst not being
ready for writing. The write end of src and the read end of dst are
not under our control, so we cannot reliably use the poller to wait
for readiness. Therefore, it seems infeasible to use the new ReadFrom
method to splice between pipes directly. In conclusion, for now, the
only optimization enabled by the new ReadFrom method on *os.File is
the copy_file_range optimization.

Fixes #36817.

Change-Id: I696372639fa0cdf704e3f65414f7321fc7d30adb
Reviewed-on: https://go-review.googlesource.com/c/go/+/229101
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-04-28 00:59:36 +00:00
Ian Lance Taylor
d422f54619 os, net: define and use os.ErrDeadlineExceeded
If an I/O operation fails because a deadline was exceeded,
return os.ErrDeadlineExceeded. We used to return poll.ErrTimeout,
an internal error, and told users to check the Timeout method.
However, there are other errors with a Timeout method that returns true,
notably syscall.ETIMEDOUT which is returned for a keep-alive timeout.
Checking errors.Is(err, os.ErrDeadlineExceeded) should permit code
to reliably tell why it failed.

This change does not affect the handling of net.Dialer.Deadline,
nor does it change the handling of net.DialContext when the context
deadline is exceeded. Those cases continue to return an error
reported as "i/o timeout" for which Timeout is true, but that error
is not os.ErrDeadlineExceeded.

Fixes #31449

Change-Id: I0323f42e944324c6f2578f00c3ac90c24fe81177
Reviewed-on: https://go-review.googlesource.com/c/go/+/228645
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2020-04-25 00:26:48 +00:00
Richard Miller
664d270727 os: correct bad PathError message from FileOpen with O_CREATE on Plan 9
On Plan 9, FileOpen with flag O_CREATE & ~O_TRUNC is done in two
steps.  First, syscall.Open is attempted, to avoid truncation when opening
an existing file.  If that fails because the file doesn't exist,
syscall.Create is used to create a new file.  If the Create fails,
for example because we are racing with another process to create a
ModeExclusive file, the PathError returned from FileOpen should reflect
the result of the Create, not the "does not exist" error from the initial
Open attempt.

Fixes #38540

Change-Id: I90c95a301de417ecdf79cd52748591edb1dbf528
Reviewed-on: https://go-review.googlesource.com/c/go/+/229099
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David du Colombier <0intro@gmail.com>
2020-04-21 11:41:40 +00:00
Ian Lance Taylor
98c6b9844b os/exec: build TestExtraFiles subprocess without cgo
Fixes #25628

Change-Id: I8b69e59f9c0123c4f65b5931d7c6d7ecc1c720e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/228639
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-04-18 19:58:12 +00:00