The number of open file descriptors reported by lsof is unreliable
because it depends on whether the parent process (the test) closed
the file descriptors it passed into the child process (lsof) before
lsof runs.
Reading /proc/self/fd directly on Linux appears to be much more
reliable and still detects any file descriptor leaks originating
from attempting to run an executable that cannot be found (issue
#5071). If /proc/self/fd is not available (e.g. on Darwin) then we
fall back to lsof and tolerate small differences in open file
descriptor counts.
Fixes#19243.
Change-Id: I052b0c129e609010f1083e43a9911cba154117bf
Reviewed-on: https://go-review.googlesource.com/37343
Run-TryBot: Michael Munday <munday@ca.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Stop reporting errors from cmd.Process.Kill; they don't matter for
purposes of this test, and they can occur if the process exits quickly.
Fixes#19211.
Fixes#19213.
Change-Id: I1a0bb9170220ca69199abb8e8811b1dde43e1897
Reviewed-on: https://go-review.googlesource.com/37309
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The below range loop will not stop when encountering
the first '.' character in a Darwin version string like "15.6.0".
for i = range osver {
if osver[i] != '.' {
continue
}
}
}
Therefore, the condition i > 2 was always satisfied and
supportsCloseOnExec was always set to true.
Since the minimum supported version of OSX for go is currently 10.8
and O_CLOEXEC is implemented from OSX 10.7 on the detection code
can be removed and support for O_CLOEXEC is always assumed to exist.
Change-Id: Idd10094d8385dd4adebc8d7a6d9e9a8f29455867
Reviewed-on: https://go-review.googlesource.com/37193
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
If the caller set ups a Credential in os/exec.Command,
os/exec.Command.Start will end up calling setgroups(2), even if no
supplementary groups were given.
Only root can call setgroups(2) on BSD kernels, which causes Start to
fail for non-root users when they try to set uid and gid for the new
process.
We fix by introducing a new field to syscall.Credential named
NoSetGroups, and setgroups(2) is only called if it is false.
We make this field with inverted logic to preserve backward
compatibility.
RELNOTES=yes
Change-Id: I3cff1f21c117a1430834f640ef21fd4e87e06804
Reviewed-on: https://go-review.googlesource.com/36697
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I don't know why it is not working. Filed issue 19111 for this.
Fixes build.
Update #19111.
Change-Id: I76f8d6aafba5951da2f3ad7d10960419cca7dd1f
Reviewed-on: https://go-review.googlesource.com/37092
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It can't work since Plan 9 does not support the runtime poller.
Fixes build.
Change-Id: I9ec33eb66019d9364c6ff6519b61b32e59498559
Reviewed-on: https://go-review.googlesource.com/37091
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This changes the os package to use the runtime poller for file I/O
where possible. When a system call blocks on a pollable descriptor,
the goroutine will be blocked on the poller but the thread will be
released to run other goroutines. When using a non-pollable
descriptor, the os package will continue to use thread-blocking system
calls as before.
For example, on GNU/Linux, the runtime poller uses epoll. epoll does
not support ordinary disk files, so they will continue to use blocking
I/O as before. The poller will be used for pipes.
Since this means that the poller is used for many more programs, this
modifies the runtime to only block waiting for the poller if there is
some goroutine that is waiting on the poller. Otherwise, there is no
point, as the poller will never make any goroutine ready. This
preserves the runtime's current simple deadlock detection.
This seems to crash FreeBSD systems, so it is disabled on FreeBSD.
This is issue 19093.
Using the poller on Windows requires opening the file with
FILE_FLAG_OVERLAPPED. We should only do that if we can remove that
flag if the program calls the Fd method. This is issue 19098.
Update #6817.
Update #7903.
Update #15021.
Update #18507.
Update #19093.
Update #19098.
Change-Id: Ia5197dcefa7c6fbcca97d19a6f8621b2abcbb1fe
Reviewed-on: https://go-review.googlesource.com/36800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
CL 20845 changed Stdin.Stat() so it returns ModeNamedPipe.
But introduced TestStatStdin does not test what Stdin.Stat()
returns when Stdin is console.
This CL adjusts both TestStatStdin and Stdin.Stat
implementations to handle console. Return ModeCharDevice
from Stdin.Stat() when Stdin is console on windows,
just like it does on unix.
Fixes#14853.
Change-Id: I54d73caee2aea45a99618d11600d8e82fe20d0c0
Reviewed-on: https://go-review.googlesource.com/34090
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This has a notable impact on systems with very large passwd files.
Before:
BenchmarkCurrent-12 30000 42546 ns/op
After:
BenchmarkCurrent-12 20000000 77.5 ns/op
Saved in perf dashboard:
https://perf.golang.org/search?q=upload:20170206.1Fixes#11625
Change-Id: Iebc9bf122cc64a4cab24ac06843c7b2bc450ded9
Reviewed-on: https://go-review.googlesource.com/36391
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Before this CL, Go programs in c-archive or c-shared buildmodes
would not handle SIGPIPE. That leads to surprising behaviour where
writes on a closed pipe or socket would raise SIGPIPE and terminate
the program. This CL changes the Go runtime to handle
SIGPIPE regardless of buildmode. In addition, SIGPIPE from non-Go
code is forwarded.
This is a refinement of CL 32796 that fixes the case where a non-default
handler for SIGPIPE is installed by the host C program.
Fixes#17393
Change-Id: Ia41186e52c1ac209d0a594bae9904166ae7df7de
Reviewed-on: https://go-review.googlesource.com/35960
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The working directory is now adjusted to match the typical Go test
working directory in main, as the old trick for adjusting earlier
stopped working with the latest version of LLDB bugs.
That means the small number of places where testdata files are
read before main is called no longer work. This CL adjusts those
reads to happen after main is called. (This has the bonus effect of
not reading some benchmark testdata files in all.bash.)
Fixes compress/bzip2, go/doc, go/parser, os, and time package
tests on the iOS builder.
Change-Id: If60f026aa7848b37511c36ac5e3985469ec25209
Reviewed-on: https://go-review.googlesource.com/35255
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The test for the race detector itself had a race of a sort not
detected by the race detector.
Fixes#18286.
Change-Id: I3265eae275aaa2869a6b6d3e8675b0d88b25831b
Reviewed-on: https://go-review.googlesource.com/34287
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reverts commit d24b57a6a1.
Reason for revert: Further complications arised (issue 18100). We'll try again in Go 1.9.
Change-Id: I5ca93d2643a4be877dd9c2d8df3359718440f02f
Reviewed-on: https://go-review.googlesource.com/33770
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Run-TryBot: Minux Ma <minux@golang.org>
Go 1.5 worked with Unicode console input but not ^Z.
Go 1.6 did not work with Unicode console input but did handle one ^Z case.
Go 1.7 did not work with Unicode console input but did handle one ^Z case.
The intent of this CL is for Go 1.8 to work with Unicode console input
and also handle all ^Z cases.
Here's a simple test program for reading from the console.
It prints a "> " prompt, calls read, prints what it gets, and repeats.
package main
import (
"fmt"
"os"
)
func main() {
p := make([]byte, 100)
fmt.Printf("> ")
for {
n, err := os.Stdin.Read(p)
fmt.Printf("[%d %q %v]\n> ", n, p[:n], err)
}
}
On Unix, typing a ^D produces a break in the input stream.
If the ^D is at the beginning of a line, then the 0 bytes returned
appear as an io.EOF:
$ go run /tmp/x.go
> hello
[6 "hello\n" <nil>]
> hello^D[5 "hello" <nil>]
> ^D[0 "" EOF]
> ^D[0 "" EOF]
> hello^Dworld
[5 "hello" <nil>]
> [6 "world\n" <nil>]
>
On Windows, the EOF character is ^Z, not ^D, and there has
been a long-standing problem that in Go programs, ^Z on Windows
does not behave in the expected way, namely like ^D on Unix.
Instead, the ^Z come through as literal ^Z characters:
C:\>c:\go1.5.4\bin\go run x.go
> ^Z
[3 "\x1a\r\n" <nil>]
> hello^Zworld
[13 "hello\x1aworld\r\n" <nil>]
>
CL 4310 attempted to fix this bug, then known as #6303,
by changing the use of ReadConsole to ReadFile.
This CL was released as part of Go 1.6 and did fix the case
of a ^Z by itself, but not as part of a larger input:
C:\>c:\go1.6.3\bin\go run x.go
> ^Z
[0 "" EOF]
> hello^Zworld
[13 "hello\x1aworld\r\n" <nil>]
>
So the fix was incomplete.
Worse, the fix broke Unicode console input.
ReadFile does not handle Unicode console input correctly.
To handle Unicode correctly, programs must use ReadConsole.
Early versions of Go used ReadFile to read the console,
leading to incorrect Unicode handling, which was filed as #4760
and fixed in CL 7312053, which switched to ReadConsole
and was released as part of Go 1.1 and still worked as of Go 1.5:
C:\>c:\go1.5.4\bin\go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[16 "hello world™\r\n" <nil>]
>
But in Go 1.6:
C:\>c:\go1.6.3\bin\go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[0 "" EOF]
>
That is, changing back to ReadFile in Go 1.6 reintroduced #4760,
which has been refiled as #17097. (We have no automated test
for this because we don't know how to simulate console input
in a test: it appears that one must actually type at a keyboard
to use the real APIs. This CL at least adds a comment warning
not to reintroduce ReadFile again.)
CL 29493 attempted to fix#17097, but it was not a complete fix:
the hello world™ example above still fails, as does Shift-JIS input,
which was filed as #17939.
CL 29493 also broke ^Z handling, which was filed as #17427.
This CL attempts the never before successfully performed trick
of simultaneously fixing Unicode console input and ^Z handling.
It changes the console input to use ReadConsole again,
as in Go 1.5, which seemed to work for all known Unicode input.
Then it adds explicit handling of ^Z in the input stream.
(In the case where standard input is a redirected file, ^Z processing
should not happen, and it does not, because this code path is only
invoked when standard input is the console.)
With this CL:
C:\>go run x.go
> hello
[7 "hello\r\n" <nil>]
> hello world™
[16 "hello world™\r\n" <nil>]
> ^Z
[0 "" EOF]
> [2 "\r\n" <nil>]
> hello^Zworld
[5 "hello" <nil>]
> [0 "" EOF]
> [7 "world\r\n" <nil>]
This almost matches Unix:
$ go run /tmp/x.go
> hello
[6 "hello\n" <nil>]
> hello world™
[15 "hello world™\n" <nil>]
> ^D
[0 "" EOF]
> [1 "\n" <nil>]
> hello^Dworld
[5 "hello" <nil>]
> [6 "world\n" <nil>]
>
The difference is in the handling of hello^Dworld / hello^Zworld.
On Unix, hello^Dworld terminates the read of hello but does not
result in a zero-length read between reading hello and world.
This is dictated by the tty driver, not any special Go code.
On Windows, in this CL, hello^Zworld inserts a zero length read
result between hello and world, which is treated as an interior EOF.
This is implemented by the Go code in this CL, but it matches the
handling of ^Z on the console in other programs:
C:\>copy con x.txt
hello^Zworld
1 file(s) copied.
C:\>type x.txt
hello
C:\>
A natural question is how to test all this. As noted above, we don't
know how to write automated tests using the actual Windows console.
CL 29493 introduced the idea of substituting a different syscall.ReadFile
implementation for testing; this CL continues that idea but substituting
for syscall.ReadConsole instead. To avoid the regression of putting
ReadFile back, this CL adds a comment warning against that.
Fixes#17427.
Fixes#17939.
Change-Id: Ibaabd0ceb2d7af501d44ac66d53f64aba3944142
Reviewed-on: https://go-review.googlesource.com/33451
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Quentin Smith <quentin@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Fixes the os test on the Android builder.
Change-Id: Ibb9db712156a620fcccf515e035475c5e2f535a5
Reviewed-on: https://go-review.googlesource.com/33650
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Before this CL, Go programs in c-archive or c-shared buildmodes
would not handle SIGPIPE. That leads to surprising behaviour where
writes on a closed pipe or socket would raise SIGPIPE and terminate
the program. This CL changes the Go runtime to handle
SIGPIPE regardless of buildmode. In addition, SIGPIPE from non-Go
code is forwarded.
Fixes#17393
Updates #16760
Change-Id: I155e82020a03a5cdc627a147c27da395662c3fe8
Reviewed-on: https://go-review.googlesource.com/32796
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TestReadStdin always fill up buffer provided by ReadFile caller full.
But we do not know if real ReadFile does the same. Add tests where
buffer is only filled with limited data.
Change-Id: I0fc776325c2b1fe60511126c439f4b0560e9d653
Reviewed-on: https://go-review.googlesource.com/33030
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add an explicit WriteString method to closeOnce that acquires the
writers lock. This overrides the one promoted from the
embedded *os.File field. The promoted one naturally does not acquire
the lock, and can therefore race with the Close method.
Fixes#17647.
Change-Id: I3460f2a0d503449481cfb2fd4628b4855ab0ecdf
Reviewed-on: https://go-review.googlesource.com/33298
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Change-Id: I9a42cb55544185ade20b2a4a9de5d39a6cfc6fc6
Reviewed-on: https://go-review.googlesource.com/33172
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Previously, `os.Clearenv()` (by way of `syscall.Clearenv`) would simply
set all environment variables' values to `""` rather than actually
unsetting them causing subsequent `os.LookupEnv` calls to return that
they were still set.
Fixes#17902
Change-Id: I54081b4b98665e9a39f55ea7582c8d40bb8a2a22
Reviewed-on: https://go-review.googlesource.com/33168
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Add tmpdir as a parameter to the closure otherwise the subsequent
modifications to tmpdir causes only the last subdirectory to be
removed.
Additionally, add the missing argument for the t.Fatalf call.
Change-Id: I3df53f9051f7ea40cf3f846d47d9cefe445e9b9d
Reviewed-on: https://go-review.googlesource.com/32892
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
// Executable returns the path name for the executable that started
// the current process. There is no guarantee that the path is still
// pointing to the correct executable. If a symlink was used to start
// the process, depending on the operating system, the result might
// be the symlink or the path it pointed to. If a stable result is
// needed, path/filepath.EvalSymlinks might help.
//
// Executable returns an absolute path unless an error occurred.
//
// The main use case is finding resources located relative to an
// executable.
//
// Executable is not supported on nacl or OpenBSD (unless procfs is
// mounted.)
func Executable() (string, error) {
return executable()
}
Fixes#12773.
Change-Id: I469738d905b12f0b633ea4d88954f8859227a88c
Reviewed-on: https://go-review.googlesource.com/16551
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Windows has a limit of 260 characters on normal paths, but it's possible
to use longer paths by using "extended-length paths" that begin with
`\\?\`. This commit attempts to transparently convert an absolute path
to an extended-length path, following the subtly different rules those
paths require. It does not attempt to handle relative paths, which
continue to be passed to the operating system unmodified.
This adds a new test, TestLongPath, to the os package. This test makes
sure that it is possible to write a path at least 400 characters long
and runs on every platform. It also tests symlinks and hardlinks, though
symlinks are not testable with our builder configuration.
HasLink is moved to internal/testenv so it can be used by multiple tests.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
has Microsoft's documentation on extended-length paths.
Fixes#3358.
Fixes#10577.
Fixes#17500.
Change-Id: I4ff6bb2ef9c9a4468d383d98379f65cf9c448218
Reviewed-on: https://go-review.googlesource.com/32451
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
TestRemoveDevNull was added in CL 31657. However, this test
was failing on Plan 9, because /dev/null was considered as
a regular file.
On Plan 9, there is no special mode to distinguish between
device files and regular files.
However, files are served by different servers. For example,
/dev/null is served by #c (devcons), while /bin/cat is served
by #M (devmnt).
We chose to consider only the files served by #M as regular
files. All files served by different servers will be considered
as device files.
Fixes#17598.
Change-Id: Ibb1c3357d742cf2a7de15fc78c9e436dc31982bb
Reviewed-on: https://go-review.googlesource.com/32152
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Current implementation of syscall.Readlink mistakenly calculates
the end offset of the PrintName field.
Also, there are some cases that the PrintName field is empty.
Instead, the CL uses SubstituteName with correct calculation.
Fixes#15978Fixes#16145
Change-Id: If3257137141129ac1c552d003726d5b9c08bb754
Reviewed-on: https://go-review.googlesource.com/31118
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Unix rejects this when new is a non-empty directory.
Other systems reject this when new is a directory, empty or not.
Make Unix reject empty directory too.
Fixes#14527.
Change-Id: Ice24b8065264c91c22cba24aa73e142386c29c87
Reviewed-on: https://go-review.googlesource.com/31358
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
cmd.StdinPipe returns an io.WriteCloser.
It's reasonable to expect the caller not to call Write and Close simultaneously,
but there is an implicit Close in cmd.Wait that's not obvious.
We already synchronize the implicit Close in cmd.Wait against
any explicit Close from the caller. Also synchronize that implicit
Close against any explicit Write from the caller.
Fixes#9307.
Change-Id: I8561e9369d6e5ac88dfbca1175549f6dfa04b8ac
Reviewed-on: https://go-review.googlesource.com/31148
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL introduces first test for readConsole. And new test
discovered couple of problems with readConsole.
Console characters consist of multiple bytes each, but byte blocks
returned by syscall.ReadFile have no character boundaries. Some
multi-byte characters might start at the end of one block, and end
at the start of next block. readConsole feeds these blocks to
syscall.MultiByteToWideChar to convert them into utf16, but if some
multi-byte characters have no ending or starting bytes, the
syscall.MultiByteToWideChar might get confused. Current version of
syscall.MultiByteToWideChar call will make
syscall.MultiByteToWideChar ignore all these not complete
multi-byte characters.
The CL solves this issue by changing processing from "randomly
sized block of bytes at a time" to "one multi-byte character at a
time". New readConsole code calls syscall.ReadFile to get 1 byte
first. Then it feeds this byte to syscall.MultiByteToWideChar.
The new syscall.MultiByteToWideChar call uses MB_ERR_INVALID_CHARS
flag to make syscall.MultiByteToWideChar return error if input is
not complete character. If syscall.MultiByteToWideChar returns
correspondent error, we read another byte and pass 2 byte buffer
into syscall.MultiByteToWideChar, and so on until success.
Old readConsole code would also sometimes return no data if user
buffer was smaller then uint16 size, which would confuse callers
that supply 1 byte buffer. This CL fixes that problem too.
Fixes#17097
Change-Id: I88136cdf6a7bf3aed5fbb9ad2c759b6c0304ce30
Reviewed-on: https://go-review.googlesource.com/29493
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
It is possible (and common) for Windows systems to use a different codepage
for console applications from that used on normal windowed application
(called ANSI codepage); for instance, most of the western Europe uses
CP850 for console (for backward compatibility with MS-DOS), while
windowed applications use a different codepage depending on the country
(eg: CP1252 aka Latin-1). The usage being changed with this commit is
specifically related to decoding input coming from the console, so the
previous usage of the ANSI codepage was wrong.
Also fixes an issue that previous did convert bytes as NFD. Go is
designed to handle single Unicode code point. This fix change behaivor
to NFC.
Fixes#16857.
Change-Id: I4f41ae83ece47321b6e9a79a2087ecbb8ac066dd
Reviewed-on: https://go-review.googlesource.com/27575
Reviewed-by: Hiroshi Ioka <hirochachacha@gmail.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
