qbit/go
1
0
Fork 0
mirror of https://github.com/golang/go synced 2024-11-14 09:20:28 -07:00
Code Releases Activity
33,900 Commits 58 Branches 402 Tags 522 MiB
93e97ef066
Commit Graph

5 Commits

Author SHA1 Message Date
Alex Brainman
39c8d2b7fa os: parse command line without shell32.dll 
Go uses CommandLineToArgV from shell32.dll to parse command
line parameters. But shell32.dll is slow to load. Implement
Windows command line parsing in Go. This should make starting
Go programs faster.

I can see these speed ups for runtime.BenchmarkRunningGoProgram

on my Windows 7 amd64:
name                old time/op  new time/op  delta
RunningGoProgram-2  11.2ms ± 1%  10.4ms ± 2%  -6.63%  (p=0.000 n=9+10)

on my Windows XP 386:
name                old time/op  new time/op  delta
RunningGoProgram-2  19.0ms ± 3%  12.1ms ± 1%  -36.20%  (p=0.000 n=10+10)

on @egonelbre Windows 10 amd64:
name                old time/op  new time/op  delta
RunningGoProgram-8  17.0ms ± 1%  15.3ms ± 2%  -9.71%  (p=0.000 n=10+10)

This CL is based on CL 22932 by John Starks.

Fixes #15588.

Change-Id: Ib14be0206544d0d4492ca1f0d91fac968be52241
Reviewed-on: https://go-review.googlesource.com/37915
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
 2017-03-24 00:53:03 +00:00
Ian Lance Taylor
c05b06a12d os: use poller for file I/O 
This changes the os package to use the runtime poller for file I/O
where possible. When a system call blocks on a pollable descriptor,
the goroutine will be blocked on the poller but the thread will be
released to run other goroutines. When using a non-pollable
descriptor, the os package will continue to use thread-blocking system
calls as before.

For example, on GNU/Linux, the runtime poller uses epoll. epoll does
not support ordinary disk files, so they will continue to use blocking
I/O as before. The poller will be used for pipes.

Since this means that the poller is used for many more programs, this
modifies the runtime to only block waiting for the poller if there is
some goroutine that is waiting on the poller. Otherwise, there is no
point, as the poller will never make any goroutine ready. This
preserves the runtime's current simple deadlock detection.

This seems to crash FreeBSD systems, so it is disabled on FreeBSD.
This is issue 19093.

Using the poller on Windows requires opening the file with
FILE_FLAG_OVERLAPPED. We should only do that if we can remove that
flag if the program calls the Fd method. This is issue 19098.

Update #6817.
Update #7903.
Update #15021.
Update #18507.
Update #19093.
Update #19098.

Change-Id: Ia5197dcefa7c6fbcca97d19a6f8621b2abcbb1fe
Reviewed-on: https://go-review.googlesource.com/36800
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
 2017-02-15 19:31:55 +00:00
Russ Cox
610d522189 os: fix handling of Windows Unicode console input and ^Z 
Go 1.5 worked with Unicode console input but not ^Z.
Go 1.6 did not work with Unicode console input but did handle one ^Z case.
Go 1.7 did not work with Unicode console input but did handle one ^Z case.

The intent of this CL is for Go 1.8 to work with Unicode console input
and also handle all ^Z cases.

Here's a simple test program for reading from the console.
It prints a "> " prompt, calls read, prints what it gets, and repeats.

	package main

	import (
	    "fmt"
	    "os"
	)

	func main() {
	    p := make([]byte, 100)
	    fmt.Printf("> ")
	    for {
	        n, err := os.Stdin.Read(p)
	        fmt.Printf("[%d %q %v]\n> ", n, p[:n], err)
	    }
	}

On Unix, typing a ^D produces a break in the input stream.
If the ^D is at the beginning of a line, then the 0 bytes returned
appear as an io.EOF:

	$ go run /tmp/x.go
	> hello
	[6 "hello\n" <nil>]
	> hello^D[5 "hello" <nil>]
	> ^D[0 "" EOF]
	> ^D[0 "" EOF]
	> hello^Dworld
	[5 "hello" <nil>]
	> [6 "world\n" <nil>]
	>

On Windows, the EOF character is ^Z, not ^D, and there has
been a long-standing problem that in Go programs, ^Z on Windows
does not behave in the expected way, namely like ^D on Unix.
Instead, the ^Z come through as literal ^Z characters:

	C:\>c:\go1.5.4\bin\go run x.go
	> ^Z
	[3 "\x1a\r\n" <nil>]
	> hello^Zworld
	[13 "hello\x1aworld\r\n" <nil>]
	>

CL 4310 attempted to fix this bug, then known as #6303,
by changing the use of ReadConsole to ReadFile.
This CL was released as part of Go 1.6 and did fix the case
of a ^Z by itself, but not as part of a larger input:

	C:\>c:\go1.6.3\bin\go run x.go
	> ^Z
	[0 "" EOF]
	> hello^Zworld
	[13 "hello\x1aworld\r\n" <nil>]
	>

So the fix was incomplete.
Worse, the fix broke Unicode console input.

ReadFile does not handle Unicode console input correctly.
To handle Unicode correctly, programs must use ReadConsole.
Early versions of Go used ReadFile to read the console,
leading to incorrect Unicode handling, which was filed as #4760
and fixed in CL 7312053, which switched to ReadConsole
and was released as part of Go 1.1 and still worked as of Go 1.5:

	C:\>c:\go1.5.4\bin\go run x.go
	> hello
	[7 "hello\r\n" <nil>]
	> hello world™
	[16 "hello world™\r\n" <nil>]
	>

But in Go 1.6:

	C:\>c:\go1.6.3\bin\go run x.go
	> hello
	[7 "hello\r\n" <nil>]
	> hello world™
	[0 "" EOF]
	>

That is, changing back to ReadFile in Go 1.6 reintroduced #4760,
which has been refiled as #17097. (We have no automated test
for this because we don't know how to simulate console input
in a test: it appears that one must actually type at a keyboard
to use the real APIs. This CL at least adds a comment warning
not to reintroduce ReadFile again.)

CL 29493 attempted to fix #17097, but it was not a complete fix:
the hello world™ example above still fails, as does Shift-JIS input,
which was filed as #17939.

CL 29493 also broke ^Z handling, which was filed as #17427.

This CL attempts the never before successfully performed trick
of simultaneously fixing Unicode console input and ^Z handling.
It changes the console input to use ReadConsole again,
as in Go 1.5, which seemed to work for all known Unicode input.
Then it adds explicit handling of ^Z in the input stream.
(In the case where standard input is a redirected file, ^Z processing
should not happen, and it does not, because this code path is only
invoked when standard input is the console.)

With this CL:

	C:\>go run x.go
	> hello
	[7 "hello\r\n" <nil>]
	> hello world™
	[16 "hello world™\r\n" <nil>]
	> ^Z
	[0 "" EOF]
	> [2 "\r\n" <nil>]
	> hello^Zworld
	[5 "hello" <nil>]
	> [0 "" EOF]
	> [7 "world\r\n" <nil>]

This almost matches Unix:

	$ go run /tmp/x.go
	> hello
	[6 "hello\n" <nil>]
	> hello world™
	[15 "hello world™\n" <nil>]
	> ^D
	[0 "" EOF]
	> [1 "\n" <nil>]
	> hello^Dworld
	[5 "hello" <nil>]
	> [6 "world\n" <nil>]
	>

The difference is in the handling of hello^Dworld / hello^Zworld.
On Unix, hello^Dworld terminates the read of hello but does not
result in a zero-length read between reading hello and world.
This is dictated by the tty driver, not any special Go code.

On Windows, in this CL, hello^Zworld inserts a zero length read
result between hello and world, which is treated as an interior EOF.
This is implemented by the Go code in this CL, but it matches the
handling of ^Z on the console in other programs:

	C:\>copy con x.txt
	hello^Zworld
	        1 file(s) copied.

	C:\>type x.txt
	hello
	C:\>

A natural question is how to test all this. As noted above, we don't
know how to write automated tests using the actual Windows console.
CL 29493 introduced the idea of substituting a different syscall.ReadFile
implementation for testing; this CL continues that idea but substituting
for syscall.ReadConsole instead. To avoid the regression of putting
ReadFile back, this CL adds a comment warning against that.

Fixes #17427.
Fixes #17939.

Change-Id: Ibaabd0ceb2d7af501d44ac66d53f64aba3944142
Reviewed-on: https://go-review.googlesource.com/33451
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Quentin Smith <quentin@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
 2016-11-29 02:13:03 +00:00
Quentin Smith
231aa9d6d7 os: use extended-length paths on Windows when possible 
Windows has a limit of 260 characters on normal paths, but it's possible
to use longer paths by using "extended-length paths" that begin with
`\\?\`. This commit attempts to transparently convert an absolute path
to an extended-length path, following the subtly different rules those
paths require. It does not attempt to handle relative paths, which
continue to be passed to the operating system unmodified.

This adds a new test, TestLongPath, to the os package. This test makes
sure that it is possible to write a path at least 400 characters long
and runs on every platform. It also tests symlinks and hardlinks, though
symlinks are not testable with our builder configuration.

HasLink is moved to internal/testenv so it can be used by multiple tests.

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365247(v=vs.85).aspx
has Microsoft's documentation on extended-length paths.

Fixes #3358.
Fixes #10577.
Fixes #17500.

Change-Id: I4ff6bb2ef9c9a4468d383d98379f65cf9c448218
Reviewed-on: https://go-review.googlesource.com/32451
Run-TryBot: Quentin Smith <quentin@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
 2016-11-07 20:31:02 +00:00
Alex Brainman
1af769da82 os: make readConsole handle its input and output correctly 
This CL introduces first test for readConsole. And new test
discovered couple of problems with readConsole.

Console characters consist of multiple bytes each, but byte blocks
returned by syscall.ReadFile have no character boundaries. Some
multi-byte characters might start at the end of one block, and end
at the start of next block. readConsole feeds these blocks to
syscall.MultiByteToWideChar to convert them into utf16, but if some
multi-byte characters have no ending or starting bytes, the
syscall.MultiByteToWideChar might get confused. Current version of
syscall.MultiByteToWideChar call will make
syscall.MultiByteToWideChar ignore all these not complete
multi-byte characters.

The CL solves this issue by changing processing from "randomly
sized block of bytes at a time" to "one multi-byte character at a
time". New readConsole code calls syscall.ReadFile to get 1 byte
first. Then it feeds this byte to syscall.MultiByteToWideChar.
The new syscall.MultiByteToWideChar call uses MB_ERR_INVALID_CHARS
flag to make syscall.MultiByteToWideChar return error if input is
not complete character. If syscall.MultiByteToWideChar returns
correspondent error, we read another byte and pass 2 byte buffer
into syscall.MultiByteToWideChar, and so on until success.

Old readConsole code would also sometimes return no data if user
buffer was smaller then uint16 size, which would confuse callers
that supply 1 byte buffer. This CL fixes that problem too.

Fixes #17097

Change-Id: I88136cdf6a7bf3aed5fbb9ad2c759b6c0304ce30
Reviewed-on: https://go-review.googlesource.com/29493
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
 2016-10-13 06:16:53 +00:00