The previous code acquired a read lock on src and a write lock on
dst for the entire duration of Splice. This resulted in deadlock,
in a situation akin to the following:
Splice is blocking, waiting to read from src.
The caller tries to close dst from another goroutine, but Close on
dst blocks in runtime.semacquire, because Splice is still holding a
write lock on it, and the Close didn't unblock any I/O.
The caller cannot unblock the read side of Splice through other means,
because they are stuck waiting in dst.Close().
Use more fine-grained locking instead: acquire the read lock on src
just before trying to splice from the source socket to the pipe,
and acquire the write lock on dst just before trying to splice from
the pipe to the destination socket.
Fixes#25985
Change-Id: I264c91c7a69bb6c5e28610e2bd801244804cf86d
Reviewed-on: https://go-review.googlesource.com/120317
Run-TryBot: Aram Hăvărneanu <aram@mgk.ro>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Instead of "inconsistent poll.fdMutex", panic with
"too many concurrent operations on a single file or socket (max 1048575)".
Fixes#25558
Change-Id: I5cad3633aa539fb6f48cca236c6656c86acfb663
Reviewed-on: https://go-review.googlesource.com/119956
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Dmitry Vyukov <dvyukov@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitry Vyukov <dvyukov@google.com>
This commit enables vet/all for the js/wasm architecture. It got
skipped initially because the codebase did not fully compile yet
for js/wasm, which made vet/all fail.
startTimer and stopTimer are not needed in the syscall package.
Removed their assembly code since their Go stubs were already gone.
Change-Id: Icaeb6d903876e51ceb1edff7631f715a98c28696
Reviewed-on: https://go-review.googlesource.com/118657
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Current SendFile implementation assumes that TransmitFile starts from
the current file position. But that appears not true for Windows 10
Version 1803.
TransmitFile documentation
https://msdn.microsoft.com/en-us/library/windows/desktop/ms740565(v=vs.85).aspx
suggests, "You can use the lpOverlapped parameter to specify a 64-bit
offset within the file at which to start the file data transfer by
setting the Offset and OffsetHigh member of the OVERLAPPED structure."
Do as it advises.
Fixes#25722
Change-Id: I241d3bf76d0d5590d4df27c6f922d637068232fb
Reviewed-on: https://go-review.googlesource.com/117816
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Hardware AES support in Go on s390x currently requires ECB, CBC
and CTR modes be available. It also requires that either the
GHASH or GCM facilities are available. The existing checks missed
some of these constraints.
While we're here simplify the cpu package on s390x, moving masking
code out of assembly and into Go code. Also, update SHA-{1,256,512}
implementations to use the cpu package since that is now trivial.
Finally I also added a test for internal/cpu on s390x which loads
/proc/cpuinfo and checks it against the flags set by internal/cpu.
Updates #25822 for changes to vet whitelist.
Change-Id: Iac4183f571643209e027f730989c60a811c928eb
Reviewed-on: https://go-review.googlesource.com/114397
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit adds the js/wasm architecture to the net package.
The net package is not supported by js/wasm, but a simple fake
networking is available so tests of other packages that require
basic TCP sockets can pass. The tests of the net package itself
are mostly disabled.
Updates #18892
Change-Id: Id287200c39f0a3e23d20ef17260ca15ccdcca032
Reviewed-on: https://go-review.googlesource.com/109995
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Each URL was manually verified to ensure it did not serve up incorrect
content.
Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
This commit adds the js/wasm architecture to the os package.
Access to the actual file system is supported through Node.js.
Updates #18892
Change-Id: I6fa642fb294ca020b2c545649d4324d981aa0408
Reviewed-on: https://go-review.googlesource.com/109977
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
I don't know why these files were not formatted. Perhaps because
their changes came from Github PRs?
Change-Id: Ida8d7b9a36f0d1064caf74ca1911696a247a9bbe
Reviewed-on: https://go-review.googlesource.com/114824
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reverts commit 1764609b8b.
Reason for revert: It breaks the dragonfly trybot, in which there are sometimes (non-deterministically) Events with the same timestamp that have to occur in a specific order.
Change-Id: I714e640c6ab5ccb23d5577e8aa98c7716ede7ad2
Reviewed-on: https://go-review.googlesource.com/114356
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When the internal/cpu package was introduced, the AES package still used
the custom crypto/internal/cipherhw package for amd64 and s390x. This
change removes that package entirely in favor of directly referencing the
cpu feature flags set and exposed by the internal/cpu package. In
addition, 5 new flags have been added to the internal/cpu s390x struct
for detecting various cipher message (KM) features.
Change-Id: I77cdd8bc1b04ab0e483b21bf1879b5801a4ba5f4
GitHub-Last-Rev: a611e3ecb1
GitHub-Pull-Request: golang/go#24766
Reviewed-on: https://go-review.googlesource.com/105695
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The existing code just used timestamps. The new code uses more fields
when timestamps are equal.
Revised to shorten code per reviewer comments.
Change-Id: Ibd0824d0acd7644484d536b1a754a0da156fac3d
Reviewed-on: https://go-review.googlesource.com/113721
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
The splice syscall is buggy prior to linux 2.6.29. Instead of returning
0 when reading a closed socket, it returns EAGAIN. While it is possible
to detect this (HAProxy falls back to recv), it is simpiler to avoid
using splice all together. the "fcntl(fd, F_GETPIPE_SZ)" syscall is used
detect buggy versions of splice as the syscall returns EINVAL on
versions prior to 2.6.35.
Fixes#25486
Change-Id: I860c029f13de2b09e95a7ba39b76ac7fca91a195
Reviewed-on: https://go-review.googlesource.com/113999
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Needs the go compiler to be build with GOEXPERIMENT=debugcpu to be active.
The GODEBUGCPU environment variable can be used to disable usage of
specific processor features in the Go standard library.
This is useful for testing and benchmarking different code paths that
are guarded by internal/cpu variable checks.
Use of processor features can not be enabled through GODEBUGCPU.
To disable usage of AVX and SSE41 cpu features on GOARCH amd64 use:
GODEBUGCPU=avx=0,sse41=0
The special "all" option can be used to disable all options:
GODEBUGCPU=all=0
Updates #12805
Updates #15403
Change-Id: I699c2e6f74d98472b6fb4b1e5ffbf29b15697aab
Reviewed-on: https://go-review.googlesource.com/91737
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This commit adds the wasm architecture to the internal/bytealg package.
Some parts of the assembly code have been extracted from WebAssembly
bytecode generated with Emscripten (which uses musl libc).
Updates #18892
Change-Id: Iba7f7158356b816c9ad03ca9223903a41a024da6
Reviewed-on: https://go-review.googlesource.com/103915
Reviewed-by: Keith Randall <khr@golang.org>
Go runtime currently only populates hwcap for ppc64 and arm64.
While the interpretation of hwcap is platform specific the hwcap
information is generally available on linux.
Changing the runtime variable name to cpu_hwcap for cpu.hwcap makes it
consistent with the general naming of runtime variables that are linked
to other packages.
Change-Id: I1e1f932a73ed624a219b9298faafbb6355e47ada
Reviewed-on: https://go-review.googlesource.com/94757
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The general policy for the current state of js/wasm is that it only
has to support tests that are also supported by nacl.
The test nilptr3.go makes assumptions about which nil checks can be
removed. Since WebAssembly does not signal on reading a null pointer,
all nil checks have to be explicit.
Updates #18892
Change-Id: I06a687860b8d22ae26b1c391499c0f5183e4c485
Reviewed-on: https://go-review.googlesource.com/110096
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This commit only moves code in preparation for the following commit
which adds the js/wasm architecture to the os package. There are no
semantic changes in this commit.
Updates #18892
Change-Id: Ia44484216f905c25395c565c34cfe6996c305ed6
Reviewed-on: https://go-review.googlesource.com/109976
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The file name suffix arm64 already limits the file to be build only on arm64.
Change-Id: I33db713041b6dec9eb00889bac3b54c727e90743
Reviewed-on: https://go-review.googlesource.com/108986
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
"Span" is a commonly used term in many distributed tracing systems
(Dapper, OpenCensus, OpenTracing, ...). They use it to refer to a
period of time, not necessarily tied into execution of underlying
processor, thread, or goroutine, unlike the "Span" of runtime/trace
package.
Since distributed tracing and go runtime execution tracing are
already similar enough to cause confusion, this CL attempts to avoid
using the same word if possible.
"Region" is being used in a certain tracing system to refer to a code
region which is pretty close to what runtime/trace.Span currently
refers to. So, replace that.
https://software.intel.com/en-us/itc-user-and-reference-guide-defining-and-recording-functions-or-regions
This CL also tweaks APIs a bit based on jbd and heschi's comments:
NewContext -> NewTask
and it now returns a Task object that exports End method.
StartSpan -> StartRegion
and it now returns a Region object that exports End method.
Also, changed WithSpan to WithRegion and it now takes func() with no
context. Another thought is to get rid of WithRegion. It is a nice
concept but in practice, it seems problematic (a lot of code churn,
and polluting stack trace). Already, the tracing concept is very low
level, and we hope this API to be used with great care.
Recommended usage will be
defer trace.StartRegion(ctx, "someRegion").End()
Left old APIs untouched in this CL. Once the usage of them are cleaned
up, they will be removed in a separate CL.
Change-Id: I73880635e437f3aad51314331a035dd1459b9f3a
Reviewed-on: https://go-review.googlesource.com/108296
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JBD <jbd@google.com>
This change adds support for the splice system call on Linux,
for the purpose of optimizing (*TCPConn).ReadFrom by reducing
copies of data from and to userspace. It does so by creating a
temporary pipe and splicing data from the source connection to the
pipe, then from the pipe to the destination connection. The pipe
serves as an in-kernel buffer for the data transfer.
No new API is added to package net, but a new Splice function is
added to package internal/poll, because using splice requires help
from the network poller. Users of the net package should benefit
from the change transparently.
This change only enables the optimization if the Reader in ReadFrom
is a TCP connection. Since splice is a more general interface, it
could, in theory, also be enabled if the Reader were a unix socket,
or the read half of a pipe.
However, benchmarks show that enabling it for unix sockets is most
likely not a net performance gain. The tcp <- unix case is also
fairly unlikely to be used very much by users of package net.
Enabling the optimization for pipes is also problematic from an
implementation perspective, since package net cannot easily get at
the *poll.FD of an *os.File. A possible solution to this would be
to dup the pipe file descriptor, register the duped descriptor with
the network poller, and work on that *poll.FD instead of the original.
However, this seems too intrusive, so it has not been done. If there
was a clean way to do it, it would probably be worth doing, since
splicing from a pipe to a socket can be done directly.
Therefore, this patch only enables the optimization for what is likely
the most common use case: tcp <- tcp.
The following benchmark compares the performance of the previous
userspace genericReadFrom code path to the new optimized code path.
The sub-benchmarks represent chunk sizes used by the writer on the
other end of the Reader passed to ReadFrom.
benchmark old ns/op new ns/op delta
BenchmarkTCPReadFrom/1024-4 4727 4954 +4.80%
BenchmarkTCPReadFrom/2048-4 4389 4301 -2.01%
BenchmarkTCPReadFrom/4096-4 4606 4534 -1.56%
BenchmarkTCPReadFrom/8192-4 5219 4779 -8.43%
BenchmarkTCPReadFrom/16384-4 8708 8008 -8.04%
BenchmarkTCPReadFrom/32768-4 16349 14973 -8.42%
BenchmarkTCPReadFrom/65536-4 35246 27406 -22.24%
BenchmarkTCPReadFrom/131072-4 72920 52382 -28.17%
BenchmarkTCPReadFrom/262144-4 149311 95094 -36.31%
BenchmarkTCPReadFrom/524288-4 306704 181856 -40.71%
BenchmarkTCPReadFrom/1048576-4 674174 357406 -46.99%
benchmark old MB/s new MB/s speedup
BenchmarkTCPReadFrom/1024-4 216.62 206.69 0.95x
BenchmarkTCPReadFrom/2048-4 466.61 476.08 1.02x
BenchmarkTCPReadFrom/4096-4 889.09 903.31 1.02x
BenchmarkTCPReadFrom/8192-4 1569.40 1714.06 1.09x
BenchmarkTCPReadFrom/16384-4 1881.42 2045.84 1.09x
BenchmarkTCPReadFrom/32768-4 2004.18 2188.41 1.09x
BenchmarkTCPReadFrom/65536-4 1859.38 2391.25 1.29x
BenchmarkTCPReadFrom/131072-4 1797.46 2502.21 1.39x
BenchmarkTCPReadFrom/262144-4 1755.69 2756.68 1.57x
BenchmarkTCPReadFrom/524288-4 1709.42 2882.98 1.69x
BenchmarkTCPReadFrom/1048576-4 1555.35 2933.84 1.89x
Fixes#10948
Change-Id: I3ce27f21f7adda8b696afdc48a91149998ae16a5
Reviewed-on: https://go-review.googlesource.com/107715
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I6e1fa67dc9d4d151c90eb19a6f736e4daa7d4fb3
Reviewed-on: https://go-review.googlesource.com/107615
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
CL 99337 introduced a file with CRLF line endings. Convert them to LF
line endings as in all other Go files.
Change-Id: I68b28fd443f05bebbbd9280d1821c4ccd33a4a8a
Reviewed-on: https://go-review.googlesource.com/108075
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
RawRead assumes the callback will perform either (a) a blocking read
and always return true, (b) a blocking read with a SO_RCVTIMEO set
returning false on WSAETIMEDOUT, or (c) a non-blocking read
returning false on WSAEWOULDBLOCK. In the latter two cases, it uses
a 0-byte overlapped read for notifications from the IOCP runtime
when the socket becomes readable before trying again.
RawWrite assumes the callback will perform blocking write and will
always return true, and makes no effort to tie into the runtime loop.
Change-Id: Ib10074e9d502c040294f41a260e561e84208652f
Reviewed-on: https://go-review.googlesource.com/76391
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
CL 92475 added WSAEMSGSIZE const to syscall package. But there
is already copy of WSAEMSGSIZE in internal/syscall/windows.
So delete syscall.WSAEMSGSIZE
Change-Id: I0b81fa5dcf846887a0cb27d8bbd7e250860627b5
Reviewed-on: https://go-review.googlesource.com/106956
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If NewFile is called with a file descriptor that is already set to
non-blocking mode, it tries to return a pollable file (one for which
SetDeadline methods work) by adding the filedes to the poll/netpoll
mechanism. If called with a filedes in blocking mode, it returns a
non-pollable file, as it always did.
Fixes#22939
Updates #24331
Change-Id: Id54c8be1b83e6d35e14e76d7df0e57a9fd64e176
Reviewed-on: https://go-review.googlesource.com/100077
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Use constant masks and align the definition of isSet with
arm64 and ppc64x.
Change-Id: I0c6eae30da5e3ce797cde0dab4a39855d4d245d9
Reviewed-on: https://go-review.googlesource.com/94759
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
runtime.alginit needs runtime/support_{aes,ssse3,sse41} feature flag
to init aeshash function but internal/cpu.init not be called yet.
This CL will call internal/cpu.initialize before runtime.alginit, so
that we can move all cpu features related code to internal/cpu.
Change-Id: I00b8e403ace3553f8c707563d95f27dade0bc853
Reviewed-on: https://go-review.googlesource.com/104636
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add the following helpers in lookup_windows.go:
1) lookupGroupName() is used to obtain the SID of a group based
on name.
2) listGroupsForUsernameAndDomain() uses NetUserGetLocalGroups()
as a WINAPI backend to obtain the list of local groups for this
user.
3) lookupUserPrimaryGroup() is now used to populate the User.Gid
field when looking up a user by name.
Implement listGroups(), lookupGroupId(), lookupGroup() and no longer
return unimplemented errors.
Do not skip Windows User.Gid tests in user_test.go.
Change-Id: I81fd41b406da51f9a4cb24e50d392a333df81141
GitHub-Last-Rev: d1448fd55d
GitHub-Pull-Request: golang/go#24222
Reviewed-on: https://go-review.googlesource.com/98137
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When snapshotting the execution time stats of a goroutine
we take into account the intermediate values kepts in the GDesc's
gdesc field. At the end of goroutine analysis, we go through all
goroutines and replace the GExecutionStat with the new snapshot.
Here the mistake was that we replaced the GExecutionStat with
the value that reflects the intermediate values, but did clear
the intermediate values. So, when the following finalizeActiveSpans
runs, it trieds to add the intermediate values again (double-counting)
when taking a snapshot.
Fix the issue by using the finalized GExecutionStat instead of
recomputing the snapshot. While we are here, refactor the finalization
logic so it can be used when processing GoEnd, GoStop events.
Change-Id: Ibdb342214c29b65d4ea37e52b1f8b83f1fb20143
Reviewed-on: https://go-review.googlesource.com/103157
Reviewed-by: Peter Weinberger <pjw@google.com>
Move part of UserSpan event processing from cmd/trace.analyzeAnnotations
to internal/trace.GoroutineStats that returns analyzed per-goroutine
execution information. Now the execution information includes list of
spans and their execution information.
cmd/trace.analyzeAnnotations utilizes the span execution information
from internal/trace.GoroutineStats and connects them with task
information.
Change-Id: Ib7f79a3ba652a4ae55cd81ea17565bcc7e241c5c
Reviewed-on: https://go-review.googlesource.com/101917
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Reviewed-by: Peter Weinberger <pjw@google.com>
As found by unparam. Picked the low-hanging fruit, consisting only of
errors that were always nil and results that were never used. Left out
those that were useful for consistency with other func signatures.
Change-Id: I06b52bbd3541f8a5d66659c909bd93cb3e172018
Reviewed-on: https://go-review.googlesource.com/102418
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reverts commit bfa8b6f8ff.
Reason for revert: This depends on another CL which is not yet submitted.
Change-Id: I50e7594f1473c911a2079fe910849a6694ac6c07
Reviewed-on: https://go-review.googlesource.com/101496
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Goroutine analysis reports the sum of all overlapping GC intervals as
the GCTime of a goroutine. The computation is done by adding the length
of a completed GC interval to 'active' goroutines when processing the
corresponding EvGCDone event. This change fixes the two corner cases
the current implementation ignores:
1) Goroutine that ends during GC. Previously, this goroutine was ignored
and GC time was undercounted. We handle this case by setting the
gcStartTime only when GC is active and handling non-zero gcStartTime
when processing EvGoStop and EvGoStart.
2) Goroutine that starts during GC. Previously, the entire GC interval
length was added to the Goroutine's GCTime which resulted in overcount
of GC time. We handle this case by computing the length of overlapped
period precisely.
Change-Id: Ifa8e82672ec341b5ff87837209f4311fa7262b7f
Reviewed-on: https://go-review.googlesource.com/100842
Reviewed-by: Heschi Kreinick <heschi@google.com>
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is an updated version of golang.org/cl/96395, with the fix to
TestUserSpan.
This reverts commit 7b6f6267e90a8e4eab37a3f2164ba882e6222adb.
Change-Id: I31eec8ba0997f9178dffef8dac608e731ab70872
Reviewed-on: https://go-review.googlesource.com/98236
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
Missed removing the argument loading from the indexbody function.
Change-Id: Ia1391231fc99771d00410a09fe80a09f08ceed02
Reviewed-on: https://go-review.googlesource.com/98575
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Make ReadSubKeyNames work even if key is opened with only
ENUMERATE_SUB_KEYs access rights mask.
Fixes#23869
Change-Id: I138bd51715fdbc3bda05607c64bde1150f4fe6b2
Reviewed-on: https://go-review.googlesource.com/97435
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Move the IndexByte function from the runtime to a new bytealg package.
The new package will eventually hold all the optimized assembly for
groveling through byte slices and strings. It seems a better home for
this code than randomly keeping it in runtime.
Once this is in, the next step is to move the other functions
(Compare, Equal, ...).
Update #19792
This change seems complicated enough that we might just declare
"not worth it" and abandon. Opinions welcome.
The core assembly is all unchanged, except minor modifications where
the code reads cpu feature bits.
The wrapper functions have been cleaned up as they are now actually
checked by vet.
Change-Id: I9fa75bee5d85db3a65b3fd3b7997e60367523796
Reviewed-on: https://go-review.googlesource.com/98016
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Even though undocumented, the assumption is the Event's link field
points to the following event in the future. The new span/task event
processing breaks the assumption.
Change-Id: I4ce2f30c67c4f525ec0a121a7e43d8bdd2ec3f77
Reviewed-on: https://go-review.googlesource.com/96395
Reviewed-by: Heschi Kreinick <heschi@google.com>
The SOCK_CLOEXEC and SOCK_NONBLOCK flags to the socket syscall and the
accept4 syscall are supported since OpenBSD 5.7.
Follows CL 40895 and CL 94295
Change-Id: Icaf35ace2ef5e73279a70d4f1a9fbf3be9371e6c
Reviewed-on: https://go-review.googlesource.com/97196
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
newUserFromSid() is extended so that the retriaval of the user home
path based on a user SID becomes possible.
(1) The primary method it uses is to lookup the Windows registry for
the following key:
HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\ProfileList\[SID]
If the key does not exist the user might not have logged in yet.
If (1) fails it falls back to (2)
(2) The second method the function uses is to look at the default home
path for users (e.g. WINAPI's GetProfilesDirectory()) and append
the username to that. The procedure is in the lines of:
c:\Users + \ + <username>
The function newUser() now requires the following arguments:
uid, gid, dir, username, domain
This is done to avoid multiple calls to usid.String() and
usid.LookupAccount("") in the case of a newUserFromSid()
call stack.
The functions current() and newUserFromSid() both call newUser()
supplying the arguments in question. The helpers
lookupUsernameAndDomain() and findHomeDirInRegistry() are
added.
This commit also updates:
- go/build/deps_test.go, so that the test now includes the
"internal/syscall/windows/registry" import.
- os/user/user_test.go, so that User.HomeDir is tested on Windows.
GitHub-Last-Rev: 25423e2a38
GitHub-Pull-Request: golang/go#23822
Change-Id: I6c3ad1c4ce3e7bc0d1add024951711f615b84ee5
Reviewed-on: https://go-review.googlesource.com/93935
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Calling UDPConn readers (Read, ReadFrom, ReadMsgUDP) to read part of
datagram returns error (in Windows), mentioning there is more data
available, and 0 as size of read data, even though part of data is
already read.
This fix makes UDPConn readers to return truncated payload size,
even there is error due more data available to read.
Fixes#14074
Updates #18056
Change-Id: Id7eec7f544dd759b2d970fa2561eef2937ec4662
Reviewed-on: https://go-review.googlesource.com/92475
Run-TryBot: Mikio Hara <mikioh.mikioh@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
Also add testdata for version 1.11 including UserTaskSpan test trace.
Change-Id: I673fb29bb3aee96a14fadc0ab860d4f5832143f5
Reviewed-on: https://go-review.googlesource.com/93795
Reviewed-by: Heschi Kreinick <heschi@google.com>
This implements the annotation API proposed in golang.org/cl/63274.
traceString is updated to protect the string map with trace.stringsLock
because the assumption that traceString is called by a single goroutine
(either at the beginning of tracing and at the end of tracing when
dumping all the symbols and function names) is no longer true.
traceString is used by the annotation apis (NewContext, StartSpan, Log)
to register frequently appearing strings (task and span names, and log
keys) after this change.
NewContext -> one or two records (EvString, EvUserTaskCreate)
end function -> one record (EvUserTaskEnd)
StartSpan -> one or two records (EvString, EvUserSpan)
span end function -> one or two records (EvString, EvUserSpan)
Log -> one or two records (EvString, EvUserLog)
EvUserLog record is of the typical record format written by traceEvent
except that it is followed by bytes that represents the value string.
In addition to runtime/trace change, this change includes
corresponding changes in internal/trace to parse the new record types.
Future work to improve efficiency:
More efficient unique task id generation instead of atomic. (per-P
counter).
Instead of a centralized trace.stringsLock, consider using per-P
string cache or something more efficient.
R=go1.11
Change-Id: Iec9276c6c51e5be441ccd52dec270f1e3b153970
Reviewed-on: https://go-review.googlesource.com/71690
Reviewed-by: Austin Clements <austin@google.com>
This CL presents the proposed user annotation API skeleton.
This CL bumps up the trace version to 1.11.
Design doc https://goo.gl/iqJfJ3
Implementation CLs are followed.
The API introduces three basic building blocks. Log, Span, and Task.
Log is for basic logging. When called, the message will be recorded
to the trace along with timestamp, goroutine id, and stack info.
trace.Log(ctx, messageType message)
Span can be thought as an extension of log to record interesting
time interval during a goroutine's execution. A span is local to a
goroutine by definition.
trace.WithSpan(ctx, "doVeryExpensiveOp", func(ctx context) {
/* do something very expensive */
})
Task is higher-level concept that aids tracing of complex operations
that encompass multiple goroutines or are asynchronous.
For example, an RPC request, a HTTP request, a file write, or a
batch job can be traced with a Task.
Note we chose to design the API around context.Context so it allows
easier integration with other tracing tools, often designed around
context.Context as well. Log and WithSpan APIs recognize the task
information embedded in the context and record it in the trace as
well. That allows the Go execution tracer to associate and group
the spans and log messages based on the task information.
In order to create a Task,
ctx, end := trace.NewContext(ctx, "myTask")
defer end()
The Go execution tracer measures the time between the task created
and the task ended for the task latency.
More discussion history in golang.org/cl/59572.
Update #16619
R=go1.11
Change-Id: I59a937048294dafd23a75cf1723c6db461b193cd
Reviewed-on: https://go-review.googlesource.com/63274
Reviewed-by: Austin Clements <austin@google.com>
Per the notice in the Go 1.10 release notes, this change drops the
support for Windows Vista or below (including Windows XP) and
simplifies the code for the sake of maintenance.
There is one exception to the above. The code related to DLL and
system calls still remains in the runtime package. The remaining code
will be refined and used for supporting upcoming Windows versions in
future.
Updates #17245Fixes#23072
Change-Id: I9e2821721f25ef9b83dfbf85be2b7ee5d9023aa5
Reviewed-on: https://go-review.googlesource.com/94255
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
NetBSD supports the SOCK_CLOEXEC and SOCK_NONBLOCK flags to the socket
syscall since version 6.0. The same version also introduced the paccept
syscall which can be used to implement syscall.Accept4.
Follows CL 40895
Change-Id: I9e4e1829b0382744c7799f4e58929a53b4e193f7
Reviewed-on: https://go-review.googlesource.com/94295
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This makes the constant names less verbose and aligns them more
with the Linux kernel which uses HWCAP_XXX for the constant names.
Change-Id: Ia7d079b59b57978adc045945951eaa1d99b41fac
Reviewed-on: https://go-review.googlesource.com/91738
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
golang.org/cl/81315 attempted to distinguish system goroutines
by examining the function name in the goroutine stack. It assumes that
the information would be available when GoSysBlock or GoInSyscall
events are processed, but it turned out the stack information is
set too late (when the goroutine gets a chance to run).
This change initializes the goroutine information entry when
processing GoCreate event which should be one of the very first
events for the every goroutine in trace.
Fixes#22574
Change-Id: I1ed37087ce2e78ed27c9b419b7d942eb4140cc69
Reviewed-on: https://go-review.googlesource.com/83595
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
os.NewFile doesn't put the fd into non-blocking mode.
In most cases, an *os.File returned by os.NewFile is in blocking mode.
Updates #7970
Updates #21856
Updates #23111
Change-Id: Iab08432e41f7ac1b5e25aaa8855d478adb7f98ed
Reviewed-on: https://go-review.googlesource.com/83995
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When we write a cached test result, we now also write a log of the
environment variables and files inspected by the test run,
along with a hash of their content. Before reusing a cached test result,
we recompute the hash of the content specified by the log, and only
use the result if that content has not changed.
This makes test caching behave correctly for tests that consult
environment variables or stat or read files or directories.
Fixes#22593.
Change-Id: I8608798e73c90e0c1911a38bf7e03e1232d784dc
Reviewed-on: https://go-review.googlesource.com/81895
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The DNS code can start goroutines and not wait for them to complete.
This does no harm, but in tests this can cause a race condition with
the test hooks that are installed and unintalled around the tests.
Add a WaitGroup that tests of DNS can use to avoid the race.
Fixes#21090
Change-Id: I6c1443a9c2378e8b89d0ab1d6390c0e3e726b0ce
Reviewed-on: https://go-review.googlesource.com/82795
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
It appears that TransmitFile Windows API does not work with Windows
pipes. So just copy data from pipe and into TCP connection manually.
Fixes#22278
Change-Id: I4810caca5345eac5bffb3176956689b8ae993256
Reviewed-on: https://go-review.googlesource.com/79775
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Test is in os/signal package because the problem is signal related.
Fixes#22838.
Change-Id: I223eeebb5fbc972910737eddef8ab9784cb984a6
Reviewed-on: https://go-review.googlesource.com/79215
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
change hash/crc32 package to use cpu package instead of using
runtime internal variables to check crc32 instruction
Change-Id: I8f88d2351bde8ed4e256f9adf822a08b9a00f532
Reviewed-on: https://go-review.googlesource.com/76490
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
I submitted two CLs which broke the build. Add temporary placeholder
with false bools to fix the build and restore old behavior.
Updates golang/go#22718 (details of why it broke)
Change-Id: I1f30624e14f631a95f4eff5aae462f1091f723a2
Reviewed-on: https://go-review.googlesource.com/77590
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This means {Read,Write}Msg{UDP,IP} now work on windows.
Fixes#9252
Change-Id: Ifb105f9ad18d61289b22d7358a95faabe73d2d02
Reviewed-on: https://go-review.googlesource.com/76393
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
CL 75253 introduced new SysProcAttr.Token field as Handle.
But we already have exact type for it - Token. Use Token
instead of Handle everywhere - it saves few type conversions
and provides better documentation for new API.
Change-Id: Ibc5407a234a1f49804de15a24b27c8e6a6eba7e0
Reviewed-on: https://go-review.googlesource.com/76314
Reviewed-by: Ian Lance Taylor <iant@golang.org>
WSASocket (unlike socket call) allows to create sockets that
will not be inherited by child process. So call WSASocket to
save on using syscall.ForkLock and calling syscall.CloseOnExec.
Some very old versions of Windows do not have that functionality.
Call socket, if WSASocket failed, to support these.
Change-Id: I2dab9fa00d1a8609dd6feae1c9cc31d4e55b8cb5
Reviewed-on: https://go-review.googlesource.com/72590
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This makes the output they print refer to the code that called them.
For example, instead of
=== RUN TestWindowsStackMemoryCgo
--- SKIP: TestWindowsStackMemoryCgo (0.00s)
testenv.go:213: skipping known flaky test ...
PASS
we see
=== RUN TestWindowsStackMemoryCgo
--- SKIP: TestWindowsStackMemoryCgo (0.00s)
crash_cgo_test.go:471: skipping known flaky test ...
PASS
Change-Id: I5f4c77c3aeab5c0e43c6dde2f15db70a6df24603
Reviewed-on: https://go-review.googlesource.com/76031
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Add support for ADX cpuid bit detection and all instructions,
implied by that bit (ADOX/ADCX). They are useful for rsa and math/big in
general.
Change-Id: Idaa93303ead48fd18b9b3da09b3e79de2f7e2193
Reviewed-on: https://go-review.googlesource.com/74850
Run-TryBot: Ilya Tocar <ilya.tocar@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
If the go install doesn't use the same flags as the main build
it can overwrite the installed standard library, leading to
flakiness and slow future tests.
Force uses of 'go install' etc to propagate $GO_GCFLAGS
or disable them entirely, to avoid problems.
As I understand it, the main place this happens is the ssacheck builder.
If there are other uses that need to run some of the now-disabled
tests we can reenable fixed tests in followup CLs.
Change-Id: Ib860a253539f402f8a96a3c00ec34f0bbf137c9a
Reviewed-on: https://go-review.googlesource.com/74470
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Add SetDeadline, SetReadDeadline, and SetWriteDeadline methods to os.File,
just as they exist today for the net package.
Fixes#22114
Change-Id: I4d390d739169b991175baba676010897dc8568fa
Reviewed-on: https://go-review.googlesource.com/71770
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
On systems that use kqueue, we always register descriptors for both
EVFILT_READ and EVFILT_WRITE. On at least FreeBSD and OpenBSD, when
the write end of a pipe is registered for EVFILT_READ and EVFILT_WRITE
events, and the read end of the pipe is closed, kqueue reports an
EVFILT_READ event with EV_EOF set, but does not report an EVFILT_WRITE
event. Since the write to the pipe is waiting for an EVFILT_WRITE
event, closing the read end of a pipe can cause the write end to hang
rather than attempt another write which will fail with EPIPE.
Fix this by treating EVFILT_READ with EV_EOF set as making both reads
and writes ready to proceed.
The real test for this is in CL 71770, which tests using various
timeouts with pipes.
Updates #22114
Change-Id: Ib23fbaaddbccd8eee77bdf18f27a7f0aa50e2742
Reviewed-on: https://go-review.googlesource.com/71973
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
No test because at present it is never called in a way that fails.
When #22114 is implemented, failure will be possible. Not including this
change in that work because this change is separable and clearly correct.
Updates #22114
Change-Id: I81eb9eec8800e8082d918c0e5fb71282f538267e
Reviewed-on: https://go-review.googlesource.com/71751
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Joe Tsai <joetsai@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
CL 36799 made SetFileCompletionNotificationModes to be called for
file handles. I don't think it is correct. Revert that change.
Fixes#22024Fixes#22207
Change-Id: I26260e8a727131cffbf60958d79eca2457495554
Reviewed-on: https://go-review.googlesource.com/69871
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Current code assumes that SetFileCompletionNotificationModes
is safe to call even if we know that it is not safe to use
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS flag. It appears (see issue #22149),
SetFileCompletionNotificationModes crashes when we call it without
FILE_SKIP_COMPLETION_PORT_ON_SUCCESS flag.
Do not call SetFileCompletionNotificationModes in that situation.
We are allowed to do that, because SetFileCompletionNotificationModes
is just an optimisation.
Fixes#22149
Change-Id: I0ad3aff4eabd8c27739417a62c286b1819ae166a
Reviewed-on: https://go-review.googlesource.com/69870
Reviewed-by: Ian Lance Taylor <iant@golang.org>
strings.LastIndexByte was introduced in go1.5 and it can be used
effectively wherever the second argument to strings.LastIndex is
exactly one byte long.
This avoids generating unnecessary string symbols and saves
a few calls to strings.LastIndex.
Change-Id: I7b5679d616197b055cffe6882a8675d24a98b574
Reviewed-on: https://go-review.googlesource.com/66372
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This permits the program to reliably know that when the Close method
returns, the descriptor has definitely been closed. This matters at
least for listeners.
Fixes#21856
Updates #7970
Change-Id: I1fd0cfd2333649e6e67c6ae956e19fdff3a35a83
Reviewed-on: https://go-review.googlesource.com/66150
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Joe Tsai <joetsai@google.com>
internal/poll package assumes that only net sockets use runtime
netpoller on windows. We get memory corruption if other file
handles are passed into runtime poller. Make FD.Init receive
and use useNetpoller argument, so FD.Init caller is explicit
about using runtime netpoller.
Fixes#21172
Change-Id: I60e2bfedf9dda9b341eb7a3e5221035db29f5739
Reviewed-on: https://go-review.googlesource.com/65810
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
In general, there are no guarantee that `go` command exist on $PATH.
This CL tries to get `go` command from $GOROOT/bin instead.
There are three kinds of code we should handle:
For normal code, the CL implements goCmd() or goCmdName().
For unit tests, the CL uses testenv.GoTool() or testenv.GoToolPath().
For integration tests, the CL sets PATH=$GOROOT/bin:$PATH in cmd/dist.
Note that make.bash sets PATH=$GOROOT/bin:$PATH in the build process.
So this change is only useful when we use toolchain manually.
Updates #21875
Change-Id: I963b9f22ea732dd735363ececde4cf94a5db5ca2
Reviewed-on: https://go-review.googlesource.com/64650
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
