Add unexported unlinkat, openat, and fstatat calls, so that
the internal/syscall/unix package can use them.
Change-Id: I1df81ecae6427211dd392ec68c9f020fe131a526
Reviewed-on: https://go-review.googlesource.com/c/148457
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Now, this is embarrassing. While preparing CL 142818, I noticed a
possible vulnerability in the existing code which I was rewriting. I
took a note to go back and assess if it was indeed an issue, and in case
start the security release process. The note unintentionally slipped
into the commit. Fortunately, there was no vulnerability.
What caught my eye was that I had fixed the calculation of the minimum
encrypted payload length from
roundUp(explicitIVLen+macSize+1, blockSize)
to (using the same variable names)
explicitIVLen + roundUp(macSize+1, blockSize)
The explicit nonce sits outside of the encrypted payload, so it should
not be part of the value rounded up to the CBC block size.
You can see that for some values of the above, the old result could be
lower than the correct value. An unexpectedly short payload might cause
a panic during decryption (a DoS vulnerability) or even more serious
issues due to the constant time code that follows it (see for example
Yet Another Padding Oracle in OpenSSL CBC Ciphersuites [1]).
In practice, explicitIVLen is either zero or equal to blockSize, so it
does not change the amount of rounding up necessary and the two
formulations happen to be identical. Nothing to see here.
It looked more suspicious than it is in part due to the fact that the
explicitIVLen definition moved farther into hc.explicitNonceLen() and
changed name from IV (which suggests a block length) to nonce (which
doesn't necessarily). But anyway it was never meant to surface or be
noted, except it slipped, so here we are for a boring explanation.
[1] https://blog.cloudflare.com/yet-another-padding-oracle-in-openssl-cbc-ciphersuites/
Change-Id: I365560dfe006513200fa877551ce7afec9115fdf
Reviewed-on: https://go-review.googlesource.com/c/147637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Miscellaneous additional conversions from raw syscalls
to using their libc equivalent.
Update #17490
Change-Id: If9ab22cc1d676c1f20fb161ebf02b0c28f71585d
Reviewed-on: https://go-review.googlesource.com/c/148257
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Pass the Go language version specified in the go.mod file to
cmd/compile.
Also, change the behavior when the go.mod file requests a Go version
that is later than the current one. Previously cmd/go would give a
fatal error in this situation. With this change it attempts the
compilation, and if (and only if) the compilation fails it adds a note
saying that the requested Go version is newer than the known version.
This is as described in https://golang.org/issue/28221.
Updates #28221.
Change-Id: I46803813e7872d4a418a3fd5299880be3b73a971
Reviewed-on: https://go-review.googlesource.com/c/147278
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
This will be used by later commits in this sequence.
Updates #28221
Change-Id: I2b22b9f88a0183636cde9509606f03f079eb33f1
Reviewed-on: https://go-review.googlesource.com/c/147277
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
There are still some references to the bare Syscall functions
in the stdlib. I will root those out in a following CL.
(This CL is big enough as it is.)
Most are in vendor directories:
cmd/vendor/golang.org/x/sys/unix/
vendor/golang_org/x/net/route/syscall.go
syscall/bpf_bsd.go
syscall/exec_unix.go
syscall/flock.go
Update #17490
Change-Id: I69ab707811530c26b652b291cadee92f5bf5c1a4
Reviewed-on: https://go-review.googlesource.com/c/141639
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Elias Naur <elias.naur@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It is possible to create certain recursive type declarations involving
alias types which cause the type-checker to produce an (invalid) type
for the alias because it is not yet available. By type-checking alias
declarations in a 2nd phase, the problem is mitigated a bit since it
requires more convoluted alias declarations for the problem to appear.
Also re-enable testing of fixedbugs/issue27232.go again (which was the
original cause for this change).
Updates #28576.
Change-Id: If6f9656a95262e6575b01c4a003094d41551564b
Reviewed-on: https://go-review.googlesource.com/c/147597
Reviewed-by: Alan Donovan <adonovan@google.com>
Sync @ fde099a (Oct 26, 2018)
Also update misc/nacl/testzip.proto to include new testdata.
Change-Id: If41590be9f395a591056e89a417b589c4ba71b1a
Reviewed-on: https://go-review.googlesource.com/c/147979
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This mainly entails writing compiler output files to a temporary
directory, as well as the corrupted files in TestVersionHandling.
Updates #28387
Change-Id: I6b3619a91fff27011c7d73daa4febd14a6c5c348
Reviewed-on: https://go-review.googlesource.com/c/146119
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This change updates the expected output of the delve debugging session
in the TestNexting internal/ssa test, aligning it with the changes
introduced in CL 147360 and earlier.
Change-Id: I1cc788d02433624a36f4690f24201569d765e5d3
Reviewed-on: https://go-review.googlesource.com/c/147998
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This change updates the expected output of the gdb debugging session
in the TestNexting internal/ssa test, aligning it with the changes
introduced in CL 147360.
Fixes the longtest builder.
Change-Id: I5b5c22e1cf5e205967ff8359dc6c1485c815428e
Reviewed-on: https://go-review.googlesource.com/c/147957
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Like every other command's -h flag. To achieve this, pass the command's
usage function to the cmdflag package, since that package is used by
multiple commands and cannot directly access *base.Command.
This also lets us get rid of testFlag1 and testFlag2, and instead have
contiguous raw strings for the test and testflag help docs.
Fixes#26999.
Change-Id: I2ebd66835ee61fa83270816a01fa312425224bb3
Reviewed-on: https://go-review.googlesource.com/c/144558
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
This change fixes a TestFormat failure in fmt_test by adding a
recently introduced new known format (%q for syntax.Error).
Fixes#28621
Change-Id: I026ec88c334549a957a692c1652a860c57e23dae
Reviewed-on: https://go-review.googlesource.com/c/147837
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In ARM64 ABI, R18 is the "platform register", the use of which is
OS specific. The OS could choose to reserve this register. In
practice, it seems fine to use R18 on Linux but not on darwin (iOS).
Rename R18 to R18_PLATFORM to prevent accidental use. There is no
R18 usage within the standard library (besides tests, which are
updated).
Fixes#26110
Change-Id: Icef7b9549e2049db1df307a0180a3c90a12d7a84
Reviewed-on: https://go-review.googlesource.com/c/147218
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Dead-code eliminating labels is tricky because there might
be gotos that can still reach them.
Bug probably introduced with CL 91056
Fixes#28616
Change-Id: I6680465134e3486dcb658896f5172606cc51b104
Reviewed-on: https://go-review.googlesource.com/c/147817
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Iskander Sharipov <iskander.sharipov@intel.com>
This change adds the vet-lite command (the future cmd/vet) and all its
dependencies from x/tools, but not its tests and their dependencies.
It was created with these commands:
$ (cd $GOPATH/src/golang.org/x/tools && git checkout c76e1ad)
$ cd GOROOT/src/cmd
$ govendor add $(go list -deps golang.org/x/tools/go/analysis/cmd/vet-lite | grep golang.org/x/tools)
$ rm -fr $(find vendor/golang.org/x/tools/ -name testdata)
$ rm $(find vendor/golang.org/x/tools/ -name \*_test.go)
I feel sure I am holding govendor wrong. Please advise.
A followup CL will make cmd/vet behave like vet-lite, initially just
for users that opt in, and soon after for all users, at which point
cmd/vet will be replaced in its entirety by a copy of vet-lite's small
main.go.
In the meantime, anyone can try the new tool using these commands:
$ go build cmd/vendor/golang.org/x/tools/go/analysis/cmd/vet-lite
$ export GOVETTOOL=$(which vet-lite)
$ go vet your/project/...
Change-Id: Iea168111a32ce62f82f9fb706385ca0f368bc869
Reviewed-on: https://go-review.googlesource.com/c/147444
Reviewed-by: Russ Cox <rsc@golang.org>
On AIX, cmd/link needs two information in order to generate a dynamic
import, the library and its object needed. Currently, cmd/link isn't
able to retrieve this object only with the name of the library.
Therefore, the library pattern in cgo_import_dynamic must be
"lib.a/obj.o".
Change-Id: Ib8b8aaa9807c9fa6af46ece4e312d58073ed6ec1
Reviewed-on: https://go-review.googlesource.com/c/146957
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Previously we panicked if the number of methods present for an embedded
field was >= 32. This change removes that limit and now StructOf
dynamically calls itself to create space for the number of methods.
Fixes#25402
Change-Id: I3b1deb119796d25f7e6eee1cdb126327b49a0b5e
GitHub-Last-Rev: 16da71ad6b
GitHub-Pull-Request: golang/go#26865
Reviewed-on: https://go-review.googlesource.com/c/128479
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
We collapse OpOffPtrs during generic rewrites.
However, we also use disjoint at the same time.
Instead of waiting for all OpOffPtrs to be collapsed
before the disjointness rules can kick in,
burrow through all OpOffPtrs immediately.
Change-Id: I60d0a70a9b4605b1817db7c4aab0c0d789651c90
Reviewed-on: https://go-review.googlesource.com/c/145206
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Michael Munday <mike.munday@ibm.com>
Reviewed-by: Keith Randall <khr@golang.org>
Consider these functions:
func f(a any) int
func g(a any) int
Prior to this change, since f and g have identical signatures,
they would share a single generated func type.
types.SubstAny makes a shallow type copy, even after instantiation,
f and g share a single generated Result type.
So if you instantiate f with any=T, call dowidth,
instantiate g with any=U, and call dowidth,
and if sizeof(T) != sizeof(U),
then the Offset of the result for f is now wrong.
I don't believe this happens at all right now, but it bit me hard when
experimenting with some other compiler changes.
And it's hard to debug. It results in rare stack corruption, causing
problems far from the actual source of the problem.
To fix this, change SubstAny to make deep copies of TSTRUCTs.
name old alloc/op new alloc/op delta
Template 35.3MB ± 0% 35.4MB ± 0% +0.23% (p=0.008 n=5+5)
Unicode 29.1MB ± 0% 29.1MB ± 0% +0.16% (p=0.008 n=5+5)
GoTypes 122MB ± 0% 122MB ± 0% +0.16% (p=0.008 n=5+5)
Compiler 513MB ± 0% 514MB ± 0% +0.19% (p=0.008 n=5+5)
SSA 1.94GB ± 0% 1.94GB ± 0% +0.01% (p=0.008 n=5+5)
Flate 24.2MB ± 0% 24.2MB ± 0% +0.08% (p=0.008 n=5+5)
GoParser 28.5MB ± 0% 28.5MB ± 0% +0.24% (p=0.008 n=5+5)
Reflect 86.2MB ± 0% 86.3MB ± 0% +0.09% (p=0.008 n=5+5)
Tar 34.9MB ± 0% 34.9MB ± 0% +0.13% (p=0.008 n=5+5)
XML 47.0MB ± 0% 47.1MB ± 0% +0.18% (p=0.008 n=5+5)
[Geo mean] 80.9MB 81.0MB +0.15%
name old allocs/op new allocs/op delta
Template 348k ± 0% 349k ± 0% +0.38% (p=0.008 n=5+5)
Unicode 340k ± 0% 340k ± 0% +0.21% (p=0.008 n=5+5)
GoTypes 1.27M ± 0% 1.28M ± 0% +0.27% (p=0.008 n=5+5)
Compiler 4.90M ± 0% 4.92M ± 0% +0.36% (p=0.008 n=5+5)
SSA 15.3M ± 0% 15.3M ± 0% +0.03% (p=0.008 n=5+5)
Flate 232k ± 0% 233k ± 0% +0.14% (p=0.008 n=5+5)
GoParser 291k ± 0% 292k ± 0% +0.42% (p=0.008 n=5+5)
Reflect 1.05M ± 0% 1.05M ± 0% +0.14% (p=0.008 n=5+5)
Tar 343k ± 0% 344k ± 0% +0.22% (p=0.008 n=5+5)
XML 428k ± 0% 430k ± 0% +0.36% (p=0.008 n=5+5)
[Geo mean] 807k 809k +0.25%
Change-Id: I62134db642206cded01920dc1d8a7da61f7ca0ac
Reviewed-on: https://go-review.googlesource.com/c/147038
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This change cleans up references to MSpan, MCache, and MCentral in the
docs via a bunch of sed invocations to better reflect the Go names for
the equivalent structures (i.e. mspan, mcache, mcentral) and their
methods (i.e. MSpan_Sweep -> mspan.sweep).
Change-Id: Ie911ac975a24bd25200a273086dd835ab78b1711
Reviewed-on: https://go-review.googlesource.com/c/147557
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
TestTracebackAncestors has a ~0.1% chance of failing with more
goroutines in the traceback than expected. This happens because
there's a window between each goroutine starting its child and that
goroutine actually exiting. The test captures its own stack trace
after everything is "done", but if this happens during that window, it
will include the goroutine that's in the process of being torn down.
Here's an example of such a failure:
https://build.golang.org/log/fad10d0625295eb79fa879f53b8b32b9d0596af8
This CL fixes this by recording the goroutines that are expected to
exit and removing them from the stack trace. With this fix, this test
passed 15,000 times with no failures.
Change-Id: I71e7c6282987a15e8b74188b9c585aa2ca97cbcd
Reviewed-on: https://go-review.googlesource.com/c/147517
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This documentation example was broken:
https://golang.org/pkg/image/png/#example_Decode.
It did not have the "io" package imported,
The package was referenced in the result type of the function.
The "playExample" function did not inspect
the result types of declared functions.
This CL adds inspecting of parameters and result types of functions.
Fixes#28492
Updates #9679
Change-Id: I6d8b11bad2db8ea8ba69039cfaa914093bdd5132
Reviewed-on: https://go-review.googlesource.com/c/146118
Run-TryBot: Yury Smolsky <yury@smolsky.by>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This change re-introduces (temporarily) a work-around for recursive
alias type declarations, originally in https://golang.org/cl/35831/
(intended as fix for #18640). The work-around was removed later
for a more comprehensive cycle detection check. That check
contained a subtle error which made the code appear to work,
while in fact creating incorrect types internally. See #25838
for details.
By re-introducing the original work-around, we eliminate problems
with many simple recursive type declarations involving aliases;
specifically cases such as #27232 and #27267. However, the more
general problem remains.
This CL also fixes the subtle error (incorrect variable use when
analyzing a type cycle) mentioned above and now issues a fatal
error with a reference to the relevant issue (rather than crashing
later during the compilation). While not great, this is better
than the current status. The long-term solution will need to
address these cycles (see #25838).
As a consequence, several old test cases are not accepted anymore
by the compiler since they happened to work accidentally only.
This CL disables parts or all code of those test cases. The issues
are: #18640, #23823, and #24939.
One of the new test cases (fixedbugs/issue27232.go) exposed a
go/types issue. The test case is excluded from the go/types test
suite and an issue was filed (#28576).
Updates #18640.
Updates #23823.
Updates #24939.
Updates #25838.
Updates #28576.
Fixes#27232.
Fixes#27267.
Change-Id: I6c2d10da98bfc6f4f445c755fcaab17fc7b214c5
Reviewed-on: https://go-review.googlesource.com/c/147286
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
As of 07e738e all spans are allocated out of a treap, and not just
large spans or spans for large objects. Also, now we have a separate
treap for spans that have been scavenged.
Change-Id: I9c2cb7b6798fc536bbd34835da2e888224fd7ed4
Reviewed-on: https://go-review.googlesource.com/c/142958
Reviewed-by: Austin Clements <austin@google.com>
This uses the mutator utilization distribution to compute the p99.9,
p99, and p95 mutator utilization topograph lines and display them
along with the MMU.
Change-Id: I8c7e0ec326aa4bc00619ec7562854253f01cc802
Reviewed-on: https://go-review.googlesource.com/c/60800
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This adds support for computing the quantiles of a mutator utilization
distribution.
Change-Id: Ia8b3ed14bf415c234e2f567360fd1b361d28bd40
Reviewed-on: https://go-review.googlesource.com/c/60799
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
The current MMU analysis considers all Ps together, so if, for
example, one of four Ps is blocked, mutator utilization is 75%.
However, this is less useful for understanding the impact on
individual goroutines because that one blocked goroutine could be
blocked for a very long time, but we still appear to have good
utilization.
Hence, this introduces a new flag that does a "per-P" analysis where
the utilization of each P is considered independently. The MMU is then
the combination of the MMU for each P's utilization function.
Change-Id: Id67b980d4d82b511d28300cdf92ccbb5ae8f0c78
Reviewed-on: https://go-review.googlesource.com/c/60797
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This adds the ability to click a point on the MMU graph to show a list
of the worst 10 mutator utilization windows of the selected size. This
list in turn links to the trace viewer to drill down on specifically
what happened in each specific window.
Change-Id: Ic1b72d8b37fbf2212211c513cf36b34788b30133
Reviewed-on: https://go-review.googlesource.com/c/60795
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This will let the trace viewer show specifically when poor utilization
happened and link to specific instances in the trace.
Change-Id: I1f03a0f9d9a7570009bb15762e7b8b6f215e9423
Reviewed-on: https://go-review.googlesource.com/c/60793
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This further optimizes MMU construction by first computing a
low-resolution summary of the utilization curve. This "band" summary
lets us compute the worst-possible window starting in each of these
low-resolution bands (even without knowing where in the band the
window falls). This in turn lets us compute precise minimum mutator
utilization only in the worst low-resolution bands until we can show
that any remaining bands can't possibly contain a worse window.
This slows down MMU construction for small traces, but these are
reasonably fast to compute either way. For large traces (e.g.,
150,000+ utilization changes) it's significantly faster.
Change-Id: Ie66454e71f3fb06be3f6173b6d91ad75c61bda48
Reviewed-on: https://go-review.googlesource.com/c/60792
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This commit speeds up MMU construction by ~10X (and reduces the number
of windows considered by ~20X) by using an observation about the
maximum slope of the windowed mutator utilization function to advance
the window time in jumps if the window's current mean mutator
utilization is much larger than the current minimum.
Change-Id: If3cba5da0c4adc37b568740f940793e491e96a51
Reviewed-on: https://go-review.googlesource.com/c/60791
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This adds an endpoint to the trace tool that plots the minimum mutator
utilization curve using information on mark assists and GC pauses from
the trace.
This commit implements a fairly straightforward O(nm) algorithm for
computing the MMU (and tests against an even more direct but slower
algorithm). Future commits will extend and optimize this algorithm.
This should be useful for debugging and understanding mutator
utilization issues like #14951, #14812, #18155. #18534, #21107,
particularly once follow-up CLs add trace cross-referencing.
Change-Id: Ic2866869e7da1e6c56ba3e809abbcb2eb9c4923a
Reviewed-on: https://go-review.googlesource.com/c/60790
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
