This change adds two new treap iteration types: one for large
unscavenged spans (contain at least one huge page) and one for small
unscavenged spans. This allows us to scavenge the huge spans first by
first iterating over the large ones, then the small ones.
Also, since we now depend on physHugePageSize being a power of two,
ensure that that's the case when it's retrieved from the OS.
For #30333.
Change-Id: I51662740205ad5e4905404a0856f5f2b2d2a5680
Reviewed-on: https://go-review.googlesource.com/c/go/+/174399
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change introduces a treapIterFilter type which represents the
power set of states described by a treapIterType.
This change then adds a treapIterFilter field to each treap node
indicating the types of spans that live in that subtree. The field is
maintained via the same mechanism used to maintain maxPages. This allows
pred, succ, start, and end to be judicious about which subtrees it will
visit, ensuring that iteration avoids traversing irrelevant territory.
Without this change, repeated scavenging attempts can end up being N^2
as the scavenger walks over what it already scavenged before finding new
spans available for scavenging.
Finally, this change also only scavenges a span once it is removed from
the treap. There was always an invariant that spans owned by the treap
may not be mutated in-place, but with this change violating that
invariant can cause issues with scavenging.
For #30333.
Change-Id: I8040b997e21c94a8d3d9c8c6accfe23cebe0c3d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/174878
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change modifies the treap implementation to support holding all
spans in a single treap, instead of keeping them all in separate treaps.
This improves ergonomics for nearly all treap-related callsites.
With that said, iteration is now more expensive, but it never occurs on
the fast path, only on scavenging-related paths.
This change opens up the opportunity for further optimizations, such as
splitting spans without treap removal (taking treap removal off the span
allocator's critical path) as well as improvements to treap iteration
(building linked lists for each iteration type and managing them on
insert/removal, since those operations should be less frequent).
For #30333.
Change-Id: I3dac97afd3682a37fda09ae8656a770e1369d0a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/174398
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Currently, when the compiler emits a symbol name in the object
file, it uses "". for the package path of the package being
compiled. This is then expanded in the linker to the actual
package path.
With CL 173938, it does not need an allocation if the symbol name
does not need expansion. In many cases, the compiler actually
knows the package path (through the -p flag), so we could just
write it out in compile time, without fixing it up in the linker.
This reduces allocations in the linker.
In case that the package path is not known (compiler's -p flag is
missing, or the object file is generated by the assembler), the
linker still does the expansion.
This reduces ~100MB allocations (~10% inuse_space) in linking
k8s.io/kubernetes/cmd/kube-apiserver on Linux/AMD64.
Also makes the linker a little faster: linking cmd/go on
Linux/AMD64:
Real 1.13 ± 1% 1.11 ± 1% -2.13% (p=0.000 n=10+10)
User 1.17 ± 3% 1.14 ± 5% -3.14% (p=0.003 n=10+10)
Sys 0.34 ±15% 0.34 ±15% ~ (p=0.986 n=10+10)
The caveat is that the object files get slightly bigger. On
Linux/AMD64, runtime.a gets 2.1% bigger, cmd/compile/internal/ssa
(which has a longer import path) gets 2.8% bigger.
This reveals that when building an unnamed plugin (e.g.
go build -buildmode=plugin x.go), the go command passes different
package paths to the compiler and to the linker. Before this CL
there seems nothing obviously broken, but given that the compiler
already emits the package's import path in various places (e.g.
debug info), I guess it is possible that this leads to some
unexpected behavior. Now that the compiler writes the package
path in more places, this disagreement actually leads to
unresolved symbols. Adjust the go command to use the same package
path for both compiling and linking.
Change-Id: I19f08981f51db577871c906e08d9e0fd588a2dd8
Reviewed-on: https://go-review.googlesource.com/c/go/+/174657
Reviewed-by: Austin Clements <austin@google.com>
This does not repair #31786, and in fact also unfixes the revert
of CL 174617. We were just getting lucky when it looked like
it was working. And unfortunately for the bug, there does not
appear to be any particular problems with the line numbers;
if anything they're a couple of extras, i.e., stepping might
repeat, rather than skip. Delve works fine either way.
Updates #31786.
Change-Id: I5c2fdc2a0265bb99773b3a85492a3db557dffee4
Reviewed-on: https://go-review.googlesource.com/c/go/+/174948
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
On my machine, these directories add up to 276mb
and account for 40% of the size of the GOROOT directory.
Once bootstrapping is complete, they are never used again.
Fixes#31851
Change-Id: Idbf8f21bae3d64655aa43761cc778677add6234a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175377
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Updates x/net/http2 to git rev 1da14a5a36f220ea3f03470682b737b1dfd5de22 for:
http2: make empty method mean GET
https://golang.org/cl/169557 (Fixesgolang/go#31061)
http2: don't hang a stream if trailers values are not provided
https://golang.org/cl/161958
Change-Id: I628af8c6d07d19e8f19ee37637243f6c242ef3a1
Reviewed-on: https://go-review.googlesource.com/c/go/+/174677
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
These are unused since CL 153837. illumos64Bit was added by CL 174457
but was never used.
Change-Id: I34a1bd41cf70f8a07e57f93a71de3c6034fcaf7b
Reviewed-on: https://go-review.googlesource.com/c/go/+/175358
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Those print statements are not a good debug helpers
and only clutter the code.
Change-Id: Ifbf450a04e6fa538af68e6352c016728edb4119a
Reviewed-on: https://go-review.googlesource.com/c/go/+/160537
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Add Unwrap methods to types which wrap an underlying error:
"encodinc/csv".ParseError
"encoding/json".MarshalerError
"net/http".transportReadFromServerError
"net".OpError
"net".DNSConfigError
"net/url".Error
"os/exec".Error
"signal/internal/pty".PtyError
"text/template".ExecError
Add os.ErrTemporary. A case could be made for putting this error
value in package net, since no exported error types in package os
include a Temporary method. However, syscall errors returned from
the os package do include this method.
Add Is methods to error types with a Timeout or Temporary method,
making errors.Is(err, os.Err{Timeout,Temporary}) equivalent to
testing the corresponding method:
"context".DeadlineExceeded
"internal/poll".TimeoutError
"net".adrinfoErrno
"net".OpError
"net".DNSError
"net/http".httpError
"net/http".tlsHandshakeTimeoutError
"net/pipe".timeoutError
"net/url".Error
Updates #30322
Updates #29934
Change-Id: I409fb20c072ea39116ebfb8c7534d493483870dc
Reviewed-on: https://go-review.googlesource.com/c/go/+/170037
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marcel van Lohuizen <mpvl@golang.org>
golang.org/cl/127495 incorrectly tested against environment specifc
details that do not form part of the script test conditions. This broke
the clang build.
Fix by removing the specific check; the existing checks are sufficient.
Change-Id: Ic6ec873df9343c809968a08dd676e210046da5fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/175179
Run-TryBot: Paul Jolly <paul@myitcv.org.uk>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
sys.HugePageSize was superceded in the last commit by physHugePageSize
which is determined dynamically by querying the operating system.
For #30333.
Change-Id: I827bfca8bdb347e989cead31564a8fffe56c66ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/173757
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
This change adds the global physHugePageSize which is initialized in
osinit(). physHugePageSize contains the system's transparent huge page
(or superpage) size in bytes.
For #30333.
Change-Id: I2f0198c40729dbbe6e6f2676cef1d57dd107562c
Reviewed-on: https://go-review.googlesource.com/c/go/+/170858
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Austin Clements <austin@google.com>
As a result of using go env, the following new environment variables are
shown as part of the env section:
+CGO_CFLAGS="-g -O2"
+CGO_CPPFLAGS=""
+CGO_CXXFLAGS="-g -O2"
+CGO_FFLAGS="-g -O2"
+CGO_LDFLAGS="-g -O2"
+PKG_CONFIG="pkg-config"
+GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build612849170=/tmp/go-build -gno-record-gcc-switches"
The diff between the web-based template and the result of go bug is now:
+GOROOT/bin/go version: go version devel +478f3a5384 Wed Mar 27 16:21:00 2019 +0000 linux/amd64
+GOROOT/bin/go tool compile -V: compile version devel +478f3a5384 Wed Mar 27 16:21:00 2019 +0000
+uname -sr: Linux 4.15.0-46-generic
+Distributor ID: Ubuntu
+Description: Ubuntu 18.04.2 LTS
+Release: 18.04
+Codename: bionic
+/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Ubuntu GLIBC 2.27-3ubuntu1) stable release version 2.27.
Fixes#26751
Change-Id: I32baca1c3c06d08068dad0041a43a1f5532bd91e
Reviewed-on: https://go-review.googlesource.com/c/go/+/127495
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
The testcase created a race between the close of the current connection
and the client grabbing a connection for the next request. The client
may receive the current connection which may be closed during its use.
We can have the trasnport close all idle connections thereby forcing the
client to receive a new connection.
Closing idle connections did not handle cleaning up host connection
counts for http/2. We will now decrement the host connection count for
http/2 connections.
Fixes#31784
Change-Id: Iefc0d0d7ed9fa3acd8b4f42004f1579fc1de63fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/174950
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Using password that returns from User.Password() won't work in this case
because password in Userinfo already unescaped. The solution is uses
User.String() to escape password back again and then stringify it to error.
Fixes#31808
Change-Id: I723aafd5a57a5b69f2dd7d3a21b82ebbd4174451
Reviewed-on: https://go-review.googlesource.com/c/go/+/175018
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
They were already skipped on tethered Android builders because
the tests are gated on GO_BUILDER_NAME being set and the Android
exec wrapper does not propagate GO_BUILDER_NAME.
Updates #31722
Change-Id: Ifd2c7daecc19a4e540d86d1f38083f43cc3e6b15
Reviewed-on: https://go-review.googlesource.com/c/go/+/175177
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
iostest.bash might not live much longer, and all.bash is much
less confusing and more explicit.
Change-Id: If42e8716bbbb02aa3f817dceaabb1aa8076aae1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/175178
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL adds intrinsics for the 64-bit addition and subtraction
functions in math/bits. These intrinsics use the condition code
to propagate the carry or borrow bit.
To make the carry chains more efficient I've removed the
'clobberFlags' property from most of the load and store
operations. Originally these ops did clobber flags when using
offsets that didn't fit in a signed 20-bit integer, however
that is no longer true.
As with other platforms the intrinsics are faster when executed
in a chain rather than a loop because currently we need to spill
and restore the carry bit between each loop iteration. We may
be able to reduce the need to do this on s390x (e.g. by using
compare-and-branch instructions that do not clobber flags) in the
future.
name old time/op new time/op delta
Add64 1.21ns ± 2% 2.03ns ± 2% +67.18% (p=0.000 n=7+10)
Add64multiple 2.98ns ± 3% 1.03ns ± 0% -65.39% (p=0.000 n=10+9)
Sub64 1.23ns ± 4% 2.03ns ± 1% +64.85% (p=0.000 n=10+10)
Sub64multiple 3.73ns ± 4% 1.04ns ± 1% -72.28% (p=0.000 n=10+8)
Change-Id: I913bbd5e19e6b95bef52f5bc4f14d6fe40119083
Reviewed-on: https://go-review.googlesource.com/c/go/+/174303
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
golang.org/cl/174498 add ONAME case to isStaticCompositeLiteral, to
detect global variable as compile-time constant.
It does report wrong for struct field, e.g:
o := one{i: two{i: 42}.i}
field i in two{i: 42} was reported as static composite literal, while it
should not.
In general, adding ONAME case for isStaticCompositeLiteral is probably
wrong.
Fixes#31782
Change-Id: Icde7d43bbb002b75df5c52b948b7126a4265e07b
Reviewed-on: https://go-review.googlesource.com/c/go/+/174837
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Int.String had no documentation and the documentation for Int.Text
did not mention the handling of the nil pointer case.
Change-Id: I9f21921e431c948545b7cabc7829e4b4e574bbe9
Reviewed-on: https://go-review.googlesource.com/c/go/+/175118
Reviewed-by: Robert Griesemer <gri@golang.org>
$ go get -u golang.org/x/sys
$ go mod vendor
$ go mod tidy
Change-Id: Ie0a4646aef41b00ec8e27bc6f7e3ec9c270c8ccb
Reviewed-on: https://go-review.googlesource.com/c/go/+/174946
Run-TryBot: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Instead of requiring exact type match, allow assignment conversions
(those conversions allowed in the language spec without a cast) on the
returned values.
Particularly useful when the type being returned is an interface type,
but the Value actually returned is a concrete value implementing that
type (as it is tricky to return a Value which has interface type).
RELNOTE=y
Fixes#28761
Change-Id: I69eef07ca51690b2086dfa1eb549db5e4724c657
Reviewed-on: https://go-review.googlesource.com/c/go/+/174531
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We compile package sort as part of the compiler bootstrap,
to make sure the compiler uses a consistent sort algorithm
no matter what version of Go it is compiled against.
(This matters for elements that compare "equal" but are distinguishable.)
Package sort was compiled in such a way as to disallow
sort.Slice entirely during bootstrap (at least with some compilers),
while cmd/internal/obj was compiled in such a way as to
make obj.SortSlice available to all compilers, precisely because
sort.Slice was not. This is all highly confusing.
Simplify by making sort.Slice available all the time.
Followup to CL 169137 and #30440
(and also CL 40114 and CL 73951).
Change-Id: I127f4e02d6c71392805d256c3a90ef7c51f9ba0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/174525
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This CL improves internal link to provide basic support for cgo and PIE:
1, add support for GOT, PLT and GOTPLT.
2, add support for following ELF relocation types which have been used by std
packages:
R_AARCH64_ADR_GOT_PAGE
R_AARCH64_LD64_GOT_LO12_NC
R_AARCH64_ADR_PREL_PG_HI21
R_AARCH64_ADD_ABS_LO12_NC
R_AARCH64_LDST8_ABS_LO12_NC
R_AARCH64_LDST32_ABS_LO12_NC
R_AARCH64_LDST64_ABS_LO12_NC
R_AARCH64_JUMP26
R_AARCH64_ABS64
R_AARCH64_PREL32
R_AARCH64_PREL64
With this change, Go toolchain can be built in internal linking mode, and
pure Go programs can be built with PIE mode in internal linking mode on arm64.
Updates #10373
The prototype of this CL is contributed by Wei Xiao <wei.xiao@arm.com>
Change-Id: I2253923c69e855fd1524d54def309a961dce6247
Reviewed-on: https://go-review.googlesource.com/c/go/+/163579
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
This change will allow android/arm64 hosts to build for android/arm,
and likewise for iOS.
Updates #31722
Change-Id: Id410bd112abbab585ebb13b61fe4d3a38a1a81fb
Reviewed-on: https://go-review.googlesource.com/c/go/+/174705
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
As pointed out by thepudds in #30634, the 'list -u' documentation states that the current version should be considered for upgrade:
The -u flag adds information about available upgrades. When the latest version of a given module is newer than the current one, list -u sets the Module's Update field to information about the newer module.
In go 1.12.4 (and current tip), an older version will be suggested as upgrade to a newer pseudo version.
Updates: #30634
Change-Id: If2c8887198884b8e7ccb3a604908065aa1f1878a
Reviewed-on: https://go-review.googlesource.com/c/go/+/174206
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Mid-stack inlining is enable now, see #19348, but we still can not
remove the special case for runtime.heapBits.nextArena, because
runtime.heapBits.next is too complex to be inlined
(cost 96 exceeds budget 80).
Change-Id: I04ea86509074afdc83a3f70d68b8a1a8829763d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/174839
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The Unwrap function performs a type assertion looking for the Wrapper
interface. The method of that interface is called Unwrap but the
interface itself is called Wrapper.
Change-Id: Ie3bf296f93b773d36015bcab2a0e6585d39783c7
GitHub-Last-Rev: 32b1a0c2f8
GitHub-Pull-Request: golang/go#31794
Reviewed-on: https://go-review.googlesource.com/c/go/+/174917
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The web package is now used for proxy fetches, so its logs shouldn't
start with "Parsing meta tags".
Change-Id: I22a7dce09e3a681544ee4b860f93c63336e547ca
Reviewed-on: https://go-review.googlesource.com/c/go/+/174740
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
golang.org/cl/174498 removes dynamic map entry handling in maplit, by
filtering the static entry only. It panics if it see a dynamic entry.
It relies on order to remove all dynamic entries.
But after recursively call order on the statics, some static entries
become dynamic, e.g OCONVIFACE node:
type i interface {
j()
}
type s struct{}
func (s) j() {}
type foo map[string]i
var f = foo{
"1": s{},
}
To fix it, we recursively call order on each static entry, if it changed
to dynamic, put entry to dynamic then.
Fixes#31777
Change-Id: I1004190ac8f2d1eaa4beb6beab989db74099b025
Reviewed-on: https://go-review.googlesource.com/c/go/+/174777
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Right now they are in a deterministic order
but one that depends on the shape of the import graph.
Sort them instead.
Change-Id: Ia0c076a0d6677a511e52acf01f38353e9895dec2
Reviewed-on: https://go-review.googlesource.com/c/go/+/174527
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The newly added functions create a copy of their input with all bytes in
invalid UTF-8 byte sequences mapped to the UTF-8 byte sequence
given as replacement parameter.
Fixes#25805
Change-Id: Iaf65f65b40c0581c6bb000f1590408d6628321d0
Reviewed-on: https://go-review.googlesource.com/c/go/+/142003
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
