For linking large binaries, the slice of Relocs consumes a large
amount of memory. We can reduce this memory consumption by
shrinking the size of the Reloc struct. This CL moves the fields
used only in external linking or only on PPC64 and S390X to a
lazily initialized side struct.
Linking k8s.io/kubernetes/cmd/kube-apiserver on Linux/AMD64,
before:
inuse_space 1240.25MB total
438.11MB 35.32% 35.32% 438.11MB 35.32% cmd/link/internal/objfile.(*objReader).readSlices
after:
inuse_space 1123.39MB total
306.85MB 27.31% 55.03% 306.85MB 27.31% cmd/link/internal/objfile.(*objReader).readSlices
Under GOGC=5 (to simulate system under memory pressure), the max
RSS reduces from ~2.05G to ~1.83G. Even with external linking the
max RSS doesn't increase.
Change-Id: I9bb8a8effaed24a8b3c4b1cfb93e6992b8740f91
Reviewed-on: https://go-review.googlesource.com/c/go/+/173437
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
A nil src (NULL database value) will result in a "nil" string,
which will never parse correctly in a ParseInt or similar
numeric conversion. The resulting error is confusing. Check
for a nil src prior to converting the value to string
if the resulting string will be parsed after that.
Closes#31274
Change-Id: I90f12cceff00fbbfdd3e343b04fa7e2596390e6d
Reviewed-on: https://go-review.googlesource.com/c/go/+/174177
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This updates the packages from text and net to use
the Unicode 11 tables.
Updates golang/go#27945
Change-Id: I096e80283f37b596f2aa8bd3e5ffec6b7045469a
Reviewed-on: https://go-review.googlesource.com/c/go/+/174057
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Copied and updated import paths.
Eventually we will probably publish
these packages somewhere in golang.org/x
(as non-internal packages) and then we will
be able to vendor them properly.
For now, copy.
sumweb.globsMatchPath moved to str.GlobsMatchPath.
Change-Id: I4585e6dc5daa423e4ca9669195d41e58e7c8c275
Reviewed-on: https://go-review.googlesource.com/c/go/+/173950
Reviewed-by: Jay Conrod <jayconrod@google.com>
Needed for Go checksum database integration in cmd/go.
Generated with "go mod vendor" in client with that CL already applied.
Change-Id: I1aa84977ba3a62504d3f4c069f48c07c67b83023
Reviewed-on: https://go-review.googlesource.com/c/go/+/173949
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Testing whether cgo is enabled in go/build is not the same as testing
whether the go tool supports cgo. They differ, for example, when using
GOARCH=386 on an amd64 system, as for a cross-build cgo is disabled by default.
Change-Id: Ib59106c92a3131b73ac6a91c0f7658a1769acf73
Reviewed-on: https://go-review.googlesource.com/c/go/+/174098
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This brings in support for openbsd/arm64.
Updated via 'go get' and 'go mod vendor'.
Updates #31656
Change-Id: I18ffbc622d542f61ca0442b9dab28271fa2cfa85
Reviewed-on: https://go-review.googlesource.com/c/go/+/174118
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
AIX doesn't allow to mmap an address range which is already mmap.
Therefore, once the region has been allocated, it must munmap before
being able to play with it.
Change-Id: I1547782f0379024f57869f1dda8c1c9bb12d831f
Reviewed-on: https://go-review.googlesource.com/c/go/+/174059
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
When the leading zero format is used, we currently don't handle the
month and year properly.
For the month, we were reporting an out of range error when getnum
returns zero of its own, as it also returns the month 0. That's
confusing, so only check the range when getnum returns a nil error.
For the year, we don't restore the value when parsing error occurs. For
example, with the incorrect input "111-01", "01" will be used to report
an error. So restore the value when an error occurs fix the problem.
Fixes#29918Fixes#29916
Change-Id: I3145f8c46813a0457766b7c302482e6b56f94ed6
Reviewed-on: https://go-review.googlesource.com/c/go/+/160338
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
I couldn't remember and couldn't tell from the docs,
so I added a test and documented what I found.
Change-Id: Ic5d837c2d620b15d7a831823e94e70080f5e5324
Reviewed-on: https://go-review.googlesource.com/c/go/+/173948
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Expand "go version" from printing just the version of the
go command itself to being able to print the version of any
go binary (when listed on the command line).
This is a simplified version of rsc.io/goversion, which will
now be deprecated in favor of "go version".
(Preparatory changes to runtime, cmd/go, and cmd/link
made this version information easier to find, allowing a
much simpler implementation than in rsc.io/goversion.)
Fixes#31624.
Change-Id: I5f6f5b8392ce86a617e32ce287f6bf2aceafccff
Reviewed-on: https://go-review.googlesource.com/c/go/+/173343
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
It is useful to be able to dig the Go version out of a binary,
even a stripped binary. rsc.io/goversion does this for x86
binaries by disassembling the binary to find runtime/proc.go's
startup-time reference to runtime.buildVersion's address.
That approach is quite fragile: the implementation doesn't
work for non-x86 and must be updated as the generated
code changes.
rsc.io/goversion finds the module version string by looking
for random 16-byte framing around the actual string.
This is less fragile but fairly kludgy and requires scanning
the entire data segment.
cmd/buildid finds the build ID by looking for an ELF note
or else falling back to scanning the beginning of the text
segment for a magic string. This has proved quite reliable
and doesn't require scanning much of the binary.
This CL makes it possible to find the Go and module versions
using a scan more like the build ID scan: a symbol early in
the writable data segment starts with a magic number and
then has pointers to the two string variables.
Setting up for "go version <binary>".
For #31624.
Change-Id: I78ea8c52fe1686b5cc5a829ca5f198104d10ebf0
Reviewed-on: https://go-review.googlesource.com/c/go/+/173342
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When reading symbol names from an object file, if a name does not
need fixup (conversion of "". to package path), then generate
a string whose backing store is in read-only memory (from an mmap
of the object file), avoiding the need for an allocation. This
yields a modest reduction in total linker heap use.
Change-Id: I95719c93026b6cc82eb6947a9d14063cf3a6679c
Reviewed-on: https://go-review.googlesource.com/c/go/+/173938
Run-TryBot: Than McIntosh <thanm@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Ps are strictly numbered from 0 to GOMAXPROCS-1, so if procresize
happens to be running on a P that's being destroyed, it moves itself
to P 0.
However, currently procresize destroys the unused Ps *before* moving
itself to P 0. This means it may briefly run on a destroyed P. This is
basically harmless, but has at least one very confusing consequence:
since destroying a P has write barriers, it may enqueue pointers to a
destroyed write barrier buffer. As far as I can tell, there are no
negative consequences of this, but this seems really fragile.
This CL swaps the order of things, so now procresize moves itself to P
0 if necessary before destroying Ps. This ensures it always has a
valid P.
This is part of refactoring for #10958 and #24543, but is a good
cleanup regardless.
Change-Id: I91a23dd6ed27e372f8d6291feec9bc991bcf9812
Reviewed-on: https://go-review.googlesource.com/c/go/+/173941
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
We're about to change some of these rules, so it's about time we wrote
them down!
For #10958, #24543.
Change-Id: I3efce0c44b53bfb6f31ce2d299809b2b4eb329f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/172857
Reviewed-by: Russ Cox <rsc@golang.org>
This does *not* update the vendored tables.
A commit updating these tables should follow soon,
Mostly generated running UNICODE_VERSION=11.0.0 in x/text.
Manually updated next.txt file.
Updates golang/go#27945.
Change-Id: I939a01e235aeca898ee9afc99a531e7ad8444e12
Reviewed-on: https://go-review.googlesource.com/c/go/+/154420
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
This change modifies the implementation of mTreap.find to find the
best-fit span with the lowest possible base address.
Fixes#31616.
Change-Id: Ib4bda0f85d7d0590326f939a243a6e4665f37d3f
Reviewed-on: https://go-review.googlesource.com/c/go/+/173479
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
When the linker's -X flag is used, it will overwrite the symbol's
content (sym.P) in addstrdata. The symbol's content may be in
read-only memory, in which case overwriting it will fault. Do
copy-on-write to fix this.
Change-Id: I34d583f44c30d187042757e19a14c1ef7d3e613c
Reviewed-on: https://go-review.googlesource.com/c/go/+/173937
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Following discussion on golang.org/issue/26334, this CL changes
the GOPROXY environment setting to be a list of proxies, tried in
sequence. The first successful or non-404/410 error is taken as
authoritative. Otherwise the next proxy is tried, and so on.
As in earlier releases, GOPROXY=direct means "connect directly",
but now it can appear in a longer list as well.
This will let companies run a proxy holding only their private modules
and let users set GOPROXY=thatproxy,publicproxy or GOPROXY=thatproxy,direct
to fall back to an alternate mechanism for fetching public modules.
Fixes#26334.
Change-Id: I642f0ae655ec307d9cdcad0830c0baac8670eb9c
Reviewed-on: https://go-review.googlesource.com/c/go/+/173441
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
- rename PasswordRedacted to Redacted
- move URL into Response in redacted form, remove from Get result list
- add Response.Err to construct non-200 errors
(otherwise GetBytes is not just a wrapper)
- make 404/410 errors satisfy Is(err, os.ErrNotExist)
Change-Id: Id15899c1e3dfd30cffb1a75ba79a9a1999913258
Reviewed-on: https://go-review.googlesource.com/c/go/+/173717
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
CL 168597 added IsNotFound field to DNSError.
However, this change broke TestLookupNonLDH on Plan 9
because LookupHost is expected to return a DNSError,
while on Plan 9, it returned an error string.
This change fixes the implementation of lookupHost on
Plan 9 to return a DNSError instead of an error string.
Fixes#31672.
Change-Id: Ia805c8965af63ddee7ccfdebb9462a5502b0269d
Reviewed-on: https://go-review.googlesource.com/c/go/+/173857
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
For powers of two (c=1<<k), the divisibility check x%c == 0 can be made
just by checking the trailing zeroes via a mask x&(c-1) == 0 even for signed
integers. This avoids division fix-ups when just divisibility check is needed.
To apply this rule, we match on the fixed-up version of the division. This is
neccessary because the mod and division rewrite rules are already applied
during the initial opt pass.
The speed up on amd64 due to elimination of unneccessary fix-up code is ~55%:
name old time/op new time/op delta
DivconstI64-4 2.08ns ± 0% 2.09ns ± 1% ~ (p=0.730 n=5+5)
DivisiblePow2constI64-4 1.78ns ± 1% 0.81ns ± 1% -54.66% (p=0.008 n=5+5)
DivconstU64-4 2.08ns ± 0% 2.08ns ± 0% ~ (p=0.683 n=5+5)
DivconstI32-4 1.53ns ± 0% 1.53ns ± 1% ~ (p=0.968 n=4+5)
DivisiblePow2constI32-4 1.79ns ± 1% 0.81ns ± 1% -54.97% (p=0.008 n=5+5)
DivconstU32-4 1.78ns ± 1% 1.80ns ± 2% ~ (p=0.206 n=5+5)
DivconstI16-4 1.54ns ± 2% 1.54ns ± 0% ~ (p=0.238 n=5+4)
DivisiblePow2constI16-4 1.78ns ± 0% 0.81ns ± 1% -54.72% (p=0.000 n=4+5)
DivconstU16-4 1.00ns ± 5% 1.01ns ± 1% ~ (p=0.119 n=5+5)
DivconstI8-4 1.54ns ± 0% 1.54ns ± 2% ~ (p=0.571 n=4+5)
DivisiblePow2constI8-4 1.78ns ± 0% 0.82ns ± 8% -53.71% (p=0.008 n=5+5)
DivconstU8-4 0.93ns ± 1% 0.93ns ± 1% ~ (p=0.643 n=5+5)
A follow-up CL will address the general case of x%c == 0 for signed integers.
Updates #15806
Change-Id: Iabadbbe369b6e0998c8ce85d038ebc236142e42a
Reviewed-on: https://go-review.googlesource.com/c/go/+/173557
Run-TryBot: Brian Kessler <brian.m.kessler@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
We already skipped blank field initialization in non-global contexts.
This change makes the global context treatment match.
Fixes#31546
Change-Id: I40acce49b0a9deb351ae0da098f4c114e425ec63
Reviewed-on: https://go-review.googlesource.com/c/go/+/173723
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The compiler couldn't quite see that reading p[2] and p[3] was safe.
This change provides a few hints to help it.
First, make sz an int throughout, rather than just when checking the input length.
Second, use <= instead of == in later comparisons.
name old time/op new time/op delta
DecodeASCIIRune-8 2.62ns ± 3% 2.60ns ± 5% ~ (p=0.126 n=18+19)
DecodeJapaneseRune-8 4.46ns ±10% 4.01ns ± 5% -10.00% (p=0.000 n=19+20)
Change-Id: I2f78a17e38156fbf8b0f5dd6c07c20d6a47e9209
Reviewed-on: https://go-review.googlesource.com/c/go/+/173662
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We were using hex literals and had the binary literal in a comment.
When I was working with this code, I always referred to the comment.
That's an indicator that we should just use the binary literal directly.
Updates #19308
Change-Id: I2279cb8efb4ae5f2e1558c15979058ab09eb4f6f
Reviewed-on: https://go-review.googlesource.com/c/go/+/173663
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This pair of packages caused a crash in gollvm, due to a glitch in the
way the front end handles empty/non-name parameters for functions that
are inline candidates.
Updates #31637.
Change-Id: I571c0658a00974dd36025e571638c0c836a3cdfd
Reviewed-on: https://go-review.googlesource.com/c/go/+/173617
Run-TryBot: Benny Siegert <bsiegert@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Revise the fix for #31540 to take into account the possibility that we
may see an alias to a name that has already been installed into
package scope. This scenario is not possible to reproduce with the
current importer unit tests; changes to the harness to enable this
scenario will be added in a later CL.
Updates #31540.
Change-Id: Ie155d5e0b998604177a78471cba2413f57d40229
Reviewed-on: https://go-review.googlesource.com/c/go/+/173440
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Go on openbsd/arm has supported external linking for a while now, so
re-enable the external linking related tests that were previously
disabled.
Fixes#10619
Change-Id: I304eeabf3b462d53b7feda17ae390bbe2fa22069
Reviewed-on: https://go-review.googlesource.com/c/go/+/173597
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
OpenBSD/arm64 reports itself as arm64 from `uname -m` - this currently
matches as gohostarch=arm, rather than gohostarch=arm64. Correct this
by matching on both aarch64 and arm64 (the alternative would be to use
`uname -p`, however that's likely to cause upset elsewhere).
Updates #31656
Change-Id: I7b2da93495d808e704caf032b3fadc984077769e
Reviewed-on: https://go-review.googlesource.com/c/go/+/173598
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
The changes in Unicode 11 exposes a bug in maketables.go.
We update the Unicode 10 tables using a new generator
to minimize the changes upgrading to Unicode 11.
This change switches over the generation from core to that in
x/text. To properly update the tables one needs to run the generate
in x/text anyway, so this makes that official.
The RangeTable generator in x/text also generates slightly compacter
tables.
Updates golang/go#27945
See CL 154443
Change-Id: I6c59e082d5b8cd9e9332a32d8971061228581d66
Reviewed-on: https://go-review.googlesource.com/c/go/+/169617
Run-TryBot: Marcel van Lohuizen <mpvl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Using commas makes it possible to put multiple tags into GOFLAGS.
The space-separated form is still recognized and will be maintained.
Alleviates #26849 somewhat.
Fixes#18800 (again).
Change-Id: I6f4cf28ea31e53e21ccbdad6ef1a0aee63b007d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/173438
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Ian Cottrell <iancottrell@google.com>
"go build -trimpath" trims the recorded file paths in the
resulting packages and executables to avoid recording
the names of any local directories. Instead, the files appear
to be stored in directories named either "go/src/..." (for the
standard library) or named after the module or package
in which the files appear.
Fixes#16860.
Change-Id: I433afeeb1fdeea641286b21693fee5e0a66d607e
Reviewed-on: https://go-review.googlesource.com/c/go/+/173345
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
This CL affects the low-level -trimpath flag provided
by both cmd/asm and cmd/compile. Previously, the flag
took the name of a single directory that would be trimmed
from recorded paths in the resulting object file.
This CL makes the flag take a semicolon-separated list of paths.
Further, each path can now end in an optional "=>replacement"
to specify what to replace that leading path prefix with,
instead of only dropping it.
A followup CL will add a mode to cmd/go that uses this
richer -trimpath to build binaries that do not contain any
local path names.
For #16860.
Change-Id: I246811750f37607c7f7a8fbecd56c5475ebe1ea5
Reviewed-on: https://go-review.googlesource.com/c/go/+/173344
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
The compiler has advanced enough that it is cheaper
to convert to strings than to go through the assembly
trampolines to call runtime.memequal.
Simplify Equal accordingly, and cull dead code from bytealg.
While we're here, simplify Equal's documentation.
Fixes#31587
Change-Id: Ie721d33f9a6cbd86b1d873398b20e7882c2c63e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/173323
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Fixes panic on nil pointer dereference error when assigning module's path and version pair to fileJSON.
Fixes#31623
Change-Id: I3f61122ba0676a1270d3ad98900af8c8e9c90935
Reviewed-on: https://go-review.googlesource.com/c/go/+/173397
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
