Now that we write ELF relocation records in mapped memory with
known sizes and offsets, we can write them in parallel.
Further speed up Asmb2 pass. Linking cmd/compile with external
linking,
Asmb2 141ms ± 4% 97ms ± 5% -30.98% (p=0.000 n=10+9)
Change-Id: I52c2b9230e90ed4421c21d7ef13a4f1e996f6054
Reviewed-on: https://go-review.googlesource.com/c/go/+/240400
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Currently, ELF relocations are generated sequentially in the heap
and flushed to output file periodically. In fact, in some cases,
the output size of the relocation records can be easily computed,
as a relocation entry has fixed size. We only need to count the
number of relocation records to compute the size.
Once the size is computed, we can mmap the output with the proper
size, and directly write relocation records in the mapped memory.
It also opens the possibility of writing relocations in parallel
(not done in this CL).
Note: on some architectures, a Go relocation may turn into
multiple ELF relocations, which makes size calculation harder.
This CL does not handle those cases, and it still writes
sequentially in the heap there.
Linking cmd/compile with external linking,
name old time/op new time/op delta
Asmb2 190ms ± 2% 141ms ± 4% -25.74% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
Asmb2_GC 66.8MB ± 0% 8.2MB ± 0% -87.79% (p=0.008 n=5+5)
name old live-B new live-B delta
Asmb2_GC 66.9M ± 0% 55.2M ± 0% -17.58% (p=0.008 n=5+5)
Change-Id: If7056bbe909dc90033eef6b9c4891fcca310602c
Reviewed-on: https://go-review.googlesource.com/c/go/+/240399
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
If mmap is used, we don't do file I/O anymore, so no need to Seek
in the file either.
Change-Id: Ic8350eb66404398420df18560f6a7acbb3fcf7e3
Reviewed-on: https://go-review.googlesource.com/c/go/+/240398
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Also crypto/tls.Config.BuildNameToCertificate.
Note that this field and method were deprecated in the Go 1.14 release,
so this change is to the 1.14 release notes.
Fixes#37626
Change-Id: If8549bc746f42a93f1903439e1b464b3e81e2c19
Reviewed-on: https://go-review.googlesource.com/c/go/+/240005
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Now that we removed the "weird thing" about runtime.etext symbol,
we can remove this special case.
Change-Id: I2e4558367758d37e898a802bcd30671c7dd6fe89
Reviewed-on: https://go-review.googlesource.com/c/go/+/240066
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Currently, on most platforms, the start/end symbols runtime.text
and runtime.etext are defined in symtab pass and assigned values
in address pass. In some cases (darwin+dynlink or AIX+external),
however, they are defined and assigned values in textaddress pass
(because they need non-zero sizes). Then their values get
overwritten in address pass. This is bad. The linker expects
their values to be consistent. In particular, in CL 239281,
findfunctab is split to two parts. The two parts need to have a
consistent view of the start/end symbols. If its value changes in
between, bad things can happen.
This CL fixes it by always defining runtime.text/etext symbols in
the textaddress pass.
Fix darwin and AIX builds.
Change-Id: Ifdc1bcb69d99be1b7e5b4fd31d473650c03e3b9d
Reviewed-on: https://go-review.googlesource.com/c/go/+/240065
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Create a new class of symbols internal to the linker. These symbols live
in the Loader, and are real smybols, but have no data, only size. After
symbols are allocated in the binary in asmb() a function is called that
is responsible for filling in the data.
This allows the linker to create large symbols, but not pay the price on
the heap memory.
Change-Id: Ib4291fc6e578478057ed2ec163d7b27426f1d5ff
Reviewed-on: https://go-review.googlesource.com/c/go/+/239280
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
It doesn't have to. The type in the aux field is authoritative.
There are cases involving casting from interface{} where pointers
have a placeholder pointer type (because the type is not known when
the IData op is generated).
The check was introduced in CL 13447.
Fixes#39459
Change-Id: Id77a57577806a271aeebd20bea5d92d08ee7aa6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/239817
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
File.Sync was returning *SyscallError instead of *PathError on Plan 9.
Adjust the error type to match other systems.
Fixes#39800
Change-Id: I844e716eb61c193ef78d29cb0b4a3ef790bb3320
Reviewed-on: https://go-review.googlesource.com/c/go/+/239857
Reviewed-by: David du Colombier <0intro@gmail.com>
The Go compiler includes special treatment for a small set of very
commonly used type symbols (26 to be exact); for these types it
doesn't bother to emit type descriptors for "normal" compilations, and
instead only generates them for the runtime package, so as to reduce
object file bloat.
This patch moves the set of type symbols in question from the
PkgIdxNone index space (in the object file) to the PkgIdxBuiltin
space, which saves some work in the compiler and loader (reduces each
package's index space slightly).
Change-Id: I039c805e05c1aef26f035e52760fd0a0af40f7a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/239658
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Change the object file writer to avoid adding entries to the object
file string table for builtin functions. This helps save some very
small amount of space in the object file.
Change-Id: Ic3b94a154e00eb4c7378b57613580c7073b841bc
Reviewed-on: https://go-review.googlesource.com/c/go/+/239657
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Now that we've added a os.File.ReadFrom method, io.CopyBuffer to a
os.File will no longer use the provided buffer.
For #16474
For #36817
For #37419
Change-Id: I79a3bf778ff93eab88e88dd9ecbb8c7ea101e868
Reviewed-on: https://go-review.googlesource.com/c/go/+/238864
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
After CL 228645 some mentions of the Deadline methods referred
to the Timeout method, and some to os.ErrDeadlineExceeded.
Stop referring to the Timeout method, to encourage ErrDeadlineExceeded.
For #31449
Change-Id: I27b8ff34f31798f38b06437546886af8cce98ca4
Reviewed-on: https://go-review.googlesource.com/c/go/+/239705
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Damien Neil <dneil@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Switched the generator to using the open source releases of the root
store rather than HTML parsing, while trying to emulate the sorting
algorithm of the table to reduce churn.
Updates #38843
Change-Id: I78608d245eabc2a35c2f98635ed5f1a531ad2ba8
Reviewed-on: https://go-review.googlesource.com/c/go/+/239557
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
On AIX, in relocsym we call Xcoffadddynrel, which adds a
relocation record to a global array. relocsym already runs in
parallel. In the past we only parallelize over segments, and
we call Xcoffadddynrel only for symbols in data segment, so it is
effectively called sequentially. In CL 239197 we started to do
more fine-grained parallelism, so we need to make sure it is safe
to call Xcoffadddynrel in parallel.
Fix AIX build.
Change-Id: I3128193995a5a99d9fa04c8e728e590f17298da3
Reviewed-on: https://go-review.googlesource.com/c/go/+/239561
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
The code in the compiler's DWARF line table generation emits line
table ops at the end of each function fragment to reset the state
machine registers back to an initial state, so that when the line
table fragments for each function are stitched together into a
compilation unit, each fragment will have a clean starting point. The
set-file/set-line ops emitted in this code were being applied to the
last row of the line table, however, meaning that they were
overwriting the existing values.
To avoid this problem, add code to advance the PC past the end of the
last instruction in the function, and switch to just using an
end-of-sequence operator at the end of each function instead of
explicit set-file/set-line ops.
Updates #39757.
Change-Id: Ieb30f83444fa86fb1f2cd53862d8cc8972bb8763
Reviewed-on: https://go-review.googlesource.com/c/go/+/239286
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Normally, packages are loaded in dependency order, and if a
Library object is not nil, it is already loaded with the actual
fingerprint. In shared build mode, however, packages may be added
not in dependency order (e.g. go install -buildmode=shared std
adds all std packages before loading them), and it is possible
that a Library's fingerprint is not yet loaded. Skip the check
in this case (when the fingerprint is the zero value).
Fixes#39777.
Change-Id: I66208e92bf687c8778963ba8e33e9bd948f82f3a
Reviewed-on: https://go-review.googlesource.com/c/go/+/239517
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Change should be non-functional.
Change-Id: I8ac835762a1aa6458d896b8815cd2d87333b55ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/239279
Run-TryBot: Jeremy Faller <jeremy@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
We can apply relocations of a symbol right after the symbol data
is copied to output buffer. This should help locality and
parallelism (parallelizing over blocks, instead of over segments).
Linking cmd/compile,
Asmb+Reloc 23.9ms ±18% 16.5ms ±11% -30.73% (p=0.008 n=5+5)
Linking cmd/compile with external linking,
Asmb+Reloc 74.0ms ± 3% 33.8ms ± 8% -54.32% (p=0.008 n=5+5)
In external linking mode, allocation goes up slightly, as we do
smaller batching now. It doesn't seem too bad.
Asmb+Reloc 15.0MB ± 0% 16.7MB ± 0% +11.22% (p=0.008 n=5+5)
Change-Id: Ide33d9ff86c39124c8f5cfc050d7badc753a1ced
Reviewed-on: https://go-review.googlesource.com/c/go/+/239197
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Follow glibc's implementation and check secondary group memberships
using Getgroups.
No test since we cannot easily change file permissions when not running
as root and the test is meaningless if running as root.
Same as CL 238722 did for x/sys/unix
Updates #39660
Change-Id: I6af50e27b255e33405558947a0ab3dfbc33b2d50
Reviewed-on: https://go-review.googlesource.com/c/go/+/238937
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
addpersrc is called very late, after we have converted to
sym.Symbols and various fields in loader representation have been
dropped. Use the Symbol representation there.
Fixes#39658.
Change-Id: I616e838655b6f01554644171317e2cc5cefabf39
Reviewed-on: https://go-review.googlesource.com/c/go/+/238779
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
The preceding paragraph suggests the test run will produce a file called trace.out.
The same name, trace.out, is used in the output from go help testflag, thus we change the go test line instead of changing the preceding paragraph.
Change-Id: Ib1fa7e49e540853e263a2399b16040ea6f41b703
GitHub-Last-Rev: 3535e62bf8
GitHub-Pull-Request: golang/go#39709
Reviewed-on: https://go-review.googlesource.com/c/go/+/238997
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
In os.Getenv and os.Setenv, instead of directly reading and writing the
Plan 9 environment device (which may be shared with other processes),
use a local copy of environment variables cached at the start of
execution. This gives the same semantics for Getenv and Setenv as on
other operating systems which don't share the environment, making it
more likely that Go programs (for example the build tests) will be
portable to Plan 9.
This doesn't preclude writing non-portable Plan 9 Go programs which make
use of the shared environment semantics (for example to have a command
which exports variable definitions to the parent shell). To do this, use
ioutil.ReadFile("/env/"+key) and
ioutil.WriteFile("/env/"+key, value, 0666)
in place of os.Getenv(key) and os.Setenv(key, value) respectively.
Note that CL 5599054 previously added env cacheing, citing efficiency
as the reason. However it made the cache write-through, with Setenv
changing the shared environment as well as the cache (so not consistent
with Posix semantics), and Clearenv breaking the sharing of the
environment between the calling thread and other threads (leading to
unpredictable behaviour). Because of these inconsistencies (#8849),
CL 158970045 removed the cacheing again.
This CL restores cacheing but without write-through. The local cache is
initialised at start of execution, manipulated by the standard functions
in syscall/env_unix.go to ensure the same semantics, and exported only
when exec'ing a new program.
Fixes#34971Fixes#25234Fixes#19388
Updates #38772
Change-Id: I2dd15516d27414afaf99ea382f0e00be37a570c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/236520
Run-TryBot: David du Colombier <0intro@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Fazlul Shahriar <fshahriar@gmail.com>
Reviewed-by: David du Colombier <0intro@gmail.com>
reflect.assignTo writes to the target using write barriers. Make sure
that the memory it is writing to is zeroed, so the write barrier does
not read pointers from uninitialized memory.
Fixes#39541
Change-Id: Ia64b2cacc193bffd0c1396bbce1dfb8182d4905b
Reviewed-on: https://go-review.googlesource.com/c/go/+/238760
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Fixes the *noov opcodes so they handle a constant argument properly.
Most of the infrastructure for this CL is in CL 238077 (the arm32 one).
Fixes#39505
Change-Id: Id424a4e18964b848f05aa42f4d78e5f2e2cdf43b
Reviewed-on: https://go-review.googlesource.com/c/go/+/237999
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Encode the flag results in an auxint field instead of having
one opcode per flag state. This helps us handle the new *noov
branches in a unified manner.
This is only for arm, arm64 is in a subsequent CL.
We could extend to other architectures as well, athough it would
only be cleanup, no behavioral change.
Update #39505
Change-Id: Ia46cea596faad540d1496c5915ab1274571543f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/238077
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
These conversion instructions set the condition code and so should
be marked as clobbering flags.
Fixes#39651.
Change-Id: I91cc9687ea70ef0551bb3139c1875071c349d43e
Reviewed-on: https://go-review.googlesource.com/c/go/+/238628
Run-TryBot: Michael Munday <mike.munday@ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This updates the ppc64 asm doc file, including information on
updates to the objdump, correcting information on operand order,
and adding some information on shifts.
Change-Id: Ib8ed53eac86c2121ea5b657c361ad92aae31cb32
Reviewed-on: https://go-review.googlesource.com/c/go/+/238237
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
By keeping a tail pointer, we can append to a patchList in constant
time, rather than in time proportional to the length of the list. This
gets rid of the quadratic compile times we were seeing for long series
of alternations.
This is basically the same change as
e9d517989f.
Fixes#39542.
Change-Id: Ib4ca0ca9c55abd1594df1984653c7d311ccf7572
Reviewed-on: https://go-review.googlesource.com/c/go/+/238079
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Ensure that the exact Request passed to Transport.RoundTrip
is returned in the Response. Do not replace the Request with
a copy when resetting the request body.
Fixes#39533
Change-Id: Ie6fb080c24b0f6625b0761b7aa542af3d2411817
Reviewed-on: https://go-review.googlesource.com/c/go/+/237560
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
