The dev.link branch was not sync'd with the new DWARF compilation
unit logic change on the master branch, and the new object file
format didn't support this.
This CL adds the new DWARF CU and file table support to the new
object file format. In the old object file, the DWARF file table
is a separate section. For now, we do the same with the new
object file, keeping it as a separate block.
While here, also refactor the loader code so it is easier for the
loader to carry per-object informations.
Change-Id: I4c317941fc0a5831acbc11ce8c2a8b7421471372
Reviewed-on: https://go-review.googlesource.com/c/go/+/198198
Reviewed-by: Austin Clements <austin@google.com>
With the old object file format, we use mmap (if supported) to
read object files and back symbol data with mapped read-only
memory.
Do the same with the new object file format. This also
significantly reduces number of syscalls made to read object
files.
Currently we still do mmap in object file level, not archive
level. This is probably ok, as there shouldn't be many archives
that contain more than one object. If this is a problem we can
change that later.
Change-Id: Icae3ef14d8ed6adbee1b5b48d420e2af22fd9604
Reviewed-on: https://go-review.googlesource.com/c/go/+/197797
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
If the new object file format is used, now we load full symbol
contents after the deadcode pass, for reachable symbols only.
We still load some informations early, like relocations and the
contents of type symbols, which are used in the deadcode pass.
If we rewrite deadcode to use index directly, we could delay more
of the loading (to sym.Symbol), and perhaps delay the creation of
sym.Symbol.
TODO: internal linking with host objects doesn't work yet.
Change-Id: I7d4880e8f150e8709ffac277e62191623440e4cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/197258
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Currently we add all text symbols to ctxt.Textp at load time,
then the deadcode pass filters out unreachable ones. This CL
delays adding symbols to ctxt.Textp to the end of the deadcode
pass, where we only add reachable ones.
Change-Id: Ie83b2958f915c5aaa004b8c5ed1f1bc275f4d1db
Reviewed-on: https://go-review.googlesource.com/c/go/+/197257
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Follow-up of Than's comment on CL 196032. For runtime.gcbits
symbols, the compiler already sets the "local" attribute
(cmd/compile/internal/gc/reflect.go:dgcptrmask). No need of the
special handling in the linker.
Change-Id: I7c53a818ed0375083f04655f3565c1c9c45f2401
Reviewed-on: https://go-review.googlesource.com/c/go/+/199642
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
This is in preparation of delaying loading symbol content to
after dead code elimination.
Change-Id: Ia4ce82d9a42cc1a76c82f6d0eb9a54b0325ee3ba
Reviewed-on: https://go-review.googlesource.com/c/go/+/197061
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Calls are code-generated in an alternate path that inherits
its positions from values, not from *SSAGenState. The
default position on *SSAGenState was marked as not-a-statement,
but this was not applied to the value itself, leading to
spurious "is statement" marks in the output (convention:
after code generation in the compiler, everything is either
definitely a statement or definitely not a statement, nothing
is in the undetermined state).
This CL causes a 35 statement regression in ssa/stmtlines_test.
This is down from the earlier 150 because of all the other
CLs preceding this one that deal with the root causes of the
missing lines (repeated lines on nested calls hid missing lines).
This also removes some line repeats from ssa/debug_test.
Change-Id: Ie9a507bd5447e906b35bbd098e3295211df2ae01
Reviewed-on: https://go-review.googlesource.com/c/go/+/188018
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
This was a cause of some statements being lost.
Change-Id: I81c95dcf3df6ed8a03b7578a27f9b21d33b3cf39
Reviewed-on: https://go-review.googlesource.com/c/go/+/198484
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
We replace the existing file with a thin wrapper around its target so
that we don't break anybody's workflow.
Updates #34388
Change-Id: I0d00371c483cb78f4be18fe987df33c79cd40f05
Reviewed-on: https://go-review.googlesource.com/c/go/+/198977
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This reduces the number of missing-statement lines.
Change-Id: Iefa56c2a253220d17d8b53210c8c6af78ee68756
Reviewed-on: https://go-review.googlesource.com/c/go/+/198483
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
This reverts CL 183841.
Fixes#34533
Reason for revert: Introduced a significant performance regression for repos with many incompatible-version tags.
Change-Id: I75d7fd76e6e1a0902b114b00167b38439e0f8221
Reviewed-on: https://go-review.googlesource.com/c/go/+/198699
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
This was a cause of some statements being lost.
Change-Id: Ia4805c2dafd7a880d485a678a48427de8930d57e
Reviewed-on: https://go-review.googlesource.com/c/go/+/198482
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
In the local package build process, when -o is pointing to an existing folder, the object
the filename is generated from files listed on the command line like when the -o is
not pointing to a folder instead of using the `importPath` that is going to be `command-line-arguments`
Fixes#34535
Change-Id: I09a7609c17a2ccdd83da32f01247c0ef473dea1e
GitHub-Last-Rev: b3224226a3
GitHub-Pull-Request: golang/go#34562
Reviewed-on: https://go-review.googlesource.com/c/go/+/197544
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Statement markers on rematerializable values were getting lost in
register allocation. This checks for that case (rematerializable
input and using value share line number, but mark is on the input)
and preserves the mark.
When combined with other CLs in this series, this CL reduces the
"nostmt" count (a line appears in the assembly, but no statement
marker) for cmd/go from 413 to 277. The rematerialized input is
usually a LEAQ (on AMD64).
The cause is "complicated"; for example, a NilCheck originally has the
statement mark (a good thing, if the NilCheck remains) but the
NilCheck is removed and the mark floats to a Block end, then to a
SliceMake. The SliceMake decomposes and goes dead without preserving
its marker (its component values are elided in other rewrites and may
target inputs with different line numbers), but before deadcode
removes it from the graph it moves the mark to an input, which at that
time happens to be a LocalAddr. This eventually transforms to a LEAQ.
Change-Id: Iff91fc2a934357fb59ec46ac87b4a9b1057d9160
Reviewed-on: https://go-review.googlesource.com/c/go/+/198480
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
OpPhi nodes tend to disappear or get rearranged,
and cause statement marks to vanish.
Change-Id: I2f5a222903b7fcd0d1a72e8f6d7e156036b23f30
Reviewed-on: https://go-review.googlesource.com/c/go/+/198481
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
According to spec, the hash must be truncated, but crypto/dsa
does not do it. We can't fix it in crypto/dsa, because it would break
verification of previously generated signatures.
In crypto/x509 however, go can't generate DSA certs, only verify them,
so the fix here should be safe.
Fixes#22017
Change-Id: Iee7e20a5d76f45da8901a7ca686063639092949f
GitHub-Last-Rev: 8041cde8d2
GitHub-Pull-Request: golang/go#34630
Reviewed-on: https://go-review.googlesource.com/c/go/+/198138
Reviewed-by: Filippo Valsorda <filippo@golang.org>
'GOFLAGS=-mod=vendor' currently causes 'go get' to always fail unless
the '-mod' flag is explicitly overwritten. Moreover, as of CL 198319
we plan to set -mod=vendor by default if a vendor directory is
present, so all users with vendor directories will be affected — not
just those who set 'GOFLAGS' explicitly.
Similarly, an explicit '-mod=readonly' argument to 'go get' is
currently ignored as a special case, but the fact that it is ignored
(rather than rejected) can be very surprising.
Rather than adding more special cases, we should remove the '-mod'
flag from 'go get' entirely.
Fixes#30345Fixes#32502
Updates #33848
Change-Id: Iecd3233ca3ef580ca3a66bd5e6ee8d86d4cbd8a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/198438
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Parse new object file format in the linker. At least we can link
a hello-world program.
Add a basic "loader", which handles symbol references in the
object file.
- mapping between local and global indices
- resolve by-name references
(TODO: the overwrite logic isn't implemented yet)
Currently we still create sym.Symbol rather early, and, after all
the object files are loaded and indexed references are resolved,
add all symbols to sym.Symbols.
The code here is probably not going in the final version. This
is basically only for debugging purposes -- to make sure the
writer and the reader work as expected.
Change-Id: I895aeea68326fabdb7e5aa1371b8cac7211a09dd
Reviewed-on: https://go-review.googlesource.com/c/go/+/196032
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
CL 188317 introduced a compiler crash during dwarf generation which
was reported as Issue #34520. After CL 188217, the issue appears to be
fixed. Add a testcase to avoid future regressions.
Fixes#34520
Change-Id: I73544a9e9baf8dbfb85c19eb6d202beea05affb6
Reviewed-on: https://go-review.googlesource.com/c/go/+/198546
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Passing ambiguous patterns, ending in `.go`, to `go list` results in them
being interpreted as Go files despite potentially being package references.
This can then result in errors on other package references.
The parsing logic is modified to check for a locally present file
corresponding to any pattern ending in `.go`. If no such file is present
the pattern is considered to be a package reference.
We're also adding a variety of non-regression tests that fail with the
original parsing code but passes after applying the fix.
Fixes#32483Fixes#34653
Change-Id: I073871da0dfc5641a359643f95ac14608fdca09b
GitHub-Last-Rev: 5abc200103
GitHub-Pull-Request: golang/go#34663
Reviewed-on: https://go-review.googlesource.com/c/go/+/198459
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
The existing text was hard to parse.
Shorten the sentences and simplify the text.
Change-Id: Ic16f486925090ea303c04e70969e5a4b27a60896
Reviewed-on: https://go-review.googlesource.com/c/go/+/198758
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Add support of parsing new object file format. We use the new
parser if the magic string matches the new one, otherwise use the
old one.
The parsed data are still filled into the current goobj API. In
the future we may consider to change the goobj API to a close
match of the object file data.
Now objdump and nm commands support new object file format.
For a reference to a symbol defined in another package, with the
new object file format we don't know its name. Write it as
pkg.<#nn> for now, where nn is its symbol index.
Change-Id: I06d05b2ca834ba36980da3c5d76aee16c3b0a483
Reviewed-on: https://go-review.googlesource.com/c/go/+/196031
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
Nilcheck would move statements from NilCheck values to others that
turned out were already dead, which leads to lost statements. Better
to eliminate the dead code first.
One "error" is removed from test/prove.go because the code is
actually dead, and the additional deadcode pass removes it before
prove can run.
Change-Id: If75926ca1acbb59c7ab9c8ef14d60a02a0a94f8b
Reviewed-on: https://go-review.googlesource.com/c/go/+/198479
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
The earlier code was picking nodes that were "poor choices" and
thus sometimes losing statements altogether.
Change-Id: Ibe5ed800ffbd3c926c0ab1bc10c77d72d3042e45
Reviewed-on: https://go-review.googlesource.com/c/go/+/198478
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Aggregate-making nodes that are later decomposed
are poor choices for statements, because the decomposition
phase turns them into multiple sub-values, some of which may be
dead. Better to look elsewhere for a statement mark.
Change-Id: Ibd9584138ab3d1384548686896a28580a2e43f54
Reviewed-on: https://go-review.googlesource.com/c/go/+/198477
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The previous version used the position of the switch statement,
which makes for potentially jumpy stepping and introduces a large
number of statements repeating the line (tricky for inserting
breaks). It also shared a single OBREAK node and this was not
really a syntax "tree".
This improves both the nostmt test (by 6 lines) and
reduces the total badness score from dwarf-goodness (by about 200).
Change-Id: I1f71b231a26f152bdb6ce9bc8f95828bb222f665
Reviewed-on: https://go-review.googlesource.com/c/go/+/188218
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
1) An empty block is treated as not-a-statement unless its line differs
from at least one of its predecessors (it might make sense to
rearrange branches in predecessors, but that is a different issue).
2) When iterating forward to choose a "good" place for a statement,
actually check that the chosen place is in fact good.
3) Refactor same line and same file into methods on XPos and Pos.
This reduces the failure rate of ssa/stmtlines_test by 7-ish lines.
(And interacts favorably with later debugging CLs.)
Change-Id: Idb7cca7068f6fc9fbfdbe25bc0da15bcfc7b9d4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/188217
Run-TryBot: David Chase <drchase@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Makes it easier to run
go build -a -gcflags=-d=export std |& grep ^BenchmarkExportSize
and get useful output for feeding into benchstat.
Change-Id: I2b52e8f5ff33b7ccb6c25b18e464513344bd9ad9
Reviewed-on: https://go-review.googlesource.com/c/go/+/198698
Reviewed-by: Robert Griesemer <gri@golang.org>
If -newobj is set, write object file in new format, which uses
indices for symbol references instead of symbol names. The file
format is described at the beginning of
cmd/internal/goobj2/objfile.go.
A new package, cmd/internal/goobj2, is introduced for reading and
writing new object files. (The package name is temporary.) It is
written in a way that trys to make the encoding as regular as
possible, and the reader and writer as symmetric as possible.
This is incomplete, and currently nothing will consume the new
object file.
Change-Id: Ifefedbf6456d760d15a9f40a28af6486c93100fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/196030
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
During package initialization, the compiler tries to optimize:
var A = "foo"
var B = A
into
var A = "foo"
var B = "foo"
so that we can statically initialize both A and B and skip emitting
dynamic initialization code to assign "B = A".
However, this isn't safe in the presence of cmd/link's -X flag, which
might overwrite an initialized string-typed variable at link time. In
particular, if cmd/link changes A's static initialization, it won't
know it also needs to change B's static initialization.
To address this, this CL disables this optimization for string-typed
variables.
Fixes#34675.
Change-Id: I1c18f3b855f6d7114aeb39f96aaaf1b452b88236
Reviewed-on: https://go-review.googlesource.com/c/go/+/198657
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It's pointless to reach all ms via allgs, and doing so introduces a
race, since the m member of a g can change underneath it. Instead
iterate directly through the allm linked list.
Updates: #31528
Updates: #34130
Change-Id: I34b88402b44339b0a5b4cd76eafd0ce6e43e2be1
Reviewed-on: https://go-review.googlesource.com/c/go/+/198417
Run-TryBot: Jason A. Donenfeld <Jason@zx2c4.com>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL is a follow up for 198080.
Added a private writeTo method to the Node interface,
in order to use the same builder for printing all nodes
in the tree. Benchmark output against master:
benchmark old ns/op new ns/op delta
BenchmarkParseLarge-8 24594994 25292054 +2.83%
BenchmarkVariableString-8 117 118 +0.85%
BenchmarkListString-8 10475 3353 -67.99%
benchmark old allocs new allocs delta
BenchmarkVariableString-8 3 3 +0.00%
BenchmarkListString-8 149 31 -79.19%
benchmark old bytes new bytes delta
BenchmarkVariableString-8 72 72 +0.00%
BenchmarkListString-8 5698 1608 -71.78%
Change-Id: I2b1cf07cda65c1b80083fb99671289423700feba
Reviewed-on: https://go-review.googlesource.com/c/go/+/198278
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If a request for a PTR record returned a response with a non-PTR
answer, goLookupPTR would loop forever. Skipping non-PTR answers
guarantees progress through the DNS response.
Fixes#34660
Change-Id: I56f9d21e5342d07e7d843d253267e93a29707904
Reviewed-on: https://go-review.googlesource.com/c/go/+/198460
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Currently cgoCheckPointer is only used with one optional argument.
Using a slice for the optional arguments is quite expensive, hence
replace it with a single interface{}. This results in ~30% improvement.
When checking struct fields, they quite often end up being without
pointers. Check this before calling cgoCheckPointer, which results in
additional ~20% improvement.
Inline some p == nil checks from cgoIsGoPointer which gives
additional ~15% improvement.
All of this translates to:
name old time/op new time/op delta
CgoCall/add-int-32 46.9ns ± 1% 46.6ns ± 1% -0.75% (p=0.000 n=18+20)
CgoCall/one-pointer-32 143ns ± 1% 87ns ± 1% -38.96% (p=0.000 n=20+20)
CgoCall/eight-pointers-32 767ns ± 0% 327ns ± 1% -57.30% (p=0.000 n=18+16)
CgoCall/eight-pointers-nil-32 110ns ± 1% 89ns ± 2% -19.10% (p=0.000 n=19+19)
CgoCall/eight-pointers-array-32 5.09µs ± 1% 3.56µs ± 2% -30.09% (p=0.000 n=19+19)
CgoCall/eight-pointers-slice-32 3.92µs ± 0% 2.57µs ± 2% -34.48% (p=0.000 n=20+20)
Change-Id: I2aa9f5ae8962a9a41a7fb1db0c300893109d0d75
Reviewed-on: https://go-review.googlesource.com/c/go/+/198081
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
