We want to error early if GOFLAGS contains any flag that isn't known to
any cmd/go command. Thus, at init time we would recursively use VisitAll
on each of the flagsets to populate a map of all registered flags.
This was unfortunate, as populating said map constituted a whole 5% of
the run-time of 'go env GOARCH'. This is because VisitAll is pretty
expensive; it copies all the maps from the flagset's map to a slice,
sorts the slice, then does one callback per flag.
First, this was a bit wasteful. We only ever needed to query the
knownFlag map if GOFLAGS wasn't empty. If it's empty, there's no work to
do, thus we can skip the map populating work.
Second and most important, we don't actually need the map at all. A
flag.FlagSet already has a Lookup method, so we can simply recursively
call those methods for each flag in GOFLAGS. Add a hasFlag func to make
that evident.
This mechanism is different; its upfront cost is none, but it will
likely mean a handful of map lookups for each flag in GOFLAGS. However,
that tradeoff is worth it; we don't expect GOFLAGS to contain thousands
of flags. The most likely scenario is less than a dozen flags, in which
case constructing a "unified" map is not at all a net win.
One possible reason the previous mechanism was that way could be
AddKnownFlag. Thankfully, the one and only use of that API was removed
last year when Bryan cleaned up flag parsing in cmd/go.
The wins for the existing benchmark with an empty GOFLAGS are
significant:
name old time/op new time/op delta
ExecGoEnv-8 575µs ± 1% 549µs ± 2% -4.44% (p=0.000 n=7+8)
name old sys-time/op new sys-time/op delta
ExecGoEnv-8 1.69ms ± 1% 1.68ms ± 2% ~ (p=0.281 n=7+8)
name old user-time/op new user-time/op delta
ExecGoEnv-8 1.80ms ± 1% 1.66ms ± 2% -8.09% (p=0.000 n=7+8)
To prove that a relatively large number of GOFLAGS isn't getting
noticeably slower, we measured that as well, via benchcmd and GOFLAGS
containing 50 valid flags:
GOFLAGS=$(yes -- -race | sed 50q) benchcmd -n 500 GoEnvGOFLAGS go env GOARCH
And the result, while noisy, shows no noticeable difference (note that
it measures 3ms instead of 0.6ms since it's sequential):
name old time/op new time/op delta
GoEnvGOFLAGS 3.04ms ±32% 3.03ms ±35% ~ (p=0.156 n=487+481)
Finally, we've improved the existing Go benchmark. Now it's parallel,
and it also reports sys-time and user-time, which are useful metrics.
Change-Id: I9b4551415cedf2f819eb184a02324b8bd919e2bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/248757
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This CL changes "T literal.M" error message to "T{...}.M". It's clearer
expression and focusing user on actual issue.
Updates #38745
Change-Id: I84b455a86742f37e0bde5bf390aa02984eecc3c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/253677
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This is a refactoring intended to break the dependency from
internal/modfetch to internal/get. No change in functionality is intended.
Change-Id: If51aba7139cc0b62ecc9ba454c055c99e8f36f0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/254364
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
'go mod download' calls WriteGoMod once via modload.ListModules when
it loads the build list. This saves sums for go.mod files needed by
MVS, but the write occurs before any zip files are downloaded.
With this change, 'go mod download' calls WriteGoMod again (and thus,
modfetch.WriteGoSum) after downloading and verifying module zip files,
so the sums of the zip files will be saved, too.
Fixes#41341
Change-Id: I7d56754aa255256ed45fd93cb154c2e6ea5f45a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/254357
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
On darwin, we put read-only data in __TEXT segment on AMD64 in
exe (non-PIE) buildmode, and in __DATA on everywhere else. This
is not ideal: things in __DATA segment are not read-only, and
being mapped R/W may use more run-time resources.
In fact, newer darwin systems support a __DATA_CONST segment,
which the dynamic linker will map it read-only after applying
relocations. Use that.
Fixes#38830.
Change-Id: Ic281e6c6ca8ef5fec4bb7c5b71c50dd5393e78ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/253919
Reviewed-by: Than McIntosh <thanm@google.com>
When -mod=readonly is set, Import will now allow imports from
replacements without explicit requirements. With -mod=mod, this would
add a new requirement but does not trigger a module lookup, so it's
determinisitic.
Before reporting an error for an unknown import with -mod=readonly,
check whether the import is valid. If there's a typo in the import,
that's more relevant.
For #40728
Change-Id: I05e138ff76ba3d0eb2e3010c15589fa363deb8d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/253745
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Keep track of whether the -mod flag was set explicitly. When
-mod=readonly is the default, we'll want to adjust our error messages
if it's set explicitly.
Also, register the -mod, -modcacherw, and -modfile flags in functions
in internal/base instead of internal/work. 'go mod' commands that
don't load packages shouldn't depend on internal/work.
For #40728
Change-Id: I272aea9e19908ba37e151baac4ea8630e90f241f
Reviewed-on: https://go-review.googlesource.com/c/go/+/253744
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
In the deadcode pass, a type symbol may be marked twice, one
without UsedInIface, one with. For the second time, don't
update the Reachparent graph, so it only records the path of
the first time the symbol is reached. This ensures the
Reachparent graph is acyclic.
TODO: add a test. (This only affects GOEXPERIMENT=fieldtrack)
Change-Id: I68e8a1a69c3830bc8aee5df946151dc22dcb2b29
Reviewed-on: https://go-review.googlesource.com/c/go/+/254297
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
That port is gone.
Change-Id: I212d435e290d1890d6cd5531be98bb692650595e
Reviewed-on: https://go-review.googlesource.com/c/go/+/254077
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
BenchmarkFullASCIIRune tests the performance of function utf8.FullRune,
which will be inlined in BenchmarkFullASCIIRune. Since the return value
of FullRune is not referenced, it will be removed as dead code.
This CL makes the FullRune functions return value referenced by a global
variable to avoid this point. In addition, this CL adds one more benchmark
to cover more code paths, and puts them together as sub benchmarks of
BenchmarkFullRune.
Change-Id: I6e79f4c087adf70e351498a4b58d7482dcd1ec4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/233979
Run-TryBot: eric fang <eric.fang@arm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This CL adds USHLL, USHLL2, UZP1, UZP2, and BIF instructions requested
by #40725. And since UXTL* are aliases of USHLL*, this CL also merges
them into one case.
Updates #40725
Change-Id: I404a4fdaf953319f72eea548175bec1097a2a816
Reviewed-on: https://go-review.googlesource.com/c/go/+/253659
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Otherwise we’d still be writing that we're building cmd/dist
even if GOROOT_BOOTSTRAP we’re wrongly set.
Change-Id: I940e18c3cebae5664f85babc9919e9eb215d5093
Reviewed-on: https://go-review.googlesource.com/c/go/+/253877
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Enable VBSL, VBIT, VCMTST, VUXTL VUXTL2 and FMOVQ SIMD
instructions required by the issue #40725. And FMOVQ
instrucion is used to move a large constant to a Vn
register.
Add test cases.
Fixes#40725
Change-Id: I1cac1922a0a0165d698a4b73a41f7a5f0a0ad549
Reviewed-on: https://go-review.googlesource.com/c/go/+/249758
Reviewed-by: Cherry Zhang <cherryyz@google.com>
I found the control flow of this function a bit tricky to reason about
due to nesting and interaction between conditions and iteration. This
change factors out a helper function that can return early instead of
mixing conditionals and 'continue' statements.
Also remove the (unused) ModuleUsedDirectly function.
For #36460
Change-Id: I60a2a5a1b32989e5a17a14e1a8c858b280cda8f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/251998
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
With lazy loading, the “build list” can be refined as packages are loaded.
Rename functions that return the build list to more precisely describe
the set of modules returned by the call.
Also eliminate a redundant call to LoadBuildList (right before
ListModules, which itself begins with the same call).
For #36460
Change-Id: I0fc4f9dd7602e0df5e166e329ee5d516d810ca53
Reviewed-on: https://go-review.googlesource.com/c/go/+/249878
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
The new semantics of the "all" package pattern can be implemented
without actually changing module loading per se. This change
implements those semantics, so that the change can be decoupled from
the changes to the module requirement graph.
For #36460
Change-Id: I0ee8b17afa8b728dc470a42a540fcc01764a4442
Reviewed-on: https://go-review.googlesource.com/c/go/+/240623
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Includes the latest fix on vet to warn unused context.WithValue result.
Fixes#41149
Change-Id: I06c204f40ef12b0f62f59b1bbdf1fe06ccd6565d
Reviewed-on: https://go-review.googlesource.com/c/go/+/252941
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Due to a bug in CL 173017, if QueryPackages found multiple candidates
for the given package and *at least* one of those candidates was not
available to add, we would reject *all* such candidates — even those
that were still viable.
Now, we return the first viable candidate, and only return an error if
*no* candidate is viable given the current build list.
Fixes#41113
Change-Id: Idb2e77244be7c0f5dd511efb142c3059925d7336
Reviewed-on: https://go-review.googlesource.com/c/go/+/251446
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
Fix a data race for clients that mutate requests after receiving a
response error which is caused by the writeLoop goroutine left
running, this can be seen on cancelled requests.
Fixes#37669
Change-Id: Ia4743c6b8abde3a7503de362cc6a3782e19e7f60
Reviewed-on: https://go-review.googlesource.com/c/go/+/251858
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
modload.Import previously performed two otherwise-separable tasks:
1. Identify which module in the build list contains the requested
package.
2. If no such module exists, search available modules to try to find
the missing package.
This change splits those two tasks into two separate unexported
functions, and reports import-resolution errors by attaching them to
the package rather than emitting them directly to stderr. That allows
'list' to report the errors, but 'list -e' to ignore them.
With the two tasks now separate, it will be easier to avoid the
overhead of resolving missing packages during lazy loading if we
discover that some existing dependency needs to be promoted to the top
level (potentially altering the main module's selected versions, and
thus suppling packages that were previously missing).
For #36460
Updates #26909
Change-Id: I32bd853b266d7cd231d1f45f92b0650d95c4bcbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/251445
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
forceStdVendor was a special-case mechanism to allow Go contributors
to use vendored dependencies by default when working in GOROOT/src.
As of Go 1.14,¹ the 'go' command uses vendored dependencies by default
within all modules, so the 'std' and 'cmd' modules no longer need to
be special cases, and we can remove this special-case code.
¹ https://golang.org/doc/go1.14#vendor
Updates #33848
Updates #30241
Change-Id: Ib2fb5841c253113b17fa86a086ce85a22ac3d121
Reviewed-on: https://go-review.googlesource.com/c/go/+/251159
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
If the user requests the 'all' pattern in addition to explicit roots
outside of 'all', we should not load the transitive dependencies of
those explicit roots as if they were *in* 'all'. Without the '-test'
flag, we should not load test dependencies of any package outside of
'all'. Even *with* the '-test' flag, we should only load test
dependencies of the requested roots, not test dependencies of other
packages that happen to be imported by those roots.
More precise tracking of membership in 'all' will be important when we
implement lazy loading, because membership in 'all' determines which
module dependencies we will record in the main module's go.mod file.
This change also reduces reliance on global state, factors out the
loading process into several smaller functions, and sets us up to
reuse the 'go mod vendor' version of the 'all' pattern for lazy
loading.
For #36460Fixes#40799
Change-Id: I5ca21c86a860daee1316f732cea131a331d8ddf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/240505
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
This avoids a deadlock on prof.signalLock between setcpuprofilerate
and cpuprof.add if a SIGPROF is delivered to the thread between the
call to setThreadCPUProfiler and acquiring prof.signalLock.
Fixes#41014
Change-Id: Ie825e8594f93a19fb1a6320ed640f4e631553596
Reviewed-on: https://go-review.googlesource.com/c/go/+/253758
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Both ReadMemStatsSlow and CheckScavengedBits iterate over the page
allocator's chunks but don't actually check if they exist. During the
development process the chunks index became sparse, so now this was a
possibility. If the runtime tests' heap is sparse we might end up
segfaulting in either one of these functions, though this will generally
be very rare.
The pattern here to return nil for a nonexistent chunk is also useful
elsewhere, so this change introduces tryChunkOf which won't throw, but
might return nil. It also updates the documentation of chunkOf.
Fixes#41296.
Change-Id: Id5ae0ca3234480de1724fdf2e3677eeedcf76fa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/253777
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Pre-resolve package index references, so it doesn't need to do a
map lookup in every cross-package symbol reference resolution. It
increases the memory usage very slightly (O(# imported packages)).
Change-Id: Ia76c97ac51f1c2c2d5ea7ae34853850ec69ef0a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/253604
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This preserves zip sums when 'go get' is run on a module that does not
have a package in the root directory. The zip must be fetched to
determine whether the package should be loaded, so we already load and
verify the sum.
Note that 'go mod tidy' may still remove these sums, since they
aren't needed to load packages.
Fixes#41103
Change-Id: I78f10a25f0392461fdc98518a7c92a38ee3233c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/251880
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Currently, the statement:
go g(uintptr(f()))
gets rewritten into:
tmp := f()
newproc(8, g, uintptr(tmp))
runtime.KeepAlive(tmp)
which doesn't guarantee that tmp is still alive by time the g call is
scheduled to run.
This CL fixes the issue, by wrapping g call in a closure:
go func(p unsafe.Pointer) {
g(uintptr(p))
}(f())
then this will be rewritten into:
tmp := f()
go func(p unsafe.Pointer) {
g(uintptr(p))
runtime.KeepAlive(p)
}(tmp)
runtime.KeepAlive(tmp) // superfluous, but harmless
So the unsafe.Pointer p will be kept alive at the time g call runs.
Updates #24491
Change-Id: Ic10821251cbb1b0073daec92b82a866c6ebaf567
Reviewed-on: https://go-review.googlesource.com/c/go/+/253457
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
It appears the machoCalcStart function is meant to align the
segment, but it doesn't. Replace it with an actual alignment
calculation. Also, use the alignment from the configuration,
instead of hardcode.
With this fix we could enable DWARF combining on macOS ARM64.
Change-Id: I19ec771b77d752b83a54c53b6ee65af78a31b8ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/253558
Reviewed-by: Than McIntosh <thanm@google.com>
In Mach-O DWARF combining, some code was written using reflection,
so it could support both 32-bit and 64-bit Mach-O files without
duplicating code. We no longer support 32-bit darwin platforms
now. 32-bit support can go. Rewrite it with direct field access,
for 64-bit only.
Change-Id: If1338c3cd37cecf603f4df0c6eb0c890eaebfe5f
Reviewed-on: https://go-review.googlesource.com/c/go/+/253557
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
On darwin, with external linking, the system linker produces STAB
(symbolic debugging) symbols in the binary's symbol table. These
include paths of the intermediate object files, like
<tmpdir>/go.o, which changes from run to run, making the build
non-reproducible.
Since we run dsymutil to produce debug info and combine them
back into the binary, we don't need those STAB symbols anymore.
Strip them after running dsymutil.
If DWARF is not enabled, we don't run dsymutil. We can pass
"-Wl,-S" to let the system linker not generate those symbols.
While here, also make it more consistent about DWARF combining.
Currently we only do DWARF combining on macOS/AMD64, when DWARF
is enabled. On ARM64, we run dsymutil, but then throw the result
away. This CL changes it to not run dsymutil (and strip) on
ARM64.
TODO: add a test. We don't do it here as it fails on some
(non-darwin) platforms.
Fixes#40979.
Change-Id: If770f7828cdb858857d6079e0585bf067f8f7a92
Reviewed-on: https://go-review.googlesource.com/c/go/+/250944
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
The post-index offset of VLD[1-4]R instructions is decided by the
"size" field not "Q" field, the current assembler uses "Q" fileld
to check the correctness of post-index offset which is not correct.
This patch fixes it.
Fixes#40725
Change-Id: If1cde7f21c6b3ee0e491649eb567700bd1475c84
Reviewed-on: https://go-review.googlesource.com/c/go/+/249757
Reviewed-by: Cherry Zhang <cherryyz@google.com>
The Value field of ast.BasicLit is a string field holding the literal
string. For CHARs and STRINGs, the BasicLit.Value literal includes
quotes, so to use the value in practise one will often need to Unquote
it.
Since this is a common gotcha (I've been bitten by this a few times),
document it, and suggest the use of the strconv.Unquote functions.
Fixes#39590
Change-Id: Ie3e13f5a2a71bb1b59e03bc5b3a16d8e2e7c01d4
Reviewed-on: https://go-review.googlesource.com/c/go/+/244960
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
