Forked from CL 111895.
The trickiest part of this is that the binary layout code (blk,
elfshbits, and various other things) assumes a constant offset between
symbols' and sections' file locations and their virtual addresses.
Compression, of course, breaks this constant offset. But we need to
assign virtual addresses to everything before compression in order to
resolve relocations before compression. As a result, compression needs
to re-compute the "address" of the DWARF sections and symbols based on
their compressed size. Luckily, these are at the end of the file, so
this doesn't perturb any other sections or symbols. (And there is, of
course, a surprising amount of code that assumes the DWARF segment
comes last, so what's one more place?)
Relevant benchmarks:
name old time/op new time/op delta
StdCmd 10.3s ± 2% 10.8s ± 1% +5.43% (p=0.000 n=30+30)
name old text-bytes new text-bytes delta
HelloSize 746kB ± 0% 746kB ± 0% ~ (all equal)
CmdGoSize 8.41MB ± 0% 8.41MB ± 0% ~ (all equal)
[Geo mean] 2.50MB 2.50MB +0.00%
name old data-bytes new data-bytes delta
HelloSize 10.6kB ± 0% 10.6kB ± 0% ~ (all equal)
CmdGoSize 252kB ± 0% 252kB ± 0% ~ (all equal)
[Geo mean] 51.5kB 51.5kB +0.00%
name old bss-bytes new bss-bytes delta
HelloSize 125kB ± 0% 125kB ± 0% ~ (all equal)
CmdGoSize 145kB ± 0% 145kB ± 0% ~ (all equal)
[Geo mean] 135kB 135kB +0.00%
name old exe-bytes new exe-bytes delta
HelloSize 1.60MB ± 0% 1.05MB ± 0% -34.39% (p=0.000 n=30+30)
CmdGoSize 16.5MB ± 0% 11.3MB ± 0% -31.76% (p=0.000 n=30+30)
[Geo mean] 5.14MB 3.44MB -33.08%
Fixes#11799.
Updates #6853.
Change-Id: I64197afe4c01a237523a943088051ee056331c6f
Reviewed-on: https://go-review.googlesource.com/118276
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is needed in addition to CL 110066 in order to be able to generate
Go type definitions for linux/riscv64 in the golang.org/x/sys/unix
package.
Change-Id: I4a27e6424aaea63283b55bd4f73b958b41f29d72
Reviewed-on: https://go-review.googlesource.com/118618
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This adds the support to enable the race detector for ppc64le.
Added runtime/race_ppc64le.s to manage the calls from Go to the
LLVM tsan functions, mostly converting from the Go ABI to the
PPC64 ABI expected by Clang generated code.
Changed racewalk.go to call racefuncenterfp instead of racefuncenter
on ppc64le to allow the caller pc to be obtained in the asm code
before calling the tsan version.
Changed the set up code for racecallbackthunk so it doesn't use
the autogenerated save and restore of the link register since that
sequence uses registers inconsistent with the normal ppc64 ABI.
Made various changes to recognize that race is supported for
ppc64le.
Ensured that tls_g is updated and accessible from race_linux_ppc64le.s
so that the race ctx can be obtained and passed to tsan functions.
This enables the race tests for ppc64le in cmd/dist/test.go and
increases the timeout when running the benchmarks with the -race
option to avoid timing out.
Updates #24354, #23731
Change-Id: Ib97dc7ac313e6313c836dc7d2fb698f9d8fba3ef
Reviewed-on: https://go-review.googlesource.com/107935
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Each URL was manually verified to ensure it did not serve up incorrect
content.
Change-Id: I4dc846227af95a73ee9a3074d0c379ff0fa955df
Reviewed-on: https://go-review.googlesource.com/115798
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
So we can have builders running go test -short=false.
Updates golang/go#12508
Change-Id: If90f0f6d9f89268c33b1d1876139ad551fecd3d8
Reviewed-on: https://go-review.googlesource.com/113435
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This commit adds the wasm architecture to the compile command.
A later commit will contain the corresponding linker changes.
Design doc: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4
The following files are generated:
- src/cmd/compile/internal/ssa/opGen.go
- src/cmd/compile/internal/ssa/rewriteWasm.go
- src/cmd/internal/obj/wasm/anames.go
Updates #18892
Change-Id: Ifb4a96a3e427aac2362a1c97967d5667450fba3b
Reviewed-on: https://go-review.googlesource.com/103295
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
When dist test was updated to run "go test" with multiple package
arguments at once, merging the logical test units into one execution,
the hack to give cmd/go twice as much time wasn't updated.
What was happening (even in the all.bash case) was that we were
merging together, say, "cmd/go" and "bytes", and because bar was
lexically earlier, the timeout calculation was based on package "byte",
even though we were actually running, say: "go test bytes cmd/go".
This explains why x/build/cmd/release was often flaky with its
all.bash, since cmd/go can't really finish in 3 minutes reliably
unless it's running by itself. If it has any competition, it runs
over.
Change-Id: I875c8c9e65940ce0ceff48215740dfadfaa89d3f
Reviewed-on: https://go-review.googlesource.com/111395
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Changes include:
1. open compilation option -msan for arm64
2. modify doc to explain -msan is also supported on linux/arm64
3. wrap msan lib API in msan_arm64.s
4. use libc for sigaction syscalls when cgo is enabled
5. use libc for mmap syscalls when cgo is enabled
Change-Id: I26ebe61ff7ce1906125f54a0182a720f9d58ec11
Reviewed-on: https://go-review.googlesource.com/109255
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
mips64 softfloat support is based on mips implementation and introduces
new enviroment variable GOMIPS64.
GOMIPS64 is a GOARCH=mips64{,le} specific option, for a choice between
hard-float and soft-float. Valid values are 'hardfloat' (default) and
'softfloat'. It is passed to the assembler as
'GOMIPS64_{hardfloat,softfloat}'.
Change-Id: I7f73078627f7cb37c588a38fb5c997fe09c56134
Reviewed-on: https://go-review.googlesource.com/108475
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Would've caught two regressions so far, and found two more.
Updates #24841
Updates #24845 (package net remains)
Change-Id: I57ad06eb54e04b8c99b5d2e7f24c77ad865224e8
Reviewed-on: https://go-review.googlesource.com/107300
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is the first commit of a series that will add WebAssembly
as an architecture target. The design document can be found at
https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4.
The GOARCH name "wasm" is the official abbreviation of WebAssembly.
The GOOS name "js" got chosen because initially the host environment
that executes WebAssembly bytecode will be web browsers and Node.js,
which both use JavaScript to embed WebAssembly. Other GOOS values
may be possible later, see:
https://github.com/WebAssembly/design/blob/master/NonWeb.md
Updates #18892
Change-Id: Ia25b4fa26bba8029c25923f48ad009fd3681933a
Reviewed-on: https://go-review.googlesource.com/102835
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Fixes#14327
Much of the code is based on the linux/amd64 code that implements these
build modes, and code is shared where possible.
Change-Id: Ia510f2023768c0edbc863aebc585929ec593b332
Reviewed-on: https://go-review.googlesource.com/93875
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Auto-detecting GOARM on Android makes as little sense as for nacl/arm
and darwin/arm.
Also update androidtest.sh to not require GOARM set.
Change-Id: Id409ce1573d3c668d00fa4b7e3562ad7ece6fef5
Reviewed-on: https://go-review.googlesource.com/98875
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Move the IndexByte function from the runtime to a new bytealg package.
The new package will eventually hold all the optimized assembly for
groveling through byte slices and strings. It seems a better home for
this code than randomly keeping it in runtime.
Once this is in, the next step is to move the other functions
(Compare, Equal, ...).
Update #19792
This change seems complicated enough that we might just declare
"not worth it" and abandon. Opinions welcome.
The core assembly is all unchanged, except minor modifications where
the code reads cpu feature bits.
The wrapper functions have been cleaned up as they are now actually
checked by vet.
Change-Id: I9fa75bee5d85db3a65b3fd3b7997e60367523796
Reviewed-on: https://go-review.googlesource.com/98016
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Both gcc and clang accept an option -fplugin=code.so to load
a plugin from the ELF shared object file code.so.
Obviously that plugin can then do anything it wants
during the build. This is contrary to the goal of "go get"
never running untrusted code during the build.
(What happens if you choose to run the result of
the build is your responsibility.)
Disallow this behavior by only allowing a small set of
known command-line flags in #cgo CFLAGS directives
(and #cgo LDFLAGS, etc).
The new restrictions can be adjusted by the environment
variables CGO_CFLAGS_ALLOW, CGO_CFLAGS_DISALLOW,
and so on. See the documentation.
In addition to excluding cgo-defined flags, we also have to
make sure that when we pass file names on the command
line, they don't look like flags. So we now refuse to build
packages containing suspicious file names like -x.go.
A wrinkle in all this is that GNU binutils uniformly accept
@foo on the command line to mean "if the file foo exists,
then substitute its contents for @foo in the command line".
So we must also reject @x.go, flags and flag arguments
beginning with @, and so on.
Fixes#23672, CVE-2018-6574.
Change-Id: I59e7c1355155c335a5c5ae0d2cf8fa7aa313940a
Reviewed-on: https://team-review.git.corp.google.com/209949
Reviewed-by: Ian Lance Taylor <iant@google.com>
When there is no go directory inside the swiglib directory then swig
was installed without Go support. Tests in misc/swig will fail when
swig is installed without Go support.
Add additional checks for the presence of a go directory in the directory
reported by 'swig -go -swiglib' to determine if misc/swig tests should
be run.
This avoids all.bash failing when swig but not swig-go is installed
using macports.
Tested on darwin with swig and with and without swig-go installed
using macports.
Fixes#23469
Change-Id: I173201221554982ea0d9f2bea70a3cb85b297cec
Reviewed-on: https://go-review.googlesource.com/88776
Reviewed-by: David Chase <drchase@google.com>
Suppose you build the Go toolchain in directory A,
move the whole thing to directory B, and then use
it from B to build a new program hello.exe, and then
run hello.exe, and hello.exe crashes with a stack
trace into the standard library.
Long ago, you'd have seen hello.exe print file names
in the A directory tree, even though the files had moved
to the B directory tree. About two years ago we changed
the compiler to write down these files with the name
"$GOROOT" (that literal string) instead of A, so that the
final link from B could replace "$GOROOT" with B,
so that hello.exe's crash would show the correct source
file paths in the stack trace. (golang.org/cl/18200)
Now suppose that you do the same thing but hello.exe
doesn't crash: it prints fmt.Println(runtime.GOROOT()).
And you run hello.exe after clearing $GOROOT from the
environment.
Long ago, you'd have seen hello.exe print A instead of B.
Before this CL, you'd still see hello.exe print A instead of B.
This case is the one instance where a moved toolchain
still divulges its origin. Not anymore. After this CL, hello.exe
will print B, because the linker sets runtime/internal/sys.DefaultGoroot
with the effective GOROOT from link time.
This makes the default result of runtime.GOROOT once again
match the file names recorded in the binary, after two years
of divergence.
With that cleared up, we can reintroduce GOROOT into the
link action ID and also reenable TestExecutableGOROOT/RelocatedExe.
When $GOROOT_FINAL is set during link, it is used
in preference to $GOROOT, as always, but it was easier
to explain the behavior above without introducing that
complication.
Fixes#22155.
Fixes#20284.
Fixes#22475.
Change-Id: Ifdaeb77fd4678fdb337cf59ee25b2cd873ec1016
Reviewed-on: https://go-review.googlesource.com/86835
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
The default test timeout is 10 minutes if unspecified.
The misc/cgo/testshared test didn't use t.timeout(sec), which respects
GO_TEST_TIMEOUT_SCALE, so all builders got the default 10 minute
timeout. arm5 needs more, though, so specify 10 minutes explicitly,
which will then get scaled accordingly on slower builders.
Change-Id: I19ecfdcd9c865f2b69524484415b8fbd2852718e
Reviewed-on: https://go-review.googlesource.com/84315
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Andrew Bonventre <andybons@golang.org>
cmd/go has grown slow, even in short mode, and it's now regularly
failing on a number of builders where it's taking over the previous 3
minute timeout. for now, give it more time.
Change-Id: If565baf71c2770880b2e2139b47e03433951331f
Reviewed-on: https://go-review.googlesource.com/84235
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Sometimes people use run.bash repeatedly
or run go tool dist test by hand for cgo tests.
Avoid test caching in that case, by request.
Refactor code so that all go test commands
share a common prefix.
If not caching is problematic it will be a one-line
change to turn caching back on.
Fixes#22758.
Change-Id: I17d721b832d97bffe26629d21f85b05dbbf2b3ec
Reviewed-on: https://go-review.googlesource.com/80735
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
GOMIPS is a GOARCH=mips{,le} specific option, for a choice between
hard-float and soft-float. Valid values are 'hardfloat' (default) and
'softfloat'. It is passed to the assembler as
'GOMIPS_{hardfloat,softfloat}'.
Note: GOMIPS will later also be used for a choice of MIPS instruction
set (mips32/mips32r2).
Updates #18162
Change-Id: I35417db8625695f09d6ccc3042431dd2eaa756a6
Reviewed-on: https://go-review.googlesource.com/37954
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
They were added in CL 78175 but doesn't run on Android (yet). Skip
them for now.
For the Android builders.
Change-Id: I3b4bfe1f0d820ab98cf50aaab1ee2fad1a44a851
Reviewed-on: https://go-review.googlesource.com/78615
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
In a previous change to cmd/dist/test.go to fix some pie
testcases, a few other tests were incorrectly dropped.
This returns the testcases that shouldn't have been removed.
Fixes#22708
Change-Id: I2f735f4fd3a378f0f45d12a99768638aeb4787c7
Reviewed-on: https://go-review.googlesource.com/77650
Run-TryBot: Russ Cox <rsc@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Some C types are declared as pointers, but C code
stores non-pointers in them. When the Go garbage
collector sees such a pointer, it gets unhappy.
Instead, for these types represent them on the Go
side with uintptr.
We need this change to handle Apple's CoreFoundation
CF*Ref types. Users of these types might need to
update their code like we do in root_cgo_darwin.go.
The only change that is required under normal
circumstances is converting some nils to 0.
A go fix module is provided to help.
Fixes#21897
RELNOTE=yes
Change-Id: I9716cfb255dc918792625f42952aa171cd31ec1b
Reviewed-on: https://go-review.googlesource.com/66332
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Adding s390x to the list of architectures that support c-shared and c-archive.
Required adding load-time initialization (via _rt0_s390x_linux_lib) and adding s390x
to the c-shared and c-archive tests.
Change-Id: I75883b2891c310fe8ce7f08c27b06895c074e123
Reviewed-on: https://go-review.googlesource.com/74910
Reviewed-by: Michael Munday <mike.munday@ibm.com>
(The tests only run when swig is already installed on the local system.)
Change-Id: I172d106a68cfc746a1058f5a4bcf6761bab88912
Reviewed-on: https://go-review.googlesource.com/78175
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Cgo has always operated by rewriting the AST and invoking go/printer.
This CL converts it to use the AST to make decisions but then apply
its edits directly to the underlying source text. This approach worked
better in rsc.io/grind (used during the C to Go conversion) and also
more recently in cmd/cover. It guarantees that all comments and
line numbers are preserved exactly.
This eliminates a lot of special concern about comments and
problems with cgo not preserving meaningful comments.
Combined with the CL changing cmd/cover to use the same
approach, it means that the combination of applying cgo and
applying cover still guarantees all comments and line numbers
are preserved exactly.
This sets us up to fix some cgo vs cover bugs by swapping
the order in which they run during the go command.
This also sets up #16623 a bit: the edit list being
accumulated here is nearly exactly what you'd want
to pass to the compiler for that issue.
Change-Id: I7611815be22e7c5c0d4fc3fa11832c42b32c4eb3
Reviewed-on: https://go-review.googlesource.com/77153
Reviewed-by: Ian Lance Taylor <iant@golang.org>
It has always been problematic that there was no way to specify
tool flags that applied only to the build of certain packages;
it was only to specify flags for all packages being built.
The usual workaround was to install all dependencies of something,
then build just that one thing with different flags. Since the
dependencies appeared to be up-to-date, they were not rebuilt
with the different flags. The new content-based staleness
(up-to-date) checks see through this trick, because they detect
changes in flags. This forces us to address the underlying problem
of providing a way to specify per-package flags.
The solution is to allow -gcflags=pattern=flags, which means
that flags apply to packages matching pattern, in addition to the
usual -gcflags=flags, which is now redefined to apply only to
the packages named on the command line.
See #22527 for discussion and rationale.
Fixes#22527.
Change-Id: I6716bed69edc324767f707b5bbf3aaa90e8e7302
Reviewed-on: https://go-review.googlesource.com/76551
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The cache will take care of keeping go test -tags lldb fast.
Installing runtime/cgo this way just makes all the checkNotStale
tests think runtime/cgo is out of date.
Should fix ios builders.
Fixes#22509.
Change-Id: If092cc4feb189eb848b6a22f6d22b89b70df219c
Reviewed-on: https://go-review.googlesource.com/76020
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The change in cmd/dist ignores debug output, instead of assuming
any output is from the template.
The change in cmd/go makes the debug output show the package name
on every line, so that interlaced prints can be deinterlaced.
Change-Id: Ic3d59ee0256271067cb9be2fde643a0e19405375
Reviewed-on: https://go-review.googlesource.com/76019
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Even though cmd/dist has historically distinguished "CC for gohostos/gohostarch"
from "CC for target goos/goarch", it has not recorded that distinction
for later use by cmd/cgo and cmd/go. Now that content-based staleness
includes the CC setting in the decision about when to rebuild packages,
the go command needs to know the details of which CC to use when.
Otherwise lots of things look out of date and (worse) may be rebuilt with
the wrong CC.
A related issue is that users may want to be able to build a toolchain
capable of cross-compiling for two different non-host targets, and
to date we've required that CC_FOR_TARGET apply to both.
This CL introduces CC_FOR_${GOOS}_${GOARCH}, so that you can
(for example) set CC_FOR_linux_arm and CC_FOR_linux_arm64
separately on a linux/ppc64 host and be able to cross-compile to
either arm or arm64 with the right toolchain.
Fixes#8161.
Half of a fix for #22509.
Change-Id: I7a43769f39d859f659d31bc96980918ba102fb83
Reviewed-on: https://go-review.googlesource.com/76018
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
I do not remember why we require deps.go to have a hard-coded
copy of the dependency information for cmd/go, when we can
read it from the source files instead. The answer probably involves
cmd/dist once being a C program.
In any event, stop doing that, which will eliminate the builder-only
failures in the builder-only deps test.
Change-Id: I0abd384c47401940ca08427b5be544812edcbe7f
Reviewed-on: https://go-review.googlesource.com/76021
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Otherwise, one can't run "go fmt" on a directory containing Go files if
none of them are buildable (e.g. because of build tags). This is
counter-intuitive, as fmt will format all Go files anyway.
If we encounter such a load error, ignore it and carry on. All other
load errors, such as when a package can't be found, should still be
shown to the user.
Add a test for the two kinds of load errors. Use fmt -n so that any
changes to the formatting of the files in testdata don't actually get
applied. The load errors still occur with -n, so the test does its job.
Fixes#22183.
Change-Id: I99d0c0cdd29015b6a3f5286a9bbff50757c78e0d
Reviewed-on: https://go-review.googlesource.com/75930
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
This CL makes "go install" behave the way many users expect:
install only the things named on the command line.
Future builds still run as fast, thanks to the new build cache (CL 75473).
To install dependencies as well (the old behavior), use "go install -i".
Actual definitions aside, what most users know and expect of "go install"
is that (1) it installs what you asked, and (2) it's fast, unlike "go build".
It was fast because it installed dependencies, but installing dependencies
confused users repeatedly (see for example #5065, #6424, #10998, #12329,
"go build" and "go test" so that they could be "fast" too, but that only
created new opportunities for confusion. We also had to add -installsuffix
and then -pkgdir, to allow "fast" even when dependencies could not be
installed in the usual place.
The recent introduction of precise content-based staleness logic means that
the go command detects the need for rebuilding packages more often than it
used to, with the consequence that "go install" rebuilds and reinstalls
dependencies more than it used to. This will create more new opportunities
for confusion and will certainly lead to more issues filed like the ones
listed above.
CL 75743 introduced a build cache, separate from the install locations.
That cache makes all operations equally incremental and fast, whether or
not the operation is "install" or "build", and whether or not "-i" is used.
Installing dependencies is no longer necessary for speed, it has confused
users in the past, and the more accurate rebuilds mean that it will confuse
users even more often in the future. This CL aims to end all that confusion
by not installing dependencies by default.
By analogy with "go build -i" and "go test -i", which still install
dependencies, this CL introduces "go install -i", which installs
dependencies in addition to the things named on the command line.
Fixes#5065.
Fixes#6424.
Fixes#10998.
Fixes#12329.
Fixes#18981.
Fixes#22469.
Another step toward #4719.
Change-Id: I3d7bc145c3a680e2f26416e182fa0dcf1e2a15e5
Reviewed-on: https://go-review.googlesource.com/75850
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This CL adds caching of successful test results, keyed by the
action ID of the test binary and its command line arguments.
Suppose you run:
go test -short std
<edit a typo in a comment in math/big/float.go>
go test -short std
Before this CL, the second go test would re-run all the tests
for the std packages. Now, the second go test will use the cached
result immediately (without any compile or link steps) for any
packages that do not transitively import math/big, and then
it will, after compiling math/big and seeing that the .a file didn't
change, reuse the cached test results for the remaining packages
without any additional compile or link steps.
Suppose that instead of editing a typo you made a substantive
change to one function, but you left the others (including their
line numbers) unchanged. Then the second go test will re-link
any of the tests that transitively depend on math/big, but it still
will not re-run the tests, because the link will result in the same
test binary as the first run.
The only cacheable test arguments are:
-cpu
-list
-parallel
-run
-short
-v
Using any other test flag disables the cache for that run.
The suggested argument to mean "turn off the cache" is -count=1
(asking "please run this 1 time, not 0").
There's an open question about re-running tests when inputs
like environment variables and input files change. For now we
will assume that users will bypass the test cache when they
need to do so, using -count=1 or "go test" with no arguments.
This CL documents the new cache but also documents the
previously-undocumented distinction between "go test" with
no arguments (now called "local directory mode") and with
arguments (now called "package list mode"). It also cleans up
a minor detail of package list mode buffering that used to change
whether test binary stderr was sent to go command stderr based
on details like exactly how many packages were listed or
how many CPUs the host system had. Clearly the file descriptor
receiving output should not depend on those, so package list mode
now consistently merges all output to stdout, where before it
mostly did that but not always.
Fixes#11193.
Change-Id: I120edef347b9ddd5b10e247bfd5bd768db9c2182
Reviewed-on: https://go-review.googlesource.com/75631
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
We build and run executables in the work directory,
and some users have $TMPDIR set noexec.
Fixes#8451.
Change-Id: I76bf2ddec84e9cb37ad9a6feb53a1a84b47aa263
Reviewed-on: https://go-review.googlesource.com/75475
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This CL adds caching of built package files in $GOCACHE, so that
a second build with a particular configuration will be able to reuse
the work done in the first build of that configuration, even if the
first build was only "go build" and not "go install", or even if there
was an intervening "go install" that wiped out the installed copy of
the first build.
The benchjuju benchmark runs go build on a specific revision of jujud 10 times.
Before this CL:
102.72u 15.29s 21.98r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
105.99u 15.55s 22.71r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
106.49u 15.70s 22.82r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
107.09u 15.72s 22.94r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
108.19u 15.85s 22.78r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
108.92u 16.00s 23.02r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
109.25u 15.82s 23.05r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
109.57u 15.96s 23.11r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
109.86u 15.97s 23.17r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
110.50u 16.05s 23.37r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
After this CL:
113.66u 17.00s 24.17r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
3.85u 0.68s 3.49r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
3.98u 0.72s 3.63r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
4.07u 0.72s 3.57r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
3.98u 0.70s 3.43r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
4.58u 0.70s 3.58r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
3.90u 0.70s 3.46r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
3.85u 0.71s 3.52r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
3.70u 0.69s 3.64r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
3.79u 0.68s 3.41r go build -o /tmp/jujud github.com/juju/juju/cmd/jujud ...
This CL reduces the overall all.bash time from 4m22s to 4m17s on my laptop.
Not much faster, but also not slower.
See also #4719, #20137, #20372.
Change-Id: I101d5363f8c55bf4825167a5f6954862739bf000
Reviewed-on: https://go-review.googlesource.com/75473
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The README is there to help people who stumble across the directory.
The access log is there to help us evaluate potential algorithms for
managing and pruning cache directories. For now the management
is manual: users have to run "go clean -cache" if they want the cache
to get smaller.
As a low-resolution version of the access log, we also update the
mtime on each cache file as they are used by the go command.
A simple refinement of go clean -cache would be to delete
(perhaps automatically) cache files that have not been used in more
than one day, or some suitable time period.
Change-Id: I1dd6309952942169d71256c4b50b723583d21fca
Reviewed-on: https://go-review.googlesource.com/75471
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
