The android emulator builders is running out of space after CL 165797
copied most of GOROOT to the device.
The pkg directory is by far the largest, so only include what seems
necessary to build the x/ repositories: pkg/android_$GOARCH and
pkg/tool/android_$GOARCH.
While here, rename the device root directory to match the exec
wrapper name and make sure the deferred cleanups actually run before
os.Exit.
Hopefully fixes the emulator builders.
Updates #23824
Change-Id: I4d1e3ab2c89fd1e5818503d323ddb87f073094da
Reviewed-on: https://go-review.googlesource.com/c/go/+/166397
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Android tests are built on the host and run on the device. To do
that, the exec wrapper copies the test binary and testdata to the device.
To enable many more tests, make the copied environment more like the host:
- Copy all of pkg from GOROOT, not just the android pkg directory.
- Copy any parent testdata directories as well as the package's own.
- Copy *.go files from the package directory. This enables misc/cgo/stdio
and misc/cgo/life tests that were invisible before so disable them explicitly.
- Always copy the GOROOT, even for tests outside GOROOT. This is expensive
but only done once per make.bash.
- Build the go tool for the device and put it in PATH. Set GOCACHE
to a writable directory and disable cgo.
While here, use a single directory for all the exec wrapper files and
delete that once per make.bash as well.
In total, this CL enables many tests in the subrepos that would need skips
without it, in particular the x/tools tests.
Fixes#11452
Updates #23824
Updates #11811
Change-Id: I2e50d8b57db9bc4637f25272a5360c8b2cf4e627
Reviewed-on: https://go-review.googlesource.com/c/go/+/165797
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
According to
https://stackoverflow.com/questions/46233200/stop-buffering-of-adb-shell-output
the adb exec-out commands avoids the buffering inherent to adb shell.
Let's see if using exec-out will fix the android builder flakyness where
exitcodes or output were sometimes missing.
Updates #30512 (perhaps fixes it).
Change-Id: Ib953ef0262b20730e0d4c332058d29c5066bfeb2
Reviewed-on: https://go-review.googlesource.com/c/164661
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Android now works with all.bash.
Change-Id: I1087308865d2eb31f02501b5798e14d11145b185
Reviewed-on: https://go-review.googlesource.com/c/164700
Run-TryBot: Elias Naur <mail@eliasnaur.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Previous CLs added symlink evaulation to GOROOT and GOPATH.
Unfortunately that only fixed tests that ran outside GOROOT.
To fix the standard library tests, evaluate symlinks in the current
working directory as well.
Change-Id: Ia406a968235ae4321a1002567520105998582d15
Reviewed-on: https://go-review.googlesource.com/c/164699
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
(A stripped down version of) $GOROOT is uploaded to the device
before running standar library tests, including many (all?)
testdata directories.
Use the --sync flag when pushing testdata directories to the device
in case it is already present.
Change-Id: If8104f9d15838c1be3623adcf831a7188303c376
Reviewed-on: https://go-review.googlesource.com/c/164338
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
We've got away with not copying the testdata directories for the
standard library because the exec wrapper also pushes almost the
entire $GOROOT tree to the device, including testdata directories.
Similar to what the iOS exec wrapper does.
Change-Id: I91ef63ef84a658fc8843002890132c64b7c1d20e
Reviewed-on: https://go-review.googlesource.com/c/163626
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Android emulator builders are soon to join the trybot set. To avoid
flaky runs, work around a longstanding adb bug where concurrent adb
commands sometimes fail.
I haven't seen the problem on actual devices until recently. It seems
that the recently added "adb wait-for-device" can introduce flakyness
with errors such as:
adb: error: failed to get feature set: protocol fault (couldn't read status): Connection reset by peer
Instead of working around that, give up and serialize use of adb
everywhere.
Fixes#23795
Updates #23824
Change-Id: If347c9981fa32ff8a1e14b7454f122ef682450a6
Reviewed-on: https://go-review.googlesource.com/c/163625
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
To run the standard library tests on Android, the androidtest.bash
script copies GOROOT to the device. Move that logic to the android
exec wrapper, thereby making androidtest.bash obsolete.
Apart from making Android less special, the sharded builder
infrastructure should now be able to run (emulated) Android builders
and trybots without special treatment.
Updates #23824
Change-Id: I41591fea9a15b38c6dcf84046ea57f1e9165eaa5
Reviewed-on: https://go-review.googlesource.com/c/163619
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
When a test binary runs for too long, the go command sends it a
SIGQUIT to force a backtrace dump. On Android, the exec wrapper
will instead receive the signal and dump its backtrace.
Forward SIGQUIT signals from the wrapper to the wrapped process
to gain useful backtraces.
Inspired by issuse 25519; this CL would have revealed the hanging
test directly in the builder log.
Change-Id: Ic362d06940d261374343a1dc09366ef54edaa631
Reviewed-on: https://go-review.googlesource.com/114137
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
There is no NL at the end of README, and that make it strange
when doing "cat misc/android/README".
Change-Id: Ib47953d7b16e8927a4d6be7d5be8de8f2ddbcc39
Reviewed-on: https://go-review.googlesource.com/114010
Reviewed-by: Elias Naur <elias.naur@gmail.com>
Add more information to misc/android/README for developing
arm and arm64 with an Android environment.
Change-Id: I0c88996b6ab0c41946a2c7e69e9c92ec7bb3be27
Reviewed-on: https://go-review.googlesource.com/112276
Reviewed-by: Elias Naur <elias.naur@gmail.com>
Introduce GOANDROID_ADB_FLAGS for additional flags to adb invocations.
With GOANDROID_ADG_FLAGS, the Android builders can distinguish between
emulator and device builds.
Change-Id: I11729926a523ee27f6a3795cb2cfb64a9454f0a5
Reviewed-on: https://go-review.googlesource.com/88795
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
On Android, the exec wrapper passes on output from adb to its parent
process by passing on os.Stderr and os.Stdout to adb. If the adb
process somehow hangs, it will keep stderr and stdout will open, in turn
blocking go test from ever returning from its cmd.Wait() even though
it has killed the exec wrapper process.
Break the short circuit by introducing a wrapper between adb and the
exec wrapper, preventing os/exec.Run from passing along the raw
file descriptors for os.Stdout and os.Stderr.
(Hopefully) fixes occasional indefinite hangs on the Android builder.
Change-Id: I1188211fbde79b4a66bf93ff8e9d0091abf34560
Reviewed-on: https://go-review.googlesource.com/42271
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Before, the Android exec wrapper expected the trailing exit code
output on its own line, like this:
PASS
exitcode=0
However, some tests can sometimes squeeze in some output after
the test harness outputs "PASS" and the newline. The
TestWriteHeapDumpFinalizers test is particularly prone to this,
since its finalizers println to standard out. When it happens, the
output looks like this:
PASS
finalizedexitcode=0
Two recent failures caused by this race:
https://build.golang.org/log/185605e1b936142c22350eef22d20e982be53c29https://build.golang.org/log/e61cf6a050551d10360bd90be3c5f58c3eb07605
Since the "exitcode=" string is always echoed after the test output,
the fix is simple: instead of looking for the last newline in the
output, look for the last exitcode string instead.
Change-Id: Icd6e53855eeba60b982ad3108289d92549328b86
Reviewed-on: https://go-review.googlesource.com/23750
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
This change includes the cleanup of temporary files created during
the binary execution as well.
Change-Id: Ic01a0a537d1daafcaa3acda1ec344aff5dcddfc2
Reviewed-on: https://go-review.googlesource.com/2903
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This CL adjusts code referring to src/pkg to refer to src.
Immediately after submitting this CL, I will submit
a change doing 'hg mv src/pkg/* src'.
That change will be too large to review with Rietveld
but will contain only the 'hg mv'.
This CL will break the build.
The followup 'hg mv' will fix it.
For more about the move, see golang.org/s/go14nopkg.
LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/134570043
