The lldb child process is killed if a test runs too long. Also
kill it when the setup times out (and is retried).
Might help with builder flakes where all 5 attempts to start up
lldb fail even though the tests before and after the timeouts
succeed. For example:
...
ok vendor/golang_org/x/net/route 37.797s
lldb setup error: command timeout (lldb start for 17s)
start timeout, trying again
lldb setup error: command timeout (lldb start for 17s)
start timeout, trying again
lldb setup error: command timeout (lldb start for 17s)
start timeout, trying again
lldb setup error: command timeout (lldb start for 17s)
start timeout, trying again
lldb setup error: command timeout (lldb start for 17s)
go_darwin_arm_exec: failed to start test harness (retry attempted)
FAIL vendor/golang_org/x/text/transform 115.185s
ok vendor/golang_org/x/text/unicode/norm 122.773s
...
Change-Id: I6638860522896491dccfa12f1e520c0f23df6d66
Reviewed-on: https://go-review.googlesource.com/67791
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
When running multiple iOS builds on the same host, GOIOS_DEVICE_ID
is used to distinguish the devices. To improve support,
- Only restart the particular device when invoking iostest.bash
with the -restart flag.
- Make the exec wrapper lock file per-device.
For the iOS builder.
Change-Id: Id6f222981f25036399a43c3202a393dba89d87cb
Reviewed-on: https://go-review.googlesource.com/57970
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
If set, GOIOS_DEVICE_ID specifies the device id for the iOS exec
wrapper. With that, a single builder can host multiple iOS devices.
Change-Id: If3cc049552f5edbd7344befda7b8d7f73b4236e2
Reviewed-on: https://go-review.googlesource.com/57296
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: JBD <jbd@google.com>
Reviewed-by: Chris Broadfoot <cbro@golang.org>
The "lldb start" phase often times out on the iOS builder. Increase
the timeout and see if that helps.
Change-Id: I92fd67cbfa90659600e713198d6b2c5c78dde20f
Reviewed-on: https://go-review.googlesource.com/41863
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Some tests need the src/runtime/textflag.h file. Make sure it is
included in iOS test runs.
Change-Id: I5e0e7ebe85679686ef15a7d336f28ac9b68a587a
Reviewed-on: https://go-review.googlesource.com/40915
Run-TryBot: Elias Naur <elias.naur@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The iOS test harness was recently changed in response to lldb bugs
to replace breakpoints with the SIGUSR2 signal (CL 34926), and to
pass the current directory in the test binary arguments (CL 35152).
Both the signal sending and working directory setup is done from
the go test driver.
However, the new method doesn't work with tests where a C program is
the test driver instead of go test: the current working directory
will not be changed and SIGUSR2 is not raised.
Instead of copying that logic into any C test program, rework the
test harness (again) to move the setup logic to the early runtime
cgo setup code. That way, the harness will run even in the library
build modes.
Then, use the app Info.plist file to pass the working
directory, removing the need to alter the arguments after running.
Finally, use the SIGINT signal instead of SIGUSR2 to avoid
manipulating the signal masks or handlers.
Fixes the testcarchive tests on iOS.
With this CL, both darwin/arm and darwin/arm64 passes all.bash.
This CL replaces CL 34926, CL 35152 as well as the fixup CL
35123 and CL 35255. They are reverted in CLs earlier in the
relation chain.
Change-Id: I8485c7db1404fbd8daa261efd1ea89e905121a3e
Reviewed-on: https://go-review.googlesource.com/36090
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This reverts commit 333f764df3.
Replaced by a improved strategy later in the CL relation chain.
Change-Id: I70a1d2f0aa5aa0d3d0ec85b5a956c6fb60d88908
Reviewed-on: https://go-review.googlesource.com/36069
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This reverts commit 593ea3b360.
Replaced by a improved strategy later in the CL relation chain.
Change-Id: I6963e4d1bf38e7028cf545a953e28054d83548
Change-Id: I6963e4d1bf38e7028cf545a953e28054d8354870
Reviewed-on: https://go-review.googlesource.com/36067
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This reverts commit 467109bf56.
Replaced by a improved strategy later in the CL relation chain.
Change-Id: Ib90813b5a6c4716b563c8496013d2d57f9c022b8
Reviewed-on: https://go-review.googlesource.com/36066
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Tests that use TestMain might never call m.Run(), and simply return
from TestMain. In that case, the iOS test harness never sees the
PASS from the testing framework and assumes the test failed.
Allow an exit with exit code 0 to also mean test success, thereby
fixing the objdump test on iOS.
Change-Id: I1fe9077b05931aa0905e41b88945cd153c5b35b6
Reviewed-on: https://go-review.googlesource.com/36065
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
If -test.timeout is not specified to go test, it will time out after
a default 10 minutes.
The iOS exec wrapper also contains a fail safe timeout mechanism for
a stuck device. However, if no explicit -test.timeout is specified,
it will use a timeout of 0, plus some constant amount.
Use the same default timeout in the exec wrapper as for go test,
10 minutes.
Change-Id: I6465ccd9f7b9ce08fa302e6697f7938a0ea9af34
Reviewed-on: https://go-review.googlesource.com/36062
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The iOS exec wrapper use the constant bundle id "golang.gotest" for
running Go programs on iOS. However, that only happens to work on
the old iOS builders where their provisioning profile covers
that bundle id.
Expand the detection script to list all available provisioning
profiles for the attached device and include the bundle id in the
GOIOS_APP_ID environment variable.
To allow the old builders to continue, the "golang.gotest" bundle
id is used as a fallback if only the app id prefix is specified in
GOIOS_APP_ID.
For the new builders.
Change-Id: I8baa1d4d57f845de851c3fad3f178e05e9a01b17
Reviewed-on: https://go-review.googlesource.com/36060
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
On (at least) macOS 10.12, the `security cms` subcommand used by the
iOS detection script will output an error to stderr. The command
otherwise succeeds, but the extra line confuses a later parsing step.
To fix it, use only stdout and ignore stderr from every command run
by detect.go.
For the new iOS builders.
Change-Id: Iee426da7926d7f987ba1be061fa92ebb853ef53d
Reviewed-on: https://go-review.googlesource.com/36059
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The working directory is now adjusted to match the typical Go test
working directory in main, as the old trick for adjusting earlier
stopped working with the latest version of LLDB bugs.
That means the small number of places where testdata files are
read before main is called no longer work. This CL adjusts those
reads to happen after main is called. (This has the bonus effect of
not reading some benchmark testdata files in all.bash.)
Fixes compress/bzip2, go/doc, go/parser, os, and time package
tests on the iOS builder.
Change-Id: If60f026aa7848b37511c36ac5e3985469ec25209
Reviewed-on: https://go-review.googlesource.com/35255
Run-TryBot: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Another change in behvaior (bug) in LLDB. Despite the fact that
LLDB can dump the symtab of our test binaries and show the function
addresses, it can no longer call the functions. This means the chdir
trick on signal is failing.
This CL uses a new trick. For iOS, the exec script passes the change
in directory as an argument, and it is processed early by the test
harness generated by cmd/go.
For the iOS builders.
Change-Id: I8f5d0f831fe18de99f097761f89c5184d5bf2afb
Reviewed-on: https://go-review.googlesource.com/35152
Reviewed-by: Elias Naur <elias.naur@gmail.com>
Run-TryBot: David Crawshaw <crawshaw@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
The iOS test harness has set a breakpoint early in the life of Go
programs so that it can change the current working directory using
information only available from the host debugger. Somewhere in the
upgrade to iOS 10 / XCode 8.2, breakpoints stopped working. This
may be an LLDB bug, or a bug in the ios-deploy LLDB scripts, it's
not clear.
Work around the problem by giving up on breakpoints. Instead, early
in the life of every test binary built for iOS, send (and ignore) a
SIGUSR2 signal. The debugger will catch this, giving the script
go_darwin_arm_exec a chance to change the working directory.
For the iOS builders.
Change-Id: I7476531985217d0c76bc176904c48379210576c2
Reviewed-on: https://go-review.googlesource.com/34926
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Xcode 8.0 has been donen't support the iOS 5 anymore
Fixes#18390.
Change-Id: Icc97e09424780c610a8fe173d0cf461d76b06da4
Reviewed-on: https://go-review.googlesource.com/34673
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The exec wrapper lock file was opened, locked and then never used
again, assuming it would close and unlock at process exit.
However, the garbage collector could collect and run the *os.File
finalizer that closes the file prematurely, rendering the lock
ineffective.
Make the lock global so that the lock is live during the entire
execution.
(Hopefully) fix the iOS builders.
Change-Id: I62429e92042a0a49c4f1ea553fdb32b6ea53a43e
Reviewed-on: https://go-review.googlesource.com/21137
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The iOS exec wrapper use complicated machinery to run a iOS binary
on a device.
Running several binaries concurrently doesn't work (reliably), which
can break tests running concurrently. For my setup, the
runtime:cpu124 and sync_cpu tests can't run reliably without one of them
crashing.
Add a file lock to the exec wrapper to serialize execution.
Fixes#14318 (for me)
Change-Id: I023610e014b327f8d66f1d2fd2e54dd0e56f2be0
Reviewed-on: https://go-review.googlesource.com/21074
Reviewed-by: David Crawshaw <crawshaw@golang.org>
A retry mechanism is in place to combat the inherent flakiness of
launching iOS test binaries. Before it covered just the starting of
lldb; expand it to cover the setup steps as well. Note that the
running of the binary itself is (still) not retried, to avoid
covering over genuine bugs.
On my test device (iPhone 5S, iOS 9.3) starting lldb can take longer
than 10 seconds, so increase the timeout for that.
Furthermore, some basic steps such as setting breakpoints in lldb
can take longer than the 1 second timeout. Increase that timeout
as well, to 2 seconds.
Finally, improve the error message for when ios-deploy is not
installed.
For #14318
Change-Id: Iba41d1bd9d023575b9454cb577b08f8cae081c2a
Reviewed-on: https://go-review.googlesource.com/21072
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Instruct lldb to pass through SIGCONT unhindered when running iOS
tests. Fixes the TestSIGCONT test in os/signal.
For #14318
Change-Id: I669264208cc3d6ecae9fbc8790e0b753a93a5e04
Reviewed-on: https://go-review.googlesource.com/21071
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The existing go_darwin_arm_exec.go script does not work with Xcode 7,
not due to any significant changes, but just ordering and timing of
statements from lldb. Unfortunately the current design of
go_darwin_arm_exec.go makes it not obvious what gets stuck where, so
this moves from a moving buffer window to a complete buffer of the
lldb output.
The result is easier code to follow, and it works with Xcode 7.
Updates #12660.
Change-Id: I3b8b890b0bf4474119482e95d84e821a86d1eaed
Reviewed-on: https://go-review.googlesource.com/16634
Reviewed-by: Michael Matloob <matloob@golang.org>
The error message should indicate the name of the unset variable,
rather than the value. The value will alwayse be empty.
Change-Id: I6f6c165074dfce857b6523703a890d205423cd28
Reviewed-on: https://go-review.googlesource.com/16555
Reviewed-by: David Crawshaw <crawshaw@golang.org>
This is a follow-up to CL 8910.
This is the version that I have tested and which works
when appID and teamID are not the same (which they appear
to be for the builder).
I am unsure how I submitted it with the wrong code.
Change-Id: I186e34e91953d082b507390c1cd2042e5419c4c9
Reviewed-on: https://go-review.googlesource.com/8943
Reviewed-by: David Crawshaw <crawshaw@golang.org>
We no longer need the EXC_BAD_ACCESS watcher as runtime/cgo contains
a mach exception handler that catches it. And now lldb only
intermittently reports process connection and exiting, so instead
just look for the PASS from Go.
Change-Id: I403266558f5a900e0b87ec1019d9baec88148d23
Reviewed-on: https://go-review.googlesource.com/8957
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Use environment variables to allow set-and-forget.
Add a script to attempt to autodetect codesign info.
Change-Id: Ic56b9c5f097b1a4117ebb89c408bc333d91f581d
Reviewed-on: https://go-review.googlesource.com/8910
Reviewed-by: David Crawshaw <crawshaw@golang.org>
The -lldb flag makes it easy to use go run and end up in a debugging
session on darwin/arm.
Change-Id: I556f93e950086a7dff4839f301b9c55f7579f87b
Reviewed-on: https://go-review.googlesource.com/8024
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
After moving the darwin/arm builder to new hardware several new flaky
error messages appeared. This provided enough information to Google
to make it clear that iOS build systems have been flaky for many
years, and that is unlikely to change any time soon.
However, all of the pain of lldb and using a breakpoint early in
program initialization gives us an advantage: all install and
initialization flakiness appears to happen before the Go program ever
gets going. So if we see an error or we timeout before we reach our
breakpoint (before any test code has executed), we can assume it is
the fault of the builder and restart without risking hiding a flaky
Go test.
This code has successfully processed the last 8 builds. I am hopeful.
Change-Id: Ide24aaae4fa7bdab9d8f4432bb85d8f2256c7606
Reviewed-on: https://go-review.googlesource.com/8241
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
The clever partial timer I added interacts badly with iOS app launch
timeout termination. A fixed timeout will be easier to debug.
Change-Id: I6eb4ee5f1431539f00fa707e8cde6f3cf86983fc
Reviewed-on: https://go-review.googlesource.com/8083
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
Split out from cl/8024 for clarity and improved approach.
Rarely, "stop reason = breakpoint" does not appear in the lldb stop
text. However the program is ready to proceed. To be a little more
robust about those cases, we wait for two seconds, and if that text
doesn't appear but a prompt does we continue and hope for the best.
Worst case, this results in a harder to read failure message.
Change-Id: Ib20aa92564cdccefd2b7260417c647cd44122b66
Reviewed-on: https://go-review.googlesource.com/8080
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
We now wait until we see the completed prompt from a command before
proceeding. This seems to cut down on a spurious error I have seen
this afternoon.
Change-Id: Ic0a3481d8c265c3c3b4449ec7ac1c2752b85b0b6
Reviewed-on: https://go-review.googlesource.com/6691
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
See golang.org/cl/6511.
Change-Id: I2145a42877ed6b78400f29c2ef18969870dab5c3
Reviewed-on: https://go-review.googlesource.com/6512
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
This script is getting very close to complete, and is complex enough
that I'd like to get what's there so far reviewed. With it the builder
is left failing on eight packages. Two of those involve correcting
GOROOT which may need modifications to this script, the others are
either a unix sockets bug I have to hunt down or are caused by lldb
getting stuck on SIGSEGV, a TODO.
Change-Id: I5ff933800167b6764b51ad195da7dcda61d59ff8
Reviewed-on: https://go-review.googlesource.com/6404
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
