This uses the new JSON interface to the build dashboard (from
golang.org/cl/2290) to find all work, and re-enables the OpenBSD
builder[*], and can do multiple things at a time. Andrew and I just
watched it fire up 8 OpenBSD VMs at once to catch up.
[*] The OpenBSD builder was disabled because it would only report
results for the main repo, not subrepos, and the old
build.golang.org/todo interface didn't understand that was
possible. Now the steps are considered separate.
Update golang/go#8642 (OpenBSD)
Update golang/go#9492 (builds in VMs)
Change-Id: Ic6c2f73ee3da218dd54ef1a33f3afc97046ea3cc
Reviewed-on: https://go-review.googlesource.com/2282
Reviewed-by: Andrew Gerrand <adg@golang.org>
Currently this is running an OpenBSD amd64 builder, but other operating
systems are now easy.
Updates golang/go#8642
Updates golang/go#9492
Change-Id: I4e1026c49ee4d253dfcf78c7606767294784f02e
Reviewed-on: https://go-review.googlesource.com/2264
Reviewed-by: Andrew Gerrand <adg@golang.org>
Currently the coordinator only handles Docker container builders. An
upcoming change will add VM builders, running VMs on GCE for OpenBSD,
Plan 9, Windows, etc.
This change refactors the handling of the coordinator's build status
in prep for VM builders. Notably, it doesn't call "docker logs"
directly to get logs. It now keeps them in memory instead. Because
they're then in memory, we might as well make it keep the details of
the last N builds too, which we kept wanting earlier.
Updates #9492
Change-Id: I6bae4a5854e7efa3f8a6186ec3670a43c98c4df2
Reviewed-on: https://go-review.googlesource.com/2262
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This isn't used yet, but will be for the new-style builders (VMs on
GCE running the buildlet, started by the coordinator).
From the code's comments:
cleanUpOldVMs periodically enumerates virtual machines and deletes
any which have a "delete-at" attribute having a unix timestamp
before the current time. These VMs are created to run a single
build and should be shut down by a controlling process. Due to
various types of failures, they might get stranded. To prevent them
from getting stranded and wasting resources forever, we instead set
the "delete-at" metadata attribute on them when created to some
time that's well beyond their expected lifetime, and then this is
the backup mechanism to delete them if they get away.
Update golang/go#8639
Update golang/go#8640
Update golang/go#8642
Change-Id: I489e97926e7ab56487571c2bf0bd255cdf49570d
Reviewed-on: https://go-review.googlesource.com/2199
Reviewed-by: Burcu Dogan <jbd@google.com>
I'm tired of figuring out what isn't cleaning up after itself, so keep
a background goroutine that looks at old containers and deletes them
as a backup measure. Verified it works by creating some dummy containers on
the machine.
Also adds df output to the HTML status page.
Change-Id: I23adc22872def882b3b9b3a4ec730017899bb966
Reviewed-on: https://go-review.googlesource.com/1537
Reviewed-by: Andrew Gerrand <adg@golang.org>
New default repo, restart watchers on failure, etc.
Change-Id: Idb2e8e6f65debd07d2fa04d998b8036a00260683
Reviewed-on: https://go-review.googlesource.com/1366
Reviewed-by: Andrew Gerrand <adg@golang.org>
Revision 184f9219148f (from review 152790043) introduced a new image,
linux-x86-clang, and the url is missing the `s` for TLS.
LGTM=adg
R=bradfitz, golang-codereviews, adg
CC=golang-codereviews
https://golang.org/cl/156480043
This change adds a 32-bit builder for gccgo and fixes some specific
configuration problems with the docker image/coordinator to speed up builds.
The go builder running on the linux-x86-gccgo image has been modified to run
`make -j16` when building to reduce build times from ~60 minutes to ~10 minutes
and the coordinator has been modified to run the command `make check-go -j16`
to reduce testsuite run times from ~30 minutes to ~10 minutes.
LGTM=bradfitz
R=bradfitz
CC=adg, golang-codereviews
https://golang.org/cl/151530043
So the optimization to avoid hg clone works again. The
builder looks at buildroot+"goroot" for an hg clone repo. If
it finds it, it can just "hg pull" instead of a slow "hg
clone".
LGTM=cmang
R=cmang, proppy
CC=golang-codereviews
https://golang.org/cl/147960044
... since it already caught a problem and was fixed in
hg rev 0b99c2137ccc.
I pushed this live already just so don't get a whole column of
fail that might hide other issues. Real users have USER set
anyway. (Perhaps we should set it for all our builders?)
LGTM=adg
R=golang-codereviews
CC=adg, dave, golang-codereviews, minux
https://golang.org/cl/137180043
Creating a new disk is the major latency cost of creating a
new instance. If we reuse a previous disk image, the VM and
CoreOS boot much quicker and start building again.
So add a mode where we don't mark the disk as delete-on-death
when we create it, and we search for a suitably-named existing
one on creation and try to reuse it.
It's a cache, essentially. It's still stateless.
LGTM=adg
R=adg
CC=golang-codereviews
https://golang.org/cl/135410043
Then make the coordinator and linux environment Makefiles have
an upload target. Amusingly, this actually worked: the Docker
images now tar + compress + upload over HTTP all in a stream,
without even knowing how large the resulting tar.gz will be until
it's done uploading.
LGTM=adg
R=adg
CC=golang-codereviews
https://golang.org/cl/140200043
Remove subversion & use newer version of the builder which
makes -verbose direct all.bash output to stdout/stderr.
LGTM=cmang
R=adg, cmang
CC=golang-codereviews
https://golang.org/cl/140910044
