mirror of
https://github.com/golang/go
synced 2024-11-26 03:07:57 -07:00
[dev.boringcrypto] misc/boring: release packaging
Add scripts and docs for packaging releases. Change-Id: I0682c92bbb2e229d2636762e49fe73513852d351 Reviewed-on: https://go-review.googlesource.com/57890 Reviewed-by: Adam Langley <agl@golang.org>
This commit is contained in:
Russ Cox
parent
94fb8224b2
commit
f48a9fb815
97
misc/boring/README.md
Normal file
97
misc/boring/README.md
Normal file
@ -0,0 +1,97 @@
|
||||
# README.md
|
||||
|
||||
This directory holds build scripts for unofficial, unsupported
|
||||
distributions of Go+BoringCrypto.
|
||||
|
||||
## Version strings
|
||||
|
||||
The distribution name for a Go+BoringCrypto release has the form `<GoVersion>b<BoringCryptoVersion>`,
|
||||
where `<GoVersion>` is the Go version the release is based on, and `<BoringCryptoVersion>` is
|
||||
an integer that increments each time there is a new release with different BoringCrypto bits.
|
||||
The `<BoringCryptoVersion>` is stored in the `VERSION` file in this directory.
|
||||
|
||||
For example, the first release is based on Go 1.8.3 is `go1.8.3b1`.
|
||||
If the BoringCrypto bits are updated, the next would be `go1.8.3b2`.
|
||||
If, after that, Go 1.9 is released and the same BoringCrypto code added to it,
|
||||
that would result in `go1.9b2`. There would likely not be a `go1.9b1`,
|
||||
since that would indicate Go 1.9 with the older BoringCrypto code.
|
||||
|
||||
## Releases
|
||||
|
||||
The `build.release` script prepares a binary release and publishes it in Google Cloud Storage
|
||||
at `gs://go-boringcrypto/`, making it available for download at
|
||||
`https://go-boringcrypto.storage.googleapis.com/<FILE>`.
|
||||
The script records each published release in the `RELEASES` file in this directory.
|
||||
|
||||
The `build.docker` script, which must be run after `build.release`, prepares a Docker image
|
||||
and publishes it on hub.docker.com in the goboring organization.
|
||||
`go1.8.3b1` is published as `goboring/golang:1.8.3b1`.
|
||||
|
||||
## Release process
|
||||
|
||||
1. If the BoringCrypto bits have been updated, increment the number in `VERSION`,
|
||||
send that change out as a CL for review, get it committed, and run `git sync`.
|
||||
|
||||
2. Run `build.release`, which will determine the base Go version and the BoringCrypto
|
||||
version, build a release, and upload it.
|
||||
|
||||
3. Run `build.docker`, which will build and upload a Docker image from the latest release.
|
||||
|
||||
4. Send out a CL with the updated `RELEASES` file and get it committed.
|
||||
|
||||
## Release process for dev.boringcrypto.go1.8.
|
||||
|
||||
In addition to the dev.boringcrypto branch, we have a dev.boringcrypto.go1.8 branch,
|
||||
which is BoringCrypto backported to the Go 1.8 release branch.
|
||||
To issue new BoringCrypto releases based on Go 1.8:
|
||||
|
||||
1. Do a regular release on the (not Go 1.8) dev.boringcrypto branch.
|
||||
|
||||
2. Change to the dev.boringcrypto.go1.8 branch and cherry-pick all
|
||||
BoringCrypto updates, including the update of the `VERSION` file.
|
||||
Mail them out and get them committed.
|
||||
|
||||
3. **Back on the (not Go 1.8) dev.boringcrypto branch**, run `build.release <commit>`,
|
||||
where `<commit>` is the latest commit on the dev.boringcrypto.go1.8 branch.
|
||||
It will build a release and upload it.
|
||||
|
||||
4. Run `build.docker`.
|
||||
|
||||
5. Send out a CL with the updated `RELEASES` file and get it committed.
|
||||
|
||||
## Building from Docker
|
||||
|
||||
A Dockerfile that starts with `FROM golang:1.8.3` can switch
|
||||
to `FROM goboring/golang:1.8.3b2` (see [goboring/golang on Docker Hub](https://hub.docker.com/r/goboring/golang/))
|
||||
and should need no other modifications.
|
||||
|
||||
## Building from Bazel
|
||||
|
||||
Using an alternate toolchain from Bazel is not as clean as it might be.
|
||||
Today, as of Bazel 0.5.3 and the bazelbuild/rules_go tag 0.5.3,
|
||||
it is necessary to define a `go-boringcrypto.bzl` file that duplicates
|
||||
some of the rules_go internal guts and then invoke its `go_repositories` rule
|
||||
instead of the standard one.
|
||||
|
||||
See https://gist.github.com/rsc/6f63d54886c9c50fa924597d7355bc93 for a minimal example.
|
||||
|
||||
Note that in the example that the Bazel `WORKSPACE` file still refers to the release as "go1.8.3" not "go1.8.3b2".
|
||||
|
||||
## Caveat
|
||||
|
||||
BoringCrypto is used for a given build only in limited circumstances:
|
||||
|
||||
- The build must be GOOS=linux, GOARCH=amd64.
|
||||
- The build must have cgo enabled.
|
||||
- The android build tag must not be specified.
|
||||
- The cmd_go_bootstrap build tag must not be specified.
|
||||
|
||||
The version string reported by `runtime.Version` does not indicate that BoringCrypto
|
||||
was actually used for the build. For example, linux/386 and non-cgo linux/amd64 binaries
|
||||
will report a version of `go1.8.3b2` but not be using BoringCrypto.
|
||||
|
||||
To check whether a given binary is using BoringCrypto, run `go tool nm` on it and check
|
||||
that it has symbols named `*_Cfunc__goboringcrypto_*`.
|
||||
|
||||
The program [rsc.io/goversion](https://godoc.org/rsc.io/goversion) will report the
|
||||
crypto implementation used by a given binary when invoked with the `-crypto` flag.
|
||||
4
misc/boring/RELEASES
Normal file
4
misc/boring/RELEASES
Normal file
@ -0,0 +1,4 @@
|
||||
# This file lists published Go+BoringCrypto releases.
|
||||
# Each line describes a single release: <version> <git commit> <URL> <sha256sum>
|
||||
go1.9rc2b2 91753387bdf7 https://go-boringcrypto.storage.googleapis.com/go1.9rc2b2.linux-amd64.tar.gz 59355a45e6970e8013060851ddb3f079afe8db52e90db520a0826a13f1b5ae5b
|
||||
go1.8.3b3 f6ff81bac156 https://go-boringcrypto.storage.googleapis.com/go1.8.3b3.linux-amd64.tar.gz 6287ad971cd268bb2684fb8b1275dea928ad527823062bc057e73036c419e7af
|
||||
1
misc/boring/VERSION
Normal file
1
misc/boring/VERSION
Normal file
@ -0,0 +1 @@
|
||||
3
|
||||
57
misc/boring/build.docker
Executable file
57
misc/boring/build.docker
Executable file
@ -0,0 +1,57 @@
|
||||
#!/bin/bash
|
||||
# Copyright 2017 The Go Authors. All rights reserved.
|
||||
# Use of this source code is governed by a BSD-style
|
||||
# license that can be found in the LICENSE file.
|
||||
|
||||
# build.docker builds and publishes a Docker image for
|
||||
# a given Go+BoringCrypto release.
|
||||
|
||||
set -e
|
||||
|
||||
# With no arguments, use the most recent release in the RELEASES file.
|
||||
case "$#" in
|
||||
0)
|
||||
version=$(tail -1 RELEASES | awk '{print $1}');;
|
||||
1)
|
||||
version="$1";;
|
||||
*)
|
||||
echo 'usage: build.docker [version]' >&2
|
||||
exit 2
|
||||
esac
|
||||
|
||||
url="$(grep "^$version " RELEASES | awk '{print $3}')"
|
||||
sha256="$(grep "^$version " RELEASES | awk '{print $4}')"
|
||||
if [ "$sha256" = "" ]; then
|
||||
echo "cannot find $version in RELEASES file" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Build a temporary directory with a Dockerfile.
|
||||
dir=$(mktemp -d)
|
||||
trap "rm -rf $dir" EXIT
|
||||
|
||||
if echo "$url" | grep '!' >/dev/null; then
|
||||
# ! is sed delimiter below. Should never happen.
|
||||
echo "URL contains an exclamation mark!" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
sed "s!UUU!$url!; s/SSS/$sha256/" dockerfile.in >$dir/Dockerfile
|
||||
cp go-wrapper $dir/go-wrapper
|
||||
|
||||
dversion=$(echo "$version" | sed 's/^go//')
|
||||
docker build -t goboring/golang:$dversion $dir
|
||||
docker run goboring/golang:$dversion go version
|
||||
docker run goboring/golang:$dversion go tool nm /usr/local/go/bin/go >$dir/nm
|
||||
if ! grep crypto/sha1.boringNewSHA1 $dir/nm >/dev/null; then
|
||||
echo 'built docker image but did NOT find sha1.boringNewSHA1 in go command!' >&2
|
||||
exit 2
|
||||
fi
|
||||
if egrep 'crypto/sha1\.\(\*digest\)' $dir/nm >/dev/null; then
|
||||
echo 'built docker image but DID find sha1.(*digest) in go command unexpectedly!' >&2
|
||||
exit 2
|
||||
fi
|
||||
docker push goboring/golang:$dversion
|
||||
|
||||
echo
|
||||
echo published as goboring/golang:$dversion
|
||||
90
misc/boring/build.release
Executable file
90
misc/boring/build.release
Executable file
@ -0,0 +1,90 @@
|
||||
#!/bin/bash
|
||||
# Copyright 2017 The Go Authors. All rights reserved.
|
||||
# Use of this source code is governed by a BSD-style
|
||||
# license that can be found in the LICENSE file.
|
||||
|
||||
# build.release builds and publishes a new Go+BoringCrypto release.
|
||||
# After running this script, the change to the RELEASES file should be
|
||||
# sent out for review and committed to the repository (but the release
|
||||
# is already done, so there's not much to review).
|
||||
|
||||
set -e
|
||||
|
||||
case "$#" in
|
||||
0)
|
||||
rev=HEAD;;
|
||||
1)
|
||||
rev="$1";;
|
||||
*)
|
||||
echo 'usage: build.release [git-rev]' >&2
|
||||
exit 2
|
||||
esac
|
||||
|
||||
# Determine commit to use.
|
||||
commit=$(git rev-parse "$rev" | awk '{print substr($1, 1, 12)}')
|
||||
if [ "$commit" = "" ]; then
|
||||
echo 'cannot find commit in git history' >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Determine base Go release from tags.
|
||||
base=$(git log --decorate=short --oneline "$rev" | grep 'tag: go' | sed 1q | sed 's/[),].*//; s/.*tag: //')
|
||||
if [ "$base" = "" ]; then
|
||||
echo "cannot find go release tag in git history for $rev" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Determine boring crypto version from file.
|
||||
boring=$(git show "$commit:misc/boring/VERSION")
|
||||
if [ "$boring" = "" ]; then
|
||||
echo "missing BORINGVERSION file in $commit" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Make sure we're not redefining a published release.
|
||||
version="${base}b${boring}"
|
||||
if grep "^$version " RELEASES >/dev/null; then
|
||||
echo "found $version in RELEASES - not rereleasing" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Show what's going on, while the release builds.
|
||||
# Good time for user to type ^C if something is wrong.
|
||||
echo >&2
|
||||
echo "building $version from $commit" >&2
|
||||
echo >&2
|
||||
git log -n1 "$commit" >&2
|
||||
echo >&2
|
||||
|
||||
# Build the release tool in a temporary GOPATH.
|
||||
dir=$(mktemp -d)
|
||||
trap "rm -rf $dir" EXIT
|
||||
export GOPATH="$dir"
|
||||
export GOBIN="$dir"
|
||||
go get -u golang.org/x/build/cmd/release
|
||||
|
||||
# Build the release.
|
||||
shortgo=$(echo "$base" | perl -pe 's/(go\d+\.\d+)(\.\d+|rc\d+)/$1/')
|
||||
$dir/release -target linux-amd64 -rev "$commit" -version "$version" -tools "release-branch.$shortgo"
|
||||
output="$version.linux-amd64.tar.gz"
|
||||
ls -l "$output"
|
||||
sha256=$(sha256sum "$output" | awk '{print $1}')
|
||||
|
||||
trap "rm -f /tmp/go.release.$$ /tmp/go.nm.$$" EXIT
|
||||
tar -xzf "$output" -O go/bin/go >/tmp/go.release.$$
|
||||
go tool nm /tmp/go.release.$$ >/tmp/go.nm.$$
|
||||
if ! grep crypto/sha1.boringNewSHA1 /tmp/go.nm.$$ >/dev/null; then
|
||||
echo 'built release but did NOT find sha1.boringNewSHA1 in go command!' >&2
|
||||
exit 2
|
||||
fi
|
||||
if egrep 'crypto/sha1\.\(\*digest\)' /tmp/go.nm.$$ >/dev/null; then
|
||||
echo 'built release but DID find sha1.(*digest) in go command unexpectedly!' >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
# Publish the release.
|
||||
gsutil cp "$output" gs://go-boringcrypto/
|
||||
url="https://go-boringcrypto.storage.googleapis.com/$output"
|
||||
|
||||
# Record that it was published.
|
||||
echo "$version $commit $url $sha256" >>RELEASES
|
||||
29
misc/boring/dockerfile.in
Normal file
29
misc/boring/dockerfile.in
Normal file
@ -0,0 +1,29 @@
|
||||
# Template for Dockerfile, used in build.docker script.
|
||||
# Based on https://github.com/docker-library/golang/blob/master/1.9-rc/stretch/Dockerfile
|
||||
FROM buildpack-deps:stretch-scm
|
||||
|
||||
# gcc for cgo
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
g++ \
|
||||
gcc \
|
||||
libc6-dev \
|
||||
make \
|
||||
pkg-config \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
ADD UUU /go.tgz
|
||||
|
||||
RUN set -eux; \
|
||||
\
|
||||
echo "SSS /go.tgz" | sha256sum -c -; \
|
||||
tar -C /usr/local -xzf /go.tgz; \
|
||||
rm /go.tgz; \
|
||||
export PATH="/usr/local/go/bin:$PATH"; \
|
||||
go version
|
||||
|
||||
ENV GOPATH /go
|
||||
ENV PATH $GOPATH/bin:/usr/local/go/bin:$PATH
|
||||
RUN mkdir -p "$GOPATH/src" "$GOPATH/bin" && chmod -R 777 "$GOPATH"
|
||||
WORKDIR $GOPATH
|
||||
|
||||
COPY go-wrapper /usr/local/bin/
|
||||
100
misc/boring/go-wrapper
Executable file
100
misc/boring/go-wrapper
Executable file
@ -0,0 +1,100 @@
|
||||
#!/bin/sh
|
||||
# Copied from https://raw.githubusercontent.com/docker-library/golang/master/1.9-rc/stretch/go-wrapper
|
||||
# Copied into Docker images.
|
||||
|
||||
set -e
|
||||
|
||||
usage() {
|
||||
base="$(basename "$0")"
|
||||
cat <<EOUSAGE
|
||||
|
||||
usage: $base command [args]
|
||||
|
||||
This script assumes that is is run from the root of your Go package (for
|
||||
example, "/go/src/app" if your GOPATH is set to "/go").
|
||||
|
||||
In Go 1.4, a feature was introduced to supply the canonical "import path" for a
|
||||
given package in a comment attached to a package statement
|
||||
(https://golang.org/s/go14customimport).
|
||||
|
||||
This script allows us to take a generic directory of Go source files such as
|
||||
"/go/src/app" and determine that the canonical "import path" of where that code
|
||||
expects to live and reference itself is "github.com/jsmith/my-cool-app". It
|
||||
will then ensure that "/go/src/github.com/jsmith/my-cool-app" is a symlink to
|
||||
"/go/src/app", which allows us to build and run it under the proper package
|
||||
name.
|
||||
|
||||
For compatibility with versions of Go older than 1.4, the "import path" may also
|
||||
be placed in a file named ".godir".
|
||||
|
||||
Available Commands:
|
||||
|
||||
$base download
|
||||
$base download -u
|
||||
(equivalent to "go get -d [args] [godir]")
|
||||
|
||||
$base install
|
||||
$base install -race
|
||||
(equivalent to "go install [args] [godir]")
|
||||
|
||||
$base run
|
||||
$base run -app -specific -arguments
|
||||
(assumes "GOPATH/bin" is in "PATH")
|
||||
|
||||
EOUSAGE
|
||||
}
|
||||
|
||||
# make sure there is a subcommand specified
|
||||
if [ "$#" -eq 0 ]; then
|
||||
usage >&2
|
||||
exit 1
|
||||
fi
|
||||
# "shift" so that "$@" becomes the remaining arguments and can be passed along to other "go" subcommands easily
|
||||
cmd="$1"
|
||||
shift
|
||||
|
||||
goDir="$(go list -e -f '{{.ImportComment}}' 2>/dev/null || true)"
|
||||
|
||||
if [ -z "$goDir" -a -s .godir ]; then
|
||||
goDir="$(cat .godir)"
|
||||
fi
|
||||
|
||||
dir="$(pwd -P)"
|
||||
if [ "$goDir" ]; then
|
||||
goPath="${GOPATH%%:*}" # this just grabs the first path listed in GOPATH, if there are multiple (which is the detection logic "go get" itself uses, too)
|
||||
goDirPath="$goPath/src/$goDir"
|
||||
mkdir -p "$(dirname "$goDirPath")"
|
||||
if [ ! -e "$goDirPath" ]; then
|
||||
ln -sfv "$dir" "$goDirPath"
|
||||
elif [ ! -L "$goDirPath" ]; then
|
||||
echo >&2 "error: $goDirPath already exists but is unexpectedly not a symlink!"
|
||||
exit 1
|
||||
fi
|
||||
goBin="$goPath/bin/$(basename "$goDir")"
|
||||
else
|
||||
goBin="$(basename "$dir")" # likely "app"
|
||||
fi
|
||||
|
||||
case "$cmd" in
|
||||
download)
|
||||
set -- go get -v -d "$@"
|
||||
if [ "$goDir" ]; then set -- "$@" "$goDir"; fi
|
||||
set -x; exec "$@"
|
||||
;;
|
||||
|
||||
install)
|
||||
set -- go install -v "$@"
|
||||
if [ "$goDir" ]; then set -- "$@" "$goDir"; fi
|
||||
set -x; exec "$@"
|
||||
;;
|
||||
|
||||
run)
|
||||
set -x; exec "$goBin" "$@"
|
||||
;;
|
||||
|
||||
*)
|
||||
echo >&2 'error: unknown command:' "$cmd"
|
||||
usage >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
Loading…
Reference in New Issue
Block a user