1
0
mirror of https://github.com/golang/go synced 2024-11-07 10:36:19 -07:00
go/src
Austin Clements a25a77aed2 runtime: block sweep completion on all sweep paths
The runtime currently has two different notions of sweep completion:

1. All spans are either swept or have begun sweeping.

2. The sweeper has *finished* sweeping all spans.

Most things depend on condition 1. Notably, GC correctness depends on
condition 1, but since all sweep operations a non-preemptible, the STW
at the beginning of GC forces condition 1 to become condition 2.

runtime.GC(), however, depends on condition 2, since the intent is to
complete a complete GC cycle, and also update the heap profile (which
can only be done after sweeping is complete).

However, the way we compute condition 2 is racy right now and may in
fact only indicate condition 1. Specifically, sweepone blocks
condition 2 until all sweepone calls are done, but there are many
other ways to enter the sweeper that don't block this. Hence, sweepone
may see that there are no more spans in the sweep list and see that
it's the last sweepone and declare sweeping done, while there's some
other sweeper still working on a span.

Fix this by making sure every entry to the sweeper participates in the
protocol that blocks condition 2. To make sure we get this right, this
CL introduces a type to track sweep blocking and (lightly) enforces
span sweep ownership via the type system. This has the nice
side-effect of abstracting the pattern of acquiring sweep ownership
that's currently repeated in many different places.

Fixes #45315.

Change-Id: I7fab30170c5ae14c8b2f10998628735b8be6d901
Reviewed-on: https://go-review.googlesource.com/c/go/+/307915
Trust: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2021-04-12 19:22:50 +00:00
..
archive archive/zip: fix character device handling in fileModeToUnixMode 2021-04-03 20:16:00 +00:00
bufio
builtin
bytes
cmd cmd/link: fix TestLargeText 2021-04-12 18:32:27 +00:00
compress
container
context all: fix spellings 2021-04-10 01:46:41 +00:00
crypto all: fix spellings 2021-04-10 01:46:41 +00:00
database/sql all: fix spellings 2021-04-10 01:46:41 +00:00
debug
embed embed, testing/fstest: small optimization for ReadDir 2021-04-07 09:59:58 +00:00
encoding
errors
expvar
flag flag: use strings.Builder instead of concatenating strings 2021-04-05 20:39:06 +00:00
fmt
go internal/goexperiment: consolidate experiment-enabled constants 2021-04-08 02:17:18 +00:00
hash
html
image
index/suffixarray
internal internal/poll: ensure that newPoolPipe doesn't return a nil pointer 2021-04-12 17:32:47 +00:00
io io/fs: implement subFS.Sub 2021-04-03 20:14:37 +00:00
log
math
mime
net net: fix (*ipStackCapabilities).probe godoc 2021-04-12 05:11:59 +00:00
os os/signal: replace os.MkdirTemp with T.TempDir 2021-04-12 00:45:40 +00:00
path path/filepath: replace os.MkdirTemp with T.TempDir 2021-04-10 08:13:32 +00:00
plugin
reflect reflect: panic on New of go:notinheap type 2021-04-09 23:54:31 +00:00
regexp
runtime runtime: block sweep completion on all sweep paths 2021-04-12 19:22:50 +00:00
sort
strconv
strings
sync sync: update misleading comment in map.go about entry type 2021-04-08 14:08:29 +00:00
syscall syscall: replace os.MkdirTemp with T.TempDir 2021-04-07 23:00:58 +00:00
testdata
testing embed, testing/fstest: small optimization for ReadDir 2021-04-07 09:59:58 +00:00
text
time time: properly quote strings containing quotes and backslashes 2021-04-06 19:18:47 +00:00
unicode
unsafe
vendor
all.bash
all.bat
all.rc
bootstrap.bash
buildall.bash
clean.bash
clean.bat
clean.rc
cmp.bash
go.mod
go.sum
make.bash
make.bat
Make.dist
make.rc
race.bash
race.bat
README.vendor
run.bash
run.bat
run.rc

Vendoring in std and cmd
========================

The Go command maintains copies of external packages needed by the
standard library in the src/vendor and src/cmd/vendor directories.

In GOPATH mode, imports of vendored packages are resolved to these
directories following normal vendor directory logic
(see golang.org/s/go15vendor).

In module mode, std and cmd are modules (defined in src/go.mod and
src/cmd/go.mod). When a package outside std or cmd is imported
by a package inside std or cmd, the import path is interpreted
as if it had a "vendor/" prefix. For example, within "crypto/tls",
an import of "golang.org/x/crypto/cryptobyte" resolves to
"vendor/golang.org/x/crypto/cryptobyte". When a package with the
same path is imported from a package outside std or cmd, it will
be resolved normally. Consequently, a binary may be built with two
copies of a package at different versions if the package is
imported normally and vendored by the standard library.

Vendored packages are internally renamed with a "vendor/" prefix
to preserve the invariant that all packages have distinct paths.
This is necessary to avoid compiler and linker conflicts. Adding
a "vendor/" prefix also maintains the invariant that standard
library packages begin with a dotless path element.

The module requirements of std and cmd do not influence version
selection in other modules. They are only considered when running
module commands like 'go get' and 'go mod vendor' from a directory
in GOROOT/src.

Maintaining vendor directories
==============================

Before updating vendor directories, ensure that module mode is enabled.
Make sure GO111MODULE=off is not set ('on' or 'auto' should work).

Requirements may be added, updated, and removed with 'go get'.
The vendor directory may be updated with 'go mod vendor'.
A typical sequence might be:

    cd src
    go get -d golang.org/x/net@latest
    go mod tidy
    go mod vendor

Use caution when passing '-u' to 'go get'. The '-u' flag updates
modules providing all transitively imported packages, not only
the module providing the target package.

Note that 'go mod vendor' only copies packages that are transitively
imported by packages in the current module. If a new package is needed,
it should be imported before running 'go mod vendor'.