mirror of
https://github.com/golang/go
synced 2024-11-18 11:55:01 -07:00
39380e8e01
The span set data structure may leak blocks due to a race in the logic to check whether it's safe to free a block. The simplest example of this race is between two poppers: 1. Popper A claims slot spanSetEntries-2. 2. Popper B claims slot spanSetEntries-1. 3. Popper A gets descheduled before it subtracts from block.used. 4. Popper B subtracts from block.used, sees that claimed spanSetEntries-1, but also that block.used != 0, so it returns. 5. Popper A comes back and subtracts from block.used, but it didn't claim spanSetEntries-1 so it also returns. The spine is left with a stale block pointer and the block later gets overwritten by pushes, never to be re-used again. The problem here is that we designate the claimer of slot spanSetEntries-1 to be the one who frees the block, but that may not be the thread that actually does the last subtraction from block.used. Fixing this problem is tricky, and the fundamental problem there is that block.used is not stable: it may be observed to be zero, but that doesn't necessarily mean you're the last popper! Do something simpler: keep a counter of how many pops have happened to a given block instead of block.used. This counter monotonically increases when a pop is _completely done_. Because this counter is monotonically increasing, and only increases when a popper is done, then we know for sure whichever popper is the last to increase it (i.e. its value is spanSetBlockEntries) is also the last popper in the block. Because the race described above still exists, the last popper may not be the one which claimed the last slot in the block, but we know for certain nobody else is popping from that block anymore so we can safely free it. Finally, because pops serialize with pushes to the same slot, we need not worry about concurrent pushers at all. Updates #37487. Change-Id: I6697219372774c8ca7d8ee6895eaa230a64ce9e1 Reviewed-on: https://go-review.googlesource.com/c/go/+/230497 Run-TryBot: Michael Knyszek <mknyszek@google.com> Reviewed-by: Michael Pratt <mpratt@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> |
||
|---|---|---|
| .. | ||
| archive | ||
| bufio | ||
| builtin | ||
| bytes | ||
| cmd | ||
| compress | ||
| container | ||
| context | ||
| crypto | ||
| database/sql | ||
| debug | ||
| encoding | ||
| errors | ||
| expvar | ||
| flag | ||
| fmt | ||
| go | ||
| hash | ||
| html | ||
| image | ||
| index/suffixarray | ||
| internal | ||
| io | ||
| log | ||
| math | ||
| mime | ||
| net | ||
| os | ||
| path | ||
| plugin | ||
| reflect | ||
| regexp | ||
| runtime | ||
| sort | ||
| strconv | ||
| strings | ||
| sync | ||
| syscall | ||
| testdata | ||
| testing | ||
| text | ||
| time | ||
| unicode | ||
| unsafe | ||
| vendor | ||
| all.bash | ||
| all.bat | ||
| all.rc | ||
| bootstrap.bash | ||
| buildall.bash | ||
| clean.bash | ||
| clean.bat | ||
| clean.rc | ||
| cmp.bash | ||
| go.mod | ||
| go.sum | ||
| iostest.bash | ||
| make.bash | ||
| make.bat | ||
| Make.dist | ||
| make.rc | ||
| race.bash | ||
| race.bat | ||
| README.vendor | ||
| run.bash | ||
| run.bat | ||
| run.rc | ||
Vendoring in std and cmd
========================
The Go command maintains copies of external packages needed by the
standard library in the src/vendor and src/cmd/vendor directories.
In GOPATH mode, imports of vendored packages are resolved to these
directories following normal vendor directory logic
(see golang.org/s/go15vendor).
In module mode, std and cmd are modules (defined in src/go.mod and
src/cmd/go.mod). When a package outside std or cmd is imported
by a package inside std or cmd, the import path is interpreted
as if it had a "vendor/" prefix. For example, within "crypto/tls",
an import of "golang.org/x/crypto/cryptobyte" resolves to
"vendor/golang.org/x/crypto/cryptobyte". When a package with the
same path is imported from a package outside std or cmd, it will
be resolved normally. Consequently, a binary may be built with two
copies of a package at different versions if the package is
imported normally and vendored by the standard library.
Vendored packages are internally renamed with a "vendor/" prefix
to preserve the invariant that all packages have distinct paths.
This is necessary to avoid compiler and linker conflicts. Adding
a "vendor/" prefix also maintains the invariant that standard
library packages begin with a dotless path element.
The module requirements of std and cmd do not influence version
selection in other modules. They are only considered when running
module commands like 'go get' and 'go mod vendor' from a directory
in GOROOT/src.
Maintaining vendor directories
==============================
Before updating vendor directories, ensure that module mode is enabled.
Make sure GO111MODULE=off is not set ('on' or 'auto' should work).
Requirements may be added, updated, and removed with 'go get'.
The vendor directory may be updated with 'go mod vendor'.
A typical sequence might be:
cd src
go get -d golang.org/x/net@latest
go mod tidy
go mod vendor
Use caution when passing '-u' to 'go get'. The '-u' flag updates
modules providing all transitively imported packages, not only
the module providing the target package.
Note that 'go mod vendor' only copies packages that are transitively
imported by packages in the current module. If a new package is needed,
it should be imported before running 'go mod vendor'.