1
0
mirror of https://github.com/golang/go synced 2024-11-16 19:34:48 -07:00
go/src
Michael Anthony Knyszek e1b5f347e7 runtime: reduce max idle mark workers during periodic GC cycles
This change reduces the maximum number of idle mark workers during
periodic (currently every 2 minutes) GC cycles to 1.

Idle mark workers soak up all available and unused Ps, up to GOMAXPROCS.
While this provides some throughput and latency benefit in general, it
can cause what appear to be massive CPU utilization spikes in otherwise
idle applications. This is mostly an issue for *very* idle applications,
ones idle enough to trigger periodic GC cycles. This spike also tends to
interact poorly with auto-scaling systems, as the system might assume
the load average is very low and suddenly see a massive burst in
activity.

The result of this change is not to bring down this 100% (of GOMAXPROCS)
CPU utilization spike to 0%, but rather

  min(25% + 1/GOMAXPROCS*100%, 100%)

Idle mark workers also do incur a small latency penalty as they must be
descheduled for other work that might pop up. Luckily the runtime is
pretty good about getting idle mark workers off of Ps, so in general
the latency benefit from shorter GC cycles outweighs this cost. But, the
cost is still non-zero and may be more significant in idle applications
that aren't invoking assists and write barriers quite as often.

We can't completely eliminate idle mark workers because they're
currently necessary for GC progress in some circumstances. Namely,
they're critical for progress when all we have is fractional workers. If
a fractional worker meets its quota, and all user goroutines are blocked
directly or indirectly on a GC cycle (via runtime.GOMAXPROCS, or
runtime.GC), the program may deadlock without GC workers, since the
fractional worker will go to sleep with nothing to wake it.

Fixes #37116.
For #44163.

Change-Id: Ib74793bb6b88d1765c52d445831310b0d11ef423
Reviewed-on: https://go-review.googlesource.com/c/go/+/393394
Reviewed-by: Michael Pratt <mpratt@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
2022-04-26 22:08:42 +00:00
..
archive
bufio bufio: implement large write forwarding in Writer.WriteString 2022-04-21 23:41:03 +00:00
builtin
bytes bytes: explode checks for n too large 2022-04-16 02:01:19 +00:00
cmd cmd/compile/internal/syntax: parser to accept ~x as unary expression 2022-04-26 02:19:42 +00:00
compress
container
context
crypto crypto/ed25519: test fixup 2022-04-25 14:52:51 +00:00
database/sql
debug debug/pe: fix off by one error in valid symbol index test 2022-04-25 15:42:53 +00:00
embed
encoding encoding/binary: add AppendVarint AppendUvarint 2022-04-15 01:19:37 +00:00
errors
expvar
flag
fmt
go go/types, types2: add loong64 to gcArchSizes 2022-04-25 13:37:03 +00:00
hash hash/maphash: use fastrand64 in MakeSeed 2022-04-21 17:46:04 +00:00
html
image
index/suffixarray
internal internal/bytealg: optimize cmpbody for ppc64le/ppc64 2022-04-22 12:12:38 +00:00
io io/ioutil: provide an equivalent for the deprecated ReadDir 2022-04-19 16:19:27 +00:00
log
math math: improve documentation of Copysign 2022-04-14 17:42:53 +00:00
mime mime: ignore non-extension globs2 entries 2022-04-15 17:07:27 +00:00
net net: skip TestDialCancel on darwin-arm64 2022-04-26 17:50:49 +00:00
os os/exec: use a TestMain to avoid hijacking stdout for helper commands 2022-04-26 14:49:07 +00:00
path
plugin
reflect reflect: support Len and Cap on pointer-to-array Value 2022-04-25 23:18:00 +00:00
regexp regexp/syntax: rename ErrInvalidDepth to ErrNestingDepth 2022-04-22 22:35:03 +00:00
runtime runtime: reduce max idle mark workers during periodic GC cycles 2022-04-26 22:08:42 +00:00
sort slices: use !{{Less}} instead of {{GreaterOrEqual}} 2022-04-25 19:12:14 +00:00
strconv
strings
sync
syscall syscall: define Syscall6 in terms of RawSyscall6 on linux 2022-04-22 19:02:29 +00:00
testdata
testing
text
time time: fix quickcheck test to avoid wraparounds 2022-04-26 02:28:58 +00:00
unicode
unsafe
vendor
all.bash
all.bat
all.rc
bootstrap.bash cmd/trace: embed static content 2022-04-21 21:18:18 +00:00
buildall.bash
clean.bash
clean.bat
clean.rc
cmp.bash
go.mod
go.sum
make.bash
make.bat
Make.dist
make.rc
race.bash
race.bat
README.vendor
run.bash
run.bat
run.rc

Vendoring in std and cmd
========================

The Go command maintains copies of external packages needed by the
standard library in the src/vendor and src/cmd/vendor directories.

In GOPATH mode, imports of vendored packages are resolved to these
directories following normal vendor directory logic
(see golang.org/s/go15vendor).

In module mode, std and cmd are modules (defined in src/go.mod and
src/cmd/go.mod). When a package outside std or cmd is imported
by a package inside std or cmd, the import path is interpreted
as if it had a "vendor/" prefix. For example, within "crypto/tls",
an import of "golang.org/x/crypto/cryptobyte" resolves to
"vendor/golang.org/x/crypto/cryptobyte". When a package with the
same path is imported from a package outside std or cmd, it will
be resolved normally. Consequently, a binary may be built with two
copies of a package at different versions if the package is
imported normally and vendored by the standard library.

Vendored packages are internally renamed with a "vendor/" prefix
to preserve the invariant that all packages have distinct paths.
This is necessary to avoid compiler and linker conflicts. Adding
a "vendor/" prefix also maintains the invariant that standard
library packages begin with a dotless path element.

The module requirements of std and cmd do not influence version
selection in other modules. They are only considered when running
module commands like 'go get' and 'go mod vendor' from a directory
in GOROOT/src.

Maintaining vendor directories
==============================

Before updating vendor directories, ensure that module mode is enabled.
Make sure GO111MODULE=off is not set ('on' or 'auto' should work).

Requirements may be added, updated, and removed with 'go get'.
The vendor directory may be updated with 'go mod vendor'.
A typical sequence might be:

    cd src
    go get -d golang.org/x/net@latest
    go mod tidy
    go mod vendor

Use caution when passing '-u' to 'go get'. The '-u' flag updates
modules providing all transitively imported packages, not only
the module providing the target package.

Note that 'go mod vendor' only copies packages that are transitively
imported by packages in the current module. If a new package is needed,
it should be imported before running 'go mod vendor'.