bc1da38c3d
Add a new function, WithDataIndependentTiming, which takes a function as an argument, and encloses it with calls to set/unset the DIT PSTATE bit on Arm64. Since DIT is OS thread-local, for the duration of the execution of WithDataIndependentTiming, we lock the goroutine to the OS thread, using LockOSThread. For long running operations, this is likely to not be performant, but we expect this to be tightly scoped around cryptographic operations that have bounded execution times. If locking to the OS thread turns out to be too slow, another option is to add a bit to the g state indicating if a goroutine has DIT enabled, and then have the scheduler enable/disable DIT when scheduling a g. Additionally, we add a new GODEBUG, dataindependenttiming, which allows setting DIT for an entire program. Running a program with dataindependenttiming=1 enables DIT for the program during initialization. In an ideal world PSTATE.DIT would be inherited from the parent thread, so we'd only need to set it in the main thread and then all subsequent threads would inherit the value. While this does happen in the Linux kernel [0], it is not the case for darwin [1]. Rather than add complex logic to only set it on darwin for each new thread, we just unconditionally set it in mstart1 and cgocallbackg1 regardless of the OS. DIT will already impose some overhead, and the cost of setting the bit is only ~two instructions (CALL, MSR), so it should be cheap enough. Fixes #66450 Updates #49702 [0]e8bdb3c8be/arch/arm64/kernel/process.c (L373)[1]8d741a5de7/osfmk/arm64/status.c (L1666)Change-Id: I78eda691ff9254b0415f2b54770e5850a0179749 Reviewed-on: https://go-review.googlesource.com/c/go/+/598336 Reviewed-by: Michael Knyszek <mknyszek@google.com> Reviewed-by: Filippo Valsorda <filippo@golang.org> Reviewed-by: Michael Pratt <mpratt@google.com> LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
18 KiB
| title | layout |
|---|---|
| Go, Backwards Compatibility, and GODEBUG | article |
Introduction
Go's emphasis on backwards compatibility is one of its key strengths. There are, however, times when we cannot maintain complete compatibility. If code depends on buggy (including insecure) behavior, then fixing the bug will break that code. New features can also have similar impacts: enabling the HTTP/2 use by the HTTP client broke programs connecting to servers with buggy HTTP/2 implementations. These kinds of changes are unavoidable and permitted by the Go 1 compatibility rules. Even so, Go provides a mechanism called GODEBUG to reduce the impact such changes have on Go developers using newer toolchains to compile old code.
A GODEBUG setting is a key=value pair
that controls the execution of certain parts of a Go program.
The environment variable GODEBUG
can hold a comma-separated list of these settings.
For example, if a Go program is running in an environment that contains
GODEBUG=http2client=0,http2server=0
then that Go program will disable the use of HTTP/2 by default in both
the HTTP client and the HTTP server.
Unrecognized settings in the GODEBUG environment variable are ignored.
It is also possible to set the default GODEBUG for a given program
(discussed below).
When preparing any change that is permitted by Go 1 compatibility but may nonetheless break some existing programs, we first engineer the change to keep as many existing programs working as possible. For the remaining programs, we define a new GODEBUG setting that allows individual programs to opt back in to the old behavior. A GODEBUG setting may not be added if doing so is infeasible, but that should be extremely rare.
GODEBUG settings added for compatibility will be maintained
for a minimum of two years (four Go releases).
Some, such as http2client and http2server,
will be maintained much longer, even indefinitely.
When possible, each GODEBUG setting has an associated
runtime/metrics counter
named /godebug/non-default-behavior/<name>:events
that counts the number of times a particular program's
behavior has changed based on a non-default value
for that setting.
For example, when GODEBUG=http2client=0 is set,
/godebug/non-default-behavior/http2client:events
counts the number of HTTP transports that the program
has configured without HTTP/2 support.
Default GODEBUG Values
When a GODEBUG setting is not listed in the environment variable,
its value is derived from three sources:
the defaults for the Go toolchain used to build the program,
amended to match the Go version listed in go.mod,
and then overridden by explicit //go:debug lines in the program.
The GODEBUG History gives the exact defaults for each Go toolchain version.
For example, Go 1.21 introduces the panicnil setting,
controlling whether panic(nil) is allowed;
it defaults to panicnil=0, making panic(nil) a run-time error.
Using panicnil=1 restores the behavior of Go 1.20 and earlier.
When compiling a work module or workspace that declares
an older Go version, the Go toolchain amends its defaults
to match that older Go version as closely as possible.
For example, when a Go 1.21 toolchain compiles a program,
if the work module's go.mod or the workspace's go.work
says go 1.20, then the program defaults to panicnil=1,
matching Go 1.20 instead of Go 1.21.
Because this method of setting GODEBUG defaults was introduced only in Go 1.21, programs listing versions of Go earlier than Go 1.20 are configured to match Go 1.20, not the older version.
To override these defaults, starting in Go 1.23, the work module's go.mod
or the workspace's go.work can list one or more godebug lines:
godebug (
default=go1.21
panicnil=1
asynctimerchan=0
)
The special key default indicates a Go version to take unspecified
settings from. This allows setting the GODEBUG defaults separately
from the Go language version in the module.
In this example, the program is asking for Go 1.21 semantics and
then asking for the old pre-Go 1.21 panic(nil) behavior and the
new Go 1.23 asynctimerchan=0 behavior.
Only the work module's go.mod is consulted for godebug directives.
Any directives in required dependency modules are ignored.
It is an error to list a godebug with an unrecognized setting.
(Toolchains older than Go 1.23 reject all godebug lines, since they do not
understand godebug at all.)
The defaults from the go and godebug lines apply to all main
packages that are built. For more fine-grained control,
starting in Go 1.21, a main package's source files
can include one or more //go:debug directives at the top of the file
(preceding the package statement).
The godebug lines in the previous example would be written:
//go:debug default=go1.21
//go:debug panicnil=1
//go:debug asynctimerchan=0
Starting in Go 1.21, the Go toolchain treats a //go:debug directive
with an unrecognized GODEBUG setting as an invalid program.
Programs with more than one //go:debug line for a given setting
are also treated as invalid.
(Older toolchains ignore //go:debug directives entirely.)
The defaults that will be compiled into a main package are reported by the command:
{{raw go list -f '{{.DefaultGODEBUG}}' my/main/package}}
Only differences from the base Go toolchain defaults are reported.
When testing a package, //go:debug lines in the *_test.go
files are treated as directives for the test's main package.
In any other context, //go:debug lines are ignored by the toolchain;
go vet reports such lines as misplaced.
GODEBUG History
This section documents the GODEBUG settings introduced and removed in each major Go release for compatibility reasons. Packages or programs may define additional settings for internal debugging purposes; for example, see the runtime documentation and the go command documentation.
Go 1.24
Go 1.24 changed the global math/rand.Seed to be a
no-op. This behavior is controlled by the randseednop setting.
For Go 1.24 it defaults to randseednop=1.
Using randseednop=0 reverts to the pre-Go 1.24 behavior.
Go 1.24 added new values for the multipathtcp setting.
The possible values for multipathtcp are now:
- "0": disable MPTCP on dialers and listeners by default
- "1": enable MPTCP on dialers and listeners by default
- "2": enable MPTCP on listeners only by default
- "3": enable MPTCP on dialers only by default
For Go 1.24, it now defaults to multipathtcp="2", thus enabled by default on listeners. Using multipathtcp="0" reverts to the pre-Go 1.24 behavior.
Go 1.24 changed the behavior of go test -json to emit build errors as JSON
instead of text.
These new JSON events are distinguished by new Action values,
but can still cause problems with CI systems that aren't robust to these events.
This behavior can be controlled with the gotestjsonbuildtext setting.
Using gotestjsonbuildtext=1 restores the 1.23 behavior.
This setting will be removed in a future release, Go 1.28 at the earliest.
Go 1.24 introduced a mechanism for enabling platform specific Data Independent
Timing (DIT) modes in the crypto/subtle package. This
mode can be enabled for an entire program with the dataindependenttiming setting.
For Go 1.24 it defaults to dataindependenttiming=0. There is no change in default
behavior from Go 1.23 when dataindependenttiming is unset.
Using dataindependenttiming=1 enables the DIT mode for the entire Go program.
When enabled, DIT will be enabled when calling into C from Go. When enabled,
calling into Go code from C will enable DIT, and disable it before returning to
C if it was not enabled when Go code was entered.
This currently only affects arm64 programs. For all other platforms it is a no-op.
Go 1.23
Go 1.23 changed the channels created by package time to be unbuffered
(synchronous), which makes correct use of the Timer.Stop
and Timer.Reset method results much easier.
The asynctimerchan setting disables this change.
There are no runtime metrics for this change,
This setting may be removed in a future release, Go 1.27 at the earliest.
Go 1.23 changed the mode bits reported by os.Lstat and os.Stat
for reparse points, which can be controlled with the winsymlink setting.
As of Go 1.23 (winsymlink=1), mount points no longer have os.ModeSymlink
set, and reparse points that are not symlinks, Unix sockets, or dedup files now
always have os.ModeIrregular set. As a result of these changes,
filepath.EvalSymlinks no longer evaluates
mount points, which was a source of many inconsistencies and bugs.
At previous versions (winsymlink=0), mount points are treated as symlinks,
and other reparse points with non-default os.ModeType bits
(such as os.ModeDir) do not have the ModeIrregular bit set.
Go 1.23 changed os.Readlink and filepath.EvalSymlinks
to avoid trying to normalize volumes to drive letters, which was not always even possible.
This behavior is controlled by the winreadlinkvolume setting.
For Go 1.23, it defaults to winreadlinkvolume=1.
Previous versions default to winreadlinkvolume=0.
Go 1.23 enabled the experimental post-quantum key exchange mechanism
X25519Kyber768Draft00 by default. The default can be reverted using the
tlskyber setting.
Go 1.23 changed the behavior of
crypto/x509.ParseCertificate to reject
serial numbers that are negative. This change can be reverted with
the x509negativeserial setting.
Go 1.23 re-enabled support in html/template for ECMAScript 6 template literals by default.
The jstmpllitinterp setting no longer has
any effect.
Go 1.23 changed the default TLS cipher suites used by clients and servers when
not explicitly configured, removing 3DES cipher suites. The default can be reverted
using the tls3des setting.
Go 1.23 changed the behavior of tls.X509KeyPair
and tls.LoadX509KeyPair to populate the
Leaf field of the returned tls.Certificate.
This behavior is controlled by the x509keypairleaf setting. For Go 1.23, it
defaults to x509keypairleaf=1. Previous versions default to
x509keypairleaf=0.
Go 1.23 changed
net/http.ServeContent,
net/http.ServeFile, and
net/http.ServeFS to
remove Cache-Control, Content-Encoding, Etag, and Last-Modified headers
when serving an error. This behavior is controlled by
the httpservecontentkeepheaders setting.
Using httpservecontentkeepheaders=1 restores the pre-Go 1.23 behavior.
Go 1.22
Go 1.22 adds a configurable limit to control the maximum acceptable RSA key size
that can be used in TLS handshakes, controlled by the tlsmaxrsasize setting.
The default is tlsmaxrsasize=8192, limiting RSA to 8192-bit keys. To avoid
denial of service attacks, this setting and default was backported to Go
1.19.13, Go 1.20.8, and Go 1.21.1.
Go 1.22 made it an error for a request or response read by a net/http
client or server to have an empty Content-Length header.
This behavior is controlled by the httplaxcontentlength setting.
Go 1.22 changed the behavior of ServeMux to accept extended
patterns and unescape both patterns and request paths by segment.
This behavior can be controlled by the
httpmuxgo121 setting.
Go 1.22 added the Alias type to go/types
for the explicit representation of type aliases.
Whether the type checker produces Alias types or not is controlled by the
gotypesalias setting.
For Go 1.22 it defaults to gotypesalias=0.
For Go 1.23, gotypesalias=1 will become the default.
This setting will be removed in a future release, Go 1.27 at the earliest.
Go 1.22 changed the default minimum TLS version supported by both servers
and clients to TLS 1.2. The default can be reverted to TLS 1.0 using the
tls10server setting.
Go 1.22 changed the default TLS cipher suites used by clients and servers when
not explicitly configured, removing the cipher suites which used RSA based key
exchange. The default can be reverted using the tlsrsakex setting.
Go 1.22 disabled
ConnectionState.ExportKeyingMaterial
when the connection supports neither TLS 1.3 nor Extended Master Secret
(implemented in Go 1.21). It can be reenabled with the tlsunsafeekm
setting.
Go 1.22 changed how the runtime interacts with transparent huge pages on Linux.
In particular, a common default Linux kernel configuration can result in
significant memory overheads, and Go 1.22 no longer works around this default.
To work around this issue without adjusting kernel settings, transparent huge
pages can be disabled for Go memory with the
disablethp setting.
This behavior was backported to Go 1.21.1, but the setting is only available
starting with Go 1.21.6.
This setting may be removed in a future release, and users impacted by this issue
should adjust their Linux configuration according to the recommendations in the
GC guide, or switch to a Linux
distribution that disables transparent huge pages altogether.
Go 1.22 added contention on runtime-internal locks to the mutex
profile. Contention on these locks is always
reported at runtime._LostContendedRuntimeLock. Complete stack traces of
runtime locks can be enabled with the runtimecontentionstacks
setting. These stack traces have
non-standard semantics, see setting documentation for details.
Go 1.22 added a new crypto/x509.Certificate
field, Policies, which supports
certificate policy OIDs with components larger than 31 bits. By default this
field is only used during parsing, when it is populated with policy OIDs, but
not used during marshaling. It can be used to marshal these larger OIDs, instead
of the existing PolicyIdentifiers field, by using the
x509usepolicies setting..
Go 1.21
Go 1.21 made it a run-time error to call panic with a nil interface value,
controlled by the panicnil setting.
Go 1.21 made it an error for html/template actions to appear inside of an ECMAScript 6
template literal, controlled by the
jstmpllitinterp setting.
This behavior was backported to Go 1.19.8+ and Go 1.20.3+.
Go 1.21 introduced a limit on the maximum number of MIME headers and multipart
forms, controlled by the
multipartmaxheaders and multipartmaxparts settings
respectively.
This behavior was backported to Go 1.19.8+ and Go 1.20.3+.
Go 1.21 adds the support of Multipath TCP but it is only used if the application
explicitly asked for it. This behavior can be controlled by the
multipathtcp setting.
There is no plan to remove any of these settings.
Go 1.20
Go 1.20 introduced support for rejecting insecure paths in tar and zip archives,
controlled by the tarinsecurepath setting
and the zipinsecurepath setting.
These default to tarinsecurepath=1 and zipinsecurepath=1,
preserving the behavior of earlier versions of Go.
A future version of Go may change the defaults to
tarinsecurepath=0 and zipinsecurepath=0.
Go 1.20 introduced automatic seeding of the
math/rand global random number generator,
controlled by the randautoseed setting.
Go 1.20 introduced the concept of fallback roots for use during certificate verification,
controlled by the x509usefallbackroots setting.
Go 1.20 removed the preinstalled .a files for the standard library
from the Go distribution.
Installations now build and cache the standard library like
packages in other modules.
The installgoroot setting
restores the installation and use of preinstalled .a files.
There is no plan to remove any of these settings.
Go 1.19
Go 1.19 made it an error for path lookups to resolve to binaries in the current directory,
controlled by the execerrdot setting.
There is no plan to remove this setting.
Go 1.19 started sending EDNS0 additional headers on DNS requests.
This can reportedly break the DNS server provided on some routers,
such as CenturyLink Zyxel C3000Z.
This can be changed by the netedns0 setting.
This setting is available in Go 1.21.12, Go 1.22.5, Go 1.23, and later.
There is no plan to remove this setting.
Go 1.18
Go 1.18 removed support for SHA1 in most X.509 certificates,
controlled by the x509sha1 setting.
This setting will be removed in a future release, Go 1.22 at the earliest.
Go 1.10
Go 1.10 changed how build caching worked and added test caching, along
with the gocacheverify, gocachehash, and gocachetest settings.
There is no plan to remove these settings.
Go 1.6
Go 1.6 introduced transparent support for HTTP/2,
controlled by the http2client, http2server, and http2debug settings.
There is no plan to remove these settings.
Go 1.5
Go 1.5 introduced a pure Go DNS resolver,
controlled by the netdns setting.
There is no plan to remove this setting.