1
0
mirror of https://github.com/golang/go synced 2024-11-24 23:07:56 -07:00
Commit Graph

61600 Commits

Author SHA1 Message Date
Maksym Sobolyev
e707d47326 runtime: utilize EVFILT_USER more effectively
Re-work kqueue_event wakeup logic to use one-shot events. In an
event of waking up a wrong thread, simply re-post the event.

This saves close to 1 system call per wakeup on average, since
chances of non-blocking poller picking it up is pretty low.
2024-11-19 12:28:25 -08:00
Filippo Valsorda
bedde1bee0 crypto: check all cpu.X86 flags for features used in assembly
These are most likely redundant, but cmd/compile/internal/amd64's
TestGoAMD64v1 turns them off when clobbering those instructions, so we
need to know to skip the assembly in those cases.

Thankfully we have Avo now that adds a helpful comment with the list of
features used by each generated function!

Also improve the error output of TestGoAMD64v1. It had broken before in
#49402 and had required the exact same patch.

Change-Id: I7fab8f36042cdff630f806723aa1d8124c294f60
Reviewed-on: https://go-review.googlesource.com/c/go/+/626876
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-19 15:48:59 +00:00
Filippo Valsorda
e66229a22a crypto/internal/fips/aes: add service indicators
For #69536

Change-Id: I485c165b1d9fcd7b09ff151bbeebc57d8908bcb8
Reviewed-on: https://go-review.googlesource.com/c/go/+/626835
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
2024-11-19 14:36:19 +00:00
Russ Cox
8f22369136 cmd/go/internal/fsys: refactor overlay consistency checks
Do the overlay consistency checks separate from constructing
the overlay data structure. This makes sure that the data structure
can be changed without worrying about losing the checks.

Change-Id: I9ff50cc366b5362adc5570f94e6caf646ddf5046
Reviewed-on: https://go-review.googlesource.com/c/go/+/628700
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2024-11-19 05:49:58 +00:00
Russ Cox
8767a090d5 cmd/go/internal/fsys: refactor vfs lookup
Refactor vfs lookup into 'func stat', which knows the internal
data structures for the vfs and returns information about a
given path. The callers can then all use stat and avoid direct
knowledge of the internal data structures.

This is setting up for a different internal data structure.

Change-Id: I496b7b3fb686cdde81b14687f65eb0bf51ec62be
Reviewed-on: https://go-review.googlesource.com/c/go/+/628699
Auto-Submit: Russ Cox <rsc@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2024-11-19 05:42:03 +00:00
Russ Cox
7ded55f941 cmd/go/internal/fsys: convert to proper ReadDir
Many releases ago we migrated
from ioutil.ReadDir, which returned []os.FileInfo,
to os.ReadDir, which returns []fs.DirEntry.
The latter is faster, but the former is expected by go/build.Context.

Convert fsys to use the new ReadDir signature.
This should make the go command faster when scanning
source trees, and it brings cmd/go up to date with the rest
of the tree.

Similarly, convert Walk to WalkDir.

Change-Id: I767a8548d7ca7cc3c05f2ff073d18070a4e8a0da
Reviewed-on: https://go-review.googlesource.com/c/go/+/628698
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
2024-11-19 05:07:42 +00:00
Russ Cox
f174c31f3e cmd/go/internal/fsys: minor cleanup
Rename canonicalize to abs.
Rename IsDirWithGoFiles to IsGoDir.
Remove Init argument.
Split OverlayPath into Actual and Renamed.
Clean up doc comments.
Other minor cleanups.

Preparation for larger changes.

Change-Id: Ida022588149a1618a63acc91e3800b09df873b6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/628697
TryBot-Bypass: Russ Cox <rsc@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2024-11-19 05:07:40 +00:00
Russ Cox
0b4cde34ac cmd/go/internal/fsys: move glob, walk code into new files
The Glob and Walk code does not depend on any of the fsys internals;
it simply uses ReadDir as an opaque abstraction.
Move it to separate files so that when working on the
actual overlay abstraction, it is out of sight, out of mind.

Change-Id: Ifa98feaaaafe5c1d8d8edce82de4fd0c78f599c4
Reviewed-on: https://go-review.googlesource.com/c/go/+/628696
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Sam Thanawalla <samthanawalla@google.com>
TryBot-Bypass: Russ Cox <rsc@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2024-11-19 05:07:37 +00:00
Ian Lance Taylor
b6aada5e81 doc/godebug: document that unrecognized GODEBUG entries are ignored
Change-Id: I6debc82913ae9fa02bee55dc87d35574f5265008
Reviewed-on: https://go-review.googlesource.com/c/go/+/629576
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-11-19 03:56:07 +00:00
Ian Lance Taylor
aae088b16a doc/godebug: fix tipo
Change-Id: Ib67d81d8e3aa33c6a506813194cc5710b3d5cdd0
Reviewed-on: https://go-review.googlesource.com/c/go/+/629575
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Auto-Submit: Ian Lance Taylor <iant@google.com>
2024-11-19 03:52:47 +00:00
Michael Anthony Knyszek
b09478b9c9 mime: disable TestLookupMallocs with ASAN
With the synchashtriemap GOEXPERIMENT, this allocates now, but just
under ASAN. That's OK, just skip the test under ASAN.

Fixes #70428.

Change-Id: I1ab4f7c06505c84309946358643715085f0db506
Reviewed-on: https://go-review.googlesource.com/c/go/+/629456
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-19 02:54:28 +00:00
Austin Clements
3e7cb78800 cmd/go: re-enable build JSON from go test -json, now with GODEBUG
This re-enables the behavior of CL 536399 (by effectively reverting CL
628955), so now go test -json again includes build output and failures
as JSON rather than text.

However, since this behavior is clearly enough to trip up some build
systems, this CL includes a GODEBUG=gotestjsonbuildtext that can be
set to 1 to revert to the old behavior.

Fixes #70402.
Updates #62067.

Cq-Include-Trybots: luci.golang.try:gotip-darwin-arm64_13,gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: I84e778cd844783dacfc83433e391b5ccb5925127
Reviewed-on: https://go-review.googlesource.com/c/go/+/629335
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Austin Clements <austin@google.com>
2024-11-19 02:39:44 +00:00
Guoqi Chen
5432cd96fd cmd/compiler,internal/runtime/atomic: optimize Cas{64,32} on loong64
In Loongson's new microstructure LA664 (Loongson-3A6000) and later, the atomic
compare-and-exchange instruction AMCAS[DB]{B,W,H,V} [1] is supported. Therefore,
the implementation of the atomic operation compare-and-swap can be selected according
to the CPUCFG flag LAMCAS: AMCASDB(full barrier) instruction is used on new
microstructures, and traditional LL-SC is used on LA464 (Loongson-3A5000) and older
microstructures. This can significantly improve the performance of Go programs on
new microstructures.

goos: linux
goarch: loong64
pkg: internal/runtime/atomic
cpu: Loongson-3A6000 @ 2500.00MHz
         |  bench.old   |  bench.new                           |
         |   sec/op     |   sec/op       vs base               |
Cas        46.84n ±  0%   22.82n ±  0%  -51.28% (p=0.000 n=20)
Cas-2      47.58n ±  0%   29.57n ±  0%  -37.85% (p=0.000 n=20)
Cas-4      43.27n ± 20%   25.31n ± 13%  -41.50% (p=0.000 n=20)
Cas64      46.85n ±  0%   22.82n ±  0%  -51.29% (p=0.000 n=20)
Cas64-2    47.43n ±  0%   29.53n ±  0%  -37.74% (p=0.002 n=20)
Cas64-4    43.18n ±  0%   25.28n ±  2%  -41.46% (p=0.000 n=20)
geomean    45.82n         25.74n        -43.82%

goos: linux
goarch: loong64
pkg: internal/runtime/atomic
cpu: Loongson-3A5000 @ 2500.00MHz
         |  bench.old  |  bench.new                         |
         |   sec/op    |   sec/op      vs base              |
Cas        50.05n ± 0%   51.26n ± 0%  +2.42% (p=0.000 n=20)
Cas-2      52.80n ± 0%   53.11n ± 0%  +0.59% (p=0.000 n=20)
Cas-4      55.97n ± 0%   57.31n ± 0%  +2.39% (p=0.000 n=20)
Cas64      50.05n ± 0%   51.26n ± 0%  +2.42% (p=0.000 n=20)
Cas64-2    52.68n ± 0%   53.11n ± 0%  +0.82% (p=0.000 n=20)
Cas64-4    55.96n ± 0%   57.26n ± 0%  +2.33% (p=0.000 n=20)
geomean    52.86n        53.83n       +1.82%

[1]: https://loongson.github.io/LoongArch-Documentation/LoongArch-ELF-ABI-EN.html

Change-Id: I9b777c63c124fb492f61c903f77061fa2b4e5322
Reviewed-on: https://go-review.googlesource.com/c/go/+/613396
Reviewed-by: Meidan Li <limeidan@loongson.cn>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Qiqi Huang <huangqiqi@loongson.cn>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-19 01:15:07 +00:00
Michael Anthony Knyszek
ec7824b6bb cmd/go: disable fuzz instrumentation for internal/godebug
This is thought to be the cause of certain recent longtest failures.
Let's try it out.

This appears to fix the longtests fuzz failures. I suspect that the
sync.Map in internal/godebug is at fault with the implementation
changing. I'm not sure yet exactly why this is a problem, maybe inlining
that didn't happen before? I don't know exactly when coverage
instrumentation happens in the compiler, but this is definitely the
problem.

For good measure, let's add internal/sync. If sync is on the list,
internal/sync should be, too.

Fixes #70429.
Fixes #70430.
Fixes #70431.

Change-Id: Ic9f49daa0956e3a50192bcc7778983682b5d12b8
Cq-Include-Trybots: luci.golang.try:gotip-linux-amd64-longtest
Reviewed-on: https://go-review.googlesource.com/c/go/+/629475
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-19 00:45:21 +00:00
Filippo Valsorda
79c0041daf crypto/internal/fips/aes/gcm: add GCMForTLS12 and GCMForTLS13
For #69536

Change-Id: I2d7b6e7b9932d0f0f582a5ab0bb871395dc2a1e8
Reviewed-on: https://go-review.googlesource.com/c/go/+/626675
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
2024-11-19 00:32:09 +00:00
Filippo Valsorda
b86f770bec crypto/internal/fips/aes/gcm: add SealWithRandomNonce
We don't expose it as an AEAD yet because the logic for that is complex
due to overlap issues. For #69981 we will make a cipher.AEAD wrapper
outside the FIPS module, but maybe a v2 interface will make it easier,
and then we'll be able to use this method more directly.

Updates #69981
For #69536

Change-Id: Id88191c01443b0dec89ff0d6c4a6289f519369d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/624916
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-11-19 00:32:04 +00:00
Filippo Valsorda
2b1a151524 crypto/internal/fips/aes/gcm: add CounterKDF
Change-Id: I236f9d8c4637793c1bda39f69573481356f8e362
Reviewed-on: https://go-review.googlesource.com/c/go/+/626436
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-11-19 00:31:19 +00:00
Filippo Valsorda
7cc488c8b5 crypto/internal/fips/aes/gcm: add CMAC
Change-Id: I5602dbf485c5c8a221e71c79961588e33f90452d
Reviewed-on: https://go-review.googlesource.com/c/go/+/626435
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2024-11-19 00:31:17 +00:00
Filippo Valsorda
7d7618971e crypto/aes,crypto/cipher: test all available implementations
TestEmptyPlaintext and TestCryptBlocks were folded into cryptotest.

Change-Id: I6131ab8582eb0e6d3a1b24bab1147a145d9766ac
Reviewed-on: https://go-review.googlesource.com/c/go/+/624738
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-19 00:31:13 +00:00
Filippo Valsorda
deee5322fc crypto/internal/fips/aes: move GCM to its own package
For #69536

Change-Id: Idcef2411c1fd4da302412609f28e438839e8aa4b
Reviewed-on: https://go-review.googlesource.com/c/go/+/624736
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
2024-11-19 00:31:11 +00:00
Filippo Valsorda
ecdc8c1b3f crypto/internal/cryptotest: add SkipTestAllocations
[                                                                     ]
[    It has been [ 0 ] days since Filippo broke a TestAllocations.    ]
[                                                                     ]

Concentrate all the skips in one place, so we don't have to re-discover
always the same ones via trial and error.

This might over-skip fixable allocations, but all these targets are not
fast anyway, so they are not worth going back for.

Removed the sysrand TestAllocations because it causes an import loop
with cryptotest and it's covered by TestAllocations in crypto/rand.

Change-Id: Icd40e97f9128e037f567147f8c9dafa758a47fac
Reviewed-on: https://go-review.googlesource.com/c/go/+/626438
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
2024-11-19 00:30:25 +00:00
Filippo Valsorda
cf531372b2 crypto/internal/fips/drbg: add CAST and service indicator
For #69536

Change-Id: If216a7537013bd9bbe90026ffe983a6bf50029c8
Reviewed-on: https://go-review.googlesource.com/c/go/+/625435
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-19 00:30:21 +00:00
Filippo Valsorda
0d8b96de05 crypto/internal/fips/drbg: implement FIPS-mode randomness generation
For #69536

Change-Id: I5bc83360fcd9114cc76314c3570977e9811fd1c6
Reviewed-on: https://go-review.googlesource.com/c/go/+/624978
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2024-11-19 00:30:18 +00:00
Filippo Valsorda
e67037fb27 crypto/internal/fips/drbg: implement CTR_DRBG
For #69536

Change-Id: I016bb723841acbda50f013db46f9d2dda200e1fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/624977
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
2024-11-19 00:30:16 +00:00
Filippo Valsorda
644628536f crypto/rand: move OS interaction to crypto/internal/sysrand
We're going to use that package as the passive entropy source for the
FIPS module, and we need to import it from a package that will be
imported by crypto/rand.

Since there is no overridable Reader now, introduced a mechanism to test
the otherwise impossible failure of the OS entropy source.

For #69536

Change-Id: I558687ed1ec896dba05b99b937970bb809de3fe7
Reviewed-on: https://go-review.googlesource.com/c/go/+/624976
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-11-19 00:29:28 +00:00
Filippo Valsorda
f705cf8f96 crypto/rand: replace crypto/aes with internal/chacha8rand for plan9
We will need to import this functionality from the FIPS module, and we
can't import AES from there. Plan 9 is not going to be FIPS validated
anyway, so we can use non-approved cryptography.

For #69536

Change-Id: I7921ec0829b576de2e80f3a7d0a9a776ff387684
Reviewed-on: https://go-review.googlesource.com/c/go/+/624975
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-11-19 00:29:26 +00:00
Filippo Valsorda
c4aea46784 crypto/aes: move to crypto/internal/fips/aes
The crypto/aes <-> crypto/cipher interfaces and the hardware support
upgrades were layered over the years, and had grown unwieldily.

Before: conditionally wrap the private crypto/aes type in private types
that implement an interface that's interface-upgraded by crypto/cipher
to replace the generic implementation in crypto/cipher.

crypto/aes depended on crypto/cipher, which is backwards.

After: provide concrete exported implementations of modes in
crypto/internal/fips/aes that crypto/cipher returns if the input Block
is the crypto/internal/fips/aes concrete implementation.

crypto/aes and crypto/cipher both depend on crypto/internal/fips/aes.

Also, made everything follow go.dev/wiki/TargetSpecific by only putting
the minimal code necessary and no exported functions in build-tagged
files.

The GCM integration still uses an interface upgrade, because the generic
implementation is complex enough that it was not trivial to duplicate.
This will be fixed in a future CL to make review easier.

For #69536

Change-Id: I21c2b93a498edb31c562b1aca824e21e8457fdff
Reviewed-on: https://go-review.googlesource.com/c/go/+/624395
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Roland Shoemaker <roland@golang.org>
Auto-Submit: Filippo Valsorda <filippo@golang.org>
2024-11-19 00:29:22 +00:00
Boris Nagaev
0240c91383 crypto/aes: speedup CTR mode on AMD64 and ARM64
The implementation runs up to 8 AES instructions in different registers
one after another in ASM code. Because CPU has instruction pipelining
and the instructions do not depend on each other, they can run in
parallel with this layout of code. This results in significant speedup
compared to the regular implementation in which blocks are processed in
the same registers so AES instructions do not run in parallel.

GCM mode already utilizes the approach.

The ASM implementation of ctrAble has most of its code in XORKeyStreamAt
method which has an additional argument, offset. It allows to use it
in a stateless way and to jump to any location in the stream. The method
does not exist in pure Go and boringcrypto implementations.

[ Mailed as CL 413594, then edited by filippo@ to manage the counter
with bits.Add64, remove bounds checks, make the assembly interface more
explicit, and to port the amd64 to Avo. Squeezed another -6.38% out. ]

goos: linux
goarch: amd64
pkg: crypto/cipher
cpu: AMD Ryzen 7 PRO 8700GE w/ Radeon 780M Graphics
            │  19df80d792  │             c8b0409d40              │
            │    sec/op    │   sec/op     vs base                │
AESCTR/50-8    64.68n ± 0%   26.89n ± 0%  -58.42% (p=0.000 n=10)
AESCTR/1K-8   1145.0n ± 0%   135.8n ± 0%  -88.14% (p=0.000 n=10)
AESCTR/8K-8   9145.0n ± 0%   917.5n ± 0%  -89.97% (p=0.000 n=10)
geomean        878.2n        149.6n       -82.96%

            │  19df80d792  │               c8b0409d40               │
            │     B/s      │      B/s       vs base                 │
AESCTR/50-8   737.2Mi ± 0%   1773.3Mi ± 0%  +140.54% (p=0.000 n=10)
AESCTR/1K-8   848.5Mi ± 0%   7156.6Mi ± 0%  +743.40% (p=0.000 n=10)
AESCTR/8K-8   853.8Mi ± 0%   8509.9Mi ± 0%  +896.70% (p=0.000 n=10)
geomean       811.4Mi         4.651Gi       +486.94%

Fixes #20967
Updates #39365
Updates #26673

Co-authored-by: Filippo Valsorda <filippo@golang.org>
Change-Id: Iaeea29fb93a56456f2e54507bc25196edb31b84b
Reviewed-on: https://go-review.googlesource.com/c/go/+/621958
Auto-Submit: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
Reviewed-by: Daniel McCarney <daniel@binaryparadox.net>
2024-11-19 00:24:58 +00:00
David Chase
170436c045 cmd/compile: strongly favor closure inlining
This tweaks the inlining cost knob for closures
specifically, they receive a doubled budget.  The
rationale for this is that closures have a lot of
"crud" in their IR that will disappear after inlining,
so the standard budget penalizes them unnecessarily.

This is also the cause of these bugs -- looking at the
code involved, these closures "should" be inlineable,
therefore tweak the parameters until behavior matches
expectations.  It's not costly in binary size, because
the only-called-from-one-site case is common (especially
for rangefunc iterators).

I can imagine better fixes and I am going to try to
get that done, but this one is small and makes things
better.

Fixes #69411, #69539.

Change-Id: I8a892c40323173a723799e0ddad69dcc2724a8f9
Reviewed-on: https://go-review.googlesource.com/c/go/+/629195
Reviewed-by: Keith Randall <khr@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Robert Griesemer <gri@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-19 00:04:51 +00:00
Damien Neil
b8fe88393b os: correctly handle errno==0 in (*Process).blockUntilWaitable
CL 627478 inadvertently returns a non-nil error
containing a syscall.Errno(0).

Change-Id: I1d6a9d0575d3ed651ddc02f30505437d0d266bb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/629515
Auto-Submit: Ian Lance Taylor <iant@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Ian Lance Taylor <iant@google.com>
Commit-Queue: Ian Lance Taylor <iant@google.com>
2024-11-18 23:26:47 +00:00
Michael Anthony Knyszek
a65f1a467f weak: move internal/weak to weak, and update according to proposal
The updates are:
- API documentation changes.
- Removal of the old package documentation discouraging linkname.
- Addition of new package documentation with some advice.
- Renaming of weak.Pointer.Strong -> weak.Pointer.Value.

Fixes #67552.

Change-Id: Ifad7e629b6d339dacaf2ca37b459d7f903e31bf8
Reviewed-on: https://go-review.googlesource.com/c/go/+/628455
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 22:29:23 +00:00
qmuntal
5e82cba9bd os/user: skip tests that create users when running on dev machines
Creating and deleting users is tricky to get right, and it's not
something we want to do on a developer machine. This change skips the
tests that create users when not running on a Go builder.

This will fix #70396, although I still don't understand why the test
user couldn't be recreated.

Fixes #70396

Change-Id: Ie7004dc209f94e72152c7d6bd8ec95cc12c79757
Reviewed-on: https://go-review.googlesource.com/c/go/+/627877
Reviewed-by: Ian Lance Taylor <iant@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@google.com>
2024-11-18 20:45:28 +00:00
Keith Randall
45869f5931 runtime: get rid of gc programs for types
Instead, have the runtime build the gc bitmaps on demand
at runtime.

Change-Id: If7a245bc62e4bce3ce80972410b0ed307d921abe
Reviewed-on: https://go-review.googlesource.com/c/go/+/616255
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@google.com>
2024-11-18 20:36:14 +00:00
khr@golang.org
7588cc9b00 cmd/link: use types (and not GC programs) to build data/bss ptrmasks
The linker knows the types of the global variables. We can use those
types to build the GC programs that describe the data and bss pointer masks.

That way we don't use the GC programs of the constituent types.
This is part of an effort to remove GC programs from the runtime.

There's a major complication in that when we're linking against a
shared library (typically, libstd.so), the relocations we need to
break apart arrays and structs into constituent types are difficult to
find. Load that additional data when linking against shared libraries.

Change-Id: I8516b24a0604479895c7b8a8a358d3cd8d421530
Reviewed-on: https://go-review.googlesource.com/c/go/+/546216
Reviewed-by: Keith Randall <khr@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
2024-11-18 20:36:04 +00:00
Michael Anthony Knyszek
aecb8faa91 sync: make HashTrieMap[any, any] the default implementation of Map
This change adds a GOEXPERIMENT, synchashtriemap, which replaces the
internals of a sync.Map with internal/sync.HashTrieMap[any, any]. The
main purpose behind this change is improved performance. Across almost
every benchmark, HashTrieMap[any, any] performs better than Map.

Also, relax TestMapClearNoAllocations to allow for one allocation.
Currently, the HashTrieMap allocates a new empty root node and stores
it: that's the whole clear operation. At the cost of some complexity, we
could allow Clear to have zero allocations by clearing the root node.
The complexity comes down to allowing threads to race to install a new
root node *or* creating a top-level mutex for installing a root node.

But I'm not sure this is worth it. Whether Clear or some other operation
takes the hit for allocating a single node almost certainly doesn't
matter. And Clear is still much, much faster in the new implementation
than the old, so I don't consider this a regression.

Change-Id: I939aa70a0edf2e850cedbea239aaf29a11a77b79
Reviewed-on: https://go-review.googlesource.com/c/go/+/608335
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
2024-11-18 20:35:42 +00:00
Michael Anthony Knyszek
06e6b8efa4 internal/sync: optimize CompareAndSwap and Swap
We observe the CompareAndSwap and Swap can both be substantially faster
if the value in each entry node is mutable. This change modifies the
map entry node to store the value indirectly, allowing us to perform
swaps for existing nodes and compare-and-swaps without taking the
parent node's lock.

Change-Id: I371343aa81a843d3a7e6bc5ac87b8a96c12ca3a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/606462
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-18 20:35:39 +00:00
Michael Anthony Knyszek
50f1888814 sync: add HashTrieMap to Map tests and benchmarks
Also, rename Map benchmarks to make them easier to single out via
regexp.

Change-Id: I4dcb066745aba1c340f56050d08539ae2976274d
Reviewed-on: https://go-review.googlesource.com/c/go/+/606461
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 20:35:37 +00:00
Michael Anthony Knyszek
5c7b7c7d60 internal/sync: add Range to HashTrieMap
This implementation is trivial given that All already exists. The
purpose of Range is to conform to sync.Map's API.

Change-Id: Icfe755b9986c46b88c7201644e562b1631a02b66
Reviewed-on: https://go-review.googlesource.com/c/go/+/606460
Reviewed-by: David Chase <drchase@google.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-18 20:35:34 +00:00
Michael Anthony Knyszek
e5c4c79cc4 internal/sync: add Clear to HashTrieMap
This change adds the Clear operation to HashTrieMap to align it with
sync.Map.

Change-Id: I46069b018725d7fe3a07d667876235534bca86f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/606459
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 20:35:30 +00:00
Michael Anthony Knyszek
083045f99d internal/sync: add Delete to HashTrieMap
This change adds the Delete operation (with the same semantics as
sync.Map's Delete) to HashTrieMap.

Change-Id: If90376ff735256444538719d670ed07e9e42870b
Reviewed-on: https://go-review.googlesource.com/c/go/+/606458
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 20:35:28 +00:00
Michael Anthony Knyszek
107e094f51 internal/sync: refactor HashTrieMap tests into per-op suites
This change does a minor refactor of the HashTrieMap tests to be better
organized into suites for each operation. It might be worthwhile to
deduplicate some code in here, but it's also helpful to see exactly
what's happening when something goes wrong.

Change-Id: I138515ee6de5aec5f3d38afe8a3bf3972afb2800
Reviewed-on: https://go-review.googlesource.com/c/go/+/606457
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-18 20:35:25 +00:00
Michael Anthony Knyszek
700c7b95ae internal/sync: add LoadAndDelete to HashTrieMap
This change adds the LoadAndDelete operation (with the same semantics as
sync.Map's LoadAndDelete) to HashTrieMap.

Change-Id: Id6777dffcd3ebc98490aa51f0e85e59a56f63074
Reviewed-on: https://go-review.googlesource.com/c/go/+/606456
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2024-11-18 20:35:22 +00:00
Michael Anthony Knyszek
872031dc10 internal/sync: add Store to HashTrieMap
This change adds the Store operation (with the same semantics as
sync.Map's Store) to HashTrieMap.

Change-Id: Ida55421f28d8cc6527f983f7f1c2fd55bb280925
Reviewed-on: https://go-review.googlesource.com/c/go/+/606455
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 20:35:19 +00:00
Michael Anthony Knyszek
28bac5640c internal/sync: add Swap to HashTrieMap
This change adds the Swap operation (with the same semantics as
sync.Map's Swap) to HashTrieMap.

Change-Id: I8697a0c8c2eb761e2452a41b868b590ccbfa5c03
Reviewed-on: https://go-review.googlesource.com/c/go/+/594064
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 20:35:14 +00:00
Michael Anthony Knyszek
85fa418fd5 internal/sync: add CompareAndSwap to HashTrieMap
This change adds the CompareAndSwap operation (with the same semantics
as sync.Map's CompareAndSwap) to HashTrieMap.

Change-Id: I86153799fc47304784333d17f0c6a7ad7682f04a
Reviewed-on: https://go-review.googlesource.com/c/go/+/594063
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 20:35:11 +00:00
Michael Anthony Knyszek
e8c5e6d635 internal/sync: make the HashTrieMap zero value ready to use
This improves ergonomics a little and aligns the HashTrieMap with
sync.Map.

Change-Id: Idb2b981a4f59a35f8670c6b5038e2bd207484483
Reviewed-on: https://go-review.googlesource.com/c/go/+/594062
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
2024-11-18 20:30:24 +00:00
Michael Anthony Knyszek
008904aba6 internal/sync: relax value type constraint for HashTrieMap
Currently the HashTrieMap requires both keys and values to be
comparable, but it's actually OK if the value is not comparable. Some
operations may fail, but others will not, and we can check comparability
dynamically on map initialization. This makes the implementation
substantially more flexible.

Change-Id: Idc9c30dfa273d80ae4d46a9eefb5c155294408aa
Reviewed-on: https://go-review.googlesource.com/c/go/+/594061
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
2024-11-18 20:30:22 +00:00
Michael Anthony Knyszek
36b5b053be internal/sync: use normal comparison for keys in HashTrieMap
There's are unnecessary calls to the key's equal function -- we can just
leverage the language here. Leave the values alone for now, we want to
relax that constraint.

Change-Id: Iccfaef030a2a29b6a24a7da41e5e816b70091c7c
Reviewed-on: https://go-review.googlesource.com/c/go/+/594060
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
2024-11-18 20:30:18 +00:00
Michael Anthony Knyszek
e51a33a0ef internal/sync: factor out lookup for CompareAndDelete in HashTrieMap
This lookup will be reused for other operations, like CompareAndSwap.

Change-Id: I6698e3c99f7ef6d2b82b9ef489ba8a0be2a71d61
Reviewed-on: https://go-review.googlesource.com/c/go/+/594059
Auto-Submit: Michael Knyszek <mknyszek@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Reviewed-by: David Chase <drchase@google.com>
2024-11-18 20:29:12 +00:00
Michael Anthony Knyszek
4804d0daca internal/sync: move HashTrieMap from internal/concurrent
This change moves internal/concurrent.HashTrieMap from
internal/concurrent into internal/sync just to clean up the packages a
bit. This is all in anticipation of using HashTrieMap from the sync
package.

Change-Id: I18c007a301f83979d72f5d6bea600c42eaf2421e
Reviewed-on: https://go-review.googlesource.com/c/go/+/594058
Auto-Submit: Michael Knyszek <mknyszek@google.com>
Reviewed-by: David Chase <drchase@google.com>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
2024-11-18 20:29:08 +00:00