qbit/go
1
0
Fork 0
mirror of https://github.com/golang/go synced 2024-11-08 07:26:11 -07:00
Code Releases Activity
ec989337c5
go/src/runtime/hash64.go

113 lines
2.7 KiB
Go
Raw Normal View History

runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
// Hashing algorithm inspired by
// xxhash: https://code.google.com/p/xxhash/
// cityhash: https://code.google.com/p/cityhash/
runtime: add js/wasm architecture This commit adds the js/wasm architecture to the runtime package. Currently WebAssembly has no support for threads yet, see https://github.com/WebAssembly/design/issues/1073. Because of that, there is no preemption of goroutines and no sysmon goroutine. Design doc: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0_Nv3OUwjEY5qVCxCup4 About WebAssembly assembly files: https://docs.google.com/document/d/1GRmy3rA4DiYtBlX-I1Jr_iHykbX8EixC3Mq0TCYqbKc Updates #18892 Change-Id: I7f12d21b5180500d55ae9fd2f7e926a1731db391 Reviewed-on: https://go-review.googlesource.com/103877 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Austin Clements <austin@google.com>
2018-03-31 15:14:17 -06:00
// +build amd64 amd64p32 arm64 mips64 mips64le ppc64 ppc64le s390x wasm
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
package runtime
import "unsafe"
const (
// Constants for multiplication: four random odd 64-bit numbers.
m1 = 16877499708836156737
m2 = 2820277070424839065
m3 = 9497967016996688599
m4 = 15839092249703872147
)
runtime: remove size argument from hash and equal algorithms The equal algorithm used to take the size equal(p, q *T, size uintptr) bool With this change, it does not equal(p, q *T) bool Similarly for the hash algorithm. The size is rarely used, as most equal functions know the size of the thing they are comparing. For instance f32equal already knows its inputs are 4 bytes in size. For cases where the size is not known, we allocate a closure (one for each size needed) that points to an assembly stub that reads the size out of the closure and calls generic code that has a size argument. Reduces the size of the go binary by 0.07%. Performance impact is not measurable. Change-Id: I6e00adf3dde7ad2974adbcff0ee91e86d2194fec Reviewed-on: https://go-review.googlesource.com/2392 Reviewed-by: Russ Cox <rsc@golang.org>
2015-01-06 17:42:48 -07:00
func memhash(p unsafe.Pointer, seed, s uintptr) uintptr {
runtime: implement aeshash for arm64 platform Fix #10109 name old time/op new time/op delta Hash5 72.3ns ± 0% 51.5ns ± 0% -28.71% (p=0.000 n=4+5) Hash16 78.8ns ± 0% 48.7ns ± 0% ~ (p=0.079 n=4+5) Hash64 196ns ±25% 73ns ±16% -62.68% (p=0.008 n=5+5) Hash1024 1.57µs ± 0% 0.27µs ± 1% -82.90% (p=0.000 n=5+4) Hash65536 96.5µs ± 0% 14.3µs ± 0% -85.14% (p=0.016 n=5+4) HashStringSpeed 156ns ± 6% 129ns ± 3% -17.56% (p=0.008 n=5+5) HashBytesSpeed 227ns ± 1% 200ns ± 1% -11.98% (p=0.008 n=5+5) HashInt32Speed 116ns ± 2% 102ns ± 0% -11.92% (p=0.016 n=5+4) HashInt64Speed 120ns ± 3% 101ns ± 2% -15.55% (p=0.008 n=5+5) HashStringArraySpeed 342ns ± 0% 306ns ± 2% -10.58% (p=0.008 n=5+5) FastrandHashiter 217ns ± 1% 217ns ± 1% ~ (p=1.000 n=5+5) name old speed new speed delta Hash5 69.1MB/s ± 0% 97.0MB/s ± 0% +40.32% (p=0.008 n=5+5) Hash16 203MB/s ± 0% 329MB/s ± 0% +61.76% (p=0.016 n=4+5) Hash64 332MB/s ±21% 881MB/s ±14% +165.66% (p=0.008 n=5+5) Hash1024 651MB/s ± 0% 3652MB/s ±17% +460.73% (p=0.008 n=5+5) Hash65536 679MB/s ± 0% 4570MB/s ± 0% +572.85% (p=0.016 n=5+4) Change-Id: I573028979f84cf2e0e087951271d5de8865dbf04 Reviewed-on: https://go-review.googlesource.com/89755 Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2018-04-03 23:15:22 -06:00
if (GOARCH == "amd64" || GOARCH == "arm64") &&
GOOS != "nacl" && useAeshash {
runtime: remove size argument from hash and equal algorithms The equal algorithm used to take the size equal(p, q *T, size uintptr) bool With this change, it does not equal(p, q *T) bool Similarly for the hash algorithm. The size is rarely used, as most equal functions know the size of the thing they are comparing. For instance f32equal already knows its inputs are 4 bytes in size. For cases where the size is not known, we allocate a closure (one for each size needed) that points to an assembly stub that reads the size out of the closure and calls generic code that has a size argument. Reduces the size of the go binary by 0.07%. Performance impact is not measurable. Change-Id: I6e00adf3dde7ad2974adbcff0ee91e86d2194fec Reviewed-on: https://go-review.googlesource.com/2392 Reviewed-by: Russ Cox <rsc@golang.org>
2015-01-06 17:42:48 -07:00
return aeshash(p, seed, s)
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
}
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
h := uint64(seed + s*hashkey[0])
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
tail:
switch {
case s == 0:
case s < 4:
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
h ^= uint64(*(*byte)(p))
h ^= uint64(*(*byte)(add(p, s>>1))) << 8
h ^= uint64(*(*byte)(add(p, s-1))) << 16
h = rotl_31(h*m1) * m2
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
case s <= 8:
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
h ^= uint64(readUnaligned32(p))
h ^= uint64(readUnaligned32(add(p, s-4))) << 32
h = rotl_31(h*m1) * m2
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
case s <= 16:
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
h ^= readUnaligned64(p)
h = rotl_31(h*m1) * m2
h ^= readUnaligned64(add(p, s-8))
h = rotl_31(h*m1) * m2
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
case s <= 32:
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
h ^= readUnaligned64(p)
h = rotl_31(h*m1) * m2
h ^= readUnaligned64(add(p, 8))
h = rotl_31(h*m1) * m2
h ^= readUnaligned64(add(p, s-16))
h = rotl_31(h*m1) * m2
h ^= readUnaligned64(add(p, s-8))
h = rotl_31(h*m1) * m2
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
default:
v1 := h
runtime: make aeshash more DOS-proof Improve the aeshash implementation to make it harder to engineer collisions. 1) Scramble the seed before xoring with the input string. This makes it harder to cancel known portions of the seed (like the size) because it mixes the per-table seed into those other parts. 2) Use table-dependent seeds for all stripes when hashing >16 byte strings. For small strings this change uses 4 aesenc ops instead of 3, so it is somewhat slower. The first two can run in parallel, though, so it isn't 33% slower. benchmark old ns/op new ns/op delta BenchmarkHash64-12 10.2 11.2 +9.80% BenchmarkHash16-12 5.71 6.13 +7.36% BenchmarkHash5-12 6.64 7.01 +5.57% BenchmarkHashBytesSpeed-12 30.3 31.9 +5.28% BenchmarkHash65536-12 2785 2882 +3.48% BenchmarkHash1024-12 53.6 55.4 +3.36% BenchmarkHashStringArraySpeed-12 54.9 56.5 +2.91% BenchmarkHashStringSpeed-12 18.7 19.2 +2.67% BenchmarkHashInt32Speed-12 14.8 15.1 +2.03% BenchmarkHashInt64Speed-12 14.5 14.5 +0.00% Change-Id: I59ea124b5cb92b1c7e8584008257347f9049996c Reviewed-on: https://go-review.googlesource.com/14124 Reviewed-by: jcd . <jcd@golang.org> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-08-31 17:26:12 -06:00
v2 := uint64(seed * hashkey[1])
v3 := uint64(seed * hashkey[2])
v4 := uint64(seed * hashkey[3])
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
for s >= 32 {
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
v1 ^= readUnaligned64(p)
v1 = rotl_31(v1*m1) * m2
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
p = add(p, 8)
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
v2 ^= readUnaligned64(p)
v2 = rotl_31(v2*m2) * m3
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
p = add(p, 8)
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
v3 ^= readUnaligned64(p)
v3 = rotl_31(v3*m3) * m4
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
p = add(p, 8)
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
v4 ^= readUnaligned64(p)
v4 = rotl_31(v4*m4) * m1
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
p = add(p, 8)
s -= 32
}
runtime: use some startup randomness in the fallback hashes Fold in some startup randomness to make the hash vary across different runs. This helps prevent attackers from choosing keys that all map to the same bucket. Also, reorganize the hash a bit. Move the *m1 multiply to after the xor of the current hash and the message. For hash quality it doesn't really matter, but for DDOS resistance it helps a lot (any processing done to the message before it is merged with the random seed is useless, as it is easily inverted by an attacker). Update #9365 Change-Id: Ib19968168e1bbc541d1d28be2701bb83e53f1e24 Reviewed-on: https://go-review.googlesource.com/2344 Reviewed-by: Ian Lance Taylor <iant@golang.org>
2015-01-06 10:06:44 -07:00
h = v1 ^ v2 ^ v3 ^ v4
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
goto tail
}
h ^= h >> 29
h *= m3
h ^= h >> 32
return uintptr(h)
}
runtime: specialize memhash32 and memhash64 AMD64 with AES support disabled: name old time/op new time/op delta MapPopulate/1 78.0ns ± 1% 75.5ns ± 1% -3.17% (p=0.000 n=10+9) MapPopulate/10 764ns ± 2% 673ns ± 2% -11.91% (p=0.000 n=10+10) MapPopulate/100 9.52µs ± 1% 8.54µs ± 1% -10.37% (p=0.000 n=8+10) MapPopulate/1000 116µs ± 2% 103µs ± 1% -10.40% (p=0.000 n=10+8) MapPopulate/10000 1.01ms ± 1% 0.90ms ± 1% -10.70% (p=0.000 n=10+10) MapPopulate/100000 9.81ms ± 1% 8.67ms ± 2% -11.54% (p=0.000 n=10+10) 386 with AES support disabled: name old time/op new time/op delta MapPopulate/1 95.3ns ± 1% 90.6ns ± 1% -4.95% (p=0.000 n=10+9) MapPopulate/10 983ns ± 2% 912ns ± 1% -7.18% (p=0.000 n=10+10) MapPopulate/100 11.9µs ± 2% 11.2µs ± 1% -6.01% (p=0.000 n=10+10) MapPopulate/1000 140µs ± 1% 131µs ± 1% -6.19% (p=0.000 n=10+10) MapPopulate/10000 1.26ms ± 2% 1.18ms ± 1% -5.93% (p=0.000 n=9+10) MapPopulate/100000 12.1ms ± 2% 11.4ms ± 1% -5.48% (p=0.000 n=10+10) Fixes #21539 Change-Id: Ice128c947c9a6a294800d6a5250d82045eb70b55 Reviewed-on: https://go-review.googlesource.com/59352 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-08-27 06:05:11 -06:00
func memhash32(p unsafe.Pointer, seed uintptr) uintptr {
h := uint64(seed + 4*hashkey[0])
v := uint64(readUnaligned32(p))
h ^= v
h ^= v << 32
h = rotl_31(h*m1) * m2
h ^= h >> 29
h *= m3
h ^= h >> 32
return uintptr(h)
}
func memhash64(p unsafe.Pointer, seed uintptr) uintptr {
h := uint64(seed + 8*hashkey[0])
runtime: fix memhash64 on big endian systems Fixes #21677. Change-Id: I869dee5f43df5d87d86922681726297e3024c562 Reviewed-on: https://go-review.googlesource.com/59810 Run-TryBot: Michael Munday <mike.munday@ibm.com> Reviewed-by: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-08-28 22:10:17 -06:00
h ^= uint64(readUnaligned32(p)) | uint64(readUnaligned32(add(p, 4)))<<32
runtime: specialize memhash32 and memhash64 AMD64 with AES support disabled: name old time/op new time/op delta MapPopulate/1 78.0ns ± 1% 75.5ns ± 1% -3.17% (p=0.000 n=10+9) MapPopulate/10 764ns ± 2% 673ns ± 2% -11.91% (p=0.000 n=10+10) MapPopulate/100 9.52µs ± 1% 8.54µs ± 1% -10.37% (p=0.000 n=8+10) MapPopulate/1000 116µs ± 2% 103µs ± 1% -10.40% (p=0.000 n=10+8) MapPopulate/10000 1.01ms ± 1% 0.90ms ± 1% -10.70% (p=0.000 n=10+10) MapPopulate/100000 9.81ms ± 1% 8.67ms ± 2% -11.54% (p=0.000 n=10+10) 386 with AES support disabled: name old time/op new time/op delta MapPopulate/1 95.3ns ± 1% 90.6ns ± 1% -4.95% (p=0.000 n=10+9) MapPopulate/10 983ns ± 2% 912ns ± 1% -7.18% (p=0.000 n=10+10) MapPopulate/100 11.9µs ± 2% 11.2µs ± 1% -6.01% (p=0.000 n=10+10) MapPopulate/1000 140µs ± 1% 131µs ± 1% -6.19% (p=0.000 n=10+10) MapPopulate/10000 1.26ms ± 2% 1.18ms ± 1% -5.93% (p=0.000 n=9+10) MapPopulate/100000 12.1ms ± 2% 11.4ms ± 1% -5.48% (p=0.000 n=10+10) Fixes #21539 Change-Id: Ice128c947c9a6a294800d6a5250d82045eb70b55 Reviewed-on: https://go-review.googlesource.com/59352 Run-TryBot: Martin Möhrmann <moehrmann@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Keith Randall <khr@golang.org>
2017-08-27 06:05:11 -06:00
h = rotl_31(h*m1) * m2
h ^= h >> 29
h *= m3
h ^= h >> 32
return uintptr(h)
}
runtime: a better fallback hash For arm and powerpc, as well as x86 without aes instructions. Contains a mixture of ideas from cityhash and xxhash. Compared to our old fallback on ARM, it's ~no slower on small objects and up to ~50% faster on large objects. More importantly, it is a much better hash function and thus has less chance of bad behavior. Fixes #8737 benchmark old ns/op new ns/op delta BenchmarkHash5 173 181 +4.62% BenchmarkHash16 252 212 -15.87% BenchmarkHash64 575 419 -27.13% BenchmarkHash1024 7173 3995 -44.31% BenchmarkHash65536 516940 313173 -39.42% BenchmarkHashStringSpeed 300 279 -7.00% BenchmarkHashBytesSpeed 478 424 -11.30% BenchmarkHashInt32Speed 217 207 -4.61% BenchmarkHashInt64Speed 262 231 -11.83% BenchmarkHashStringArraySpeed 609 631 +3.61% Change-Id: I0a9335028f32b10ad484966e3019987973afd3eb Reviewed-on: https://go-review.googlesource.com/1360 Reviewed-by: Russ Cox <rsc@golang.org>
2014-12-10 18:02:58 -07:00
// Note: in order to get the compiler to issue rotl instructions, we
// need to constant fold the shift amount by hand.
// TODO: convince the compiler to issue rotl instructions after inlining.
func rotl_31(x uint64) uint64 {
return (x << 31) | (x >> (64 - 31))
}
Copy Permalink