qbit/go
1
0
Fork 0
mirror of https://github.com/golang/go synced 2024-11-19 14:54:43 -07:00
Code Releases Activity
4feb47bc76
go/src/runtime/memclr_amd64.s

178 lines
3.6 KiB
ArmAsm
Raw Normal View History

runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
// Copyright 2014 The Go Authors. All rights reserved.
// Use of this source code is governed by a BSD-style
// license that can be found in the LICENSE file.
runtime: use unoptimized memmove and memclr on Plan 9 On Plan 9, the kernel disallows the use of floating point instructions while handling a note. Previously, we worked around this by using a simple loop in place of memmove. When I added that work-around, I verified that all paths from the note handler didn't end up calling memmove. Now that memclr is using SSE instructions, the same process will have to be done again. Instead of doing that, however, this CL just punts and uses unoptimized functions everywhere on Plan 9. LGTM=rsc R=rsc, 0intro CC=golang-codereviews https://golang.org/cl/73830044
2014-03-12 19:12:25 -06:00
// +build !plan9
runtime: use new #include "textflag.h" I did this just to clean things up, but it will be important when we drop the pkg directory later. LGTM=bradfitz R=r, bradfitz CC=golang-codereviews https://golang.org/cl/132600043
2014-09-04 21:05:18 -06:00
#include "textflag.h"
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
runtime: fix windows/386 build The difference between the old and the new (from earlier) code is that we set stackguard = stack.lo + StackGuard, while the old code set stackguard = stack.lo. That 512 bytes appears to be the difference between the profileloop function running and not running. We don't know how big the system stack is, but it is likely MUCH bigger than 4k. Give Go/C 8k. TBR=iant CC=golang-codereviews https://golang.org/cl/140440044
2014-09-09 15:12:05 -06:00
// NOTE: Windows externalthreadhandler expects memclr to preserve DX.
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
// void runtime·memclr(void*, uintptr)
TEXT runtime·memclr(SB), NOSPLIT, $0-16
MOVQ ptr+0(FP), DI
MOVQ n+8(FP), BX
XORQ AX, AX
// MOVOU seems always faster than REP STOSQ.
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
tail:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
TESTQ BX, BX
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JEQ _0
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
CMPQ BX, $2
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JBE _1or2
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
CMPQ BX, $4
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JBE _3or4
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
CMPQ BX, $8
runtime: memmove/memclr pointers atomically Make sure that we're moving or zeroing pointers atomically. Anything that is a multiple of pointer size and at least pointer aligned might have pointers in it. All the code looks ok except for the 1-pointer-sized moves. Fixes #13160 Update #12552 Change-Id: Ib97d9b918fa9f4cc5c56c67ed90255b7fdfb7b45 Reviewed-on: https://go-review.googlesource.com/16668 Reviewed-by: Dmitry Vyukov <dvyukov@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-11-05 13:39:56 -07:00
JB _5through7
JE _8
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
CMPQ BX, $16
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JBE _9through16
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
PXOR X0, X0
CMPQ BX, $32
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JBE _17through32
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
CMPQ BX, $64
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JBE _33through64
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
CMPQ BX, $128
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JBE _65through128
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
CMPQ BX, $256
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JBE _129through256
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
CMPB runtime·support_avx2(SB), $1
JE loop_preheader_avx2
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
// TODO: use branch table and BSR to make this just a single dispatch
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
// TODO: for really big clears, use MOVNTDQ, even without AVX2.
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
loop:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVOU X0, 0(DI)
MOVOU X0, 16(DI)
MOVOU X0, 32(DI)
MOVOU X0, 48(DI)
MOVOU X0, 64(DI)
MOVOU X0, 80(DI)
MOVOU X0, 96(DI)
MOVOU X0, 112(DI)
MOVOU X0, 128(DI)
MOVOU X0, 144(DI)
MOVOU X0, 160(DI)
MOVOU X0, 176(DI)
MOVOU X0, 192(DI)
MOVOU X0, 208(DI)
MOVOU X0, 224(DI)
MOVOU X0, 240(DI)
SUBQ $256, BX
ADDQ $256, DI
CMPQ BX, $256
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
JAE loop
JMP tail
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
loop_preheader_avx2:
cmd/asm: correct, complete newly added AVX instructions Use the standard names, for discoverability. Use the standard register arguments, for correctness. Implement all possible arguments, for completeness. Enable the corresponding tests now that everything is standard. Update the uses in package runtime. Fixes #14068. Change-Id: I8e1af9a41e7d02d98c2a82af3d4cdb3e9204824f Reviewed-on: https://go-review.googlesource.com/18852 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2016-01-22 20:25:15 -07:00
VPXOR Y0, Y0, Y0
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
// For smaller sizes MOVNTDQ may be faster or slower depending on hardware.
// For larger sizes it is always faster, even on dual Xeons with 30M cache.
// TODO take into account actual LLC size. E. g. glibc uses LLC size/2.
CMPQ BX, $0x2000000
JAE loop_preheader_avx2_huge
loop_avx2:
cmd/asm: correct, complete newly added AVX instructions Use the standard names, for discoverability. Use the standard register arguments, for correctness. Implement all possible arguments, for completeness. Enable the corresponding tests now that everything is standard. Update the uses in package runtime. Fixes #14068. Change-Id: I8e1af9a41e7d02d98c2a82af3d4cdb3e9204824f Reviewed-on: https://go-review.googlesource.com/18852 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2016-01-22 20:25:15 -07:00
VMOVDQU Y0, 0(DI)
VMOVDQU Y0, 32(DI)
VMOVDQU Y0, 64(DI)
VMOVDQU Y0, 96(DI)
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
SUBQ $128, BX
ADDQ $128, DI
CMPQ BX, $128
JAE loop_avx2
cmd/asm: correct, complete newly added AVX instructions Use the standard names, for discoverability. Use the standard register arguments, for correctness. Implement all possible arguments, for completeness. Enable the corresponding tests now that everything is standard. Update the uses in package runtime. Fixes #14068. Change-Id: I8e1af9a41e7d02d98c2a82af3d4cdb3e9204824f Reviewed-on: https://go-review.googlesource.com/18852 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2016-01-22 20:25:15 -07:00
VMOVDQU Y0, -32(DI)(BX*1)
VMOVDQU Y0, -64(DI)(BX*1)
VMOVDQU Y0, -96(DI)(BX*1)
VMOVDQU Y0, -128(DI)(BX*1)
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
VZEROUPPER
RET
loop_preheader_avx2_huge:
// Align to 32 byte boundary
cmd/asm: correct, complete newly added AVX instructions Use the standard names, for discoverability. Use the standard register arguments, for correctness. Implement all possible arguments, for completeness. Enable the corresponding tests now that everything is standard. Update the uses in package runtime. Fixes #14068. Change-Id: I8e1af9a41e7d02d98c2a82af3d4cdb3e9204824f Reviewed-on: https://go-review.googlesource.com/18852 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2016-01-22 20:25:15 -07:00
VMOVDQU Y0, 0(DI)
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
MOVQ DI, SI
ADDQ $32, DI
ANDQ $~31, DI
SUBQ DI, SI
ADDQ SI, BX
loop_avx2_huge:
cmd/asm: correct, complete newly added AVX instructions Use the standard names, for discoverability. Use the standard register arguments, for correctness. Implement all possible arguments, for completeness. Enable the corresponding tests now that everything is standard. Update the uses in package runtime. Fixes #14068. Change-Id: I8e1af9a41e7d02d98c2a82af3d4cdb3e9204824f Reviewed-on: https://go-review.googlesource.com/18852 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2016-01-22 20:25:15 -07:00
VMOVNTDQ Y0, 0(DI)
VMOVNTDQ Y0, 32(DI)
VMOVNTDQ Y0, 64(DI)
VMOVNTDQ Y0, 96(DI)
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
SUBQ $128, BX
ADDQ $128, DI
CMPQ BX, $128
JAE loop_avx2_huge
// In the desciption of MOVNTDQ in [1]
// "... fencing operation implemented with the SFENCE or MFENCE instruction
// should be used in conjunction with MOVNTDQ instructions..."
// [1] 64-ia-32-architectures-software-developer-manual-325462.pdf
SFENCE
cmd/asm: correct, complete newly added AVX instructions Use the standard names, for discoverability. Use the standard register arguments, for correctness. Implement all possible arguments, for completeness. Enable the corresponding tests now that everything is standard. Update the uses in package runtime. Fixes #14068. Change-Id: I8e1af9a41e7d02d98c2a82af3d4cdb3e9204824f Reviewed-on: https://go-review.googlesource.com/18852 Run-TryBot: Russ Cox <rsc@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rob Pike <r@golang.org>
2016-01-22 20:25:15 -07:00
VMOVDQU Y0, -32(DI)(BX*1)
VMOVDQU Y0, -64(DI)(BX*1)
VMOVDQU Y0, -96(DI)(BX*1)
VMOVDQU Y0, -128(DI)(BX*1)
runtime: speed up memclr with avx2 on amd64 Results are a bit noisy, but show good improvement (haswell) name old time/op new time/op delta Memclr5-48 6.06ns ± 8% 5.65ns ± 8% -6.81% (p=0.000 n=20+20) Memclr16-48 5.75ns ± 6% 5.71ns ± 6% ~ (p=0.545 n=20+19) Memclr64-48 6.54ns ± 5% 6.14ns ± 9% -6.12% (p=0.000 n=18+20) Memclr256-48 10.1ns ±12% 9.9ns ±14% ~ (p=0.285 n=20+19) Memclr4096-48 104ns ± 8% 57ns ±15% -44.98% (p=0.000 n=20+20) Memclr65536-48 2.45µs ± 5% 2.43µs ± 8% ~ (p=0.665 n=16+20) Memclr1M-48 58.7µs ±13% 56.4µs ±11% -3.92% (p=0.033 n=20+19) Memclr4M-48 233µs ± 9% 234µs ± 9% ~ (p=0.728 n=20+19) Memclr8M-48 469µs ±11% 472µs ±16% ~ (p=0.947 n=20+20) Memclr16M-48 947µs ±10% 916µs ±10% ~ (p=0.050 n=20+19) Memclr64M-48 10.9ms ±10% 4.5ms ± 9% -58.43% (p=0.000 n=20+20) GoMemclr5-48 3.80ns ±13% 3.38ns ± 6% -11.02% (p=0.000 n=20+20) GoMemclr16-48 3.34ns ±15% 3.40ns ± 9% ~ (p=0.351 n=20+20) GoMemclr64-48 4.10ns ±15% 4.04ns ±10% ~ (p=1.000 n=20+19) GoMemclr256-48 7.75ns ±20% 7.88ns ± 9% ~ (p=0.227 n=20+19) name old speed new speed delta Memclr5-48 826MB/s ± 7% 886MB/s ± 8% +7.32% (p=0.000 n=20+20) Memclr16-48 2.78GB/s ± 5% 2.81GB/s ± 6% ~ (p=0.550 n=20+19) Memclr64-48 9.79GB/s ± 5% 10.44GB/s ±10% +6.64% (p=0.000 n=18+20) Memclr256-48 25.4GB/s ±14% 25.6GB/s ±12% ~ (p=0.647 n=20+19) Memclr4096-48 39.4GB/s ± 8% 72.0GB/s ±13% +82.81% (p=0.000 n=20+20) Memclr65536-48 26.6GB/s ± 6% 27.0GB/s ± 9% ~ (p=0.517 n=17+20) Memclr1M-48 17.9GB/s ±12% 18.5GB/s ±11% ~ (p=0.068 n=20+20) Memclr4M-48 18.0GB/s ± 9% 17.8GB/s ±14% ~ (p=0.547 n=20+20) Memclr8M-48 17.9GB/s ±10% 17.8GB/s ±14% ~ (p=0.947 n=20+20) Memclr16M-48 17.8GB/s ± 9% 18.4GB/s ± 9% ~ (p=0.050 n=20+19) Memclr64M-48 6.19GB/s ±10% 14.87GB/s ± 9% +140.11% (p=0.000 n=20+20) GoMemclr5-48 1.31GB/s ±10% 1.48GB/s ± 6% +13.06% (p=0.000 n=19+20) GoMemclr16-48 4.81GB/s ±14% 4.71GB/s ± 8% ~ (p=0.341 n=20+20) GoMemclr64-48 15.7GB/s ±13% 15.8GB/s ±11% ~ (p=0.967 n=20+19) Change-Id: I393f3f20e2f31538d1b1dd70d6e5c201c106a095 Reviewed-on: https://go-review.googlesource.com/16773 Run-TryBot: Ilya Tocar <ilya.tocar@intel.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Klaus Post <klauspost@gmail.com> Reviewed-by: Keith Randall <khr@golang.org>
2015-10-30 08:54:39 -06:00
VZEROUPPER
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_1or2:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVB AX, (DI)
MOVB AX, -1(DI)(BX*1)
runtime: prevent pointless jmp in amd64 and 386 memmove 6a and 8a rearrange memmove such that the fallthrough from move_1or2 to move_0 ends up being a JMP to a RET. Insert an explicit RET to prevent such silliness. Do the same for memclr as prophylaxis. benchmark old ns/op new ns/op delta BenchmarkMemmove1 4.59 4.13 -10.02% BenchmarkMemmove2 4.58 4.13 -9.83% LGTM=khr R=golang-codereviews, dvyukov, minux, ruiu, bradfitz, khr CC=golang-codereviews https://golang.org/cl/120930043
2014-08-01 07:21:08 -06:00
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_0:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_3or4:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVW AX, (DI)
MOVW AX, -2(DI)(BX*1)
RET
runtime: memmove/memclr pointers atomically Make sure that we're moving or zeroing pointers atomically. Anything that is a multiple of pointer size and at least pointer aligned might have pointers in it. All the code looks ok except for the 1-pointer-sized moves. Fixes #13160 Update #12552 Change-Id: Ib97d9b918fa9f4cc5c56c67ed90255b7fdfb7b45 Reviewed-on: https://go-review.googlesource.com/16668 Reviewed-by: Dmitry Vyukov <dvyukov@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-11-05 13:39:56 -07:00
_5through7:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVL AX, (DI)
MOVL AX, -4(DI)(BX*1)
RET
runtime: memmove/memclr pointers atomically Make sure that we're moving or zeroing pointers atomically. Anything that is a multiple of pointer size and at least pointer aligned might have pointers in it. All the code looks ok except for the 1-pointer-sized moves. Fixes #13160 Update #12552 Change-Id: Ib97d9b918fa9f4cc5c56c67ed90255b7fdfb7b45 Reviewed-on: https://go-review.googlesource.com/16668 Reviewed-by: Dmitry Vyukov <dvyukov@google.com> Run-TryBot: Keith Randall <khr@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org>
2015-11-05 13:39:56 -07:00
_8:
// We need a separate case for 8 to make sure we clear pointers atomically.
MOVQ AX, (DI)
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_9through16:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVQ AX, (DI)
MOVQ AX, -8(DI)(BX*1)
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_17through32:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVOU X0, (DI)
MOVOU X0, -16(DI)(BX*1)
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_33through64:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVOU X0, (DI)
MOVOU X0, 16(DI)
MOVOU X0, -32(DI)(BX*1)
MOVOU X0, -16(DI)(BX*1)
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_65through128:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVOU X0, (DI)
MOVOU X0, 16(DI)
MOVOU X0, 32(DI)
MOVOU X0, 48(DI)
MOVOU X0, -64(DI)(BX*1)
MOVOU X0, -48(DI)(BX*1)
MOVOU X0, -32(DI)(BX*1)
MOVOU X0, -16(DI)(BX*1)
RET
[dev.power64] cmd/5a, cmd/6a, cmd/8a, cmd/9a: make labels function-scoped I removed support for jumping between functions years ago, as part of doing the instruction layout for each function separately. Given that, it makes sense to treat labels as function-scoped. This lets each function have its own 'loop' label, for example. Makes the assembly much cleaner and removes the last reason anyone would reach for the 123(PC) form instead. Note that this is on the dev.power64 branch, but it changes all the assemblers. The change will ship in Go 1.5 (perhaps after being ported into the new assembler). Came up as part of CL 167730043. LGTM=r R=r CC=austin, dave, golang-codereviews, minux https://golang.org/cl/159670043
2014-10-28 19:50:16 -06:00
_129through256:
runtime: faster memclr on x86. Use explicit SSE writes instead of REP STOSQ. benchmark old ns/op new ns/op delta BenchmarkMemclr5 22 5 -73.62% BenchmarkMemclr16 27 5 -78.49% BenchmarkMemclr64 28 6 -76.43% BenchmarkMemclr256 34 8 -74.94% BenchmarkMemclr4096 112 84 -24.73% BenchmarkMemclr65536 1902 1920 +0.95% LGTM=dvyukov R=golang-codereviews, dvyukov CC=golang-codereviews https://golang.org/cl/60090044
2014-02-06 18:43:22 -07:00
MOVOU X0, (DI)
MOVOU X0, 16(DI)
MOVOU X0, 32(DI)
MOVOU X0, 48(DI)
MOVOU X0, 64(DI)
MOVOU X0, 80(DI)
MOVOU X0, 96(DI)
MOVOU X0, 112(DI)
MOVOU X0, -128(DI)(BX*1)
MOVOU X0, -112(DI)(BX*1)
MOVOU X0, -96(DI)(BX*1)
MOVOU X0, -80(DI)(BX*1)
MOVOU X0, -64(DI)(BX*1)
MOVOU X0, -48(DI)(BX*1)
MOVOU X0, -32(DI)(BX*1)
MOVOU X0, -16(DI)(BX*1)
RET
Copy Permalink