mirror of
https://github.com/golang/go
synced 2024-11-08 19:06:26 -07:00
2b415549b8
This replaces the contiguous heap arena mapping with a potentially sparse mapping that can support heap mappings anywhere in the address space. This has several advantages over the current approach: * There is no longer any limit on the size of the Go heap. (Currently it's limited to 512GB.) Hence, this fixes #10460. * It eliminates many failures modes of heap initialization and growing. In particular it eliminates any possibility of panicking with an address space conflict. This can happen for many reasons and even causes a low but steady rate of TSAN test failures because of conflicts with the TSAN runtime. See #16936 and #11993. * It eliminates the notion of "non-reserved" heap, which was added because creating huge address space reservations (particularly on 64-bit) led to huge process VSIZE. This was at best confusing and at worst conflicted badly with ulimit -v. However, the non-reserved heap logic is complicated, can race with other mappings in non-pure Go binaries (e.g., #18976), and requires that the entire heap be either reserved or non-reserved. We currently maintain the latter property, but it's quite difficult to convince yourself of that, and hence difficult to keep correct. This logic is still present, but will be removed in the next CL. * It fixes problems on 32-bit where skipping over parts of the address space leads to mapping huge (and never-to-be-used) metadata structures. See #19831. This also completely rewrites and significantly simplifies mheap.sysAlloc, which has been a source of many bugs. E.g., #21044, #20259, #18651, and #13143 (and maybe #23222). This change also makes it possible to allocate individual objects larger than 512GB. As a result, a few tests that expected huge allocations to fail needed to be changed to make even larger allocations. However, at the moment attempting to allocate a humongous object may cause the program to freeze for several minutes on Linux as we fall back to probing every page with addrspace_free. That logic (and this failure mode) will be removed in the next CL. Fixes #10460. Fixes #22204 (since it rewrites the code involved). This slightly slows down compilebench and the x/benchmarks garbage benchmark. name old time/op new time/op delta Template 184ms ± 1% 185ms ± 1% ~ (p=0.065 n=10+9) Unicode 86.9ms ± 3% 86.3ms ± 1% ~ (p=0.631 n=10+10) GoTypes 599ms ± 0% 602ms ± 0% +0.56% (p=0.000 n=10+9) Compiler 2.87s ± 1% 2.89s ± 1% +0.51% (p=0.002 n=9+10) SSA 7.29s ± 1% 7.25s ± 1% ~ (p=0.182 n=10+9) Flate 118ms ± 2% 118ms ± 1% ~ (p=0.113 n=9+9) GoParser 147ms ± 1% 148ms ± 1% +1.07% (p=0.003 n=9+10) Reflect 401ms ± 1% 404ms ± 1% +0.71% (p=0.003 n=10+9) Tar 175ms ± 1% 175ms ± 1% ~ (p=0.604 n=9+10) XML 209ms ± 1% 210ms ± 1% ~ (p=0.052 n=10+10) (https://perf.golang.org/search?q=upload:20171231.4) name old time/op new time/op delta Garbage/benchmem-MB=64-12 2.23ms ± 1% 2.25ms ± 1% +0.84% (p=0.000 n=19+19) (https://perf.golang.org/search?q=upload:20171231.3) Relative to the start of the sparse heap changes (starting at and including "runtime: fix various contiguous bitmap assumptions"), overall slowdown is roughly 1% on GC-intensive benchmarks: name old time/op new time/op delta Template 183ms ± 1% 185ms ± 1% +1.32% (p=0.000 n=9+9) Unicode 84.9ms ± 2% 86.3ms ± 1% +1.65% (p=0.000 n=9+10) GoTypes 595ms ± 1% 602ms ± 0% +1.19% (p=0.000 n=9+9) Compiler 2.86s ± 0% 2.89s ± 1% +0.91% (p=0.000 n=9+10) SSA 7.19s ± 0% 7.25s ± 1% +0.75% (p=0.000 n=8+9) Flate 117ms ± 1% 118ms ± 1% +1.10% (p=0.000 n=10+9) GoParser 146ms ± 2% 148ms ± 1% +1.48% (p=0.002 n=10+10) Reflect 398ms ± 1% 404ms ± 1% +1.51% (p=0.000 n=10+9) Tar 173ms ± 1% 175ms ± 1% +1.17% (p=0.000 n=10+10) XML 208ms ± 1% 210ms ± 1% +0.62% (p=0.011 n=10+10) [Geo mean] 369ms 373ms +1.17% (https://perf.golang.org/search?q=upload:20180101.2) name old time/op new time/op delta Garbage/benchmem-MB=64-12 2.22ms ± 1% 2.25ms ± 1% +1.51% (p=0.000 n=20+19) (https://perf.golang.org/search?q=upload:20180101.3) Change-Id: I5daf4cfec24b252e5a57001f0a6c03f22479d0f0 Reviewed-on: https://go-review.googlesource.com/85887 Run-TryBot: Austin Clements <austin@google.com> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Rick Hudson <rlh@golang.org>
80 lines
1.3 KiB
Go
80 lines
1.3 KiB
Go
// run
|
|
|
|
// Copyright 2010 The Go Authors. All rights reserved.
|
|
// Use of this source code is governed by a BSD-style
|
|
// license that can be found in the LICENSE file.
|
|
|
|
// https://golang.org/issue/589
|
|
|
|
package main
|
|
|
|
import "unsafe"
|
|
|
|
var bug = false
|
|
|
|
var minus1 = -1
|
|
var five = 5
|
|
var big int64 = 10 | 1<<40
|
|
|
|
type block [1 << 19]byte
|
|
|
|
var g1 []block
|
|
|
|
func shouldfail(f func(), desc string) {
|
|
defer func() { recover() }()
|
|
f()
|
|
if !bug {
|
|
println("BUG")
|
|
bug = true
|
|
}
|
|
println("didn't crash: ", desc)
|
|
}
|
|
|
|
func badlen() {
|
|
g1 = make([]block, minus1)
|
|
}
|
|
|
|
func biglen() {
|
|
g1 = make([]block, big)
|
|
}
|
|
|
|
func badcap() {
|
|
g1 = make([]block, 10, minus1)
|
|
}
|
|
|
|
func badcap1() {
|
|
g1 = make([]block, 10, five)
|
|
}
|
|
|
|
func bigcap() {
|
|
g1 = make([]block, 10, big)
|
|
}
|
|
|
|
type cblock [1<<16 - 1]byte
|
|
|
|
var g4 chan cblock
|
|
|
|
func badchancap() {
|
|
g4 = make(chan cblock, minus1)
|
|
}
|
|
|
|
func bigchancap() {
|
|
g4 = make(chan cblock, big)
|
|
}
|
|
|
|
func overflowchan() {
|
|
const ptrSize = unsafe.Sizeof(uintptr(0))
|
|
g4 = make(chan cblock, 1<<(30*(ptrSize/4)))
|
|
}
|
|
|
|
func main() {
|
|
shouldfail(badlen, "badlen")
|
|
shouldfail(biglen, "biglen")
|
|
shouldfail(badcap, "badcap")
|
|
shouldfail(badcap1, "badcap1")
|
|
shouldfail(bigcap, "bigcap")
|
|
shouldfail(badchancap, "badchancap")
|
|
shouldfail(bigchancap, "bigchancap")
|
|
shouldfail(overflowchan, "overflowchan")
|
|
}
|