1
0
mirror of https://github.com/golang/go synced 2024-10-05 10:31:22 -06:00
Commit Graph

72 Commits

Author SHA1 Message Date
Dmitriy Vyukov
83d4cd758c runtime: minor code style improvements (followup to change 9778049)
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9693044
2013-06-02 01:45:26 +04:00
Dmitriy Vyukov
86da989ee5 runtime: introduce helper persistentalloc() function
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
Reincarnation of committed and rolled back https://golang.org/cl/9805043
The latent bugs that it revealed are fixed:
https://golang.org/cl/9837049
https://golang.org/cl/9778048

R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9778049
2013-05-31 10:42:30 +04:00
Keith Randall
d6f89d735e runtime: set MSpan.limit properly for large spans.
Then use the limit to make sure MHeap_LookupMaybe & inlined
copies don't return a span if the pointer is beyond the limit.
Use this fact to optimize all call sites.

R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/9869045
2013-05-30 21:32:20 -07:00
Dmitriy Vyukov
e17281b397 runtime: rename mheap.maps to mheap.spans
as was dicussed in cl/9791044

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9853046
2013-05-30 17:09:58 +04:00
Dmitriy Vyukov
8bbb08533d runtime: make mheap statically allocated again
This depends on: 9791044: runtime: allocate page table lazily
Once page table is moved out of heap, the heap becomes small.
This removes unnecessary dereferences during heap access.
No logical changes.

R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9802043
2013-05-28 22:14:47 +04:00
Dmitriy Vyukov
671814b904 runtime: allocate page table lazily
This removes the 256MB memory allocation at startup,
which conflicts with ulimit.
Also will allow to eliminate an unnecessary memory dereference in GC,
because the page table is usually mapped at known address.
Update #5049.
Update #5236.

R=golang-dev, khr, r, khr, rsc
CC=golang-dev
https://golang.org/cl/9791044
2013-05-28 22:04:34 +04:00
Dmitriy Vyukov
828c68f8d8 undo CL 9805043 / 776aba85ece8
multiple failures on amd64

««« original CL description
runtime: introduce helper persistentalloc() function
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.

R=golang-dev, daniel.morsing, khr
CC=golang-dev
https://golang.org/cl/9805043
»»»

R=golang-dev
CC=golang-dev
https://golang.org/cl/9822043
2013-05-28 11:14:39 +04:00
Dmitriy Vyukov
5166013f75 runtime: inline MCache_Alloc() into mallocgc()
benchmark                    old ns/op    new ns/op    delta
BenchmarkMalloc8                    68           62   -8.63%
BenchmarkMalloc16                   75           69   -7.94%
BenchmarkMallocTypeInfo8           102           98   -3.73%
BenchmarkMallocTypeInfo16          108          103   -4.63%

R=golang-dev, dave, khr
CC=golang-dev
https://golang.org/cl/9790043
2013-05-28 11:05:55 +04:00
Dmitriy Vyukov
47e0a3d7b1 runtime: introduce helper persistentalloc() function
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.

R=golang-dev, daniel.morsing, khr
CC=golang-dev
https://golang.org/cl/9805043
2013-05-28 10:47:35 +04:00
Dmitriy Vyukov
5782f4117d runtime: introduce cnewarray() to simplify allocation of typed arrays
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/9648044
2013-05-27 11:29:11 +04:00
Dmitriy Vyukov
c075d82cca runtime: fix and speedup malloc stats
Currently per-sizeclass stats are lost for destroyed MCache's. This patch fixes this.
Also, only update mstats.heap_alloc on heap operations, because that's the only
stat that needs to be promptly updated. Everything else needs to be up-to-date only in ReadMemStats().

R=golang-dev, remyoudompheng, dave, iant
CC=golang-dev
https://golang.org/cl/9207047
2013-05-22 22:22:57 +04:00
Dmitriy Vyukov
5a89b35bca runtime: inline size to class conversion in malloc()
Also change table type from int32[] to int8[] to save space in L1$.

benchmark          old ns/op    new ns/op    delta
BenchmarkMalloc           42           40   -4.68%

R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/9199044
2013-05-15 11:02:33 +04:00
Carl Shapiro
c80f5b9bf7 runtime: use a distinct pattern to mark free blocks in need of zeroing
R=golang-dev, dvyukov, khr, cshapiro
CC=golang-dev
https://golang.org/cl/8392043
2013-04-04 14:18:52 -07:00
Ian Lance Taylor
515353a290 pkg/runtime: ignore runtime.memlimit when allocating heap
For Go 1.1, stop checking the rlimit, because it broke now
that mheap is allocated using SysAlloc.  See issue 5049.

R=r
CC=golang-dev
https://golang.org/cl/7741050
2013-03-26 14:01:12 -07:00
Jan Ziak
2001f0c28e runtime: prevent garbage collection during hashmap insertion (fix 2)
Fixes #5074 in multi-threaded scenarios.

R=golang-dev, daniel.morsing, dave, dvyukov, bradfitz, rsc
CC=golang-dev, remyoudompheng
https://golang.org/cl/7916043
2013-03-20 20:36:33 +01:00
Dmitriy Vyukov
76959e2cc6 runtime: do not memprofile settype_flush
Fixes #4850.

R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/7745044
2013-03-14 19:11:29 +04:00
Jan Ziak
94955f9b40 runtime: check the value returned by runtime·SysAlloc
R=golang-dev, rsc
CC=golang-dev, minux.ma
https://golang.org/cl/7424047
2013-03-01 00:21:08 -05:00
Russ Cox
8a6ff3ab34 runtime: allocate heap metadata at run time
Before, the mheap structure was in the bss,
but it's quite large (today, 256 MB, much of
which is never actually paged in), and it makes
Go binaries run afoul of exec-time bss size
limits on some BSD systems.

Fixes #4447.

R=golang-dev, dave, minux.ma, remyoudompheng, iant
CC=golang-dev
https://golang.org/cl/7307122
2013-02-15 14:27:03 -05:00
Dmitriy Vyukov
691455f780 runtime: move stack management related code to stack.c
No code changes.
This is mainly in preparation to scheduler changes,
oldstack/newstack are not related to scheduling.

R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/7311085
2013-02-14 12:37:55 +04:00
Shenghou Ma
4019d0e424 runtime: avoid defining the same variable in more than one translation unit
For gccgo runtime and Darwin where -fno-common is the default.

R=iant, dave
CC=golang-dev
https://golang.org/cl/7094061
2013-01-26 09:57:06 +08:00
Dmitriy Vyukov
7847f328cd runtime: faster segmented stacks
benchmark                      old ns/op    new ns/op    delta
BenchmarkStackGrowth                 665          548  -17.59%
BenchmarkStackGrowth-2               333          274  -17.72%
BenchmarkStackGrowth-4               224          168  -25.00%
BenchmarkStackGrowth-8               124           91  -26.21%
BenchmarkStackGrowth-16               82           70  -14.55%
BenchmarkStackGrowth-32               73           59  -19.49%

R=nigeltao, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/7026044
2013-01-10 11:36:40 +04:00
Dmitriy Vyukov
f82db7d9e4 runtime: less aggressive per-thread stack segment caching
Introduce global stack segment cache and limit per-thread cache size.
This greatly reduces StackSys memory on workloads that create lots of threads.

benchmark                      old ns/op    new ns/op    delta
BenchmarkStackGrowth                 665          656   -1.35%
BenchmarkStackGrowth-2               333          328   -1.50%
BenchmarkStackGrowth-4               224          172  -23.21%
BenchmarkStackGrowth-8               124           91  -26.13%
BenchmarkStackGrowth-16               82           47  -41.94%
BenchmarkStackGrowth-32               73           40  -44.79%

BenchmarkStackGrowthDeep           97231        94391   -2.92%
BenchmarkStackGrowthDeep-2         47230        58562  +23.99%
BenchmarkStackGrowthDeep-4         24993        49356  +97.48%
BenchmarkStackGrowthDeep-8         15105        30072  +99.09%
BenchmarkStackGrowthDeep-16        10005        15623  +56.15%
BenchmarkStackGrowthDeep-32        12517        13069   +4.41%

TestStackMem#1,MB                  310          12       -96.13%
TestStackMem#2,MB                  296          14       -95.27%
TestStackMem#3,MB                  479          14       -97.08%

TestStackMem#1,sec                 3.22         2.26     -29.81%
TestStackMem#2,sec                 2.43         2.15     -11.52%
TestStackMem#3,sec                 2.50         2.38      -4.80%

R=sougou, no.smile.face, rsc
CC=golang-dev, msolomon
https://golang.org/cl/7029044
2013-01-10 09:57:06 +04:00
Russ Cox
ec59b840f9 runtime: coalesce 0-size allocations
Fixes #3996.

R=ken2
CC=golang-dev
https://golang.org/cl/7001052
2012-12-22 16:42:22 -05:00
Jingcheng Zhang
70e967b7bc runtime: use "mp" and "gp" instead of "m" and "g" for local variable name to avoid confusion with the global "m" and "g".
R=golang-dev, minux.ma, rsc
CC=bradfitz, golang-dev
https://golang.org/cl/6939064
2012-12-19 00:30:29 +08:00
Russ Cox
9799a5a4fd runtime: allow up to 128 GB of allocated memory
Incorporates code from CL 6828055.

Fixes #2142.

R=golang-dev, iant, devon.odell
CC=golang-dev
https://golang.org/cl/6826088
2012-11-13 12:45:08 -05:00
Dmitriy Vyukov
1a19f01a68 runtime/race: lazily allocate shadow memory
Currently race detector runtime maps shadow memory eagerly at process startup.
It works poorly on Windows, because Windows requires reservation in swap file
(especially problematic if several Go program runs at the same, each consuming GBs
of memory).
With this change race detector maps shadow memory lazily, so Go runtime must notify
about all new heap memory.
It will help with Windows port, but also eliminates scary 16TB virtual mememory
consumption in top output (which sometimes confuses some monitoring scripts).

R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6811085
2012-11-07 12:48:58 +04:00
Shenghou Ma
7c44edf425 runtime, runtime/race: add missing if(raceenabled), update package docs of pkg race
R=dvyukov
CC=golang-dev
https://golang.org/cl/6733058
2012-10-23 02:33:51 +08:00
Jan Ziak
4a191c2c1b runtime: store types of allocated objects
R=rsc
CC=golang-dev
https://golang.org/cl/6569057
2012-10-21 17:41:32 -04:00
Shenghou Ma
c1b7ddc6aa runtime: update docs for MemStats.PauseNs
PauseNs is a circular buffer of recent pause times, and the
most recent one is at [((NumGC-1)+256)%256].

   Also fix comments cross-linking the Go and C definition of
various structs.

R=golang-dev, rsc, bradfitz
CC=golang-dev
https://golang.org/cl/6657047
2012-10-22 01:08:13 +08:00
Dmitriy Vyukov
6273c7324f runtime: add missing if(raceenabled)
R=0xe2.0x9a.0x9b, minux.ma, iant, dave
CC=golang-dev
https://golang.org/cl/6654052
2012-10-15 13:54:31 +04:00
Dmitriy Vyukov
2f6cbc74f1 race: runtime changes
This is a part of a bigger change that adds data race detection feature:
https://golang.org/cl/6456044

R=rsc
CC=gobot, golang-dev
https://golang.org/cl/6535050
2012-10-07 22:05:32 +04:00
Jan Ziak
f8c58373e5 runtime: add types to MSpan
R=rsc
CC=golang-dev
https://golang.org/cl/6554060
2012-09-24 20:08:05 -04:00
Russ Cox
0b08c9483f runtime: prepare for 64-bit ints
This CL makes the runtime understand that the type of
the len or cap of a map, slice, or string is 'int', not 'int32',
and it is also careful to distinguish between function arguments
and results of type 'int' vs type 'int32'.

In the runtime, the new typedefs 'intgo' and 'uintgo' refer
to Go int and uint. The C types int and uint continue to be
unavailable (cause intentional compile errors).

This CL does not change the meaning of int, but it should make
the eventual change of the meaning of int on amd64 a bit
smoother.

Update #2188.

R=iant, r, dave, remyoudompheng
CC=golang-dev
https://golang.org/cl/6551067
2012-09-24 14:58:34 -04:00
Dmitriy Vyukov
ed516df4e4 runtime: add freemcache() function
It will be required for scheduler that maintains
GOMAXPROCS MCache's.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/6350062
2012-07-01 13:10:01 +04:00
Dave Cheney
0b09425b5c runtime: use uintptr where possible in malloc stats
linux/arm OMAP4 pandaboard

benchmark                 old ns/op    new ns/op    delta
BenchmarkBinaryTree17   68723297000  37026214000  -46.12%
BenchmarkFannkuch11     34962402000  35958435000   +2.85%
BenchmarkGobDecode        137298600    124182150   -9.55%
BenchmarkGobEncode         60717160     60006700   -1.17%
BenchmarkGzip            5647156000   5550873000   -1.70%
BenchmarkGunzip          1196350000   1198670000   +0.19%
BenchmarkJSONEncode       863012800    782898000   -9.28%
BenchmarkJSONDecode      3312989000   2781800000  -16.03%
BenchmarkMandelbrot200     45727540     45703120   -0.05%
BenchmarkParse             74781800     59990840  -19.78%
BenchmarkRevcomp          140043650    139462300   -0.42%
BenchmarkTemplate        6467682000   5832153000   -9.83%

benchmark                  old MB/s     new MB/s  speedup
BenchmarkGobDecode             5.59         6.18    1.11x
BenchmarkGobEncode            12.64        12.79    1.01x
BenchmarkGzip                  3.44         3.50    1.02x
BenchmarkGunzip               16.22        16.19    1.00x
BenchmarkJSONEncode            2.25         2.48    1.10x
BenchmarkJSONDecode            0.59         0.70    1.19x
BenchmarkParse                 0.77         0.97    1.26x
BenchmarkRevcomp              18.15        18.23    1.00x
BenchmarkTemplate              0.30         0.33    1.10x

darwin/386 core duo

benchmark                 old ns/op    new ns/op    delta
BenchmarkBinaryTree17   10591616577   9678245733   -8.62%
BenchmarkFannkuch11     10758473315  10749303846   -0.09%
BenchmarkGobDecode         34379785     34121250   -0.75%
BenchmarkGobEncode         23523721     23475750   -0.20%
BenchmarkGzip            2486191492   2446539568   -1.59%
BenchmarkGunzip           444179328    444250293   +0.02%
BenchmarkJSONEncode       221138507    219757826   -0.62%
BenchmarkJSONDecode      1056034428   1048975133   -0.67%
BenchmarkMandelbrot200     19862516     19868346   +0.03%
BenchmarkRevcomp         3742610872   3724821662   -0.48%
BenchmarkTemplate         960283112    944791517   -1.61%

benchmark                  old MB/s     new MB/s  speedup
BenchmarkGobDecode            22.33        22.49    1.01x
BenchmarkGobEncode            32.63        32.69    1.00x
BenchmarkGzip                  7.80         7.93    1.02x
BenchmarkGunzip               43.69        43.68    1.00x
BenchmarkJSONEncode            8.77         8.83    1.01x
BenchmarkJSONDecode            1.84         1.85    1.01x
BenchmarkRevcomp              67.91        68.24    1.00x
BenchmarkTemplate              2.02         2.05    1.01x

R=rsc, 0xe2.0x9a.0x9b, mirtchovski
CC=golang-dev, minux.ma
https://golang.org/cl/6297047
2012-06-08 17:35:14 -04:00
Jan Ziak
ede6718cd7 runtime: move type kinds into a separate file
R=rsc
CC=golang-dev
https://golang.org/cl/6285047
2012-06-06 17:20:02 -04:00
Jan Ziak
773685b4a3 runtime: fix counting of free objects
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6206056
2012-05-15 11:48:58 -04:00
Dmitriy Vyukov
c1c851bbe8 runtime: avoid unnecessary zeroization of huge memory blocks
+move zeroization out of the heap mutex

R=golang-dev, iant, rsc
CC=golang-dev
https://golang.org/cl/6094050
2012-05-02 18:01:11 +04:00
Rémy Oudompheng
3dcedb620c runtime: try extending arena size in 32-bit allocator.
If it didn't reach the limit, we can try extending the arena
before resorting to random memory mappings and praying for the
kernel to be kind.

Fixes #3173.

R=rsc, rsc
CC=golang-dev
https://golang.org/cl/5725045
2012-03-07 14:21:45 -05:00
Russ Cox
102274a30e runtime: size arena to fit in virtual address space limit
For Brad.
Now FreeBSD/386 binaries run on nearlyfreespeech.net.

Fixes #2302.

R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5700060
2012-02-24 15:28:51 -05:00
Russ Cox
5bcad92f07 ld: add NOPTRBSS for large, pointer-free uninitialized data
cc: add #pragma textflag to set it
runtime: mark mheap to go into noptr-bss.
        remove special case in garbage collector

Remove the ARM from.flag field created by CL 5687044.
The DUPOK flag was already in p->reg, so keep using that.

Otherwise test/nilptr.go creates a very large binary.
Should fix the arm build.
Diagnosed by minux.ma; replacement for CL 5690044.

R=golang-dev, minux.ma, r
CC=golang-dev
https://golang.org/cl/5686060
2012-02-21 22:08:42 -05:00
Russ Cox
03f2289f7e runtime: API
Delete Alloc, Free, Lookup, Semacquire, Semrelease

Fixes #2955.

R=golang-dev, r, bradfitz
CC=golang-dev
https://golang.org/cl/5675093
2012-02-19 00:11:44 -05:00
Russ Cox
1253c75cf5 runtime: fix compiler warnings
R=golang-dev, gri
CC=golang-dev
https://golang.org/cl/5642068
2012-02-09 16:48:52 +11:00
Shenghou Ma
073aeff785 runtime: fix "SysReserve returned unaligned address" bug on 32-bit systems
R=rsc
CC=golang-dev
https://golang.org/cl/5642064
2012-02-09 09:25:10 +11:00
Paul Borman
d37a8b73c5 runtime: drop to 32 bit malloc if 64 bit will not work
On 64 bit UML it is not possible to reserve memory at 0xF8<<32.
Detect when linux cannot use these high virtual memory addresses
and drop back to the 32 bit memory allocator.

R=rsc, cw
CC=golang-dev
https://golang.org/cl/5634050
2012-02-08 14:39:16 -05:00
Maxim Pimenov
dcdc309c7c runtime: fix typo in comment
R=golang-dev
CC=golang-dev
https://golang.org/cl/5529059
2012-01-10 12:56:25 -08:00
Russ Cox
851f30136d runtime: make more build-friendly
Collapse the arch,os-specific directories into the main directory
by renaming xxx/foo.c to foo_xxx.c, and so on.

There are no substantial edits here, except to the Makefile.
The assumption is that the Go tool will #define GOOS_darwin
and GOARCH_amd64 and will make any file named something
like signals_darwin.h available as signals_GOOS.h during the
build.  This replaces what used to be done with -I$(GOOS).

There is still work to be done to make runtime build with
standard tools, but this is a big step.  After this we will have
to write a script to generate all the generated files so they
can be checked in (instead of generated during the build).

R=r, iant, r, lucio.dere
CC=golang-dev
https://golang.org/cl/5490053
2011-12-16 15:33:58 -05:00
Dmitriy Vyukov
1135fc3978 runtime: fix crash if user sets MemProfileRate=0
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5284044
2011-10-16 10:49:24 +03:00
Dmitriy Vyukov
bf9c778fe2 gc: pass FlagNoPointers to runtime.new
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/5151043
2011-10-13 11:06:55 +03:00
Dmitriy Vyukov
c14b2689f0 runtime: faster finalizers
Linux/amd64, 2 x Intel Xeon E5620, 8 HT cores, 2.40GHz
benchmark                    old ns/op    new ns/op    delta
BenchmarkFinalizer              420.00       261.00  -37.86%
BenchmarkFinalizer-2            985.00       201.00  -79.59%
BenchmarkFinalizer-4           1077.00       244.00  -77.34%
BenchmarkFinalizer-8           1155.00       180.00  -84.42%
BenchmarkFinalizer-16          1182.00       184.00  -84.43%

BenchmarkFinalizerRun          2128.00      1378.00  -35.24%
BenchmarkFinalizerRun-2        1655.00      1418.00  -14.32%
BenchmarkFinalizerRun-4        1634.00      1522.00   -6.85%
BenchmarkFinalizerRun-8        2213.00      1581.00  -28.56%
BenchmarkFinalizerRun-16       2424.00      1599.00  -34.03%

Darwin/amd64, Intel L9600, 2 cores, 2.13GHz
benchmark                    old ns/op    new ns/op    delta
BenchmarkChanCreation          1451.00       926.00  -36.18%
BenchmarkChanCreation-2        3124.00      1412.00  -54.80%
BenchmarkChanCreation-4        6121.00      2628.00  -57.07%

BenchmarkFinalizer              684.00       420.00  -38.60%
BenchmarkFinalizer-2          11195.00       398.00  -96.44%
BenchmarkFinalizer-4          15862.00       654.00  -95.88%

BenchmarkFinalizerRun          2025.00      1397.00  -31.01%
BenchmarkFinalizerRun-2        3920.00      1447.00  -63.09%
BenchmarkFinalizerRun-4        9471.00      1545.00  -83.69%

R=golang-dev, cw, rsc
CC=golang-dev
https://golang.org/cl/4963057
2011-10-06 18:42:51 +03:00