When the gctrace GODEBUG option is enabled, it will now report three
heap sizes: the heap size at the beginning of the GC cycle, the heap
size at the end of the GC cycle before sweeping, and marked heap size,
which is the amount of heap that will be retained until the next GC
cycle.
Change-Id: Ie13f8a6d5c609bc9cc47c7555960ab55b37b5f1c
Reviewed-on: https://go-review.googlesource.com/8430
Reviewed-by: Rick Hudson <rlh@golang.org>
In the STW collector, next_gc was both the heap size to trigger GC at
as well as the goal heap size.
Early in the concurrent collector's development, next_gc was the goal
heap size, but was also used as the heap size to trigger GC at. This
meant we always overshot the goal because of allocation during
concurrent GC.
Currently, next_gc is still the goal heap size, but we trigger
concurrent GC at 7/8*GOGC heap growth. This complicates
shouldtriggergc, but was necessary because of the incremental
maintenance of next_gc.
Now we simply compute next_gc for the next cycle during mark
termination. Hence, it's now easy to take the simpler route and
redefine next_gc as the heap size at which the next GC triggers. We
can directly compute this with the 7/8 backoff during mark termination
and shouldtriggergc can simply test if the live heap size has grown
over the next_gc trigger.
This will also simplify later changes once we start setting next_gc in
more sophisticated ways.
Change-Id: I872be4ae06b4f7a0d7f7967360a054bd36b90eea
Reviewed-on: https://go-review.googlesource.com/8420
Reviewed-by: Russ Cox <rsc@golang.org>
Currently there are two main consumers of memstats.heap_alloc:
updatememstats (aka ReadMemStats) and shouldtriggergc.
updatememstats recomputes heap_alloc from the ground up, so we don't
need to keep heap_alloc up to date for it. shouldtriggergc wants to
know how many bytes were marked by the previous GC plus how many bytes
have been allocated since then, but this *isn't* what heap_alloc
tracks. heap_alloc also includes objects that are not marked and
haven't yet been swept.
Introduce a new memstat called heap_live that actually tracks what
shouldtriggergc wants to know and stop keeping heap_alloc up to date.
Unlike heap_alloc, heap_live follows a simple sawtooth that drops
during each mark termination and increases monotonically between GCs.
heap_alloc, on the other hand, has much more complicated behavior: it
may drop during sweep termination, slowly decreases from background
sweeping between GCs, is roughly unaffected by allocation as long as
there are unswept spans (because we sweep and allocate at the same
rate), and may go up after background sweeping is done depending on
the GC trigger.
heap_live simplifies computing next_gc and using it to figure out when
to trigger garbage collection. Currently, we guess next_gc at the end
of a cycle and update it as we sweep and get a better idea of how much
heap was marked. Now, since we're directly tracking how much heap is
marked, we can directly compute next_gc.
This also corrects bugs that could cause us to trigger GC early.
Currently, in any case where sweep termination actually finds spans to
sweep, heap_alloc is an overestimation of live heap, so we'll trigger
GC too early. heap_live, on the other hand, is unaffected by sweeping.
Change-Id: I1f96807b6ed60d4156e8173a8e68745ffc742388
Reviewed-on: https://go-review.googlesource.com/8389
Reviewed-by: Russ Cox <rsc@golang.org>
This tracks the number of heap bytes marked by a GC cycle. We'll use
this information to precisely trigger the next GC cycle.
Currently this aggregates the work counter in gcWork and dispose
atomically aggregates this into a global work counter. dispose happens
relatively infrequently, so the contention on the global counter
should be low. If this turns out to be an issue, we can reduce the
number of disposes, and if it's still a problem, we can switch to
per-P counters.
Change-Id: I1bc377cb2e802ef61c2968602b63146d52e7f5db
Reviewed-on: https://go-review.googlesource.com/8388
Reviewed-by: Russ Cox <rsc@golang.org>
It referred to the wrong architecture.
Fixes#10355.
Change-Id: I5b9d31c9f04f3106b93f94fa68c848b2518b128e
Reviewed-on: https://go-review.googlesource.com/8495
Reviewed-by: Dave Cheney <dave@cheney.net>
This fixes the formerly extremely slow conversion of floating-point
constants with large exponents (e.g., "const c = 1e1000000000" could
stall the machine).
Change-Id: I36e02158e3334d32b18743ec0c259fec77baa74f
Reviewed-on: https://go-review.googlesource.com/8466
Reviewed-by: Alan Donovan <adonovan@google.com>
As noted on recently on golang-nuts, there's currently no way to know
the total size of a strings.Reader or bytes.Reader when using ReadAt
on them. Most callers resort to wrapping it in an io.SectionReader to
retain that information.
The SizeReaderAt abstraction (an io.ReaderAt with a Size() int64
method) has proven useful as a way of expressing a concurrency-safe
read-only number of bytes.
As one example, see http://talks.golang.org/2013/oscon-dl.slide#49 and
the rest of that presentation for its use in dl.google.com.
SizeReaderAt is also used in the open source google-api-go-client, and
within Google's internal codebase, where it exists in a public package
created in 2013 with the package comment: "These may migrate to the
standard library after we have enough experience with their feel."
I'm still as happy with the SizeReaderAt abstraction and its
composabilty as I was in 2013, so I'd like to make these two Readers
also be SizeReaderAts.
Fixes#9667
Change-Id: Ie6f145ada419dd116280472d8c029f046d5edf70
Reviewed-on: https://go-review.googlesource.com/3199
Reviewed-by: Andrew Gerrand <adg@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Now, only a zero deadline is interpreted as noDeadline. Any other time
in the past yields an immediate timeout.
TestConnectDeadlineInThePast already covers this case. We just need to
un-skip it for plan9, where dialChannel is used.
Change-Id: I995fd1a632c31f8004dac772c3d7c43a2a5853b0
Reviewed-on: https://go-review.googlesource.com/8435
Reviewed-by: Mikio Hara <mikioh.mikioh@gmail.com>
The the has been deleted.
Change-Id: I4290105435d4f1fd10c7014f913a3147ddeb3c2b
Reviewed-on: https://go-review.googlesource.com/8469
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Previously, a production rule like
A: B C D
would cause yacc to check that A and B have the same declared types,
but then it would generate an implicit action of { $$ = $3 } (i.e.,
copy the value from D), even if A and D have different types.
Fixes#10192.
Change-Id: I51cfd7baa0011557141dca33b7af1d892cc6f49e
Reviewed-on: https://go-review.googlesource.com/7780
Reviewed-by: Russ Cox <rsc@golang.org>
This is a follow on to 28f33b4a which removes one of the boolean flags
and adds a test for the key-driven cipher selection.
Change-Id: If2a400de807eb19110352912a9f467491cc8986c
Reviewed-on: https://go-review.googlesource.com/8428
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Adam Langley <agl@golang.org>
Reviewed-by: Jacob Haven <jacob@cloudflare.com>
I wrote some code that added a function in gentext() by sticking it
after Ctxt.Etextp and was very confused when this wasn't written out
sometimes. It turned out that Etextp was not updated by deadcode() so
if the last function is not reachable, my new function was never
seen. This changes deadcode() to update Etextp to the last reachable
funtion.
Change-Id: Ib6a3e7c67ccfb8a15826ce9e0ef046732b5e25d2
Reviewed-on: https://go-review.googlesource.com/8233
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Obtaining the actual size of the underlying storage of the buffer can
be very useful in various scenarios. Long running programs which write
and read large amounts of data to buffers might have to recycle
buffers in order to avoid holding onto potentially huge buffers.
For example, a piece of code which buffers a lot of data in a buffer
might need to release the big buffer and start again with a smaller
buffer after it finished processing the huge amount of data.
In cases where pools of bytes.Buffer are used, being able to check the
size of the allocated data can be very useful.
Instead of forking bytes.Buffer or writing new code, we can export the
Cap() method.
Change-Id: I79d4f0a3cff53b9419d82c8122964761e9e38566
Reviewed-on: https://go-review.googlesource.com/8342
Reviewed-by: Rob Pike <r@golang.org>
Add one option, which is the motivating example, a way to control
what happens when a map is indexed with a key that is not in the map.
Rather than do something specific for that case, we provide a simple
general option mechanism to avoid adding API if something else
comes up. This general approach also makes it easy for html/template
to track (and adapt, should that become important).
New method: Option(option string...). The option strings are key=value
pairs or just simple strings (no =).
New option:
missingkey: Control the behavior during execution if a map is
indexed with a key that is not present in the map.
"missingkey=default" or "missingkey=invalid"
The default behavior: Do nothing and continue execution.
If printed, the result of the index operation is the string
"<no value>".
"missingkey=zero"
The operation returns the zero value for the map type's element.
"missingkey=error"
Execution stops immediately with an error.
Fixes#6288.
Change-Id: Id811e2b99dc05aff324d517faac113ef3c25293a
Reviewed-on: https://go-review.googlesource.com/8462
Reviewed-by: Robert Griesemer <gri@golang.org>
This CL extends cmd/yacc to expose a yyErrorVerbose variable that
changes the error messages from just "syntax error" to "syntax error:
unexpected ${tokname}".
It also moves the yyToknames table generation to after rules have been
processed so that entries can be generated for tokens that aren't
mentioned in the preamble (e.g., '.' in the case of go.y).
Lastly, it restores gc's old code for applying yytfix to yyToknames,
except that substituting "LLITERAL" with litbuf happens in Yyerror.
Fixes#9968.
Change-Id: Icec188d11fdabc1dae31b8a471c35b5c7f6deec7
Reviewed-on: https://go-review.googlesource.com/8432
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Aside from removing the superfluous comment near syms, this diff is
entirely mechanically generated via Emacs's query-replace-regexp to
replace "^\tstruct {\n[^}]*}" with "\t".
Change-Id: Ide7e4b5995f6a121b3f57415e033933ac5c7431a
Reviewed-on: https://go-review.googlesource.com/8427
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Russ Cox <rsc@golang.org>
These registers are not available for programs to use. Prior to this
change, the compiler would crash attempting to use ZR as a general
purpose register. Other programs would compile but on execution would
overwrite the G register and cause havoc.
Fixes linux/arm64 build.
Fixes#10304Fixes#10320
Change-Id: I5cf51d3b77cfe3db7dd6377324950cafb02f8d8b
Reviewed-on: https://go-review.googlesource.com/8456
Reviewed-by: Minux Ma <minux@golang.org>
The original implementation used 16 int "words" but only 29 bits per word
for a total of 16*29 = 464 bits, with a space consumption of 16*64 = 1024
bits on a 64 bit machine. Switching to 512 bits increases precision while
still using (in the worst case) half the amount of memory per mp value on
a 64 bit machine.
Also: Decreased permitted number of least-significant mantissa bits which
may be incorrect when considering if a precise floating-point constant is
an integer from 29 to 16 bits.
Change-Id: Iee9287056f0e9aa4f06ceac0724ff4674f710c53
Reviewed-on: https://go-review.googlesource.com/8429
Reviewed-by: Russ Cox <rsc@golang.org>
I first prototyped this change in Sept 2011, and I discarded it
because it made no difference in the obvious benchmark loop.
It still makes no difference in the obvious benchmark loop,
but in a less obvious one, doing some extra computation
around the calls to Sqrt, not making the call does have a
significant effect.
benchmark old ns/op new ns/op delta
BenchmarkSqrt 4.56 4.57 +0.22%
BenchmarkSqrtIndirect 4.56 4.56 +0.00%
BenchmarkSqrtGo 69.4 69.4 +0.00%
BenchmarkSqrtPrime 4417 3647 -17.43%
This is a warmup for using hardware expansions for some
calls to 1-line assembly routines in the runtime (for example getg).
Change-Id: Ie66be23f8c09d0f7dc4ddd7ca8a93cfce28f55a4
Reviewed-on: https://go-review.googlesource.com/8356
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
No test cases yet, but I found this while double checking the
proginfo table.
Change-Id: Ib59675c117c676c1298bcab8765ca6a8fd234de8
Reviewed-on: https://go-review.googlesource.com/8431
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
I guess we need more builders.
Change-Id: I309e3df7608b9eef9339196fdc50dedf5f9422e4
Reviewed-on: https://go-review.googlesource.com/8434
Reviewed-by: Michael Hudson-Doyle <michael.hudson@canonical.com>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
This change makes use of the socktest package instead of the non-thread
safe variable syscall.SocketDisableIPv6 for simulating unreachable
external networks.
Also adds -ipv4 flag, -ipv6 flag already exists, as a control knob for
testing on each of IPv4-only, IPv6-only and dual IP stack kernels.
Fixes#7687.
Change-Id: I82002007fd526e8cf4de207f935e721df049a22f
Reviewed-on: https://go-review.googlesource.com/8390
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This is Part 2 of the change, see Part 1 here: in https://go-review.googlesource.com/#/c/7692/
Suggested by iant@, we use the library initialization entry point to:
- create a new OS thread and run the "regular" runtime init stack on
that thread
- return immediately from the main (i.e., loader) thread
- at the first CGO invocation, we wait for the runtime initialization
to complete.
The above mechanism is implemented only on linux_amd64. Next step is to
support it on linux_arm. Other platforms don't yet support shared library
compiling/linking, but we intend to use the same strategy there as well.
Change-Id: Ib2c81b1b83bee837134084b75a3beecfb8de6bf4
Reviewed-on: https://go-review.googlesource.com/8094
Run-TryBot: Srdjan Petrovic <spetrovic@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This change makes TestDialTimeoutFDLeak work on almost all the supported
platforms.
Updates #4384.
Change-Id: I3608f438003003f9b7cfa17c9e5fe7077700fd60
Reviewed-on: https://go-review.googlesource.com/8392
Reviewed-by: Ian Lance Taylor <iant@golang.org>
A CSR containing challengePassword or unstructuredName Attributes
(included in default OpenSSL prompts) would break ASN.1 parsing.
This updates the parsing structures to allow but then ignore these
fields.
See this CFSSL issue: https://github.com/cloudflare/cfssl/issues/115
Change-Id: I26a3bf1794589d27e6e763da88ae32276f0170c7
Reviewed-on: https://go-review.googlesource.com/8160
Reviewed-by: Adam Langley <agl@golang.org>
It's a single, package-wide init function specified for the net package.
Change-Id: Id5894d65e1a92297cc16803cc5e4d4eef0b4b099
Reviewed-on: https://go-review.googlesource.com/8391
Reviewed-by: Ian Lance Taylor <iant@golang.org>
This tracks both total CPU time used by GC and the total time
available to all Ps since the beginning of the program and uses this
to derive a cumulative CPU usage percent for the gctrace line.
Change-Id: Ica85372b8dd45f7621909b325d5ac713a9b0d015
Reviewed-on: https://go-review.googlesource.com/8350
Reviewed-by: Russ Cox <rsc@golang.org>
GODEBUG=gctrace=1 turns on a per-GC cycle trace line. The current line
is left over from the STW garbage collector and includes a lot of
information that is no longer meaningful for the concurrent GC and
doesn't include a lot of information that is important.
Replace this line with a new line designed for the new garbage
collector.
This new line is focused more on helping the user understand the
impact of the garbage collector on their program and less on telling
us, the runtime developers, everything that's happening inside
GC. It's designed to fit in 80 columns and intentionally omit some
potentially useful things that were in the old line. We might want a
"verbose" mode that adds information for us.
We'll be able to further simplify the line once we eliminate the STW
around enabling the write barrier. Then we'll have just one STW phase,
one concurrent phase, and one more STW phase, so we'll be able to
reduce the number of times from five to three.
Change-Id: Icc30939fe4576fb4491b4eac811649395727aa2a
Reviewed-on: https://go-review.googlesource.com/8208
Reviewed-by: Russ Cox <rsc@golang.org>
This change replaces all direct ECDSA/RSA sign and decrypt operations
with calls through the crypto.Signer and crypto.Decrypter interfaces.
This is a follow-up to https://go-review.googlesource.com/#/c/3900/
which added crypto.Decrypter and implemented it for RSA.
Change-Id: Ie0f3928448b285f329efcd3a93ca3fd5e3b3e42d
Reviewed-on: https://go-review.googlesource.com/7804
Reviewed-by: Adam Langley <agl@golang.org>
All multi-precision arithmetic is now based on math/big.
- passes all.bash
- added test cases for fixed bugs
Fixes#7740.
Fixes#6866.
Change-Id: I67268b91766970ced3b928260053ccdce8753d58
Reviewed-on: https://go-review.googlesource.com/7912
Reviewed-by: Russ Cox <rsc@golang.org>
This restores go.errors from before 3af0d79 along with a fixed up
version of the bisonerrors AWK script, translated to Go.
However, this means Yyerror needs access to the yacc parser's state,
which is currently private. To workaround that, add a "state"
accessor method like the Lookahead method added in c7fa3c6.
Update issue #9968.
Change-Id: Ib868789e92fdb7d135442120a392457923e50121
Reviewed-on: https://go-review.googlesource.com/7270
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This is vendored copy of the pure-Go version of math/big.
To update, run vendor.bash in place.
This will permit the use of the new big.Float functionality in
gc (which is not available in 1.4, the version used for bootstrapping).
Change-Id: I4dcdea875d54710005ca3fdea2e0e30422b1b46d
Reviewed-on: https://go-review.googlesource.com/7857
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Yacc generates a bunch of global variables of the form
var yyFoo = []int{...}
where yyFoo is never subsequently modified to point to a different
slice. Since these variables are implicitly compiled as
var yyFoo = ([...]int{...})[:]
anyway, by simply converting them all to
var yyFoo = [...]int{...}
we save sizeof(sliceStruct) bytes of data memory for each variable and
also make len(yyFoo) into compile-time constant expressions, which
shaves some bytes off text size:
$ size 6g.before 6g.after
text data bss dec hex filename
4598019 605968 342700 5546687 54a2bf 6g.before
4597810 605552 342700 5546062 54a04e 6g.after
Change-Id: I53c7aa6efdb2d52738013e9d337a59afbfcb2494
Reviewed-on: https://go-review.googlesource.com/7520
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Russ Cox <rsc@golang.org>
This change cleans up some of the uglyness introduced in 8fc73a39ef
by moving the gc.Use_sse into the gc.Arch struct and adjusting its
zero value to be more useful.
Change-Id: I26ff5d9ac57b3f25e936519e443de6583cdafa56
Reviewed-on: https://go-review.googlesource.com/7994
Reviewed-by: Russ Cox <rsc@golang.org>
On arm64, CMP $foo, R is encoded as from=$foo, reg=R, not as from=$foo,
to=R. The progtable entry for ACMP incorrectly described the latter
form. Because of this, the registerizer was not accounting the registers
used in CMP instructions and was incorrectly re-assigning those registers.
This was an old problem, but it only became apparent after b115c35
(cmd/internal/gc: move cgen, regalloc, et al to portable code). Previous
to this commit, the compiler used a slightly larger register set for the
temps than it used for register variables. Since it had plenty registers
dedicated to temps, the registers used in CMP instruction never clashed
with registers assigned to register variables.
Fixes#10253
Change-Id: Iedf4bd882bd59440dff310ac0f81e0f53d80d7ed
Reviewed-on: https://go-review.googlesource.com/8387
Reviewed-by: Aram Hăvărneanu <aram@mgk.ro>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
Fixes#9053
Change-Id: I7b9af3647800671f065c366a6adf9278e64ebec9
Reviewed-on: https://go-review.googlesource.com/8400
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Currently hashmap is riddled with code that attempts to force a GC on
the next allocation if checkgc is set. This no longer works as
originally intended with the concurrent collector, and is apparently
no longer used anyway.
Remove checkgc.
Change-Id: Ia6c17c405fa8821dc2e6af28d506c1133ab1ca0c
Reviewed-on: https://go-review.googlesource.com/8355
Reviewed-by: Keith Randall <khr@golang.org>
This tries to clarify that Alloc and HeapAlloc are tied to how much
freeing has been done by the sweeper.
Change-Id: Id8320074bd75de791f39ec01bac99afe28052d02
Reviewed-on: https://go-review.googlesource.com/8354
Reviewed-by: Rick Hudson <rlh@golang.org>
