This CL refactors the existing listenerSockaddr function into several
methods on netFD.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave, alex.brainman, dvyukov, remyoudompheng
CC=golang-dev
https://golang.org/cl/12023043
Updates #6046.
This CL just does maxstring and concatstring. There are other functions
to fix but doing them a few at a time will help isolate any (unlikely)
breakages these changes bring up in architectures I can't test
myself.
R=golang-dev, dave, iant
CC=golang-dev
https://golang.org/cl/12519044
I broke it with the darwin getwd attrlist stuff (0583e9d36dd).
plan9 doesn't have syscall.ENOTSUP.
It's in api/go1.txt as a symbol always available (not context-specific):
pkg syscall, const ENOTSUP Errno
... but plan9 isn't considered by cmd/api, so it only looks
universally available. Alternatively, we could add a fake ENOTSUP
to plan9, but they were making efforts earlier to clean their
syscall package, so I'd prefer not to dump more in it.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12509044
This change replaces the hard-coded switch on compression method
in zipfile reader and writer with a map into which users can
register compressors and decompressors in their init()s.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/12421043
NetBSD and OpenBSD are broken like OS X is. Good to know.
Drop required count from avg/2 to avg/3, because the
Plan 9 builder just barely missed avg/2 in one of its runs.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/12548043
Looks like latest FreeBSD doesn't set address family identifer
for RTAX_NETMASK stuff; probably RTAX_GENMASK too, not confirmed.
This CL tries to identify address families by using the length of
each socket address if possible.
The issue is confirmed on FreeBSD 9.1.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12332043
Unlike the existing net package own pollster, runtime-integrated
network pollster on BSD variants, actually kqueue, requires a socket
that has beed passed to syscall.Listen previously for a stream
listener.
This CL separates pollDesc.Init (actually runtime_pollOpen) from newFD
to allow control of each state of sockets and adds init method to netFD
instead. Upcoming CLs will rearrange the call order of runtime-integrated
pollster and syscall functions like the following;
- For dialers that open active connections, runtime_pollOpen will be
called in between syscall.Bind and syscall.Connect.
- For stream listeners that open passive stream connections,
runtime_pollOpen will be called just after syscall.Listen.
- For datagram listeners that open datagram connections,
runtime_pollOpen will be called just after syscall.Bind.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=dvyukov, alex.brainman, minux.ma
CC=golang-dev
https://golang.org/cl/8608044
Update #6046.
This CL just does findnull and findnullw. There are other functions
to fix but doing them a few at a time will help isolate any (unlikely)
breakages these changes bring up in architectures I can't test
myself.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/12520043
Embed all data necessary for read/write operations directly into netFD.
benchmark old ns/op new ns/op delta
BenchmarkTCP4Persistent 27669 23341 -15.64%
BenchmarkTCP4Persistent-2 18173 12558 -30.90%
BenchmarkTCP4Persistent-4 10390 7319 -29.56%
This change will intentionally break all builders to see
how many allocations they do per read/write.
This will be fixed soon afterwards.
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/12413043
gcpc/gcsp are used by GC in similar situation.
gcpc/gcsp are also more stable than gp->sched,
because gp->sched is mutated by entersyscall/exitsyscall
in morestack and mcall. So it has higher chances of being inconsistent.
Also, rename gcpc/gcsp to syscallpc/syscallsp.
This is the same as reverted change 12250043
with save marked as textflag 7.
The problem was that if save calls morestack,
then subsequent lessstack spoils g->sched.pc/sp.
And that bad values were remembered in g->syscallpc/sp.
Entersyscallblock had the same problem,
but it was never triggered to date.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12478043
Basically a partial rollback of 12053043 until I can
figure out what is really going on.
Fixes bug 6051.
R=golang-dev
CC=golang-dev
https://golang.org/cl/12496043
This means that pprof will no longer report profiles on OS X.
That's unfortunate, but the profiles were often wrong and, worse,
it was difficult to tell whether the profile was wrong or not.
The workarounds were making the scheduler more complex,
possibly caused a deadlock (see issue 5519), and did not actually
deliver reliable results.
It may be possible for adventurous users to apply a patch to
their kernels to get working results, or perhaps having no results
will encourage someone to do the work of creating a profiling
thread like on Windows. Issue 6047 has details.
Fixes#5519.
Fixes#6047.
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/12429045
This means that in the common case (modern kernel), we only
make 1 system call to dup instead of two, and we also avoid
grabbing the syscall.ForkLock.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/12476043
you do reflect.call with too big an argument list.
Not worth the hassle.
Fixes#6023Fixes#6033
R=golang-dev, bradfitz, dave
CC=golang-dev
https://golang.org/cl/12485043
While we're here, add a test for the same functionality in gzip,
which was already implemented, and add bzip2 CRC checks.
Fixes#5772.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/12387044
Break all 386 builders.
««« original CL description
runtime: use gcpc/gcsp during traceback of goroutines in syscalls
gcpc/gcsp are used by GC in similar situation.
gcpc/gcsp are also more stable than gp->sched,
because gp->sched is mutated by entersyscall/exitsyscall
in morestack and mcall. So it has higher chances of being inconsistent.
Also, rename gcpc/gcsp to syscallpc/syscallsp.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12250043
»»»
R=rsc
CC=golang-dev
https://golang.org/cl/12424045
It was needed for the old scheduler,
because there temporary could be more threads than gomaxprocs.
In the new scheduler gomaxprocs is always respected.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12438043
gcpc/gcsp are used by GC in similar situation.
gcpc/gcsp are also more stable than gp->sched,
because gp->sched is mutated by entersyscall/exitsyscall
in morestack and mcall. So it has higher chances of being inconsistent.
Also, rename gcpc/gcsp to syscallpc/syscallsp.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12250043
In the event that code tries to use a hash function that isn't compiled
in and panics, give the developer a fighting chance of figuring out
which hash function it needed.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12420045
Runtime netpoll supports at most one read waiter
and at most one write waiter. It's responsibility
of net package to ensure that. Currently windows
implementation allows more than one waiter in Accept.
It leads to "fatal error: netpollblock: double wait".
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12400045
Whether the keys are concatenated or separate (or a mixture) depends on the server.
Fixes#5979.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12433043
Windows dynamic priority boosting assumes that a process has different types
of dedicated threads -- GUI, IO, computational, etc. Go processes use
equivalent threads that all do a mix of GUI, IO, computations, etc.
In such context dynamic priority boosting does nothing but harm, so turn it off.
In particular, if 2 goroutines do heavy IO on a server uniprocessor machine,
windows rejects to schedule timer thread for 2+ seconds when priority boosting is enabled.
Fixes#5971.
R=alex.brainman
CC=golang-dev
https://golang.org/cl/12406043
The test isn't checking deliberate panics so catching them just makes the code longer.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/12420043
This CL makes IPAddr, UDPAddr and TCPAddr implement sockaddr
interface, UnixAddr is already sockaddr interface compliant, and
reduces unnecessary conversions between net.Addr, net.sockaddr and
syscall.Sockaddr.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave, bradfitz
CC=golang-dev
https://golang.org/cl/12010043
It's okay to preempt at ordinary function calls because
compilers arrange that there are no live registers to save
on entry to the function call.
The software floating point routines are function calls
masquerading as individual machine instructions. They are
expected to keep all the registers intact. In particular,
they are expected not to clobber all the floating point
registers.
The floating point registers are kept per-M, because they
are not live at non-preemptive goroutine scheduling events,
and so keeping them per-M reduces the number of 132-byte
register blocks we are keeping in memory.
Because they are per-M, allowing the goroutine to be
rescheduled during software floating point simulation
would mean some other goroutine could overwrite the registers
or perhaps the goroutine would continue running on a different
M entirely.
Disallow preemption during the software floating point
routines to make sure that a function full of floating point
instructions has the same floating point registers throughout
its execution.
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/12298043
Per suggestion from Russ in February. Then strings.IndexByte
can be implemented in terms of the shared code in pkg runtime.
Update #3751
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/12289043
I used just enough of the data provided by Matt in Issue 5915 to trigger
issue 5915. As luck would have it, using slightly less of it triggered
issue 5962.
Fixes#5915.
Fixes#5962.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12288043
This allows to at least determine goroutine "identity".
Now it looks like:
goroutine 12 [running]:
goroutine running on other thread; stack unavailable
created by testing.RunTests
src/pkg/testing/testing.go:440 +0x88e
R=golang-dev, r, rsc
CC=golang-dev
https://golang.org/cl/12248043
We see timeouts in these tests on some platforms,
but not on the others. The hypothesis is that
the problematic platforms are slow uniprocessors.
Stack traces do not suggest that the process
is completely hang, and it is able to schedule
the alarm goroutine. And if it actually hangs,
we still will be able to detect that.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/12253043
With dup3, we can avoid an extra system call on some machines
while holding syscall.ForkLock. Currently we have to
syscall.Dup + syscall.CloseOnExec.
On machines with Linux and a new enough kernel, this can just
be dup3.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/12170045
Preemption during the software floating point code
could cause m (R9) to change, so that when the
original registers were restored at the end of the
floating point handler, the changed and correct m
would be replaced by the old and incorrect m.
TBR=dvyukov
CC=golang-dev
https://golang.org/cl/11883045
Operations on int64 are very stack consuming with 5c.
Fixes netbsd/arm build.
Before: TEXT runtime.timediv+0(SB),7,$52-16
After: TEXT runtime.timediv+0(SB),7,$44-16
The stack usage is unchanged on 386:
TEXT runtime.timediv+0(SB),7,$8-16
R=golang-dev, dvyukov, bradfitz
CC=golang-dev
https://golang.org/cl/12182044
This patch introduces specialized functions for initial
and final permutations, and precomputes the output of the
third permutation on the S-box elements.
benchmark old ns/op new ns/op delta
BenchmarkEncrypt 3581 1226 -65.76%
BenchmarkDecrypt 3590 1224 -65.91%
benchmark old MB/s new MB/s speedup
BenchmarkEncrypt 2.23 6.52 2.92x
BenchmarkDecrypt 2.23 6.53 2.93x
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12072045
Update #5139.
Double wakeup on Note was reported several times,
but no reliable reproducer.
There also was a strange report about weird value of epoll fd.
Maybe it's corruption of global data...
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/12182043
Sysmon thread parks if no goroutines are running (runtime.sched.npidle ==
runtime.gomaxprocs).
Currently it's unparked when a goroutine enters syscall, it was enough
to retake P's from blocking syscalls.
But it's not enough for reliable goroutine preemption. We need to ensure that
sysmon runs if any goroutines are running.
R=rsc
CC=golang-dev
https://golang.org/cl/12176043
Submitted with some unrelated changes that were not intended to go in.
««« original CL description
runtime: do not park sysmon thread if any goroutines are running
Sysmon thread parks if no goroutines are running (runtime.sched.npidle == runtime.gomaxprocs).
Currently it's unparked when a goroutine enters syscall, it was enough
to retake P's from blocking syscalls.
But it's not enough for reliable goroutine preemption. We need to ensure that
sysmon runs if any goroutines are running.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12167043
»»»
R=rsc
CC=golang-dev
https://golang.org/cl/12171044
This is required to properly unwind reflect.methodValueCall/makeFuncStub.
Fixes#5954.
Stats for 'go install std':
61849 total INSTCALL
24655 currently have ArgSize metadata
27278 have ArgSize metadata with this change
godoc size before: 11351888, after: 11364288
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12163043
Sysmon thread parks if no goroutines are running (runtime.sched.npidle == runtime.gomaxprocs).
Currently it's unparked when a goroutine enters syscall, it was enough
to retake P's from blocking syscalls.
But it's not enough for reliable goroutine preemption. We need to ensure that
sysmon runs if any goroutines are running.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12167043
Adds layout cases with seconds for stdISO8601 and stdNumTZ with and without colons. Update time.Format to append seconds for those cases.
Fixes#4934.
R=golang-dev, r, bradfitz
CC=golang-dev
https://golang.org/cl/8132044
This means that printing a Node will produce output that can be used as valid input.
It won't be exactly the same - some spacing may be different - but it will mean the same.
Fixes#4593.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/12006047
When scanning input and "white space" is permitted, a carriage return
followed immediately by a newline (\r\n) is treated exactly the same
as a plain newline (\n). I hope this makes it work better on Windows.
We do it everywhere, not just on Windows, since why not?
Fixes#5391.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/12142043
When comparing strings, check these (in order):
- length mismatch => not equal
- string pointer equal => equal
- if length is short:
- memeq on body
- if length is long:
- compare first&last few bytes, if different => not equal
- save entry as a possible match
- after checking every entry, if there is only one possible
match, use memeq on that entry. Otherwise, fallback to hash.
benchmark old ns/op new ns/op delta
BenchmarkSameLengthMap 43 4 -89.77%
Fixes#5194.
Update #3885.
R=golang-dev, bradfitz, khr, rsc
CC=golang-dev
https://golang.org/cl/12128044
The prefix was not uniformly applied and is probably better left off anyway.
Fixes#4944.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/12140043
struct Hmap is the header for a map value.
CL 8377046 made flags a uint32 so that it could be updated atomically,
but that bumped the struct to 56 bytes, which allocates as 64 bytes (on amd64).
hash0 is initialized from runtime.fastrand1, which returns a uint32,
so the top 32 bits were always zero anyway. Declare it as a uint32
to reclaim 4 bytes and bring the Hmap size back down to a 48-byte allocation.
Fixes#5237.
R=golang-dev, khr, khr
CC=bradfitz, dvyukov, golang-dev
https://golang.org/cl/12034047
If netFD is closed by finalizer, runtime netpoll descriptor is not freed.
R=golang-dev, dave, alex.brainman
CC=golang-dev
https://golang.org/cl/12037043
EscapeText now escapes 0xFFFD returned from DecodeRune as 0xFFFD, rather than passing through the original byte.
Fixes#5880.
R=golang-dev, r, bradfitz, adg
CC=golang-dev
https://golang.org/cl/11975043
notetsleep: nosplit stack overflow
120 assumed on entry to notetsleep
96 after notetsleep uses 24
88 on entry to runtime.semasleep
32 after runtime.semasleep uses 56
24 on entry to runtime.nanotime
-8 after runtime.nanotime uses 32
Nanotime seems to be using only 24 bytes of stack space.
Unless I am missing something.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12041044
notetsleep: nosplit stack overflow
120 assumed on entry to notetsleep
80 after notetsleep uses 40
72 on entry to runtime.futexsleep
16 after runtime.futexsleep uses 56
8 on entry to runtime.printf
-16 after runtime.printf uses 24
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/12047043
Split stack checks (morestack) corrupt g->sched,
but g->sched must be preserved consistent for GC/traceback.
The change implements runtime.notetsleepg function,
which does entersyscall/exitsyscall and is carefully arranged
to not call any split functions in between.
R=rsc
CC=golang-dev
https://golang.org/cl/11575044
Close netpoll descriptor along with socket.
Ensure that error paths close the descriptor as well.
R=golang-dev, mikioh.mikioh, alex.brainman
CC=golang-dev
https://golang.org/cl/11987043
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, fvbommel, dave
CC=golang-dev
https://golang.org/cl/11984043
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11932044
This CL extends existing sockaddr interface to accommodate not only
internet protocol family endpoint addressess but unix network family
endpoint addresses.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11979043
If netpoll has been told to block, it must not return with nil,
otherwise scheduler assumes that netpoll is disabled.
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/11920044
Make it accept type, combine flags.
Several reasons for the change:
1. mallocgc and settype must be atomic wrt GC
2. settype is called from only one place now
3. it will help performance (eventually settype
functionality must be combined with markallocated)
4. flags are easier to read now (no mallocgc(sz, 0, 1, 0) anymore)
R=golang-dev, iant, nightlyone, rsc, dave, khr, bradfitz, r
CC=golang-dev
https://golang.org/cl/10136043
The rlimit arguments for prlimit are reversed for linux 32-bit (386 and arm).
Getrlimit becomes Setrlimit and vice versa.
Fixes#5949.
R=iant, mikioh.mikioh, rsc
CC=golang-dev
https://golang.org/cl/11803043
An ARM version of md5block.go with a big improvement in
throughput (up to 2.5x) and a reduction in object size (21%).
Code size
Before 3100 bytes
After 2424 bytes
21% smaller
Benchmarks on Rasperry Pi
benchmark old ns/op new ns/op delta
BenchmarkHash8Bytes 11703 6636 -43.30%
BenchmarkHash1K 38057 21881 -42.50%
BenchmarkHash8K 208131 142735 -31.42%
BenchmarkHash8BytesUnaligned 11457 6570 -42.66%
BenchmarkHash1KUnaligned 69334 26841 -61.29%
BenchmarkHash8KUnaligned 455120 182223 -59.96%
benchmark old MB/s new MB/s speedup
BenchmarkHash8Bytes 0.68 1.21 1.78x
BenchmarkHash1K 26.91 46.80 1.74x
BenchmarkHash8K 39.36 57.39 1.46x
BenchmarkHash8BytesUnaligned 0.70 1.22 1.74x
BenchmarkHash1KUnaligned 14.77 38.15 2.58x
BenchmarkHash8KUnaligned 18.00 44.96 2.50x
benchmark old allocs new allocs delta
BenchmarkHash8Bytes 1 0 -100.00%
BenchmarkHash1K 2 0 -100.00%
BenchmarkHash8K 2 0 -100.00%
BenchmarkHash8BytesUnaligned 1 0 -100.00%
BenchmarkHash1KUnaligned 2 0 -100.00%
BenchmarkHash8KUnaligned 2 0 -100.00%
benchmark old bytes new bytes delta
BenchmarkHash8Bytes 64 0 -100.00%
BenchmarkHash1K 128 0 -100.00%
BenchmarkHash8K 128 0 -100.00%
BenchmarkHash8BytesUnaligned 64 0 -100.00%
BenchmarkHash1KUnaligned 128 0 -100.00%
BenchmarkHash8KUnaligned 128 0 -100.00%
This also adds another test which makes sure that the sums
over larger blocks work properly. I wrote this test when I was
worried about memory corruption.
R=golang-dev, dave, bradfitz, rsc, ajstarks
CC=golang-dev, minux.ma, remyoudompheng
https://golang.org/cl/11648043
Revision 15629 (8d71734a0cb0) removed the serverConn interface
that was introduce in revision 7718 (ee5e80c62862). The
serverConn interface was there for use by gccgo on Solaris,
and it is still needed there. Solaris does not support
connecting to the syslog daemon over TCP, and gccgo simply
calls the C library function. This CL restores the
interface.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/11737043
Currently Darwin and FreeBSD support and NetBSD and OpenBSD do not
support EV_RECEIPT flag. We will drop use of EV_RECEIPT for now.
Also enables to build runtime-integrated network pollster on
freebsd/amd64,386 and openbsd/amd64,386. It just does build but never
runs pollster stuff.
This is in preparation for runtime-integrated network pollster for BSD
variants.
Update #5199
R=dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/11759044
Debugging the Windows breakage I noticed that SEH
only exists on 386, so we can balance the two stacks
a little more on amd64 and reclaim another word.
Now we're down to just one word consumed by
cgocallback_gofunc, having reclaimed 25% of the
overall budget (4 words out of 16).
Separately, fix windows/386 - the SEH must be on the
m0 stack, as must the saved SP, so we are forced to have
a three-word frame for 386. It matters much less for
386, because there 128 bytes gives 32 words to use.
R=dvyukov, alex.brainman
CC=golang-dev
https://golang.org/cl/11551044
The existing function, IsOneOf, is hard to use. Since the slice comes
before the rune, in parallelism with the other Is functions, the slice
is clumsy to build. This CL adds a nicer-signatured In function of
equivalent functionality (its implementation is identical) that's much
easier to use. Compare:
unicode.IsOneOf([]*unicode.RangeTable{unicode.Letter, unicode.Number}, r)
unicode.In(r, unicode.Letter, unicode.Number)
R=golang-dev, adg
CC=golang-dev
https://golang.org/cl/11672044
Tying preemption to stack splits means that we have to able to
complete the call to exitsyscall (inside cgocallbackg at least for now)
without any stack split checks, meaning that the whole sequence
has to work within 128 bytes of stack, unless we increase the size
of the red zone. This CL frees up 24 bytes along that critical path
on amd64. (The 32-bit systems have plenty of space because all
their words are smaller.)
R=dvyukov
CC=golang-dev
https://golang.org/cl/11676043
When making an HTTPS client request, respect the
ServerName field in the tls.Config.
Fixes#5829
R=golang-dev, agl, adg
CC=golang-dev
https://golang.org/cl/11691043
Phrases like "returns whether or not the image is opaque" could be
describing what the function does (it always returns, regardless of
the opacity) or what it returns (a boolean indicating the opacity).
Even when the "or not" is missing, the phrasing is bizarre.
Go with "reports whether", which is still clunky but at least makes
it clear we're talking about the return value.
These were edited by hand. A few were cleaned up in other ways.
R=golang-dev, dsymonds
CC=golang-dev
https://golang.org/cl/11699043
Change use of x+(SP) to access the stack frame into x-(SP)
Fixes#5925.
R=golang-dev, bradfitz, dave, remyoudompheng, nick, rsc
CC=dave cheney <dave, golang-dev
https://golang.org/cl/11647043
notetsleepg is the same as notetsleep, but is called on user g.
It includes entersyscall/exitsyscall and will help to avoid
split stack functions in syscall status.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/11681043
Calls into math/rand are very slow, especially under race
detector because of heap accesses.
go test -bench . -run none -benchtime .1s
Before: 23.0s
After: 17.4s
Fixes#5837.
R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/11564044
This CL introduces a FUNCDATA number for runtime-specific
garbage collection metadata, changes the C and Go compilers
to emit that metadata, and changes the runtime to expect it.
The old pseudo-instructions that carried this information
are gone, as is the linker code to process them.
R=golang-dev, dvyukov, cshapiro
CC=golang-dev
https://golang.org/cl/11406044
whose argument size is unknown (C vararg functions, and
assembly code without an explicit specification).
We used to use 0 to mean "unknown" and 1 to mean "zero".
Now we use ArgsSizeUnknown (0x80000000) to mean "unknown".
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/11590043
If the network is not polled for 10ms, sysmon starts polling network
on every iteration (every 20us) until another thread blocks in netpoll.
Fixes#5922.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11569043
It is an expensive test to run, and even more so with -race,
and causes timeouts on builders. It is doubtful that it would
find a race that other tests in this package wouldn't, so there
is little loss in excluding it.
Update #5837.
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11568043
It assumes that the m will not change, and the m may
change if the goroutine is preempted.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/11560043
If we start a garbage collection on g0 during a
stack split or unsplit, we'll see morestack or lessstack
at the top of the stack. Record an argument frame size
for those, and record that they terminate the stack.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11533043
Deferreturn is synthesizing a new call frame.
It must not be interrupted between copying the args there
and fixing up the program counter, or else the stack will
be in an inconsistent state, one that will confuse the
garbage collector.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11522043
With preemption, _sfloat2 can show up in stack traces.
Write the function prototype in a way that accurately
shows the frame size and the fact that it might contain
pointers.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11523043
Windows was the only one seeing this bug reliably in the builder,
but it was easy to reproduce using 'GOGC=1 go test strconv'.
concatstring looked like it took only one string, but in fact it
takes a long list of strings. Add an explicit ... so that the traceback
will not use the "fixed" frame size and instead look at the
frame size metadata recorded by the caller.
R=golang-dev
TBR=golang-dev
CC=golang-dev
https://golang.org/cl/11531043
Be consistent with os.File, strings.Reader, bytes.Reader, etc,
which all allow seeks past the end.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/11403043
Otherwise the tests in pkg/runtime fail:
runtime: unknown argument frame size for runtime.deferreturn called from 0x48657b [runtime_test.func·022]
fatal error: invalid stack
...
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/11483043
Update #543
I believe the runtime is strong enough now to reenable
preemption during the function prologue.
Assuming this is or can be made stable, it will be in Go 1.2.
More aggressive preemption is not planned for Go 1.2.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/11433045
Currently preemption signal g->stackguard0==StackPreempt
can be lost if it is received when preemption is disabled
(e.g. m->lock!=0). This change duplicates the preemption
signal in g->preempt and restores g->stackguard0
when preemption is enabled.
Update #543.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10792043
With this CL, I believe the runtime always knows
the frame size during the gc walk. There is no fallback
to "assume entire stack frame of caller" anymore.
R=golang-dev, khr, cshapiro, dvyukov
CC=golang-dev
https://golang.org/cl/11374044
Add support for ECDHE-ECDSA (RFC4492), which uses an ephemeral server
key pair to perform ECDH with ECDSA signatures. Like ECDHE-RSA,
ECDHE-ECDSA also provides PFS.
R=agl
CC=golang-dev
https://golang.org/cl/7006047
I have not done the system call stubs in sys_*.s.
I hope to avoid that, because those do not block, so those
frames will not appear in stack traces during garbage
collection.
R=golang-dev, dvyukov, khr
CC=golang-dev
https://golang.org/cl/11360043
While we're here, fix Syscall9 on NetBSD and OpenBSD:
it was storing the results into the wrong memory locations.
I guess no one uses that function's results on those systems.
Part of cleaning up stack traces and argument frame information.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11355044
Design at http://golang.org/s/go12symtab.
This enables some cleanup of the garbage collector metadata
that will be done in future CLs.
This CL does not move the old symtab and pclntab back into
an unmapped section of the file. That's a bit tricky and will be
done separately.
Fixes#4020.
R=golang-dev, dave, cshapiro, iant, r
CC=golang-dev, nigeltao
https://golang.org/cl/11085043
Race instrumentation can allocate, switch stacks, preempt, etc.
All that is not allowed in between fork and exec.
Fixes#4840.
R=golang-dev, daniel.morsing, dave
CC=golang-dev
https://golang.org/cl/11324044
A type switch on a value with map index expressions,
could get a spurious instrumentation from a OTYPESW node.
These nodes do not need instrumentation because after
walk the type switch has been turned into a sequence
of ifs.
Fixes#5890.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/11308043
Sets both the duration from the last data packet to the first
keep alive packet and the duration between keep alive packets to be
the passed duration.
I've tested the function on both Darwin (10.8.4) and 4.2 Linux.
I've compiled (make.bash) for all the OS's and tested (all.bash)
on Darwin and Linux.
R=golang-dev, dave, rsc, dvyukov, presotto+facebook, nick
CC=golang-dev, veyron-team
https://golang.org/cl/11130044
Before:
$ go test -c -cover fmt
$ ./fmt.test -test.covermode=set
PASS
coverage: 65.1% of statements in strconv
$
After:
$ go test -c -cover fmt
$ ./fmt.test
PASS
coverage: 65.1% of statements in strconv
$
In addition to being cumbersome, the old flag didn't make sense:
the cover mode cannot be changed after the binary has been built.
Another useful effect of this CL is that if you happen to do
$ go test -c -covermode=atomic fmt
and then forget you did that and run benchmarks,
the final line of the output (the coverage summary) reminds you
that you are benchmarking with coverage enabled, which might
not be what you want.
$ ./fmt.test -test.bench .
PASS
BenchmarkSprintfEmpty 10000000 217 ns/op
BenchmarkSprintfString 2000000 755 ns/op
BenchmarkSprintfInt 2000000 774 ns/op
BenchmarkSprintfIntInt 1000000 1363 ns/op
BenchmarkSprintfPrefixedInt 1000000 1501 ns/op
BenchmarkSprintfFloat 1000000 1257 ns/op
BenchmarkManyArgs 500000 5346 ns/op
BenchmarkScanInts 1000 2562402 ns/op
BenchmarkScanRecursiveInt 500 3189457 ns/op
coverage: 91.4% of statements
$
As part of passing the new mode setting in via _testmain.go, merge
the two registration mechanisms into one extensible mechanism
(a struct).
R=r
CC=golang-dev
https://golang.org/cl/11219043
I want to think more carefully about this.
We put this in because Marshal encoded named []byte but Unmarshal rejected them.
And we noticed that Marshal's behavior was undocumented so we documented it.
But I am starting to think the docs and Unmarshal were correct and Marshal's
behavior was the problem.
Rolling back to give us more time to think.
««« original CL description
json: unmarshal types that are byte slices.
The json package cheerfully would marshal
type S struct {
IP net.IP
}
but would give an error when unmarshalling. This change allows any
type whose concrete type is a byte slice to be unmarshalled from a
string.
Fixes#5086.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/11161044
»»»
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/11042046
In practice, rejecting an entire structure due to a single invalid byte
in a string is just too picky, and too hard to track down.
Be consistent with the bulk of the standard library by converting
invalid UTF-8 into UTF-8 with replacement runes.
R=golang-dev, crawshaw
CC=golang-dev
https://golang.org/cl/11211045
If the stack frame size is larger than the known-unmapped region at the
bottom of the address space, then the stack split prologue cannot use the usual
condition:
SP - size >= stackguard
because SP - size may wrap around to a very large number.
Instead, if the stack frame is large, the prologue tests:
SP - stackguard >= size
(This ends up being a few instructions more expensive, so we don't do it always.)
Preemption requests register by setting stackguard to a very large value, so
that the first test (SP - size >= stackguard) cannot possibly succeed.
Unfortunately, that same very large value causes a wraparound in the
second test (SP - stackguard >= size), making it succeed incorrectly.
To avoid *that* wraparound, we have to amend the test:
stackguard != StackPreempt && SP - stackguard >= size
This test is only used for functions with large frames, which essentially
always split the stack, so the cost of the few instructions is noise.
This CL and CL 11085043 together fix the known issues with preemption,
at the beginning of a function, so we will be able to try turning it on again.
R=ken2
CC=golang-dev
https://golang.org/cl/11205043
This is a transcript before this change. I've capitalized the text being removed.
Note that it is always near another line that already says fmt, marked with <<<
$ cd $GOROOT/src/pkg/fmt
$ go test -cover
PASS
coverage FOR FMT: 91.3% of statements
ok fmt 0.040s <<<
$ go test -coverpkg strconv
PASS
coverage FOR FMT: 64.9% of statements in strconv
ok fmt 0.039s <<<
$ go test -cover -c
$ ./fmt.test -test.covermode=set <<<
PASS
coverage FOR FMT: 91.3% of statements
$ go test -coverpkg strconv -c
$ ./fmt.test -test.covermode=set <<<
PASS
coverage FOR FMT: 64.9% of statements in strconv
That the summary printed by 'go test [options] fmt' is unchanged:
$ go test -cover fmt
ok fmt 0.040s coverage: 91.3% of statements
$ go test -coverpkg strconv fmt
ok fmt 0.038s coverage: 64.9% of statements in strconv
R=r
CC=gobot, golang-dev
https://golang.org/cl/10932045
The current cas64 definition hard-codes the x86 behavior
of updating *old with the new value when the cas fails.
This is inconsistent with cas32 and casp.
Make it consistent.
This means that the cas64 uses will be epsilon less efficient
than they might be, because they have to do an unnecessary
memory load on x86. But so be it. Code clarity and consistency
is more important.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10909045
The json package cheerfully would marshal
type S struct {
IP net.IP
}
but would give an error when unmarshalling. This change allows any
type whose concrete type is a byte slice to be unmarshalled from a
string.
Fixes#5086.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/11161044
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
This is reincarnation of cl/9776044 with the bug fixed.
The bug was due to code added after cl/9776044 was created:
if(tick - (((uint64)tick*0x4325c53fu)>>36)*61 == 0 && runtime·sched.runqsize > 0) {
runtime·lock(&runtime·sched);
gp = globrunqget(m->p, 1);
runtime·unlock(&runtime·sched);
}
If M gets gp from global runq here, it does not reset m->spinning.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10743044
Currently it crashes as follows:
fatal error: unknown pc
...
goroutine 71698 [runnable]:
runtime.racegoend()
src/pkg/runtime/race.c:171
runtime.goexit()
src/pkg/runtime/proc.c:1276 +0x9
created by runtime_test.testConcurrentReadsAfterGrowth
src/pkg/runtime/map_test.go:264 +0x332
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10674047
Merging a couple of CLs into one, since they collided in my client
and I'm lazy.
1) Fix up output in "go test -cover" case.
We need to tell the testing package the name of the package being tested
and the name of the package being covered. It can then sort out the report.
2) Filter out the _test.go files from coverage processing. We want to measure
what the tests cover, not what's covered in the tests,
The coverage for encoding/gob goes from 82.2% to 88.4%.
There may be a cleaner way to do this - suggestions welcome - but ça suffit.
Fixes#5810.
R=rsc
CC=golang-dev
https://golang.org/cl/10868047
Part 3 of several.
* Linux has grown a SetsockoptByte.
* SetsockoptIPMreqn is handled directly by syscall_linux.go and syscall_freebsd.go.
R=golang-dev, mikioh.mikioh, r, bradfitz
CC=golang-dev
https://golang.org/cl/10775043
The lzw.NewReader doc comment says, "It is the caller's responsibility
to call Close on the ReadCloser when finished reading."
Thanks to Andrew Bonventre for noticing this.
R=r, dsymonds, adg
CC=andybons, golang-dev
https://golang.org/cl/10821043
ReadMIMEHeader is used by net/http, net/mail, and
mime/multipart.
Don't do so many small allocations. Calculate up front
how much we'll probably need.
benchmark old ns/op new ns/op delta
BenchmarkReadMIMEHeader 8433 7467 -11.45%
benchmark old allocs new allocs delta
BenchmarkReadMIMEHeader 23 14 -39.13%
benchmark old bytes new bytes delta
BenchmarkReadMIMEHeader 1705 1343 -21.23%
R=golang-dev, r, iant, adg
CC=golang-dev
https://golang.org/cl/8179043
This does not include AES-GCM yet. Also, it assumes that the handshake and
certificate signature hash are always SHA-256, which is true of the ciphersuites
that we currently support.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10762044
using m->tls[0] to save ucontext pointer is not re-entry safe, and
the old code didn't set it before the early return when signal is
received on non-Go threads.
so misc/cgo/test used to hang when testing issue 5337.
R=golang-dev, bradfitz, rsc
CC=golang-dev
https://golang.org/cl/10076045
When deleting a timer, a panic due to nil deref
would leave a lock held, possibly leading to a deadlock
in a defer. Instead return false on a nil timer.
Fixes#5745.
R=golang-dev, daniel.morsing, dvyukov, rsc, iant
CC=golang-dev
https://golang.org/cl/10373047
This CL provides stable in-place sorting by use of
bottom up merge sort with in-place merging done by
the SymMerge algorithm from P.-S. Kim and A. Kutzner.
The additional space needed for stable sorting (in the form of
stack space) is logarithmic in the inputs size n.
Number of calls to Less and Swap grow like O(n * log n) and
O(n * log n * log n):
Stable sorting random data uses significantly more calls
to Swap than the unstable quicksort implementation (5 times more
on n=100, 10 times more on n=1e4 and 23 times more on n=1e8).
The number of calls to Less is practically the same for Sort and
Stable.
Stable sorting 1 million random integers takes 5 times longer
than using Sort.
BenchmarkSortString1K 50000 328662 ns/op
BenchmarkStableString1K 50000 380231 ns/op 1.15 slower
BenchmarkSortInt1K 50000 157336 ns/op
BenchmarkStableInt1K 50000 191167 ns/op 1.22 slower
BenchmarkSortInt64K 1000 14466297 ns/op
BenchmarkStableInt64K 500 16190266 ns/op 1.12 slower
BenchmarkSort1e2 200000 64923 ns/op
BenchmarkStable1e2 50000 167128 ns/op 2.57 slower
BenchmarkSort1e4 1000 14540613 ns/op
BenchmarkStable1e4 100 58117289 ns/op 4.00 slower
BenchmarkSort1e6 5 2429631508 ns/op
BenchmarkStable1e6 1 12077036952 ns/op 4.97 slower
R=golang-dev, bradfitz, iant, 0xjnml, rsc
CC=golang-dev
https://golang.org/cl/9612044
There are various problems, and both Dmitriy and I
will be away for the next week. Make the runtime a bit
more stable while we're gone.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10848043
fn can clearly hold a closure in memory.
argp/pc point into stack and so can hold
in memory a block that was previously
a large stack serment.
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/10784043
Depending on net/http means depending on cgo.
When the tree is in a shaky state it's nice to see sync/atomic
pass even if cgo or net causes broken binaries.
R=golang-dev, dave, r
CC=golang-dev
https://golang.org/cl/10753044
After loading a frame of a GIF, check that each pixel
is inside the frame's palette.
Fixes#5401.
R=nigeltao, r
CC=golang-dev
https://golang.org/cl/10597043
On amd64 the frames are very close to the limit for a
nosplit (textflag 7) function, in part because the C compiler
does not make any attempt to reclaim space allocated for
completely registerized variables. Avoid a few short-lived
variables to reclaim two words.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10758043
Currently it replaces GOGCTRACE env var (GODEBUG=gctrace=1).
The plan is to extend it with other type of debug tracing,
e.g. GODEBUG=gctrace=1,schedtrace=100.
R=rsc
CC=bradfitz, daniel.morsing, gobot, golang-dev
https://golang.org/cl/10026045
The last patch for preemptive scheduler,
with this change stoptheworld issues preemption
requests every 100us.
Update #543.
R=golang-dev, daniel.morsing, rsc
CC=golang-dev
https://golang.org/cl/10264044
On x86 it is a few words lower on the stack than m->morebuf.sp
so it is a more precise check. Enabling the check requires recording
a valid gp->sched in reflect.call too. This is a good thing in general,
since it will make stack traces during reflect.call work better, and it
may be useful for preemption too.
R=dvyukov
CC=golang-dev
https://golang.org/cl/10709043
runtime.entersyscall() sets g->status = Gsyscall,
then calls runtime.lock() which causes stack split.
runtime.newstack() resets g->status to Grunning.
This will lead to crash during GC (world is not stopped) or GC will scan stack incorrectly.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10696043
Also use 2048-bit RSA keys as default in generate_cert.go,
as recommended by the NIST.
R=golang-dev, rsc, bradfitz
CC=golang-dev
https://golang.org/cl/10676043
On my 64-bit machine, despite being 32-bit code, fixed-base
multiplications are 7.1x faster and arbitary multiplications are 2.6x
faster.
It is difficult to review this change. However, the code is essentially
the same as code that has been open-sourced in Chromium. There it has
been successfully performing P-256 operations for several months on
many machines so the arithmetic of the code should be sound.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10551044
Failure on bot:
http://build.golang.org/log/f4c648906e1289ec2237c1d0880fb1a8b1852a08
««« original CL description
runtime: fix CPU underutilization
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
R=rsc
TBR=rsc
CC=gobot, golang-dev
https://golang.org/cl/9776044
»»»
R=golang-dev
CC=golang-dev
https://golang.org/cl/10692043
runtime.newproc/ready are deliberately sloppy about waking new M's,
they only ensure that there is at least 1 spinning M.
Currently to compensate for that, schedule() checks if the current P
has local work and there are no spinning M's, it wakes up another one.
It does not work if goroutines do not call schedule.
With this change a spinning M wakes up another M when it finds work to do.
It's also not ideal, but it fixes the underutilization.
A proper check would require to know the exact number of runnable G's,
but it's too expensive to maintain.
Fixes#5586.
R=rsc
CC=gobot, golang-dev
https://golang.org/cl/9776044
Current code can print more arguments than necessary
and also incorrectly prints "...".
Update #5723.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10689043
Until now, the goroutine state has been scattered during the
execution of newstack and oldstack. It's all there, and those routines
know how to get back to a working goroutine, but other pieces of
the system, like stack traces, do not. If something does interrupt
the newstack or oldstack execution, the rest of the system can't
understand the goroutine. For example, if newstack decides there
is an overflow and calls throw, the stack tracer wouldn't dump the
goroutine correctly.
For newstack to save a useful state snapshot, it needs to be able
to rewind the PC in the function that triggered the split back to
the beginning of the function. (The PC is a few instructions in, just
after the call to morestack.) To make that possible, we change the
prologues to insert a jmp back to the beginning of the function
after the call to morestack. That is, the prologue used to be roughly:
TEXT myfunc
check for split
jmpcond nosplit
call morestack
nosplit:
sub $xxx, sp
Now an extra instruction is inserted after the call:
TEXT myfunc
start:
check for split
jmpcond nosplit
call morestack
jmp start
nosplit:
sub $xxx, sp
The jmp is not executed directly. It is decoded and simulated by
runtime.rewindmorestack to discover the beginning of the function,
and then the call to morestack returns directly to the start label
instead of to the jump instruction. So logically the jmp is still
executed, just not by the cpu.
The prologue thus repeats in the case of a function that needs a
stack split, but against the cost of the split itself, the extra few
instructions are noise. The repeated prologue has the nice effect of
making a stack split double-check that the new stack is big enough:
if morestack happens to return on a too-small stack, we'll now notice
before corruption happens.
The ability for newstack to rewind to the beginning of the function
should help preemption too. If newstack decides that it was called
for preemption instead of a stack split, it now has the goroutine state
correctly paused if rescheduling is needed, and when the goroutine
can run again, it can return to the start label on its original stack
and re-execute the split check.
Here is an example of a split stack overflow showing the full
trace, without any special cases in the stack printer.
(This one was triggered by making the split check incorrect.)
runtime: newstack framesize=0x0 argsize=0x18 sp=0x6aebd0 stack=[0x6b0000, 0x6b0fa0]
morebuf={pc:0x69f5b sp:0x6aebd8 lr:0x0}
sched={pc:0x68880 sp:0x6aebd0 lr:0x0 ctxt:0x34e700}
runtime: split stack overflow: 0x6aebd0 < 0x6b0000
fatal error: runtime: split stack overflow
goroutine 1 [stack split]:
runtime.mallocgc(0x290, 0x100000000, 0x1)
/Users/rsc/g/go/src/pkg/runtime/zmalloc_darwin_amd64.c:21 fp=0x6aebd8
runtime.new()
/Users/rsc/g/go/src/pkg/runtime/zmalloc_darwin_amd64.c:682 +0x5b fp=0x6aec08
go/build.(*Context).Import(0x5ae340, 0xc210030c71, 0xa, 0xc2100b4380, 0x1b, ...)
/Users/rsc/g/go/src/pkg/go/build/build.go:424 +0x3a fp=0x6b00a0
main.loadImport(0xc210030c71, 0xa, 0xc2100b4380, 0x1b, 0xc2100b42c0, ...)
/Users/rsc/g/go/src/cmd/go/pkg.go:249 +0x371 fp=0x6b01a8
main.(*Package).load(0xc21017c800, 0xc2100b42c0, 0xc2101828c0, 0x0, 0x0, ...)
/Users/rsc/g/go/src/cmd/go/pkg.go:431 +0x2801 fp=0x6b0c98
main.loadPackage(0x369040, 0x7, 0xc2100b42c0, 0x0)
/Users/rsc/g/go/src/cmd/go/pkg.go:709 +0x857 fp=0x6b0f80
----- stack segment boundary -----
main.(*builder).action(0xc2100902a0, 0x0, 0x0, 0xc2100e6c00, 0xc2100e5750, ...)
/Users/rsc/g/go/src/cmd/go/build.go:539 +0x437 fp=0x6b14a0
main.(*builder).action(0xc2100902a0, 0x0, 0x0, 0xc21015b400, 0x2, ...)
/Users/rsc/g/go/src/cmd/go/build.go:528 +0x1d2 fp=0x6b1658
main.(*builder).test(0xc2100902a0, 0xc210092000, 0x0, 0x0, 0xc21008ff60, ...)
/Users/rsc/g/go/src/cmd/go/test.go:622 +0x1b53 fp=0x6b1f68
----- stack segment boundary -----
main.runTest(0x5a6b20, 0xc21000a020, 0x2, 0x2)
/Users/rsc/g/go/src/cmd/go/test.go:366 +0xd09 fp=0x6a5cf0
main.main()
/Users/rsc/g/go/src/cmd/go/main.go:161 +0x4f9 fp=0x6a5f78
runtime.main()
/Users/rsc/g/go/src/pkg/runtime/proc.c:183 +0x92 fp=0x6a5fa0
runtime.goexit()
/Users/rsc/g/go/src/pkg/runtime/proc.c:1266 fp=0x6a5fa8
And here is a seg fault during oldstack:
SIGSEGV: segmentation violation
PC=0x1b2a6
runtime.oldstack()
/Users/rsc/g/go/src/pkg/runtime/stack.c:159 +0x76
runtime.lessstack()
/Users/rsc/g/go/src/pkg/runtime/asm_amd64.s:270 +0x22
goroutine 1 [stack unsplit]:
fmt.(*pp).printArg(0x2102e64e0, 0xe5c80, 0x2102c9220, 0x73, 0x0, ...)
/Users/rsc/g/go/src/pkg/fmt/print.go:818 +0x3d3 fp=0x221031e6f8
fmt.(*pp).doPrintf(0x2102e64e0, 0x12fb20, 0x2, 0x221031eb98, 0x1, ...)
/Users/rsc/g/go/src/pkg/fmt/print.go:1183 +0x15cb fp=0x221031eaf0
fmt.Sprintf(0x12fb20, 0x2, 0x221031eb98, 0x1, 0x1, ...)
/Users/rsc/g/go/src/pkg/fmt/print.go:234 +0x67 fp=0x221031eb40
flag.(*stringValue).String(0x2102c9210, 0x1, 0x0)
/Users/rsc/g/go/src/pkg/flag/flag.go:180 +0xb3 fp=0x221031ebb0
flag.(*FlagSet).Var(0x2102f6000, 0x293d38, 0x2102c9210, 0x143490, 0xa, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:633 +0x40 fp=0x221031eca0
flag.(*FlagSet).StringVar(0x2102f6000, 0x2102c9210, 0x143490, 0xa, 0x12fa60, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:550 +0x91 fp=0x221031ece8
flag.(*FlagSet).String(0x2102f6000, 0x143490, 0xa, 0x12fa60, 0x0, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:563 +0x87 fp=0x221031ed38
flag.String(0x143490, 0xa, 0x12fa60, 0x0, 0x161950, ...)
/Users/rsc/g/go/src/pkg/flag/flag.go:570 +0x6b fp=0x221031ed80
testing.init()
/Users/rsc/g/go/src/pkg/testing/testing.go:-531 +0xbb fp=0x221031edc0
strings_test.init()
/Users/rsc/g/go/src/pkg/strings/strings_test.go:1115 +0x62 fp=0x221031ef70
main.init()
strings/_test/_testmain.go:90 +0x3d fp=0x221031ef78
runtime.main()
/Users/rsc/g/go/src/pkg/runtime/proc.c:180 +0x8a fp=0x221031efa0
runtime.goexit()
/Users/rsc/g/go/src/pkg/runtime/proc.c:1269 fp=0x221031efa8
goroutine 2 [runnable]:
runtime.MHeap_Scavenger()
/Users/rsc/g/go/src/pkg/runtime/mheap.c:438
runtime.goexit()
/Users/rsc/g/go/src/pkg/runtime/proc.c:1269
created by runtime.main
/Users/rsc/g/go/src/pkg/runtime/proc.c:166
rax 0x23ccc0
rbx 0x23ccc0
rcx 0x0
rdx 0x38
rdi 0x2102c0170
rsi 0x221032cfe0
rbp 0x221032cfa0
rsp 0x7fff5fbff5b0
r8 0x2102c0120
r9 0x221032cfa0
r10 0x221032c000
r11 0x104ce8
r12 0xe5c80
r13 0x1be82baac718
r14 0x13091135f7d69200
r15 0x0
rip 0x1b2a6
rflags 0x10246
cs 0x2b
fs 0x0
gs 0x0
Fixes#5723.
R=r, dvyukov, go.peter.90, dave, iant
CC=golang-dev
https://golang.org/cl/10360048
Setenv("AN_ENV_VAR", "") deletes AN_ENV_VAR instead of setting it
to "" at this moment. Also Getenv("AN_ENV_VAR") returns "not found",
if AN_ENV_VAR is "". Change it, so they behave like unix.
Fixes#5610
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10594043
Makes it easy to ask the simple question, what is the hash of this data?
Also fix the commentary and prints in Sum256.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10630043
Makes it easy to ask the simple question, what is the hash of this data?
Also mark block as non-escaping.
R=golang-dev, agl
CC=golang-dev
https://golang.org/cl/10624044
Currently more than 1 gorutine can execute raceWrite() in Wait()
in the following scenario:
1. goroutine 1 executes first check of wg.counter, sees that it's == 0
2. goroutine 2 executes first check of wg.counter, sees that it's == 0
3. goroutine 2 locks the mutex, sees that he is the first waiter and executes raceWrite()
4. goroutine 2 block on the semaphore
5. goroutine 3 executes Done() and unblocks goroutine 2
6. goroutine 1 lock the mutex, sees that he is the first waiter and executes raceWrite()
It produces the following false report:
WARNING: DATA RACE
Write by goroutine 35:
sync.raceWrite()
src/pkg/sync/race.go:41 +0x33
sync.(*WaitGroup).Wait()
src/pkg/sync/waitgroup.go:103 +0xae
command-line-arguments_test.TestNoRaceWaitGroupMultipleWait2()
src/pkg/runtime/race/testdata/waitgroup_test.go:156 +0x19a
testing.tRunner()
src/pkg/testing/testing.go:361 +0x108
Previous write by goroutine 36:
sync.raceWrite()
src/pkg/sync/race.go:41 +0x33
sync.(*WaitGroup).Wait()
src/pkg/sync/waitgroup.go:103 +0xae
command-line-arguments_test.func·012()
src/pkg/runtime/race/testdata/waitgroup_test.go:148 +0x4d
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10424043
Resubmit 3c2cddfbdaec now that windows callbacks
are not generated during runtime.
Fixes#5494
R=golang-dev, minux.ma, rsc
CC=golang-dev
https://golang.org/cl/10487043
- change runtime_pollWait so it does not return
closed or timeout if IO is ready - windows must
know if IO has completed or not even after
interruption;
- add (*pollDesc).Prepare(mode int) that can be
used for both read and write, same for Wait;
- introduce runtime_pollWaitCanceled and expose
it in net as (*pollDesc).WaitCanceled(mode int);
Full windows netpoll changes are
here https://golang.org/cl/8670044/.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/10485043
The compiler still gets the escape analysis wrong, but the annotation here is correct.
R=golang-dev, dave, bradfitz
CC=golang-dev
https://golang.org/cl/10514046
No semantic change.
I found the wording distracting in a couple of instances and was moved to improve it.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/10478048
If a server response contains a Content-Length and the body is short,
the Transport should end in io.ErrUnexpectedEOF, not io.EOF.
Fixes#5738
R=golang-dev, kevlar, r
CC=golang-dev
https://golang.org/cl/10237050
until we decide what to do with issues 5659/5736.
Profiling with race detector is not very useful in general,
and now it makes race builders red.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10523043
Time is a tiny struct, so the compiler copies a Time by
copying each of the three fields.
The layout of a time on amd64 is [ptr int32 gap32 ptr].
Copying a Time onto a location that formerly held a pointer in the
second word changes only the low 32 bits, creating a different
but still plausible pointer. This confuses the garbage collector
when it appears in argument or result frames.
To avoid this problem, declare nsec as uintptr, so that there is
no gap on amd64 anymore, and therefore no partial pointers.
Note that rearranging the fields to put the int32 last still leaves
a gap - [ptr ptr int32 gap32] - because Time must have a total
size that is ptr-width aligned.
Update #5749
This CL is enough to fix the problem, but we should still do
the other actions listed in the initial report. We're not too far
from completely precise collection.
R=golang-dev, dvyukov, r
CC=golang-dev
https://golang.org/cl/10504043
The tradition is to use _posix when the platform extends beyond unix variants. As windows has its own file, rename to the more usual _unix.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10320043
If time.Sub results in a value that won't fit in a Duration (int64),
return either the min or max int64 value as appropriate.
Fixes#5011.
R=golang-dev, bradfitz, r, rsc
CC=golang-dev
https://golang.org/cl/10328043
trivial: it is not a serious problem to leak a fd in a short lived process, but it was obscuring my investigation of issue 5593.
R=golang-dev, iant, bradfitz
CC=golang-dev
https://golang.org/cl/10391043
The old code worked, somewhat on accident, but was confusing,
and had a useless assignment to the inner err. It worked
because url.Parse parses just about anything, so the outer err
was always nil, so it always fell through to the bottom return
statement, even without the "err = nil" line.
Instead, just have two return statements, and add a comment.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/10448044
TLS clients send ciphersuites in preference order (most prefereable
first). This change alters the order so that ECDHE comes before plain
RSA, and RC4 comes before AES (because of the Lucky13 attack).
This is unlikely to have much effect: as a server, the code uses the
client's ciphersuite order by default and, as a client, the non-Go
server probably imposes its order.
R=golang-dev, r, raggi, jsing
CC=golang-dev
https://golang.org/cl/10372045
The single flag -cover provides the default simplest behavior.
The other flags, -covermode and -coverprofile, provide more
control. The three flags interconnect to work well.
R=rsc, adg
CC=golang-dev
https://golang.org/cl/10364044
Move the data dumper to the testing package, where it has access
to file I/O.
Print a percentage value at the end of the run.
R=rsc, adg
CC=golang-dev
https://golang.org/cl/10264045
use correct field count when resolving nameservers via /net/dns on Plan 9.
we incorrectly check for 4 fields instead of 3 when parsing the result of /net/dns, and get no results
R=golang-dev, ality
CC=golang-dev
https://golang.org/cl/10182044
Don't panic when the underlying Writer isn't a Closer. And
document what Close does and clarify that it's not a Flush.
R=golang-dev, agl
CC=golang-dev
https://golang.org/cl/10310043
If first GC runs concurrently with setGCPercent,
it can overwrite gcpercent value with default.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/10242047
Currently global runqueue is starved if a group of goroutines
constantly respawn each other (local runqueue never becomes empty).
Fixes#5639.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/10042044
It was used to request large stack segment for GC
when it was running not on g0.
Now GC is running on g0 with large stack,
and it is not needed anymore.
R=golang-dev, dave
CC=golang-dev
https://golang.org/cl/10242045
No need to change to Grunnable state.
Add some more checks for Grunning state.
R=golang-dev, rsc, khr, dvyukov
CC=golang-dev
https://golang.org/cl/10186045
In Issue 5625, Russ says: "We should at least have a cache of
inflight lookups, so that 100 simultaneous dials of one host
name don't do the work 100x. That's easy and (assume we forget
the answer once they all get it) doesn't pose any consistency
problems. It just merges simultaneous work."
This brings in singleflight (unexported) from Google /
Camlistore, but without its tests. Maybe we should put it
somewhere in the standard library. But not now.
Update #5625
R=golang-dev, iant, cespare, rsc, dave, rogpeppe, remyoudompheng
CC=golang-dev
https://golang.org/cl/10079043
The previous implementation would only record access to
the address of the array but the memory access to the whole
memory range must be recorded instead.
R=golang-dev, dvyukov, r
CC=golang-dev
https://golang.org/cl/8053044
These are required to correctly determine the End() of the node.
Also set these fields in go/parser and use them in go/printer.
This is a backward-compatible API change.
R=gri, r
CC=golang-dev
https://golang.org/cl/10189043
Instrumentation of ntest expression should go to ntest->init.
Same for nincr.
Fixes#5340.
R=golang-dev, daniel.morsing
CC=golang-dev
https://golang.org/cl/10026046
Obscure misfeature now fixed: When run from "go test", profiles were always
written in the package's source directory. This change puts them in the directory
where "go test" is run.
Also fix a couple of problems causing errors in testing.after to go unreported
unless -v was set.
R=rsc, minux.ma, iant, alex.brainman
CC=golang-dev
https://golang.org/cl/10234044
Add gostartcall and gostartcallfn.
The old gogocall = gostartcall + gogo.
The old gogocallfn = gostartcallfn + gogo.
R=dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/10036044
In starttheworld() we assume that P's with local work
are situated in the beginning of idle P list.
However, once we start the first M, it can execute all local G's
and steal G's from other P's.
That breaks the assumption above. Thus starttheworld() will fail
to start some P's with local work.
It seems that it can not lead to very bad things, but still
it's wrong and breaks other assumtions
(e.g. we can have a spinning M with local work).
The fix is to collect all P's with local work first,
and only then start them.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/10051045
The garbage collection routine addframeroots is duplicating
logic in the traceback routine that calls it, sometimes correctly,
sometimes incorrectly, sometimes incompletely.
Pass necessary information to addframeroots instead of
deriving it anew.
Should make addframeroots significantly more robust.
It's certainly smaller.
Also try to standardize on uintptr for saved pc, sp values.
Will make CL 10036044 trivial.
R=golang-dev, dave, dvyukov
CC=golang-dev
https://golang.org/cl/10169045
There's no reason to use a different name on each architecture,
and doing so makes it impossible for portable code to refer to
the original Go runtime entry point. Rename it _rt0_go everywhere.
This is a global search and replace only.
R=golang-dev, bradfitz, minux.ma
CC=golang-dev
https://golang.org/cl/10196043
The first identifier in an Object Identifer must be between 0 and 2
inclusive. The range of values that the second one can take depends
on the value of the first one.
The two first identifiers are not necessarily encoded in a single octet,
but in a varint.
R=golang-dev, agl
CC=golang-dev
https://golang.org/cl/10140046
Do not synchronize Add(1) with Wait().
Imitate read on first Add(1) and write on Wait(),
it allows to catch common misuses of WaitGroup:
- Add() called in the additional goroutine itself
- incorrect reuse of WaitGroup with multiple waiters
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/10093044
Also reduce FixAlloc allocation granulatiry from 128k to 16k,
small programs do not need that much memory for MCache's and MSpan's.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/10140044
Especially important for Windows because it reserves VM
only in multiple of 64k.
R=golang-dev, alex.brainman
CC=golang-dev
https://golang.org/cl/10082048
Count only number of frees, everything else is derivable
and does not need to be counted on every malloc.
benchmark old ns/op new ns/op delta
BenchmarkMalloc8 68 66 -3.07%
BenchmarkMalloc16 75 70 -6.48%
BenchmarkMallocTypeInfo8 102 97 -4.80%
BenchmarkMallocTypeInfo16 108 105 -2.78%
R=golang-dev, dave, rsc
CC=golang-dev
https://golang.org/cl/9776043
CFLAGS comes before CPPFLAGS.
Also fix one typo CPPCFLAGS.
Cleanup for CL 8248043.
R=golang-dev, iant, alberto.garcia.hierro
CC=golang-dev
https://golang.org/cl/9965045
The significant change between TLS 1.0 and 1.1 is the addition of an explicit IV in the case of CBC encrypted records. Support for TLS 1.1 is needed in order to support TLS 1.2.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/7880043
Remove unnecessary ( ) around == in && clause.
Add { } around multiline if body, even though it's one statement.
Add runtime: prefix to printed errors.
R=cshapiro, iant
CC=golang-dev
https://golang.org/cl/9685047
This is part of preemptive scheduler.
stackguard0 is checked in split stack checks and can be set to StackPreempt.
stackguard is not set to StackPreempt (holds the original value).
R=golang-dev, daniel.morsing, iant
CC=golang-dev
https://golang.org/cl/9875043
Fixes#5599.
Thanks to minux.ma for the suggested fix.
As we now have a harness to test testing internal functions I added some coverage for testing.roundUp, as it is the main consumer of roundDown10.
R=minux.ma, kr, r
CC=golang-dev
https://golang.org/cl/9926043
Before this change, grow work was done only
during map writes to ensure multithreaded safety.
This can lead to maps remaining in a partially
grown state for a long time, potentially forever.
This change allows grow work to happen during reads,
which will lead to grow work finishing sooner, making
the resulting map smaller and faster.
Grow work is not done in parallel. Reads can
happen in parallel while grow work is happening.
R=golang-dev, dvyukov, khr, iant
CC=golang-dev
https://golang.org/cl/8852047
instead of regular g stack. We do this so that the g stack
we're currently running on is no longer changing. Cuts
the root set down a bit (g0 stacks are not scanned, and
we don't need to scan gc's internal state). Also an
enabler for copyable stacks.
R=golang-dev, cshapiro, khr, 0xe2.0x9a.0x9b, dvyukov, rsc, iant
CC=golang-dev
https://golang.org/cl/9754044
* Add a CXXFiles field to Package, which includes .cc, .cpp and .cxx files.
* CXXFiles are compiled using g++, which can be overridden using the CXX environment variable.
* Include .hh, .hpp and .hxx files in HFiles.
* Add support for CPPFLAGS (used for both C and C++) and CXXFLAGS (used only for C++) in cgo directive.
* Changed pkg-config cgo directive to modify CPPFLAGS rather than CFLAGS, so both C and C++ files get any flag returned by pkg-config --cflags.
Fixes#1476.
R=iant, r
CC=bradfitz, gobot, golang-dev, iant, minux.ma, remyoudompheng, seb.binet
https://golang.org/cl/8248043
mheap.map become a pointer, so nelem(h->map) returns 1 rather than the map size.
As the result coalescing with subsequent spans does not happen.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9649046
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
Reincarnation of committed and rolled back https://golang.org/cl/9805043
The latent bugs that it revealed are fixed:
https://golang.org/cl/9837049https://golang.org/cl/9778048
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9778049
Then use the limit to make sure MHeap_LookupMaybe & inlined
copies don't return a span if the pointer is beyond the limit.
Use this fact to optimize all call sites.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/9869045
As the code now says:
We are forced to return a float64 because the API is silly, but do
the division as integers so we can ask if AllocsPerRun()==1
instead of AllocsPerRun()<2.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/9837049
A nosplits was assumed to have no argument information and no
pointer map. However, nosplits created by the linker often
have both. This change uses the pointer map size as an
alternate source of argument size when processing a nosplit.
In addition, the symbol table construction pointer map size
and argument size consistency check is strengthened. If a
nptrs is greater than 0 it must be equal to the number of
argument words.
R=golang-dev, khr, khr
CC=golang-dev
https://golang.org/cl/9666047
to avoid unintentionally clobber R9/R10.
Thanks Lucio for the suggestion.
PS: yes, this could be considered a big change (but not an API change), but
as it turns out even temporarily changes R9/R10 in user code is unsafe and
leads to very hard to diagnose problems later, better to disable using R9/R10
when the user first uses it.
See CL 6300043 and CL 6305100 for two problems caused by misusing R9/R10.
R=golang-dev, khr, rsc
CC=golang-dev
https://golang.org/cl/9840043
The old code put the index before the period in the precision;
it should be after so it's always before the star, as documented.
A little trickier to do in one pass but compensated for by more
tests and catching a couple of other error cases.
R=rsc
CC=golang-dev
https://golang.org/cl/9751044
Currently we only check the leaf node's issuer against the list of
distinguished names in the server's CertificateRequest message. This
will fail if the client certiciate has more than one certificate in
the path and the leaf node issuer isn't in the list of distinguished
names, but the issuer's issuer was in the distinguished names.
R=agl, agl
CC=gobot, golang-dev
https://golang.org/cl/9795043
This is needed for preemptive scheduler, because during
stoptheworld we want to wait with timeout and re-preempt
M's on timeout.
R=golang-dev, remyoudompheng, iant
CC=golang-dev
https://golang.org/cl/9375043
With this change the compiler emits a bitmap for each function
covering its stack frame arguments area. If an argument word
is known to contain a pointer, a bit is set. The garbage
collector reads this information when scanning the stack by
frames and uses it to ignores locations known to not contain a
pointer.
R=golang-dev, bradfitz, daniel.morsing, dvyukov, khr, khr, iant, cshapiro
CC=golang-dev
https://golang.org/cl/9223046
This depends on: 9791044: runtime: allocate page table lazily
Once page table is moved out of heap, the heap becomes small.
This removes unnecessary dereferences during heap access.
No logical changes.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9802043
This removes the 256MB memory allocation at startup,
which conflicts with ulimit.
Also will allow to eliminate an unnecessary memory dereference in GC,
because the page table is usually mapped at known address.
Update #5049.
Update #5236.
R=golang-dev, khr, r, khr, rsc
CC=golang-dev
https://golang.org/cl/9791044
Currently the test closes random files descriptors,
which leads to hang (in particular if netpoll fd is closed).
Try to open only fd 3, since the parent process expects it to be fd 3 anyway.
Fixes#5571.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9778048
The 'n' variable is used during rescan initiation in GC_END case,
but it's overwritten with chan capacity in GC_CHAN case.
As the result rescan is done with the wrong object size.
Fixes#5554.
R=golang-dev, khr
CC=golang-dev
https://golang.org/cl/9831043
multiple failures on amd64
««« original CL description
runtime: introduce helper persistentalloc() function
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
R=golang-dev, daniel.morsing, khr
CC=golang-dev
https://golang.org/cl/9805043
»»»
R=golang-dev
CC=golang-dev
https://golang.org/cl/9822043
It is a caching wrapper around SysAlloc() that can allocate small chunks.
Use it for symtab allocations. Reduces number of symtab walks from 4 to 3
(reduces buildfuncs time from 10ms to 7.5ms on a large binary,
reduces initial heap size by 680K on the same binary).
Also can be used for type info allocation, itab allocation.
There are also several places in GC where we do the same thing,
they can be changed to use persistentalloc().
Also can be used in FixAlloc, because each instance of FixAlloc allocates
in 128K regions, which is too eager.
R=golang-dev, daniel.morsing, khr
CC=golang-dev
https://golang.org/cl/9805043
Variables in data sections of 32-bit executables interfere with
garbage collector's ability to free objects and/or unnecessarily
slow down the garbage collector.
This changeset moves some static variables to .noptr sections.
'files' in symtab.c is now allocated dynamically.
R=golang-dev, dvyukov, minux.ma
CC=golang-dev
https://golang.org/cl/9786044
This text is added to doc.go:
Explicit argument indexes:
In Printf, Sprintf, and Fprintf, the default behavior is for each
formatting verb to format successive arguments passed in the call.
However, the notation [n] immediately before the verb indicates that the
nth one-indexed argument is to be formatted instead. The same notation
before a '*' for a width or precision selects the argument index holding
the value. After processing a bracketed expression [n], arguments n+1,
n+2, etc. will be processed unless otherwise directed.
For example,
fmt.Sprintf("%[2]d %[1]d\n", 11, 22)
will yield "22, 11", while
fmt.Sprintf("%[3]*[2].*[1]f", 12.0, 2, 6),
equivalent to
fmt.Sprintf("%6.2f", 12.0),
will yield " 12.00". Because an explicit index affects subsequent verbs,
this notation can be used to print the same values multiple times
by resetting the index for the first argument to be repeated:
fmt.Sprintf("%d %d %#[1]x %#x", 16, 17)
will yield "16 17 0x10 0x11".
The notation chosen differs from that in C, but I believe it's easier to read
and to remember (we're indexing the arguments), and compatibility with
C's printf was never a strong goal anyway.
While we're here, change the word "field" to "arg" or "argument" in the
code; it was being misused and was confusing.
R=rsc, bradfitz, rogpeppe, minux.ma, peter.armitage
CC=golang-dev
https://golang.org/cl/9680043
crypto/x509 has ended up with a variety of error formats. This change makes them all start with "x509: ".
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9736043
According to X.690, only 0 and 255 are allowed as values
for encoded booleans. Also added some test for parsing
booleans
R=golang-dev, agl, r
CC=golang-dev
https://golang.org/cl/9692043
The problem was that server handlers block on done<-,
the goroutine that reads from done blocks on count<-,
and the main goroutine that is supposed to read from count
waits for server handlers to exit.
Fixes#5547.
R=golang-dev, dave, bradfitz
CC=golang-dev
https://golang.org/cl/9722043
This only affects calls where both ReaderFrom and WriterTo are implemented. WriterTo can issue one large write, while ReaderFrom must Read until EOF, potentially reallocating when out of memory. With one large Write, the Writer only needs to allocate once.
This also helps in ioutil.Discard since we can avoid copying memory when the Reader implements WriterTo.
R=golang-dev, dsymonds, remyoudompheng, bradfitz
CC=golang-dev, minux.ma
https://golang.org/cl/9462044
This change contains an implementation of the RSASSA-PSS signature
algorithm described in RFC 3447.
R=agl, agl
CC=gobot, golang-dev, r
https://golang.org/cl/9438043
This CL adds missing IPv6 socket options which are required
to control IPv6 as described in RFC 3493, RFC 3542.
Update #5538
R=golang-dev, dave, iant
CC=golang-dev
https://golang.org/cl/9373046
1) go/doc:
- create correct ast.FuncType
- use more commonly used variable names in a test case
2) make ast.FuncType.Pos robust in case of incorrect ASTs
R=golang-dev
CC=golang-dev
https://golang.org/cl/9651044
This is needed for preemptive scheduler, because the goroutine
can be preempted at surprising points.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/9376043
When cgo is used, runtime creates an additional M to handle callbacks on threads not created by Go.
This effectively disabled deadlock detection, which is a right thing, because Go program can be blocked
and only serve callbacks on external threads.
This also disables deadlock detection under race detector, because it happens to use cgo.
With this change the additional M is created lazily on first cgo call. So deadlock detector
works for programs that import "C", "net" or "net/http/pprof" but do not use them in fact.
Also fixes deadlock detector under race detector.
It should be fine to create the M later, because C code can not call into Go before first cgo call,
because C code does not know when Go initialization has completed. So a Go program need to call into C
first either to create an external thread, or notify a thread created in global ctor that Go
initialization has completed.
Fixes#4973.
Fixes#5475.
R=golang-dev, minux.ma, iant
CC=golang-dev
https://golang.org/cl/9303046
The original code was correct. The count returned must be the length
of the input slice, not the length of the formatted message.
««« original CL description
log/syslog: report errors from Fprintf
Thanks to chiparus for identifying this.
Fixes#5541.
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/9658043
»»»
R=golang-dev, iant
CC=golang-dev
https://golang.org/cl/9644044
Currently per-sizeclass stats are lost for destroyed MCache's. This patch fixes this.
Also, only update mstats.heap_alloc on heap operations, because that's the only
stat that needs to be promptly updated. Everything else needs to be up-to-date only in ReadMemStats().
R=golang-dev, remyoudompheng, dave, iant
CC=golang-dev
https://golang.org/cl/9207047
The nlistmin/size thresholds are copied from tcmalloc,
but are unnecesary for Go malloc. We do not do explicit
frees into MCache. For sparse cases when we do (mainly hashmap),
simpler logic will do.
R=rsc, dave, iant
CC=gobot, golang-dev, r, remyoudompheng
https://golang.org/cl/9373043
A bufio.Writer.Flush marks the usual end of a Writer's
life. Recycle its internal buffer on those explicit flushes,
but not on normal, as-needed internal flushes.
benchmark old ns/op new ns/op delta
BenchmarkWriterEmpty 1959 727 -62.89%
benchmark old allocs new allocs delta
BenchmarkWriterEmpty 2 1 -50.00%
benchmark old bytes new bytes delta
BenchmarkWriterEmpty 4215 83 -98.03%
R=gri, iant
CC=gobot, golang-dev, voidlogic7
https://golang.org/cl/9459044
This should have been removed in 45c12efb4635. Not a correctness
issue, but unnecessary work.
This CL also adds paranoia checks in removeDep so this doesn't
happen again.
Fixes#5502
R=adg
CC=gobot, golang-dev, google
https://golang.org/cl/9543043
undo CL 8478044 / 0d28fd55e721
Lack of consensus.
««« original CL description
time: add Time.FormatAppend
This is a version of Time.Format that doesn't require allocation.
Fixes#5192
Update #5195
R=r
CC=gobot, golang-dev
https://golang.org/cl/8478044
»»»
R=r
CC=golang-dev
https://golang.org/cl/9462049
This isn't clearly a bug on Go's part, but it triggers a bug in Firefox
which means that crypto/tls and net/http cannot be wired up together
unless NextProtos includes "http/1.1". When net/http sets up the
tls.Config, it does this and so works fine. But anyone setting up the
tls.Config themselves will hit the Firefox bug.
Fixes#5445.
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/9539045
Several places used io.WriteString unnecessarily when the
static type already implemented WriteString. No need to
check for it at runtime.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9608043
It contains the LHS of the range clause and gets
instrumented by racewalk, but it doesn't have any meaning.
Fixes#5446.
R=golang-dev, dvyukov, daniel.morsing, r
CC=golang-dev
https://golang.org/cl/9560044
Failures caused by errors like invalid signatures or missing hash
functions cause rather generic, unhelpful error messages because no
trust chain can be constructed: "x509: certificate signed by unknown
authority."
With this change, authority errors may contain the reason why an
arbitary candidate step in the chain was rejected. For example, in the
event of a missing hash function the error looks like:
x509: certificate signed by unknown authority (possibly because of
"crypto/x509: cannot verify signature: algorithm unimplemented" while
trying to verify candidate authority certificate 'Thawte SGC CA')
Fixes 5058.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9104051
Share garbage between different bufio Readers. When a Reader
has zero buffered data, put its buffer into a pool.
This acknowledges that most bufio.Readers eventually get
read to completion, and their buffers are then no longer
needed.
benchmark old ns/op new ns/op delta
BenchmarkReaderEmpty 2993 1058 -64.65%
benchmark old allocs new allocs delta
BenchmarkReaderEmpty 3 2 -33.33%
benchmark old bytes new bytes delta
BenchmarkReaderEmpty 4278 133 -96.89%
Update #5100
R=r
CC=adg, dvyukov, gobot, golang-dev, rogpeppe
https://golang.org/cl/8819049
The stack scanner for not started goroutines ignored the arguments
area when its size was unknown. With this change, the distance
between the stack pointer and the stack base will be used instead.
Fixes#5486
R=golang-dev, bradfitz, iant, dvyukov
CC=golang-dev
https://golang.org/cl/9440043
A test added in b37d2fdcc4d9 didn't work with some values of GOMAXPROCS
because the defer statements were in the wrong order: the Pipe could be
closed before the TLS Client was.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/9187047
If a slice points to an array embedded in a struct,
the whole struct can be incorrectly scanned as the slice buffer.
Fixes#5443.
R=cshapiro, iant, r, cshapiro, minux.ma
CC=bradfitz, gobot, golang-dev
https://golang.org/cl/9372044
Allocs of size 16 can bypass atomic set of the allocated bit, while allocs of size 8 can not.
Allocs with and w/o type info hit different paths inside of malloc.
Current results on linux/amd64:
BenchmarkMalloc8 50000000 43.6 ns/op
BenchmarkMalloc16 50000000 46.7 ns/op
BenchmarkMallocTypeInfo8 50000000 61.3 ns/op
BenchmarkMallocTypeInfo16 50000000 63.5 ns/op
R=golang-dev, remyoudompheng, minux.ma, bradfitz, iant
CC=golang-dev
https://golang.org/cl/9090045
for checking for page boundary. Also avoid boundary check
when >=16 bytes are hashed.
benchmark old ns/op new ns/op delta
BenchmarkHashStringSpeed 23 22 -0.43%
BenchmarkHashBytesSpeed 44 42 -3.61%
BenchmarkHashStringArraySpeed 71 68 -4.05%
R=iant, khr
CC=gobot, golang-dev, google
https://golang.org/cl/9123046
Finer-grained transfers were relevant with per-M caches,
with per-P caches they are not relevant and harmful for performance.
For few small size classes where it makes difference,
it's fine to grab the whole span (4K).
benchmark old ns/op new ns/op delta
BenchmarkMalloc 42 40 -4.45%
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/9374043
The PKCS#1 spec requires that the PS padding in an RSA message be at
least 8 bytes long. We were not previously checking this. This isn't
important in the most common situation (session key encryption), but
the impact is unclear in other cases.
This change enforces the specified minimum size.
R=golang-dev, bradfitz
CC=golang-dev
https://golang.org/cl/9222045
OpenSSL can be configured to send empty records in order to randomise
the CBC IV. This is an early version of 1/n-1 record splitting (that Go
does) and is quite reasonable, but it results in tls.Conn.Read
returning (0, nil).
This change ignores up to 100 consecutive, empty records to avoid
returning (0, nil) to callers.
Fixes 5309.
R=golang-dev, r, minux.ma
CC=golang-dev
https://golang.org/cl/8852044
This patch resulted from a bit of quick optimisation in response to a
golang-nuts post. It looks like one could save a couple other copies in
this function, but this addresses the inner loop and is fairly simple.
benchmark old ns/op new ns/op delta
BenchmarkGCD10x10 1964 1711 -12.88%
BenchmarkGCD10x100 2019 1736 -14.02%
BenchmarkGCD10x1000 2471 2171 -12.14%
BenchmarkGCD10x10000 6040 5778 -4.34%
BenchmarkGCD10x100000 43204 43025 -0.41%
BenchmarkGCD100x100 11004 8520 -22.57%
BenchmarkGCD100x1000 11820 9446 -20.08%
BenchmarkGCD100x10000 23846 21382 -10.33%
BenchmarkGCD100x100000 133691 131505 -1.64%
BenchmarkGCD1000x1000 120041 95591 -20.37%
BenchmarkGCD1000x10000 136887 113600 -17.01%
BenchmarkGCD1000x100000 295370 273912 -7.26%
BenchmarkGCD10000x10000 2556126 2205198 -13.73%
BenchmarkGCD10000x100000 3159512 2808038 -11.12%
BenchmarkGCD100000x100000 150543094 139986045 -7.01%
R=gri, remyoudompheng
CC=bradfitz, gobot, golang-dev, gri
https://golang.org/cl/9424043
This is needed for preemptive scheduler,
it will preempt only when m->locks==0,
and we do not want to be preempted while
we have not completely unlocked the lock.
R=golang-dev, khr, iant
CC=golang-dev
https://golang.org/cl/9196047
Also change table type from int32[] to int8[] to save space in L1$.
benchmark old ns/op new ns/op delta
BenchmarkMalloc 42 40 -4.68%
R=golang-dev, bradfitz, r
CC=golang-dev
https://golang.org/cl/9199044
Trying to lookup user's display name with directory services can
take several seconds when user's computer is not in a domain.
As a workaround, check if computer is joined in a domain first,
and don't use directory services if it is not.
Additionally, don't leak tokens in user.Current().
Fixes#5298.
R=golang-dev, bradfitz, alex.brainman, lucio.dere
CC=golang-dev
https://golang.org/cl/8541047
The *Encoder is almost always garbage. It doesn't need an
encodeState inside of it (and its bytes.Buffer), since it's
only needed locally inside of Encode.
benchmark old ns/op new ns/op delta
BenchmarkEncoderEncode 2562 2553 -0.35%
benchmark old bytes new bytes delta
BenchmarkEncoderEncode 283 102 -63.96%
R=r
CC=gobot, golang-dev
https://golang.org/cl/9365044
Move the documentation from race.go to doc.go, because
race.go uses +build race, so it's not normally parsed by go doc.
Rephrase the documentation for end users, provide link to race
detector manual.
Fixes#5444.
R=golang-dev, minux.ma, adg, r
CC=golang-dev
https://golang.org/cl/9144050
runtime.park() can access freed select descriptor
due to a racing free in another thread.
See the comment for details.
Slightly modified version of dvyukov's CL 9259045.
No test yet. Before this CL, the test described in issue 5422
would fail about every 40 times for me. With this CL, I ran
the test 5900 times with no failures.
Fixes#5422.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9311043
The linker can generate split stack prolog when a textflag 7 function
makes an indirect function call. If it happens, badsignal() crashes
trying to dereference g.
Fixes#5337.
R=bradfitz, dave, adg, iant, r, minux.ma
CC=adonovan, golang-dev
https://golang.org/cl/9226043
This must have been from when "error" was a good variable
name for an "os.Error". But we use "err" these days.
R=golang-dev, dsymonds, r
CC=golang-dev
https://golang.org/cl/9132045
runtime.setmg() calls another function (cgo_save_gm), so it must save
LR onto stack.
Re-enabled TestCthread test in misc/cgo/test.
Fixes#4863.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/9019043
The refcounting of driver Conns was completedly busted and
would leak (be held open forever) with any reasonable
load. This was a significant regression from Go 1.0.
The core of this patch is removing one line:
s.db.addDep(dc, s)
A database conn (dc) is a resource that be re-created any time
(but cached for speed) should not be held open forever with a
dependency refcount just because the Stmt (s) is alive (which
typically last for long periods of time, like forever).
The meat of the patch is new tests. In fixing the real issue,
a lot of tests then failed due to the fakedb_test.go's paranoia
about closing a fakeConn while it has open fakeStmts on it. I
could've ignored that, but that's been a problem in the past for
other bugs.
Instead, I now track per-Conn open statements and close them
when the the conn closes. The proper way to do this would've
been making *driverStmt a finalCloser and using the dep mechanism,
but it was much more invasive. Added a TODO instead.
I'd like to give a way for drivers to opt-out of caring about
driver.Stmt closes before a driver.Conn close, but that's a TODO
for the future, and that TODO is added in this CL.
I know this is very late for Go 1.1, but database/sql is
currently nearly useless without this.
I'd like to believe all these database/sql bugs in the past
release cycle are the result of increased usage, number of
drivers, and good feedback from increasingly-capable Go
developers, and not the result of me sucking. It's also hard
with all the real drivers being out-of-tree, so I'm having to
add more and more hooks to fakedb_test.go to simulate things
which real drivers end up doing.
Fixes#5323
R=golang-dev, snaury, gwenn.kahz, google, r
CC=golang-dev
https://golang.org/cl/8836045
It works on i386, but fails on amd64 and arm.
««« original CL description
runtime: prevent the GC from seeing the content of a frame in runfinq()
Fixes#5348.
R=golang-dev, dvyukov
CC=golang-dev
https://golang.org/cl/8954044
»»»
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/8695051
