2014-06-17 00:03:03 -06:00
|
|
|
// Copyright 2014 The Go Authors. All rights reserved.
|
|
|
|
|
// Use of this source code is governed by a BSD-style
|
|
|
|
|
// license that can be found in the LICENSE file.
|
|
|
|
|
|
|
|
|
|
package runtime
|
|
|
|
|
|
|
|
|
|
import "unsafe"
|
|
|
|
|
|
|
|
|
|
// Declarations for runtime services implemented in C or assembly.
|
|
|
|
|
|
2014-08-29 14:20:48 -06:00
|
|
|
const ptrSize = 4 << (^uintptr(0) >> 63) // unsafe.Sizeof(uintptr(0)) but an ideal const
|
runtime: convert traceback*.c to Go
The two converted files were nearly identical.
Instead of continuing that duplication, I merged them
into a single traceback.go.
Tested on arm, amd64, amd64p32, and 386.
LGTM=r
R=golang-codereviews, remyoudompheng, dave, r
CC=dvyukov, golang-codereviews, iant, khr
https://golang.org/cl/134200044
2014-09-02 13:12:53 -06:00
|
|
|
const regSize = 4 << (^uintreg(0) >> 63) // unsafe.Sizeof(uintreg(0)) but an ideal const
|
2014-07-16 15:16:19 -06:00
|
|
|
|
|
|
|
|
// Should be a built-in for unsafe.Pointer?
|
2014-09-03 09:49:43 -06:00
|
|
|
//go:nosplit
|
2014-07-16 15:16:19 -06:00
|
|
|
func add(p unsafe.Pointer, x uintptr) unsafe.Pointer {
|
|
|
|
|
return unsafe.Pointer(uintptr(p) + x)
|
|
|
|
|
}
|
|
|
|
|
|
2014-08-21 10:41:09 -06:00
|
|
|
func getg() *g
|
2014-07-30 10:01:52 -06:00
|
|
|
|
runtime: reject onM calls from gsignal stack
The implementation and use patterns of onM assume
that they run on either the m->curg or m->g0 stack.
Calling onM from m->gsignal has two problems:
(1) When not on g0, onM switches to g0 and then "back" to curg.
If we didn't start at curg, bad things happen.
(2) The use of scalararg/ptrarg to pass C arguments and results
assumes that there is only one onM call at a time.
If a gsignal starts running, it may have interrupted the
setup/teardown of the args for an onM on the curg or g0 stack.
Using scalararg/ptrarg itself would smash those.
We can fix (1) by remembering what g was running before the switch.
We can fix (2) by requiring that uses of onM that might happen
on a signal handling stack must save the old scalararg/ptrarg
and restore them after the call, instead of zeroing them.
The only sane way to do this is to introduce a separate
onM_signalsafe that omits the signal check, and then if you
see a call to onM_signalsafe you know the surrounding code
must preserve the old scalararg/ptrarg values.
(The implementation would be that onM_signalsafe just calls
fn if on the signal stack or else jumps to onM. It's not necessary
to have two whole copies of the function.)
(2) is not a problem if the caller and callee are both Go and
a closure is used instead of the scalararg/ptrarg slots.
For now, I think we can avoid calling onM from code executing
on gsignal stacks, so just reject it.
In the long term, (2) goes away (as do the scalararg/ptrarg slots)
once everything is in Go, and at that point fixing (1) would be
trivial and maybe worth doing just for regularity.
LGTM=iant
R=golang-codereviews, iant
CC=dvyukov, golang-codereviews, khr, r
https://golang.org/cl/135400043
2014-09-03 22:10:10 -06:00
|
|
|
// mcall switches from the g to the g0 stack and invokes fn(g),
|
|
|
|
|
// where g is the goroutine that made the call.
|
|
|
|
|
// mcall saves g's current PC/SP in g->sched so that it can be restored later.
|
|
|
|
|
// It is up to fn to arrange for that later execution, typically by recording
|
|
|
|
|
// g in a data structure, causing something to call ready(g) later.
|
|
|
|
|
// mcall returns to the original goroutine g later, when g has been rescheduled.
|
|
|
|
|
// fn must not return at all; typically it ends by calling schedule, to let the m
|
|
|
|
|
// run other goroutines.
|
|
|
|
|
//
|
|
|
|
|
// mcall can only be called from g stacks (not g0, not gsignal).
|
|
|
|
|
//go:noescape
|
|
|
|
|
func mcall(fn func(*g))
|
|
|
|
|
|
[dev.cc] runtime: delete scalararg, ptrarg; rename onM to systemstack
Scalararg and ptrarg are not "signal safe".
Go code filling them out can be interrupted by a signal,
and then the signal handler runs, and if it also ends up
in Go code that uses scalararg or ptrarg, now the old
values have been smashed.
For the pieces of code that do need to run in a signal handler,
we introduced onM_signalok, which is really just onM
except that the _signalok is meant to convey that the caller
asserts that scalarg and ptrarg will be restored to their old
values after the call (instead of the usual behavior, zeroing them).
Scalararg and ptrarg are also untyped and therefore error-prone.
Go code can always pass a closure instead of using scalararg
and ptrarg; they were only really necessary for C code.
And there's no more C code.
For all these reasons, delete scalararg and ptrarg, converting
the few remaining references to use closures.
Once those are gone, there is no need for a distinction between
onM and onM_signalok, so replace both with a single function
equivalent to the current onM_signalok (that is, it can be called
on any of the curg, g0, and gsignal stacks).
The name onM and the phrase 'm stack' are misnomers,
because on most system an M has two system stacks:
the main thread stack and the signal handling stack.
Correct the misnomer by naming the replacement function systemstack.
Fix a few references to "M stack" in code.
The main motivation for this change is to eliminate scalararg/ptrarg.
Rick and I have already seen them cause problems because
the calling sequence m.ptrarg[0] = p is a heap pointer assignment,
so it gets a write barrier. The write barrier also uses onM, so it has
all the same problems as if it were being invoked by a signal handler.
We worked around this by saving and restoring the old values
and by calling onM_signalok, but there's no point in keeping this nice
home for bugs around any longer.
This CL also changes funcline to return the file name as a result
instead of filling in a passed-in *string. (The *string signature is
left over from when the code was written in and called from C.)
That's arguably an unrelated change, except that once I had done
the ptrarg/scalararg/onM cleanup I started getting false positives
about the *string argument escaping (not allowed in package runtime).
The compiler is wrong, but the easiest fix is to write the code like
Go code instead of like C code. I am a bit worried that the compiler
is wrong because of some use of uninitialized memory in the escape
analysis. If that's the reason, it will go away when we convert the
compiler to Go. (And if not, we'll debug it the next time.)
LGTM=khr
R=r, khr
CC=austin, golang-codereviews, iant, rlh
https://golang.org/cl/174950043
2014-11-12 12:54:31 -07:00
|
|
|
// systemstack runs fn on a system stack.
|
|
|
|
|
// If systemstack is called from the per-OS-thread (g0) stack, or
|
|
|
|
|
// if systemstack is called from the signal handling (gsignal) stack,
|
|
|
|
|
// systemstack calls fn directly and returns.
|
|
|
|
|
// Otherwise, systemstack is being called from the limited stack
|
|
|
|
|
// of an ordinary goroutine. In this case, systemstack switches
|
|
|
|
|
// to the per-OS-thread stack, calls fn, and switches back.
|
|
|
|
|
// It is common to use a func literal as the argument, in order
|
|
|
|
|
// to share inputs and outputs with the code around the call
|
|
|
|
|
// to system stack:
|
runtime: reject onM calls from gsignal stack
The implementation and use patterns of onM assume
that they run on either the m->curg or m->g0 stack.
Calling onM from m->gsignal has two problems:
(1) When not on g0, onM switches to g0 and then "back" to curg.
If we didn't start at curg, bad things happen.
(2) The use of scalararg/ptrarg to pass C arguments and results
assumes that there is only one onM call at a time.
If a gsignal starts running, it may have interrupted the
setup/teardown of the args for an onM on the curg or g0 stack.
Using scalararg/ptrarg itself would smash those.
We can fix (1) by remembering what g was running before the switch.
We can fix (2) by requiring that uses of onM that might happen
on a signal handling stack must save the old scalararg/ptrarg
and restore them after the call, instead of zeroing them.
The only sane way to do this is to introduce a separate
onM_signalsafe that omits the signal check, and then if you
see a call to onM_signalsafe you know the surrounding code
must preserve the old scalararg/ptrarg values.
(The implementation would be that onM_signalsafe just calls
fn if on the signal stack or else jumps to onM. It's not necessary
to have two whole copies of the function.)
(2) is not a problem if the caller and callee are both Go and
a closure is used instead of the scalararg/ptrarg slots.
For now, I think we can avoid calling onM from code executing
on gsignal stacks, so just reject it.
In the long term, (2) goes away (as do the scalararg/ptrarg slots)
once everything is in Go, and at that point fixing (1) would be
trivial and maybe worth doing just for regularity.
LGTM=iant
R=golang-codereviews, iant
CC=dvyukov, golang-codereviews, khr, r
https://golang.org/cl/135400043
2014-09-03 22:10:10 -06:00
|
|
|
//
|
[dev.cc] runtime: delete scalararg, ptrarg; rename onM to systemstack
Scalararg and ptrarg are not "signal safe".
Go code filling them out can be interrupted by a signal,
and then the signal handler runs, and if it also ends up
in Go code that uses scalararg or ptrarg, now the old
values have been smashed.
For the pieces of code that do need to run in a signal handler,
we introduced onM_signalok, which is really just onM
except that the _signalok is meant to convey that the caller
asserts that scalarg and ptrarg will be restored to their old
values after the call (instead of the usual behavior, zeroing them).
Scalararg and ptrarg are also untyped and therefore error-prone.
Go code can always pass a closure instead of using scalararg
and ptrarg; they were only really necessary for C code.
And there's no more C code.
For all these reasons, delete scalararg and ptrarg, converting
the few remaining references to use closures.
Once those are gone, there is no need for a distinction between
onM and onM_signalok, so replace both with a single function
equivalent to the current onM_signalok (that is, it can be called
on any of the curg, g0, and gsignal stacks).
The name onM and the phrase 'm stack' are misnomers,
because on most system an M has two system stacks:
the main thread stack and the signal handling stack.
Correct the misnomer by naming the replacement function systemstack.
Fix a few references to "M stack" in code.
The main motivation for this change is to eliminate scalararg/ptrarg.
Rick and I have already seen them cause problems because
the calling sequence m.ptrarg[0] = p is a heap pointer assignment,
so it gets a write barrier. The write barrier also uses onM, so it has
all the same problems as if it were being invoked by a signal handler.
We worked around this by saving and restoring the old values
and by calling onM_signalok, but there's no point in keeping this nice
home for bugs around any longer.
This CL also changes funcline to return the file name as a result
instead of filling in a passed-in *string. (The *string signature is
left over from when the code was written in and called from C.)
That's arguably an unrelated change, except that once I had done
the ptrarg/scalararg/onM cleanup I started getting false positives
about the *string argument escaping (not allowed in package runtime).
The compiler is wrong, but the easiest fix is to write the code like
Go code instead of like C code. I am a bit worried that the compiler
is wrong because of some use of uninitialized memory in the escape
analysis. If that's the reason, it will go away when we convert the
compiler to Go. (And if not, we'll debug it the next time.)
LGTM=khr
R=r, khr
CC=austin, golang-codereviews, iant, rlh
https://golang.org/cl/174950043
2014-11-12 12:54:31 -07:00
|
|
|
// ... set up y ...
|
|
|
|
|
// systemstack(func() {
|
|
|
|
|
// x = bigcall(y)
|
|
|
|
|
// })
|
|
|
|
|
// ... use x ...
|
2014-09-11 10:08:30 -06:00
|
|
|
//
|
|
|
|
|
//go:noescape
|
[dev.cc] runtime: delete scalararg, ptrarg; rename onM to systemstack
Scalararg and ptrarg are not "signal safe".
Go code filling them out can be interrupted by a signal,
and then the signal handler runs, and if it also ends up
in Go code that uses scalararg or ptrarg, now the old
values have been smashed.
For the pieces of code that do need to run in a signal handler,
we introduced onM_signalok, which is really just onM
except that the _signalok is meant to convey that the caller
asserts that scalarg and ptrarg will be restored to their old
values after the call (instead of the usual behavior, zeroing them).
Scalararg and ptrarg are also untyped and therefore error-prone.
Go code can always pass a closure instead of using scalararg
and ptrarg; they were only really necessary for C code.
And there's no more C code.
For all these reasons, delete scalararg and ptrarg, converting
the few remaining references to use closures.
Once those are gone, there is no need for a distinction between
onM and onM_signalok, so replace both with a single function
equivalent to the current onM_signalok (that is, it can be called
on any of the curg, g0, and gsignal stacks).
The name onM and the phrase 'm stack' are misnomers,
because on most system an M has two system stacks:
the main thread stack and the signal handling stack.
Correct the misnomer by naming the replacement function systemstack.
Fix a few references to "M stack" in code.
The main motivation for this change is to eliminate scalararg/ptrarg.
Rick and I have already seen them cause problems because
the calling sequence m.ptrarg[0] = p is a heap pointer assignment,
so it gets a write barrier. The write barrier also uses onM, so it has
all the same problems as if it were being invoked by a signal handler.
We worked around this by saving and restoring the old values
and by calling onM_signalok, but there's no point in keeping this nice
home for bugs around any longer.
This CL also changes funcline to return the file name as a result
instead of filling in a passed-in *string. (The *string signature is
left over from when the code was written in and called from C.)
That's arguably an unrelated change, except that once I had done
the ptrarg/scalararg/onM cleanup I started getting false positives
about the *string argument escaping (not allowed in package runtime).
The compiler is wrong, but the easiest fix is to write the code like
Go code instead of like C code. I am a bit worried that the compiler
is wrong because of some use of uninitialized memory in the escape
analysis. If that's the reason, it will go away when we convert the
compiler to Go. (And if not, we'll debug it the next time.)
LGTM=khr
R=r, khr
CC=austin, golang-codereviews, iant, rlh
https://golang.org/cl/174950043
2014-11-12 12:54:31 -07:00
|
|
|
func systemstack(fn func())
|
2014-09-11 10:08:30 -06:00
|
|
|
|
[dev.cc] runtime: delete scalararg, ptrarg; rename onM to systemstack
Scalararg and ptrarg are not "signal safe".
Go code filling them out can be interrupted by a signal,
and then the signal handler runs, and if it also ends up
in Go code that uses scalararg or ptrarg, now the old
values have been smashed.
For the pieces of code that do need to run in a signal handler,
we introduced onM_signalok, which is really just onM
except that the _signalok is meant to convey that the caller
asserts that scalarg and ptrarg will be restored to their old
values after the call (instead of the usual behavior, zeroing them).
Scalararg and ptrarg are also untyped and therefore error-prone.
Go code can always pass a closure instead of using scalararg
and ptrarg; they were only really necessary for C code.
And there's no more C code.
For all these reasons, delete scalararg and ptrarg, converting
the few remaining references to use closures.
Once those are gone, there is no need for a distinction between
onM and onM_signalok, so replace both with a single function
equivalent to the current onM_signalok (that is, it can be called
on any of the curg, g0, and gsignal stacks).
The name onM and the phrase 'm stack' are misnomers,
because on most system an M has two system stacks:
the main thread stack and the signal handling stack.
Correct the misnomer by naming the replacement function systemstack.
Fix a few references to "M stack" in code.
The main motivation for this change is to eliminate scalararg/ptrarg.
Rick and I have already seen them cause problems because
the calling sequence m.ptrarg[0] = p is a heap pointer assignment,
so it gets a write barrier. The write barrier also uses onM, so it has
all the same problems as if it were being invoked by a signal handler.
We worked around this by saving and restoring the old values
and by calling onM_signalok, but there's no point in keeping this nice
home for bugs around any longer.
This CL also changes funcline to return the file name as a result
instead of filling in a passed-in *string. (The *string signature is
left over from when the code was written in and called from C.)
That's arguably an unrelated change, except that once I had done
the ptrarg/scalararg/onM cleanup I started getting false positives
about the *string argument escaping (not allowed in package runtime).
The compiler is wrong, but the easiest fix is to write the code like
Go code instead of like C code. I am a bit worried that the compiler
is wrong because of some use of uninitialized memory in the escape
analysis. If that's the reason, it will go away when we convert the
compiler to Go. (And if not, we'll debug it the next time.)
LGTM=khr
R=r, khr
CC=austin, golang-codereviews, iant, rlh
https://golang.org/cl/174950043
2014-11-12 12:54:31 -07:00
|
|
|
func badsystemstack() {
|
2014-12-27 21:58:00 -07:00
|
|
|
throw("systemstack called from unexpected goroutine")
|
runtime: reject onM calls from gsignal stack
The implementation and use patterns of onM assume
that they run on either the m->curg or m->g0 stack.
Calling onM from m->gsignal has two problems:
(1) When not on g0, onM switches to g0 and then "back" to curg.
If we didn't start at curg, bad things happen.
(2) The use of scalararg/ptrarg to pass C arguments and results
assumes that there is only one onM call at a time.
If a gsignal starts running, it may have interrupted the
setup/teardown of the args for an onM on the curg or g0 stack.
Using scalararg/ptrarg itself would smash those.
We can fix (1) by remembering what g was running before the switch.
We can fix (2) by requiring that uses of onM that might happen
on a signal handling stack must save the old scalararg/ptrarg
and restore them after the call, instead of zeroing them.
The only sane way to do this is to introduce a separate
onM_signalsafe that omits the signal check, and then if you
see a call to onM_signalsafe you know the surrounding code
must preserve the old scalararg/ptrarg values.
(The implementation would be that onM_signalsafe just calls
fn if on the signal stack or else jumps to onM. It's not necessary
to have two whole copies of the function.)
(2) is not a problem if the caller and callee are both Go and
a closure is used instead of the scalararg/ptrarg slots.
For now, I think we can avoid calling onM from code executing
on gsignal stacks, so just reject it.
In the long term, (2) goes away (as do the scalararg/ptrarg slots)
once everything is in Go, and at that point fixing (1) would be
trivial and maybe worth doing just for regularity.
LGTM=iant
R=golang-codereviews, iant
CC=dvyukov, golang-codereviews, khr, r
https://golang.org/cl/135400043
2014-09-03 22:10:10 -06:00
|
|
|
}
|
|
|
|
|
|
2014-07-16 15:16:19 -06:00
|
|
|
// memclr clears n bytes starting at ptr.
|
|
|
|
|
// in memclr_*.s
|
2014-08-07 14:58:42 -06:00
|
|
|
//go:noescape
|
2014-07-16 15:16:19 -06:00
|
|
|
func memclr(ptr unsafe.Pointer, n uintptr)
|
|
|
|
|
|
2014-12-22 12:31:55 -07:00
|
|
|
//go:linkname reflect_memclr reflect.memclr
|
|
|
|
|
func reflect_memclr(ptr unsafe.Pointer, n uintptr) {
|
|
|
|
|
memclr(ptr, n)
|
|
|
|
|
}
|
|
|
|
|
|
2014-07-16 15:16:19 -06:00
|
|
|
// memmove copies n bytes from "from" to "to".
|
|
|
|
|
// in memmove_*.s
|
2014-08-07 14:58:42 -06:00
|
|
|
//go:noescape
|
2014-12-22 11:27:53 -07:00
|
|
|
func memmove(to, from unsafe.Pointer, n uintptr)
|
|
|
|
|
|
|
|
|
|
//go:linkname reflect_memmove reflect.memmove
|
|
|
|
|
func reflect_memmove(to, from unsafe.Pointer, n uintptr) {
|
|
|
|
|
memmove(to, from, n)
|
|
|
|
|
}
|
2014-07-16 15:16:19 -06:00
|
|
|
|
|
|
|
|
// exported value for testing
|
|
|
|
|
var hashLoad = loadFactor
|
|
|
|
|
|
2014-09-09 15:32:53 -06:00
|
|
|
// in asm_*.s
|
|
|
|
|
func fastrand1() uint32
|
|
|
|
|
|
2014-07-16 15:16:19 -06:00
|
|
|
// in asm_*.s
|
|
|
|
|
//go:noescape
|
2014-08-07 15:52:55 -06:00
|
|
|
func memeq(a, b unsafe.Pointer, size uintptr) bool
|
2014-07-16 15:16:19 -06:00
|
|
|
|
2014-07-31 16:07:05 -06:00
|
|
|
// noescape hides a pointer from escape analysis. noescape is
|
|
|
|
|
// the identity function but escape analysis doesn't think the
|
|
|
|
|
// output depends on the input. noescape is inlined and currently
|
|
|
|
|
// compiles down to a single xor instruction.
|
|
|
|
|
// USE CAREFULLY!
|
2014-09-03 09:49:43 -06:00
|
|
|
//go:nosplit
|
2014-07-31 16:07:05 -06:00
|
|
|
func noescape(p unsafe.Pointer) unsafe.Pointer {
|
|
|
|
|
x := uintptr(p)
|
|
|
|
|
return unsafe.Pointer(x ^ 0)
|
|
|
|
|
}
|
2014-08-07 14:58:42 -06:00
|
|
|
|
cmd/cc, runtime: convert C compilers to use Go calling convention
To date, the C compilers and Go compilers differed only in how
values were returned from functions. This made it difficult to call
Go from C or C from Go if return values were involved. It also made
assembly called from Go and assembly called from C different.
This CL changes the C compiler to use the Go conventions, passing
results on the stack, after the arguments.
[Exception: this does not apply to C ... functions, because you can't
know where on the stack the arguments end.]
By doing this, the CL makes it possible to rewrite C functions into Go
one at a time, without worrying about which languages call that
function or which languages it calls.
This CL also updates all the assembly files in package runtime to use
the new conventions. Argument references of the form 40(SP) have
been rewritten to the form name+10(FP) instead, and there are now
Go func prototypes for every assembly function called from C or Go.
This means that 'go vet runtime' checks effectively every assembly
function, and go vet's output was used to automate the bulk of the
conversion.
Some functions, like seek and nsec on Plan 9, needed to be rewritten.
Many assembly routines called from C were reading arguments
incorrectly, using MOVL instead of MOVQ or vice versa, especially on
the less used systems like openbsd.
These were found by go vet and have been corrected too.
If we're lucky, this may reduce flakiness on those systems.
Tested on:
darwin/386
darwin/amd64
linux/arm
linux/386
linux/amd64
If this breaks another system, the bug is almost certainly in the
sys_$GOOS_$GOARCH.s file, since the rest of the CL is tested
by the combination of the above systems.
LGTM=dvyukov, iant
R=golang-codereviews, 0intro, dave, alex.brainman, dvyukov, iant
CC=golang-codereviews, josharian, r
https://golang.org/cl/135830043
2014-08-27 09:32:17 -06:00
|
|
|
func cgocallback(fn, frame unsafe.Pointer, framesize uintptr)
|
|
|
|
|
func gogo(buf *gobuf)
|
|
|
|
|
func gosave(buf *gobuf)
|
|
|
|
|
func mincore(addr unsafe.Pointer, n uintptr, dst *byte) int32
|
2014-09-03 09:49:43 -06:00
|
|
|
|
|
|
|
|
//go:noescape
|
|
|
|
|
func jmpdefer(fv *funcval, argp uintptr)
|
cmd/cc, runtime: convert C compilers to use Go calling convention
To date, the C compilers and Go compilers differed only in how
values were returned from functions. This made it difficult to call
Go from C or C from Go if return values were involved. It also made
assembly called from Go and assembly called from C different.
This CL changes the C compiler to use the Go conventions, passing
results on the stack, after the arguments.
[Exception: this does not apply to C ... functions, because you can't
know where on the stack the arguments end.]
By doing this, the CL makes it possible to rewrite C functions into Go
one at a time, without worrying about which languages call that
function or which languages it calls.
This CL also updates all the assembly files in package runtime to use
the new conventions. Argument references of the form 40(SP) have
been rewritten to the form name+10(FP) instead, and there are now
Go func prototypes for every assembly function called from C or Go.
This means that 'go vet runtime' checks effectively every assembly
function, and go vet's output was used to automate the bulk of the
conversion.
Some functions, like seek and nsec on Plan 9, needed to be rewritten.
Many assembly routines called from C were reading arguments
incorrectly, using MOVL instead of MOVQ or vice versa, especially on
the less used systems like openbsd.
These were found by go vet and have been corrected too.
If we're lucky, this may reduce flakiness on those systems.
Tested on:
darwin/386
darwin/amd64
linux/arm
linux/386
linux/amd64
If this breaks another system, the bug is almost certainly in the
sys_$GOOS_$GOARCH.s file, since the rest of the CL is tested
by the combination of the above systems.
LGTM=dvyukov, iant
R=golang-codereviews, 0intro, dave, alex.brainman, dvyukov, iant
CC=golang-codereviews, josharian, r
https://golang.org/cl/135830043
2014-08-27 09:32:17 -06:00
|
|
|
func exit1(code int32)
|
|
|
|
|
func asminit()
|
|
|
|
|
func setg(gg *g)
|
|
|
|
|
func breakpoint()
|
2014-10-21 15:46:07 -06:00
|
|
|
|
2014-12-30 11:59:55 -07:00
|
|
|
// reflectcall calls fn with a copy of the n argument bytes pointed at by arg.
|
|
|
|
|
// After fn returns, reflectcall copies n-retoffset result bytes
|
|
|
|
|
// back into arg+retoffset before returning. If copying result bytes back,
|
|
|
|
|
// the caller should pass the argument frame type as argtype, so that
|
|
|
|
|
// call can execute appropriate write barriers during the copy.
|
|
|
|
|
// Package reflect passes a frame type. In package runtime, there is only
|
|
|
|
|
// one call that copies results back, in cgocallbackg1, and it does NOT pass a
|
|
|
|
|
// frame type, meaning there are no write barriers invoked. See that call
|
|
|
|
|
// site for justification.
|
|
|
|
|
func reflectcall(argtype *_type, fn, arg unsafe.Pointer, argsize uint32, retoffset uint32)
|
|
|
|
|
|
2014-09-04 19:12:31 -06:00
|
|
|
func procyield(cycles uint32)
|
cmd/cc, runtime: convert C compilers to use Go calling convention
To date, the C compilers and Go compilers differed only in how
values were returned from functions. This made it difficult to call
Go from C or C from Go if return values were involved. It also made
assembly called from Go and assembly called from C different.
This CL changes the C compiler to use the Go conventions, passing
results on the stack, after the arguments.
[Exception: this does not apply to C ... functions, because you can't
know where on the stack the arguments end.]
By doing this, the CL makes it possible to rewrite C functions into Go
one at a time, without worrying about which languages call that
function or which languages it calls.
This CL also updates all the assembly files in package runtime to use
the new conventions. Argument references of the form 40(SP) have
been rewritten to the form name+10(FP) instead, and there are now
Go func prototypes for every assembly function called from C or Go.
This means that 'go vet runtime' checks effectively every assembly
function, and go vet's output was used to automate the bulk of the
conversion.
Some functions, like seek and nsec on Plan 9, needed to be rewritten.
Many assembly routines called from C were reading arguments
incorrectly, using MOVL instead of MOVQ or vice versa, especially on
the less used systems like openbsd.
These were found by go vet and have been corrected too.
If we're lucky, this may reduce flakiness on those systems.
Tested on:
darwin/386
darwin/amd64
linux/arm
linux/386
linux/amd64
If this breaks another system, the bug is almost certainly in the
sys_$GOOS_$GOARCH.s file, since the rest of the CL is tested
by the combination of the above systems.
LGTM=dvyukov, iant
R=golang-codereviews, 0intro, dave, alex.brainman, dvyukov, iant
CC=golang-codereviews, josharian, r
https://golang.org/cl/135830043
2014-08-27 09:32:17 -06:00
|
|
|
func cgocallback_gofunc(fv *funcval, frame unsafe.Pointer, framesize uintptr)
|
2014-09-03 09:49:43 -06:00
|
|
|
func goexit()
|
cmd/cc, runtime: preserve C runtime type names in generated Go
uintptr or uint64 in the runtime C were turning into uint in the Go,
bool was turning into uint8, and so on. Fix that.
Also delete Go wrappers for C functions.
The C functions can be called directly now
(but still eventually need to be converted to Go).
LGTM=bradfitz, minux, iant
R=golang-codereviews, bradfitz, iant, minux
CC=golang-codereviews, khr, r
https://golang.org/cl/138740043
2014-08-27 19:59:49 -06:00
|
|
|
|
|
|
|
|
//go:noescape
|
|
|
|
|
func cas(ptr *uint32, old, new uint32) bool
|
|
|
|
|
|
2014-12-22 20:50:42 -07:00
|
|
|
// NO go:noescape annotation; see atomic_pointer.go.
|
2014-11-11 15:09:09 -07:00
|
|
|
func casp1(ptr *unsafe.Pointer, old, new unsafe.Pointer) bool
|
|
|
|
|
|
|
|
|
|
func nop() // call to prevent inlining of function body
|
cmd/cc, runtime: preserve C runtime type names in generated Go
uintptr or uint64 in the runtime C were turning into uint in the Go,
bool was turning into uint8, and so on. Fix that.
Also delete Go wrappers for C functions.
The C functions can be called directly now
(but still eventually need to be converted to Go).
LGTM=bradfitz, minux, iant
R=golang-codereviews, bradfitz, iant, minux
CC=golang-codereviews, khr, r
https://golang.org/cl/138740043
2014-08-27 19:59:49 -06:00
|
|
|
|
|
|
|
|
//go:noescape
|
|
|
|
|
func casuintptr(ptr *uintptr, old, new uintptr) bool
|
|
|
|
|
|
2014-08-29 14:20:48 -06:00
|
|
|
//go:noescape
|
|
|
|
|
func atomicstoreuintptr(ptr *uintptr, new uintptr)
|
|
|
|
|
|
|
|
|
|
//go:noescape
|
|
|
|
|
func atomicloaduintptr(ptr *uintptr) uintptr
|
|
|
|
|
|
2014-08-30 12:03:28 -06:00
|
|
|
//go:noescape
|
|
|
|
|
func atomicloaduint(ptr *uint) uint
|
|
|
|
|
|
cmd/cc, runtime: preserve C runtime type names in generated Go
uintptr or uint64 in the runtime C were turning into uint in the Go,
bool was turning into uint8, and so on. Fix that.
Also delete Go wrappers for C functions.
The C functions can be called directly now
(but still eventually need to be converted to Go).
LGTM=bradfitz, minux, iant
R=golang-codereviews, bradfitz, iant, minux
CC=golang-codereviews, khr, r
https://golang.org/cl/138740043
2014-08-27 19:59:49 -06:00
|
|
|
//go:noescape
|
|
|
|
|
func setcallerpc(argp unsafe.Pointer, pc uintptr)
|
|
|
|
|
|
runtime: avoid gentraceback of self on user goroutine stack
Gentraceback may grow the stack.
One of the gentraceback wrappers may grow the stack.
One of the gentraceback callback calls may grow the stack.
Various stack pointers are stored in various stack locations
as type uintptr during the execution of these calls.
If the stack does grow, these stack pointers will not be
updated and will start trying to decode stack memory that
is no longer valid.
It may be possible to change the type of the stack pointer
variables to be unsafe.Pointer, but that's pretty subtle and
may still have problems, even if we catch every last one.
An easier, more obviously correct fix is to require that
gentraceback of the currently running goroutine must run
on the g0 stack, not on the goroutine's own stack.
Not doing this causes faults when you set
StackFromSystem = 1
StackFaultOnFree = 1
The new check in gentraceback will catch future lapses.
The more general problem is calling getcallersp but then
calling a function that might relocate the stack, which would
invalidate the result of getcallersp. Add note to stubs.go
declaration of getcallersp explaining the problem, and
check all existing calls to getcallersp. Most needed fixes.
This affects Callers, Stack, and nearly all the runtime
profiling routines. It does not affect stack copying directly
nor garbage collection.
LGTM=khr
R=khr, bradfitz
CC=golang-codereviews, r
https://golang.org/cl/167060043
2014-11-05 21:01:48 -07:00
|
|
|
// getcallerpc returns the program counter (PC) of its caller's caller.
|
|
|
|
|
// getcallersp returns the stack pointer (SP) of its caller's caller.
|
|
|
|
|
// For both, the argp must be a pointer to the caller's first function argument.
|
|
|
|
|
// The implementation may or may not use argp, depending on
|
|
|
|
|
// the architecture.
|
|
|
|
|
//
|
|
|
|
|
// For example:
|
|
|
|
|
//
|
|
|
|
|
// func f(arg1, arg2, arg3 int) {
|
|
|
|
|
// pc := getcallerpc(unsafe.Pointer(&arg1))
|
2014-11-17 15:55:15 -07:00
|
|
|
// sp := getcallersp(unsafe.Pointer(&arg1))
|
runtime: avoid gentraceback of self on user goroutine stack
Gentraceback may grow the stack.
One of the gentraceback wrappers may grow the stack.
One of the gentraceback callback calls may grow the stack.
Various stack pointers are stored in various stack locations
as type uintptr during the execution of these calls.
If the stack does grow, these stack pointers will not be
updated and will start trying to decode stack memory that
is no longer valid.
It may be possible to change the type of the stack pointer
variables to be unsafe.Pointer, but that's pretty subtle and
may still have problems, even if we catch every last one.
An easier, more obviously correct fix is to require that
gentraceback of the currently running goroutine must run
on the g0 stack, not on the goroutine's own stack.
Not doing this causes faults when you set
StackFromSystem = 1
StackFaultOnFree = 1
The new check in gentraceback will catch future lapses.
The more general problem is calling getcallersp but then
calling a function that might relocate the stack, which would
invalidate the result of getcallersp. Add note to stubs.go
declaration of getcallersp explaining the problem, and
check all existing calls to getcallersp. Most needed fixes.
This affects Callers, Stack, and nearly all the runtime
profiling routines. It does not affect stack copying directly
nor garbage collection.
LGTM=khr
R=khr, bradfitz
CC=golang-codereviews, r
https://golang.org/cl/167060043
2014-11-05 21:01:48 -07:00
|
|
|
// }
|
|
|
|
|
//
|
|
|
|
|
// These two lines find the PC and SP immediately following
|
|
|
|
|
// the call to f (where f will return).
|
|
|
|
|
//
|
|
|
|
|
// The call to getcallerpc and getcallersp must be done in the
|
|
|
|
|
// frame being asked about. It would not be correct for f to pass &arg1
|
|
|
|
|
// to another function g and let g call getcallerpc/getcallersp.
|
|
|
|
|
// The call inside g might return information about g's caller or
|
|
|
|
|
// information about f's caller or complete garbage.
|
|
|
|
|
//
|
|
|
|
|
// The result of getcallersp is correct at the time of the return,
|
|
|
|
|
// but it may be invalidated by any subsequent call to a function
|
|
|
|
|
// that might relocate the stack in order to grow or shrink it.
|
|
|
|
|
// A general rule is that the result of getcallersp should be used
|
|
|
|
|
// immediately and can only be passed to nosplit functions.
|
|
|
|
|
|
cmd/cc, runtime: preserve C runtime type names in generated Go
uintptr or uint64 in the runtime C were turning into uint in the Go,
bool was turning into uint8, and so on. Fix that.
Also delete Go wrappers for C functions.
The C functions can be called directly now
(but still eventually need to be converted to Go).
LGTM=bradfitz, minux, iant
R=golang-codereviews, bradfitz, iant, minux
CC=golang-codereviews, khr, r
https://golang.org/cl/138740043
2014-08-27 19:59:49 -06:00
|
|
|
//go:noescape
|
|
|
|
|
func getcallerpc(argp unsafe.Pointer) uintptr
|
|
|
|
|
|
|
|
|
|
//go:noescape
|
|
|
|
|
func getcallersp(argp unsafe.Pointer) uintptr
|
2014-08-30 12:53:47 -06:00
|
|
|
|
|
|
|
|
//go:noescape
|
|
|
|
|
func asmcgocall(fn, arg unsafe.Pointer)
|
|
|
|
|
|
2014-09-04 12:40:40 -06:00
|
|
|
//go:noescape
|
|
|
|
|
func asmcgocall_errno(fn, arg unsafe.Pointer) int32
|
|
|
|
|
|
2014-11-11 15:09:09 -07:00
|
|
|
// argp used in Defer structs when there is no argp.
|
runtime: convert traceback*.c to Go
The two converted files were nearly identical.
Instead of continuing that duplication, I merged them
into a single traceback.go.
Tested on arm, amd64, amd64p32, and 386.
LGTM=r
R=golang-codereviews, remyoudompheng, dave, r
CC=dvyukov, golang-codereviews, iant, khr
https://golang.org/cl/134200044
2014-09-02 13:12:53 -06:00
|
|
|
const _NoArgs = ^uintptr(0)
|
|
|
|
|
|
2014-09-03 11:02:48 -06:00
|
|
|
func morestack()
|
|
|
|
|
func rt0_go()
|
2014-09-03 09:49:43 -06:00
|
|
|
|
|
|
|
|
// return0 is a stub used to return 0 from deferproc.
|
|
|
|
|
// It is called at the very end of deferproc to signal
|
|
|
|
|
// the calling Go function that it should not jump
|
|
|
|
|
// to deferreturn.
|
|
|
|
|
// in asm_*.s
|
|
|
|
|
func return0()
|
2014-09-09 15:32:53 -06:00
|
|
|
|
2014-12-22 11:27:53 -07:00
|
|
|
//go:linkname time_now time.now
|
|
|
|
|
func time_now() (sec int64, nsec int32)
|
2014-10-15 11:12:16 -06:00
|
|
|
|
|
|
|
|
// in asm_*.s
|
|
|
|
|
// not called directly; definitions here supply type information for traceback.
|
|
|
|
|
func call16(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call32(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call64(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call128(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call256(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call512(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call1024(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call2048(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call4096(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call8192(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call16384(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call32768(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call65536(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call131072(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call262144(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call524288(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call1048576(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call2097152(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call4194304(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call8388608(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call16777216(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call33554432(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call67108864(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call134217728(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call268435456(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call536870912(fn, arg unsafe.Pointer, n, retoffset uint32)
|
|
|
|
|
func call1073741824(fn, arg unsafe.Pointer, n, retoffset uint32)
|
2014-11-11 15:09:09 -07:00
|
|
|
|
[dev.cc] runtime: delete scalararg, ptrarg; rename onM to systemstack
Scalararg and ptrarg are not "signal safe".
Go code filling them out can be interrupted by a signal,
and then the signal handler runs, and if it also ends up
in Go code that uses scalararg or ptrarg, now the old
values have been smashed.
For the pieces of code that do need to run in a signal handler,
we introduced onM_signalok, which is really just onM
except that the _signalok is meant to convey that the caller
asserts that scalarg and ptrarg will be restored to their old
values after the call (instead of the usual behavior, zeroing them).
Scalararg and ptrarg are also untyped and therefore error-prone.
Go code can always pass a closure instead of using scalararg
and ptrarg; they were only really necessary for C code.
And there's no more C code.
For all these reasons, delete scalararg and ptrarg, converting
the few remaining references to use closures.
Once those are gone, there is no need for a distinction between
onM and onM_signalok, so replace both with a single function
equivalent to the current onM_signalok (that is, it can be called
on any of the curg, g0, and gsignal stacks).
The name onM and the phrase 'm stack' are misnomers,
because on most system an M has two system stacks:
the main thread stack and the signal handling stack.
Correct the misnomer by naming the replacement function systemstack.
Fix a few references to "M stack" in code.
The main motivation for this change is to eliminate scalararg/ptrarg.
Rick and I have already seen them cause problems because
the calling sequence m.ptrarg[0] = p is a heap pointer assignment,
so it gets a write barrier. The write barrier also uses onM, so it has
all the same problems as if it were being invoked by a signal handler.
We worked around this by saving and restoring the old values
and by calling onM_signalok, but there's no point in keeping this nice
home for bugs around any longer.
This CL also changes funcline to return the file name as a result
instead of filling in a passed-in *string. (The *string signature is
left over from when the code was written in and called from C.)
That's arguably an unrelated change, except that once I had done
the ptrarg/scalararg/onM cleanup I started getting false positives
about the *string argument escaping (not allowed in package runtime).
The compiler is wrong, but the easiest fix is to write the code like
Go code instead of like C code. I am a bit worried that the compiler
is wrong because of some use of uninitialized memory in the escape
analysis. If that's the reason, it will go away when we convert the
compiler to Go. (And if not, we'll debug it the next time.)
LGTM=khr
R=r, khr
CC=austin, golang-codereviews, iant, rlh
https://golang.org/cl/174950043
2014-11-12 12:54:31 -07:00
|
|
|
func systemstack_switch()
|
2014-11-21 13:57:10 -07:00
|
|
|
|
|
|
|
|
func prefetcht0(addr uintptr)
|
|
|
|
|
func prefetcht1(addr uintptr)
|
|
|
|
|
func prefetcht2(addr uintptr)
|
|
|
|
|
func prefetchnta(addr uintptr)
|
