Currently, escape analysis is able to record at most one dereference
when a parameter leaks to the heap; that is, at call sites, it can't
distinguish between any of these three functions:
func x1(p ****int) { sink = *p }
func x2(p ****int) { sink = **p }
func x3(p ****int) { sink = ***p }
Similarly, it's limited to recording parameter leaks to only the first
4 parameters, and only up to 6 dereferences.
All of these limitations are due to the awkward encoding scheme used
at the moment.
This CL replaces the encoding scheme with a simple [8]uint8 array,
which can handle up to the first 7 parameters, and up to 254
dereferences, which ought to be enough for anyone. And if not, it's
much more easily increased.
Shrinks export data size geometric mean for Kubernetes by 0.07%.
Fixes#33981.
Change-Id: I10a94b9accac9a0c91490e0d6d458316f5ca1e13
Reviewed-on: https://go-review.googlesource.com/c/go/+/197680
Reviewed-by: Cherry Zhang <cherryyz@google.com>
This information is redundant with the position information already
provided. Also, no other -m diagnostics print out function name.
While here, report parameter leak diagnostics against the parameter
declaration position rather than the function, and use Warnl for
"moved to heap" messages.
Test cases updated programmatically by removing the first word from
every "no match for" error emitted by run.go:
go run run.go |& \
sed -E -n 's/^(.*):(.*): no match for `([^ ]* (.*))` in:$/\1!\2!\3!\4/p' | \
while IFS='!' read -r fn line before after; do
before=$(echo "$before" | sed 's/[.[\*^$()+?{|]/\\&/g')
after=$(echo "$after" | sed -E 's/(\&|\\)/\\&/g')
fn=$(find . -name "${fn}" | head -1)
sed -i -E -e "${line}s/\"${before}\"/\"${after}\"/" "${fn}"
done
Passes toolstash-check.
Change-Id: I6e02486b1409e4a8dbb2b9b816d22095835426b5
Reviewed-on: https://go-review.googlesource.com/c/go/+/195040
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
In general, a conversion to interface type may require values to be
boxed, which in turn necessitates escape analysis to determine whether
the boxed representation can be stack allocated.
However, esc.go used to unconditionally print escape analysis
decisions about OCONVIFACE, even for conversions that don't require
boxing (e.g., pointers, channels, maps, functions).
For test compatibility with esc.go, escape.go similarly printed these
useless diagnostics. This CL removes the diagnostics, and updates test
expectations accordingly.
Change-Id: I97c57a4a08e44d265bba516c78426ff4f2bf1e12
Reviewed-on: https://go-review.googlesource.com/c/go/+/192697
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Prep for subsequent CLs to remove old escape analysis pass.
This CL removes -newescape=true from tests that use it, and deletes
tests that use -newescape=false. (For history, see CL 170447.)
Notably, this removes escape_because.go without any replacement, but
this is being tracked by #31489.
Change-Id: I6f6058d58fff2c5d210cb1d2713200cc9f501ca7
Reviewed-on: https://go-review.googlesource.com/c/go/+/187617
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
The new escape analysis implementation tries to emit debugging
diagnostics that are compatible with the existing implementation, but
there's a handful of cases that are easier to handle by updating the
test expectations instead.
For regress tests that need updating, the original file is copied to
oldescapeXXX.go.go with -newescape=false added to the //errorcheck
line, while the file is updated in place with -newescape=true and new
test requirements.
Notable test changes:
1) escape_because.go looks for a lot of detailed internal debugging
messages that are fairly particular to how esc.go works and that I
haven't attempted to port over to escape.go yet.
2) There are a lot of "leaking param: x to result ~r1 level=-1"
messages for code like
func(p *int) *T { return &T{p} }
that were simply wrong. Here &T must be heap allocated unconditionally
(because it's being returned); and since p is stored into it, p
escapes unconditionally too. esc.go incorrectly reports that p escapes
conditionally only if the returned pointer escaped.
3) esc.go used to print each "leaking param" analysis result as it
discovered them, which could lead to redundant messages (e.g., that a
param leaks at level=0 and level=1). escape.go instead prints
everything at the end, once it knows the shortest path to each sink.
4) esc.go didn't precisely model direct-interface types, resulting in
some values unnecessarily escaping to the heap when stored into
non-escaping interface values.
5) For functions written in assembly, esc.go only printed "does not
escape" messages, whereas escape.go prints "does not escape" or
"leaking param" as appropriate, consistent with the behavior for
functions written in Go.
6) 12 tests included "BAD" annotations identifying cases where esc.go
was unnecessarily heap allocating something. These are all fixed by
escape.go.
Updates #23109.
Change-Id: Iabc9eb14c94c9cadde3b183478d1fd54f013502f
Reviewed-on: https://go-review.googlesource.com/c/go/+/170447
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
For most nodes (e.g., OPTRLIT, OMAKESLICE, OCONVIFACE), escape
analysis prints "escapes to heap" or "does not escape" to indicate
whether that node's allocation can be heap or stack allocated.
These messages are also emitted for OADDR, even though OADDR does not
actually allocate anything itself. Moreover, it's redundant because
escape analysis already prints "moved to heap" diagnostics when an
OADDR node like "&x" causes x to require heap allocation.
Because OADDR nodes don't allocate memory, my escape analysis rewrite
doesn't naturally emit the "escapes to heap" / "does not escape"
diagnostics for them. It's also non-trivial to replicate the exact
semantics esc.go uses for OADDR.
Since there are so many of these messages, I'm disabling them in this
CL by themselves. I modified esc.go to suppress the Warnl calls
without any other behavior changes, and then used a shell script to
automatically remove any ERROR messages mentioned by run.go in
"missing error" or "no match for" lines.
Fixes#16300.
Updates #23109.
Change-Id: I3993e2743c3ff83ccd0893f4e73b366ff8871a57
Reviewed-on: https://go-review.googlesource.com/c/go/+/170319
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: David Chase <drchase@google.com>
Consider the following code:
func f(x []*T) interface{} {
return x
}
It returns an interface that holds a heap copy of x (by calling
convT2I or friend), therefore x escape to heap. The current
escape analysis only recognizes that x flows to the result. This
is not sufficient, since if the result does not escape, x's
content may be stack allocated and this will result a
heap-to-stack pointer, which is bad.
Fix this by realizing that if a CONVIFACE escapes and we're
converting from a non-direct interface type, the data needs to
escape to heap.
Running "toolstash -cmp" on std & cmd, the generated machine code
are identical for all packages. However, the export data (escape
tags) differ in the following packages. It looks to me that all
are similar to the "f" above, where the parameter should escape
to heap.
io/ioutil/ioutil.go:118
old: leaking param: r to result ~r1 level=0
new: leaking param: r
image/image.go:943
old: leaking param: p to result ~r0 level=1
new: leaking param content: p
net/url/url.go:200
old: leaking param: s to result ~r2 level=0
new: leaking param: s
(as a consequence)
net/url/url.go:183
old: leaking param: s to result ~r1 level=0
new: leaking param: s
net/url/url.go:194
old: leaking param: s to result ~r1 level=0
new: leaking param: s
net/url/url.go:699
old: leaking param: u to result ~r0 level=1
new: leaking param: u
net/url/url.go:775
old: (*URL).String u does not escape
new: leaking param content: u
net/url/url.go:1038
old: leaking param: u to result ~r0 level=1
new: leaking param: u
net/url/url.go:1099
old: (*URL).MarshalBinary u does not escape
new: leaking param content: u
flag/flag.go:235
old: leaking param: s to result ~r0 level=1
new: leaking param content: s
go/scanner/errors.go:105
old: leaking param: p to result ~r0 level=0
new: leaking param: p
database/sql/sql.go:204
old: leaking param: ns to result ~r0 level=0
new: leaking param: ns
go/constant/value.go:303
old: leaking param: re to result ~r2 level=0, leaking param: im to result ~r2 level=0
new: leaking param: re, leaking param: im
go/constant/value.go:846
old: leaking param: x to result ~r1 level=0
new: leaking param: x
encoding/xml/xml.go:518
old: leaking param: d to result ~r1 level=2
new: leaking param content: d
encoding/xml/xml.go:122
old: leaking param: leaking param: t to result ~r1 level=0
new: leaking param: t
crypto/x509/verify.go:506
old: leaking param: c to result ~r8 level=0
new: leaking param: c
crypto/x509/verify.go:563
old: leaking param: c to result ~r3 level=0, leaking param content: c
new: leaking param: c
crypto/x509/verify.go:615
old: (nothing)
new: leaking closure reference c
crypto/x509/verify.go:996
old: leaking param: c to result ~r1 level=0, leaking param content: c
new: leaking param: c
net/http/filetransport.go:30
old: leaking param: fs to result ~r1 level=0
new: leaking param: fs
net/http/h2_bundle.go:2684
old: leaking param: mh to result ~r0 level=2
new: leaking param content: mh
net/http/h2_bundle.go:7352
old: http2checkConnHeaders req does not escape
new: leaking param content: req
net/http/pprof/pprof.go:221
old: leaking param: name to result ~r1 level=0
new: leaking param: name
cmd/internal/bio/must.go:21
old: leaking param: w to result ~r1 level=0
new: leaking param: w
Fixes#29353.
Change-Id: I7e7798ae773728028b0dcae5bccb3ada51189c68
Reviewed-on: https://go-review.googlesource.com/c/162829
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
For OINDEX and other Left+Right nodes, we want the whole
node to be considered as "may affect memory" if either
of Left or Right affect memory. Initial implementation
only considered node as such if both Left and Right were non-safe.
Change-Id: Icfb965a0b4c24d8f83f3722216db068dad2eba95
Reviewed-on: https://go-review.googlesource.com/133275
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Teach escape analysis to recognize these assignment patterns
as not causing the src to leak:
val.x = val.y
val.x[i] = val.y[j]
val.x1.x2 = val.x1.y2
... etc
Helps to avoid "leaking param" with assignments showed above.
The implementation is based on somewhat similiar xs=xs[a:b]
special case that is ignored by the escape analysis.
We may figure out more generalized version of this,
but this one looks like a safe step into that direction.
Updates #14858
Change-Id: I6fe5bfedec9c03bdc1d7624883324a523bd11fde
Reviewed-on: https://go-review.googlesource.com/126395
Run-TryBot: Iskander Sharipov <iskander.sharipov@intel.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
This includes the following information in the per-function summary:
outK = paramJ encoded in outK bits for paramJ
outK = *paramJ encoded in outK bits for paramJ
heap = paramJ EscHeap
heap = *paramJ EscContentEscapes
Note that (currently) if the address of a parameter is taken and
returned, necessarily a heap allocation occurred to contain that
reference, and the heap can never refer to stack, therefore the
parameter and everything downstream from it escapes to the heap.
The per-function summary information now has a tuneable number of bits
(2 is probably noticeably better than 1, 3 is likely overkill, but it
is now easy to check and the -m debugging output includes information
that allows you to figure out if more would be better.)
A new test was added to check pointer flow through struct-typed and
*struct-typed parameters and returns; some of these are sensitive to
the number of summary bits, and ought to yield better results with a
more competent escape analysis algorithm. Another new test checks
(some) correctness with array parameters, results, and operations.
The old analysis inferred a piece of plan9 runtime was non-escaping by
counteracting overconservative analysis with buggy analysis; with the
bug fixed, the result was too conservative (and it's not easy to fix
in this framework) so the source code was tweaked to get the desired
result. A test was added against the discovered bug.
The escape analysis was further improved splitting the "level" into
3 parts, one tracking the conventional "level" and the other two
computing the highest-level-suffix-from-copy, which is used to
generally model the cancelling effect of indirection applied to
address-of.
With the improved escape analysis enabled, it was necessary to
modify one of the runtime tests because it now attempts to allocate
too much on the (small, fixed-size) G0 (system) stack and this
failed the test.
Compiling src/std after touching src/runtime/*.go with -m logging
turned on shows 420 fewer heap allocation sites (10538 vs 10968).
Profiling allocations in src/html/template with
for i in {1..5} ;
do go tool 6g -memprofile=mastx.${i}.prof -memprofilerate=1 *.go;
go tool pprof -alloc_objects -text mastx.${i}.prof ;
done
showed a 15% reduction in allocations performed by the compiler.
Update #3753
Update #4720Fixes#10466
Change-Id: I0fd97d5f5ac527b45f49e2218d158a6e89951432
Reviewed-on: https://go-review.googlesource.com/8202
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
