1
0
mirror of https://github.com/golang/go synced 2024-11-24 12:20:17 -07:00
Commit Graph

31881 Commits

Author SHA1 Message Date
Kevin Burke
c2eb06193f os/user: add non-cgo versions of Lookup, LookupId
If you cross compile for a Unix target and call user.Lookup("root")
or user.LookupId("0"), we'll try to read the answer out of
/etc/passwd instead of returning an "unimplemented" error.

The equivalent cgo function calls getpwuid_r in glibc, which
may reach out to the NSS database or allow callers to register
extensions. The pure Go implementation only reads from /etc/passwd.

Change-Id: I56a302d634b15ba5097f9f0d6a758c68e486ba6d
Reviewed-on: https://go-review.googlesource.com/37664
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-04 17:37:29 +00:00
Giovanni Bajo
4fc45ae879 cmd/compile: improve generic rules for BCE based on AND operations.
Match more patterns generated by the compiler where the index for
a bound check is bounded through a AND operation, with different
register sizes.

These rules trigger a dozen of times in a bootstrap.

Change-Id: Ic9fff16f21d08580f19a366c3ee1a372e58357d1
Reviewed-on: https://go-review.googlesource.com/37442
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-04 16:13:07 +00:00
Austin Clements
4a7cf960c3 runtime: make ReadMemStats STW for < 25µs
Currently ReadMemStats stops the world for ~1.7 ms/GB of heap because
it collects statistics from every single span. For large heaps, this
can be quite costly. This is particularly unfortunate because many
production infrastructures call this function regularly to collect and
report statistics.

Fix this by tracking the necessary cumulative statistics in the
mcaches. ReadMemStats still has to stop the world to stabilize these
statistics, but there are only O(GOMAXPROCS) mcaches to collect
statistics from, so this pause is only 25µs even at GOMAXPROCS=100.

Fixes #13613.

Change-Id: I3c0a4e14833f4760dab675efc1916e73b4c0032a
Reviewed-on: https://go-review.googlesource.com/34937
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-03-04 02:56:37 +00:00
Austin Clements
3399fd254d runtime: remove unused gcstats
The gcstats structure is no longer consumed by anything and no longer
tracks statistics that are particularly relevant to the concurrent
garbage collector. Remove it. (Having statistics is probably a good
idea, but these aren't the stats we need these days and we don't have
a way to get them out of the runtime.)

In preparation for #13613.

Change-Id: Ib63e2f9067850668f9dcbfd4ed89aab4a6622c3f
Reviewed-on: https://go-review.googlesource.com/34936
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-03-04 02:56:35 +00:00
Elias Naur
7523baed09 misc/ios,cmd/go, runtime/cgo: fix iOS test harness (again)
The iOS test harness was recently changed in response to lldb bugs
to replace breakpoints with the SIGUSR2 signal (CL 34926), and to
pass the current directory in the test binary arguments (CL 35152).
Both the signal sending and working directory setup is done from
the go test driver.

However, the new method doesn't work with tests where a C program is
the test driver instead of go test: the current working directory
will not be changed and SIGUSR2 is not raised.

Instead of copying that logic into any C test program, rework the
test harness (again) to move the setup logic to the early runtime
cgo setup code. That way, the harness will run even in the library
build modes.

Then, use the app Info.plist file to pass the working
directory, removing the need to alter the arguments after running.

Finally, use the SIGINT signal instead of SIGUSR2 to avoid
manipulating the signal masks or handlers.

Fixes the testcarchive tests on iOS.

With this CL, both darwin/arm and darwin/arm64 passes all.bash.

This CL replaces CL 34926, CL 35152 as well as the fixup CL
35123 and CL 35255. They are reverted in CLs earlier in the
relation chain.

Change-Id: I8485c7db1404fbd8daa261efd1ea89e905121a3e
Reviewed-on: https://go-review.googlesource.com/36090
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2017-03-04 01:43:13 +00:00
Elias Naur
fec40bd106 Revert "cmd/go, misc: switch from breakpoint to SIGUSR2"
This reverts commit 333f764df3.

Replaced by a improved strategy later in the CL relation chain.

Change-Id: I70a1d2f0aa5aa0d3d0ec85b5a956c6fb60d88908
Reviewed-on: https://go-review.googlesource.com/36069
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2017-03-04 01:21:47 +00:00
Josh Bleecher Snyder
03e2a4d1f1 cmd/compile: cull UINF
It was used with Node.Ullman, which is now gone.

Change-Id: I83b167645659ae7ef70043b7915d642e42ca524f
Reviewed-on: https://go-review.googlesource.com/37761
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-04 01:20:58 +00:00
Dmitri Shuralyov
e646d07329 go/build: fix lack of error for Import of nonexistent local import path
When calling build.Import, normally, an error is returned if the
directory doesn't exist. However, that didn't happen for local
import paths when build.FindOnly ImportMode was used.

This change fixes that, and adds tests. It also makes the error
value more consistent in all scenarios where it occurs.

When calling build.Import with a local import path, the package
can only exist in a single deterministic directory. That makes
it possible verify that directory exists earlier in the path,
and return a "cannot find package" error if it doesn't.
Previously, this occurred only when build.FindOnly ImportMode
was not set. It occurred quite late, after getting past Found
label, to line that calls ctxt.readDir. Doing so would return
an error like "no such file or directory" when the directory
does not exist.

Fixes #17863.
Updates #17888 (relevant issue I ran into while working on this CL).

Change-Id: If6a6996ac6176ac203a88bd31419748f88d89d7c
Reviewed-on: https://go-review.googlesource.com/33158
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-04 01:14:35 +00:00
Elias Naur
2dc714e1cf Revert "cmd/go: add comment about SIGUSR2 on iOS"
This reverts commit 4f0aac52d9.

Replaced by a improved strategy later in the CL relation chain.

Change-Id: Iff0333f172443bb5b01a42ad06b01edeb6aa15bc
Reviewed-on: https://go-review.googlesource.com/36068
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2017-03-04 00:46:31 +00:00
Elias Naur
3ce5371aaf Revert "cmd/go, misc: rework cwd handling for iOS tests"
This reverts commit 593ea3b360.

Replaced by a improved strategy later in the CL relation chain.

Change-Id: I6963e4d1bf38e7028cf545a953e28054d83548
Change-Id: I6963e4d1bf38e7028cf545a953e28054d8354870
Reviewed-on: https://go-review.googlesource.com/36067
Reviewed-by: David Crawshaw <crawshaw@golang.org>
2017-03-04 00:18:42 +00:00
Elias Naur
2b780af08e Revert "all: test adjustments for the iOS builder"
This reverts commit 467109bf56.

Replaced by a improved strategy later in the CL relation chain.

Change-Id: Ib90813b5a6c4716b563c8496013d2d57f9c022b8
Reviewed-on: https://go-review.googlesource.com/36066
Reviewed-by: David Crawshaw <crawshaw@golang.org>
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-04 00:01:14 +00:00
Matthew Dempsky
870d079c76 cmd/compile/internal/gc: replace Node.Ullman with Node.HasCall
Since switching to SSA, the only remaining use for the Ullman field
was in tracking whether or not an expression contained a function
call. Give it a new name and encode it in our fancy new bitset field.

Passes toolstash-check.

Change-Id: I95b7f9cb053856320c0d66efe14996667e6011c2
Reviewed-on: https://go-review.googlesource.com/37721
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-03-03 22:35:44 +00:00
David Lazar
9fd359a29a cmd/compile: include position info in exported function bodies
This gives accurate line numbers to inlined functions from another
package. Previously AST nodes from another package would get the line
number of the import statement for that package.

The following benchmark results show how the size of package export data
is impacted by this change. These benchmarks were created by compiling
the go1 benchmark and running `go tool pack x` to extract the export
data from the resulting .a files.

name                                          old bytes   new bytes    delta
bufio                                         3.59k ± 0%   4.17k ± 0%  +16.25%
bytes                                         5.51k ± 0%   6.40k ± 0%  +16.21%
compress/bzip2                                2.69k ± 0%   3.21k ± 0%  +19.74%
compress/flate                                5.14k ± 0%   5.57k ± 0%   +8.43%
compress/gzip                                 8.91k ± 0%  10.46k ± 0%  +17.32%
container/list                                1.76k ± 0%   2.13k ± 0%  +21.51%
context                                       4.51k ± 0%   5.47k ± 0%  +21.43%
crypto                                        1.11k ± 0%   1.13k ± 0%   +1.90%
crypto/aes                                      475 ± 0%     475 ± 0%   +0.00%
crypto/cipher                                 1.18k ± 0%   1.18k ± 0%   +0.00%
crypto/des                                      502 ± 0%     502 ± 0%   +0.00%
crypto/dsa                                    5.96k ± 0%   6.54k ± 0%   +9.82%
crypto/ecdsa                                  6.93k ± 0%   7.69k ± 0%  +10.91%
crypto/elliptic                               6.53k ± 0%   7.17k ± 0%   +9.72%
crypto/hmac                                     464 ± 0%     464 ± 0%   +0.00%
crypto/internal/cipherhw                        313 ± 0%     313 ± 0%   +0.00%
crypto/md5                                      695 ± 0%     711 ± 0%   +2.30%
crypto/rand                                   5.62k ± 0%   6.21k ± 0%  +10.44%
crypto/rc4                                      512 ± 0%     512 ± 0%   +0.00%
crypto/rsa                                    7.31k ± 0%   8.10k ± 0%  +10.86%
crypto/sha1                                     760 ± 0%     777 ± 0%   +2.24%
crypto/sha256                                   523 ± 0%     523 ± 0%   +0.00%
crypto/sha512                                   663 ± 0%     663 ± 0%   +0.00%
crypto/subtle                                   873 ± 0%    1007 ± 0%  +15.35%
crypto/tls                                    29.6k ± 0%   33.8k ± 0%  +14.03%
crypto/x509                                   18.7k ± 0%   21.0k ± 0%  +12.56%
crypto/x509/pkix                              10.6k ± 0%   12.2k ± 0%  +15.22%
encoding                                        473 ± 0%     473 ± 0%   +0.00%
encoding/asn1                                 1.42k ± 0%   1.50k ± 0%   +5.99%
encoding/base64                               1.69k ± 0%   1.80k ± 0%   +6.88%
encoding/binary                               2.76k ± 0%   3.51k ± 0%  +27.09%
encoding/gob                                  13.5k ± 0%   15.2k ± 0%  +12.98%
encoding/hex                                    857 ± 0%     881 ± 0%   +2.80%
encoding/json                                 12.5k ± 0%   14.9k ± 0%  +19.37%
encoding/pem                                    484 ± 0%     484 ± 0%   +0.00%
errors                                          361 ± 0%     370 ± 0%   +2.49%
flag                                          10.5k ± 0%   12.1k ± 0%  +14.92%
fmt                                           1.42k ± 0%   1.42k ± 0%   +0.00%
go/ast                                        15.8k ± 0%   17.5k ± 0%  +10.31%
go/parser                                     8.13k ± 0%   9.86k ± 0%  +21.28%
go/scanner                                    3.94k ± 0%   4.53k ± 0%  +14.73%
go/token                                      3.53k ± 0%   3.75k ± 0%   +6.11%
hash                                            507 ± 0%     507 ± 0%   +0.00%
hash/crc32                                      685 ± 0%     685 ± 0%   +0.00%
internal/nettrace                               474 ± 0%     474 ± 0%   +0.00%
internal/poll                                 7.23k ± 0%   8.38k ± 0%  +15.90%
internal/race                                   511 ± 0%     515 ± 0%   +0.78%
internal/singleflight                           969 ± 0%    1075 ± 0%  +10.94%
internal/syscall/unix                           427 ± 0%     427 ± 0%   +0.00%
io                                            3.52k ± 0%   3.69k ± 0%   +4.82%
io/ioutil                                     8.48k ± 0%   9.90k ± 0%  +16.72%
log                                           5.06k ± 0%   5.98k ± 0%  +18.15%
math                                          4.02k ± 0%   4.35k ± 0%   +8.26%
math/big                                      9.28k ± 0%   9.94k ± 0%   +7.13%
math/bits                                     3.47k ± 0%   4.33k ± 0%  +24.83%
math/rand                                     1.30k ± 0%   1.32k ± 0%   +2.00%
mime                                          2.98k ± 0%   3.70k ± 0%  +24.21%
mime/multipart                                3.68k ± 0%   4.22k ± 0%  +14.65%
mime/quotedprintable                          2.26k ± 0%   2.65k ± 0%  +17.60%
net                                           23.0k ± 0%   25.7k ± 0%  +11.74%
net/http                                      59.1k ± 0%   66.7k ± 0%  +13.00%
net/http/httptest                             35.3k ± 0%   40.9k ± 0%  +15.80%
net/http/httptrace                            15.3k ± 0%   17.6k ± 0%  +15.26%
net/http/internal                             2.77k ± 0%   3.27k ± 0%  +17.89%
net/textproto                                 4.60k ± 0%   5.25k ± 0%  +14.22%
net/url                                       1.73k ± 0%   1.84k ± 0%   +6.59%
os                                            14.3k ± 0%   16.4k ± 0%  +14.86%
path                                            589 ± 0%     606 ± 0%   +2.89%
path/filepath                                 5.07k ± 0%   6.17k ± 0%  +21.79%
reflect                                       6.43k ± 0%   6.81k ± 0%   +5.90%
regexp                                        5.88k ± 0%   6.46k ± 0%   +9.77%
regexp/syntax                                 3.24k ± 0%   3.29k ± 0%   +1.73%
runtime                                       13.1k ± 0%   14.9k ± 0%  +13.73%
runtime/cgo                                     229 ± 0%     229 ± 0%   +0.00%
runtime/debug                                 4.23k ± 0%   5.15k ± 0%  +21.79%
runtime/internal/atomic                         905 ± 0%     905 ± 0%   +0.00%
runtime/internal/sys                          2.04k ± 0%   2.20k ± 0%   +7.64%
runtime/pprof                                 4.73k ± 0%   5.65k ± 0%  +19.41%
runtime/trace                                   354 ± 0%     354 ± 0%   +0.00%
sort                                          1.68k ± 0%   1.85k ± 0%  +10.17%
strconv                                       1.85k ± 0%   1.95k ± 0%   +5.51%
strings                                       3.98k ± 0%   4.53k ± 0%  +13.91%
sync                                          1.52k ± 0%   1.58k ± 0%   +4.28%
sync/atomic                                   1.60k ± 0%   1.74k ± 0%   +8.50%
syscall                                       53.3k ± 0%   54.3k ± 0%   +1.84%
testing                                       8.77k ± 0%  10.09k ± 0%  +14.96%
testing/internal/testdeps                       598 ± 0%     600 ± 0%   +0.33%
text/tabwriter                                3.63k ± 0%   4.41k ± 0%  +21.64%
text/template                                 15.7k ± 0%   18.1k ± 0%  +15.67%
text/template/parse                           9.12k ± 0%  10.35k ± 0%  +13.48%
time                                          6.38k ± 0%   7.14k ± 0%  +11.81%
unicode                                       4.62k ± 0%   4.66k ± 0%   +0.98%
unicode/utf16                                   707 ± 0%     791 ± 0%  +11.88%
unicode/utf8                                  1.06k ± 0%   1.20k ± 0%  +12.63%
vendor/golang_org/x/crypto/chacha20poly1305   1.26k ± 0%   1.43k ± 0%  +13.54%
vendor/golang_org/x/crypto/curve25519           392 ± 0%     392 ± 0%   +0.00%
vendor/golang_org/x/crypto/poly1305             426 ± 0%     426 ± 0%   +0.00%
vendor/golang_org/x/net/http2/hpack           4.75k ± 0%   5.77k ± 0%  +21.42%
vendor/golang_org/x/net/idna                    355 ± 0%     355 ± 0%   +0.00%
vendor/golang_org/x/net/lex/httplex             616 ± 0%     644 ± 0%   +4.55%
vendor/golang_org/x/net/proxy                 7.76k ± 0%   9.58k ± 0%  +23.37%
vendor/golang_org/x/text/transform            1.31k ± 0%   1.32k ± 0%   +0.46%
vendor/golang_org/x/text/unicode/norm         5.89k ± 0%   6.84k ± 0%  +16.06%
vendor/golang_org/x/text/width                1.24k ± 0%   1.27k ± 0%   +2.66%
[Geo mean]                                    2.51k        2.74k        +9.14%

Change-Id: I9ded911bb0ff63c530795fc85253d76b56d8abbc
Reviewed-on: https://go-review.googlesource.com/37239
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2017-03-03 21:29:40 +00:00
David Lazar
0824ae6dc1 cmd/compile: add flag for debugging PC-value tables
For example, `-d pctab=pctoinline` prints the PC-inline table and
inlining tree for every function.

Change-Id: Ia6b9ce4d83eed0b494318d40ffe06481ec5d58ab
Reviewed-on: https://go-review.googlesource.com/37235
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2017-03-03 21:29:38 +00:00
David Lazar
301149b9e4 cmd/internal/obj: avoid duplicate file name symbols
The meaning of Version=1 was overloaded: it was reserved for file name
symbols (to avoid conflicts with non-file name symbols), but was also
used to mean "give me a fresh version number for this symbol."

With the new inlining tree, the same file name symbol can appear in
multiple entries, but each one would become a distinct symbol with its
own version number.

Now, we avoid duplicating symbols by using Version=0 for file name
symbols and we avoid conflicts with other symbols by prefixing the
symbol name with "gofile..".

Change-Id: I8d0374053b8cdb6a9ca7fb71871b69b4dd369a9c
Reviewed-on: https://go-review.googlesource.com/37234
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
2017-03-03 21:29:36 +00:00
David Lazar
781fd3998e runtime: use inlining tables to generate accurate tracebacks
The code in https://play.golang.org/p/aYQPrTtzoK now produces the
following stack trace:

goroutine 1 [running]:
main.(*point).negate(...)
	/tmp/go/main.go:8
main.main()
	/tmp/go/main.go:14 +0x23

Previously the stack trace missed the inlined call:

goroutine 1 [running]:
main.main()
	/tmp/go/main.go:14 +0x23

Fixes #10152.
Updates #19348.

Change-Id: Ib43c67012f53da0ef1a1e69bcafb65b57d9cecb2
Reviewed-on: https://go-review.googlesource.com/37233
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2017-03-03 21:29:34 +00:00
David Lazar
1c6ef9aeed cmd/compile: copy literals when inlining
Without this, literals keep their original source positions through
inlining, which results in strange jumps in line numbers of inlined
function bodies. By copying literals, inlining can update their source
position like other nodes.

Fixes #15453.

Change-Id: Iad5d9bbfe183883794213266dc30e31bab89ee69
Reviewed-on: https://go-review.googlesource.com/37232
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Russ Cox <rsc@golang.org>
2017-03-03 21:29:32 +00:00
David Lazar
699175a11a cmd/compile,link: generate PC-value tables with inlining information
In order to generate accurate tracebacks, the runtime needs to know the
inlined call stack for a given PC. This creates two tables per function
for this purpose. The first table is the inlining tree (stored in the
function's funcdata), which has a node containing the file, line, and
function name for every inlined call. The second table is a PC-value
table that maps each PC to a node in the inlining tree (or -1 if the PC
is not the result of inlining).

To give the appearance that inlining hasn't happened, the runtime also
needs the original source position information of inlined AST nodes.
Previously the compiler plastered over the line numbers of inlined AST
nodes with the line number of the call. This meant that the PC-line
table mapped each PC to line number of the outermost call in its inlined
call stack, with no way to access the innermost line number.

Now the compiler retains line numbers of inlined AST nodes and writes
the innermost source position information to the PC-line and PC-file
tables. Some tools and tests expect to see outermost line numbers, so we
provide the OutermostLine function for displaying line info.

To keep track of the inlined call stack for an AST node, we extend the
src.PosBase type with an index into a global inlining tree. Every time
the compiler inlines a call, it creates a node in the global inlining
tree for the call, and writes its index to the PosBase of every inlined
AST node. The parent of this node is the inlining tree index of the
call. -1 signifies no parent.

For each function, the compiler creates a local inlining tree and a
PC-value table mapping each PC to an index in the local tree.  These are
written to an object file, which is read by the linker.  The linker
re-encodes these tables compactly by deduplicating function names and
file names.

This change increases the size of binaries by 4-5%. For example, this is
how the go1 benchmark binary is impacted by this change:

section             old bytes   new bytes   delta
.text               3.49M ± 0%  3.49M ± 0%   +0.06%
.rodata             1.12M ± 0%  1.21M ± 0%   +8.21%
.gopclntab          1.50M ± 0%  1.68M ± 0%  +11.89%
.debug_line          338k ± 0%   435k ± 0%  +28.78%
Total               9.21M ± 0%  9.58M ± 0%   +4.01%

Updates #19348.

Change-Id: Ic4f180c3b516018138236b0c35e0218270d957d3
Reviewed-on: https://go-review.googlesource.com/37231
Run-TryBot: David Lazar <lazard@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2017-03-03 21:29:30 +00:00
Aliaksandr Valialkin
ed70f37e73 cmd/compile: pack bool fields in Node, Name, Func and Type structs to bitsets
This reduces compiler memory usage by up to 4% - see compilebench
results below.

name       old time/op     new time/op     delta
Template       245ms ± 4%      241ms ± 2%  -1.88%  (p=0.029 n=10+10)
Unicode        126ms ± 3%      124ms ± 3%    ~     (p=0.105 n=10+10)
GoTypes        805ms ± 2%      813ms ± 3%    ~     (p=0.515 n=8+10)
Compiler       3.95s ± 2%      3.83s ± 1%  -2.96%  (p=0.000 n=9+10)
MakeBash       47.4s ± 4%      46.6s ± 1%  -1.59%  (p=0.028 n=9+10)

name       old user-ns/op  new user-ns/op  delta
Template        324M ± 5%       326M ± 3%    ~     (p=0.935 n=10+10)
Unicode         186M ± 5%       178M ±10%    ~     (p=0.067 n=9+10)
GoTypes        1.08G ± 7%      1.09G ± 4%    ~     (p=0.956 n=10+10)
Compiler       5.34G ± 4%      5.31G ± 1%    ~     (p=0.501 n=10+8)

name       old alloc/op    new alloc/op    delta
Template      41.0MB ± 0%     39.8MB ± 0%  -3.03%  (p=0.000 n=10+10)
Unicode       32.3MB ± 0%     31.0MB ± 0%  -4.13%  (p=0.000 n=10+10)
GoTypes        119MB ± 0%      116MB ± 0%  -2.39%  (p=0.000 n=10+10)
Compiler       499MB ± 0%      487MB ± 0%  -2.48%  (p=0.000 n=10+10)

name       old allocs/op   new allocs/op   delta
Template        380k ± 1%       379k ± 1%    ~     (p=0.436 n=10+10)
Unicode         324k ± 1%       324k ± 0%    ~     (p=0.853 n=10+10)
GoTypes        1.15M ± 0%      1.15M ± 0%    ~     (p=0.481 n=10+10)
Compiler       4.41M ± 0%      4.41M ± 0%  -0.12%  (p=0.007 n=10+10)

name       old text-bytes  new text-bytes  delta
HelloSize       623k ± 0%       623k ± 0%    ~     (all equal)
CmdGoSize      6.64M ± 0%      6.64M ± 0%    ~     (all equal)

name       old data-bytes  new data-bytes  delta
HelloSize      5.81k ± 0%      5.81k ± 0%    ~     (all equal)
CmdGoSize       238k ± 0%       238k ± 0%    ~     (all equal)

name       old bss-bytes   new bss-bytes   delta
HelloSize       134k ± 0%       134k ± 0%    ~     (all equal)
CmdGoSize       152k ± 0%       152k ± 0%    ~     (all equal)

name       old exe-bytes   new exe-bytes   delta
HelloSize       967k ± 0%       967k ± 0%    ~     (all equal)
CmdGoSize      10.2M ± 0%      10.2M ± 0%    ~     (all equal)

Change-Id: I1f40af738254892bd6c8ba2eb43390b175753d52
Reviewed-on: https://go-review.googlesource.com/37445
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-03 21:06:03 +00:00
Johan Brandhorst
fbf4dd91b9 net/http/httptest: add Client and Certificate methods to Server
Adds a function for easily accessing the x509.Certificate
of a Server, if there is one. Also adds a helper function
for getting a http.Client suitable for use with the server.

This makes the steps required to test a httptest
TLS server simpler.

Fixes #18411

Change-Id: I2e78fe1e54e31bed9c641be2d9a099f698c7bbde
Reviewed-on: https://go-review.googlesource.com/34639
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-03-03 21:02:17 +00:00
Matthew Dempsky
02e36f8c87 cmd/compile/internal/ssa: remove Hmul{8,16}{,u} ops
Change-Id: I90865921584ae4bdfb6c220d439b14593d72b6f9
Reviewed-on: https://go-review.googlesource.com/37752
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2017-03-03 20:47:36 +00:00
Cherry Zhang
c8eaeb8cba cmd/compile: remove zeroing after newobject
The Zero op right after newobject has been removed. But this rule
does not cover Store of constant zero (for SSA-able types). Add
rules to cover Store op as well.

Updates #19027.

Change-Id: I5d2b62eeca0aa9ce8dc7205b264b779de01c660b
Reviewed-on: https://go-review.googlesource.com/36836
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-03 20:36:54 +00:00
Cherry Zhang
9b480521d8 cmd/compile: fix optimization of Zero newobject on amd64p32
On amd64p32, PtrSize and RegSize don't agree, and function return
value is aligned with RegSize. Fix this rule. Other architectures
are not affected, where PtrSize and RegSize are the same.

Change-Id: If187d3dfde3dc3b931b8e97db5eeff49a781551b
Reviewed-on: https://go-review.googlesource.com/37720
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-03 20:36:31 +00:00
Matthew Dempsky
d8a0f74801 cmd/compile/internal/gc: remove OHMUL Op
Previously the compiler rewrote constant division into OHMUL
operations, but that rewriting was moved to SSA in CL 37015. Now OHMUL
is unused, so we can get rid of it.

Change-Id: Ib6fc7c2b6435510bafb5735b3b4f42cfd8ed8cdb
Reviewed-on: https://go-review.googlesource.com/37750
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-03 17:47:53 +00:00
Austin Clements
77f64c50db runtime: clarify access to mheap_.busy
There are two accesses to mheap_.busy that are guarded by checks
against len(mheap_.free). This works because both lists are (and must
be) the same length, but it makes the code less clear. Change these to
use len(mheap_.busy) so the access more clearly parallels the check.

Fixes #18944.

Change-Id: I9bacbd3663988df351ed4396ae9018bc71018311
Reviewed-on: https://go-review.googlesource.com/36354
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-03-03 17:02:18 +00:00
Austin Clements
b50b728587 runtime: simplify sweep allocation counting
Currently sweep counts the number of allocated objects, computes the
number of free objects from that, then re-computes the number of
allocated objects from that. Simplify and clean this up by skipping
these intermediate steps.

Change-Id: I3ed98e371eb54bbcab7c8530466c4ab5fde35f0a
Reviewed-on: https://go-review.googlesource.com/34935
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Marvin Stenger <marvin.stenger94@gmail.com>
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-03-03 17:02:16 +00:00
Austin Clements
f1ba75f8c5 runtime: don't rescan finalizers queue during mark termination
Currently we scan the finalizers queue both during concurrent mark and
during mark termination. This costs roughly 20ns per queued finalizer
and about 1ns per unused finalizer queue slot (allocated queue length
never decreases), which can drive up STW time if there are many
finalizers.

However, we only add finalizers to this queue during sweeping, which
means that the second scan will never find anything new. Hence, we can
fix this by simply not scanning the finalizers queue during mark
termination. This brings the STW time under the 100µs goal even with
1,000,000 queued finalizers.

Fixes #18869.

Change-Id: I4ce5620c66fb7f13ebeb39ca313ce57047d1d0fb
Reviewed-on: https://go-review.googlesource.com/36013
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-03-03 17:02:14 +00:00
Austin Clements
98da2d1f91 runtime: remove wbufptr
Since workbuf is now marked go:notinheap, the write barrier-preventing
wrapper type wbufptr is no longer necessary. Remove it.

Change-Id: I3e5b5803a1547d65de1c1a9c22458a38e08549b7
Reviewed-on: https://go-review.googlesource.com/35971
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2017-03-03 17:02:12 +00:00
Austin Clements
8eb14e9de5 cmd/compile: accept string debug flags
The compiler's -d flag accepts string-valued flags, but currently only
for SSA debug flags. Extend it to support string values for other
flags. This also makes the syntax somewhat more sane so flag=value and
flag:value now both accept integers and strings.

Change-Id: Idd144d8479a430970cc1688f824bffe0a56ed2df
Reviewed-on: https://go-review.googlesource.com/37345
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2017-03-03 15:50:49 +00:00
Cherry Zhang
5bfd1ef036 cmd/compile: get rid of "volatile" in SSA
A value is "volatile" if it is a pointer to the argument region
on stack which will be clobbered by function call. This is used
to make sure the value is safe when inserting write barrier calls.
The writebarrier pass can tell whether a value is such a pointer.
Therefore no need to mark it when building SSA and thread this
information through.

Passes "toolstash -cmp" on std.

Updates #17583.

Change-Id: Idc5fc0d710152b94b3c504ce8db55ea9ff5b5195
Reviewed-on: https://go-review.googlesource.com/36835
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-03 13:26:15 +00:00
Will Storey
4775b7feb1 image/gif: handle an extra data sub-block byte.
This changes the decoder's behaviour when there is stray/extra data
found after an image is decompressed (e.g., data sub-blocks after an LZW
End of Information Code). Instead of raising an error, we silently skip
over such data until we find the end of the image data marked by a Block
Terminator. We skip at most one byte as sample problem GIFs exhibit this
property.

GIFs should not have and do not need such stray data (though the
specification is arguably ambiguous). However GIFs with such properties
have been seen in the wild.

Fixes #16146

Change-Id: Ie7e69052bab5256b4834992304e6ca58e93c1879
Reviewed-on: https://go-review.googlesource.com/37258
Reviewed-by: Nigel Tao <nigeltao@golang.org>
Run-TryBot: Nigel Tao <nigeltao@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 23:49:32 +00:00
Josh Bleecher Snyder
9b15c13dc5 runtime/pprof: fix data race between Profile.Add and Profile.WriteTo
p.m is accessed in WriteTo without holding p.mu.
Move the access inside the critical section.

The race detector catches this bug using this program:


package main

import (
	"os"
	"runtime/pprof"
	"time"
)

func main() {
	p := pprof.NewProfile("ABC")
	go func() {
		p.WriteTo(os.Stdout, 1)
		time.Sleep(time.Second)
	}()
	p.Add("abc", 0)
	time.Sleep(time.Second)
}


$ go run -race x.go 
==================
WARNING: DATA RACE
Write at 0x00c42007c240 by main goroutine:
  runtime.mapassign()
      /Users/josh/go/tip/src/runtime/hashmap.go:485 +0x0
  runtime/pprof.(*Profile).Add()
      /Users/josh/go/tip/src/runtime/pprof/pprof.go:281 +0x255
  main.main()
      /Users/josh/go/tip/src/p.go:15 +0x9d

Previous read at 0x00c42007c240 by goroutine 6:
  runtime/pprof.(*Profile).WriteTo()
      /Users/josh/go/tip/src/runtime/pprof/pprof.go:314 +0xc5
  main.main.func1()
      /Users/josh/go/tip/src/x.go:12 +0x69

Goroutine 6 (running) created at:
  main.main()
      /Users/josh/go/tip/src/x.go:11 +0x6e
==================
ABC profile: total 1
1 @ 0x110ccb4 0x111aeee 0x1055053 0x107f031

Found 1 data race(s)
exit status 66


(Exit status 66?)

Change-Id: I49d884dc3af9cce2209057a3448fe6bf50653523
Reviewed-on: https://go-review.googlesource.com/37730
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-03-02 23:30:07 +00:00
Robert Griesemer
72359cf840 go/types: don't exclude package unsafe from a Package's Imports list
There's no good reason to exclude it and it only makes the code more
complicated and less consistent. Having it in the list provides an
easy way to detect if a package uses operations from package unsafe.

Change-Id: I2f9b0485db0a680bd82f3b93a350b048db3f7701
Reviewed-on: https://go-review.googlesource.com/37694
Reviewed-by: Alan Donovan <adonovan@google.com>
2017-03-02 22:43:41 +00:00
Robert Griesemer
359ca5ccc8 go/types: support type checking of external tests with gotype
- renamed -a flag to -t
- added -x flag to specify external test files
- improved documentation and usage string

Change-Id: I7c850bd28a10ceaa55d599c22db07774147aa3f7
Reviewed-on: https://go-review.googlesource.com/37656
Reviewed-by: Alan Donovan <adonovan@google.com>
2017-03-02 22:41:26 +00:00
Robert Griesemer
9eac1c87a6 go/types: gotype to always report the same first error now
The old code may have reported different errors given an
erroneous package depending on the order in which files
were parsed concurrently. The new code always reports
errors in "file order", independent of processing order.

Also:
- simplified parsing code and internal concurrency control
- removed -seq flag which didn't really add useful functionality

Change-Id: I18e24e630f458f2bc107a7b83926ae761d63c334
Reviewed-on: https://go-review.googlesource.com/37655
Reviewed-by: Alan Donovan <adonovan@google.com>
2017-03-02 22:29:45 +00:00
Josh Bleecher Snyder
3a90bfb253 cmd/dist, cmd/compile: eliminate mergeEnvLists copies
This is now handled by os/exec.

Updates #12868

Change-Id: Ic21a6ff76a9b9517437ff1acf3a9195f9604bb45
Reviewed-on: https://go-review.googlesource.com/37698
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-03-02 22:26:23 +00:00
Brad Fitzpatrick
9d29be468e net/http: clean up Transport.RoundTrip error handling
If I put a 10 millisecond sleep at testHookWaitResLoop, before the big
select in (*persistConn).roundTrip, two flakes immediately started
happening, TestTransportBodyReadError (#19231) and
TestTransportPersistConnReadLoopEOF.

The problem was that there are many ways for a RoundTrip call to fail
(errors reading from Request.Body while writing the response, errors
writing the response, errors reading the response due to server
closes, errors due to servers sending malformed responses,
cancelations, timeouts, etc.), and many of those failures then tear
down the TCP connection, causing more failures, since there are always
at least three goroutines involved (reading, writing, RoundTripping).

Because the errors were communicated over buffered channels to a giant
select, the error returned to the caller was a function of which
random select case was called, which was why a 10ms delay before the
select brought out so many bugs. (several fixed in my previous CLs the past
few days).

Instead, track the error explicitly in the transportRequest, guarded
by a mutex.

In addition, this CL now:

* differentiates between the two ways writing a request can fail: the
  io.Copy reading from the Request.Body or the io.Copy writing to the
  network. A new io.Reader type notes read errors from the
  Request.Body. The read-from-body vs write-to-network errors are now
  prioritized differently.

* unifies the two mapRoundTripErrorFromXXX methods into one
  mapRoundTripError method since their logic is now the same.

* adds a (*Request).WithT(*testing.T) method in export_test.go, usable
  by tests, to call t.Logf at points during RoundTrip. This is disabled
  behind a constant except when debugging.

* documents and deflakes TestClientRedirectContext

I've tested this CL with high -count values, with/without -race,
with/without delays before the select, etc. So far it seems robust.

Fixes #19231 (TestTransportBodyReadError flake)
Updates #14203 (source of errors unclear; they're now tracked more)
Updates #15935 (document Transport errors more; at least understood more now)

Change-Id: I3cccc3607f369724b5344763e35ad2b7ea415738
Reviewed-on: https://go-review.googlesource.com/37495
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2017-03-02 22:06:09 +00:00
Mike Danese
87649d32ad crypto/tls: make Config.Clone also clone the GetClientCertificate field
Using GetClientCertificate with the http client is currently completely
broken because inside the transport we clone the tls.Config and pass it
off to the tls.Client. Since tls.Config.Clone() does not pass forward
the GetClientCertificate field, GetClientCertificate is ignored in this
context.

Fixes #19264

Change-Id: Ie214f9f0039ac7c3a2dab8ffd14d30668bdb4c71
Signed-off-by: Mike Danese <mikedanese@google.com>
Reviewed-on: https://go-review.googlesource.com/37541
Reviewed-by: Filippo Valsorda <hi@filippo.io>
Reviewed-by: Adam Langley <agl@golang.org>
Run-TryBot: Adam Langley <agl@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 19:43:07 +00:00
Josh Bleecher Snyder
2f5aea7c13 Revert "Revert "cmd/vet/all: remove pprof from the whitelist""
This reverts commit 9bd1cc3fa1.

Reason for revert: New fixes in from upstream. Try this again.

Change-Id: Iea46f32857e8467f8d5a49b31e20a52fda8bce60
Reviewed-on: https://go-review.googlesource.com/37693
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2017-03-02 19:10:21 +00:00
Josh Bleecher Snyder
85bae0a9df cmd/vet/all: remove printf hacks
Now that vet loads from source,
fmt can always be correctly resolved,
so the fmt.Formatter type is always available,
so we can reinstate the check.

Change-Id: I17f0c7fccf6960c9415de8774b15123135d57be8
Reviewed-on: https://go-review.googlesource.com/37692
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 18:43:40 +00:00
Josh Bleecher Snyder
8a93546d68 cmd/vet/all: vet using only source
This simplifies the code and speeds it up.
It also allows us to eliminate some other TODOs;
those will come in a follow-up CL.

Running for the host platform, before:

real	0m9.907s
user	0m14.566s
sys	0m1.058s

After:

real	0m7.841s
user	0m12.339s
sys	0m0.572s

Running for a single non-host platform, before:

real	0m8.784s
user	0m15.451s
sys	0m3.445s

After:

real	0m7.681s
user	0m12.122s
sys	0m0.577s

Running for all platforms, before:

real	7m4.480s
user	8m43.398s
sys	1m15.683s

After:

real	4m37.596s
user	7m30.729s
sys	0m18.533s

It also makes my laptop considerably more
responsive while running for all platforms.

Change-Id: I748689fea0d2d4ef61aca2ce5524d03d8fafa5ca
Reviewed-on: https://go-review.googlesource.com/37691
Reviewed-by: Rob Pike <r@golang.org>
2017-03-02 18:43:33 +00:00
Josh Bleecher Snyder
ddbee9abd4 cmd/vet: support importing from source
Add a -source flag to cmd/vet that instructs
it to typecheck purely from source code.

Updates #16086
Fixes #19332

Change-Id: Ic83d0f14d5bb837a329d539b2873aeccdf7bf669
Reviewed-on: https://go-review.googlesource.com/37690
Reviewed-by: Rob Pike <r@golang.org>
2017-03-02 18:43:23 +00:00
Josh Bleecher Snyder
7e74d43291 cmd/vet: refactor to support alternative importers
Instead of constructing the importer in init, do it lazily as needed.
This lets us select the importer using a command line flag.
The addition of the command line flag will come in a follow-up CL.

Change-Id: Ieb3a5f01a34fb5bd220a95086daf5d6b37e83bb5
Reviewed-on: https://go-review.googlesource.com/37669
Reviewed-by: Rob Pike <r@golang.org>
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 18:43:12 +00:00
Heschi Kreinick
4de2efe927 cmd/trace: traces may end with pending mark assists
There's no guarantee that all in-progress mark assists will finish
before the trace does. Don't crash if that happens.

I haven't added a test because there's quite a bit of ceremony involved
and the bug is fairly straightforward.

Change-Id: Ia1369a8e2260fc6a328ad204a1eab1063d2e2c90
Reviewed-on: https://go-review.googlesource.com/37540
Reviewed-by: Austin Clements <austin@google.com>
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 18:33:54 +00:00
Josh Bleecher Snyder
04fc887761 runtime: delay marking maps as writing until after first alg call
Fixes #19359

Change-Id: I196b47cf0471915b6dc63785e8542aa1876ff695
Reviewed-on: https://go-review.googlesource.com/37665
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-02 17:38:30 +00:00
Matthew Dempsky
0ee9c46cb1 cmd/compile: add missing WBs for reflect.{Slice,String}Header.Data
Fixes #19168.

Change-Id: I3f4fcc0b189c53819ac29ef8de86fdad76a17488
Reviewed-on: https://go-review.googlesource.com/37663
Reviewed-by: Keith Randall <khr@golang.org>
2017-03-02 17:21:50 +00:00
Josh Bleecher Snyder
3d77bc081d cmd/vet/all: use SizesFor to calculate archbits
Change-Id: I99706807782f11e8d24baf953424a9e292a2cbac
Reviewed-on: https://go-review.googlesource.com/37668
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 17:13:44 +00:00
Josh Bleecher Snyder
99fbccbd93 cmd/vet: use types.SizesFor
This eliminates a duplicate copy of
the SizesFor map.

Change-Id: I51e44ea8ee860901086616e3f4dfa32aaa9b4d2d
Reviewed-on: https://go-review.googlesource.com/37667
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2017-03-02 17:13:36 +00:00
Josh Bleecher Snyder
32a1736d24 go/types: add a compiler param to SizesFor
The current StdSizes most closely matches
the gc compiler, and the uses I know of that care
which compiler the sizes are for are all for
the gc compiler, so call the existing
implementation "gc".

Updates #17586
Fixes #19351

Change-Id: I2bdd694518fbe233473896321a1f9758b46ed79b
Reviewed-on: https://go-review.googlesource.com/37666
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2017-03-02 17:13:28 +00:00
Josh Bleecher Snyder
542a60fbde cmd/compile: don't crash when slicing non-slice
Fixes #19323

Change-Id: I92d1bdefb15de6178a577a4fa0f0dc004f791904
Reviewed-on: https://go-review.googlesource.com/37584
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2017-03-02 15:48:19 +00:00