1
0
mirror of https://github.com/golang/go synced 2024-11-23 06:20:07 -07:00
Commit Graph

38566 Commits

Author SHA1 Message Date
Austin Clements
5333550bdc runtime: implement efficient page reclaimer
When we attempt to allocate an N page span (either for a large
allocation or when an mcentral runs dry), we first try to sweep spans
to release N pages. Currently, this can be extremely expensive:
sweeping a span to emptiness is the hardest thing to ask for and the
sweeper generally doesn't know where to even look for potentially
fruitful results. Since this is on the critical path of many
allocations, this is unfortunate.

This CL changes how we reclaim empty spans. Instead of trying lots of
spans and hoping for the best, it uses the newly introduced span marks
to efficiently find empty spans. The span marks (and in-use bits) are
in a dense bitmap, so these spans can be found with an efficient
sequential memory scan. This approach can scan for unmarked spans at
about 300 GB/ms and can free unmarked spans at about 32 MB/ms. We
could probably significantly improve the rate at which is can free
unmarked spans, but that's a separate issue.

Like the current reclaimer, this is still linear in the number of
spans that are swept, but the constant factor is now so vanishingly
small that it doesn't matter.

The benchmark in #18155 demonstrates both significant page reclaiming
delays, and object reclaiming delays. With "-retain-count=20000000
-preallocate=true -loop-count=3", the benchmark demonstrates several
page reclaiming delays on the order of 40ms. After this change, the
page reclaims are insignificant. The longest sweeps are still ~150ms,
but are object reclaiming delays. We'll address those in the next
several CLs.

Updates #18155.

Fixes #21378 by completely replacing the logic that had that bug.

Change-Id: Iad80eec11d7fc262d02c8f0761ac6998425c4064
Reviewed-on: https://go-review.googlesource.com/c/138959
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2018-11-15 19:27:11 +00:00
Austin Clements
ba1698e963 runtime: mark span when marking any object on the span
This adds a mark bit for each span that is set if any objects on the
span are marked. This will be used for sweeping.

For #18155.

The impact of this is negligible for most benchmarks, and < 1% for
GC-heavy benchmarks.

name                       old time/op  new time/op  delta
Garbage/benchmem-MB=64-12  2.18ms ± 0%  2.20ms ± 1%  +0.88%  (p=0.000 n=16+18)

(https://perf.golang.org/search?q=upload:20180928.1)

name                      old time/op    new time/op    delta
BinaryTree17-12              2.68s ± 1%     2.68s ± 1%    ~     (p=0.707 n=17+19)
Fannkuch11-12                2.28s ± 0%     2.39s ± 0%  +4.95%  (p=0.000 n=19+18)
FmtFprintfEmpty-12          40.3ns ± 4%    39.4ns ± 2%  -2.27%  (p=0.000 n=17+18)
FmtFprintfString-12         67.9ns ± 1%    68.3ns ± 1%  +0.55%  (p=0.000 n=18+19)
FmtFprintfInt-12            75.7ns ± 1%    76.1ns ± 1%  +0.44%  (p=0.005 n=18+19)
FmtFprintfIntInt-12          123ns ± 1%     121ns ± 1%  -1.00%  (p=0.000 n=18+18)
FmtFprintfPrefixedInt-12     150ns ± 0%     148ns ± 0%  -1.33%  (p=0.000 n=16+13)
FmtFprintfFloat-12           208ns ± 0%     204ns ± 0%  -1.92%  (p=0.000 n=13+17)
FmtManyArgs-12               501ns ± 1%     498ns ± 0%  -0.55%  (p=0.000 n=19+17)
GobDecode-12                6.24ms ± 0%    6.25ms ± 1%    ~     (p=0.113 n=20+19)
GobEncode-12                5.33ms ± 0%    5.29ms ± 1%  -0.72%  (p=0.000 n=20+18)
Gzip-12                      220ms ± 1%     218ms ± 1%  -1.02%  (p=0.000 n=19+19)
Gunzip-12                   35.5ms ± 0%    35.7ms ± 0%  +0.45%  (p=0.000 n=16+18)
HTTPClientServer-12         77.9µs ± 1%    77.7µs ± 1%  -0.30%  (p=0.047 n=20+19)
JSONEncode-12               8.82ms ± 0%    8.93ms ± 0%  +1.20%  (p=0.000 n=18+17)
JSONDecode-12               47.3ms ± 0%    47.0ms ± 0%  -0.49%  (p=0.000 n=17+18)
Mandelbrot200-12            3.69ms ± 0%    3.68ms ± 0%  -0.25%  (p=0.000 n=19+18)
GoParse-12                  3.13ms ± 1%    3.13ms ± 1%    ~     (p=0.640 n=20+20)
RegexpMatchEasy0_32-12      76.2ns ± 1%    76.2ns ± 1%    ~     (p=0.818 n=20+19)
RegexpMatchEasy0_1K-12       226ns ± 0%     226ns ± 0%  -0.22%  (p=0.001 n=17+18)
RegexpMatchEasy1_32-12      71.9ns ± 1%    72.0ns ± 1%    ~     (p=0.653 n=18+18)
RegexpMatchEasy1_1K-12       355ns ± 1%     356ns ± 1%    ~     (p=0.160 n=18+19)
RegexpMatchMedium_32-12      106ns ± 1%     106ns ± 1%    ~     (p=0.325 n=17+20)
RegexpMatchMedium_1K-12     31.1µs ± 2%    31.2µs ± 0%  +0.59%  (p=0.007 n=19+15)
RegexpMatchHard_32-12       1.54µs ± 2%    1.53µs ± 2%  -0.78%  (p=0.021 n=17+18)
RegexpMatchHard_1K-12       46.0µs ± 1%    45.9µs ± 1%  -0.31%  (p=0.025 n=17+19)
Revcomp-12                   391ms ± 1%     394ms ± 2%  +0.80%  (p=0.000 n=17+19)
Template-12                 59.9ms ± 1%    59.9ms ± 1%    ~     (p=0.428 n=20+19)
TimeParse-12                 304ns ± 1%     312ns ± 0%  +2.88%  (p=0.000 n=20+17)
TimeFormat-12                318ns ± 0%     326ns ± 0%  +2.64%  (p=0.000 n=20+17)

(https://perf.golang.org/search?q=upload:20180928.2)

Change-Id: I336b9bf054113580a24103192904c8c76593e90e
Reviewed-on: https://go-review.googlesource.com/c/138958
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2018-11-15 19:27:09 +00:00
Austin Clements
69e666e4f7 runtime: record in-use spans in a page-indexed bitmap
This adds a bitmap indexed by page number that marks the starts of
in-use spans. This will be used to quickly find in-use spans with no
marked objects for sweeping.

For #18155.

Change-Id: Icee56f029cde502447193e136fa54a74c74326dd
Reviewed-on: https://go-review.googlesource.com/c/138957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2018-11-15 19:27:06 +00:00
Austin Clements
e500ffd88c runtime: track all heap arenas in a slice
Currently, there's no efficient way to iterate over the Go heap. We're
going to need this for fast free page sweeping, so this CL adds a
slice of all allocated heap arenas. This will also be useful for
generational GC.

For #18155.

Change-Id: I58d126cfb9c3f61b3125d80b74ccb1b2169efbcc
Reviewed-on: https://go-review.googlesource.com/c/138076
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2018-11-15 19:14:10 +00:00
Alan Donovan
b2df0bd5f5 cmd/vet/all: use x/tools/go/analysis/cmd/vet not cmd/vet
cmd/vet/all applies vet to all packages in the standard tree.
It is run for every configuration using this command:
  GO_BUILDER_NAME=misc-vetall go tool dist test
by the misc-vetall builder (see chart at build.golang.org).

Ideally we would switch to 'go vet', but it effectively does a partial
build. This means that its analysis has accurate type information, so
it reports slightly fewer spurious diagnostics. However, it is more
than twice as slow.

Instead, cmd/vet/all builds and runs
golang.org/x/tools/go/analysis/cmd/vet, which uses x/tools/go/packages
to load the entire std lib from source. It takes about 4min to run all
OS/ARCH pairs. An important consequence is that golang.org/x/tools
must be on your $GOPATH to run cmd/vet/all. The test has been
temporarily modified to warn and skip if this is not the case.

This is a preparatory step for switching to the new
cmd/vet based on vet-lite.

Whitelist changes:
- The two "deadcode" diagnostics removed from the whitelist were due
  to if-conditions that could now be proven false.
- The asmdecl warnings are now printed with the log.Printf prefix,
  so they are discarded by the parser and needn't be whitelisted.

Change-Id: I6486508b0de2cd947c897523af086a408cbaf4a8
Reviewed-on: https://go-review.googlesource.com/c/149097
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-15 18:46:33 +00:00
Alan Donovan
a18638c0f2 cmd/vendor: update to golang.org/x/tools@f62bfb54
Change-Id: I3b3035784ce89ba2ac5ab8f6448c45a3d38fa97d
Reviewed-on: https://go-review.googlesource.com/c/149778
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-15 17:40:19 +00:00
Keith Randall
82c260099d x/net/route: use libc calls on Darwin
Starting with 1.12, we must use syscall versions
of sysctl instead of the raw syscall.

An identical CL went into the source copy at golang.org/x/net/route.
This is just a cherry pick of that CL.
(CL: https://go-review.googlesource.com/c/net/+/148597)

Change-Id: I6286ab3e49f82512491afb5bcf349e89ab5645ab
Reviewed-on: https://go-review.googlesource.com/c/149637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-14 22:39:08 +00:00
Austin Clements
8dba66dfeb cmd/link: fix isStmt DWARF info
When CL 147160 introduced function ABIs encoded as symbol versions in
the linker, it became slightly more complicated to look up derived
DWARF symbols. It fixed this by introducing a dwarfFuncSym function to
hide this logic, but missed one derived lookup that was done in the
object reader itself. As a result, we lost the isStmt tables from the
compiler, so every PC was marked as a statement in the DWARF info.

Fix this by moving this derived lookup out of the object reader and
into the DWARF code and calling dwarfFuncSym to get the correctly
versioned symbol.

Should fix the linux-amd64-longtest builder.

Updates #27539.

Change-Id: If40d5ba28bab1918ac4ad18fbb5103666b6d978b
Reviewed-on: https://go-review.googlesource.com/c/149605
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2018-11-14 22:01:38 +00:00
Martin Möhrmann
2de53906e1 internal/cpu: move GODEBUGCPU options into GODEBUG
Change internal/cpu feature configuration to use
GODEBUG=cpu.feature1=value,cpu.feature2=value...
instead of GODEBUGCPU=feature1=value,feature2=value... .

This is not a backwards compatibility breaking change
since GODEBUGCPU was introduced in go1.11 as an
undocumented compiler experiment.

Fixes #28757

Change-Id: Ib21b3fed2334baeeb061a722ab1eb513d1137e87
Reviewed-on: https://go-review.googlesource.com/c/149578
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-14 21:47:50 +00:00
Alan Donovan
89a7917493 cmd/vendor/golang.org/x/tools: update to 7d6b83ca
Also, add a script for future updates.

Change-Id: I2565d1f26532b9dd7cf9d8ce198ba08fb3d53407
Reviewed-on: https://go-review.googlesource.com/c/149604
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2018-11-14 21:08:11 +00:00
Brad Fitzpatrick
bfd9b94069 net/http: make Transport respect {X-,}Idempotency-Key header
Fixes #19943

Change-Id: I5e0fefe44791d7b3556095d726c2a753ec551ef2
Reviewed-on: https://go-review.googlesource.com/c/147457
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-11-14 21:07:02 +00:00
Keith Randall
0c7762cd18 cmd/compile: assume unsafe pointer arithmetic generates non-nil results
I've never seen a case where unsafe arithmetic is used to generate a nil.
(Something like var x uintptr; unsafe.Pointer(x - x).)
We can assume that if someone is doing arithmetic with pointers, the
result will be non-nil. Our unsafe rules already forbid this, although
we should be more explicit.

RELNOTE=It is invalid to convert a nil unsafe.Pointer to uintptr and back, with arithmetic.
(This was already invalid, but this statement has been added for clarification.)

Fixes #27180

Change-Id: I1880b7725a9fd99e4613799930fdad9aaa99e8f0
Reviewed-on: https://go-review.googlesource.com/c/146058
Reviewed-by: Austin Clements <austin@google.com>
2018-11-14 21:01:36 +00:00
Ali Rizvi-Santiago
a063a2284a cmd/go: allow the user to specify ar via an environment variable
This allows one to customize which ar to use by fetching its path
from the environment. This way one can swap it out for a
different implementation.

Change-Id: I40d8cbd8a69e97b5254e66081d9bf0b726c10366
GitHub-Last-Rev: 4aa1d631ea
GitHub-Pull-Request: golang/go#28746
Reviewed-on: https://go-review.googlesource.com/c/149117
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-14 20:33:24 +00:00
Martin Möhrmann
f36e92dbfc fmt: avoid allocation when formatting byte slice arguments with verb s
fmtBytes is in the top 10 callers of runtime.slicebytetostring according
to Google wide profiling data.

Avoid the string conversion of the input byte slice in fmtBytes by calling
a newly added specialized fmtS function for byte slices.

Expand tests for verb s with widths to test strings and byte slice arguments.

SprintfTruncateString     157ns ± 4%     156ns ± 3%     ~     (p=0.122 n=20+20)
SprintfTruncateBytes      188ns ± 2%     155ns ± 3%  -18.00%  (p=0.000 n=20+19)

name                   old alloc/op   new alloc/op   delta
SprintfTruncateString     16.0B ± 0%     16.0B ± 0%     ~     (all equal)
SprintfTruncateBytes      64.0B ± 0%     16.0B ± 0%  -75.00%  (p=0.000 n=20+20)

name                   old allocs/op  new allocs/op  delta
SprintfTruncateString      1.00 ± 0%      1.00 ± 0%     ~     (all equal)
SprintfTruncateBytes       2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=20+20)

Change-Id: I461bf514d4232b39bd9c812f7faa4e5ef693a03b
Reviewed-on: https://go-review.googlesource.com/c/145284
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2018-11-14 20:32:15 +00:00
Martin Möhrmann
75798e8ada runtime: make processor capability variable naming platform specific
The current support_XXX variables are specific for the
amd64 and 386 platforms.

Prefix processor capability variables by architecture to have a
consistent naming scheme and avoid reuse of the existing
variables for new platforms.

This also aligns naming of runtime variables closer with internal/cpu
processor capability variable names.

Change-Id: I3eabb29a03874678851376185d3a62e73c1aff1d
Reviewed-on: https://go-review.googlesource.com/c/91435
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-11-14 20:30:31 +00:00
Mark Rushakoff
529ea7c0de cmd/go: don't panic when go run is passed ... under nonexistent dir
Given a nonexistent directory above a wildcard:

    go run ./nonexistent/...

Print this error instead of panicking:

    go run: no packages loaded from ./nonexistent/...

Fixes #28696.

Change-Id: Iaa3bc5c78b14ef858d931778e1bc55ca626c5571
GitHub-Last-Rev: bb1a80483a
GitHub-Pull-Request: golang/go#28703
Reviewed-on: https://go-review.googlesource.com/c/148821
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-11-14 19:59:17 +00:00
Milan Knezevic
c92e73b702 cmd/compile/internal/gc: OMUL should be evaluated when using soft-float
When using soft-float, OMUL might be rewritten to function call
so we should ensure it was evaluated first.

Fixes #28688

Change-Id: I30b87501782fff62d35151f394a1c22b0d490c6c
Reviewed-on: https://go-review.googlesource.com/c/148837
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-11-14 18:52:15 +00:00
Clément Chigot
0a72e8eefb cmd/go: fix TestScript/vet_asm for OS without amd64
Add GOOS=linux to vet_asm script to avoid errors on OS not working on
amd64 architecture.

Change-Id: Ic3db43618008ae8e8bc08738c95c200dd4e916d7
Reviewed-on: https://go-review.googlesource.com/c/149577
Reviewed-by: Alan Donovan <adonovan@google.com>
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-14 14:45:03 +00:00
Emmanuel T Odeke
6d620fc42e test: move empty header file in builddir, buildrundir to temp directory
Move the empty header file created by "builddir", "buildrundir"
directives to t.tempDir. The file was accidentally placed in the
same directory as the source code and this was a vestige of CL 146999.

Fixes #28781

Change-Id: I3d2ada5f9e8bf4ce4f015b9bd379b311592fe3ce
Reviewed-on: https://go-review.googlesource.com/c/149458
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-11-14 00:22:40 +00:00
Ian Lance Taylor
69397422c0 cmd/compile: by default accept any language
The bootstrap stage 1 compiler was defaulting to the language version
used by the bootstrap compiler itself, typically 1.4. Normally this
doesn't matter since the bootstrap code has to build with 1.4 anyhow,
but it broke the boringcrypto branch which uses cgo during the
bootstrap, as cgo now generates code that uses type aliases.

Change-Id: I8a8312bb9ca4befaf65c00a8d71a78566075c2f7
Reviewed-on: https://go-review.googlesource.com/c/149459
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2018-11-14 00:08:05 +00:00
Austin Clements
a3c70e28ed test: fix ABI mismatch in fixedbugs/issue19507
Because run.go doesn't pass the package being compiled to the compiler
via the -p flag, it can't match up the main·f symbol from the
assembler with the "func f" stub in Go, so it doesn't produce the
correct assembly stub.

Fix this by removing the package prefix from the assembly definition.

Alternatively, we could make run.go pass -p to the compiler, but it's
nicer to remove these package prefixes anyway.

Should fix the linux-arm builder, which was broken by the introduction
of function ABIs in CL 147160.

Updates #27539.

Change-Id: Id62b7701e1108a21a5ad48ffdb5dad4356c273a6
Reviewed-on: https://go-review.googlesource.com/c/149483
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2018-11-13 23:44:52 +00:00
Keith Randall
0098f8aeac runtime: when using explicit argmap, also use arglen
When we set an explicit argmap, we may want only a prefix of that
argmap.  Argmap is set when the function is reflect.makeFuncStub or
reflect.methodValueCall. In this case, arglen specifies how much of
the args section is actually live. (It could be either all the args +
results, or just the args.)

Fixes #28750

Change-Id: Idf060607f15a298ac591016994e58e22f7f92d83
Reviewed-on: https://go-review.googlesource.com/c/149217
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2018-11-13 22:52:09 +00:00
Keith Randall
df2bb9817b runtime: during map delete, update entries after new last element
When we delete an element, and it was the last element in the bucket,
update the slots between the new last element and the old last element
with the marker that says "no more elements beyond here".

Change-Id: I8efeeddf4c9b9fc491c678f84220a5a5094c9c76
Reviewed-on: https://go-review.googlesource.com/c/142438
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-11-13 21:24:57 +00:00
Daniel Martí
62b850f1c5 cmd/vet: rewrite method check to use go/types
Now that vet can rely on go/types, there's no reason to do extra work to
avoid using it. The rewrite lets us get rid of the field list flattening
code, as well as the slight verbosity that comes with go/printer.

While at it, make the testdata/method.go expected errors be more
specific, to make sure that we're not breaking the warnings that are
printed.

Finally, update whitelist/all.txt, since the reported errors now include
qualified types.

Change-Id: I760a1b3b1f60e4a478c9dc43bd7f584a8459593e
Reviewed-on: https://go-review.googlesource.com/c/148919
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Alan Donovan <adonovan@google.com>
2018-11-13 18:11:58 +00:00
Samuel Kelemen
160ddf76e3 cmd/internal/obj/arm64: fix spelling in arm64 doc
ln5: "instrutions" => "instructions";
ln159: "immedate" => "immediate";

Change-Id: Ifb94a9c145d1911ed92f12883213245beee2bd67
GitHub-Last-Rev: 78627835e7
GitHub-Pull-Request: golang/go#28776
Reviewed-on: https://go-review.googlesource.com/c/149378
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-13 18:06:17 +00:00
Hana Kim
45e9c5538b runtime/debug: add API to read module info in binary
When module is enabled, the go tool embeds build information
related to the module in the binary including the dependencies
and the replace information (See
src/cmd/go/internal/modload.PackageBuildInfo).

The newly introduced ReadBuildInfo reads the information and
makes it accessible programmatically.

Update #26404

Change-Id: Ide37022d609b4a8fb6b5ce02afabb73f04fbb532
Reviewed-on: https://go-review.googlesource.com/c/144220
Run-TryBot: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2018-11-13 17:16:48 +00:00
Fangming.Fang
978cfa8e46 cmd,runtime: enable race detector on arm64
Changes include:
1. enable compiler option -race for arm64
2. add runtime/race_arm64.s to manage the calls from Go to the compiler-rt runtime
3. change racewalk.go to call racefuncenterfp instead of racefuncenter on arm64 to
   allow the caller pc to be obtained in the asm code before calling the tsan version
4. race_linux_arm64.syso comes from compiler-rt which just supports 48bit VA, compiler-rt
   is fetched from master branch which latest commit is 3aa2b775d08f903f804246af10b

Fixes #25682

Change-Id: I04364c580b8157fd117deecae74a4656ba16e005
Reviewed-on: https://go-review.googlesource.com/c/138675
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-11-13 16:57:22 +00:00
Alan Donovan
e787b13328 cmd/go: vet: pass non-.go files to vet tool
The "gofiles" cache entry has been renamed "srcfiles", and it includes
non-Go files (.s, .c, .cxx) that belong to the package. It does not
include raw cgo files.

Added regression test.

Fixes #27665

Change-Id: I4884fe9b4f823f50705f8c2d357a04a8e567734f
Reviewed-on: https://go-review.googlesource.com/c/148904
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-11-13 16:10:01 +00:00
Russ Cox
b075dfba80 cmd/go/internal/modload: skip go.mod directories on all systems, not just Plan 9
I see no reason Plan 9 should be special cased.
A directory named go.mod is not useful on any system.
Followup to CL 129804.

Change-Id: I9cc91b5934b17650bfdb07370aa73aeae445968c
Reviewed-on: https://go-review.googlesource.com/c/149337
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-11-13 15:58:52 +00:00
Than McIntosh
43edf21eff cmd/cgo: fix typo in gccgo name mangling recipe
The code to implement new-style gccgo name mangling had a recipe that
didn't quite match that of the compiler (incorrect handling for '.').
This showed up as a failure in the gotools cgo test if the directory
containing the test run included a "." character.

[This is a copy of https://golang.org/cl/147917].

Change-Id: Ia94728ecead879c8d223eb6cee6c102a8af1c86e
Reviewed-on: https://go-review.googlesource.com/c/147937
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-11-13 15:34:54 +00:00
Russ Cox
8f7173dcde cmd/go: revert "remove unnecessary else conditions"
This reverts CL 144137.

Reason for revert: The justification for the original commit
was that golint said so, but golint is wrong. The code reads
more clearly the original way.

Change-Id: I960f286ed66fec67aabd953e7b69993f60b00bca
Reviewed-on: https://go-review.googlesource.com/c/149339
Reviewed-by: Russ Cox <rsc@golang.org>
2018-11-13 15:15:18 +00:00
Martin Garton
e51b19a993 bufio: make Reader.Peek invalidate Unreads
Since Reader.Peek potentially reads from the underlying io.Reader,
discarding previous buffers, UnreadRune and UnreadByte cannot
necessarily work.  Change Peek to invalidate the unread buffers in all
cases (as allowed according to the documentation) and thus prevent
hiding bugs in the caller.

(This change was previoiusly merged and then reverted due concern about
being too close to a release)

Fixes #18556

Change-Id: I9027d75aa834d4b27703f37711ba25de04d89f3c
GitHub-Last-Rev: 917ef1e511
GitHub-Pull-Request: golang/go#28768
Reviewed-on: https://go-review.googlesource.com/c/149297
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-13 15:08:13 +00:00
Russ Cox
9d025bdafe container/heap: adjust wording in comments
Followup to CL 129779 but also some other minor tweaks.

Change-Id: Id71455d8a14f5e33f82c942c9e892da56c49d17c
Reviewed-on: https://go-review.googlesource.com/c/149257
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-13 13:42:59 +00:00
Russ Cox
a48a666bc8 go/build: do not suggest runtime.Version in comment
The form of runtime.Version is not guaranteed to be helpful.
Do not suggest it. (The suggestion was added in CL 136215.)

Change-Id: I3227d2e66b6ce860b7e62d7ba531c18fb173823c
Reviewed-on: https://go-review.googlesource.com/c/149258
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-13 13:35:15 +00:00
Russ Cox
de50ea3cd8 bufio: put notes about len(p) together
CL 145577 added the part about io.ReadFull to read len(p)
but it should be next to the existing sentence about not
reading len(p) bytes.

Change-Id: Idfa037c59a3085d44d5da6129188473db0e96d23
Reviewed-on: https://go-review.googlesource.com/c/148903
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
2018-11-13 03:25:22 +00:00
Brad Fitzpatrick
ee55f0856a net/http/httputil: make ReverseProxy automatically proxy WebSocket requests
Fixes #26937

Change-Id: I6cdc1bad4cf476cd2ea1462b53444eccd8841e14
Reviewed-on: https://go-review.googlesource.com/c/146437
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-11-13 01:42:47 +00:00
Robert Griesemer
de578dcdd6 spec: be clearer about definition of string length
Adjusted spec to explicitly define the string length as the
number of bytes of the string; the prose now matches the prose
for arrays. Made analogous change for slices.

Fixes #28736.

Change-Id: I47cab321c87de0a4c482f5466b819b2cc8993fd1
Reviewed-on: https://go-review.googlesource.com/c/149077
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-11-13 00:42:32 +00:00
Ian Lance Taylor
7ebe35093d cmd/compile: correct check for valid -lang version
Change-Id: Iad10d0a2dbc8e12e9f776c6cfb34070f584fd439
Reviewed-on: https://go-review.googlesource.com/c/149057
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-11-13 00:08:32 +00:00
Emmanuel T Odeke
5d39260079 net: preserve unexpired context values for LookupIPAddr
To avoid any cancelation of the parent context from affecting
lookupGroup operations, Resolver.LookupIPAddr previously used
an entirely new context created from context.Background().
However, this meant that all the values in the parent context
with which LookupIPAddr was invoked were dropped.

This change provides a custom context implementation
that only preserves values of the parent context by composing
context.Background() and the parent context. It only falls back
to the parent context to perform value lookups if the parent
context has not yet expired.
This context is never canceled, and has no deadlines.

Fixes #28600

Change-Id: If2f570caa26c65bad638b7102c35c79d5e429fea
Reviewed-on: https://go-review.googlesource.com/c/148698
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-12 23:34:54 +00:00
Filippo Valsorda
70e3b1df4a crypto/tls: don't modify Config.Certificates in BuildNameToCertificate
The Config does not own the memory pointed to by the Certificate slice.
Instead, opportunistically use Certificate.Leaf and let the application
set it if it desires the performance gain.

This is a partial rollback of CL 107627. See the linked issue for the
full explanation.

Fixes #28744

Change-Id: I33ce9e6712e3f87939d9d0932a06d24e48ba4567
Reviewed-on: https://go-review.googlesource.com/c/149098
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-12 23:25:21 +00:00
Ali Rizvi-Santiago
595bc63e8f runtime/cgo: added missing includes for errno.h to the windows gcc stubs.
This adds the includes for errno.h to the windows stubs
for runtime/cgo so that "errno" is properly declared.

Due to "errno" not being properly declared, the compiler is
forced to assume it's an external which leaves it up to the
linker. This is an issue in some implementations as errno
might be a macro which results in an unresolved symbol error
during linking.

runtime/cgo/gcc_libinit_windows.c: added include
runtime/cgo/gcc_windows_386.c: added include
runtime/cgo/gcc_windows_amd64.c: added include

Change-Id: I77167d02f7409462979135efc55cf50bbc6bd363
GitHub-Last-Rev: 90da06ee3c
GitHub-Pull-Request: golang/go#28747
Reviewed-on: https://go-review.googlesource.com/c/149118
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-12 23:05:28 +00:00
Austin Clements
5cf2b4c2d3 cmd/compile: fix race on initializing Sym symFunc flag
SSA lowering can create PFUNC ONAME nodes when compiling method calls.
Since we generally initialize the node's Sym to a func when we set its
class to PFUNC, we did this here, too. Unfortunately, since SSA
compilation is concurrent, this can cause a race if two function
compilations try to initialize the same symbol.

Luckily, we don't need to do this at all, since we're actually just
wrapping an ONAME node around an existing Sym that's already marked as
a function symbol.

Fixes the linux-amd64-racecompile builder, which was broken by CL
147158.

Updates #27539.

Change-Id: I8ddfce6e66a08ce53998c5bfa6f5a423c1ffc1eb
Reviewed-on: https://go-review.googlesource.com/c/149158
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2018-11-12 22:38:02 +00:00
Austin Clements
b52db19b98 cmd/compile: create "init" symbol earlier
We create the "init" symbol and mark it as a function before compiling
to SSA because SSA can initialize this symbol, but it turns out we do
it slightly too late. peekitabs, at least, can also create the "init"
LSym. Move this initialization to just after type-checking.

Fixes the linux-amd64-ssacheck and the android-arm64-wiko-fever
builders.

Updates #27539.

Change-Id: If145952c79d39f75c93b24e35e67fe026dd08329
Reviewed-on: https://go-review.googlesource.com/c/149137
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-11-12 22:25:35 +00:00
Austin Clements
891f99eb43 cmd/compile: fix TestFormats
This fixes the linux-amd64-longtest builder, which was broken by CL
147160.

Updates #27539.

Change-Id: If6e69581ef503bba2449ec9bacaa31f34f59beb1
Reviewed-on: https://go-review.googlesource.com/c/149157
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-12 21:54:58 +00:00
Austin Clements
685aca45dc cmd/compile, cmd/link: separate stable and internal ABIs
This implements compiler and linker support for separating the
function calling ABI into two ABIs: a stable and an internal ABI. At
the moment, the two ABIs are identical, but we'll be able to evolve
the internal ABI without breaking existing assembly code that depends
on the stable ABI for calling to and from Go.

The Go compiler generates internal ABI symbols for all Go functions.
It uses the symabis information produced by the assembler to create
ABI wrappers whenever it encounters a body-less Go function that's
defined in assembly or a Go function that's referenced from assembly.

Since the two ABIs are currently identical, for the moment this is
implemented using "ABI alias" symbols, which are just forwarding
references to the native ABI symbol for a function. This way there's
no actual code involved in the ABI wrapper, which is good because
we're not deriving any benefit from it right now. Once the ABIs
diverge, we can eliminate ABI aliases.

The linker represents these different ABIs internally as different
versions of the same symbol. This way, the linker keeps us honest,
since every symbol definition and reference also specifies its
version. The linker is responsible for resolving ABI aliases.

Fixes #27539.

Change-Id: I197c52ec9f8fc435db8f7a4259029b20f6d65e95
Reviewed-on: https://go-review.googlesource.com/c/147160
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2018-11-12 20:46:55 +00:00
Austin Clements
1794ee6829 cmd/link: nice error message on ABI mismatch
Currently, if a symbol is only defined under one ABI and referenced
under another ABI, you simply get a "relocation target X not defined".
This is confusing because it seems like the symbol is defined.

This CL enhances the error message in this case to be "relocation
target X not defined for <ABI> (but is defined for <ABI>)".

For #27539.

Change-Id: If857a1882c3fe9af5346797d5295ca1fe50ae565
Reviewed-on: https://go-review.googlesource.com/c/147159
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-11-12 20:46:52 +00:00
Austin Clements
16e6cd9a4d cmd/compile: mark function Syms
In order to mark the obj.LSyms produced by the compiler with the
correct ABI, we need to know which types.Syms refer to function
symbols. This CL adds a flag to types.Syms to mark symbols for
functions, and sets this flag everywhere we create a PFUNC-class node,
and in the one place where we directly create function symbols without
always wrapping them in a PFUNC node (methodSym).

We'll use this information to construct obj.LSyms with correct ABI
information.

For #27539.

Change-Id: Ie3ac8bf3da013e449e78f6ca85546a055f275463
Reviewed-on: https://go-review.googlesource.com/c/147158
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2018-11-12 20:46:50 +00:00
Austin Clements
c5718b6b26 cmd/internal/obj, cmd/link: record ABIs and aliases in Go obj files
This repurposes the "version" field of a symbol reference in the Go
object file format to be an ABI field. Currently, this is just 0 or 1
depending on whether the symbol is static (the linker turns it into a
different internal version number), so it's already only tenuously a
symbol version. We change this to be -1 for static symbols and
otherwise by the ABI number.

This also adds a separate list of ABI alias symbols to be recorded in
the object file. The ABI aliases must be a separate list and not just
part of the symbol definitions because it's possible to have a symbol
defined in one package and the alias "defined" in a different package.
For example, this can happen if a symbol is defined in assembly in one
package and stubbed in a different package. The stub triggers the
generation of the ABI alias, but in a different package from the
definition.

For #27539.

Change-Id: I015c9fe54690c027de6ef77e22b5585976a01587
Reviewed-on: https://go-review.googlesource.com/c/147157
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2018-11-12 20:46:48 +00:00
Austin Clements
07544c7e80 cmd/go, cmd/dist: collect known cross-package uses of runtime symbols
This extends cmd/go's symabis support to collect known cross-package
uses of runtime symbols from other "basically runtime" packages in
std. This avoids having to declare a large number of ABI0 symbols in
the runtime for a small number of known cross-package references.

For cmd/dist, we use a simpler but less efficient approach and tell
the compiler to generate ABI wrappers for everything.

Change-Id: Ifaed94efdcff42e7345ab11b4d2fb880fb1a24e8
Reviewed-on: https://go-review.googlesource.com/c/147257
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-12 20:46:43 +00:00
Austin Clements
0f5dfbcfd7 cmd/go, cmd/dist: plumb symabis from assembler to compiler
For #27539.

Change-Id: I0e27f142224e820205fb0e65ad03be7eba93da14
Reviewed-on: https://go-review.googlesource.com/c/146999
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-12 20:46:41 +00:00