1
0
mirror of https://github.com/golang/go synced 2024-11-06 09:36:16 -07:00
Commit Graph

40651 Commits

Author SHA1 Message Date
Russ Cox
fbaf881cc6 net/http: fix Transport.MaxConnsPerHost limits & idle pool races
There were at least three races in the implementation of the pool of
idle HTTP connections before this CL.

The first race is that HTTP/2 connections can be shared for many
requests, but each requesting goroutine would take the connection out
of the pool and then immediately return it before using it; this
created unnecessary, tiny little race windows during which another
goroutine might dial a second connection instead of reusing the first.
This CL changes the idle pool to just leave the HTTP/2 connection in
the pool permanently (until there is reason to close it), instead of
doing the take-it-out-put-it-back dance race.

The second race is that “is there an idle connection?” and
“register to wait for an idle connection” were implemented as two
separate steps, in different critical sections. So a client could end
up registered to wait for an idle connection and be waiting or perhaps
dialing, not having noticed the idle connection sitting in the pool
that arrived between the two steps.

The third race is that t.getIdleConnCh assumes that the inability to
send on the channel means the client doesn't need the result, when it
could mean that the client has not yet entered the select.
That is, the main dial does:

	idleConnCh := t.getIdleConnCh(cm)
	select {
	case v := <-dialc:
		...
	case pc := <-idleConnCh
		...
	...
	}

But then tryPutIdleConn does:

	waitingDialer := t.idleConnCh[key] // what getIdleConnCh(cm) returned
	select {
	case waitingDialer <- pconn:
		// We're done ...
		return nil
	default:
		if waitingDialer != nil {
			// They had populated this, but their dial won
			// first, so we can clean up this map entry.
			delete(t.idleConnCh, key)
		}
	}

If the client has returned from getIdleConnCh but not yet reached the
select, tryPutIdleConn will be unable to do the send, incorrectly
conclude that the client does not care anymore, and put the connection
in the idle pool instead, again leaving the client dialing unnecessarily
while a connection sits in the idle pool.

(It's also odd that the success case does not clean up the map entry,
and also that the map has room for only a single waiting goroutine for
a given host.)

None of these races mattered too much before Go 1.11: at most they
meant that connections were not reused quite as promptly as possible,
or a few more than necessary would be created. But Go 1.11 added
Transport.MaxConnsPerHost, which limited the number of connections
created for a given host. The default is 0 (unlimited), but if a user
did explicitly impose a low limit (2 is common), all these misplaced
conns could easily add up to the entire limit, causing a deadlock.
This was causing intermittent timeouts in TestTransportMaxConnsPerHost.

The addition of the MaxConnsPerHost support added its own races.

For example, here t.incHostConnCount could increment the count
and return a channel ready for receiving, and then the client would
not receive from it nor ever issue the decrement, because the select
need not evaluate these two cases in order:

	select {
	case <-t.incHostConnCount(cmKey):
		// count below conn per host limit; proceed
	case pc := <-t.getIdleConnCh(cm):
		if trace != nil && trace.GotConn != nil {
			trace.GotConn(httptrace.GotConnInfo{Conn: pc.conn, Reused: pc.isReused()})
		}
		return pc, nil
	...
	}

Obviously, unmatched increments are another way to get to a deadlock.
TestTransportMaxConnsPerHost deadlocked approximately 100% of
the time with a small random sleep added between incHostConnCount
and the select:

	ch := t.incHostConnCount(cmKey):
	time.Sleep(time.Duration(rand.Intn(10))*time.Millisecond)
	select {
	case <-ch
		// count below conn per host limit; proceed
	case pc := <-t.getIdleConnCh(cm):
		...
	}

The limit also did not properly apply to HTTP/2, because of the
decrement being attached to the underlying net.Conn.Close
and net/http not having access to the underlying HTTP/2 conn.
The alternate decrements for HTTP/2 may also have introduced
spurious decrements (discussion in #29889). Perhaps those
spurious decrements or other races caused the other intermittent
non-deadlock failures in TestTransportMaxConnsPerHost,
in which the HTTP/2 phase created too many connections (#31982).

This CL replaces the buggy, racy code with new code that is hopefully
neither buggy nor racy.

Fixes #29889.
Fixes #31982.
Fixes #32336.

Change-Id: I0dfac3a6fe8a6cdf5f0853722781fe2ec071ac97
Reviewed-on: https://go-review.googlesource.com/c/go/+/184262
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-07-08 14:31:42 +00:00
Than McIntosh
ddc8439b2f test: add new test case for gccgo compiler bug
Test case that causes incorrect compiler error from gccgo.

Updates #32922

Change-Id: I59432a8e8770cf03eda293f6d110c081c18fa88b
Reviewed-on: https://go-review.googlesource.com/c/go/+/184918
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-07-08 13:20:23 +00:00
Daniel Martí
13327f219e encoding/json: obey SetEscapeHTML in all MarshalJSON cases
It wasn't obeyed in the case where the MarshalJSON method uses a pointer
receiver, and the encoder grabs the address of a value to find that
method. addrMarshalerEncoder is the function that does this work, but it
ignored opts.escapeHTML.

Here's the before and after of the added test case, which was failing
before the fix. Now the two cases are correct and consistent.

	{"NonPtr":"<str>","Ptr":"\u003cstr\u003e"}
	{"NonPtr":"<str>","Ptr":"<str>"}

Fixes #32896.

Change-Id: Idc53077ece074973558bd3bb5ad036380db0d02c
Reviewed-on: https://go-review.googlesource.com/c/go/+/184757
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Caleb Spare <cespare@gmail.com>
2019-07-06 13:25:59 +00:00
Austin Clements
a2fb5cd823 cmd/cgo: accept weak dynamic imports
cgo produces dynamic imports for Go binaries by scanning the dynamic
imports table of a binary produced by the system C compiler and
linker. Currently, since it uses elf.File.ImportedSymbols, it only
reads global symbols. Unfortunately, recent versions of lld emit weak
symbol imports for several pthread symbols, which means the cgo tool
doesn't emit dynamic imports for them, which ultimately causes linking
of cgo binaries to fail.

Fix this by using elf.File.DynamicSymbols instead and filtering down
to both global and weak symbols.

Fixes #31912.

Change-Id: If346a7eca6733e3bfa2cccf74a9cda02a3e81d38
Reviewed-on: https://go-review.googlesource.com/c/go/+/184100
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-07-05 18:18:28 +00:00
Austin Clements
7aac3436dd debug/elf: add version information to all dynamic symbols
Currently, File.ImportedSymbols is the only API that exposes the GNU
symbol version information for dynamic symbols. Unfortunately, it also
filters to specific types of symbols, and only returns symbol names.

The cgo tool is going to need symbol version information for more
symbols. In order to support this and make the API more orthogonal,
this CL adds version information to the Symbol type and updates
File.DynamicSymbols to fill this in. This has the downside of
increasing the size of Symbol, but seems to be the most natural API
for exposing this. I also explored 1) adding a method to get the
version information for the i'th dynamic symbol, but we don't use
symbol indexes anywhere else in the API, and it's not clear if this
index would be 0-based or 1-based, and 2) adding a
DynamicSymbolVersions method that returns a slice of version
information that parallels the DynamicSymbols slice, but that's less
efficient to implement and harder to use.

For #31912.

Change-Id: I69052ac3894f7af2aa9561f7085275130e0cf717
Reviewed-on: https://go-review.googlesource.com/c/go/+/184099
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-07-05 18:18:26 +00:00
Keith Randall
9a00e64633 test: add another test for issue 32680
Update #32680

Change-Id: I0318c22c22c5cd6ab6441d1aa2d1a40d20d71242
Reviewed-on: https://go-review.googlesource.com/c/go/+/185137
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: David Chase <drchase@google.com>
2019-07-05 12:11:36 +00:00
Mohit Agarwal
e94472a369 doc/go1.13: fix links and a closing tag
- fix link for `Time.Format`
- fix closing tag for `go get`
- add links for `runtime.Caller`, `runtime.Callers`
- remove link for `TypedArrayOf` since it has been removed (CL 177537)

Change-Id: I1dc38226e6d91c68fbd2f02c1acfad5327f4ebe8
Reviewed-on: https://go-review.googlesource.com/c/go/+/185038
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-07-05 00:44:51 +00:00
Patrik Lundin
04e2e81ecb net/http: stop ExampleServer_Shutdown from hanging on error
Running the example code when not having permissions
to bind to port 80 will cause the program to hang after
printing the error message.

Change-Id: I2433ba2629b362fc8f1731e40cab5eea72ec354f
GitHub-Last-Rev: 0bb3dc08b6
GitHub-Pull-Request: golang/go#32947
Reviewed-on: https://go-review.googlesource.com/c/go/+/185157
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-07-05 00:26:14 +00:00
Kyle Shannon
adcb2b1e7a lib/time: update tz data to 2019b
Updates #22487

Change-Id: I3c933c68b5c3c5e78e3a9656ea2dcdbcbb59533f
Reviewed-on: https://go-review.googlesource.com/c/go/+/184577
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-07-03 23:08:27 +00:00
Jay Conrod
beeb8fea8d cmd/go: update tests for new @upgrade query
This should have been part of CL 184440.

Updates #32846

Change-Id: I78a1326f4a67b3b526859bd15cb9653b4a8551a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/184920
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-07-03 20:29:17 +00:00
Jay Conrod
a3a0cc2e1a cmd/go: restore @latest behavior and support @upgrade in 'go get'
'go get path@latest' may now downgrade a module required at a
pre-release or pseudo-version newer than the latest released
version. This restores the 1.12 behavior and the ability to easily
roll back from a temporary development version.

'go get path@upgrade' is like @latest but will not downgrade.
If no version suffix is specified ('go get path'), @upgrade is
implied.

Fixes #32846

Change-Id: Ibec0628292ab1c484716a5add0950d7a7ee45f47
Reviewed-on: https://go-review.googlesource.com/c/go/+/184440
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-07-03 18:15:45 +00:00
Jay Conrod
b412fde53a go/build: document characters allowed in build tags
The documentation just said "alphanumeric", but underscores and dots
are also accepted.

Fixes #32886

Change-Id: I1ba872a220d5c5bf64f1d851ddba9eb3b1afb89a
Reviewed-on: https://go-review.googlesource.com/c/go/+/184917
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-07-03 14:55:39 +00:00
Cherry Zhang
1c1e517005 test: add a test for gccgo bug #32901
This CL adds a test for gccgo bug #32901: not all the type
descriptors are registered and thus deduplicated with types
created by reflection. It needs a few levels of indirect imports
to trigger this bug.

Updates #32901.

Change-Id: Idbd89bedd63fea746769f2687f3f31c9767e5ec0
Reviewed-on: https://go-review.googlesource.com/c/go/+/184718
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-07-03 14:12:22 +00:00
Tobias Klauser
e2fdce9f16 doc/go1.13: fix typo
Change-Id: I99e76c0c12050289be5b353595eb21fbabe7c01e
Reviewed-on: https://go-review.googlesource.com/c/go/+/184597
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-07-03 07:14:40 +00:00
Bryan C. Mills
54c6b814ac cmd/go: fix TestScript/mod_sumdb_golang to avoid assumptions about @v/list
I accidentally fetched an invalid version of rsc.io/quote from
proxy.golang.org, which the proxy then cached and now includes in
https://proxy.golang.org/rsc.io/quote/@v/list.

That causes 'go get rsc.io/quote` to resolve to a different version
depending on whether the proxy is used.

Adjust the test to fetch an explicit version instead, since the choice
of 'latest' is mostly irrelevant to the checksum database logic that
the test is intended to verify.

Updates #32805
Fixes #32900

Change-Id: I075b1f62e8c71545d0fb2dd4bd77ba525fc2a36d
Reviewed-on: https://go-review.googlesource.com/c/go/+/184719
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-07-02 20:07:52 +00:00
Than McIntosh
e62c3d3503 test: add testcase for gccgo compile failure
Test case that caused a compiler crash in gofrontend, related to
exporting inlinable function bodies.

Updates #32778

Change-Id: Iacf1753825d5359da43e5e281189876d4c3dd3c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/183851
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-07-02 16:34:55 +00:00
Keith Randall
fbde753a58 cmd/compile: make duplicate anonymous interface output deterministic
Taking over CL 162240, the original CL hasn't been making progress.
I just took the parts that fix the immediate issue. I left the
signatslice changes out, I don't think they are necessary.

Fixes #30202

Change-Id: I5b347605f0841dd925d5a73150b8bf269fa82464
Reviewed-on: https://go-review.googlesource.com/c/go/+/183852
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-07-02 13:52:21 +00:00
Ian Lance Taylor
d410642f49 runtime: use correct register in darwin/386 pipe_trampoline
Updates #31264

Change-Id: I745744dd3fdaa432d70e8dc9336547017bac89ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/184377
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Elias Naur <mail@eliasnaur.com>
2019-07-01 21:30:23 +00:00
Russ Cox
bc593eac2d sync: document implementation of Once.Do
It's not correct to use atomic.CompareAndSwap to implement Once.Do,
and we don't, but why we don't is a question that has come up
twice on golang-dev in the past few months.
Add a comment to help others with the same question.

Change-Id: Ia89ec9715cc5442c6e7f13e57a49c6cfe664d32c
Reviewed-on: https://go-review.googlesource.com/c/go/+/184261
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ingo Oeser <nightlyone@googlemail.com>
2019-07-01 14:45:49 +00:00
Ian Lance Taylor
c485e8b559 runtime: use a pipe to wake up signal_recv on Darwin
The implementation of semaphores, and therefore notes, used on Darwin
is not async-signal-safe. The runtime has one case where a note needs
to be woken up from a signal handler: the call to notewakeup in sigsend.
That notewakeup call is only called on a single note, and it doesn't
need the full functionality of notes: nothing ever does a timed wait on it.
So change that one note to use a different implementation on Darwin,
based on a pipe. This lets the wakeup code use the write call, which is
async-signal-safe.

Fixes #31264

Change-Id: If705072d7a961dd908ea9d639c8d12b222c64806
Reviewed-on: https://go-review.googlesource.com/c/go/+/184169
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2019-06-30 05:48:31 +00:00
Russ Cox
623d653db7 crypto/tls: deflake localPipe in tests
The localPipe implementation assumes that every successful net.Dial
results in exactly one successful listener.Accept. I don't believe this
is guaranteed by essentially any operating system. For this test, we're
seeing flakes on dragonfly (#29583).

But see also #19519, flakes due to the same assumption on FreeBSD
and macOS in package net's own tests.

This CL rewrites localPipe to try a few times to get a matching pair
of connections on the dial and accept side.

Fixes #29583.

Change-Id: Idb045b18c404eae457f091df20456c5ae879a291
Reviewed-on: https://go-review.googlesource.com/c/go/+/184157
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-29 13:17:15 +00:00
Russ Cox
2e0cd2aef5 net: deflake TestVariousDeadlines
TestVariousDeadlines starts a client and server.
The client dials the server, sets a timeout on the connection,
reads from it, gets a timeout error, closes the connection.
The server writes an infinite stream of a's to each connection
it accepts.

The test was trying to run these in lockstep:
run a client dial+read+timeout+close,
wait for server to accept+write+error out on write to closed connection,
repeat.

On FreeBSD 11.2 and less frequently on macOS we see
the test timeout waiting for the server to do its half of
the lockstep dance.

I believe the problem is that the client can do its step
of the dance with such a short timeout that the read,
timeout, and close happens before the server ever returns
from the accept(2) system call. For the purposes of testing
the client-side read timeout, this is fine. But I suspect
that under some circumstances, the "TCP-accepted"
connection does not translate into a "socket-layer-accepted"
connection that triggers a return from accept(2).
That is, the Go server never sees the connection at all.
And the test sits there waiting for it to acknowledge
being done with a connection it never started with.

Fix the problem by not trying to lockstep with the server.

This definitely fixes the flake, since the specific line that
was calling t.Fatal is now deleted.

This exposes a different flake, seen on a trybot run for an
early version of this CL, in which the client's io.Copy does
not stop within the time allotted. The problem now is that
there is no guarantee that a read beyond the deadline with
available data returns an error instead of the available data,
yet the test assumes this guarantee, and in fact the opposite
is usually true - we don't bother checking the deadline unless
the read needs to block. That is, deadlines don't cut off a
flood of available data, yet this test thinks they do.

This CL therefore also changes the server not to send an
infinite flood of data - don't send any data at all - so that
the read deadline is guaranteed to be exercised.

Fixes #19519.

Change-Id: I58057c3ed94ac2aebab140ea597f317abae6e65e
Reviewed-on: https://go-review.googlesource.com/c/go/+/184137
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-28 20:10:05 +00:00
Russ Cox
91c385b393 runtime: fix pprof cpu profile corruption on arm/mips/mipsle
CL 42652 changed the profile handler for mips/mipsle to
avoid recording a profile when in atomic functions, for fear
of interrupting the 32-bit simulation of a 64-bit atomic with
a lock. The profile logger itself uses 64-bit atomics and might
deadlock (#20146).

The change was to accumulate a count of dropped profile events
and then send the count when the next ordinary event was sent:

	if prof.hz != 0 {
	+	if (GOARCH == "mips" || GOARCH == "mipsle") && lostAtomic64Count > 0 {
	+		cpuprof.addLostAtomic64(lostAtomic64Count)
	+		lostAtomic64Count = 0
	+	}
 		cpuprof.add(gp, stk[:n])
 	}

CL 117057 extended this behavior to include GOARCH == "arm".

Unfortunately, the inserted cpuprof.addLostAtomic64 differs from
the original cpuprof.add in that it neglects to acquire the lock
protecting the profile buffer.

This has caused a steady stream of flakes on the arm builders
for the past 12 months, ever since CL 117057 landed.

This CL moves the lostAtomic count into the profile buffer and
then lets the existing addExtra calls take care of it, instead of
duplicating the locking logic.

Fixes #24991.

Change-Id: Ia386c40034fcf46b31f080ce18f2420df4bb8004
Reviewed-on: https://go-review.googlesource.com/c/go/+/184164
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2019-06-28 20:09:48 +00:00
Dmitri Shuralyov
3b040b7e80 cmd/doc: provide working directory to build.Import calls
The current cmd/doc implementation uses go/build.Import in a few
places to check whether a package is findable and importable.
go/build has limited support for finding packages in modules,
but to do so, build.Import requires knowing the source directory
to use when performing the lookup (so it can find the go.mod file).
Otherwise, it only looks inside the GOPATH workspace.

Start passing the current working directory to build.Import calls,
so that it can correctly look for packages in modules when in cmd/doc
is executed in module mode.

Before this change, cmd/doc in module mode could mistakenly find and
use a package in the GOPATH workspace, instead of the current module.

Since the result of os.Getwd is needed in even more places, assign it
to a local variable in parseArgs now.

Fixes #28992
Updates #26504

Change-Id: I7571618e18420d2d3b3890cc69ade2d97b1962bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/183991
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-28 20:00:59 +00:00
Diogo Pinela
67f181bfd8 doc: fix spec links in Go 1.13 release notes
When reading tip.golang.org/doc/go1.13.html, the spec links in
the "Changes to the language" section should point to the updated
spec, not the old one.

Change-Id: I6b13ca0b4c722ed52b84a12a680bece876a4e478
Reviewed-on: https://go-review.googlesource.com/c/go/+/184118
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2019-06-27 22:31:35 +00:00
Robert Griesemer
b97ec8e57f text/scanner: remove AllowDigitSeparator flag again
The scanner was changed to accept the new Go number literal syntax
of which separators are a part. Making them opt-in is inconsistent
with the rest of the changes. For comparison, the strconv package
also accepts the new number literals including separators with the
various conversion routines, if no explicit number base is given.

Updates #28493.

Change-Id: Ifaae2225a9565364610813658bfe692901dd3ccd
Reviewed-on: https://go-review.googlesource.com/c/go/+/184080
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
2019-06-27 21:13:53 +00:00
Robert Griesemer
f205ae3305 go/importer: update documentation on ForCompiler lookup parameter
Change-Id: I80dff181a79858b52ba77c3f38f77a744b423afa
Reviewed-on: https://go-review.googlesource.com/c/go/+/184079
Reviewed-by: Michael Matloob <matloob@golang.org>
2019-06-27 20:43:26 +00:00
Jay Conrod
f1a8ca30fc go/build: don't check if srcDir in GOPATH when deciding to use modules
go/build.Context.Import loads package information using 'go list' when
in module mode. It does this when GO111MODULE is not "off", there is a
go.mod file in any parent directory, and neither the path nor the
source directory are in GOROOT. Import no longer checks whether the
source directory is in GOPATH if GO111MODULE=auto or unset.

Also fixed subdirectory checks that did not handle relative source
directory paths. mod_gobuild_import should have failed when we changed
the meaning of GO111MODULE=auto but didn't because of this.

Fixes #32799

Change-Id: Ic5210b7e00cb58f91ea9455b67b49d5aed4eec63
Reviewed-on: https://go-review.googlesource.com/c/go/+/184098
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-06-27 19:58:49 +00:00
Bryan C. Mills
0d4de70c1c misc/cgo/errors: align code snippets in ptr_test.go
Change-Id: Ic3e2819617375df653116d21d7361a46085250d5
Reviewed-on: https://go-review.googlesource.com/c/go/+/183986
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-06-27 17:26:05 +00:00
Damien Neil
e301e165ac Revert "syscall: use Ctty before fd shuffle"
This reverts commit 103b5b6692.

Reason for revert: Breaks valid existing programs.

Updates #29458

Change-Id: I7ace4ae404cf2a8b0e15e646663c50115f74b758
Reviewed-on: https://go-review.googlesource.com/c/go/+/183939
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Greg Thelen <gthelen@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-06-27 16:53:02 +00:00
Baokun Lee
998a98984b fmt: fix typo in errors.go
Fixes #32802

Change-Id: I756ca49285130b45777bd29de440db296d9632e9
Reviewed-on: https://go-review.googlesource.com/c/go/+/184057
Run-TryBot: Baokun Lee <nototon@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2019-06-27 04:16:38 +00:00
Heschi Kreinick
f938b9b33b cmd/go: fix typo in sumdb cache path
Put the sumdb cache in the usual pkg/mod/cache/download dir, rather
than the new pkg/mod/download/cache dir which I presume was a typo.

Change-Id: Id162f24db30f509353178ca0c8cc7a4dabc927e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/183318
Run-TryBot: Heschi Kreinick <heschi@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-06-26 20:26:48 +00:00
Bryan C. Mills
9acd2d60e7 misc/cgo/errors: fix TestPointerChecks when GO111MODULE=on
Change-Id: I4e9e46f03c9c43df1d0c6995f3baedd2e1a04c6a
Reviewed-on: https://go-review.googlesource.com/c/go/+/183985
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-06-26 20:17:35 +00:00
Bryan C. Mills
ab94ebbdcd doc/go1.13: make cmd/go paragraphs more concise
Looking at the live release notes on tip.golang.org, the Modules
section is much more verbose than the other sections.

To some extent that's to be expected, but too much detail in the
release notes might discourage folks from consulting the actual
documentation. Ensure that topics have clear links and omit
unnecessary details.

Change-Id: I1ccbc1697fccaf7ca7094c606bd11696c46d87f0
Reviewed-on: https://go-review.googlesource.com/c/go/+/183987
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-06-26 20:12:13 +00:00
Jay Conrod
fc26cbacff doc/go1.13: describe new 'go version' behavior
Change-Id: I18e2546f89e68e77d6e829acc997851751a44f0c
Reviewed-on: https://go-review.googlesource.com/c/go/+/183983
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-26 20:10:05 +00:00
Austin Clements
9caaac2c92 sync: only check for successful PopHeads in long mode
In TestPoolDequeue it's surprisingly common for the queue to stay
nearly empty the whole time and for a racing PopTail to happen in the
window between the producer doing a PushHead and doing a PopHead. In
short mode, there are only 100 PopTail attempts. On linux/amd64, it's
not uncommon for this to fail 50% of the time. On linux/arm64, it's
not uncommon for this to fail 100% of the time, causing the test to
fail.

This CL fixes this by only checking for a successful PopTail in long
mode. Long mode makes 200,000 PopTail attempts, and has never been
observed to fail.

Fixes #31422.

Change-Id: If464d55eb94fcb0b8d78fbc441d35be9f056a290
Reviewed-on: https://go-review.googlesource.com/c/go/+/183981
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-26 19:48:42 +00:00
Austin Clements
c19b3a60da sync: make TestPoolDequeue termination condition more robust
TestPoolDequeue creates P-1 consumer goroutines and 1 producer
goroutine. Currently, if a consumer goroutine pops the last value from
the dequeue, it sets a flag that stops all consumers, but the producer
also periodically pops from the dequeue and doesn't set this flag.

Hence, if the producer were to pop the last element, the consumers
will continue to run and the test won't terminate. This CL fixes this
by also setting the termination flag in the producer.

I believe it's impossible for this to happen right now because the
producer only pops after pushing an element for which j%10==0 and the
last element is either 999 or 1999999, which means it should never try
to pop after pushing the last element. However, we shouldn't depend on
this reasoning.

Change-Id: Icd2bc8d7cb9a79ebfcec99e367c8a2ba76e027d8
Reviewed-on: https://go-review.googlesource.com/c/go/+/183980
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-26 19:48:40 +00:00
Austin Clements
93c05627f9 sync: fix pool wrap-around test
TestPoolDequeue in long mode does a little more than 1<<21 pushes.
This was originally because the head and tail indexes were 21 bits and
the intent was to test wrap-around behavior. However, in the final
version they were both 32 bits, so the test no longer tested
wrap-around.

It would take too long to reach 32-bit wrap around in a test, so
instead we initialize the poolDequeue with indexes that are already
nearly at their limit. This keeps the knowledge of the maximum index
in one place, and lets us test wrap-around even in short mode.

Change-Id: Ib9b8d85b6d9b5be235849c2b32e81c809e806579
Reviewed-on: https://go-review.googlesource.com/c/go/+/183979
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-26 19:48:39 +00:00
Bryan C. Mills
4b41f7f818 cmd/go/internal/modfetch: halt proxy fallback if the proxy returns a non-404/410 response for @latest
The @latest proxy endpoint is optional. If a proxy returns a 404 for
it, and returns an @v/list with no matching versions, then we should
allow module lookup to try other module paths. However, if the proxy
returns some other error (say, a 403 or 505), then the result of the
lookup is ambiguous, and we should report the actual error rather than
"no matching versions for query".

(This fix was prompted by discussion with Dmitri on CL 183619.)

Updates #32715
Updates #26334

Change-Id: I6d510a5ac24d48d9bc5037c3c747ac50695c663f
Reviewed-on: https://go-review.googlesource.com/c/go/+/183845
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-06-26 19:14:29 +00:00
Bryan C. Mills
070e2dda02 cmd/go/internal/modfetch/codehost: don't try to execute a nonexistent fetch function
This fixes a panic observed in
https://build.golang.org/log/1242a032cac29fd3dffb24055fdeff0202b546be.

Change-Id: I029b9d69cabb87091734bd33d7fd39d30b12f9c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/183917
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-06-26 19:12:30 +00:00
Bryan C. Mills
4047f01d31 cmd/go: move mod_get_svn test to vcs-test.golang.org
The test currently usses llvm.org, which seems to be very flaky today.

Change-Id: I3d01476d53f94d9170dbb087e3f3cf99581cdb4d
Reviewed-on: https://go-review.googlesource.com/c/go/+/183847
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-06-26 19:11:12 +00:00
Bryan C. Mills
06f709a04a doc/go1.13: move 'go get -t' paragraph to be adjacent to 'go get -u' changes
As suggested by thepudds in CL 183630.

Also adjust the paragraph to harmonize the transitions between the
newly-adjacent paragraphs.

Change-Id: Ie85abea946db81804c1995d27be4951d5db6b812
Reviewed-on: https://go-review.googlesource.com/c/go/+/183918
Reviewed-by: Jay Conrod <jayconrod@google.com>
2019-06-26 18:38:59 +00:00
Bryan C. Mills
b21c837008 doc/go1.13: describe the change in behavior of GO111MODULE=auto
Updates #31857

Change-Id: Id0dc66246f768a2c730c67cd7a4a7830547f22a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/183922
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Katie Hockman <katie@golang.org>
2019-06-26 18:35:40 +00:00
Andrew Bonventre
60f14fddfe api: promote next to go1.13
Change-Id: I394f942822a0db27418d3fbb25df6227ea199a5f
Reviewed-on: https://go-review.googlesource.com/c/go/+/183919
Run-TryBot: Andrew Bonventre <andybons@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-26 16:35:14 +00:00
Cherry Zhang
4ea7aa7cf3 cmd/compile, runtime: use R20, R21 in ARM64's Duff's devices
Currently we use R16 and R17 for ARM64's Duff's devices.
According to ARM64 ABI, R16 and R17 can be used by the (external)
linker as scratch registers in trampolines. So don't use these
registers to pass information across functions.

It seems unlikely that calling Duff's devices would need a
trampoline in normal cases. But it could happen if the call
target is out of the 128 MB direct jump limit.

The choice of R20 and R21 is kind of arbitrary. The register
allocator allocates from low-numbered registers. High numbered
registers are chosen so it is unlikely to hold a live value and
forces a spill.

Fixes #32773.

Change-Id: Id22d555b5afeadd4efcf62797d1580d641c39218
Reviewed-on: https://go-review.googlesource.com/c/go/+/183842
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2019-06-26 16:01:47 +00:00
Jay Conrod
24f7d89a73 cmd/go: 'go get' should not delete binaries when run from $GOBIN
When 'go install' is run without arguments in a directory containing a
main package, it deletes an executable file with the same name as the
package (presumably created by 'go build' previously).

'go get' in module mode executes the same code after updating and
downloading modules. However, the special case was misfiring because
we passed an empty list of patterns to InstallPackages.

Fixes #32766

Change-Id: I19aca64ee1fb5a216777dd7d559e8e6a45b3e90c
Reviewed-on: https://go-review.googlesource.com/c/go/+/183846
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2019-06-26 15:02:32 +00:00
Tobias Klauser
9bf62783d2 syscall: fix nil pointer dereference in Getdirentries on 32-bit freebsd 12
Don't attempt to dereference basep if it's nil, just pass it to
getdirentries_freebsd12 as is.

Ported from x/sys/unix CL 183223

Change-Id: Id1c4e0eb6ff36dd39524da8194fed9a5957bce61
Reviewed-on: https://go-review.googlesource.com/c/go/+/183797
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-06-26 06:27:56 +00:00
Robert Griesemer
adf888376d go/types: avoid race condition with dot-imported objects
It would be nice to have a test, but it requires running
this under the race detector which is a bit complicated
to set up; yet the fix is trivial. Verified manually that
it doesn't trip the race detector.

Fixes #32154.

Change-Id: I20bd746a07945c802f0476a1d8b1dfd83c87dae8
Reviewed-on: https://go-review.googlesource.com/c/go/+/183849
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2019-06-26 03:59:57 +00:00
Ian Lance Taylor
6d1aaf143c misc/cgo/test: use char, not int, so test works on big-endian systems
Updates #32579
Fixes #32770

Change-Id: I32d1dea7505e8ad22e11a9806e10d096924b729b
Reviewed-on: https://go-review.googlesource.com/c/go/+/183778
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2019-06-25 21:47:04 +00:00
Keith Randall
343b7fa626 cmd/compile: don't mark argument array as noalg
It ends up making two similar types, [N]uint8 of both
alg and noalg varieties. Comparsions between the two then
don't come out equal when they should.
In particular, the type *[N]uint8 has an Elem pointer which
must point to one of the above two types; it can't point to both.
Thus allocating a *[N]uint8 and dereferencing it might be a
different type than a [N]uint8.

The fix is easy. Making a small test for this is really hard. It
requires that both a argless defer and the test be imported by a
common parent package. This is why a main binary doesn't see this
issue, but a test does (as Agniva noticed), because there's a wrapper
package that imports both the test and the defer.

Types like [N]uint8 don't really need to be marked noalg anyway,
as the generated code (if any) will be shared among all
vanilla memory types of the same size.

Fixes #32595

Change-Id: If7b77fa6ed56cd4495601c3f90170682d853b82f
Reviewed-on: https://go-review.googlesource.com/c/go/+/182357
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2019-06-25 21:30:39 +00:00