1
0
mirror of https://github.com/golang/go synced 2024-09-25 11:10:13 -06:00
Commit Graph

38599 Commits

Author SHA1 Message Date
Michael Ellis
dcb1363a1f cmd/go: parse dot-separated identifiers in build metadata
Per https://semver.org/#spec-item-10, build metadata may include
a series of dot separated identifiers.

Fixes #28647

Change-Id: I98655c62584a822953df71fba32b4a2cafe7a04b
Reviewed-on: https://go-review.googlesource.com/c/148835
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-11-19 20:34:23 +00:00
Alan Donovan
8bb3ae18db cmd/vendor: eliminate vet-lite
cmd/vet, now simplified to a single function call is now authoritative,
not a copy of vet-lite.

The update-xtools.sh script now uses the imports of cmd/vet as the
roots for vendoring.

Change-Id: I4faef3fcf3db10b3a3930726e8d0720a3c8395da
Reviewed-on: https://go-review.googlesource.com/c/150297
Run-TryBot: Alan Donovan <adonovan@google.com>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2018-11-19 17:33:37 +00:00
Alan Donovan
42b46585f1 cmd/go: improve go vet documentation
- restore and rework cmd/vet/doc.go, which was clobbered during the vet-lite switch.
- document go vet -vettool=prog flag and how to run an alternative checker.
- make 'go vet -help' show how to list vet tool's flags.  Example:

	$ go vet -help
	usage: go vet [-n] [-x] [-vettool prog] [build flags] [vet flags] [packages]
	Run 'go help vet' for details.
	Run 'go tool vet help' for the vet tool's flags.

	$ go vet -vettool=~/bin/myvet -help
	usage: go vet [-n] [-x] [-vettool prog] [build flags] [vet flags] [packages]
	Run 'go help vet' for details.
	Run '~/bin/myvet help' for the vet tool's flags.

Updates #28840

Change-Id: Ieb79dfe29e1df074f865bc9a9d47b44199675d7d
Reviewed-on: https://go-review.googlesource.com/c/147018
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-19 15:20:54 +00:00
Daniel Martí
3c92bdc7db cmd/vendor: update to golang.org/x/tools@139d099f
Mainly to pull the fix for the regression in #28792.

Change-Id: If71ae783fd9a9e3935186b49fdf501ba098235a2
Reviewed-on: https://go-review.googlesource.com/c/150161
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
2018-11-19 14:34:32 +00:00
Josh Bleecher Snyder
bc43889566 cmd/compile: bulk rename
This change does a bulk rename of several identifiers in the compiler.
See #27167 and https://docs.google.com/document/d/19_ExiylD9MRfeAjKIfEsMU1_RGhuxB9sA0b5Zv7byVI/
for context and for discussion of these particular renames.

Commands run to generate this change:

gorename -from '"cmd/compile/internal/gc".OPROC' -to OGO
gorename -from '"cmd/compile/internal/gc".OCOM' -to OBITNOT
gorename -from '"cmd/compile/internal/gc".OMINUS' -to ONEG
gorename -from '"cmd/compile/internal/gc".OIND' -to ODEREF
gorename -from '"cmd/compile/internal/gc".OARRAYBYTESTR' -to OBYTES2STR
gorename -from '"cmd/compile/internal/gc".OARRAYBYTESTRTMP' -to OBYTES2STRTMP
gorename -from '"cmd/compile/internal/gc".OARRAYRUNESTR' -to ORUNES2STR
gorename -from '"cmd/compile/internal/gc".OSTRARRAYBYTE' -to OSTR2BYTES
gorename -from '"cmd/compile/internal/gc".OSTRARRAYBYTETMP' -to OSTR2BYTESTMP
gorename -from '"cmd/compile/internal/gc".OSTRARRAYRUNE' -to OSTR2RUNES

gorename -from '"cmd/compile/internal/gc".Etop' -to ctxStmt
gorename -from '"cmd/compile/internal/gc".Erv' -to ctxExpr
gorename -from '"cmd/compile/internal/gc".Ecall' -to ctxCallee
gorename -from '"cmd/compile/internal/gc".Efnstruct' -to ctxMultiOK
gorename -from '"cmd/compile/internal/gc".Easgn' -to ctxAssign
gorename -from '"cmd/compile/internal/gc".Ecomplit' -to ctxCompLit

Not altered: parameters and local variables (mostly in typecheck.go) named top,
which should probably now be called ctx (and which should probably have a named type).
Also not altered: Field called Top in gc.Func.

gorename -from '"cmd/compile/internal/gc".Node.Isddd' -to IsDDD
gorename -from '"cmd/compile/internal/gc".Node.SetIsddd' -to SetIsDDD
gorename -from '"cmd/compile/internal/gc".nodeIsddd' -to nodeIsDDD
gorename -from '"cmd/compile/internal/types".Field.Isddd' -to IsDDD
gorename -from '"cmd/compile/internal/types".Field.SetIsddd' -to SetIsDDD
gorename -from '"cmd/compile/internal/types".fieldIsddd' -to fieldIsDDD

Not altered: function gc.hasddd, params and local variables called isddd
Also not altered: fmt.go prints nodes using "isddd(%v)".

cd cmd/compile/internal/gc; go generate

I then manually found impacted comments using exact string match
and fixed them up by hand. The comment changes were trivial.

Passes toolstash-check.

Fixes #27167. If this experiment is deemed a success,
we will open a new tracking issue for renames to do
at the end of the 1.13 cycles.

Change-Id: I2dc541533d2ab0d06cb3d31d65df205ecfb151e8
Reviewed-on: https://go-review.googlesource.com/c/150140
Run-TryBot: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2018-11-19 00:02:53 +00:00
Ian Lance Taylor
404ab866d1 cmd/go: packages that use SWIG depend on "unsafe"
Fixes #28834

Change-Id: I95d6874e62d36974415f43843881a4ae85b3c7ce
Reviewed-on: https://go-review.googlesource.com/c/149964
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Alan Donovan <adonovan@google.com>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
2018-11-18 17:00:40 +00:00
Tobias Klauser
760ac1dd33 os: make RemoveAll("") fail silently on unix
CL 146020 changed the behavior of RemoveAll("") on unix systems using
the *at functions to return syscall.EINVAL instead of nil. Adjust the
*at implementation to retain this behavior as is the case on the *noat
systems.

Additionally, also make sure RemoveAll("") on systems not using the "at
functions (e.g. nacl and js/wasm) follow the same behavior (which wasn't
the case previously).

Fixes #28830

Change-Id: I8383c1423fefe871d18ff49134a1d23077ec6867
Reviewed-on: https://go-review.googlesource.com/c/150158
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: roger peppe <rogpeppe@gmail.com>
2018-11-18 16:13:13 +00:00
sevki
5001a38cc1 crypto/hmac: rename CheckHMAC to ValidHMAC in package docs
Procedure names should reflect what they do; function names
should reflect what they return. Functions are used in
expressions, often in things like if's, so they need
to read appropriately.

        if CheckHMAC(a, b, key)

is unhelpful because we can't deduce whether CheckHMAC
returns true on error or non­-error; instead

        if ValidHMAC(a, b, key)

makes the point clear and makes a future mistake
in using the routine less likely.

https://www.lysator.liu.se/c/pikestyle.html

Change-Id: I7c4b1981c90c8d7475ddd8ec18dee3db2e0f42df
GitHub-Last-Rev: 32199a418b
GitHub-Pull-Request: golang/go#28823
Reviewed-on: https://go-review.googlesource.com/c/149857
Reviewed-by: Filippo Valsorda <filippo@golang.org>
2018-11-17 21:29:23 +00:00
Elias Naur
a889aaf8bf runtime: don't use thread local storage before it is set up on iOS
CL 138675 added a call to runtime.save_g which uses thread local
storage to store g. On iOS however, that storage was not initialized
 yet. Move the call to below _cgo_init where it is set up.

Change-Id: I14538d3e7d56ff35a6fa02c47bca306d24c38010
Reviewed-on: https://go-review.googlesource.com/c/150157
Run-TryBot: Elias Naur <elias.naur@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-11-17 20:33:30 +00:00
Alan Donovan
1f388bc806 cmd/vet: remove pkgfact analyzer, left in by mistake
Also, document process for updating vendored x/tools.

Change-Id: I826744603ae0752e508a6db7334a2bf9adaf1289
Reviewed-on: https://go-review.googlesource.com/c/149963
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-17 13:25:36 +00:00
Ian Lance Taylor
ba8f6fa0ca cmd/cgo: recognized untyped Go constants as untyped constants
Fixes #28772

Change-Id: I9446d95fb73fbcbb1cd9a4d2156ebc91bc9e91cb
Reviewed-on: https://go-review.googlesource.com/c/149858
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-16 23:30:19 +00:00
Austin Clements
9255688610 cmd/asm: rename -symabis to -gensymabis
Currently, both asm and compile have a -symabis flag, but in asm it's
a boolean flag that means to generate a symbol ABIs file and in the
compiler its a string flag giving the path of the symbol ABIs file to
consume. I'm worried about this false symmetry biting us in the
future, so rename asm's flag to -gensymabis.

Updates #27539.

Change-Id: I8b9c18a852d2838099718f8989813f19d82e7434
Reviewed-on: https://go-review.googlesource.com/c/149818
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-16 21:57:50 +00:00
Brad Fitzpatrick
b358987666 cmd/vet/all: remove skip when x/tools isn't in $GOPATH
Now that the build system has been updated to install x/tools in
$GOPATH (CL 149658), depend on it being there and don't ignore
failures to build the tool.

Update to CL 149097.

Change-Id: I72fde347217533697068b6a6773696cc2b83e9ed
Reviewed-on: https://go-review.googlesource.com/c/150017
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alan Donovan <adonovan@google.com>
2018-11-16 21:11:27 +00:00
Alan Donovan
a3f64237ad cmd/vet: basic tests of go vet -json -c=N flags
Change-Id: I787592a5d92ff9329ecdfcf879e491af66c8b749
Reviewed-on: https://go-review.googlesource.com/c/149962
Run-TryBot: Alan Donovan <adonovan@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-16 21:05:53 +00:00
Alan Donovan
b411c4268e cmd/vendor: update to golang.org/x/tools@8e5aba0a
- Adds 'go vet -json' and 'go vet -c=1' flags
- Removes the pkgfact analyzer, included by mistake.

Change-Id: Id3f1879af479efc567ea0508a1de7a37db5bee89
Reviewed-on: https://go-review.googlesource.com/c/149961
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Michael Matloob <matloob@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-16 20:37:19 +00:00
marwan-at-work
92caeef892 cmd/go: accept @hash/branch in mod download
Go get in mod-enabled packages lets you do go get "pkg@<hash>" or "pkg@<branch>".
Go internally will switch the hash or branch into a pseudo version.
Go mod download should do the same. The bug lay in the fact that the disk cache
was not being written when Go converted the hash/branch into a pseudo version.

Fixes #27947

Change-Id: I94c29a5c95f69ab18a9cd7a2ecade128047c5e36
GitHub-Last-Rev: 668634b3e7
GitHub-Pull-Request: golang/go#28042
Reviewed-on: https://go-review.googlesource.com/c/140257
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-16 19:46:17 +00:00
Alan Donovan
35244d8fd1 cmd/go: vet: provide package ID to the vet tool
This field, which matches the IDs used by go list, will enable all vet
drivers to produce JSON output in a consistent format (a map from
package ID to analysis name to result).

Change-Id: Icac703b944de55df42c996dc2f672005014ad57a
Reviewed-on: https://go-review.googlesource.com/c/149960
Reviewed-by: Russ Cox <rsc@golang.org>
2018-11-16 19:11:37 +00:00
Russ Cox
2a07c50a5c build: clear GO111MODULE during make.bash etc
The standard build assumes the variable is unset.
Make it so, like we do for GOFLAGS, GOBIN, and so on.

Change-Id: I4ad5695f8021b08bd1a35dd99112970a813d247c
Reviewed-on: https://go-review.googlesource.com/c/149959
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-16 18:54:22 +00:00
Alan Donovan
55e1fc930b cmd/vet: reenable cgo test
The reason the 386 trybot was happy but 'GOARCH=386 go test cmd/vet'
was not is that CgoEnabled defaults to false in a cross build;
I have no idea why. Now we ask the go command for the effective
value so that the test works in both cases.

Also, remove stale comment.

Fixes #28829

Change-Id: I1210af34da6986f47924059de5c1f08b2824ace9
Reviewed-on: https://go-review.googlesource.com/c/149958
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-16 17:51:20 +00:00
Russ Cox
b7ba523355 doc/go_spec: tweak wording to avoid 'explicit assignment' misreading
This text changed in CL 139099 to add "explicit" in front of "conversion".
But now "explicit conversion or assignment" reads like it might mean
"explicit [conversion or assignment]" when what is meant is
"[explicit conversion] or assignment". To make clear that explicit does
not apply to assignment, use "assignment or explicit conversion".

Change-Id: I8ff7a5b3ecd9f562793502fa6808242f22264f28
Reviewed-on: https://go-review.googlesource.com/c/149340
Reviewed-by: Robert Griesemer <gri@golang.org>
2018-11-16 17:50:06 +00:00
Russ Cox
94f7795d05 cmd/go: fix experiment isolation in cache key
In general we don't assume that the go command knows the
specific version of the compiler being used, including which
experiments the compiler was built with. Let the compiler tell us,
instead of importing cmd/internal/objabi from cmd/go.

Replacement for CL 128735.

Change-Id: Iaa07f46e19764d0fb14a1c89979bea7bb7139b9c
Reviewed-on: https://go-review.googlesource.com/c/149338
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-11-16 17:50:00 +00:00
Russ Cox
f53a986856 cmd/go: fix detection of unexpected files in downloaded zips
A bug in the old code was indirectly causing a confusing print,
but CL 131635 fixed the print instead of the surrounding code.
Fix the surrounding code, restore the old print, and test that the
error is actually reported (it was being ignored in a direct go get
but displaying in go build).

Change-Id: I03c21380fce481060c443b0cc820f3617497fdd9
Reviewed-on: https://go-review.googlesource.com/c/149317
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2018-11-16 17:49:55 +00:00
Robert Griesemer
69010963aa cmd/compile: provide updating mechanism for format test
The compiler's Format test verifies that the correct format
strings for the given arguments are used in the compiler
sources. The format strings are fairly specialized which is
why we cannot use go vet; and the mapping is based on a
hard-wired map.

In the past, if that map got out of sync with the compiler
sources, it was necessary to manually update the map. This
change introduces an update mechanism which simply requires
the test to be run with the -u flag.

(Formerly, the -u flag was used to automatically rewrite
format strings; now we use -r for that.)

Change-Id: I9259566a6120a13cf34b143875975ada62697890
Reviewed-on: https://go-review.googlesource.com/c/149460
Run-TryBot: Robert Griesemer <gri@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
2018-11-16 17:33:54 +00:00
Daniel Theophanes
8d335084cc database/sql: add examples for opening and testing a DB pool
Show two larger application examples. One example that
could be used in a CLI, the other in a long running
service. These demonstarates different strategies for
handling DB.Ping errors in context.

Fixes #23738

Change-Id: Id01213caf1f47917239a7506b01d30e37db74d31
Reviewed-on: https://go-review.googlesource.com/c/101216
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-16 17:17:09 +00:00
Leigh McCulloch
8f4bc468a7 testing: add example to package doc
The package doc for the testing package doesn't have a simple
example demonstrating how to write a test with an expectation. The doc
has simple examples for benchmarks, examples, and skipping, and it would be
useful for people new to writing tests in Go.

Also moved the skip example further down because it references tests and
benchmarks but benchmarks haven't been discussed in detail until the
next section. Skip is also a less used feature and it seems misplaced to
sit so high up in the package documentation. As an example, Skip is used
570 times the Go code repository which is significantly less than Error
and Fatal that are used 23,303 times.

Also changed 'sample' to 'simple' in other places in the package documentation
to keep the language used consistent when describing the small examples.

Fixes #27839

Change-Id: Ie01a3751986ee61adf2a2f2eda59cc182342baa7
GitHub-Last-Rev: 7357bfdcd2
GitHub-Pull-Request: golang/go#27840
Reviewed-on: https://go-review.googlesource.com/c/137175
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-11-16 16:49:09 +00:00
mbj36
b1e8846dfb net/http: fix typo in the SameSite docs
Fixes #28244

Change-Id: I3ca36fd513f5543af0c8af254d267254c7d5e803
GitHub-Last-Rev: 83b16fac4e
GitHub-Pull-Request: golang/go#28302
Reviewed-on: https://go-review.googlesource.com/c/143480
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-16 16:39:22 +00:00
Alan Donovan
41660d0086 cmd/vet: fix two failing test cases
Select linux/arm64 for the asm test.

Disable the cgo test for now.
Will fix properly in a follow-up.
Filed Issue 28829 to track it.

Updates #28829

Change-Id: Ic05f619700b06e91c43f8c150b089b8e77d92c85
Reviewed-on: https://go-review.googlesource.com/c/149937
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-16 14:58:23 +00:00
Vladimir Kovpak
9ffd5f31dc reflect: add comment for String method of Kind struct
On reflect documentation page only this function
doesn't have description, this commit add simple description.

Change-Id: Idcda89ddd1f6fdd1938c4030e89ebdc186255ce6
GitHub-Last-Rev: 1553b834bb
GitHub-Pull-Request: golang/go#28818
Reviewed-on: https://go-review.googlesource.com/c/149721
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-16 14:29:14 +00:00
Tobias Klauser
48e4d36fed misc/cgo/testsanitizers: gofmt
Change-Id: I4e7328bb89f504dbca3948b8565d22c44d41db3d
Reviewed-on: https://go-review.googlesource.com/c/149917
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-16 13:54:47 +00:00
Ian Lance Taylor
3096b85d37 cmd/cgo: fix comment grammar
Change-Id: I9c881943685177ce14841da53ccaed301c4955dd
Reviewed-on: https://go-review.googlesource.com/c/149859
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2018-11-16 13:47:05 +00:00
yuuji.yaginuma
03eb137aa3 cmd/go: correctly suggest tidy instead of nonexistent fix for -fix
CL 129682 removed go mod fix but unfortunately
we hadn't updated the source code hence running
   go mod -fix
would suggest
   go mod fix
which is a nonexistent command.

This change fixes that to instead suggest
   go mod tidy

Change-Id: Ie0d7c90805034e9fe6df24afaa15340c44d4f426
GitHub-Last-Rev: 5ae1340954
GitHub-Pull-Request: golang/go#28402
Reviewed-on: https://go-review.googlesource.com/c/144838
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-15 23:37:20 +00:00
Alan Donovan
cb4130996f cmd/vet: switch to x/tools/go/analysis implementation
This change deletes the legacy implementation of vet, replacing it
with a short main.go that merely selects the desired analyzers and
calls into the "unitchecker" implementation vendored from
golang.org/x/tools/go/analysis.

Unlike the full vet checker (x/tools/go/analysis/cmd/vet), the 'lite'
unitchecker cannot also be run standalone (as 'go tool vet' or
cmd/vet); it must be invoked by 'go vet'.
This design was chosen to avoid vendoring many
additional dependencies into GOROOT, in particular go/packages. If
go/packages should someday become part of the standard library, there
will be considerable opportunity for simplification.

This change also patches the vendored analysisflag package
(by adding patch.go) so that it fully supports the build
system's -V flag protocol.

Also:
- remove stale internal/unitchecker/ tree
  (belonged in https://go-review.googlesource.com/c/149778).
- move vet legacy flags (-all, -v, -source, -tags) into analysisflags
  as all drivers will need them, not just unitchecker.
  I will upstream this change.

A sampling of tests from the cmd/vet testsuite have been preserved as
a smoke test, to ensure that each analyzer is being run, and for
convenience when evaluating changes. Comprehensive tests for each
analyzer live upstream in x/tools. The tests have been heavily reduced
and reorganized so that they conform to the structure required by 'go
vet'.

Change-Id: I84b38caeef733e65deb95234b3b87b5f61046def
Reviewed-on: https://go-review.googlesource.com/c/149609
Reviewed-by: Russ Cox <rsc@golang.org>
2018-11-15 21:51:50 +00:00
Austin Clements
b00a6d8bfe runtime: eliminate mheap.busy* lists
The old whole-page reclaimer was the only thing that used the busy
span lists. Remove them so nothing uses them any more.

Change-Id: I4007dd2be08b9ef41bfdb0c387215c73c392cc4c
Reviewed-on: https://go-review.googlesource.com/c/138960
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2018-11-15 19:27:13 +00:00
Austin Clements
5333550bdc runtime: implement efficient page reclaimer
When we attempt to allocate an N page span (either for a large
allocation or when an mcentral runs dry), we first try to sweep spans
to release N pages. Currently, this can be extremely expensive:
sweeping a span to emptiness is the hardest thing to ask for and the
sweeper generally doesn't know where to even look for potentially
fruitful results. Since this is on the critical path of many
allocations, this is unfortunate.

This CL changes how we reclaim empty spans. Instead of trying lots of
spans and hoping for the best, it uses the newly introduced span marks
to efficiently find empty spans. The span marks (and in-use bits) are
in a dense bitmap, so these spans can be found with an efficient
sequential memory scan. This approach can scan for unmarked spans at
about 300 GB/ms and can free unmarked spans at about 32 MB/ms. We
could probably significantly improve the rate at which is can free
unmarked spans, but that's a separate issue.

Like the current reclaimer, this is still linear in the number of
spans that are swept, but the constant factor is now so vanishingly
small that it doesn't matter.

The benchmark in #18155 demonstrates both significant page reclaiming
delays, and object reclaiming delays. With "-retain-count=20000000
-preallocate=true -loop-count=3", the benchmark demonstrates several
page reclaiming delays on the order of 40ms. After this change, the
page reclaims are insignificant. The longest sweeps are still ~150ms,
but are object reclaiming delays. We'll address those in the next
several CLs.

Updates #18155.

Fixes #21378 by completely replacing the logic that had that bug.

Change-Id: Iad80eec11d7fc262d02c8f0761ac6998425c4064
Reviewed-on: https://go-review.googlesource.com/c/138959
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2018-11-15 19:27:11 +00:00
Austin Clements
ba1698e963 runtime: mark span when marking any object on the span
This adds a mark bit for each span that is set if any objects on the
span are marked. This will be used for sweeping.

For #18155.

The impact of this is negligible for most benchmarks, and < 1% for
GC-heavy benchmarks.

name                       old time/op  new time/op  delta
Garbage/benchmem-MB=64-12  2.18ms ± 0%  2.20ms ± 1%  +0.88%  (p=0.000 n=16+18)

(https://perf.golang.org/search?q=upload:20180928.1)

name                      old time/op    new time/op    delta
BinaryTree17-12              2.68s ± 1%     2.68s ± 1%    ~     (p=0.707 n=17+19)
Fannkuch11-12                2.28s ± 0%     2.39s ± 0%  +4.95%  (p=0.000 n=19+18)
FmtFprintfEmpty-12          40.3ns ± 4%    39.4ns ± 2%  -2.27%  (p=0.000 n=17+18)
FmtFprintfString-12         67.9ns ± 1%    68.3ns ± 1%  +0.55%  (p=0.000 n=18+19)
FmtFprintfInt-12            75.7ns ± 1%    76.1ns ± 1%  +0.44%  (p=0.005 n=18+19)
FmtFprintfIntInt-12          123ns ± 1%     121ns ± 1%  -1.00%  (p=0.000 n=18+18)
FmtFprintfPrefixedInt-12     150ns ± 0%     148ns ± 0%  -1.33%  (p=0.000 n=16+13)
FmtFprintfFloat-12           208ns ± 0%     204ns ± 0%  -1.92%  (p=0.000 n=13+17)
FmtManyArgs-12               501ns ± 1%     498ns ± 0%  -0.55%  (p=0.000 n=19+17)
GobDecode-12                6.24ms ± 0%    6.25ms ± 1%    ~     (p=0.113 n=20+19)
GobEncode-12                5.33ms ± 0%    5.29ms ± 1%  -0.72%  (p=0.000 n=20+18)
Gzip-12                      220ms ± 1%     218ms ± 1%  -1.02%  (p=0.000 n=19+19)
Gunzip-12                   35.5ms ± 0%    35.7ms ± 0%  +0.45%  (p=0.000 n=16+18)
HTTPClientServer-12         77.9µs ± 1%    77.7µs ± 1%  -0.30%  (p=0.047 n=20+19)
JSONEncode-12               8.82ms ± 0%    8.93ms ± 0%  +1.20%  (p=0.000 n=18+17)
JSONDecode-12               47.3ms ± 0%    47.0ms ± 0%  -0.49%  (p=0.000 n=17+18)
Mandelbrot200-12            3.69ms ± 0%    3.68ms ± 0%  -0.25%  (p=0.000 n=19+18)
GoParse-12                  3.13ms ± 1%    3.13ms ± 1%    ~     (p=0.640 n=20+20)
RegexpMatchEasy0_32-12      76.2ns ± 1%    76.2ns ± 1%    ~     (p=0.818 n=20+19)
RegexpMatchEasy0_1K-12       226ns ± 0%     226ns ± 0%  -0.22%  (p=0.001 n=17+18)
RegexpMatchEasy1_32-12      71.9ns ± 1%    72.0ns ± 1%    ~     (p=0.653 n=18+18)
RegexpMatchEasy1_1K-12       355ns ± 1%     356ns ± 1%    ~     (p=0.160 n=18+19)
RegexpMatchMedium_32-12      106ns ± 1%     106ns ± 1%    ~     (p=0.325 n=17+20)
RegexpMatchMedium_1K-12     31.1µs ± 2%    31.2µs ± 0%  +0.59%  (p=0.007 n=19+15)
RegexpMatchHard_32-12       1.54µs ± 2%    1.53µs ± 2%  -0.78%  (p=0.021 n=17+18)
RegexpMatchHard_1K-12       46.0µs ± 1%    45.9µs ± 1%  -0.31%  (p=0.025 n=17+19)
Revcomp-12                   391ms ± 1%     394ms ± 2%  +0.80%  (p=0.000 n=17+19)
Template-12                 59.9ms ± 1%    59.9ms ± 1%    ~     (p=0.428 n=20+19)
TimeParse-12                 304ns ± 1%     312ns ± 0%  +2.88%  (p=0.000 n=20+17)
TimeFormat-12                318ns ± 0%     326ns ± 0%  +2.64%  (p=0.000 n=20+17)

(https://perf.golang.org/search?q=upload:20180928.2)

Change-Id: I336b9bf054113580a24103192904c8c76593e90e
Reviewed-on: https://go-review.googlesource.com/c/138958
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2018-11-15 19:27:09 +00:00
Austin Clements
69e666e4f7 runtime: record in-use spans in a page-indexed bitmap
This adds a bitmap indexed by page number that marks the starts of
in-use spans. This will be used to quickly find in-use spans with no
marked objects for sweeping.

For #18155.

Change-Id: Icee56f029cde502447193e136fa54a74c74326dd
Reviewed-on: https://go-review.googlesource.com/c/138957
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
2018-11-15 19:27:06 +00:00
Austin Clements
e500ffd88c runtime: track all heap arenas in a slice
Currently, there's no efficient way to iterate over the Go heap. We're
going to need this for fast free page sweeping, so this CL adds a
slice of all allocated heap arenas. This will also be useful for
generational GC.

For #18155.

Change-Id: I58d126cfb9c3f61b3125d80b74ccb1b2169efbcc
Reviewed-on: https://go-review.googlesource.com/c/138076
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rick Hudson <rlh@golang.org>
2018-11-15 19:14:10 +00:00
Alan Donovan
b2df0bd5f5 cmd/vet/all: use x/tools/go/analysis/cmd/vet not cmd/vet
cmd/vet/all applies vet to all packages in the standard tree.
It is run for every configuration using this command:
  GO_BUILDER_NAME=misc-vetall go tool dist test
by the misc-vetall builder (see chart at build.golang.org).

Ideally we would switch to 'go vet', but it effectively does a partial
build. This means that its analysis has accurate type information, so
it reports slightly fewer spurious diagnostics. However, it is more
than twice as slow.

Instead, cmd/vet/all builds and runs
golang.org/x/tools/go/analysis/cmd/vet, which uses x/tools/go/packages
to load the entire std lib from source. It takes about 4min to run all
OS/ARCH pairs. An important consequence is that golang.org/x/tools
must be on your $GOPATH to run cmd/vet/all. The test has been
temporarily modified to warn and skip if this is not the case.

This is a preparatory step for switching to the new
cmd/vet based on vet-lite.

Whitelist changes:
- The two "deadcode" diagnostics removed from the whitelist were due
  to if-conditions that could now be proven false.
- The asmdecl warnings are now printed with the log.Printf prefix,
  so they are discarded by the parser and needn't be whitelisted.

Change-Id: I6486508b0de2cd947c897523af086a408cbaf4a8
Reviewed-on: https://go-review.googlesource.com/c/149097
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2018-11-15 18:46:33 +00:00
Alan Donovan
a18638c0f2 cmd/vendor: update to golang.org/x/tools@f62bfb54
Change-Id: I3b3035784ce89ba2ac5ab8f6448c45a3d38fa97d
Reviewed-on: https://go-review.googlesource.com/c/149778
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-15 17:40:19 +00:00
Keith Randall
82c260099d x/net/route: use libc calls on Darwin
Starting with 1.12, we must use syscall versions
of sysctl instead of the raw syscall.

An identical CL went into the source copy at golang.org/x/net/route.
This is just a cherry pick of that CL.
(CL: https://go-review.googlesource.com/c/net/+/148597)

Change-Id: I6286ab3e49f82512491afb5bcf349e89ab5645ab
Reviewed-on: https://go-review.googlesource.com/c/149637
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-14 22:39:08 +00:00
Austin Clements
8dba66dfeb cmd/link: fix isStmt DWARF info
When CL 147160 introduced function ABIs encoded as symbol versions in
the linker, it became slightly more complicated to look up derived
DWARF symbols. It fixed this by introducing a dwarfFuncSym function to
hide this logic, but missed one derived lookup that was done in the
object reader itself. As a result, we lost the isStmt tables from the
compiler, so every PC was marked as a statement in the DWARF info.

Fix this by moving this derived lookup out of the object reader and
into the DWARF code and calling dwarfFuncSym to get the correctly
versioned symbol.

Should fix the linux-amd64-longtest builder.

Updates #27539.

Change-Id: If40d5ba28bab1918ac4ad18fbb5103666b6d978b
Reviewed-on: https://go-review.googlesource.com/c/149605
Run-TryBot: Austin Clements <austin@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: David Chase <drchase@google.com>
2018-11-14 22:01:38 +00:00
Martin Möhrmann
2de53906e1 internal/cpu: move GODEBUGCPU options into GODEBUG
Change internal/cpu feature configuration to use
GODEBUG=cpu.feature1=value,cpu.feature2=value...
instead of GODEBUGCPU=feature1=value,feature2=value... .

This is not a backwards compatibility breaking change
since GODEBUGCPU was introduced in go1.11 as an
undocumented compiler experiment.

Fixes #28757

Change-Id: Ib21b3fed2334baeeb061a722ab1eb513d1137e87
Reviewed-on: https://go-review.googlesource.com/c/149578
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-11-14 21:47:50 +00:00
Alan Donovan
89a7917493 cmd/vendor/golang.org/x/tools: update to 7d6b83ca
Also, add a script for future updates.

Change-Id: I2565d1f26532b9dd7cf9d8ce198ba08fb3d53407
Reviewed-on: https://go-review.googlesource.com/c/149604
Run-TryBot: Alan Donovan <adonovan@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
2018-11-14 21:08:11 +00:00
Brad Fitzpatrick
bfd9b94069 net/http: make Transport respect {X-,}Idempotency-Key header
Fixes #19943

Change-Id: I5e0fefe44791d7b3556095d726c2a753ec551ef2
Reviewed-on: https://go-review.googlesource.com/c/147457
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2018-11-14 21:07:02 +00:00
Keith Randall
0c7762cd18 cmd/compile: assume unsafe pointer arithmetic generates non-nil results
I've never seen a case where unsafe arithmetic is used to generate a nil.
(Something like var x uintptr; unsafe.Pointer(x - x).)
We can assume that if someone is doing arithmetic with pointers, the
result will be non-nil. Our unsafe rules already forbid this, although
we should be more explicit.

RELNOTE=It is invalid to convert a nil unsafe.Pointer to uintptr and back, with arithmetic.
(This was already invalid, but this statement has been added for clarification.)

Fixes #27180

Change-Id: I1880b7725a9fd99e4613799930fdad9aaa99e8f0
Reviewed-on: https://go-review.googlesource.com/c/146058
Reviewed-by: Austin Clements <austin@google.com>
2018-11-14 21:01:36 +00:00
Ali Rizvi-Santiago
a063a2284a cmd/go: allow the user to specify ar via an environment variable
This allows one to customize which ar to use by fetching its path
from the environment. This way one can swap it out for a
different implementation.

Change-Id: I40d8cbd8a69e97b5254e66081d9bf0b726c10366
GitHub-Last-Rev: 4aa1d631ea
GitHub-Pull-Request: golang/go#28746
Reviewed-on: https://go-review.googlesource.com/c/149117
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2018-11-14 20:33:24 +00:00
Martin Möhrmann
f36e92dbfc fmt: avoid allocation when formatting byte slice arguments with verb s
fmtBytes is in the top 10 callers of runtime.slicebytetostring according
to Google wide profiling data.

Avoid the string conversion of the input byte slice in fmtBytes by calling
a newly added specialized fmtS function for byte slices.

Expand tests for verb s with widths to test strings and byte slice arguments.

SprintfTruncateString     157ns ± 4%     156ns ± 3%     ~     (p=0.122 n=20+20)
SprintfTruncateBytes      188ns ± 2%     155ns ± 3%  -18.00%  (p=0.000 n=20+19)

name                   old alloc/op   new alloc/op   delta
SprintfTruncateString     16.0B ± 0%     16.0B ± 0%     ~     (all equal)
SprintfTruncateBytes      64.0B ± 0%     16.0B ± 0%  -75.00%  (p=0.000 n=20+20)

name                   old allocs/op  new allocs/op  delta
SprintfTruncateString      1.00 ± 0%      1.00 ± 0%     ~     (all equal)
SprintfTruncateBytes       2.00 ± 0%      1.00 ± 0%  -50.00%  (p=0.000 n=20+20)

Change-Id: I461bf514d4232b39bd9c812f7faa4e5ef693a03b
Reviewed-on: https://go-review.googlesource.com/c/145284
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2018-11-14 20:32:15 +00:00
Martin Möhrmann
75798e8ada runtime: make processor capability variable naming platform specific
The current support_XXX variables are specific for the
amd64 and 386 platforms.

Prefix processor capability variables by architecture to have a
consistent naming scheme and avoid reuse of the existing
variables for new platforms.

This also aligns naming of runtime variables closer with internal/cpu
processor capability variable names.

Change-Id: I3eabb29a03874678851376185d3a62e73c1aff1d
Reviewed-on: https://go-review.googlesource.com/c/91435
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
2018-11-14 20:30:31 +00:00
Mark Rushakoff
529ea7c0de cmd/go: don't panic when go run is passed ... under nonexistent dir
Given a nonexistent directory above a wildcard:

    go run ./nonexistent/...

Print this error instead of panicking:

    go run: no packages loaded from ./nonexistent/...

Fixes #28696.

Change-Id: Iaa3bc5c78b14ef858d931778e1bc55ca626c5571
GitHub-Last-Rev: bb1a80483a
GitHub-Pull-Request: golang/go#28703
Reviewed-on: https://go-review.googlesource.com/c/148821
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2018-11-14 19:59:17 +00:00
Milan Knezevic
c92e73b702 cmd/compile/internal/gc: OMUL should be evaluated when using soft-float
When using soft-float, OMUL might be rewritten to function call
so we should ensure it was evaluated first.

Fixes #28688

Change-Id: I30b87501782fff62d35151f394a1c22b0d490c6c
Reviewed-on: https://go-review.googlesource.com/c/148837
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2018-11-14 18:52:15 +00:00