1
0
mirror of https://github.com/golang/go synced 2024-11-12 13:10:33 -07:00
Commit Graph

45065 Commits

Author SHA1 Message Date
Changkun Ou
a408139bb0 testing: fix panicking tests hang if Cleanup calls FailNow
Previously, it was impossible to call FailNow in a Cleanup.
Because it can terminate a panicking goroutine and cause its
parent hangs on t.signal channel. This CL sends the signal
in a deferred call to prevent the hang.

Fixes #41355

Change-Id: I4552d3a7ea763ef86817bf9b50c0e37fb34bf20f
Reviewed-on: https://go-review.googlesource.com/c/go/+/254637
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-14 20:27:49 +00:00
Bryan C. Mills
5764653429 cmd/api: omit outside dependencies when listing the packages in "std"
As of CL 251159, when 'go list -deps std' is run within GOROOT/src, it
treats the vendored external dependencies as real module dependencies,
not standard-library "vendor/" packages (which still exist in that
case, but are treated as distinct packages outside the "std" module).

Fixes #41358
Updates #30241

Change-Id: Ic23eae9829d90e74a340d49ca9052e9191597410
Reviewed-on: https://go-review.googlesource.com/c/go/+/254738
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Trust: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-14 19:45:28 +00:00
Martin Möhrmann
14c7caae50 runtime: add 24 byte allocation size class
This CL introduces a 24 byte allocation size class which
fits 3 pointers on 64 bit and 6 pointers on 32 bit architectures.

Notably this new size class fits a slice header on 64 bit
architectures exactly while previously a 32 byte size class
would have been used for allocating a slice header on the heap.

The main complexity added with this CL is that heapBitsSetType
needs to handle objects that aren't 16-byte aligned but contain
more than a single pointer on 64-bit architectures.

Due to having a non 16 byte aligned size class on 32 bit a
h.shift of 2 is now possible which means a heap bitmap byte might
only be partially written. Due to this already having been
possible on 64 bit before the heap bitmap code only needed
minor adjustments for 32 bit doublecheck code paths.

Note that this CL changes the slice capacity allocated by append
for slice growth to a target capacity of 17 to 24 bytes.

On 64 bit architectures the capacity of the slice returned by
append([]byte{}, make([]byte, 24)...)) is 32 bytes before and
24 bytes after this CL. Depending on allocation patterns of the
specific Go program this can increase the number of total
alloctions as subsequent appends to the slice can trigger slice
growth earlier than before. On the other side if the slice is
never appended to again above its capacity this will lower heap
usage by 8 bytes.

This CL changes the set of size classes reported in the
runtime.MemStats.BySize array due to it being limited to a
total of 61 size classes. The new 24 byte size class is now
included and the 20480 byte size class is not included anymore.

Fixes #8885

name                      old time/op       new time/op       delta
Template                        196ms ± 3%        194ms ± 2%    ~     (p=0.247 n=10+10)
Unicode                        85.6ms ±16%       88.1ms ± 1%    ~     (p=0.165 n=10+10)
GoTypes                         673ms ± 2%        668ms ± 2%    ~     (p=0.258 n=9+9)
Compiler                        3.14s ± 6%        3.08s ± 1%    ~     (p=0.243 n=10+9)
SSA                             6.82s ± 1%        6.76s ± 1%  -0.87%  (p=0.006 n=9+10)
Flate                           128ms ± 7%        127ms ± 3%    ~     (p=0.739 n=10+10)
GoParser                        154ms ± 3%        153ms ± 4%    ~     (p=0.730 n=9+9)
Reflect                         404ms ± 1%        412ms ± 4%  +1.99%  (p=0.022 n=9+10)
Tar                             172ms ± 4%        170ms ± 4%    ~     (p=0.065 n=10+9)
XML                             231ms ± 4%        230ms ± 3%    ~     (p=0.912 n=10+10)
LinkCompiler                    341ms ± 1%        339ms ± 1%    ~     (p=0.243 n=9+10)
ExternalLinkCompiler            1.72s ± 1%        1.72s ± 1%    ~     (p=0.661 n=9+10)
LinkWithoutDebugCompiler        221ms ± 2%        221ms ± 2%    ~     (p=0.529 n=10+10)
StdCmd                          18.4s ± 3%        18.2s ± 1%    ~     (p=0.515 n=10+8)

name                      old user-time/op  new user-time/op  delta
Template                        238ms ± 4%        243ms ± 6%    ~     (p=0.661 n=9+10)
Unicode                         116ms ± 6%        113ms ± 3%  -3.37%  (p=0.035 n=9+10)
GoTypes                         854ms ± 2%        848ms ± 2%    ~     (p=0.604 n=9+10)
Compiler                        4.10s ± 1%        4.11s ± 1%    ~     (p=0.481 n=8+9)
SSA                             9.49s ± 1%        9.41s ± 1%  -0.92%  (p=0.001 n=9+10)
Flate                           149ms ± 6%        151ms ± 7%    ~     (p=0.481 n=10+10)
GoParser                        189ms ± 2%        190ms ± 2%    ~     (p=0.497 n=9+10)
Reflect                         511ms ± 2%        508ms ± 2%    ~     (p=0.211 n=9+10)
Tar                             215ms ± 4%        212ms ± 3%    ~     (p=0.105 n=10+10)
XML                             288ms ± 2%        288ms ± 2%    ~     (p=0.971 n=10+10)
LinkCompiler                    559ms ± 4%        557ms ± 1%    ~     (p=0.968 n=9+10)
ExternalLinkCompiler            1.78s ± 1%        1.77s ± 1%    ~     (p=0.055 n=8+10)
LinkWithoutDebugCompiler        245ms ± 3%        245ms ± 2%    ~     (p=0.684 n=10+10)

name                      old alloc/op      new alloc/op      delta
Template                       34.8MB ± 0%       34.4MB ± 0%  -0.95%  (p=0.000 n=9+10)
Unicode                        28.6MB ± 0%       28.3MB ± 0%  -0.95%  (p=0.000 n=10+10)
GoTypes                         115MB ± 0%        114MB ± 0%  -1.02%  (p=0.000 n=10+9)
Compiler                        554MB ± 0%        549MB ± 0%  -0.86%  (p=0.000 n=9+10)
SSA                            1.28GB ± 0%       1.27GB ± 0%  -0.83%  (p=0.000 n=10+10)
Flate                          21.8MB ± 0%       21.6MB ± 0%  -0.87%  (p=0.000 n=8+10)
GoParser                       26.7MB ± 0%       26.4MB ± 0%  -0.97%  (p=0.000 n=10+9)
Reflect                        75.0MB ± 0%       74.1MB ± 0%  -1.18%  (p=0.000 n=10+10)
Tar                            32.6MB ± 0%       32.3MB ± 0%  -0.94%  (p=0.000 n=10+7)
XML                            41.5MB ± 0%       41.2MB ± 0%  -0.90%  (p=0.000 n=10+8)
LinkCompiler                    105MB ± 0%        104MB ± 0%  -0.94%  (p=0.000 n=10+10)
ExternalLinkCompiler            153MB ± 0%        152MB ± 0%  -0.69%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler       63.7MB ± 0%       63.6MB ± 0%  -0.13%  (p=0.000 n=10+10)

name                      old allocs/op     new allocs/op     delta
Template                         336k ± 0%         336k ± 0%  +0.02%  (p=0.002 n=10+10)
Unicode                          332k ± 0%         332k ± 0%    ~     (p=0.447 n=10+10)
GoTypes                         1.16M ± 0%        1.16M ± 0%  +0.01%  (p=0.001 n=10+10)
Compiler                        4.92M ± 0%        4.92M ± 0%  +0.01%  (p=0.000 n=10+10)
SSA                             11.9M ± 0%        11.9M ± 0%  +0.02%  (p=0.000 n=9+10)
Flate                            214k ± 0%         214k ± 0%  +0.02%  (p=0.032 n=10+8)
GoParser                         270k ± 0%         270k ± 0%  +0.02%  (p=0.004 n=10+9)
Reflect                          877k ± 0%         877k ± 0%  +0.01%  (p=0.000 n=10+10)
Tar                              313k ± 0%         313k ± 0%    ~     (p=0.075 n=9+10)
XML                              387k ± 0%         387k ± 0%  +0.02%  (p=0.007 n=10+10)
LinkCompiler                     455k ± 0%         456k ± 0%  +0.08%  (p=0.000 n=10+9)
ExternalLinkCompiler             670k ± 0%         671k ± 0%  +0.06%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler         113k ± 0%         113k ± 0%    ~     (p=0.149 n=10+10)

name                      old maxRSS/op     new maxRSS/op     delta
Template                        34.1M ± 1%        34.1M ± 1%    ~     (p=0.853 n=10+10)
Unicode                         35.1M ± 1%        34.6M ± 1%  -1.43%  (p=0.000 n=10+10)
GoTypes                         72.8M ± 3%        73.3M ± 2%    ~     (p=0.724 n=10+10)
Compiler                         288M ± 3%         295M ± 4%    ~     (p=0.393 n=10+10)
SSA                              630M ± 1%         622M ± 1%  -1.18%  (p=0.001 n=10+10)
Flate                           26.0M ± 1%        26.2M ± 2%    ~     (p=0.493 n=10+10)
GoParser                        28.6M ± 1%        28.5M ± 2%    ~     (p=0.256 n=10+10)
Reflect                         55.5M ± 2%        55.4M ± 1%    ~     (p=0.436 n=10+10)
Tar                             33.0M ± 1%        32.8M ± 2%    ~     (p=0.075 n=10+10)
XML                             38.7M ± 1%        39.0M ± 1%    ~     (p=0.053 n=9+10)
LinkCompiler                     164M ± 1%         164M ± 1%  -0.27%  (p=0.029 n=10+10)
ExternalLinkCompiler             174M ± 0%         173M ± 0%  -0.33%  (p=0.002 n=9+10)
LinkWithoutDebugCompiler         137M ± 0%         136M ± 2%    ~     (p=0.825 n=9+10)

Change-Id: I9ecf2a10024513abef8fbfbe519e44e0b29b6167
Reviewed-on: https://go-review.googlesource.com/c/go/+/242258
Trust: Martin Möhrmann <moehrmann@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-14 19:21:56 +00:00
Cherry Zhang
c0c396bd6a misc/ios: quote paths
The paths may contain spaces. Quote them.

Change-Id: I1f67085a1e7c40f60282c2fea7104fb44a01e310
Reviewed-on: https://go-review.googlesource.com/c/go/+/254739
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-14 18:28:23 +00:00
Clément Chigot
86dbeefe1f syscall: fix fsync for read-only files on aix
AIX fsync syscall doesn't work on read-only files. Using fsync_range
instead allows syscall.Fsync to work on any files.

Fixes #41372

Change-Id: I66d33e847875496af53da60828c1bddf6c2b76b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/254657
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-14 17:07:47 +00:00
Roberto Clapis
4f5cd0c033 net/http/cgi,net/http/fcgi: add Content-Type detection
This CL ensures that responses served via CGI and FastCGI
have a Content-Type header based on the content of the
response if not explicitly set by handlers.

If the implementers of the handler did not explicitly
specify a Content-Type both CGI implementations would default
to "text/html", potentially causing cross-site scripting.

Thanks to RedTeam Pentesting GmbH for reporting this.

Fixes #40928
Fixes CVE-2020-24553

Change-Id: I82cfc396309b5ab2e8d6e9a87eda8ea7e3799473
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/823217
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/252179
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2020-09-14 15:42:03 +00:00
Dominic Della Valle
66e66e7113 make.bat: fix compare between GOROOT and srcdir paths, when either contains whitespace.
CL 96455 brings CL 57753 to Windows
However, a path comparison within it was left unquoted.
If the Go source directory resides in a path containing whitespace,
the interpreter will compare against the first portion of the path string,
and treat the remainder as an expression.
This patch amends that.

For example, consider the path
`C:\Users\Dominic Della Valle\Projects\Go\goroot\src`
Issuing `make.bat` will print out `'Della' is not recognized as an internal or external command, operable program or batch file.` before proceeding.

Change-Id: Ifcec159baeec940c29c61aa721c64c13c6fd8c14
GitHub-Last-Rev: 809ddbb4db
GitHub-Pull-Request: golang/go#41319
Reviewed-on: https://go-review.googlesource.com/c/go/+/253898
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Giovanni Bajo <rasky@develer.com>
Trust: Giovanni Bajo <rasky@develer.com>
2020-09-13 19:17:09 +00:00
Cuong Manh Le
afb5fca25a test: fix flaky test for issue24491
runtime.GC() doesn't guarantee the finalizer has run, so use a channel
instead to make sure finalizer was run in call to "after()".

Fixes #41361

Change-Id: I69c801e29aea49757ea72c52e8db13239de19ddc
Reviewed-on: https://go-review.googlesource.com/c/go/+/254401
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2020-09-13 07:54:42 +00:00
Cuong Manh Le
5f1b12bfbe cmd/compile: remove nodeNeedsWrapper flag
CL 254397 attached OVARLIVE nodes to OCALLxxx nodes Nbody.

The NeedsWrapper flag is now redundant with n.Nbody.Len() > 0
condition, so use that condition instead and remove the flag.

Passes toolstash-check.

Change-Id: Iebc3e674d3c0040a876ca4be05025943d2b4fb31
Reviewed-on: https://go-review.googlesource.com/c/go/+/254398
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-09-13 04:35:35 +00:00
Cuong Manh Le
1f45216694 cmd/compile: attach OVARLIVE nodes to OCALLxxx
So we can insert theses OVARLIVE nodes right after OpStaticCall in SSA.

This helps fixing issue that unsafe-uintptr arguments are not kept alive
during return statement, or can be kept alive longer than expected.

Fixes #24491

Change-Id: Ic04a5d1bbb5c90dcfae65bd95cdd1da393a66800
Reviewed-on: https://go-review.googlesource.com/c/go/+/254397
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-09-13 04:35:22 +00:00
Carlos Alexandro Becker
95bb00d108 encoding/json: implement Is on all errors
Allows users to check:

      errors.Is(err, &UnmarshalTypeError{})
      errors.Is(err, &UnmarshalFieldError{})
      errors.Is(err, &InvalidUnmarshalError{})
      errors.Is(err, &UnsupportedValueError{})
      errors.Is(err, &MarshalerError{})

which is the recommended way of checking for kinds of errors.

SyntaxError.Is was implemented in CL 253037.
As and Unwrap relevant methods will be added in future CLs.

Change-Id: I1f8a503b8fdc0f3afdfe9669a91f3af8d960e028
GitHub-Last-Rev: 930cda5384
GitHub-Pull-Request: golang/go#41360
Reviewed-on: https://go-review.googlesource.com/c/go/+/254537
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-13 03:19:24 +00:00
Carlos Alexandro Becker
b3ef90ec73 encoding/json: implement Is on SyntaxError
Allows users to check:

      errors.Is(err, &json.SyntaxError{})

which is the recommended way of checking for kinds of errors.

Change-Id: I20dc805f20212765e9936a82d9cb7822e73ec4ef
GitHub-Last-Rev: e2627ccf8e
GitHub-Pull-Request: golang/go#41210
Reviewed-on: https://go-review.googlesource.com/c/go/+/253037
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-12 19:42:29 +00:00
Daniel Martí
92b2b8860d cmd/go: avoid flag.FlagSet.VisitAll at init time
We want to error early if GOFLAGS contains any flag that isn't known to
any cmd/go command. Thus, at init time we would recursively use VisitAll
on each of the flagsets to populate a map of all registered flags.

This was unfortunate, as populating said map constituted a whole 5% of
the run-time of 'go env GOARCH'. This is because VisitAll is pretty
expensive; it copies all the maps from the flagset's map to a slice,
sorts the slice, then does one callback per flag.

First, this was a bit wasteful. We only ever needed to query the
knownFlag map if GOFLAGS wasn't empty. If it's empty, there's no work to
do, thus we can skip the map populating work.

Second and most important, we don't actually need the map at all. A
flag.FlagSet already has a Lookup method, so we can simply recursively
call those methods for each flag in GOFLAGS. Add a hasFlag func to make
that evident.

This mechanism is different; its upfront cost is none, but it will
likely mean a handful of map lookups for each flag in GOFLAGS. However,
that tradeoff is worth it; we don't expect GOFLAGS to contain thousands
of flags. The most likely scenario is less than a dozen flags, in which
case constructing a "unified" map is not at all a net win.

One possible reason the previous mechanism was that way could be
AddKnownFlag. Thankfully, the one and only use of that API was removed
last year when Bryan cleaned up flag parsing in cmd/go.

The wins for the existing benchmark with an empty GOFLAGS are
significant:

	name         old time/op       new time/op       delta
	ExecGoEnv-8        575µs ± 1%        549µs ± 2%  -4.44%  (p=0.000 n=7+8)

	name         old sys-time/op   new sys-time/op   delta
	ExecGoEnv-8       1.69ms ± 1%       1.68ms ± 2%    ~     (p=0.281 n=7+8)

	name         old user-time/op  new user-time/op  delta
	ExecGoEnv-8       1.80ms ± 1%       1.66ms ± 2%  -8.09%  (p=0.000 n=7+8)

To prove that a relatively large number of GOFLAGS isn't getting
noticeably slower, we measured that as well, via benchcmd and GOFLAGS
containing 50 valid flags:

	GOFLAGS=$(yes -- -race | sed 50q) benchcmd -n 500 GoEnvGOFLAGS go env GOARCH

And the result, while noisy, shows no noticeable difference (note that
it measures 3ms instead of 0.6ms since it's sequential):

	name          old time/op         new time/op         delta
	GoEnvGOFLAGS         3.04ms ±32%         3.03ms ±35%    ~     (p=0.156 n=487+481)

Finally, we've improved the existing Go benchmark. Now it's parallel,
and it also reports sys-time and user-time, which are useful metrics.

Change-Id: I9b4551415cedf2f819eb184a02324b8bd919e2bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/248757
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-12 12:39:50 +00:00
Cuong Manh Le
806f478499 cmd/compile: don't report not enough args error if call is undefined
Fixes #38745

Change-Id: I2fbd8b512a8cf911b81a087162c74416116efea5
Reviewed-on: https://go-review.googlesource.com/c/go/+/253678
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-09-12 08:50:45 +00:00
Cuong Manh Le
2c95e3a6a8 cmd/compile: use clearer error message for stuct literal
This CL changes "T literal.M" error message to "T{...}.M". It's clearer
expression and focusing user on actual issue.

Updates #38745

Change-Id: I84b455a86742f37e0bde5bf390aa02984eecc3c9
Reviewed-on: https://go-review.googlesource.com/c/go/+/253677
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-09-12 08:31:49 +00:00
Constantin Konstantinidis
d7384f3612 os: implement File.Chmod on Windows
Fixes: #39606

Change-Id: I4def67ef18bd3ff866b140f6e76cdabe5d51a1c5
Reviewed-on: https://go-review.googlesource.com/c/go/+/250077
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
2020-09-11 22:10:02 +00:00
Jay Conrod
07c1788357 cmd/go: move repository resolution from internal/get to internal/vcs
This is a refactoring intended to break the dependency from
internal/modfetch to internal/get. No change in functionality is intended.

Change-Id: If51aba7139cc0b62ecc9ba454c055c99e8f36f0f
Reviewed-on: https://go-review.googlesource.com/c/go/+/254364
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-11 18:14:49 +00:00
Jay Conrod
86ee84c40e cmd/go: move get.Insecure to cfg.Insecure to break dependency cycle
Change-Id: If9c73ff5adc7e080a48ecc6b35ce40822193d66f
Reviewed-on: https://go-review.googlesource.com/c/go/+/254363
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-11 18:14:42 +00:00
Cherry Zhang
3ab22052fb [dev.link] all: merge branch 'master' into dev.link
Clean merge.

Change-Id: Ib773b0bc00fd99d494f9331c3613bcc8285e48e3
2020-09-11 12:07:44 -04:00
Jay Conrod
b459bc8152 cmd/go: make 'go mod download' update go.sum after downloads are complete
'go mod download' calls WriteGoMod once via modload.ListModules when
it loads the build list. This saves sums for go.mod files needed by
MVS, but the write occurs before any zip files are downloaded.

With this change, 'go mod download' calls WriteGoMod again (and thus,
modfetch.WriteGoSum) after downloading and verifying module zip files,
so the sums of the zip files will be saved, too.

Fixes #41341

Change-Id: I7d56754aa255256ed45fd93cb154c2e6ea5f45a9
Reviewed-on: https://go-review.googlesource.com/c/go/+/254357
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-11 15:53:52 +00:00
Cherry Zhang
1ed4f12f4a cmd/link: add a test to test RODATA is indeed read-only
Updates #38830.

Change-Id: Ie1f6ccef40a773f038aac587dfc26bf70a1a8536
Reviewed-on: https://go-review.googlesource.com/c/go/+/253921
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-11 15:41:20 +00:00
Cherry Zhang
ffd95aadcd cmd/link: put read-only data in __DATA_CONST segment
On darwin, we put read-only data in __TEXT segment on AMD64 in
exe (non-PIE) buildmode, and in __DATA on everywhere else. This
is not ideal: things in __DATA segment are not read-only, and
being mapped R/W may use more run-time resources.

In fact, newer darwin systems support a __DATA_CONST segment,
which the dynamic linker will map it read-only after applying
relocations. Use that.

Fixes #38830.

Change-Id: Ic281e6c6ca8ef5fec4bb7c5b71c50dd5393e78ae
Reviewed-on: https://go-review.googlesource.com/c/go/+/253919
Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-11 15:07:26 +00:00
Cherry Zhang
a531bd5a59 cmd/internal/objfile: recognize Mach-O __DATA_CONST segment as read-only data
Updates #38830.

Change-Id: I826c6b0a42bc8e48fcda556250ca4a95c73987eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/253918
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-11 15:07:03 +00:00
Jay Conrod
9214677e7d cmd/go: refactor modload.Import for better -mod=readonly errors
When -mod=readonly is set, Import will now allow imports from
replacements without explicit requirements. With -mod=mod, this would
add a new requirement but does not trigger a module lookup, so it's
determinisitic.

Before reporting an error for an unknown import with -mod=readonly,
check whether the import is valid. If there's a typo in the import,
that's more relevant.

For #40728

Change-Id: I05e138ff76ba3d0eb2e3010c15589fa363deb8d3
Reviewed-on: https://go-review.googlesource.com/c/go/+/253745
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-11 14:22:26 +00:00
Jay Conrod
6e3df749b1 cmd/go: refactor -mod flag parsing
Keep track of whether the -mod flag was set explicitly. When
-mod=readonly is the default, we'll want to adjust our error messages
if it's set explicitly.

Also, register the -mod, -modcacherw, and -modfile flags in functions
in internal/base instead of internal/work. 'go mod' commands that
don't load packages shouldn't depend on internal/work.

For #40728

Change-Id: I272aea9e19908ba37e151baac4ea8630e90f241f
Reviewed-on: https://go-review.googlesource.com/c/go/+/253744
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-11 14:22:17 +00:00
Cherry Zhang
b22af9b407 cmd/link: record only the first occurance in Reachparent graph
In the deadcode pass, a type symbol may be marked twice, one
without UsedInIface, one with. For the second time, don't
update the Reachparent graph, so it only records the path of
the first time the symbol is reached. This ensures the
Reachparent graph is acyclic.

TODO: add a test. (This only affects GOEXPERIMENT=fieldtrack)

Change-Id: I68e8a1a69c3830bc8aee5df946151dc22dcb2b29
Reviewed-on: https://go-review.googlesource.com/c/go/+/254297
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-11 13:53:03 +00:00
Jay Conrod
03a6860691 cmd/go: update tests to work with -mod=readonly on by default
For #40728

Change-Id: Ic2b025ff75c6e73c0cb58c1737e44e2a41c71571
Reviewed-on: https://go-review.googlesource.com/c/go/+/253837
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-11 13:08:54 +00:00
Cherry Zhang
d277a36123 runtime: remove darwin/arm specifc code
That port is gone.

Change-Id: I212d435e290d1890d6cd5531be98bb692650595e
Reviewed-on: https://go-review.googlesource.com/c/go/+/254077
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-11 02:58:36 +00:00
eric fang
a1762c2cc6 unicode/utf8: refactor benchmarks for FullRune function
BenchmarkFullASCIIRune tests the performance of function utf8.FullRune,
which will be inlined in BenchmarkFullASCIIRune. Since the return value
of FullRune is not referenced, it will be removed as dead code.

This CL makes the FullRune functions return value referenced by a global
variable to avoid this point. In addition, this CL adds one more benchmark
to cover more code paths, and puts them together as sub benchmarks of
BenchmarkFullRune.

Change-Id: I6e79f4c087adf70e351498a4b58d7482dcd1ec4a
Reviewed-on: https://go-review.googlesource.com/c/go/+/233979
Run-TryBot: eric fang <eric.fang@arm.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-10 20:25:45 +00:00
Junchen Li
d7ab277eed cmd/asm: add more SIMD instructions on arm64
This CL adds USHLL, USHLL2, UZP1, UZP2, and BIF instructions requested
by #40725. And since UXTL* are aliases of USHLL*, this CL also merges
them into one case.

Updates #40725

Change-Id: I404a4fdaf953319f72eea548175bec1097a2a816
Reviewed-on: https://go-review.googlesource.com/c/go/+/253659
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-10 15:48:36 +00:00
Cherry Zhang
9b2df72b63 cmd/link: add copyright header
Change-Id: I44f57019bb8e659d4aa3da8b13e8bd9a20b9d2e1
Reviewed-on: https://go-review.googlesource.com/c/go/+/253920
Reviewed-by: Than McIntosh <thanm@google.com>
2020-09-10 15:38:17 +00:00
ipriver
8098dbb30e runtime: update docs for GOMAXPROCS
https://github.com/golang/go/blob/master/doc/effective_go.html#L3211 is used to update the docs comment for `GOMAXPROCS` function.

Fixes #41275

Change-Id: I39f58e93a267c6e9f3ac6638ed51acbe5284ada2
GitHub-Last-Rev: e45c8ac587
GitHub-Pull-Request: golang/go#41276
Reviewed-on: https://go-review.googlesource.com/c/go/+/253537
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-10 14:44:25 +00:00
Cuong Manh Le
07d19b2597 all: check GOROOT_BOOTSTRAP executable before bootsrappping
Otherwise we’d still be writing that we're building cmd/dist
even if GOROOT_BOOTSTRAP we’re wrongly set.

Change-Id: I940e18c3cebae5664f85babc9919e9eb215d5093
Reviewed-on: https://go-review.googlesource.com/c/go/+/253877
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-10 12:11:16 +00:00
fanzha02
dfdc3880b0 cmd/internal/obj/arm64: enable some SIMD instructions
Enable VBSL, VBIT, VCMTST, VUXTL VUXTL2 and FMOVQ SIMD
instructions required by the issue #40725. And FMOVQ
instrucion is used to move a large constant to a Vn
register.

Add test cases.

Fixes #40725

Change-Id: I1cac1922a0a0165d698a4b73a41f7a5f0a0ad549
Reviewed-on: https://go-review.googlesource.com/c/go/+/249758
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-10 02:22:19 +00:00
Bryan C. Mills
aa476ba6f4 cmd/go/internal/modload: refactor pathInModuleCache
I found the control flow of this function a bit tricky to reason about
due to nesting and interaction between conditions and iteration. This
change factors out a helper function that can return early instead of
mixing conditionals and 'continue' statements.

Also remove the (unused) ModuleUsedDirectly function.

For #36460

Change-Id: I60a2a5a1b32989e5a17a14e1a8c858b280cda8f2
Reviewed-on: https://go-review.googlesource.com/c/go/+/251998
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09 22:39:28 +00:00
Bryan C. Mills
564b350c08 cmd/go/internal/modload: rename LoadBuildList and BuildList
With lazy loading, the “build list” can be refined as packages are loaded.
Rename functions that return the build list to more precisely describe
the set of modules returned by the call.

Also eliminate a redundant call to LoadBuildList (right before
ListModules, which itself begins with the same call).

For #36460

Change-Id: I0fc4f9dd7602e0df5e166e329ee5d516d810ca53
Reviewed-on: https://go-review.googlesource.com/c/go/+/249878
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09 22:39:02 +00:00
Bryan C. Mills
521393e7e0 cmd/go/internal/modget: move MVS code to a separate file
For #36460

Change-Id: Ie81c03df18c6987527da765d5f6575556340cb01
Reviewed-on: https://go-review.googlesource.com/c/go/+/249877
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-09 22:38:35 +00:00
Bryan C. Mills
363fb4bcc8 cmd/go/internal/modload: consolidate buildList and associated functions into one file
Change-Id: I310c37c7f0ce5581f07cf6e27d1f6361d03b92ef
Reviewed-on: https://go-review.googlesource.com/c/go/+/244077
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-09 22:38:16 +00:00
Bryan C. Mills
d27ebc7b86 cmd/go/internal/modload: implement the "all" pattern for lazy loading
The new semantics of the "all" package pattern can be implemented
without actually changing module loading per se. This change
implements those semantics, so that the change can be decoupled from
the changes to the module requirement graph.

For #36460

Change-Id: I0ee8b17afa8b728dc470a42a540fcc01764a4442
Reviewed-on: https://go-review.googlesource.com/c/go/+/240623
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09 22:37:22 +00:00
Tzu-Chiao Yeh
b4944ef310 cmd: update golang.org/x/tools to v0.0.0-20200901153117-6e59e24738da
Includes the latest fix on vet to warn unused context.WithValue result.

Fixes #41149

Change-Id: I06c204f40ef12b0f62f59b1bbdf1fe06ccd6565d
Reviewed-on: https://go-review.googlesource.com/c/go/+/252941
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-09 22:03:39 +00:00
Bryan C. Mills
cd91ab5d96 cmd/go/internal/modload: fix spurious import resolution error
Due to a bug in CL 173017, if QueryPackages found multiple candidates
for the given package and *at least* one of those candidates was not
available to add, we would reject *all* such candidates — even those
that were still viable.

Now, we return the first viable candidate, and only return an error if
*no* candidate is viable given the current build list.

Fixes #41113

Change-Id: Idb2e77244be7c0f5dd511efb142c3059925d7336
Reviewed-on: https://go-review.googlesource.com/c/go/+/251446
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09 21:29:10 +00:00
Steven Hartland
2c8d2a0c51 net/http: fix data race due to writeLoop goroutine left running
Fix a data race for clients that mutate requests after receiving a
response error which is caused by the writeLoop goroutine left
running, this can be seen on cancelled requests.

Fixes #37669

Change-Id: Ia4743c6b8abde3a7503de362cc6a3782e19e7f60
Reviewed-on: https://go-review.googlesource.com/c/go/+/251858
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-09 21:25:35 +00:00
Bryan C. Mills
015a5a5c5c cmd/go/internal/modload: rework import resolution
modload.Import previously performed two otherwise-separable tasks:

1. Identify which module in the build list contains the requested
   package.

2. If no such module exists, search available modules to try to find
   the missing package.

This change splits those two tasks into two separate unexported
functions, and reports import-resolution errors by attaching them to
the package rather than emitting them directly to stderr. That allows
'list' to report the errors, but 'list -e' to ignore them.

With the two tasks now separate, it will be easier to avoid the
overhead of resolving missing packages during lazy loading if we
discover that some existing dependency needs to be promoted to the top
level (potentially altering the main module's selected versions, and
thus suppling packages that were previously missing).

For #36460
Updates #26909

Change-Id: I32bd853b266d7cd231d1f45f92b0650d95c4bcbd
Reviewed-on: https://go-review.googlesource.com/c/go/+/251445
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09 20:53:04 +00:00
Bryan C. Mills
26d27f96fe cmd/go/internal/modload: remove (*loader).forceStdVendor
forceStdVendor was a special-case mechanism to allow Go contributors
to use vendored dependencies by default when working in GOROOT/src.

As of Go 1.14,¹ the 'go' command uses vendored dependencies by default
within all modules, so the 'std' and 'cmd' modules no longer need to
be special cases, and we can remove this special-case code.

¹ https://golang.org/doc/go1.14#vendor

Updates #33848
Updates #30241

Change-Id: Ib2fb5841c253113b17fa86a086ce85a22ac3d121
Reviewed-on: https://go-review.googlesource.com/c/go/+/251159
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09 20:47:23 +00:00
Bryan C. Mills
b96d32bd92 cmd/go/internal/modload: track which packages are in 'all' during loading
If the user requests the 'all' pattern in addition to explicit roots
outside of 'all', we should not load the transitive dependencies of
those explicit roots as if they were *in* 'all'. Without the '-test'
flag, we should not load test dependencies of any package outside of
'all'. Even *with* the '-test' flag, we should only load test
dependencies of the requested roots, not test dependencies of other
packages that happen to be imported by those roots.

More precise tracking of membership in 'all' will be important when we
implement lazy loading, because membership in 'all' determines which
module dependencies we will record in the main module's go.mod file.

This change also reduces reliance on global state, factors out the
loading process into several smaller functions, and sets us up to
reuse the 'go mod vendor' version of the 'all' pattern for lazy
loading.

For #36460
Fixes #40799

Change-Id: I5ca21c86a860daee1316f732cea131a331d8ddf9
Reviewed-on: https://go-review.googlesource.com/c/go/+/240505
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-09 20:46:48 +00:00
Ian Lance Taylor
2556eb76c8 runtime: ignore SIGPROF if profiling disable for thread
This avoids a deadlock on prof.signalLock between setcpuprofilerate
and cpuprof.add if a SIGPROF is delivered to the thread between the
call to setThreadCPUProfiler and acquiring prof.signalLock.

Fixes #41014

Change-Id: Ie825e8594f93a19fb1a6320ed640f4e631553596
Reviewed-on: https://go-review.googlesource.com/c/go/+/253758
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-09 18:04:10 +00:00
Michael Anthony Knyszek
34835df048 runtime: fix ReadMemStatsSlow's and CheckScavengedBits' chunk iteration
Both ReadMemStatsSlow and CheckScavengedBits iterate over the page
allocator's chunks but don't actually check if they exist. During the
development process the chunks index became sparse, so now this was a
possibility. If the runtime tests' heap is sparse we might end up
segfaulting in either one of these functions, though this will generally
be very rare.

The pattern here to return nil for a nonexistent chunk is also useful
elsewhere, so this change introduces tryChunkOf which won't throw, but
might return nil. It also updates the documentation of chunkOf.

Fixes #41296.

Change-Id: Id5ae0ca3234480de1724fdf2e3677eeedcf76fa0
Reviewed-on: https://go-review.googlesource.com/c/go/+/253777
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-09 17:48:56 +00:00
Cuong Manh Le
9ef3ee3396 cmd/link: remove unnecessary type conversion for nitablinks
It's already an uint64.

Change-Id: Ic4cdb957aa4f9245c1ea3f946bcb740f116dd04b
Reviewed-on: https://go-review.googlesource.com/c/go/+/253679
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-09 17:18:41 +00:00
Cherry Zhang
83ed734df0 cmd/link: pre-resolve package reference
Pre-resolve package index references, so it doesn't need to do a
map lookup in every cross-package symbol reference resolution. It
increases the memory usage very slightly (O(# imported packages)).

Change-Id: Ia76c97ac51f1c2c2d5ea7ae34853850ec69ef0a8
Reviewed-on: https://go-review.googlesource.com/c/go/+/253604
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-09 15:15:33 +00:00
Jay Conrod
bdad428570 cmd/go: make 'go get' preserve sums for content of new requirements
This preserves zip sums when 'go get' is run on a module that does not
have a package in the root directory. The zip must be fetched to
determine whether the package should be loaded, so we already load and
verify the sum.

Note that 'go mod tidy' may still remove these sums, since they
aren't needed to load packages.

Fixes #41103

Change-Id: I78f10a25f0392461fdc98518a7c92a38ee3233c3
Reviewed-on: https://go-review.googlesource.com/c/go/+/251880
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-09 13:30:35 +00:00