1
0
mirror of https://github.com/golang/go synced 2024-11-14 08:50:22 -07:00
Commit Graph

45434 Commits

Author SHA1 Message Date
Russ Cox
b1f76f7a22 os: add DirFS
It will inevitably be important to be able to pass an operating system
directory to code written to expect an fs.FS.

os.DirFS provides the conversion.

For #41190.

Change-Id: Id1a8fcbe4c7a30de2c47dea0504e9481a88b1b39
Reviewed-on: https://go-review.googlesource.com/c/go/+/243911
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 17:52:48 +00:00
Russ Cox
90c924ff88 testing/fstest: new package for testing file system code
This change adds basic test helpers for file system code.

The type MapFS is a simple map-based file system for use when
exercising general file system code.

The func TestFS tests a file system implementation.

For #41190.

Change-Id: I5a2036f57e733915ad508651ad7317749794423c
Reviewed-on: https://go-review.googlesource.com/c/go/+/243910
Trust: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 17:52:41 +00:00
Than McIntosh
d595712540 cmd/asm: rename "compiling runtime" flag
Rename the assembler "-compilingRuntime" flag to "-compiling-runtime",
to be more consistent with the flag style of other Go commands.

Change-Id: I8cc5cbf0b9b34d1dd4e9fa499d3fec8c1ef10b6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/263857
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-20 17:47:31 +00:00
Bryan C. Mills
f62d3202bf cmd/go/internal/renameio: include ios in the darwin test-flake mitigation
Because the "ios" build constraint implies "darwin", it is already
included in the general "darwin" flakiness workaround in
cmd/go/internal/robustio. We just need to relax the renameio test
to avoid false-positives there.

I do not expect this change to drive the rate of false-positives down
to zero, but it should at least reduce noise on the build dashboard.

For #42066

Change-Id: Ia33dbd33295fce5b3261b4831f2807ce29b82e65
Reviewed-on: https://go-review.googlesource.com/c/go/+/263777
Trust: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-20 17:36:42 +00:00
Michael Anthony Knyszek
05f5ae74bc runtime: fix scavenging tests for pallocChunkBytes huge pages and larger
Currently the scavenging tests implicitly assume that the system huge
page size is always strictly less than 4 MiB, or pallocChunkBytes. This
leads to failures on systems with huge pages of this size, and larger.

Filter out those tests on such platforms and add a test for the 4 MiB
case. The scavenger is already equipped to handle this case.

Huge page sizes > 4 MiB are effectively ignored, so also add a test case
to ensure that happens.

Unfortunately we can't actually run these tests in our CI because they
require the platform to provide the right huge page size, but we really
should just parameterize this value so we can test it (there's a TODO
about this already).

Fixes #42053.

Change-Id: Ia576cbf67e178a14a178a893967efbed27d6eb17
Reviewed-on: https://go-review.googlesource.com/c/go/+/263837
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
2020-10-20 15:19:09 +00:00
Daniel Martí
0bf507efe9 cmd/go: add BuildID to list -json -export
That is, the following two pieces of shell code are now equivalent:

	$ go tool buildid $(go list -export -f {{.Export}} strings)
	v_0VqA6yzwuMg2dn4u57/PXcIR2Pb8Mi9yRdcdkwe

	$ go list -export -f {{.BuildID}} strings
	v_0VqA6yzwuMg2dn4u57/PXcIR2Pb8Mi9yRdcdkwe

This does not expose any information that wasn't available before, but
makes this workflow simpler and faster. In the first example, we have to
execute two programs, and 'go tool buildid' has to re-open the export
data file to read the build ID.

With the new mechanism, 'go list -export' already has the build ID
ready, so we can simply print it out. Moreover, when listing lots of
related packages like './...', we can now obtain all their build IDs at
once.

Fixes #37281.

Change-Id: I8e2f65a08391b3df1a628c6e06e708b8c8cb7865
Reviewed-on: https://go-review.googlesource.com/c/go/+/263542
Trust: Daniel Martí <mvdan@mvdan.cc>
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-10-20 15:17:15 +00:00
Tobias Klauser
0709e58bee net/http/cgi: preserve env vars on illumos
Preserve the same environment variables as on solaris.

Spotted while reviewing CL 263577.

Change-Id: Id479dcf83d6231e9ef1fd2404b400d10082e1d0b
Reviewed-on: https://go-review.googlesource.com/c/go/+/263802
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20 14:04:18 +00:00
Martin Möhrmann
feb984c96b internal/cpu: make architectures without initialization work explicit
When cpu_no_init.go was created most architectures did not have
code in the doinit function. Currently only mips(le), riscv64 and
wasm do not have empty doinit functions.

Keeping cpu_no_init.go around does not reduce the work to satisfy
the build process when adding support for new architectures.

To support a new architecture a new file or build directive has to
be added to an existing file at any rate to define the constant
CacheLinePadSize. A new empty doinit can then be created in the
new file or the existing doinit can be reused when adding the
additional build directive.

Change-Id: I58a97f8cdf1cf1be85c37f4550c40750358aa031
Reviewed-on: https://go-review.googlesource.com/c/go/+/263801
Trust: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
2020-10-20 12:10:43 +00:00
Martin Möhrmann
de932da453 internal/cpu: consolidate arm64 feature detection
Move code to detect and mask arm64 CPU features from
runtime to internal/cpu.

Change-Id: Ib784e2ff056e8def125d68827b852f07a3eff0db
Reviewed-on: https://go-review.googlesource.com/c/go/+/261878
Trust: Martin Möhrmann <moehrmann@google.com>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
2020-10-20 11:09:52 +00:00
Nikhil Benesch
55b2d479d7 net/http/cgi: preserve LD_LIBRARY_PATH on NetBSD, too
This makes the behavior of the cgi package on NetBSD consistent with its
behavior on the other BSDs. It is also necessary for the test suite to
pass on NetBSD with gccgo (see CL 261137).

Change-Id: I531a6e0954f895a921204dcc61e3f3e90860a23d
Reviewed-on: https://go-review.googlesource.com/c/go/+/263577
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Benny Siegert <bsiegert@gmail.com>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Trust: Benny Siegert <bsiegert@gmail.com>
2020-10-20 08:08:15 +00:00
Tobias Klauser
310984bf54 syscall, cmd/go/internal/modload: add and use Access on aix
Implement Access using Faccessat on aix following golang.org/x/sys/unix
CL 262897 and switch cmd/go/internal/modload to use it to implement
hasWritePerm.

Change-Id: I682e44737ac2bac5a203ac1c9ddd277810454426
Reviewed-on: https://go-review.googlesource.com/c/go/+/263540
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-10-20 07:33:13 +00:00
Russ Cox
fcb9d6b5d0 io/fs: add FS, File, ReadDirFile; move DirEntry from os
These are the core interfaces for the io/fs design.
See #41190 and https://golang.org/s/draft-iofs-design for details.

DirEntry was left behind in the previous move from os
but is needed for ReadDirFile, so it moves in this commit.

Also apply a couple comment changes suggested in
the review of CL 261540.

For #41190.

Change-Id: I087741545139ed30b9ba5db728a0bad71129500b
Reviewed-on: https://go-review.googlesource.com/c/go/+/243908
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-20 02:32:46 +00:00
Russ Cox
9ad090c5fe testing/iotest: add TestReader to test readers
There are many reader behaviors that are subtle and
worth testing, and it's nice to have one complete tester
instead of many incomplete ones.

For #41190, which will use this as part of a larger
file system implementation tester.

Change-Id: Ib4cc7fae94b0d9b45dfacadc52baa77ad3761322
Reviewed-on: https://go-review.googlesource.com/c/go/+/243909
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20 02:32:44 +00:00
Russ Cox
7bb721b938 all: update references to symbols moved from os to io/fs
The old os references are still valid, but update our code
to reflect best practices and get used to the new locations.

Code compiled with the bootstrap toolchain
(cmd/asm, cmd/dist, cmd/compile, debug/elf)
must remain Go 1.4-compatible and is excluded.

For #41190.

Change-Id: I8f9526977867c10a221e2f392f78d7dec073f1bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/243907
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 02:32:42 +00:00
Russ Cox
d4da735091 io/fs: move FileInfo, FileMode, PathError, ErrInvalid, ... from os to io/fs
First step of creating the new io/fs package.

For #41190.

Change-Id: I1339b1abdd533b0f1deab283628088b2f706fb5b
Reviewed-on: https://go-review.googlesource.com/c/go/+/243906
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20 02:32:41 +00:00
Cherry Zhang
627959eb04 cmd/link: support cgo internal/linking on darwin/arm64
Cgo programs work as well. Still not enabled by default for now.

Enable internal linking tests.

Updates #38485.

Change-Id: I8324a5c263fba221eb4e67d71207ca84fa241e6c
Reviewed-on: https://go-review.googlesource.com/c/go/+/263637
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-20 02:25:52 +00:00
Cuong Manh Le
bccdd31252 cmd/compile: use type position for error message in align.go
This helps the compiler reports the right place where the type declared,
instead of relying on global lineno, which maybe set to wrong value at
the time the error is reported.

Fixes #42058

Change-Id: I06d34aa9b0236d122f4a0d72e66675ded022baac
Reviewed-on: https://go-review.googlesource.com/c/go/+/263597
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-20 02:05:01 +00:00
Russ Cox
a505312c89 api: update next.txt
A bunch of new API hasn't been added yet and is cluttering all.bash output.

Change-Id: Ic7f255cce8c162716b9a1144f78b5f9ce6220b33
Reviewed-on: https://go-review.googlesource.com/c/go/+/262878
Trust: Russ Cox <rsc@golang.org>
Trust: Rob Pike <r@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-10-20 00:59:23 +00:00
Russ Cox
2291cae2af os: use keyed literals for PathError
Necessary to move PathError to io/fs.

For #41190.

Change-Id: I05e87675f38a22f0570d4366b751b6169f7a1b13
Reviewed-on: https://go-review.googlesource.com/c/go/+/243900
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-20 00:59:22 +00:00
Russ Cox
a4ede9f9a6 os: add File.ReadDir method and DirEntry type
ReadDir provides a portable, efficient way to read a directory
and discover the type of directory entries.
This enables a more efficient file system walk, yet to be added.

See #41467 for the proposal review for the API.

Fixes #41467.

Change-Id: I461a526793ae46df48821aa448b04f1705546739
Reviewed-on: https://go-review.googlesource.com/c/go/+/261540
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
2020-10-20 00:59:20 +00:00
Dan Scales
8fe372c7b3 cmd/compile: allowing inlining of functions with OCALLPART
OCALLPART is exported in its original form, which is as an OXDOT.

The body of the method value wrapper created in makepartialcall() was
not being typechecked, and that was causing a problem during escape
analysis, so I added code to typecheck the body.

The go executable got slightly bigger with this change (13598111 ->
13598905), because of extra exported methods with OCALLPART (I
believe), while the text size got slightly smaller (9686964 ->
9686643).

This is mainly part of the work to make sure all function bodies can
be exported (for purposes of generics), but might as well fix the
OCALLPART inlining bug as well.

Fixes #18493

Change-Id: If7aa055ff78ed7a6330c6a1e22f836ec567d04fd
Reviewed-on: https://go-review.googlesource.com/c/go/+/263620
Run-TryBot: Dan Scales <danscales@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-10-20 00:07:42 +00:00
Michael Anthony Knyszek
2754d91164 runtime: add lock rank partial-order edge between fin and mheap
finlock may be held across a write barrier, which could then acquire the
mheap lock. Notably, this occurs in the mp.unlockf write in gopark where
finlock is held by the finalizer goroutines and is going to sleep.

Fixes #42062.

Change-Id: Icf76637ae6fc12795436272633dca3d473780875
Reviewed-on: https://go-review.googlesource.com/c/go/+/263678
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dan Scales <danscales@google.com>
2020-10-19 23:35:51 +00:00
hk
70cad2744b cmd/compile/internal/gc: use plural nouns in comments
Change-Id: I99a5b5b88495cfaef16d41ee4b962c5d0b3488aa
GitHub-Last-Rev: fa0d895b3a
GitHub-Pull-Request: golang/go#42059
Reviewed-on: https://go-review.googlesource.com/c/go/+/263617
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Keith Randall <khr@golang.org>
Trust: Robert Griesemer <gri@golang.org>
2020-10-19 21:40:56 +00:00
Matthew Dempsky
c216ae80c9 cmd/compile: fix ICE in reporting of invalid recursive types
asNode(t.Nod).Name.Param will be nil for builtin types (i.e., the
universal predeclared types and unsafe.Pointer). These types can't be
part of a cycle anyway, so we can just skip them.

Fixes #42075.

Change-Id: Ic7a44de65c6bfd16936545dee25e36de8850acf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/263717
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Daniel Martí <mvdan@mvdan.cc>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2020-10-19 21:30:43 +00:00
Bryan C. Mills
5647d01ab7 Revert "net/http: test that ParseMultipartForm returns an error for int overflow"
This reverts CL 254977.

Reason for revert: introduced test failures on longtest builders.

Change-Id: I75e868245f980189ad85dd4103d9178989e06ecf
Reviewed-on: https://go-review.googlesource.com/c/go/+/263658
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
2020-10-19 19:51:19 +00:00
Than McIntosh
4d1cecdee8 cmd/dist,cmd/go: broaden use of asm macro GOEXPERIMENT_REGABI
This extends a change made in https://golang.org/cl/252258 to the go
command (to define an asm macro when GOEXPERIMENT=regabi is in
effect); we need this same macro during the bootstrap build in order
to build the runtime correctly.

In addition, expand the set of packages where the macro is applied to
{runtime, reflect, syscall, runtime/internal/*}, and move the logic
for deciding when something is a "runtime package" out of the
assembler and into cmd/{go,dist}, introducing a new assembler command
line flag instead.

Updates #27539, #40724.

Change-Id: Ifcc7f029f56873584de1e543c55b0d3e54ad6c49
Reviewed-on: https://go-review.googlesource.com/c/go/+/262317
Trust: Than McIntosh <thanm@google.com>
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-19 19:27:54 +00:00
Klaus Post
ab541a0560 compress/flate: fix corrupted output
The fastest compression mode can pick up a false match for every 2GB
of input data resulting in incorrectly decompressed data.

Since matches are allowed to be up to and including at maxMatchOffset
we must offset the buffer by an additional element to prevent the first
4 bytes to match after an out-of-reach value after shiftOffsets has
been called.

We offset by `maxMatchOffset + 1` so offset 0 in the table will now
fail the `if offset > maxMatchOffset` in all cases.

Fixes #41420

Change-Id: If1fbe01728e132b8a207e3f3f439edd832dcc710
GitHub-Last-Rev: 50fabab0da
GitHub-Pull-Request: golang/go#41477
Reviewed-on: https://go-review.googlesource.com/c/go/+/255879
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Joe Tsai <thebrokentoaster@gmail.com>
Trust: Matthew Dempsky <mdempsky@google.com>
2020-10-19 18:48:17 +00:00
Cherry Zhang
06839e3c9a cmd/link: run generator functions along with symbol writing
Like we do for applying relocations, for generator symbols, run
the generator function along with symbol writing. This will
probably have better locality and parallelism.

Linking cmd/compile,

Asmb          29.9ms ± 5%    19.1ms ±12%  -36.18%  (p=0.000 n=10+9)
TotalTime      351ms ± 3%     339ms ± 2%   -3.51%  (p=0.000 n=11+10)

Change-Id: I9cda6718bf70b3bcf1b7a501a845d6136234d2ee
Reviewed-on: https://go-review.googlesource.com/c/go/+/263640
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-19 18:31:53 +00:00
Cherry Zhang
2ff9e01d72 cmd/link: fix TestDWARFiOS
Use GOOS=ios.

Run "xcodebuild -showsdks" correctly.

Wrap testDWARF in subtests, as it calls t.Parallel, otherwise
the two calls of testDWARF will cause t.Parallel being called
twice, which panics.

Updates #38485.

Change-Id: I614c8daa99c83cbfd05a4cfa041968d49279f16a
Reviewed-on: https://go-review.googlesource.com/c/go/+/263639
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Jeremy Faller <jeremy@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-19 18:31:39 +00:00
Cherry Zhang
0040adfd0f runtime: define ios/arm64 entry points
Updates #38485.

Change-Id: I030346c7f0c3ce89209588525b210284fdea4efd
Reviewed-on: https://go-review.googlesource.com/c/go/+/263638
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-19 18:31:05 +00:00
Cherry Zhang
bb44c90a94 Revert "cmd/link: put C static symbols in the symbol table"
This reverts CL 263259.

Reason for revert: break AIX builder.

Change-Id: I3abb0b1f509961a06f1567d8c87679b0ca5782d7
Reviewed-on: https://go-review.googlesource.com/c/go/+/263660
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-10-19 18:25:30 +00:00
hk
178b0f59ff cmd/compile/internal/gc: remove an unused method
Change-Id: Ib496d1e9a47d2cf69509ffd0009038b7d34a0149
GitHub-Last-Rev: 5b5589dd03
GitHub-Pull-Request: golang/go#42060
Reviewed-on: https://go-review.googlesource.com/c/go/+/263618
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Robert Griesemer <gri@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-19 17:23:00 +00:00
avivklas
58eadc232e net/http: test that ParseMultipartForm returns an error for int overflow
ParseMultipartForm has been changed to return an error if maxMemory
parameter + 10MB causes int overflows. This adds a test for the new
behaviour.

For #40430

Change-Id: I4f66ce8a9382940182011d22a84ee52b1d1364cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/254977
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
2020-10-19 17:19:18 +00:00
Aofei Sheng
9c017ff30d net/http/httputil: copy response header back to http.ResponseWriter in ReverseProxy.handleUpgradeResponse
Fixes: #41634

Change-Id: Ib78cc37a4d2ca0753d567eafb616238e4103484e
Reviewed-on: https://go-review.googlesource.com/c/go/+/257777
Reviewed-by: Damien Neil <dneil@google.com>
Trust: Damien Neil <dneil@google.com>
Trust: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Damien Neil <dneil@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-19 17:17:28 +00:00
Katie Hockman
2a206c7fcc crypto/hmac: panic if reusing hash.Hash values
Also put Reset in the correct place for the other
benchmarks.

name           old time/op    new time/op    delta
NewWriteSum-8    1.01µs ± 0%    1.01µs ± 1%   ~     (p=0.945 n=9+9)

name           old speed      new speed      delta
NewWriteSum-8  31.7MB/s ± 0%  31.6MB/s ± 1%   ~     (p=0.948 n=9+9)

name           old alloc/op   new alloc/op   delta
NewWriteSum-8      544B ± 0%      544B ± 0%   ~     (all equal)

name           old allocs/op  new allocs/op  delta
NewWriteSum-8      7.00 ± 0%      7.00 ± 0%   ~     (all equal)

Fixes #41089

Change-Id: I3dae660adbe4993963130bf3c2636bd53899164b
Reviewed-on: https://go-review.googlesource.com/c/go/+/261960
Trust: Katie Hockman <katie@golang.org>
Trust: Roland Shoemaker <roland@golang.org>
Run-TryBot: Katie Hockman <katie@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Filippo Valsorda <filippo@golang.org>
Reviewed-by: Roland Shoemaker <roland@golang.org>
2020-10-19 15:00:02 +00:00
Tobias Klauser
19f6422e00 internal/bytealg: add assembly implementation of Count/CountString for riscv64
Simple single-byte loop count for now, to be further improved in future
CLs.

Benchmark on linux/riscv64 (HiFive Unleashed):

name               old time/op    new time/op     delta
CountSingle/10-4      190ns ± 1%      145ns ± 1%  -23.66%  (p=0.000 n=10+9)
CountSingle/32-4      422ns ± 1%      268ns ± 0%  -36.43%  (p=0.000 n=10+7)
CountSingle/4K-4     43.3µs ± 0%     23.8µs ± 0%  -45.09%  (p=0.000 n=8+10)
CountSingle/4M-4     54.2ms ± 1%     33.3ms ± 1%  -38.48%  (p=0.000 n=10+10)
CountSingle/64M-4     1.52s ± 1%      1.20s ± 1%  -21.20%  (p=0.000 n=9+9)

name               old speed      new speed       delta
CountSingle/10-4   52.7MB/s ± 1%   69.1MB/s ± 1%  +31.03%  (p=0.000 n=10+9)
CountSingle/32-4   75.9MB/s ± 1%  119.5MB/s ± 0%  +57.34%  (p=0.000 n=10+8)
CountSingle/4K-4   94.6MB/s ± 0%  172.2MB/s ± 0%  +82.10%  (p=0.000 n=8+10)
CountSingle/4M-4   77.4MB/s ± 1%  125.8MB/s ± 1%  +62.54%  (p=0.000 n=10+10)
CountSingle/64M-4  44.2MB/s ± 1%   56.1MB/s ± 1%  +26.91%  (p=0.000 n=9+9)

Change-Id: I2a6bd50d22d5f598517bb3c5a50066c54280cac5
Reviewed-on: https://go-review.googlesource.com/c/go/+/263541
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Joel Sing <joel@sing.id.au>
2020-10-19 14:55:41 +00:00
Cherry Zhang
6398b4725c cmd/link: put C static symbols in the symbol table
We don't put Go static symbols in the symbol table, as they are
always compiler-generated (there is no way to define a static
symbol in user code in Go). We retain static symbols in assembly
code, as it may be user-defined. Also retain static symbols in C.

Change-Id: Ie0425bb4df33f183857b1fd5ba4b2bdfdc497571
Reviewed-on: https://go-review.googlesource.com/c/go/+/263259
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Than McIntosh <thanm@google.com>
2020-10-19 13:50:22 +00:00
Alberto Donizetti
3853272573 Revert "test: add regression test from #41474"
This reverts CL 263097.

Reason for revert: broke the noopt builder.

Change-Id: Ie36d2c3ed9449b4425732072db624c8e18f965f3
Reviewed-on: https://go-review.googlesource.com/c/go/+/263537
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
2020-10-19 13:49:50 +00:00
Than McIntosh
53094ac844 cmd: go get golang.org/x/tools@d1624618 && go mod vendor
This brings in CL 262197, which enables handling of ABI selector
syntax in the asmdecl checker.

Updates #27539, #40724.

Change-Id: I088f5eeacb3588ed49c4aa2277641049bf5b9083
Reviewed-on: https://go-review.googlesource.com/c/go/+/262318
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Than McIntosh <thanm@google.com>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-19 12:40:52 +00:00
Than McIntosh
dd58239dd2 cmd/asm: allow def/ref of func ABI when compiling runtime
Function symbols defined and referenced by assembly source currently
always default to ABI0; this patch adds preliminary support for
accepting an explicit ABI selector clause for func defs/refs. This
functionality is currently only enabled when compiling runtime-related
packages (runtime, syscall, reflect). Examples:

  TEXT ·DefinedAbi0Symbol<ABI0>(SB),NOSPLIT,$0
        RET

  TEXT ·DefinedAbi1Symbol<ABIInternal>(SB),NOSPLIT,$0
        CALL    ·AbiZerolSym<ABI0>(SB)
	...
        JMP     ·AbiInternalSym<ABIInternal>(SB)
        RET

Also included is a small change to the code in the compiler that reads
the symabis file emitted by the assembler.

New behavior is currently gated under GOEXPERIMENT=regabi.

Updates #27539, #40724.

Change-Id: Ia22221fe26df0fa002191cfb13bdfaaa38d7df38
Reviewed-on: https://go-review.googlesource.com/c/go/+/260477
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Than McIntosh <thanm@google.com>
2020-10-19 11:25:35 +00:00
Joel Sing
9499a2e108 cmd/dist: detect gohostarch for openbsd/mips64
On OpenBSD/octeon `uname -m` returns 'octeon' and we need `uname -p` to determine
the processor type (mips64).

Update #40995

Change-Id: I37c53ca2c372ab958fff17917d82d41f4b0a0393
Reviewed-on: https://go-review.googlesource.com/c/go/+/250583
Trust: Joel Sing <joel@sing.id.au>
Run-TryBot: Joel Sing <joel@sing.id.au>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-18 08:32:19 +00:00
Joel Sing
15eaa870e1 cmd/link: add support for external linking on linux/riscv64
Fixes #36739

Change-Id: Id7573b343786360c72524f9f27d2a8f08d379cf3
Reviewed-on: https://go-review.googlesource.com/c/go/+/243517
Trust: Joel Sing <joel@sing.id.au>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-10-18 08:30:18 +00:00
Alex Brainman
515e6a9b12 runtime: use CreateWaitableTimerEx to implement usleep
@jstarks suggested that recent versions of Windows provide access to high resolution timers. See

https://github.com/golang/go/issues/8687#issuecomment-656259353

for details.

I tried to run this C program on my Windows 10 computer

```
 #include <stdio.h>
 #include <Windows.h>

 #pragma comment(lib, "Winmm.lib")

// Apparently this is already defined when I use msvc cl.
//#define CREATE_WAITABLE_TIMER_HIGH_RESOLUTION = 0x00000002;

int usleep(HANDLE timer, LONGLONG d) {
	LARGE_INTEGER liDueTime;
	DWORD ret;
	LARGE_INTEGER StartingTime, EndingTime, ElapsedMicroseconds;
	LARGE_INTEGER Frequency;

	QueryPerformanceFrequency(&Frequency);
	QueryPerformanceCounter(&StartingTime);

	liDueTime.QuadPart = d;
	liDueTime.QuadPart = liDueTime.QuadPart * 10;	// us into 100 of ns units
	liDueTime.QuadPart = -liDueTime.QuadPart;	// negative for relative dure time

	if (!SetWaitableTimer(timer, &liDueTime, 0, NULL, NULL, 0)) {
		printf("SetWaitableTimer failed: errno=%d\n", GetLastError());
		return 1;
	}

	ret = WaitForSingleObject(timer, INFINITE);
	if (ret != WAIT_OBJECT_0) {
		printf("WaitForSingleObject failed: ret=%d errno=%d\n", ret, GetLastError());
		return 1;
	}

	QueryPerformanceCounter(&EndingTime);
	ElapsedMicroseconds.QuadPart = EndingTime.QuadPart - StartingTime.QuadPart;
	ElapsedMicroseconds.QuadPart *= 1000000;
	ElapsedMicroseconds.QuadPart /= Frequency.QuadPart;

	printf("delay is %lld us - slept for %lld us\n", d, ElapsedMicroseconds.QuadPart);

	return 0;
}

int testTimer(DWORD createFlag)
{
	HANDLE timer;

	timer = CreateWaitableTimerEx(NULL, NULL, createFlag, TIMER_ALL_ACCESS);
	if (timer == NULL) {
		printf("CreateWaitableTimerEx failed: errno=%d\n", GetLastError());
		return 1;
	}

	usleep(timer, 1000LL);
	usleep(timer, 100LL);
	usleep(timer, 10LL);
	usleep(timer, 1LL);

	CloseHandle(timer);

	return 0;
}

int main()
{
	printf("\n1. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is off - timeBeginPeriod is off\n");
	testTimer(0);

	printf("\n2. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is on - timeBeginPeriod is off\n");
	testTimer(CREATE_WAITABLE_TIMER_HIGH_RESOLUTION);

	timeBeginPeriod(1);

	printf("\n3. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is off - timeBeginPeriod is on\n");
	testTimer(0);

	printf("\n4. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is on - timeBeginPeriod is on\n");
	testTimer(CREATE_WAITABLE_TIMER_HIGH_RESOLUTION);
}
```

and I see this output

```
1. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is off - timeBeginPeriod is off
delay is 1000 us - slept for 4045 us
delay is 100 us - slept for 3915 us
delay is 10 us - slept for 3291 us
delay is 1 us - slept for 2234 us

2. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is on - timeBeginPeriod is off
delay is 1000 us - slept for 1076 us
delay is 100 us - slept for 569 us
delay is 10 us - slept for 585 us
delay is 1 us - slept for 17 us

3. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is off - timeBeginPeriod is on
delay is 1000 us - slept for 742 us
delay is 100 us - slept for 893 us
delay is 10 us - slept for 414 us
delay is 1 us - slept for 920 us

4. CREATE_WAITABLE_TIMER_HIGH_RESOLUTION is on - timeBeginPeriod is on
delay is 1000 us - slept for 1466 us
delay is 100 us - slept for 559 us
delay is 10 us - slept for 535 us
delay is 1 us - slept for 5 us
```

That shows, that indeed using CREATE_WAITABLE_TIMER_HIGH_RESOLUTION
will provide sleeps as low as about 500 microseconds, while our
current approach provides about 1 millisecond sleep.

New approach also does not require for timeBeginPeriod to be on,
so this change solves long standing problem with go programs draining
laptop battery, because it calls timeBeginPeriod.

This change will only run on systems where
CREATE_WAITABLE_TIMER_HIGH_RESOLUTION flag is available. If not
available, the runtime will fallback to original code that uses
timeBeginPeriod.

This is how this change affects benchmark reported in issue #14790

name               old time/op  new time/op  delta
ChanToSyscallPing  1.05ms ± 2%  0.68ms ±11%  -35.43%  (p=0.000 n=10+10)

The benchmark was run with GOMAXPROCS set to 1.

Fixes #8687
Updates #14790

Change-Id: I5b97ba58289c088c17c05292e12e45285c467eae
Reviewed-on: https://go-review.googlesource.com/c/go/+/248699
Run-TryBot: Alex Brainman <alex.brainman@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Alex Brainman <alex.brainman@gmail.com>
Reviewed-by: Austin Clements <austin@google.com>
2020-10-18 08:22:01 +00:00
zikaeroh
fc981654c7 sort: fix grammar in updated Less comment
The rewritten comment didn't sound right to my ears. Tweak it to be
grammatically correct.

Change-Id: Iae7d9f8810fff78cfd964bb3117099bce4479c14
Reviewed-on: https://go-review.googlesource.com/c/go/+/263180
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Robert Griesemer <gri@golang.org>
2020-10-17 22:58:33 +00:00
Matthew Dempsky
76a615b20a cmd/compile: fix defer/go calls to variadic unsafe-uintptr functions
Before generating wrapper function, turn any f(a, b, []T{c, d, e}...)
calls back into f(a, b, c, d, e). This allows the existing code for
recognizing and specially handling unsafe.Pointer->uintptr conversions
to correctly handle variadic arguments too.

Fixes #41460.

Change-Id: I0a1255abdd1bd5dafd3e89547aedd4aec878394c
Reviewed-on: https://go-review.googlesource.com/c/go/+/263297
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2020-10-17 21:30:53 +00:00
Bryan C. Mills
30119bcca9 cmd/go/internal/modload: fix sort condition in (*replacementRepo).Versions
In CL 258220 I added replacement versions to the repo versions used in
the modload.Query functions. The versions are computed from a map in
the modfile index, which has a nondeterministic iteration order.

I added a short-circuit condition to skip sorting in the (vastly
common) case where no replacement versions are added. However, while
cleaning up the change I accidentally deleted the line of code that
sets that condition. As a result, the test of that functionality
(mod_get_replaced) has been failing nondeterministically.

This change fixes the condition by comparing the slices before and
after adding versions, rather than by setting a separate variable.
The test now passes reliably (tested with -count=200).

Updates #41577
Updates #41416
Updates #37438
Updates #26241

Change-Id: I49a66a3a5510da00ef42b47f20a168de66100db6
Reviewed-on: https://go-review.googlesource.com/c/go/+/263266
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2020-10-17 19:22:49 +00:00
Alberto Donizetti
c8f6135d4f test: add regression test from #41474
This issue was fixed with multiple individual compiler optimizations,
each of which had their own respective test cases. This CL just adds
the capstone test case to demonstrate that the issue has been fixed
and doesn't regress again.

Updates #41474.

Change-Id: Iae752d4b0e7b83ee356b946843340a4fbc254058
Reviewed-on: https://go-review.googlesource.com/c/go/+/263097
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-17 07:18:20 +00:00
Cherry Zhang
5faa828651 cmd/link: use GOOS=ios for TestBuildForTvOS
Updates #38485.

Fix darwin-amd64-10_15 build.

Change-Id: I1833c23788acafc9530bb91fb6182fc5cb44f6cd
Reviewed-on: https://go-review.googlesource.com/c/go/+/263265
Trust: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-10-17 02:48:03 +00:00
Ian Lance Taylor
11cfb48df1 syscall: use MustHaveExec in TestExec
For #41702

Change-Id: Ib2b15e52aa1fef2f5e644b316c726150252fa9f8
Reviewed-on: https://go-review.googlesource.com/c/go/+/262738
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-10-17 00:32:32 +00:00
Nigel Tao
a2eb53c571 strconv: use the Eisel-Lemire ParseFloat algorithm
Also fix BenchmarkAtof64Random* to initialize the test data when none
of the TestAtof* tests are run.

Passing "go test -test.count=5 -test.run=xxx -test.bench=Atof64" on to
benchstat:

name                  old time/op  new time/op  delta
Atof64Decimal-4       47.9ns ± 0%  48.3ns ± 1%     ~     (p=0.238 n=4+5)
Atof64Float-4         58.3ns ± 3%  57.7ns ± 0%     ~     (p=0.151 n=5+5)
Atof64FloatExp-4       107ns ± 0%    71ns ± 1%  -33.89%  (p=0.016 n=4+5)
Atof64Big-4            163ns ± 0%   166ns ± 2%     ~     (p=0.159 n=4+5)
Atof64RandomBits-4     299ns ± 1%   166ns ± 1%  -44.41%  (p=0.008 n=5+5)
Atof64RandomFloats-4   188ns ± 1%   144ns ± 0%  -23.03%  (p=0.008 n=5+5)

The canada.json file from github.com/miloyip/nativejson-benchmark is
full of geospatial coordinates (i.e. numbers). With this program:

    src, _ := ioutil.ReadFile("canada.json")
    for i := 0; i < 5; i++ {
        now := time.Now()
        for j := 0; j < 10; j++ {
            dst := interface{}(nil)
            if err := json.Unmarshal(src, &dst); err != nil {
                log.Fatal(err)
            }
        }
        fmt.Println(time.Since(now))
    }

Median of the 5 printed numbers, lower is better.
Before: 760.819549ms
After:  702.651646ms
Ratio:  1.08x

The new detailedPowersOfTen table weighs in at 596 * 16 = 9536 bytes,
but some of that weight gain can be clawed back, in a follow-up commit,
that folds in the existing powersOfTen table in extfloat.go.

RELNOTE=yes

Change-Id: I3953110deaa1f5f6941e88e8417c4665b649ed80
Reviewed-on: https://go-review.googlesource.com/c/go/+/260858
Run-TryBot: Nigel Tao <nigeltao@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Trust: Nigel Tao <nigeltao@golang.org>
2020-10-17 00:26:52 +00:00