1
0
mirror of https://github.com/golang/go synced 2024-11-08 19:46:23 -07:00
Commit Graph

45052 Commits

Author SHA1 Message Date
Lynn Boger
967465da29 cmd/compile: use combined shifts to improve array addressing on ppc64x
This change adds rules to find pairs of instructions that can
be combined into a single shifts. These instruction sequences
are common in array addressing within loops. Improvements can
be seen in many crypto packages and the hash packages.

These are based on the extended mnemonics found in the ISA
sections C.8.1 and C.8.2.

Some rules in PPC64.rules were moved because the ordering prevented
some matching.

The following results were generated on power9.

hash/crc32:
    CRC32/poly=Koopman/size=40/align=0          195ns ± 0%     163ns ± 0%  -16.41%
    CRC32/poly=Koopman/size=40/align=1          200ns ± 0%     163ns ± 0%  -18.50%
    CRC32/poly=Koopman/size=512/align=0        1.98µs ± 0%    1.67µs ± 0%  -15.46%
    CRC32/poly=Koopman/size=512/align=1        1.98µs ± 0%    1.69µs ± 0%  -14.80%
    CRC32/poly=Koopman/size=1kB/align=0        3.90µs ± 0%    3.31µs ± 0%  -15.27%
    CRC32/poly=Koopman/size=1kB/align=1        3.85µs ± 0%    3.31µs ± 0%  -14.15%
    CRC32/poly=Koopman/size=4kB/align=0        15.3µs ± 0%    13.1µs ± 0%  -14.22%
    CRC32/poly=Koopman/size=4kB/align=1        15.4µs ± 0%    13.1µs ± 0%  -14.79%
    CRC32/poly=Koopman/size=32kB/align=0        137µs ± 0%     105µs ± 0%  -23.56%
    CRC32/poly=Koopman/size=32kB/align=1        137µs ± 0%     105µs ± 0%  -23.53%

crypto/rc4:
    RC4_128    733ns ± 0%    650ns ± 0%  -11.32%  (p=1.000 n=1+1)
    RC4_1K    5.80µs ± 0%   5.17µs ± 0%  -10.89%  (p=1.000 n=1+1)
    RC4_8K    45.7µs ± 0%   40.8µs ± 0%  -10.73%  (p=1.000 n=1+1)

crypto/sha1:
    Hash8Bytes       635ns ± 0%     613ns ± 0%   -3.46%  (p=1.000 n=1+1)
    Hash320Bytes    2.30µs ± 0%    2.18µs ± 0%   -5.38%  (p=1.000 n=1+1)
    Hash1K          5.88µs ± 0%    5.38µs ± 0%   -8.62%  (p=1.000 n=1+1)
    Hash8K          42.0µs ± 0%    37.9µs ± 0%   -9.75%  (p=1.000 n=1+1)

There are other improvements found in golang.org/x/crypto which are all in the
range of 5-15%.

Change-Id: I193471fbcf674151ffe2edab212799d9b08dfb8c
Reviewed-on: https://go-review.googlesource.com/c/go/+/252097
Trust: Lynn Boger <laboger@linux.vnet.ibm.com>
Run-TryBot: Lynn Boger <laboger@linux.vnet.ibm.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Carlos Eduardo Seo <cseo@linux.vnet.ibm.com>
2020-09-17 12:37:40 +00:00
Joel Sing
0dde60a5fe cmd/internal/obj/riscv: clean up lowerJALR
This cleans up the last of the direct obj.Prog rewriting, removing lowerJALR
and replacing it with correct handling for AJALR during instruction encoding.

Change-Id: Ieea125bde30d4c0edd2d9ed1e50160543aa8f330
Reviewed-on: https://go-review.googlesource.com/c/go/+/249077
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Joel Sing <joel@sing.id.au>
2020-09-17 08:21:04 +00:00
Tobias Klauser
7f24142b7b syscall, cmd/go/internal/lockedfile/internal/filelock: add and use Flock on illumos
Copy the syscall wrapper from golang.org/x/sys/unix CL 255377 to provide
Flock on illumos and switch cmd/go/internal/lockedfile/internal/filelock
to use it.

Fixes #35618

Change-Id: I876a2b782329a988fa85361fb1ea58eb6f329af1
Reviewed-on: https://go-review.googlesource.com/c/go/+/255258
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-17 06:20:06 +00:00
Matthew Dempsky
f5d59d0e38 cmd/compile: skip looking for OCLOSURE nodes in xtop
xtop holds package's top-level declaration statements, but OCLOSURE
only appears in expression contexts. xtop will instead hold the
synthetic ODCLFUNC representing OCLOSURE's function body.

This CL makes the loop consistent with the later phases that only look
for ODCLFUNC nodes in xtop.

Passes toolstash-check.

Change-Id: I852a10ef1bf75bb3351e3da0357ca8b2e26aec6e
Reviewed-on: https://go-review.googlesource.com/c/go/+/255340
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-16 23:00:02 +00:00
David Chase
396688af7e cmd/compile: make translation to calls for SSA look more "value-oriented"
The existing translation assumes an in-memory return values, thus it returns
the address of the result(s).  Most consumers immediately load from the
address to get the value, and in late call expansion that is the favored idiom,
and it is also the favored idiom when arguments and results use registers
instead of memory.

Change-Id: Ie0ccc70f399682a42509d847b330ef3956462d56
Reviewed-on: https://go-review.googlesource.com/c/go/+/240186
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-16 21:02:46 +00:00
David Chase
39da81da5e cmd/compile: populate AuxCall fields for OpClosureCall
Change-Id: Ib5f62826d5249c1727b57d9f8ff2f3a1d6dc5032
Reviewed-on: https://go-review.googlesource.com/c/go/+/240185
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-16 20:59:25 +00:00
David Chase
acde81e0a9 cmd/compile: initialize ACArgs and ACResults AuxCall fields for static and interface calls.
Extend use of AuxCall

Change-Id: I68b6d9bad09506532e1415fd70d44cf6c15b4b93
Reviewed-on: https://go-review.googlesource.com/c/go/+/239081
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-16 20:59:03 +00:00
David Chase
3c85e995ef cmd/compile: extend ssa.AuxCall to closure and interface calls
Also introduce helper methods.

Change-Id: I11a744ed002bae0ca9ebabba3206e1c14147e03d
Reviewed-on: https://go-review.googlesource.com/c/go/+/239080
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-16 20:58:14 +00:00
David Chase
b4ef49e527 cmd/compile: introduce special ssa Aux type for calls
This is prerequisite to moving call expansion later into SSA,
and probably a good idea anyway.  Passes tests.

This is the first minimal CL that does a 1-for-1 substitution
of *ssa.AuxCall for *obj.LSym.  Next step (next CL) is to make
this change for all calls so that additional information can
be stored in AuxCall.

Change-Id: Ia3a7715648fd9fb1a176850767a726e6f5b959eb
Reviewed-on: https://go-review.googlesource.com/c/go/+/237680
Trust: David Chase <drchase@google.com>
Run-TryBot: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-16 20:57:24 +00:00
Alberto Donizetti
7ee35cb301 cmd/compile: be more specific in cannot assign errors
"cannot assign to" compiler errors are very laconic: they never
explain why the lhs cannot be assigned to (with one exception, when
assigning to a struct field in a map).

This change makes them a little more specific, in two more cases: when
assigning to a string, or to a const; by giving a very brief reason
why the lhs cannot be assigned to.

Change-Id: I244cca7fc3c3814e00e0ccadeec62f747c293979
Reviewed-on: https://go-review.googlesource.com/c/go/+/255199
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-09-16 20:04:58 +00:00
Michael Anthony Knyszek
10dfb1dd3d runtime: actually fix locking in BenchmarkMSpanCountAlloc
I just submitted CL 255297 which mostly fixed this problem, but totally
forgot to actually acquire/release the heap lock. Oops.

Updates #41391.

Change-Id: I45b42f20a9fc765c4de52476db3654d4bfe9feb3
Reviewed-on: https://go-review.googlesource.com/c/go/+/255298
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-16 17:36:58 +00:00
Keith Randall
37f261010f cmd/compile: make go:notinheap error message friendlier for cgo
Update #40954

Change-Id: Ifaab7349631ccb12fc892882bbdf7f0ebf3d845f
Reviewed-on: https://go-review.googlesource.com/c/go/+/251158
Run-TryBot: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Keith Randall <khr@golang.org>
2020-09-16 17:28:13 +00:00
Keith Randall
42b023d7b9 cmd/cgo: use go:notinheap for anonymous structs
They can't reasonably be allocated on the heap. Not a huge deal, but
it has an interesting and useful side effect.

After CL 249917, the compiler and runtime treat pointers to
go:notinheap types as uintptrs instead of real pointers (no write
barrier, not processed during stack scanning, ...). That feature is
exactly what we want for cgo to fix #40954. All the cases we have of
pointers declared in C, but which might actually be filled with
non-pointer data, are of this form (JNI's jobject heirarch, Darwin's
CFType heirarchy, ...).

Fixes #40954

Change-Id: I44a3b9bc2513d4287107e39d0cbbd0efd46a3aae
Reviewed-on: https://go-review.googlesource.com/c/go/+/250940
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Keith Randall <khr@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-16 17:26:46 +00:00
Keith Randall
4f915911e8 cmd/compile: allow aliases to go:notinheap types
The alias doesn't need to be marked go:notinheap. It gets its
notinheap-ness from the target type.

Without this change, the type alias test in the notinheap.go file
generates these two errors:

notinheap.go:62: misplaced compiler directive
notinheap.go:63: type nih must be go:notinheap

The first is a result of go:notinheap pragmas not applying
to type alias declarations.
The second is the result of then trying to match the notinheap-ness
of the alias and the target type.

Add a few more go:notinheap tests while we are here.

Update #40954

Change-Id: I067ec47698df6e9e593e080d67796fd05a1d480f
Reviewed-on: https://go-review.googlesource.com/c/go/+/250939
Run-TryBot: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Keith Randall <khr@golang.org>
2020-09-16 17:24:33 +00:00
Carl Johnson
16328513bf flag: add Func
Fixes #39557

Change-Id: Ida578f7484335e8c6bf927255f75377eda63b563
GitHub-Last-Rev: b97294f766
GitHub-Pull-Request: golang/go#39880
Reviewed-on: https://go-review.googlesource.com/c/go/+/240014
Reviewed-by: Russ Cox <rsc@golang.org>
Trust: Ian Lance Taylor <iant@golang.org>
2020-09-16 17:13:14 +00:00
Michael Anthony Knyszek
2ae2a94857 runtime: fix leak and locking in BenchmarkMSpanCountAlloc
CL 249917 made the mspan in MSpanCountAlloc no longer stack-allocated
(for good reason), but then allocated an mspan on each call and did not
free it, resulting in a leak. That allocation was also not protected by
the heap lock, which could lead to data corruption of mheap fields and
the spanalloc.

To fix this, export some functions to allocate/free dummy mspans from
spanalloc (with proper locking) and allocate just one up-front for the
benchmark, freeing it at the end. Then, update MSpanCountAlloc to accept
a dummy mspan.

Note that we need to allocate the dummy mspan up-front otherwise we
measure things like heap locking and fixalloc performance instead of
what we actually want to measure: how fast we can do a popcount on the
mark bits.

Fixes #41391.

Change-Id: If6629a6ec1ece639c7fb78532045837a8c872c04
Reviewed-on: https://go-review.googlesource.com/c/go/+/255297
Run-TryBot: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Michael Knyszek <mknyszek@google.com>
2020-09-16 17:07:38 +00:00
Joel Sing
37aa653570 cmd/link: make it easier to debug an elfrelocsect size mismatch
Change-Id: I54976b004b4db006509f5e0781b1c2e46cfa09ab
Reviewed-on: https://go-review.googlesource.com/c/go/+/244577
Run-TryBot: Than McIntosh <thanm@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
Trust: Joel Sing <joel@sing.id.au>
2020-09-16 16:07:00 +00:00
diaxu01
a86b6f23f0 cmd/internal/obj/arm64: optimize the instruction of moving long effective stack address
Currently, when the offset of "MOVD $offset(Rn), Rd" is a large positive
constant or a negative constant, the assembler will load this offset from
the constant pool.This patch gets rid of the constant pool by encoding the
offset into two ADD instructions if it's a large positive constant or one
SUB instruction if negative. For very large negative offset, it is rarely
used, here we don't optimize this case.

Optimized case 1: MOVD $-0x100000(R7), R0
Before: LDR 0x67670(constant pool), R27; ADD R27.UXTX, R0, R7
After: SUB $0x100000, R7, R0

Optimized case 2: MOVD $0x123468(R7), R0
Before: LDR 0x67670(constant pool), R27; ADD R27.UXTX, R0, R7
After: ADD $0x123000, R7, R27; ADD $0x000468, R27, R0

1. Binary size before/after.
binary                 size change
pkg/linux_arm64        +4KB
pkg/tool/linux_arm64   no change
go                     no change
gofmt                  no change

2. go1 benckmark.
name                      old time/op                new time/op                delta
pkg:test/bench/go1 goos:linux goarch:arm64
BinaryTree17-64           7335721401.800000ns +-40%  6264542009.800000ns +-14%    ~     (p=0.421 n=5+5)
Fannkuch11-64             3886551822.600000ns +- 0%  3875870590.200000ns +- 0%    ~     (p=0.151 n=5+5)
FmtFprintfEmpty-64                82.960000ns +- 1%          83.900000ns +- 2%  +1.13%  (p=0.048 n=5+5)
FmtFprintfString-64              149.200000ns +- 1%         148.000000ns +- 0%  -0.80%  (p=0.016 n=5+4)
FmtFprintfInt-64                 177.000000ns +- 0%         178.400000ns +- 2%    ~     (p=0.794 n=4+5)
FmtFprintfIntInt-64              240.200000ns +- 2%         239.400000ns +- 4%    ~     (p=0.302 n=5+5)
FmtFprintfPrefixedInt-64         300.400000ns +- 0%         299.200000ns +- 1%    ~     (p=0.119 n=5+5)
FmtFprintfFloat-64               360.000000ns +- 0%         361.600000ns +- 3%    ~     (p=0.349 n=4+5)
FmtManyArgs-64                  1064.400000ns +- 1%        1061.400000ns +- 0%    ~     (p=0.087 n=5+5)
GobDecode-64                12080404.400000ns +- 2%    11637601.000000ns +- 1%  -3.67%  (p=0.008 n=5+5)
GobEncode-64                 8474973.800000ns +- 2%     7977801.600000ns +- 2%  -5.87%  (p=0.008 n=5+5)
Gzip-64                    416501238.400000ns +- 0%   410463405.400000ns +- 0%  -1.45%  (p=0.008 n=5+5)
Gunzip-64                   58088415.200000ns +- 0%    58826209.600000ns +- 0%  +1.27%  (p=0.008 n=5+5)
HTTPClientServer-64           128660.200000ns +-23%      117840.800000ns +- 8%    ~     (p=0.222 n=5+5)
JSONEncode-64               17547746.800000ns +- 4%    17216180.000000ns +- 1%    ~     (p=0.222 n=5+5)
JSONDecode-64               80879896.000000ns +- 1%    80063737.200000ns +- 0%  -1.01%  (p=0.008 n=5+5)
Mandelbrot200-64             5484901.600000ns +- 0%     5483614.400000ns +- 0%    ~     (p=0.310 n=5+5)
GoParse-64                   6201166.800000ns +- 6%     6150920.600000ns +- 1%    ~     (p=0.548 n=5+5)
RegexpMatchEasy0_32-64           135.000000ns +- 0%         139.200000ns +- 7%    ~     (p=0.643 n=5+5)
RegexpMatchEasy0_1K-64           484.600000ns +- 2%         483.800000ns +- 2%    ~     (p=0.984 n=5+5)
RegexpMatchEasy1_32-64           128.000000ns +- 1%         124.600000ns +- 1%  -2.66%  (p=0.008 n=5+5)
RegexpMatchEasy1_1K-64           769.400000ns +- 2%         761.400000ns +- 1%    ~     (p=0.460 n=5+5)
RegexpMatchMedium_32-64           12.900000ns +- 0%          12.500000ns +- 0%  -3.10%  (p=0.008 n=5+5)
RegexpMatchMedium_1K-64        57879.200000ns +- 1%       56512.200000ns +- 0%  -2.36%  (p=0.008 n=5+5)
RegexpMatchHard_32-64           3091.600000ns +- 1%        3071.000000ns +- 0%  -0.67%  (p=0.048 n=5+5)
RegexpMatchHard_1K-64          92941.200000ns +- 1%       92794.000000ns +- 0%    ~     (p=1.000 n=5+5)
Revcomp-64                1695605187.000000ns +-54%  1821697637.400000ns +-47%    ~     (p=1.000 n=5+5)
Template-64                112839686.800000ns +- 1%   109964069.200000ns +- 3%    ~     (p=0.095 n=5+5)
TimeParse-64                     587.000000ns +- 0%         587.000000ns +- 0%    ~     (all equal)
TimeFormat-64                    586.000000ns +- 1%         584.200000ns +- 1%    ~     (p=0.659 n=5+5)
[Geo mean]                      81804.262218ns             80694.712973ns       -1.36%

name                      old speed                  new speed                  delta
pkg:test/bench/go1 goos:linux goarch:arm64
GobDecode-64                         63.6MB/s +- 2%             66.0MB/s +- 1%  +3.78%  (p=0.008 n=5+5)
GobEncode-64                         90.6MB/s +- 2%             96.2MB/s +- 2%  +6.23%  (p=0.008 n=5+5)
Gzip-64                              46.6MB/s +- 0%             47.3MB/s +- 0%  +1.47%  (p=0.008 n=5+5)
Gunzip-64                             334MB/s +- 0%              330MB/s +- 0%  -1.25%  (p=0.008 n=5+5)
JSONEncode-64                         111MB/s +- 4%              113MB/s +- 1%    ~     (p=0.222 n=5+5)
JSONDecode-64                        24.0MB/s +- 1%             24.2MB/s +- 0%  +1.02%  (p=0.008 n=5+5)
GoParse-64                           9.35MB/s +- 6%             9.42MB/s +- 1%    ~     (p=0.571 n=5+5)
RegexpMatchEasy0_32-64                237MB/s +- 0%              231MB/s +- 7%    ~     (p=0.690 n=5+5)
RegexpMatchEasy0_1K-64               2.11GB/s +- 2%             2.12GB/s +- 2%    ~     (p=1.000 n=5+5)
RegexpMatchEasy1_32-64                250MB/s +- 1%              257MB/s +- 1%  +2.63%  (p=0.008 n=5+5)
RegexpMatchEasy1_1K-64               1.33GB/s +- 2%             1.35GB/s +- 1%    ~     (p=0.548 n=5+5)
RegexpMatchMedium_32-64              77.6MB/s +- 0%             79.8MB/s +- 0%  +2.80%  (p=0.008 n=5+5)
RegexpMatchMedium_1K-64              17.7MB/s +- 1%             18.1MB/s +- 0%  +2.41%  (p=0.008 n=5+5)
RegexpMatchHard_32-64                10.4MB/s +- 1%             10.4MB/s +- 0%    ~     (p=0.056 n=5+5)
RegexpMatchHard_1K-64                11.0MB/s +- 1%             11.0MB/s +- 0%    ~     (p=0.984 n=5+5)
Revcomp-64                            188MB/s +-71%              155MB/s +-71%    ~     (p=1.000 n=5+5)
Template-64                          17.2MB/s +- 1%             17.7MB/s +- 3%    ~     (p=0.095 n=5+5)
[Geo mean]                            79.2MB/s                   79.3MB/s       +0.24%

Change-Id: I593ac3e7037afafc3605ad4b0cfb51d5dd88015d
Reviewed-on: https://go-review.googlesource.com/c/go/+/232438
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
2020-09-16 14:56:18 +00:00
Tobias Klauser
e82c9bd816 os, internal/syscall/unix: use pipe2 instead of pipe on illumos
Illumos provides the pipe2 syscall. Add a wrapper to
internal/syscall/unix and use it to implement os.Pipe.

Change-Id: I26ecdbcae1e8d51f80e2bc8a86fb129826387b1f
Reviewed-on: https://go-review.googlesource.com/c/go/+/254981
Trust: Tobias Klauser <tobias.klauser@gmail.com>
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-16 08:49:14 +00:00
Martin Möhrmann
790fa1c546 cmd/compile: unify reflect, string and slice copy runtime functions
Use a common runtime slicecopy function to copy strings or slices
into slices. This deduplicates similar code previously used in
reflect.slicecopy and runtime.stringslicecopy.

Change-Id: I09572ff0647a9e12bb5c6989689ce1c43f16b7f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/254658
Run-TryBot: Martin Möhrmann <moehrmann@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Martin Möhrmann <moehrmann@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-16 04:37:14 +00:00
Ian Lance Taylor
eaa97fbf20 cmd/cgo: don't translate bitfields into Go fields
The cgo tool would sometimes emit a bitfield at an offset that did not
correspond to the C offset, such as for the example in the new test.

Change-Id: I61b2ca10ee44a42f81c13ed12865f2060168fed5
Reviewed-on: https://go-review.googlesource.com/c/go/+/252378
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-16 03:02:13 +00:00
Henrique Vicente
b6dbaef68f os/signal: add NotifyContext to cancel context using system signals
Fixes #37255

Change-Id: Ic0fde3498afefed6e4447f8476e4da7c1faa7145
Reviewed-on: https://go-review.googlesource.com/c/go/+/219640
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Giovanni Bajo <rasky@develer.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-15 23:14:33 +00:00
Bryan C. Mills
8248b5791c cmd/go/internal/modget: factor out functions for argument resolution
For #37438
For #41315
For #36460

Change-Id: I17041c35ec91ff6ffb547e0f32572673d191b1ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/254820
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-15 21:37:17 +00:00
Jay Conrod
dbde566219 cmd/go: default to -mod=readonly in most commands
For #40728

Change-Id: I6618f1b5a632e8b353a483a83bb0cdf4ef6df72c
Reviewed-on: https://go-review.googlesource.com/c/go/+/251881
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-15 18:53:55 +00:00
Bryan C. Mills
3ab825de9d cmd/go/internal/modget: warn about unmatched packages exactly once
Due to an inverted condition, we were emitting a "matched no packages"
warning twice in some cases and not at all in others.

For #41315

Change-Id: I472cd2d4f75811c8734852f2bdd7346f4c612816
Reviewed-on: https://go-review.googlesource.com/c/go/+/254819
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Michael Matloob <matloob@golang.org>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-15 18:48:09 +00:00
Daniel Martí
de0957dc08 cmd/go: relax version's error on unexpected flags
In https://golang.org/cl/221397 we made commands like "go version -v"
error, since both of the command's flags only make sense when arguments
follow them. Without arguments, the command only reports Go's own
version, and the flags are most likely a mistake.

However, the script below is entirely reasonable:

	export GOFLAGS=-v # make all Go commands verbose
	go version
	go build

After the previous CL, "go version" would error. Instead, only error if
the flag was passed explicitly, and not via GOFLAGS.

The patch does mean that we won't error on "GOFLAGS=-v go version -v",
but that very unlikely false negative is okay. The error is only meant
to help the user not misuse the flags, anyway - it's not a critical
error of any sort.

To reuse inGOFLAGS, we move it to the base package and export it there,
since it's where the rest of the GOFLAGS funcs are.

Fixes #41264.

Change-Id: I74003dd25d94bacf9ac507b5cad778fd65233321
Reviewed-on: https://go-review.googlesource.com/c/go/+/254157
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Daniel Martí <mvdan@mvdan.cc>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
2020-09-15 15:30:56 +00:00
Jay Conrod
f1c400a063 cmd/go: fix broken mod_outside test
Since CL 254369, 'go mod graph' now reports an error when invoked
outside a module. This broke the mod_outside test, which expected
'go mod graph' to succeed with no output.

Change-Id: Ic30ee68f1f4c4d33795bdf7df70a7631fb9395e5
Reviewed-on: https://go-review.googlesource.com/c/go/+/255017
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
2020-09-15 14:51:32 +00:00
Jay Conrod
03875bd9bc cmd/go: add modload.NeedRoot mode for commands that need module root
This makes error reporting a bit more consistent for 'go mod'
subcommands. Most of these commands only work in module mode when a
go.mod file is present.

Setting modload.ForceUseModules reports an error when GO111MODULE=off.

Setting modload.RootMode to modload.NeedRoot reports an error when no
go.mod file is present.

Change-Id: I1daa8d2971cb8658e0c804765839d903734a412e
Reviewed-on: https://go-review.googlesource.com/c/go/+/254369
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-15 12:46:25 +00:00
Jay Conrod
e306363612 cmd/go: implement 'go install pkg@version'
With this change, 'go install' will install executables in module mode
without using or modifying the module in the current directory, if
there is one.

For #40276

Change-Id: I922e71719b3a4e0c779ce7a30429355fc29930bf
Reviewed-on: https://go-review.googlesource.com/c/go/+/254365
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Reviewed-by: Michael Matloob <matloob@golang.org>
2020-09-15 12:45:59 +00:00
Xiangdong Ji
ea33523877 cmd/compile: rewrite some ARM64 rules to use typed aux
Passes toolstash-check -all.

Change-Id: I7ec36bc048f3031c8201107e6fc5d1257271dbf1
Reviewed-on: https://go-review.googlesource.com/c/go/+/234379
Run-TryBot: Alberto Donizetti <alb.donizetti@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-15 07:05:36 +00:00
Tobias Klauser
3b64e6b010 internal/poll, internal/syscall/unix, net: enable writev on illumos
Illumos supports iovec read/write. Add the writev wrapper to
internal/syscall/unix and use it to implement internal/poll.writev for
net.(*netFD).writeBuffers.

Change-Id: Ie256c2f96aba8e61fb21991788789a049425f792
Reviewed-on: https://go-review.googlesource.com/c/go/+/254638
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Tobias Klauser <tobias.klauser@gmail.com>
2020-09-15 05:08:53 +00:00
Matthew Dempsky
bae9cf6517 test: fix inline.go to pass linux-amd64-noopt
Updates #33485.

Change-Id: I3330860cdff1e9797466a7630bcdb7792c465b06
Reviewed-on: https://go-review.googlesource.com/c/go/+/254938
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
2020-09-15 02:52:12 +00:00
Cuong Manh Le
d20298e1c7 cmd/compile: make funccompile non-reentrant
Currently, there's awkward reentrancy issue with funccompile:

    funccompile -> compile -> dtypesym -> geneq/genhash/genwrapper -> funccompile

Though it's not a problem at this moment, some attempts by @mdempsky to
move order/walk/instrument into buildssa was failed, due to SSA cache
corruption.

This commit fixes that reentrancy issue, by making generated functions
to be pumped through the same compile workqueue that normal functions
are compiled. We do this by adding them to xtop, instead of calling
funccompile directly in geneq/genhash/genwrapper. In dumpdata, we look
for uncompiled functions in xtop instead of compilequeue, then finish
compiling them.

Updates #38463
Fixes #33485

Change-Id: Ic9f0ce45b56ae2ff3862f17fd979253ddc144bb5
Reviewed-on: https://go-review.googlesource.com/c/go/+/254617
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Keith Randall <khr@golang.org>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2020-09-15 02:05:31 +00:00
Matthew Dempsky
f4936d09fd cmd/compile: call fninit earlier
This allows the global initializers function to go through normal
mid-end optimizations (e.g., inlining, escape analysis) like any other
function.

Updates #33485.

Change-Id: I9bcfe98b8628d1aca09b4c238d8d3b74c69010a5
Reviewed-on: https://go-review.googlesource.com/c/go/+/254839
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
2020-09-14 23:42:44 +00:00
Matthew Dempsky
237410547b cmd/compile: better dclcontext handling in func{hdr,body}
funchdr and funcbody currently assume that either (1) Curfn == nil &&
dclcontext == PEXTERN, or (2) Curfn != nil && dclcontext == PAUTO.
This is a reasonable assumption during parsing. However, these
functions end up getting used in other contexts, and not all callers
are so disciplined about Curfn/dclcontext handling.

This CL changes them to save/restore arbitrary Curfn/dclcontext pairs
instead. This is necessary for the followup CL, which pushes fninit
earlier. Otherwise, Curfn/dclcontext fall out of sync, and funchdr
panics.

Passes toolstash-check.

Updates #33485.

Change-Id: I19b1be23db1bad6475345ae5c81bbdc66291a3a7
Reviewed-on: https://go-review.googlesource.com/c/go/+/254838
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Trust: Matthew Dempsky <mdempsky@google.com>
2020-09-14 23:42:35 +00:00
Damien Neil
506eb0a9b1 Revert "encoding/json: implement Is on SyntaxError"
This reverts CL 253037.

Reason for revert: The recommended way to check for a type of error is errors.As. API changes should also start with a proposal.

Change-Id: I62896717aa47ed491c2c4775d2b05d80e5e9cde3
Reviewed-on: https://go-review.googlesource.com/c/go/+/254837
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-14 21:33:53 +00:00
Damien Neil
114719e16e Revert "encoding/json: implement Is on all errors"
This reverts CL 254537.

Reason for revert: Reason for revert: The recommended way to check for a type of error is errors.As. API changes should also start with a proposal.

Change-Id: I07c37428575e99c80b17525833a61831d10963bb
Reviewed-on: https://go-review.googlesource.com/c/go/+/254857
Trust: Damien Neil <dneil@google.com>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-14 21:33:20 +00:00
Changkun Ou
a408139bb0 testing: fix panicking tests hang if Cleanup calls FailNow
Previously, it was impossible to call FailNow in a Cleanup.
Because it can terminate a panicking goroutine and cause its
parent hangs on t.signal channel. This CL sends the signal
in a deferred call to prevent the hang.

Fixes #41355

Change-Id: I4552d3a7ea763ef86817bf9b50c0e37fb34bf20f
Reviewed-on: https://go-review.googlesource.com/c/go/+/254637
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-14 20:27:49 +00:00
Bryan C. Mills
5764653429 cmd/api: omit outside dependencies when listing the packages in "std"
As of CL 251159, when 'go list -deps std' is run within GOROOT/src, it
treats the vendored external dependencies as real module dependencies,
not standard-library "vendor/" packages (which still exist in that
case, but are treated as distinct packages outside the "std" module).

Fixes #41358
Updates #30241

Change-Id: Ic23eae9829d90e74a340d49ca9052e9191597410
Reviewed-on: https://go-review.googlesource.com/c/go/+/254738
Run-TryBot: Bryan C. Mills <bcmills@google.com>
Trust: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
2020-09-14 19:45:28 +00:00
Martin Möhrmann
14c7caae50 runtime: add 24 byte allocation size class
This CL introduces a 24 byte allocation size class which
fits 3 pointers on 64 bit and 6 pointers on 32 bit architectures.

Notably this new size class fits a slice header on 64 bit
architectures exactly while previously a 32 byte size class
would have been used for allocating a slice header on the heap.

The main complexity added with this CL is that heapBitsSetType
needs to handle objects that aren't 16-byte aligned but contain
more than a single pointer on 64-bit architectures.

Due to having a non 16 byte aligned size class on 32 bit a
h.shift of 2 is now possible which means a heap bitmap byte might
only be partially written. Due to this already having been
possible on 64 bit before the heap bitmap code only needed
minor adjustments for 32 bit doublecheck code paths.

Note that this CL changes the slice capacity allocated by append
for slice growth to a target capacity of 17 to 24 bytes.

On 64 bit architectures the capacity of the slice returned by
append([]byte{}, make([]byte, 24)...)) is 32 bytes before and
24 bytes after this CL. Depending on allocation patterns of the
specific Go program this can increase the number of total
alloctions as subsequent appends to the slice can trigger slice
growth earlier than before. On the other side if the slice is
never appended to again above its capacity this will lower heap
usage by 8 bytes.

This CL changes the set of size classes reported in the
runtime.MemStats.BySize array due to it being limited to a
total of 61 size classes. The new 24 byte size class is now
included and the 20480 byte size class is not included anymore.

Fixes #8885

name                      old time/op       new time/op       delta
Template                        196ms ± 3%        194ms ± 2%    ~     (p=0.247 n=10+10)
Unicode                        85.6ms ±16%       88.1ms ± 1%    ~     (p=0.165 n=10+10)
GoTypes                         673ms ± 2%        668ms ± 2%    ~     (p=0.258 n=9+9)
Compiler                        3.14s ± 6%        3.08s ± 1%    ~     (p=0.243 n=10+9)
SSA                             6.82s ± 1%        6.76s ± 1%  -0.87%  (p=0.006 n=9+10)
Flate                           128ms ± 7%        127ms ± 3%    ~     (p=0.739 n=10+10)
GoParser                        154ms ± 3%        153ms ± 4%    ~     (p=0.730 n=9+9)
Reflect                         404ms ± 1%        412ms ± 4%  +1.99%  (p=0.022 n=9+10)
Tar                             172ms ± 4%        170ms ± 4%    ~     (p=0.065 n=10+9)
XML                             231ms ± 4%        230ms ± 3%    ~     (p=0.912 n=10+10)
LinkCompiler                    341ms ± 1%        339ms ± 1%    ~     (p=0.243 n=9+10)
ExternalLinkCompiler            1.72s ± 1%        1.72s ± 1%    ~     (p=0.661 n=9+10)
LinkWithoutDebugCompiler        221ms ± 2%        221ms ± 2%    ~     (p=0.529 n=10+10)
StdCmd                          18.4s ± 3%        18.2s ± 1%    ~     (p=0.515 n=10+8)

name                      old user-time/op  new user-time/op  delta
Template                        238ms ± 4%        243ms ± 6%    ~     (p=0.661 n=9+10)
Unicode                         116ms ± 6%        113ms ± 3%  -3.37%  (p=0.035 n=9+10)
GoTypes                         854ms ± 2%        848ms ± 2%    ~     (p=0.604 n=9+10)
Compiler                        4.10s ± 1%        4.11s ± 1%    ~     (p=0.481 n=8+9)
SSA                             9.49s ± 1%        9.41s ± 1%  -0.92%  (p=0.001 n=9+10)
Flate                           149ms ± 6%        151ms ± 7%    ~     (p=0.481 n=10+10)
GoParser                        189ms ± 2%        190ms ± 2%    ~     (p=0.497 n=9+10)
Reflect                         511ms ± 2%        508ms ± 2%    ~     (p=0.211 n=9+10)
Tar                             215ms ± 4%        212ms ± 3%    ~     (p=0.105 n=10+10)
XML                             288ms ± 2%        288ms ± 2%    ~     (p=0.971 n=10+10)
LinkCompiler                    559ms ± 4%        557ms ± 1%    ~     (p=0.968 n=9+10)
ExternalLinkCompiler            1.78s ± 1%        1.77s ± 1%    ~     (p=0.055 n=8+10)
LinkWithoutDebugCompiler        245ms ± 3%        245ms ± 2%    ~     (p=0.684 n=10+10)

name                      old alloc/op      new alloc/op      delta
Template                       34.8MB ± 0%       34.4MB ± 0%  -0.95%  (p=0.000 n=9+10)
Unicode                        28.6MB ± 0%       28.3MB ± 0%  -0.95%  (p=0.000 n=10+10)
GoTypes                         115MB ± 0%        114MB ± 0%  -1.02%  (p=0.000 n=10+9)
Compiler                        554MB ± 0%        549MB ± 0%  -0.86%  (p=0.000 n=9+10)
SSA                            1.28GB ± 0%       1.27GB ± 0%  -0.83%  (p=0.000 n=10+10)
Flate                          21.8MB ± 0%       21.6MB ± 0%  -0.87%  (p=0.000 n=8+10)
GoParser                       26.7MB ± 0%       26.4MB ± 0%  -0.97%  (p=0.000 n=10+9)
Reflect                        75.0MB ± 0%       74.1MB ± 0%  -1.18%  (p=0.000 n=10+10)
Tar                            32.6MB ± 0%       32.3MB ± 0%  -0.94%  (p=0.000 n=10+7)
XML                            41.5MB ± 0%       41.2MB ± 0%  -0.90%  (p=0.000 n=10+8)
LinkCompiler                    105MB ± 0%        104MB ± 0%  -0.94%  (p=0.000 n=10+10)
ExternalLinkCompiler            153MB ± 0%        152MB ± 0%  -0.69%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler       63.7MB ± 0%       63.6MB ± 0%  -0.13%  (p=0.000 n=10+10)

name                      old allocs/op     new allocs/op     delta
Template                         336k ± 0%         336k ± 0%  +0.02%  (p=0.002 n=10+10)
Unicode                          332k ± 0%         332k ± 0%    ~     (p=0.447 n=10+10)
GoTypes                         1.16M ± 0%        1.16M ± 0%  +0.01%  (p=0.001 n=10+10)
Compiler                        4.92M ± 0%        4.92M ± 0%  +0.01%  (p=0.000 n=10+10)
SSA                             11.9M ± 0%        11.9M ± 0%  +0.02%  (p=0.000 n=9+10)
Flate                            214k ± 0%         214k ± 0%  +0.02%  (p=0.032 n=10+8)
GoParser                         270k ± 0%         270k ± 0%  +0.02%  (p=0.004 n=10+9)
Reflect                          877k ± 0%         877k ± 0%  +0.01%  (p=0.000 n=10+10)
Tar                              313k ± 0%         313k ± 0%    ~     (p=0.075 n=9+10)
XML                              387k ± 0%         387k ± 0%  +0.02%  (p=0.007 n=10+10)
LinkCompiler                     455k ± 0%         456k ± 0%  +0.08%  (p=0.000 n=10+9)
ExternalLinkCompiler             670k ± 0%         671k ± 0%  +0.06%  (p=0.000 n=10+10)
LinkWithoutDebugCompiler         113k ± 0%         113k ± 0%    ~     (p=0.149 n=10+10)

name                      old maxRSS/op     new maxRSS/op     delta
Template                        34.1M ± 1%        34.1M ± 1%    ~     (p=0.853 n=10+10)
Unicode                         35.1M ± 1%        34.6M ± 1%  -1.43%  (p=0.000 n=10+10)
GoTypes                         72.8M ± 3%        73.3M ± 2%    ~     (p=0.724 n=10+10)
Compiler                         288M ± 3%         295M ± 4%    ~     (p=0.393 n=10+10)
SSA                              630M ± 1%         622M ± 1%  -1.18%  (p=0.001 n=10+10)
Flate                           26.0M ± 1%        26.2M ± 2%    ~     (p=0.493 n=10+10)
GoParser                        28.6M ± 1%        28.5M ± 2%    ~     (p=0.256 n=10+10)
Reflect                         55.5M ± 2%        55.4M ± 1%    ~     (p=0.436 n=10+10)
Tar                             33.0M ± 1%        32.8M ± 2%    ~     (p=0.075 n=10+10)
XML                             38.7M ± 1%        39.0M ± 1%    ~     (p=0.053 n=9+10)
LinkCompiler                     164M ± 1%         164M ± 1%  -0.27%  (p=0.029 n=10+10)
ExternalLinkCompiler             174M ± 0%         173M ± 0%  -0.33%  (p=0.002 n=9+10)
LinkWithoutDebugCompiler         137M ± 0%         136M ± 2%    ~     (p=0.825 n=9+10)

Change-Id: I9ecf2a10024513abef8fbfbe519e44e0b29b6167
Reviewed-on: https://go-review.googlesource.com/c/go/+/242258
Trust: Martin Möhrmann <moehrmann@google.com>
Trust: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Martin Möhrmann <martisch@uos.de>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Keith Randall <khr@golang.org>
2020-09-14 19:21:56 +00:00
Cherry Zhang
c0c396bd6a misc/ios: quote paths
The paths may contain spaces. Quote them.

Change-Id: I1f67085a1e7c40f60282c2fea7104fb44a01e310
Reviewed-on: https://go-review.googlesource.com/c/go/+/254739
Run-TryBot: Cherry Zhang <cherryyz@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
2020-09-14 18:28:23 +00:00
Clément Chigot
86dbeefe1f syscall: fix fsync for read-only files on aix
AIX fsync syscall doesn't work on read-only files. Using fsync_range
instead allows syscall.Fsync to work on any files.

Fixes #41372

Change-Id: I66d33e847875496af53da60828c1bddf6c2b76b7
Reviewed-on: https://go-review.googlesource.com/c/go/+/254657
Run-TryBot: Tobias Klauser <tobias.klauser@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
2020-09-14 17:07:47 +00:00
Roberto Clapis
4f5cd0c033 net/http/cgi,net/http/fcgi: add Content-Type detection
This CL ensures that responses served via CGI and FastCGI
have a Content-Type header based on the content of the
response if not explicitly set by handlers.

If the implementers of the handler did not explicitly
specify a Content-Type both CGI implementations would default
to "text/html", potentially causing cross-site scripting.

Thanks to RedTeam Pentesting GmbH for reporting this.

Fixes #40928
Fixes CVE-2020-24553

Change-Id: I82cfc396309b5ab2e8d6e9a87eda8ea7e3799473
Reviewed-on: https://team-review.git.corp.google.com/c/golang/go-private/+/823217
Reviewed-by: Russ Cox <rsc@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/252179
Run-TryBot: Filippo Valsorda <filippo@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Katie Hockman <katie@golang.org>
2020-09-14 15:42:03 +00:00
Dominic Della Valle
66e66e7113 make.bat: fix compare between GOROOT and srcdir paths, when either contains whitespace.
CL 96455 brings CL 57753 to Windows
However, a path comparison within it was left unquoted.
If the Go source directory resides in a path containing whitespace,
the interpreter will compare against the first portion of the path string,
and treat the remainder as an expression.
This patch amends that.

For example, consider the path
`C:\Users\Dominic Della Valle\Projects\Go\goroot\src`
Issuing `make.bat` will print out `'Della' is not recognized as an internal or external command, operable program or batch file.` before proceeding.

Change-Id: Ifcec159baeec940c29c61aa721c64c13c6fd8c14
GitHub-Last-Rev: 809ddbb4db
GitHub-Pull-Request: golang/go#41319
Reviewed-on: https://go-review.googlesource.com/c/go/+/253898
Run-TryBot: Giovanni Bajo <rasky@develer.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Giovanni Bajo <rasky@develer.com>
Trust: Giovanni Bajo <rasky@develer.com>
2020-09-13 19:17:09 +00:00
Cuong Manh Le
afb5fca25a test: fix flaky test for issue24491
runtime.GC() doesn't guarantee the finalizer has run, so use a channel
instead to make sure finalizer was run in call to "after()".

Fixes #41361

Change-Id: I69c801e29aea49757ea72c52e8db13239de19ddc
Reviewed-on: https://go-review.googlesource.com/c/go/+/254401
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
2020-09-13 07:54:42 +00:00
Cuong Manh Le
5f1b12bfbe cmd/compile: remove nodeNeedsWrapper flag
CL 254397 attached OVARLIVE nodes to OCALLxxx nodes Nbody.

The NeedsWrapper flag is now redundant with n.Nbody.Len() > 0
condition, so use that condition instead and remove the flag.

Passes toolstash-check.

Change-Id: Iebc3e674d3c0040a876ca4be05025943d2b4fb31
Reviewed-on: https://go-review.googlesource.com/c/go/+/254398
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-09-13 04:35:35 +00:00
Cuong Manh Le
1f45216694 cmd/compile: attach OVARLIVE nodes to OCALLxxx
So we can insert theses OVARLIVE nodes right after OpStaticCall in SSA.

This helps fixing issue that unsafe-uintptr arguments are not kept alive
during return statement, or can be kept alive longer than expected.

Fixes #24491

Change-Id: Ic04a5d1bbb5c90dcfae65bd95cdd1da393a66800
Reviewed-on: https://go-review.googlesource.com/c/go/+/254397
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
2020-09-13 04:35:22 +00:00
Carlos Alexandro Becker
95bb00d108 encoding/json: implement Is on all errors
Allows users to check:

      errors.Is(err, &UnmarshalTypeError{})
      errors.Is(err, &UnmarshalFieldError{})
      errors.Is(err, &InvalidUnmarshalError{})
      errors.Is(err, &UnsupportedValueError{})
      errors.Is(err, &MarshalerError{})

which is the recommended way of checking for kinds of errors.

SyntaxError.Is was implemented in CL 253037.
As and Unwrap relevant methods will be added in future CLs.

Change-Id: I1f8a503b8fdc0f3afdfe9669a91f3af8d960e028
GitHub-Last-Rev: 930cda5384
GitHub-Pull-Request: golang/go#41360
Reviewed-on: https://go-review.googlesource.com/c/go/+/254537
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Trust: Emmanuel Odeke <emm.odeke@gmail.com>
2020-09-13 03:19:24 +00:00
Carlos Alexandro Becker
b3ef90ec73 encoding/json: implement Is on SyntaxError
Allows users to check:

      errors.Is(err, &json.SyntaxError{})

which is the recommended way of checking for kinds of errors.

Change-Id: I20dc805f20212765e9936a82d9cb7822e73ec4ef
GitHub-Last-Rev: e2627ccf8e
GitHub-Pull-Request: golang/go#41210
Reviewed-on: https://go-review.googlesource.com/c/go/+/253037
Reviewed-by: Emmanuel Odeke <emm.odeke@gmail.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-12 19:42:29 +00:00
Daniel Martí
92b2b8860d cmd/go: avoid flag.FlagSet.VisitAll at init time
We want to error early if GOFLAGS contains any flag that isn't known to
any cmd/go command. Thus, at init time we would recursively use VisitAll
on each of the flagsets to populate a map of all registered flags.

This was unfortunate, as populating said map constituted a whole 5% of
the run-time of 'go env GOARCH'. This is because VisitAll is pretty
expensive; it copies all the maps from the flagset's map to a slice,
sorts the slice, then does one callback per flag.

First, this was a bit wasteful. We only ever needed to query the
knownFlag map if GOFLAGS wasn't empty. If it's empty, there's no work to
do, thus we can skip the map populating work.

Second and most important, we don't actually need the map at all. A
flag.FlagSet already has a Lookup method, so we can simply recursively
call those methods for each flag in GOFLAGS. Add a hasFlag func to make
that evident.

This mechanism is different; its upfront cost is none, but it will
likely mean a handful of map lookups for each flag in GOFLAGS. However,
that tradeoff is worth it; we don't expect GOFLAGS to contain thousands
of flags. The most likely scenario is less than a dozen flags, in which
case constructing a "unified" map is not at all a net win.

One possible reason the previous mechanism was that way could be
AddKnownFlag. Thankfully, the one and only use of that API was removed
last year when Bryan cleaned up flag parsing in cmd/go.

The wins for the existing benchmark with an empty GOFLAGS are
significant:

	name         old time/op       new time/op       delta
	ExecGoEnv-8        575µs ± 1%        549µs ± 2%  -4.44%  (p=0.000 n=7+8)

	name         old sys-time/op   new sys-time/op   delta
	ExecGoEnv-8       1.69ms ± 1%       1.68ms ± 2%    ~     (p=0.281 n=7+8)

	name         old user-time/op  new user-time/op  delta
	ExecGoEnv-8       1.80ms ± 1%       1.66ms ± 2%  -8.09%  (p=0.000 n=7+8)

To prove that a relatively large number of GOFLAGS isn't getting
noticeably slower, we measured that as well, via benchcmd and GOFLAGS
containing 50 valid flags:

	GOFLAGS=$(yes -- -race | sed 50q) benchcmd -n 500 GoEnvGOFLAGS go env GOARCH

And the result, while noisy, shows no noticeable difference (note that
it measures 3ms instead of 0.6ms since it's sequential):

	name          old time/op         new time/op         delta
	GoEnvGOFLAGS         3.04ms ±32%         3.03ms ±35%    ~     (p=0.156 n=487+481)

Finally, we've improved the existing Go benchmark. Now it's parallel,
and it also reports sys-time and user-time, which are useful metrics.

Change-Id: I9b4551415cedf2f819eb184a02324b8bd919e2bd
Reviewed-on: https://go-review.googlesource.com/c/go/+/248757
Reviewed-by: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
2020-09-12 12:39:50 +00:00