Updates #48367.
Change-Id: Ib8fc6d9dd7c3c6a70fefe077615f51a71d9c42ed
Reviewed-on: https://go-review.googlesource.com/c/go/+/361899
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Dmitri Shuralyov <dmitshur@golang.org>
A small touchup after CL 361894.
For #47694.
Change-Id: Ifc161516f897f727195d21351a3c8eda7b6e327e
Reviewed-on: https://go-review.googlesource.com/c/go/+/361895
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Run-TryBot: Carlos Amedee <carlos@golang.org>
For #47694.
Change-Id: Ia80a1859bd0fc6f08d27293f519c22fd9a804fd2
Reviewed-on: https://go-review.googlesource.com/c/go/+/361894
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
In modules that specify 'go 1.17' or higher, the go.mod file
explicitly requires modules for all packages transitively imported by
the main module. Users tend to use 'go mod download' to prepare for
testing the main module itself, so we should only download those
relevant modules.
In 'go 1.16' and earlier modules, we continue to download all modules
in the module graph (because we cannot in general tell which ones are
relevant without loading the full package import graph).
'go mod download all' continues to download every module in
'go list all', as it did before.
Fixes#44435
Change-Id: I3f286c0e2549d6688b3832ff116e6cd77a19401c
Reviewed-on: https://go-review.googlesource.com/c/go/+/357310
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Michael Matloob <matloob@golang.org>
This reverts CL 359096.
Updates #47788.
Reason for revert: -buildmode=shared may have actually been working in a few very specific cases. We should not remove -buildmode=shared until we have implemented an alternative to support those few cases.
Change-Id: Ia962b06abacc11f6f29fc29d092773be175e32f1
Reviewed-on: https://go-review.googlesource.com/c/go/+/359575
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Change-Id: I032791a3cda1916099b0fd2955dcca4e69763660
Reviewed-on: https://go-review.googlesource.com/c/go/+/357958
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
It never worked in module mode (or with a read-only GOROOT).
A proposal to drop it was filed (and approved) in
https://golang.org/issue/47788.
Fixes#47788
Change-Id: I0c12f38eb0c5dfe9384fbdb49ed202301fa4273d
Reviewed-on: https://go-review.googlesource.com/c/go/+/359096
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Mention future versions will require the COMPAT_FREEBSD12 kernel option
set in the kernel.
For #47694.
Change-Id: Ia94c4f9dbb38c68025d3c1d12dd2e241a5480a6b
Reviewed-on: https://go-review.googlesource.com/c/go/+/354971
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Tobias Klauser <tobias.klauser@gmail.com>
For #43684
Change-Id: I9ce47de82203ec87e7d3683f56e6c6d61ae255f5
Reviewed-on: https://go-review.googlesource.com/c/go/+/352151
Trust: Jay Conrod <jayconrod@google.com>
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
gofmt is pretty heavily CPU-bound, since parsing and formatting 1MiB
of Go code takes much longer than reading that amount of bytes from
disk. However, parsing and manipulating a large Go source file is very
difficult to parallelize, so we continue to process each file in its
own goroutine.
A Go module may contain a large number of Go source files, so we need
to bound the amount of work in flight. However, because the
distribution of sizes for Go source files varies widely — from tiny
doc.go files containing a single package comment all the way up to
massive API wrappers generated by automated tools — the amount of
time, work, and memory overhead needed to process each file also
varies. To account for this variability, we limit the in-flight work
by bytes of input rather than by number of files. That allows us to
make progress on many small files while we wait for work on a handful
of large files to complete.
The gofmt tool has a well-defined output format on stdout, which was
previously deterministic. We keep it deterministic by printing the
results of each file in order, using a lazily-synchronized io.Writer
(loosly inspired by Haskell's IO monad). After a file has been
formatted in memory, we keep it in memory (again, limited by the
corresponding number of input bytes) until the output for all previous
files has been flushed. This adds a bit of latency compared to
emitting the output in nondeterministic order, but a little extra
latency seems worth the cost to preserve output stability.
This change is based on Daniel Martí's work in CL 284139, but using a
weighted semaphore and ephemeral goroutines instead of a worker pool
and batches. Benchmark results are similar, and I find the concurrency
in this approach a bit easier to reason about.
In the batching-based approach, the batch size allows us to "look
ahead" to find large files and start processing them early. To keep
the CPUs saturated and prevent stragglers, we would need to tune the
batch size to be about the same as the largest input files. If the
batch size is set too high, a large batch of small files could turn
into a straggler, but if the batch size is set too low, the largest
files in the data set won't be started early enough and we'll end up
with a large-file straggler.
One possible alternative would be to sort by file size instead of
batching: identify all of the files to be processed, sort from largest
to smallest, and then process the largest files first so that the
"tail" of processing covers the smallest files. However, that approach
would still fail to saturate available CPU when disk latency is high,
would require buffering an arbitrary amount of metadata in order to
sort by size, and (perhaps most importantly!) would not allow the
`gofmt` binary to preserve the same (deterministic) output order that
it has today.
In contrast, with a semaphore we can produce the same deterministic
output as ever using only one tuning parameter: the memory footprint,
expressed as a rough lower bound on the amount of RAM available per
thread. While we're below the memory limit, we can run arbitrarily
many disk operations arbitrarily far ahead, and process the results of
those operations whenever they become avaliable. Then it's up to the
kernel (not us) to schedule the disk operations for throughput and
latency, and it's up to the runtime (not us) to schedule the
goroutines so that they complete quickly.
In practice, even a modest assumption of a few megabytes per thread
seems to provide a nice speedup, and it should scale reasonably even
to machines with vastly different ratios of CPU to disk. (In practice,
I expect that most 'gofmt' invocations will work with files on at most
one physical disk, so the CPU:disk ratio should vary more-or-less
directly with the thread count, whereas the CPU:memory ratio is
more-or-less independent of thread count.)
name \ time/op baseline.txt 284139.txt simplified.txt
GofmtGorootCmd 11.9s ± 2% 2.7s ± 3% 2.8s ± 5%
name \ user-time/op baseline.txt 284139.txt simplified.txt
GofmtGorootCmd 13.5s ± 2% 14.4s ± 1% 14.7s ± 1%
name \ sys-time/op baseline.txt 284139.txt simplified.txt
GofmtGorootCmd 465ms ± 8% 229ms ±28% 232ms ±31%
name \ peak-RSS-bytes baseline.txt 284139.txt simplified.txt
GofmtGorootCmd 77.7MB ± 4% 162.2MB ±10% 192.9MB ±15%
For #43566
Change-Id: I4ba251eb4d188a3bd1901039086be57f0b341910
Reviewed-on: https://go-review.googlesource.com/c/go/+/317975
Trust: Bryan C. Mills <bcmills@google.com>
Trust: Daniel Martí <mvdan@mvdan.cc>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
Thanks to @bodar (Github) for finding this.
Fixes#48422.
Change-Id: I031c3d82a02db1d204e2b86b494d89784d37f073
Reviewed-on: https://go-review.googlesource.com/c/go/+/350409
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
CL 85215 added prose to provide some minimal intuition for the
definition of a "terminating statement". While the original definition
was perfectly fine, the added prose was actually incorrect: If the
terminating statement is a goto, it might jump to a labeled statement
following that goto in the same block (it could be the very next
statement), and thus a terminating statement does not in fact
"prevent execution of all statements that lexically appear after
it in the same block".
Rather than explaining the special case for gotos with targets that
are lexically following the goto in the same block, this CL opts for
a simpler approach.
Thanks to @3bodar (Github) for finding this.
Fixes#48323.
Change-Id: I8031346250341d038938a1ce6a75d3e687d32c37
Reviewed-on: https://go-review.googlesource.com/c/go/+/349172
Trust: Robert Griesemer <gri@golang.org>
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Emmanuel Odeke <emmanuel@orijtech.com>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Change-Id: I33bde4835d3b83fafd55beea483f6236c4c62840
Reviewed-on: https://go-review.googlesource.com/c/go/+/338990
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Trust: Than McIntosh <thanm@google.com>
So it's clear to the reader that if "M" is a promoted method from
embedded field "T", then "x.M" will be expanded to "x.T.M" during the
evaluation of the method value.
Fixes#47863
Change-Id: Id3b82127a2054584b6842c487f6e15c3102dc9fe
Reviewed-on: https://go-review.googlesource.com/c/go/+/344209
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Rob Pike <r@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
This CL adds a new syscall.SyscallN API.
The proposal discussion also suggests the API should not only for
Windows but other platforms. However, the existing API set already
contain differences between platforms, hence the CL only implements
the Windows platform.
Moreover, although the API offers variadic parameters, the permitted
parameters remains up to a limit, which is selected as 42, and arguably
large enough.
Fixes#46552
Change-Id: I66b49988a304d9fc178c7cd5de46d0b75e167a4f
Reviewed-on: https://go-review.googlesource.com/c/go/+/336550
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Trust: Matthew Dempsky <mdempsky@google.com>
Trust: Jason A. Donenfeld <Jason@zx2c4.com>
This template is based on CL 295209 and previous ones like it.
Continue to eagerly include often-used sections, and clarify that
the TODO is about completing the section, or removing if it turns
out not to be needed.
Move the Go 1.17 release notes to x/website, since that's the new
home for past Go release notes as of CL 291711. They're added to
x/website in CL 342089.
For #47694
Updates #44513
Change-Id: I24962eed800d1509bdf71b7d7f819a683eb96f8f
Reviewed-on: https://go-review.googlesource.com/c/go/+/342070
Trust: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
The current text is slightly inaccurate. Make it more correct.
Change-Id: Iebe0051b74649d13982d7eefe3697f9e69c9b75d
Reviewed-on: https://go-review.googlesource.com/c/go/+/340449
Trust: Russ Cox <rsc@golang.org>
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-by: Damien Neil <dneil@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Fixes#44513.
Change-Id: I82c44a681b1fa67df123af86ee02a980b13acdc8
Reviewed-on: https://go-review.googlesource.com/c/go/+/341673
Trust: Dmitri Shuralyov <dmitshur@golang.org>
Trust: Carlos Amedee <carlos@golang.org>
Reviewed-by: Carlos Amedee <carlos@golang.org>
Run-TryBot: Carlos Amedee <carlos@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
For #22446
Change-Id: I71a30761a28e81c50b7089d5a28be99c736c2dc8
Reviewed-on: https://go-review.googlesource.com/c/go/+/341332
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Heschi Kreinick <heschi@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
The current document uses the "top" and "bottom" when talking
about the address within a frame, which may easily lead to
misunderstandings. This patch directly uses "high address"
and "low address" to make the expression clearer.
Change-Id: I7469330bbdc158672d7f0314fe6680ebdd9ab79a
Reviewed-on: https://go-review.googlesource.com/c/go/+/339369
Trust: fannie zhang <Fannie.Zhang@arm.com>
Reviewed-by: Austin Clements <austin@google.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Writing CL 333629 clarified my thinking about the behavioral changes
associated with lazy loading. There are really two interrelated
changes — graph pruning, and lazy loading proper — that are both made
possible by the added redundancy in the go.mod file.
(I had initially approached the whole cluster of features as “lazy
loading” because that was the starting point for the design. Graph
pruning came into the picture when we looked at how to bound the
worst-case behavior of lazy loading, but it is really the more
important of the two aspects of the design.)
Note that this change adds links to doc anchors added in CL 333629.
Fixes#36460Fixes#47397
Change-Id: I0ef4af57f647bf5ee210ea7099191fb4befa2cc1
Reviewed-on: https://go-review.googlesource.com/c/go/+/335135
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
There is an example for nil slice already, so adding example for non-nil
zero length slice, too, clarifying to the reader that the result is also
non-nil and different from nil slice case.
Updates #395
Change-Id: I019db1b1a1c0c621161ecaaacab5a4d888764b1a
Reviewed-on: https://go-review.googlesource.com/c/go/+/336890
Trust: Cuong Manh Le <cuong.manhle.vn@gmail.com>
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Cuong Manh Le <cuong.manhle.vn@gmail.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Robert Griesemer <gri@golang.org>
Updates #44513
Change-Id: Ia0c6b48bde2719f3a99cb216b6166d82159198d1
Reviewed-on: https://go-review.googlesource.com/c/go/+/336930
Trust: Alberto Donizetti <alb.donizetti@gmail.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
For #395
For #46746
Change-Id: I4bfc094cf1cecd27ce48e31f92384cf470f371a6
Reviewed-on: https://go-review.googlesource.com/c/go/+/334669
Trust: Ian Lance Taylor <iant@golang.org>
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
Reviewed-by: Joe Tsai <thebrokentoaster@gmail.com>
For #46229
Change-Id: I54d01d90f2b0c892d76121f1350c0e8cf4b2772f
Reviewed-on: https://go-review.googlesource.com/c/go/+/334729
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
For #19367
Change-Id: If0ff8ddba3b6b48e2e198cf3653e73284c7572a3
Reviewed-on: https://go-review.googlesource.com/c/go/+/332409
Trust: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Robert Griesemer <gri@golang.org>
Reviewed-by: Keith Randall <khr@golang.org>
These types already implemented the Image interface. They should also
implement the RGBA64Image interface (new in Go 1.17)
Updates #44808
Change-Id: I9a2b13e305997088ae874efb95ad9e1648f94812
Reviewed-on: https://go-review.googlesource.com/c/go/+/331570
Trust: Nigel Tao <nigeltao@golang.org>
Run-TryBot: Nigel Tao <nigeltao@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Dmitri Shuralyov <dmitshur@golang.org>
Add unsafe.Add and unsafe.Slice to the list of built-in functions
which are not permitted in statement context. The compiler and
type checker already enforce this restriction, this just fixes
a documentation oversight.
For #19367.
For #40481.
Change-Id: Iabc63a8db048eaf40a5f5b5573fdf00b79d54119
Reviewed-on: https://go-review.googlesource.com/c/go/+/329925
Trust: Robert Griesemer <gri@golang.org>
Run-TryBot: Robert Griesemer <gri@golang.org>
Reviewed-by: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: Rob Pike <r@golang.org>
TryBot-Result: Go Bot <gobot@golang.org>
For #46366
Change-Id: I8417e6e4dbb8cb56ff7afc16893a01b7bb938217
Reviewed-on: https://go-review.googlesource.com/c/go/+/329529
Trust: Bryan C. Mills <bcmills@google.com>
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
