1
0
mirror of https://github.com/golang/go synced 2024-09-29 14:24:32 -06:00
go/doc/go1.18.html

578 lines
23 KiB
HTML
Raw Normal View History

<!--{
"Title": "Go 1.18 Release Notes",
"Path": "/doc/go1.18"
}-->
<!--
NOTE: In this document and others in this directory, the convention is to
set fixed-width phrases with non-fixed-width spaces, as in
<code>hello</code> <code>world</code>.
Do not send CLs removing the interior tags from such phrases.
-->
<style>
main ul li { margin: 0.5em 0; }
</style>
<h2 id="introduction">DRAFT RELEASE NOTES — Introduction to Go 1.18</h2>
<p>
<strong>
Go 1.18 is not yet released. These are work-in-progress
release notes. Go 1.18 is expected to be released in February 2022.
</strong>
</p>
<h2 id="language">Changes to the language</h2>
<p>
TODO: complete this section
</p>
<h3 id="bug_fixes">Bug fixes</h3>
<p>
The Go 1.18 compiler now correctly reports <code>declared but not used</code> errors
for variables that are set inside a function literal but are never used. Before Go 1.18,
the compiler did not report an error in such cases. This fixes long-outstanding compiler
issue <a href="https://golang.org/issue/8560">#8560</a>. As a result of this change,
(possibly incorrect) programs may not compile anymore. The necessary fix is
straightforward: fix the program if it was in fact incorrect, or use the offending
variable, for instance by assigning it to the blank identifier <code>_</code>.
Since <code>go vet</code> always pointed out this error, the number of affected
programs is likely very small.
</p>
<p>
The Go 1.18 compiler now reports an overflow when passing a rune constant expression
such as <code>'1' << 32</code> as an argument to the predeclared functions
<code>print</code> and <code>println</code>, consistent with the behavior of
user-defined functions. Before Go 1.18, the compiler did not report an error
in such cases but silently accepted such constant arguments if they fit into an
<code>int64</code>. As a result of this change, (possibly incorrect) programs
may not compile anymore. The necessary fix is straightforward: fix the program if it
was in fact incorrect, or explicitly convert the offending argument to the correct type.
Since <code>go vet</code> always pointed out this error, the number of affected
programs is likely very small.
</p>
<h3 id="generics">Generics</h3>
<p>
Go 1.18 includes an implementation of generics as described
by <a href="https://go.googlesource.com/proposal/+/refs/heads/master/design/43651-type-parameters.md">the
proposal</a>.
</p>
<p>
The current generics implementation has the following limitations:
<ul>
<li><!-- issue 47631 -->
The Go compiler cannot currently handle type declarations inside generic functions
or methods. We hope to provide support for this feature in Go 1.19.
</li>
</ul>
</p>
<h2 id="ports">Ports</h2>
<h3 id="freebsd">FreeBSD</h3>
<p>
Go 1.18 is the last release that is supported on FreeBSD 11.x, which has
already reached end-of-life. Go 1.19 will require FreeBSD 12.2+ or FreeBSD
13.0+.
FreeBSD 13.0+ will require a kernel with the COMPAT_FREEBSD12 option set (this is the default).
</p>
<h3 id="amd64">AMD64</h3>
<p><!-- CL 349595 -->
Go 1.18 introduces the new <code>GOAMD64</code> environment variable which selects
a version of the AMD64 architecture. Allowed values are <code>v1</code>,
<code>v2</code>, <code>v3</code>, or <code>v4</code>. Each higher level requires,
and takes advantage of, additional processor features. A detailed description of the
versions is <a href="https://en.wikipedia.org/wiki/X86-64#Microarchitecture_levels">here</a>.
</p>
<p>
The <code>GOAMD64</code> environment variable defaults to <code>v1</code>.
</p>
<h3 id="riscv">RISC-V</h3>
<p><!-- golang.org/issue/47100, CL 334872 -->
The 64-bit RISC-V architecture on Linux (the <code>linux/riscv64</code> port)
now supports the <code>c-archive</code> and <code>c-shared</code> build modes.
</p>
<h3 id="ios">iOS</h3>
<p><!-- golang.org/issue/48076, golang.org/issue/49616 -->
On iOS (the <code>ios/arm64</code> port)
and iOS simulator running on AMD64-based macOS (the <code>ios/amd64</code> port),
Go 1.18 now requires iOS 12 or later; support for previous versions has been discontinued.
</p>
<h2 id="tools">Tools</h2>
<h3 id="go-command">Go command</h3>
<p><!-- golang.org/issue/43684 -->
<code>go</code> <code>get</code> no longer builds or installs packages in
module-aware mode. <code>go</code> <code>get</code> is now dedicated to
adjusting dependencies in <code>go.mod</code>. Effectively, the
<code>-d</code> flag is always enabled. To install the latest version
of an executable outside the context of the current module, use
<a href="https://golang.org/ref/mod#go-install"><code>go</code>
<code>install</code> <code>example.com/cmd@latest</code></a>. Any
<a href="https://golang.org/ref/mod#version-queries">version query</a>
may be used instead of <code>latest</code>. This form of <code>go</code>
<code>install</code> was added in Go 1.16, so projects supporting older
versions may need to provide install instructions for both <code>go</code>
<code>install</code> and <code>go</code> <code>get</code>. <code>go</code>
<code>get</code> now reports an error when used outside a module, since there
is no <code>go.mod</code> file to update. In GOPATH mode (with
<code>GO111MODULE=off</code>), <code>go</code> <code>get</code> still builds
and installs packages, as before.
</p>
<p><!-- golang.org/issue/37475 -->
The <code>go</code> command now embeds version control information in
binaries including the currently checked-out revision, commit time, and a
flag indicating whether edited or untracked files are present. Version
control information is embedded if the <code>go</code> command is invoked in
a directory within a Git, Mercurial, Fossil, or Bazaar repository, and the
<code>main</code> package and its containing main module are in the same
repository. This information may be omitted using the flag
<code>-buildvcs=false</code>.
</p>
<p><!-- golang.org/issue/37475 -->
Additionally, the <code>go</code> command embeds information about the build
including build and tool tags (set with <code>-tags</code>), compiler,
assembler, and linker flags (like <code>-gcflags</code>), whether cgo was
enabled, and if it was, the values of the cgo environment variables
(like <code>CGO_CFLAGS</code>). This information may be omitted using the
flag <code>-buildinfo=false</code>. Both VCS and build information may be
read together with module information using <code>go</code>
<code>version</code> <code>-m</code> <code>file</code> or
<code>runtime/debug.ReadBuildInfo</code> (for the currently running binary)
or the new <a href="#debug/buildinfo"><code>debug/buildinfo</code></a>
package.
</p>
<p><!-- https://golang.org/issue/44435 -->
If the main module's <code>go.mod</code> file
specifies <a href="/ref/mod#go-mod-file-go"><code>go</code> <code>1.17</code></a>
or higher, <code>go</code> <code>mod</code> <code>download</code> without
arguments now downloads source code for only the modules
explicitly <a href="/ref/mod#go-mod-file-require">required</a> in the main
module's <code>go.mod</code> file. (In a <code>go</code> <code>1.17</code> or
higher module, that set already includes all dependencies needed to build the
packages and tests in the main module.)
To also download source code for transitive dependencies, use
<code>go</code> <code>mod</code> <code>download</code> <code>all</code>.
</p>
<p><!-- https://golang.org/issue/47327 -->
The <code>go</code> <code>mod</code> <code>vendor</code> subcommand now
supports a <code>-o</code> flag to set the output directory.
(Other <code>go</code> commands still read from the <code>vendor</code>
directory at the module root when loading packages
with <code>-mod=vendor</code>, so the main use for this flag is for
third-party tools that need to collect package source code.)
</p>
<p><!-- CL 298612 -->
The <code>go</code> <code>build</code> command and related commands
now support an <code>-asan</code> flag that enables interoperation
with C (or C++) code compiled with the address sanitizer (C compiler
option <code>-fsanitize=address</code>).
</p>
cmd/gofmt: format files in parallel gofmt is pretty heavily CPU-bound, since parsing and formatting 1MiB of Go code takes much longer than reading that amount of bytes from disk. However, parsing and manipulating a large Go source file is very difficult to parallelize, so we continue to process each file in its own goroutine. A Go module may contain a large number of Go source files, so we need to bound the amount of work in flight. However, because the distribution of sizes for Go source files varies widely — from tiny doc.go files containing a single package comment all the way up to massive API wrappers generated by automated tools — the amount of time, work, and memory overhead needed to process each file also varies. To account for this variability, we limit the in-flight work by bytes of input rather than by number of files. That allows us to make progress on many small files while we wait for work on a handful of large files to complete. The gofmt tool has a well-defined output format on stdout, which was previously deterministic. We keep it deterministic by printing the results of each file in order, using a lazily-synchronized io.Writer (loosly inspired by Haskell's IO monad). After a file has been formatted in memory, we keep it in memory (again, limited by the corresponding number of input bytes) until the output for all previous files has been flushed. This adds a bit of latency compared to emitting the output in nondeterministic order, but a little extra latency seems worth the cost to preserve output stability. This change is based on Daniel Martí's work in CL 284139, but using a weighted semaphore and ephemeral goroutines instead of a worker pool and batches. Benchmark results are similar, and I find the concurrency in this approach a bit easier to reason about. In the batching-based approach, the batch size allows us to "look ahead" to find large files and start processing them early. To keep the CPUs saturated and prevent stragglers, we would need to tune the batch size to be about the same as the largest input files. If the batch size is set too high, a large batch of small files could turn into a straggler, but if the batch size is set too low, the largest files in the data set won't be started early enough and we'll end up with a large-file straggler. One possible alternative would be to sort by file size instead of batching: identify all of the files to be processed, sort from largest to smallest, and then process the largest files first so that the "tail" of processing covers the smallest files. However, that approach would still fail to saturate available CPU when disk latency is high, would require buffering an arbitrary amount of metadata in order to sort by size, and (perhaps most importantly!) would not allow the `gofmt` binary to preserve the same (deterministic) output order that it has today. In contrast, with a semaphore we can produce the same deterministic output as ever using only one tuning parameter: the memory footprint, expressed as a rough lower bound on the amount of RAM available per thread. While we're below the memory limit, we can run arbitrarily many disk operations arbitrarily far ahead, and process the results of those operations whenever they become avaliable. Then it's up to the kernel (not us) to schedule the disk operations for throughput and latency, and it's up to the runtime (not us) to schedule the goroutines so that they complete quickly. In practice, even a modest assumption of a few megabytes per thread seems to provide a nice speedup, and it should scale reasonably even to machines with vastly different ratios of CPU to disk. (In practice, I expect that most 'gofmt' invocations will work with files on at most one physical disk, so the CPU:disk ratio should vary more-or-less directly with the thread count, whereas the CPU:memory ratio is more-or-less independent of thread count.) name \ time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 11.9s ± 2% 2.7s ± 3% 2.8s ± 5% name \ user-time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 13.5s ± 2% 14.4s ± 1% 14.7s ± 1% name \ sys-time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 465ms ± 8% 229ms ±28% 232ms ±31% name \ peak-RSS-bytes baseline.txt 284139.txt simplified.txt GofmtGorootCmd 77.7MB ± 4% 162.2MB ±10% 192.9MB ±15% For #43566 Change-Id: I4ba251eb4d188a3bd1901039086be57f0b341910 Reviewed-on: https://go-review.googlesource.com/c/go/+/317975 Trust: Bryan C. Mills <bcmills@google.com> Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2021-01-16 11:03:31 -07:00
<h3 id="gofmt"><code>gofmt</code></h3>
<p><!-- https://golang.org/issue/43566 -->
<code>gofmt</code> now reads and formats input files concurrently, with a
memory limit proportional to <code>GOMAXPROCS</code>. On a machine with
multiple CPUs, <code>gofmt</code> should now be significantly faster.
cmd/gofmt: format files in parallel gofmt is pretty heavily CPU-bound, since parsing and formatting 1MiB of Go code takes much longer than reading that amount of bytes from disk. However, parsing and manipulating a large Go source file is very difficult to parallelize, so we continue to process each file in its own goroutine. A Go module may contain a large number of Go source files, so we need to bound the amount of work in flight. However, because the distribution of sizes for Go source files varies widely — from tiny doc.go files containing a single package comment all the way up to massive API wrappers generated by automated tools — the amount of time, work, and memory overhead needed to process each file also varies. To account for this variability, we limit the in-flight work by bytes of input rather than by number of files. That allows us to make progress on many small files while we wait for work on a handful of large files to complete. The gofmt tool has a well-defined output format on stdout, which was previously deterministic. We keep it deterministic by printing the results of each file in order, using a lazily-synchronized io.Writer (loosly inspired by Haskell's IO monad). After a file has been formatted in memory, we keep it in memory (again, limited by the corresponding number of input bytes) until the output for all previous files has been flushed. This adds a bit of latency compared to emitting the output in nondeterministic order, but a little extra latency seems worth the cost to preserve output stability. This change is based on Daniel Martí's work in CL 284139, but using a weighted semaphore and ephemeral goroutines instead of a worker pool and batches. Benchmark results are similar, and I find the concurrency in this approach a bit easier to reason about. In the batching-based approach, the batch size allows us to "look ahead" to find large files and start processing them early. To keep the CPUs saturated and prevent stragglers, we would need to tune the batch size to be about the same as the largest input files. If the batch size is set too high, a large batch of small files could turn into a straggler, but if the batch size is set too low, the largest files in the data set won't be started early enough and we'll end up with a large-file straggler. One possible alternative would be to sort by file size instead of batching: identify all of the files to be processed, sort from largest to smallest, and then process the largest files first so that the "tail" of processing covers the smallest files. However, that approach would still fail to saturate available CPU when disk latency is high, would require buffering an arbitrary amount of metadata in order to sort by size, and (perhaps most importantly!) would not allow the `gofmt` binary to preserve the same (deterministic) output order that it has today. In contrast, with a semaphore we can produce the same deterministic output as ever using only one tuning parameter: the memory footprint, expressed as a rough lower bound on the amount of RAM available per thread. While we're below the memory limit, we can run arbitrarily many disk operations arbitrarily far ahead, and process the results of those operations whenever they become avaliable. Then it's up to the kernel (not us) to schedule the disk operations for throughput and latency, and it's up to the runtime (not us) to schedule the goroutines so that they complete quickly. In practice, even a modest assumption of a few megabytes per thread seems to provide a nice speedup, and it should scale reasonably even to machines with vastly different ratios of CPU to disk. (In practice, I expect that most 'gofmt' invocations will work with files on at most one physical disk, so the CPU:disk ratio should vary more-or-less directly with the thread count, whereas the CPU:memory ratio is more-or-less independent of thread count.) name \ time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 11.9s ± 2% 2.7s ± 3% 2.8s ± 5% name \ user-time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 13.5s ± 2% 14.4s ± 1% 14.7s ± 1% name \ sys-time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 465ms ± 8% 229ms ±28% 232ms ±31% name \ peak-RSS-bytes baseline.txt 284139.txt simplified.txt GofmtGorootCmd 77.7MB ± 4% 162.2MB ±10% 192.9MB ±15% For #43566 Change-Id: I4ba251eb4d188a3bd1901039086be57f0b341910 Reviewed-on: https://go-review.googlesource.com/c/go/+/317975 Trust: Bryan C. Mills <bcmills@google.com> Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2021-01-16 11:03:31 -07:00
</p>
<h2 id="runtime">Runtime</h2>
<p><!-- https://golang.org/issue/49759 -->
The <code>windows/arm</code> and <code>windows/arm64</code> ports now support
non-cooperative preemption, bringing that capability to all four Windows
ports, which should hopefully address subtle bugs encountered when calling
into Win32 functions that block for extended periods of time.
</p>
<p><!-- https://github.com/golang/go/issues/44167-->
The garbage collector now includes non-heap sources of garbage collector work
(e.g., stack scanning) when determining how frequently to run. As a result,
garbage collector overhead is more predictable when these sources are
significant. For most applications these changes will be negligible; however,
some Go applications may now use less memory and spend more time on garbage
collection, or vice versa, than before. The intended workaround is to tweak
<code>GOGC</code> where necessary.
</p>
<p>
The runtime now returns memory to the operating system more efficiently and has
been tuned to work more aggressively as a result.
</p>
<h2 id="compiler">Compiler</h2>
<p><!-- https://golang.org/issue/40724 -->
Go 1.17 <a href=go1.17#compiler>implemented</a> a new way of passing
function arguments and results using registers instead of the stack
on 64-bit x86 architecture on selected operating systems.
Go 1.18 expands the supported platforms to include 64-bit ARM (<code>GOARCH=arm64</code>),
big- and little-endian 64-bit PowerPC (<code>GOARCH=ppc64</code>, <code>ppc64le</code>),
as well as 64-bit x86 architecture (<code>GOARCH=amd64</code>)
on all operating systems.
On 64-bit ARM and 64-bit PowerPC systems, benchmarking shows
performance improvements of 10% or more.
</p>
<p>
As <a href=go1.17#compiler>mentioned</a> in the Go 1.17 release notes,
this change does not affect the functionality of any safe Go code and
is designed to have no impact on most assembly code. See the
<a href=go1.17#compiler>Go 1.17 release notes</a> for more details.
</p>
<p><!-- CL 352057, https://golang.org/issue/45728 -->
Go 1.17 generally improved the formatting of arguments in stack traces,
but could print inaccurate values for arguments passed in registers.
This is improved in Go 1.18 by printing a question mark (<code>?</code>)
after each value that may be inaccurate.
</p>
<p><!-- CL 298611 -->
The new compiler <code>-asan</code> option supports the
new <code>go</code> command <code>-asan</code> option.
</p>
<h2 id="linker">Linker</h2>
<p><!-- CL 298610 -->
The new linker <code>-asan</code> option supports the
new <code>go</code> command <code>-asan</code> option.
</p>
<h2 id="library">Core library</h2>
<h3 id="constraints">New <code>constraints</code> package</h3>
<p><!-- CL 349709 -->
The new <a href="/pkg/constraints/"><code>constraints</code></a> package
defines a set of useful constraints that can be used with type parameters of
generic functions.
</p>
<h3 id="netip">New <code>net/netip</code> package</h3>
<p>
The new <a href="/pkg/net/netip/"><code>net/netip</code></a>
package defines a new IP address type, <a href="/pkg/net/netip/#Addr"><code>Addr</code></a>.
Compared to the existing
<a href="/pkg/net/#IP"><code>net.IP</code></a> type, the <code>netip.Addr</code> type takes less
memory, is immutable, and is comparable so it supports <code>==</code>
and can be used as a map key.
</p>
<p>
In addition to <code>Addr</code>, the package defines
<a href="/pkg/net/netip/#AddrPort"><code>AddrPort</code></a>, representing
an IP and port, and
<a href="/pkg/net/netip/#Prefix"><code>Prefix</code></a>, representing
a network CIDR prefix.
</p>
<p>
The <code>net</code> package now has methods to send and receive UDP packets
using <code>netip.Addr</code> values instead of the relatively heavy
<code>*net.UDPAddr</code> values.
</p>
<h3>TODO</h3>
<p>
TODO: complete this section
</p>
<h3 id="minor_library_changes">Minor changes to the library</h3>
<p>
As always, there are various minor changes and updates to the library,
made with the Go 1 <a href="/doc/go1compat">promise of compatibility</a>
in mind.
</p>
<p>
TODO: complete this section
</p>
<dl id="bufio"><dt><a href="/pkg/bufio/">bufio</a></dt>
<dd>
<p><!-- CL 345569 -->
The new <a href="/pkg/bufio#Writer.AvailableBuffer"><code>Writer.AvailableBuffer</code></a>
method returns an empty buffer with a possibly non-empty capacity for use
with append-like APIs. After appending, the buffer can be provided to a
succeeding <code>Write</code> call and possibly avoid any copying.
</p>
<p><!-- CL 345570 -->
The methods <a href="/pkg/bufio#Reader.Reset"><code>Reader.Reset</code></a> and
<a href="/pkg/bufio#Writer.Reset"><code>Writer.Reset</code></a>
now use the default buffer size when called on objects with a
<code>nil</code> buffer.
</p>
</dd>
</dl><!-- bufio -->
<dl id="bytes"><dt><a href="/pkg/bytes/">bytes</a></dt>
<dd>
<p><!-- CL 323318, CL 332771 -->
<a href="/pkg/bytes/#Trim"><code>Trim</code></a>, <a href="/pkg/bytes/#TrimLeft"><code>TrimLeft</code></a>,
and <a href="/pkg/bytes/#TrimRight"><code>TrimRight</code></a> are now allocation free and, especially for
small ASCII cutsets, up to 10 times faster.
</p>
<p><!-- CL 359485 -->
The <a href="/pkg/bytes/#Title"><code>Title</code></a> function is now deprecated. It doesn't
handle Unicode punctuation and language-specific capitalization rules, and is superseded by the
<a href="https://golang.org/x/text/cases">golang.org/x/text/cases</a> package.
</p>
</dd>
</dl><!-- bytes -->
<dl id="crypto/tls"><dt><a href="/pkg/crypto/tls/">crypto/tls</a></dt>
<dd>
<p><!-- CL 325250 -->
The new <a href="/pkg/crypto/tls/#Conn.NetConn"><code>Conn.NetConn</code></a>
method allows access to the underlying
<a href="/pkg/net#Conn"><code>net.Conn</code></a>.
</p>
</dd>
</dl><!-- crypto/tls -->
<dl id="debug/buildinfo"><dt><a href="/pkg/debug/buildinfo">debug/buildinfo</a></dt>
<dd>
<p><!-- golang.org/issue/39301 -->
This new package provides access to module versions, version control
information, and build flags embedded in executable files built by
the <code>go</code> command. The same information is also available via
<a href="/pkg/runtime/debug#ReadBuildInfo"><code>runtime/debug.ReadBuildInfo</code></a>
for the currently running binary and via <code>go</code>
<code>version</code> <code>-m</code> on the command line.
</p>
</dd>
</dl>
<dl id="image/draw"><dt><a href="/pkg/image/draw/">image/draw</a></dt>
<dd>
<p><!-- CL 340049 -->
The <code>Draw</code> and <code>DrawMask</code> fallback implementations
(used when the arguments are not the most common image types) are now
faster when those arguments implement the optional
<a href="/pkg/image/draw/#RGBA64Image"><code>draw.RGBA64Image</code></a>
and <a href="/pkg/image/#RGBA64Image"><code>image.RGBA64Image</code></a>
interfaces that were added in Go 1.17.
</p>
</dd>
</dl><!-- image/draw -->
<dl id="net"><dt><a href="/pkg/net/">net</a></dt>
<dd>
<p><!-- CL 340261 -->
<a href="/pkg/net#Error"><code>net.Error.Temporary</code></a> has been deprecated.
</p>
</dd>
</dl><!-- net -->
<dl id="net/http"><dt><a href="/pkg/net/http/">net/http</a></dt>
<dd>
<p><!-- CL 330852 -->
On WebAssembly targets, the <code>Dial</code>, <code>DialContext</code>,
<code>DialTLS</code> and <code>DialTLSContext</code> method fields in
<a href="/pkg/net/http/#Transport"><code>Transport</code></a>
will now be correctly used, if specified, for making HTTP requests.
</p>
<p><!-- CL 338590 -->
The new
<a href="/pkg/net/http#Cookie.Valid"><code>Cookie.Valid</code></a>
method reports whether the cookie is valid.
</p>
</dd>
</dl><!-- net/http -->
<dl id="os/user"><dt><a href="/pkg/os/user/">os/user</a></dt>
<dd>
<p><!-- CL 330753 -->
<a href="/pkg/os/user#User.GroupIds"><code>User.GroupIds</code></a>.
now uses a Go native implementation when cgo is not available.
</p>
</dd>
</dl><!-- os/user -->
<dl id="reflect"><dt><a href="/pkg/reflect/">reflect</a></dt>
<dd>
<p><!-- CL 356049, CL 320929 -->
The new
<a href="/pkg/reflect/#Value.SetIterKey"><code>Value.SetIterKey</code></a>
and <a href="/pkg/reflect/#Value.SetIterValue"><code>Value.SetIterValue</code></a>
methods set a Value using a map iterator as the source. They are equivalent to
<code>Value.Set(iter.Key())</code> and <code>Value.Set(iter.Value())</code> but
do fewer allocations.
</p>
<p><!-- CL 350691 -->
The new
<a href="/pkg/reflect/#Value.UnsafePointer"><code>Value.UnsafePointer</code></a>
method returns the Value's value as an <a href="/pkg/unsafe/#Pointer"><code>unsafe.Pointer</code></a>.
This allows callers to migrate from <a href="/pkg/reflect/#Value.UnsafeAddr"><code>Value.UnsafeAddr</code></a>
and <a href="/pkg/reflect/#Value.Pointer"><code>Value.Pointer</code></a>
to eliminate the need to perform uintptr to unsafe.Pointer conversions at the callsite (as unsafe.Pointer rules require).
</p>
<p><!-- CL 321891 -->
The new
<a href="/pkg/reflect/#MapIter.Reset"><code>MapIter.Reset</code></a>
method changes its receiver to iterate over a
different map. The use of
<a href="/pkg/reflect/#MapIter.Reset"><code>MapIter.Reset</code></a>
allows allocation-free iteration
over many maps.
</p>
<p><!-- CL 352131 -->
A number of methods (
<a href="/pkg/reflect#Value.CanInt"><code>Value.CanInt</code></a>,
<a href="/pkg/reflect#Value.CanUint"><code>Value.CanUint</code></a>,
<a href="/pkg/reflect#Value.CanFloat"><code>Value.CanFloat</code></a>,
<a href="/pkg/reflect#Value.CanComplex"><code>Value.CanComplex</code></a>
)
have been added to
<a href="/pkg/reflect#Value"><code>Value</code></a>
to test if a conversion is safe.
</p>
<p><!-- CL 357962 -->
<a href="/pkg/reflect#Value.FieldByIndexErr"><code>Value.FieldByIndexErr</code></a>
has been added to avoid the panic that occurs in
<a href="/pkg/reflect#Value.FieldByIndex"><code>Value.FieldByIndex</code></a>
when stepping through a nil pointer to an embedded struct.
</p>
</dd>
</dl><!-- reflect -->
<dl id="regexp"><dt><a href="/pkg/regexp/">regexp</a></dt>
<dd>
<p><!-- CL 354569 -->
TODO: <a href="https://golang.org/cl/354569">https://golang.org/cl/354569</a>: document and implement that invalid UTF-8 bytes are the same as U+FFFD
</p>
</dd>
</dl><!-- regexp -->
<dl id="strconv"><dt><a href="/pkg/strconv/">strconv</a></dt>
<dd>
<p><!-- CL 343877 -->
TODO: <a href="https://golang.org/cl/343877">https://golang.org/cl/343877</a>: reject surrogate halves in Unquote
</p>
</dd>
</dl><!-- strconv -->
<dl id="strings"><dt><a href="/pkg/strings/">strings</a></dt>
<dd>
<p><!-- CL 345849 -->
The new <a href="/pkg/strings/#Clone"><code>Clone</code></a> function copies the input
<code>string</code> without the returned cloned <code>string</code> referencing
the input string's memory.
</p>
<p><!-- CL 323318, CL 332771 -->
<a href="/pkg/strings/#Trim"><code>Trim</code></a>, <a href="/pkg/strings/#TrimLeft"><code>TrimLeft</code></a>,
and <a href="/pkg/strings/#TrimRight"><code>TrimRight</code></a> are now allocation free and, especially for
small ASCII cutsets, up to 10 times faster.
</p>
<p><!-- CL 359485 -->
The <a href="/pkg/strings/#Title"><code>Title</code></a> function is now deprecated. It doesn't
handle Unicode punctuation and language-specific capitalization rules, and is superseded by the
<a href="https://golang.org/x/text/cases">golang.org/x/text/cases</a> package.
</p>
</dd>
</dl><!-- strings -->
<dl id="sync"><dt><a href="/pkg/sync/">sync</a></dt>
<dd>
<p><!-- CL 319769 -->
TODO: <a href="https://golang.org/cl/319769">https://golang.org/cl/319769</a>: add Mutex.TryLock, RWMutex.TryLock, RWMutex.TryRLock
</p>
</dd>
</dl><!-- sync -->
<dl id="syscall"><dt><a href="/pkg/syscall/">syscall</a></dt>
<dd>
<p><!-- CL 336550 -->
The new function <a href="/pkg/syscall/?GOOS=windows#SyscallN"><code>SyscallN</code></a>
has been introduced for Windows, allowing for calls with arbitrary number
of arguments. As a result,
<a href="/pkg/syscall/?GOOS=windows#Syscall"><code>Syscall</code></a>,
<a href="/pkg/syscall/?GOOS=windows#Syscall6"><code>Syscall6</code></a>,
<a href="/pkg/syscall/?GOOS=windows#Syscall9"><code>Syscall9</code></a>,
<a href="/pkg/syscall/?GOOS=windows#Syscall12"><code>Syscall12</code></a>,
<a href="/pkg/syscall/?GOOS=windows#Syscall15"><code>Syscall15</code></a>, and
<a href="/pkg/syscall/?GOOS=windows#Syscall18"><code>Syscall18</code></a> are
deprecated in favor of <a href="/pkg/syscall/?GOOS=windows#SyscallN"><code>SyscallN</code></a>.
</p>
<p><!-- CL 355570 -->
TODO: <a href="https://golang.org/cl/355570">https://golang.org/cl/355570</a>: add support for SysProcAttr.Pdeathsig on FreeBSD
</p>
</dd>
cmd/gofmt: format files in parallel gofmt is pretty heavily CPU-bound, since parsing and formatting 1MiB of Go code takes much longer than reading that amount of bytes from disk. However, parsing and manipulating a large Go source file is very difficult to parallelize, so we continue to process each file in its own goroutine. A Go module may contain a large number of Go source files, so we need to bound the amount of work in flight. However, because the distribution of sizes for Go source files varies widely — from tiny doc.go files containing a single package comment all the way up to massive API wrappers generated by automated tools — the amount of time, work, and memory overhead needed to process each file also varies. To account for this variability, we limit the in-flight work by bytes of input rather than by number of files. That allows us to make progress on many small files while we wait for work on a handful of large files to complete. The gofmt tool has a well-defined output format on stdout, which was previously deterministic. We keep it deterministic by printing the results of each file in order, using a lazily-synchronized io.Writer (loosly inspired by Haskell's IO monad). After a file has been formatted in memory, we keep it in memory (again, limited by the corresponding number of input bytes) until the output for all previous files has been flushed. This adds a bit of latency compared to emitting the output in nondeterministic order, but a little extra latency seems worth the cost to preserve output stability. This change is based on Daniel Martí's work in CL 284139, but using a weighted semaphore and ephemeral goroutines instead of a worker pool and batches. Benchmark results are similar, and I find the concurrency in this approach a bit easier to reason about. In the batching-based approach, the batch size allows us to "look ahead" to find large files and start processing them early. To keep the CPUs saturated and prevent stragglers, we would need to tune the batch size to be about the same as the largest input files. If the batch size is set too high, a large batch of small files could turn into a straggler, but if the batch size is set too low, the largest files in the data set won't be started early enough and we'll end up with a large-file straggler. One possible alternative would be to sort by file size instead of batching: identify all of the files to be processed, sort from largest to smallest, and then process the largest files first so that the "tail" of processing covers the smallest files. However, that approach would still fail to saturate available CPU when disk latency is high, would require buffering an arbitrary amount of metadata in order to sort by size, and (perhaps most importantly!) would not allow the `gofmt` binary to preserve the same (deterministic) output order that it has today. In contrast, with a semaphore we can produce the same deterministic output as ever using only one tuning parameter: the memory footprint, expressed as a rough lower bound on the amount of RAM available per thread. While we're below the memory limit, we can run arbitrarily many disk operations arbitrarily far ahead, and process the results of those operations whenever they become avaliable. Then it's up to the kernel (not us) to schedule the disk operations for throughput and latency, and it's up to the runtime (not us) to schedule the goroutines so that they complete quickly. In practice, even a modest assumption of a few megabytes per thread seems to provide a nice speedup, and it should scale reasonably even to machines with vastly different ratios of CPU to disk. (In practice, I expect that most 'gofmt' invocations will work with files on at most one physical disk, so the CPU:disk ratio should vary more-or-less directly with the thread count, whereas the CPU:memory ratio is more-or-less independent of thread count.) name \ time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 11.9s ± 2% 2.7s ± 3% 2.8s ± 5% name \ user-time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 13.5s ± 2% 14.4s ± 1% 14.7s ± 1% name \ sys-time/op baseline.txt 284139.txt simplified.txt GofmtGorootCmd 465ms ± 8% 229ms ±28% 232ms ±31% name \ peak-RSS-bytes baseline.txt 284139.txt simplified.txt GofmtGorootCmd 77.7MB ± 4% 162.2MB ±10% 192.9MB ±15% For #43566 Change-Id: I4ba251eb4d188a3bd1901039086be57f0b341910 Reviewed-on: https://go-review.googlesource.com/c/go/+/317975 Trust: Bryan C. Mills <bcmills@google.com> Trust: Daniel Martí <mvdan@mvdan.cc> Run-TryBot: Bryan C. Mills <bcmills@google.com> TryBot-Result: Go Bot <gobot@golang.org> Reviewed-by: Daniel Martí <mvdan@mvdan.cc>
2021-01-16 11:03:31 -07:00
</dl><!-- syscall -->
<dl id="syscall/js"><dt><a href="/pkg/syscall/js/">syscall/js</a></dt>
<dd>
<p><!-- CL 356430 -->
TODO: <a href="https://golang.org/cl/356430">https://golang.org/cl/356430</a>: remove Wrapper interface
</p>
</dd>
</dl><!-- syscall/js -->
<dl id="testing"><dt><a href="/pkg/testing/">testing</a></dt>
<dd>
<p><!-- CL 343883 -->
TODO: <a href="https://golang.org/cl/343883">https://golang.org/cl/343883</a>: increase alternation precedence
</p>
<p><!-- CL 356669 -->
TODO: <a href="https://golang.org/cl/356669">https://golang.org/cl/356669</a>: skip extra -count iterations if there are no tests
</p>
</dd>
</dl><!-- testing -->
<dl id="text/template"><dt><a href="/pkg/text/template/">text/template</a></dt>
<dd>
<p><!-- CL 321490 -->
The <code>and</code> function no longer always evaluates all arguments; it
stops evaluating arguments after the first argument that evaluates to
false. Similarly, the <code>or</code> function now stops evaluating
arguments after the first argument that evaluates to true. This makes a
difference if any of the arguments is a function call.
</p>
</dd>
</dl><!-- text/template -->
<dl id="unicode/utf8"><dt><a href="/pkg/unicode/utf8/">unicode/utf8</a></dt>
<dd>
<p><!-- CL 345571 -->
The <a href="/pkg/unicode/utf8/#AppendRune"><code>AppendRune</code></a> function appends the UTF-8 new
encoding of a <code>rune</code> to a <code>[]byte</code>.
</p>
</dd>
</dl><!-- unicode/utf8 -->