1
0
mirror of https://github.com/golang/go synced 2024-11-17 00:04:40 -07:00
The Go programming language
Go to file
Martin Möhrmann f74de24fbd strings: correctly handle invalid utf8 sequences in Map
When an invalid UTF-8 byte sequence is decoded in a range loop over a string
a utf8.RuneError rune is returned. This is not distinguishable from decoding
the valid '\uFFFD' sequence representing utf8.RuneError from a string without
further checks within the range loop.

The previous Map code did not do any extra checks and would thereby not map
invalid UTF-8 byte sequences correctly when those were mapping to utf8.RuneError.

Fix this by adding the extra checks necessary to distinguish the decoding
of invalid utf8 byte sequences from decoding the sequence for utf8.RuneError
when the mapping of a rune is utf8.RuneError.

This fix does not result in a measureable performance regression:
name                old time/op  new time/op  delta
ByteByteMap         1.05µs ± 3%  1.03µs ± 3%   ~     (p=0.118 n=10+10)
Map/identity/ASCII   169ns ± 2%   170ns ± 1%   ~     (p=0.501 n=9+10)
Map/identity/Greek   298ns ± 1%   303ns ± 4%   ~     (p=0.338 n=10+10)
Map/change/ASCII     323ns ± 3%   325ns ± 4%   ~     (p=0.679 n=8+10)
Map/change/Greek     628ns ± 5%   635ns ± 1%   ~     (p=0.460 n=10+9)
MapNoChanges         120ns ± 4%   119ns ± 1%   ~     (p=0.496 n=10+9)

Fixes #26305

Change-Id: I70e99fa244983c5040756fa4549ac1e8cb6022c3
Reviewed-on: https://go-review.googlesource.com/c/131495
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
2018-10-03 21:27:10 +00:00
.github .github: use HTTPS for Stack Overflow link in SUPPORT.md 2018-06-12 11:08:14 +00:00
api os: add ModeCharDevice to ModeType 2018-09-13 09:52:57 +00:00
doc doc: document Go 1.11.1 2018-10-01 20:32:50 +00:00
lib/time lib/time: update vendored tzdata to release 2018e 2018-06-11 14:47:06 +00:00
misc misc/wasm: add mention of polyfill for Edge support 2018-10-02 17:25:11 +00:00
src strings: correctly handle invalid utf8 sequences in Map 2018-10-03 21:27:10 +00:00
test test: stress test for stack objects 2018-10-03 19:54:29 +00:00
.gitattributes
.gitignore .gitignore: ignore src/cmd/dist/dist 2017-10-28 21:55:49 +00:00
AUTHORS A+C: add VMware as author, Venil Noronha as contributor 2018-08-31 02:14:24 +00:00
CONTRIBUTING.md all: restore changes from faulty merge/revert 2018-02-12 20:13:59 +00:00
CONTRIBUTORS A+C: add VMware as author, Venil Noronha as contributor 2018-08-31 02:14:24 +00:00
favicon.ico
LICENSE
PATENTS
README.md README: linkify some paths 2018-06-06 18:07:01 +00:00
robots.txt

The Go Programming Language

Go is an open source programming language that makes it easy to build simple, reliable, and efficient software.

Gopher image Gopher image by Renee French, licensed under Creative Commons 3.0 Attributions license.

Our canonical Git repository is located at https://go.googlesource.com/go. There is a mirror of the repository at https://github.com/golang/go.

Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.

Download and Install

Binary Distributions

Official binary distributions are available at https://golang.org/dl/.

After downloading a binary release, visit https://golang.org/doc/install or load doc/install.html in your web browser for installation instructions.

Install From Source

If a binary distribution is not available for your combination of operating system and architecture, visit https://golang.org/doc/install/source or load doc/install-source.html in your web browser for source installation instructions.

Contributing

Go is the work of thousands of contributors. We appreciate your help!

To contribute, please read the contribution guidelines: https://golang.org/doc/contribute.html

Note that the Go project uses the issue tracker for bug reports and proposals only. See https://golang.org/wiki/Questions for a list of places to ask questions about the Go language.