1
0
mirror of https://github.com/golang/go synced 2024-10-05 12:11:22 -06:00
Commit Graph

7 Commits

Author SHA1 Message Date
Marcel van Lohuizen
56a76c88f8 exp/locale/collate: from the regression test we derive that the spec
dictates a CJK rune is only part of a certain specified range if it
is explicitly defined in the Unicode Codepoint Database.
Fixed the code and some of the tests accordingly.

R=r
CC=golang-dev
https://golang.org/cl/6160044
2012-05-07 11:51:40 +02:00
Marcel van Lohuizen
10838165d8 exp/locale/collate: fixed two bugs uncovered by regression tests.
The first bug was that tertiary ignorables had the same colElem as
implicit colElems, yielding unexpected results. The current encoding
ensures that a non-implicit colElem is never 0.  This fix uncovered
another bug of the trie that indexed incorrectly into the null block.
This was caused by an unfinished optimization that would avoid the
need to max out the most-significant bits of continuation bytes.
This bug was also present in the trie used in exp/norm and has been
fixed there as well. The appearence of the bug was rare, as the lower
blocks happened to be nearly nil.

R=r
CC=golang-dev
https://golang.org/cl/6127070
2012-05-02 17:01:41 +02:00
Marcel van Lohuizen
fdce27f7b8 exp/locale/collate: Added Builder type for generating a complete
collation table. At this moment, it only implements the generation of
a root table.

R=r
CC=golang-dev
https://golang.org/cl/6039047
2012-04-25 13:19:35 +02:00
Marcel van Lohuizen
52f0afe0db exp/locale/collate: Added skeleton for the higher-level types to provide
context for change lists of lower-level types. The public APIs are defined
in builder.go and collate.go. Type table is the glue between the lower and
higher level code and might be a good starting point for understanding the
collation code.

R=r, r
CC=golang-dev
https://golang.org/cl/5999053
2012-04-25 13:19:00 +02:00
Marcel van Lohuizen
bcf48c7971 exp/locale/collate: added trie for associating colElems to runes.
The trie code looks a lot like the trie in exp/norm. It uses different
types, however.  Also, there is only a lookup for []byte and the unsafe
lookup methods have been dropped, as well as sparse mode.
There is now a method for generating a trie. To output Go code, one now needs
to first generate a trie and then call print() on it.

R=r, r, mpvl
CC=golang-dev
https://golang.org/cl/5966064
2012-04-25 13:16:57 +02:00
Marcel van Lohuizen
bb3f3c9775 exp/locale/collate: added representation for collation elements
(see http://www.unicode.org/reports/tr10/).

R=r, r
CC=golang-dev
https://golang.org/cl/5981048
2012-04-25 13:16:24 +02:00
Marcel van Lohuizen
e456d015fb exp/locale/collate: implementation of trie that is used for detecting contractions.
(See http://www.unicode.org/reports/tr10/#Contractions.)  Each rune that is at the
start of any contraction is associated a trie. This trie, in turn, may be shared
by other runes that have the same set of suffixes.

R=r, r
CC=golang-dev
https://golang.org/cl/5970066
2012-04-25 13:15:48 +02:00