This extends cmd/yacc with support for
%error { tokens } : message
syntax to specify custom error messages to use instead of the default
generic ones. This allows merging go.errors into go.y and removing
the yaccerrors.go tool.
Updates #9968.
Change-Id: I781219c568b86472755f877f48401eaeab00ead5
Reviewed-on: https://go-review.googlesource.com/8563
Reviewed-by: Russ Cox <rsc@golang.org>
Bison includes suggestions about what tokens are expected in the
current state when there's only four or fewer of them. For example:
syntax error: unexpected literal 2.01, expecting semicolon or newline or }
This CL adds the same functionality to cmd/yacc, which fully restores
the previous error message behavior from Go 1.4.
Updates #9968.
Change-Id: I2c1a1677c6d829a829d812c05e8813aa8829d09c
Reviewed-on: https://go-review.googlesource.com/8494
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
Previously, a production rule like
A: B C D
would cause yacc to check that A and B have the same declared types,
but then it would generate an implicit action of { $$ = $3 } (i.e.,
copy the value from D), even if A and D have different types.
Fixes#10192.
Change-Id: I51cfd7baa0011557141dca33b7af1d892cc6f49e
Reviewed-on: https://go-review.googlesource.com/7780
Reviewed-by: Russ Cox <rsc@golang.org>
This CL extends cmd/yacc to expose a yyErrorVerbose variable that
changes the error messages from just "syntax error" to "syntax error:
unexpected ${tokname}".
It also moves the yyToknames table generation to after rules have been
processed so that entries can be generated for tokens that aren't
mentioned in the preamble (e.g., '.' in the case of go.y).
Lastly, it restores gc's old code for applying yytfix to yyToknames,
except that substituting "LLITERAL" with litbuf happens in Yyerror.
Fixes#9968.
Change-Id: Icec188d11fdabc1dae31b8a471c35b5c7f6deec7
Reviewed-on: https://go-review.googlesource.com/8432
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
This restores go.errors from before 3af0d79 along with a fixed up
version of the bisonerrors AWK script, translated to Go.
However, this means Yyerror needs access to the yacc parser's state,
which is currently private. To workaround that, add a "state"
accessor method like the Lookahead method added in c7fa3c6.
Update issue #9968.
Change-Id: Ib868789e92fdb7d135442120a392457923e50121
Reviewed-on: https://go-review.googlesource.com/7270
Reviewed-by: Russ Cox <rsc@golang.org>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Yacc generates a bunch of global variables of the form
var yyFoo = []int{...}
where yyFoo is never subsequently modified to point to a different
slice. Since these variables are implicitly compiled as
var yyFoo = ([...]int{...})[:]
anyway, by simply converting them all to
var yyFoo = [...]int{...}
we save sizeof(sliceStruct) bytes of data memory for each variable and
also make len(yyFoo) into compile-time constant expressions, which
shaves some bytes off text size:
$ size 6g.before 6g.after
text data bss dec hex filename
4598019 605968 342700 5546687 54a2bf 6g.before
4597810 605552 342700 5546062 54a04e 6g.after
Change-Id: I53c7aa6efdb2d52738013e9d337a59afbfcb2494
Reviewed-on: https://go-review.googlesource.com/7520
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Dave Cheney <dave@cheney.net>
Reviewed-by: Josh Bleecher Snyder <josharian@gmail.com>
Reviewed-by: Russ Cox <rsc@golang.org>
Historically, yacc has supported various kinds of inspections
and manipulations of the parser state, exposed as global variables.
The Go implementation of yacc puts that state (properly) in local
stack variables, so it can only be exposed explicitly.
There is now an explicit parser type, yyParser, returned by a
constructor, yyNewParser.
type yyParser interface {
Parse(yyLexer) int
Lookahead() int
}
Parse runs a parse. A call to the top-level func Parse
is equivalent to calling yyNewParser().Parse, but constructing
the parser explicitly makes it possible to access additional
parser methods, such as Lookahead.
Lookahead can be called during grammar actions to read
(but not consume) the value of the current lookahead token,
as returned by yylex.Lex. If there is no current lookahead token,
Lookahead returns -1. Invoking Lookahead corresponds to
reading the global variable yychar in a traditional Unix yacc grammar.
To support Lookahead, the internal parsing code now separates
the return value from Lex (yychar) from the reencoding used
by the parsing tables (yytoken). This has the effect that grammars
that read yychar directly in the action (possible since the actions
are in the same function that declares yychar) now correctly see values
from the Lex return value space, not the internal reencoding space.
This can fix bugs in ported grammars not even using SetParse and Lookahead.
(The reencoding was added on Plan 9 for large character sets.
No Plan 9 programs using yacc looked at yychar.)
Other methods may be added to yyParser later as needed.
Obvious candidates include equivalents for the traditional
yyclearin and yyerrok macros.
Change-Id: Iaf7649efcf97e09f44d1f5bc74bb563a11f225de
Reviewed-on: https://go-review.googlesource.com/4850
Reviewed-by: Rob Pike <r@golang.org>
When the compiler echoes back an expression, it shows the
generated yacc expression. Change the generated code to
use a slice so that $3 shows up as yyDollar[3] in such messages.
Consider changing testdata/expr/expr.y to say:
$$.Sub(float64($1), $3)
(The float64 conversion is incorrect.)
Before:
expr.y:70[expr.go:486]: cannot convert exprS[exprpt - 2].num (type *big.Rat) to type float64
After:
expr.y:70[expr.go:492]: cannot convert exprDollar[1].num (type *big.Rat) to type float64
Change-Id: I74e494069df588e62299d1fccb282f3658d8f8f4
Reviewed-on: https://go-review.googlesource.com/4630
Reviewed-by: Rob Pike <r@golang.org>
CL 149110043 changed yacc to no longer keep a leading space
for quoted tokens. That is OK by itself but unfortunately
yacc was relying on that leading space to notice which tokens
it should not output as const declarations.
Add a few such tokens to expr.y, although it won't make any
immediate difference as we seem to have no tests for yacc.
LGTM=rsc
R=golang-codereviews, rsc
CC=golang-codereviews
https://golang.org/cl/152720043
From issue 7967 I learned:
1) yacc accepts either 'x' or "x" to mean token value 0x78
2) yacc also accepts 'xyz' and "XYZ" to mean token value 0x78
Use strconv.Unquote to simplify the handling of quoted
strings and check that each has only one rune.
Although this does clean things up, it makes 'x' and "x"
treated as different internally (now they are stored as
`'x'` and `"x"`; before they were both ` x`). Grammars that
use both interchangeably will now die with an error
similar to the one from issue 7967:
yacc bug -- cannot have 2 different Ts with same value
"+" and '+'
The echoing of the quotes should make clear what is going on.
The other semantic change caused by using strconv.Unquote
is that '\"' and "\'" are no longer valid. Like in Go, they must be
spelled without the backslash: '"' and "'".
On the other hand, now yacc and Go agree about what character
and string literals mean.
LGTM=r
R=r
CC=golang-codereviews
https://golang.org/cl/149110043
This will keep the go command from trying to build it
when the cmd/ tree is no longer a special case.
Also update doc.go to refer to the correct location.
(It was incorrect even before this CL.)
LGTM=r
R=iant, r
CC=golang-codereviews
https://golang.org/cl/134560043
It now serves as an example for go generate as well as for yacc.
NOTE: Depends on go generate, which is not yet checked in.
This is a proof of concept of the approach.
That is https://golang.org/cl/125580044.
LGTM=rsc
R=rsc
CC=golang-codereviews
https://golang.org/cl/125620044
The units example is nice but is covered by the Lucent
license, which may be a concern for some people making a
commercial source code distribution of Go.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/13283045
Also:
- faster code for example extraction
- simplify handling of command documentation:
all "main" packages are treated as commands
- various minor cleanups along the way
For commands written in Go, any doc.go file containing
documentation must now be part of package main (rather
then package documentation), otherwise the documentation
won't show up in godoc (it will still build, though).
For commands written in C, documentation may still be
in doc.go files defining package documentation, but the
recommended way is to explicitly ignore those files with
a +build ignore constraint to define package main.
Fixes#4806.
R=adg, rsc, dave, bradfitz
CC=golang-dev
https://golang.org/cl/7333046
The parser depends on it but the client might not import it, so make sure it's there.
Fixes#4038.
R=golang-dev, rsc
CC=golang-dev
https://golang.org/cl/6497094
In the example "units" program for goyacc, the exchange rates were
reciprocals of the correct amounts. Turn them right-side-up
and update them to current figures.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/6495053
Consequently, remove many package Makefiles,
and shorten the few that remain.
gomake becomes 'go tool make'.
Turn off test phases of run.bash that do not work,
flagged with $BROKEN. Future CLs will restore these,
but this seemed like a big enough CL already.
R=golang-dev, r
CC=golang-dev
https://golang.org/cl/5601057
1) create go-tool dir in make.bash
2) clean up stale binaries in make.bash
3) add 'tool' command to go
4) convert goyacc->yacc as a first test tool
Since goyacc stands alone, it's a safe trial.
R=rsc
CC=golang-dev
https://golang.org/cl/5576061