The Flex scanner first includes the driver declaration, then the parser's to get the set of defined tokens.
%{ /* -*- C++ -*- */ # include <cstdlib> # include <cerrno> # include <climits> # include <string> # include "calc++-driver.hh" # include "calc++-parser.hh" /* Work around an incompatibility in flex (at least versions 2.5.31 through 2.5.33): it generates code that does not conform to C89. See Debian bug 333231 <http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=333231>. */ # undef yywrap # define yywrap() 1 /* By default yylex returns int, we use token_type. Unfortunately yyterminate by default returns 0, which is not of token_type. */ #define yyterminate() return token::END %}
Because there is no #include
-like feature we don't need
yywrap
, we don't need unput
either, and we parse an
actual file, this is not an interactive session with the user.
Finally we enable the scanner tracing features.
%option noyywrap nounput batch debug
Abbreviations allow for more readable rules.
id [a-zA-Z][a-zA-Z_0-9]* int [0-9]+ blank [ \t]
The following paragraph suffices to track locations accurately. Each
time yylex
is invoked, the begin position is moved onto the end
position. Then when a pattern is matched, the end position is
advanced of its width. In case it matched ends of lines, the end
cursor is adjusted, and each time blanks are matched, the begin cursor
is moved onto the end cursor to effectively ignore the blanks
preceding tokens. Comments would be treated equally.
%{ # define YY_USER_ACTION yylloc->columns (yyleng); %} %% %{ yylloc->step (); %} {blank}+ yylloc->step (); [\n]+ yylloc->lines (yyleng); yylloc->step ();
The rules are simple, just note the use of the driver to report errors.
It is convenient to use a typedef to shorten
yy::calcxx_parser::token::identifier
into
token::identifier
for instance.
%{ typedef yy::calcxx_parser::token token; %} /* Convert ints to the actual type of tokens. */ [-+*/] return yy::calcxx_parser::token_type (yytext[0]); ":=" return token::ASSIGN; {int} { errno = 0; long n = strtol (yytext, NULL, 10); if (! (INT_MIN <= n && n <= INT_MAX && errno != ERANGE)) driver.error (*yylloc, "integer is out of range"); yylval->ival = n; return token::NUMBER; } {id} { yylval->sval = new std::string (yytext); return token::IDENTIFIER; } . driver.error (*yylloc, "invalid character"); %%
Finally, because the scanner related driver's member function depend on the scanner's data, it is simpler to implement them in this file.
void calcxx_driver::scan_begin () { yy_flex_debug = trace_scanning; if (file.empty () || file == "-") yyin = stdin; else if (!(yyin = fopen (file.c_str (), "r"))) { error ("cannot open " + file + ": " + strerror(errno)); exit (EXIT_FAILURE); } } void calcxx_driver::scan_end () { fclose (yyin); }