Puma Reference Manual | Puma: Puma::Token Class Reference |
#include <Puma/Token.h>
A token is created by a scanner (see Puma::CScanner) as part a token chain (Puma::Unit). It encapsulates a lexical unit of the scanned text.
The information provided about a lexical unit is
Public Types | |
enum | { ID_END_OF_FILE = -1, ID_UNKNOWN = -2, ID_ERROR = -3, ID_WARNING = -4 } |
Special token types. More... | |
Public Member Functions | |
Token (int type=0, LanguageID lang=LanguageID(0), const char *text="") | |
Constructor. | |
Token (const Token ©) | |
Copy-constructor. | |
virtual | ~Token () |
Destructor. | |
void | print (ostream &os) const |
Print the token text on the given stream. | |
virtual ListElement * | duplicate () |
Duplicate this token. | |
Unit * | unit () const |
Get the unit this token belongs to. | |
void | reset (int type=0, const char *text=0, LanguageID lang=LanguageID(0)) |
Reset the token object. | |
void | location (const Location &loc) |
Set the location of the token (usually the line and column in a file). | |
void | cont_lines (Array< int > *cl) |
Set the continues lines appeared in this token. | |
int | line_breaks () const |
Count the number of line breaks in the text of this token. | |
void | macro_generated () |
Set the token as being macro generated. | |
void | macro_call () |
Set the token as being a macro call. | |
const char * | text () const |
Get the token text. | |
DString & | dtext () const |
Get the token text. | |
char * | get_static_text () const |
Get the static token text. | |
int | type () const |
Get the type of the token (see Puma::CTokens). | |
const Location & | location () const |
Get the location of the token (usually the line and column in a file). | |
bool | is_macro_generated () const |
Check if the token is macro generated. | |
bool | is_macro_call () const |
Check if the token is a macro call. | |
bool | is_core () const |
Check if this is a core language token. | |
bool | is_identifier () const |
Check if the token is an identifier. | |
bool | is_keyword () const |
Check if the token is a keyword. | |
bool | is_wildcard () const |
Check if this is a wildcard token. | |
bool | is_preprocessor () const |
Check if this is a preprocessor token. | |
bool | is_directive () const |
Check if this is a compiler directive. | |
bool | is_whitespace () const |
Check if this is a white-space token. | |
bool | is_comment () const |
Check if this is a comment. | |
bool | is_macro_op () const |
Check if this is function-like macro operand. | |
bool | is_open () const |
Check if this is a left parenthesis. | |
bool | is_comma () const |
Check if this is a comma. | |
bool | is_close () const |
Check if this is a right parenthesis. | |
void * | operator new (size_t) |
Own operator new reusing memory. | |
void | operator delete (void *) |
Own delete operator. | |
Static Public Attributes | |
static LanguageID | comment_id |
Comment tokens. | |
static LanguageID | pre_id |
Preprocessor tokens. | |
static LanguageID | dir_id |
Compiler directives. | |
static LanguageID | cpp_id |
C/C++ core language tokens. | |
static LanguageID | white_id |
White-spaces. | |
static LanguageID | wildcard_id |
Wildcards. | |
static LanguageID | keyword_id |
Keywords. | |
static LanguageID | open_id |
Left parenthesis. | |
static LanguageID | macro_op_id |
Function-like macro operands. | |
static LanguageID | identifier_id |
Identifiers. | |
static LanguageID | comma_id |
Comma. | |
static LanguageID | close_id |
Right parenthesis. |
anonymous enum |
Puma::Token::Token | ( | int | type = 0 , |
|
LanguageID | lang = LanguageID(0) , |
|||
const char * | text = "" | |||
) |
Constructor.
type | The token type. | |
lang | The language of the token. | |
text | The optional token text (defaults to the empty string). |
Puma::Token::Token | ( | const Token & | copy | ) |
Copy-constructor.
copy | The token to copy. |
virtual Puma::Token::~Token | ( | ) | [virtual] |
Destructor.
Destroys the token text.
void Puma::Token::print | ( | ostream & | os | ) | const |
Print the token text on the given stream.
os | The output stream. |
virtual ListElement* Puma::Token::duplicate | ( | ) | [inline, virtual] |
Duplicate this token.
Implements Puma::ListElement.
Unit* Puma::Token::unit | ( | ) | const [inline] |
Get the unit this token belongs to.
void Puma::Token::reset | ( | int | type = 0 , |
|
const char * | text = 0 , |
|||
LanguageID | lang = LanguageID(0) | |||
) |
Reset the token object.
type | The new token type. | |
text | The new token text. | |
lang | The new token language. |
void Puma::Token::location | ( | const Location & | loc | ) | [inline] |
Set the location of the token (usually the line and column in a file).
loc | The location. |
void Puma::Token::cont_lines | ( | Array< int > * | cl | ) | [inline] |
Set the continues lines appeared in this token.
cl | The continues line array. |
int Puma::Token::line_breaks | ( | ) | const |
Count the number of line breaks in the text of this token.
void Puma::Token::macro_generated | ( | ) | [inline] |
Set the token as being macro generated.
void Puma::Token::macro_call | ( | ) | [inline] |
Set the token as being a macro call.
const char* Puma::Token::text | ( | ) | const |
Get the token text.
DString& Puma::Token::dtext | ( | ) | const |
Get the token text.
char* Puma::Token::get_static_text | ( | ) | const |
Get the static token text.
Only for keywords, operators, and so on with a constant token text.
int Puma::Token::type | ( | ) | const [inline] |
Get the type of the token (see Puma::CTokens).
const Location& Puma::Token::location | ( | ) | const [inline] |
Get the location of the token (usually the line and column in a file).
bool Puma::Token::is_macro_generated | ( | ) | const [inline] |
Check if the token is macro generated.
bool Puma::Token::is_macro_call | ( | ) | const [inline] |
Check if the token is a macro call.
bool Puma::Token::is_core | ( | ) | const [inline] |
Check if this is a core language token.
bool Puma::Token::is_identifier | ( | ) | const [inline] |
Check if the token is an identifier.
bool Puma::Token::is_keyword | ( | ) | const [inline] |
Check if the token is a keyword.
bool Puma::Token::is_wildcard | ( | ) | const [inline] |
Check if this is a wildcard token.
bool Puma::Token::is_preprocessor | ( | ) | const [inline] |
Check if this is a preprocessor token.
bool Puma::Token::is_directive | ( | ) | const [inline] |
Check if this is a compiler directive.
bool Puma::Token::is_whitespace | ( | ) | const [inline] |
Check if this is a white-space token.
bool Puma::Token::is_comment | ( | ) | const [inline] |
Check if this is a comment.
bool Puma::Token::is_macro_op | ( | ) | const [inline] |
Check if this is function-like macro operand.
bool Puma::Token::is_open | ( | ) | const [inline] |
Check if this is a left parenthesis.
bool Puma::Token::is_comma | ( | ) | const [inline] |
Check if this is a comma.
bool Puma::Token::is_close | ( | ) | const [inline] |
Check if this is a right parenthesis.
void* Puma::Token::operator new | ( | size_t | ) |
Own operator new reusing memory.
void Puma::Token::operator delete | ( | void * | ) |
Own delete operator.
LanguageID Puma::Token::comment_id [static] |
Comment tokens.
LanguageID Puma::Token::pre_id [static] |
Preprocessor tokens.
LanguageID Puma::Token::dir_id [static] |
Compiler directives.
LanguageID Puma::Token::cpp_id [static] |
C/C++ core language tokens.
LanguageID Puma::Token::white_id [static] |
White-spaces.
LanguageID Puma::Token::wildcard_id [static] |
Wildcards.
LanguageID Puma::Token::keyword_id [static] |
Keywords.
LanguageID Puma::Token::open_id [static] |
Left parenthesis.
LanguageID Puma::Token::macro_op_id [static] |
Function-like macro operands.
LanguageID Puma::Token::identifier_id [static] |
Identifiers.
LanguageID Puma::Token::comma_id [static] |
Comma.
LanguageID Puma::Token::close_id [static] |
Right parenthesis.