haskell-lexer-1.0: A fully compliant Haskell 98 lexer.Source codeContentsIndex
Language.Haskell.Lexer
Synopsis
type PosToken = (Token, (Pos, String))
data Token
= Varid
| Conid
| Varsym
| Consym
| Reservedid
| Reservedop
| Specialid
| IntLit
| FloatLit
| CharLit
| StringLit
| Qvarid
| Qconid
| Qvarsym
| Qconsym
| Special
| Whitespace
| NestedCommentStart
| NestedComment
| LiterateComment
| Commentstart
| Comment
| ErrorToken
| GotEOF
| TheRest
| ModuleName
| ModuleAlias
| Layout
| Indent Int
| Open Int
lexerPass0 :: String -> [PosToken]
lexerPass0' :: Pos -> String -> [PosToken]
lexerPass1 :: String -> [PosToken]
rmSpace :: [PosToken] -> [PosToken]
layoutPre :: [PosToken] -> [PosToken]
data Pos = Pos {
char :: !Int
line :: !Int
column :: !Int
}
simpPos :: Pos -> (Int, Int)
startPos :: Pos
nextPos :: Pos -> String -> Pos
nextPos1 :: Pos -> Char -> Pos
Documentation
type PosToken = (Token, (Pos, String))Source
data Token Source
Haskell token classifications:
Constructors
VaridVariable
ConidConstructor
VarsymVariable operator
ConsymConstructor operator
ReservedidReserved keyword
ReservedopReserved operator
Specialid
IntLitIntegral numeric literal
FloatLitFractional numeric literal
CharLitCharacter literal
StringLitString literal
QvaridQualified variable
QconidQualified constructor
QvarsymQualified variable operator
QconsymQualified constructor operator
Special
WhitespaceWhite space
NestedCommentStartInternal: causes a call to an external function
NestedCommentA nested comment ({- ... -})
LiterateCommentNot handled by the lexer
CommentstartDashes
CommentThe stuff after the dashes
ErrorToken
GotEOF
TheRest
ModuleName
ModuleAliasrecognized in a later pass
Layoutfor implicit braces
Indent Int<n>, to preceed first token on each line
Open Int{n}, after let, where, do or of, if not followed by a "{"
lexerPass0 :: String -> [PosToken]Source
Tokenize and add position information. Preserves white space, and does not insert extra tokens due to layout.
lexerPass0' :: Pos -> String -> [PosToken]Source
Same as lexerPass0, except that it uses the given start position.
lexerPass1 :: String -> [PosToken]Source
The function lexerPass1 handles the part of lexical analysis that can be done independently of the parser---the tokenization and the addition of the extra layout tokens <n> and {n}, as specified in section 9.3 of the revised Haskell 98 Report.
rmSpace :: [PosToken] -> [PosToken]Source
Remove token that are not meaningful (e.g., white space and comments).
layoutPre :: [PosToken] -> [PosToken]Source
This is an implementation of Haskell layout, as specified in section 9.3 of the revised Haskell 98 report. This preprocessor inserts the extra <n> and {n} tokens.
data Pos Source
The posisiotn within a file.
Constructors
Pos
char :: !Int
line :: !Int
column :: !Int
simpPos :: Pos -> (Int, Int)Source
The line and column numbers of a position.
startPos :: PosSource
The first column is designated column 1, not 0.
nextPos :: Pos -> String -> PosSource
Advance position by a string.
nextPos1 :: Pos -> Char -> PosSource
Advance position by a single character.
Produced by Haddock version 2.6.0