Lexical

The lexical analysis is independent of the syntax parsing and the semantic analysis. The lexical analyzer splits the source text up into tokens. The lexical grammar describes the syntax of those tokens. The grammar is designed to be suitable for high speed scanning and to make it easy to write a correct scanner for it. It has a minimum of special case rules and there is only one phase of translation.

Source Text

For now, only one format of the source text is allowed UTF8 without BOM.

Character Set

                
                    Character:
                        any Unicode character

End of File

                
                    EndOfFile:
                    physical end of the file
                    \u0000
                    \u001A

The source text is terminated by whichever comes first.

End of Line

                
                    EndOfLine:
                        \u000D
                        \u000A
                        \u000D \u000A
                        \u2028
                        \u2029
                        EndOfFile

White Space

                
                WhiteSpace:
                    Space
                    Space WhiteSpace

                Space:
                    \u0020
                    \u0009
                    \u000B
                    \u000C

Comments

Tokens

Keywords