Home My Page Projects Code Snippets Project Openings SML/NJ
Summary Activity Forums Tracker Lists Tasks Docs Surveys News SCM Files

SCM Repository

[smlnj] View of /sml/trunk/src/lexgen/doc/ReleaseNotes/release-notes.tex
ViewVC logotype

View of /sml/trunk/src/lexgen/doc/ReleaseNotes/release-notes.tex

Parent Directory Parent Directory | Revision Log Revision Log

Revision 1902 - (download) (as text) (annotate)
Wed Mar 1 04:44:30 2006 UTC (13 years, 6 months ago) by mblume
File size: 2336 byte(s)
ml-flex -> lexgen

\usepackage[top=1.25in, bottom=1.25in, left=1.33in, right=1.33in]{geometry}


\title{\flex{} Release Notes}
\author{Aaron Turon\\



\flex{} is a scanner generator written in Standard ML.  It replaces the older
ML-Lex tool.  For information about its implementation, see the
\flex{} Implementation Notes. This document describes the features and usage of

At the moment, \flex{} is meant to provide an almost drop-in replacement for
ML-Lex.  As such, the basic feature set and specification format described in
the ML-Lex user guide is almost entirely applicable to \flex{}; here we describe
the points of departure.  Note that in the future \flex{} will have its own
specification format which may be quite different.  The new format will allow
for more substantial breaks from ML-Lex, including full unicode support.  The
effort so far has been directed toward building up the basic ML-Lex-compatible

The major differences between \flex{} and ML-Lex are the following:

\item \flex{} uses regular expression derivatives~\cite{derivatives} to
its automata and generates a control-flow-based lexer, rather than a
table-based lexer.

\item \texttt{REJECT()} and \texttt{yylineno} do not have substantial
performance cost.

\item There is no \texttt{yypos} off by 2 bug.

\item \texttt{\%full} is not presently supported (but will be soon).

\item \texttt{yylineno} is no longer a reference cell.

\item A new function, \texttt{makeLexer'}, allows the generated lexer to work
directly with a \texttt{TextIO.StreamIO.instream}.  Doing so gives a modest
performance improvement.

\item Several new commandline options are available:

\tt --dot & output Graphviz .DOT files for visualizing the DFAs \\
\tt --dump & dump detailed DFA description for the whole lexer \\
\tt --report & give a short summary \\
    & (list start states and number of DFA nodes
per start state) \\
\tt --match & interactive matching mode (for lexer testing)




ViewVC Help
Powered by ViewVC 1.0.0