jb -- Parser and Lexer Generation for Java

----------------------------------------------------------------------------

Last Updated: 9 October 1997
Latest Version: jb-5.4

[ Description | Retrieving | Change Log | Mailing List | Acknowledgments ]

Description

The jb system takes parsers generated using the Gnu Bison parser generator
system and translates them to execute in Java (tm) Jb takes the C file
output by Bison and scans it to extract the parse tables and constants. Jb
then scans various template files specified by the user and inserts the
extracted information at specified points in the templates.

In addition to generating parsers, jb provides two methods of generating
corresponding lexers.

  1. Flex -- the jb system (starting with version 3.0) can also take lexers
     generated using the Gnu flex generator and translate them to execute in
     Java; this is accomplished with a program called jf.

  2. yylex.generic -- A generic ad-hoc lexer that can be modified to produce
     lexers for typical programming languages.

Retrieving jb

FTP

Various versions of jb are available via anonymous ftp. The following
references are symbolic links to the latest versions.

Source:
     ftp://ftp.cs.colorado.edu/pub/cs/distribs/arcadia/jb.tar
Information file:
     ftp://ftp.cs.colorado.edu/pub/cs/distribs/arcadia/jb.txt

SRM

Various versions of jb can also be obtained from SRM -- the Serl Software
Release Manager at

http://www.cs.colorado.edu/cgi-bin/cgiwrap/~serl/getmainpage.tcl?GROUP=serl

Dependencies:

   * Bison -- Jb has been tested with bison version 1.24 and 1.25. Other
     versions probably will work as well as long as the bison output parser
     is not wildly changed, but jb will issue a warning when using other
     versions.

   * Flex -- Jb has been tested with flex version 2.5.2 and 2.5.3.

   * Java -- As of Jb version 5, Jb produces code targeted at Java version
     1.1 (including 1.1.1 and 1.1.2).

   * Starwave Regular Expression Package -- I use this package (version
     1.10) to provide regular expression support. This package was produced
     by Jonathan Payne (jpayne@starwave.com).

Mailing List

If you are interested in receiving occasional mailings about this system,
please send your preferred email address to the contact address below and
mentioning the name of this system.

Acknowledgments

This work is sponsored by the Air Force Material Command, Rome Laboratory,
and the Defense Advanced Research Projects Agency under Contract Number
F30602-94-C-0253.

----------------------------------------------------------------------------
Dennis Heimbigner <dennis@cs.colorado.edu>
----------------------------------------------------------------------------

                         SERL <serl@cs.colorado.edu>
                 [ Research | People | Software | Papers ]

----------------------------------------------------------------------------
----------------------------------------------------------------------------

Change Log

----------------------------------------------------------------------------

Last Updated: 4 July 1997

[ Version 5 | Version 4 | Version 3 ]

Changes Incorporated into Version 5

Minor version levels are indicated in parentheses.

   * (0) Modified all files to compile and execute under JDK-1.1. This
     primarily involved the following changes to the following directories
        o regexp:
             + Moved various classes out of (e.g.) Regexp.java and into
               their own files.
             + Replaced deprecated String class methods.
        o jbf:
             + Converted InputStream and OutputStream classes to use
               corresponding Reader and Writer classes
   * (0) Fixed bug caused by YYtokentypes.tokennames not handling a null
     argument.
   * (1) Fixed bug in which -YYtokentypes does not replace all instances in
     template.
   * (1) Placed linebreaks in generated YYtokentypes.java to make it
     palatable to editors.
   * (2) Provided mechanism to allow for compilation under jdk 1.0.2.
   * (2) Fix miscellaneous errors in the documentation.
   * (3) Allowed for the insertion of code pieces (``%{...%}'') into the
     rules section. These code pieces are inserted into the beginning of the
     yylex procedure in the generated lexer.
   * (4) Fixed the -package option to add the trailing semicolon. Also fixed
     the -package documentation to clarify its effect. Thanks to Gerhard
     Wickler for finding this bug.

Changes Incorporated into Version 4

Minor version levels are indicated in parentheses.

   * (0) Rebuilt jb and jf to be Java programs. Unfortunately, Java is not a
     good string processing language, so the new programs are detectably
     slower than the older Tcl versions.
   * (0) Replaced the use of sed by a java program called subst.
   * (0) Rebuilt the mechanism by which tokens are passed between the parser
     and the lexer. The change attempts to avoid the creation of a new token
     for every lexeme in the input and attempts to avoid re-allocating so
     many strings. This destroys compatibility with earlier versions of jb.
   * (0) Added support for the bison %union construct to allow typing of
     tokens and rules. This can simplify semantic actions by automating some
     casting of values.
   * (1) The static initializer in class YYtokentypes is incorrect. In the
     file jbf/yytokentypes.template, line 1 should be substituted for line
     2.
       1. for(int i=0;i<tokenmax;i++) {Tokentype[i] = new Integer(i);}
       2. for(int i=0;i<=tokenmax;i++) {Tokentype[i] = new Integer(i);}
   * (2) Incorporated suggestion from Steven G. Parker
     (sparker@taz.cs.utah.edu), who notes as follows. ``By default, Java
     doesn't buffer File*Streams. So each read/write to that stream
     translates to a read/write system call. If that isn't bad enough, the
     implementation of threads in Java makes each read/write system call
     take about 4 calls - to getpid, fcntl, and some other junk. Adding the
     buffer makes it read in 4K chunks. For my one test case, it knocked it
     down from around 60 seconds to around 20 seconds. ...You have to flush
     them manually - I had a program that would chop off the tail end
     because the buffer wasn't getting flushed when the program exited.''

Changes Incorporated into Version 3

Minor version levels are indicated in parentheses.

   * (0) Major renaming of various classes to avoid name clashes. This
     destroys compatibility with earlier versions of jb.
   * (0) Moved the calc and idl examples to a general examples directory and
     added a grammar for Java.
   * (0) Renamed the bison package to be named "jbf" (java+bison+flex).
   * (0) Added the ability to store line number and character positions;
     this also required modifications to jbf/yylex.generic.
   * (0) Introduced a general nonterminal mechanism that can be used to
     build an intermediate representation tree (ala the Arcadia IRIS
     mechanism) for any parse. Both the new non-terminal class (YYnonterm)
     and the old token class (now called YYtoken) are both subtypes of a new
     class called YYnode. The names and an index of the non-terminals is
     inserted into generated class YYtokentypes (was Tokentypes).
   * (0) Dropped JavaLex in favor of the Gnu Flex system for generating
     lexers. Flex produced much smaller lexers, and ones that are
     competitive in size to the generic lexer.
   * (1) Fixed bug in YYlexbuffer that miscounted the number of lines after
     purge was finished.

----------------------------------------------------------------------------

                         SERL <serl@cs.colorado.edu>
                 [ Research | People | Software | Papers ]