jb -- Parser and Lexer Generation for Java

Last Updated: 9 October 1997
Latest Version: jb-5.4

[ Description | Retrieving | Change Log | Mailing List | Acknowledgments ]

Description

The jb system takes parsers generated using the Gnu Bison parser generator system and translates them to execute in Java (tm) Jb takes the C file output by Bison and scans it to extract the parse tables and constants. Jb then scans various template files specified by the user and inserts the extracted information at specified points in the templates.

In addition to generating parsers, jb provides two methods of generating corresponding lexers.

Flex -- the jb system (starting with version 3.0) can also take lexers generated using the Gnu flex generator and translate them to execute in Java; this is accomplished with a program called jf.
yylex.generic -- A generic ad-hoc lexer that can be modified to produce lexers for typical programming languages.

Retrieving jb

FTP

Various versions of jb are available via anonymous ftp. The following references are symbolic links to the latest versions.

Source:: ftp://ftp.cs.colorado.edu/pub/cs/distribs/arcadia/jb.tar
Information file:: ftp://ftp.cs.colorado.edu/pub/cs/distribs/arcadia/jb.txt

SRM

Various versions of jb can also be obtained from SRM -- the Serl Software Release Manager at

http://www.cs.colorado.edu/cgi-bin/cgiwrap/~serl/getmainpage.tcl?GROUP=serl

Dependencies:

Bison -- Jb has been tested with bison version 1.24 and 1.25. Other versions probably will work as well as long as the bison output parser is not wildly changed, but jb will issue a warning when using other versions.
Flex -- Jb has been tested with flex version 2.5.2 and 2.5.3.
Java -- As of Jb version 5, Jb produces code targeted at Java version 1.1 (including 1.1.1 and 1.1.2).
Starwave Regular Expression Package -- I use this package (version 1.10) to provide regular expression support. This package was produced by Jonathan Payne (jpayne@starwave.com).

Mailing List

If you are interested in receiving occasional mailings about this system, please send your preferred email address to the contact address below and mentioning the name of this system.

Acknowledgments

This work is sponsored by the Air Force Material Command, Rome Laboratory, and the Defense Advanced Research Projects Agency under Contract Number F30602-94-C-0253.

Dennis Heimbigner <dennis@cs.colorado.edu>

SERL <serl@cs.colorado.edu>
[ Research | People | Software | Papers ]

SERL: jb -- Bison Parser and Lexer Generating Java

Change Log

Last Updated: 4 July 1997

[ Version 5 | Version 4 | Version 3 ]

Changes Incorporated into Version 5

Minor version levels are indicated in parentheses.

(0) Modified all files to compile and execute under JDK-1.1. This primarily involved the following changes to the following directories
- regexp:
  - Moved various classes out of (e.g.) Regexp.java and into their own files.
  - Replaced deprecated String class methods.
- jbf:
  - Converted InputStream and OutputStream classes to use corresponding Reader and Writer classes
(0) Fixed bug caused by YYtokentypes.tokennames not handling a null argument.
(1) Fixed bug in which -YYtokentypes does not replace all instances in template.
(1) Placed linebreaks in generated YYtokentypes.java to make it palatable to editors.
(2) Provided mechanism to allow for compilation under jdk 1.0.2.
(2) Fix miscellaneous errors in the documentation.
(3) Allowed for the insertion of code pieces (``%{...%}'') into the rules section. These code pieces are inserted into the beginning of the yylex procedure in the generated lexer.
(4) Fixed the -package option to add the trailing semicolon. Also fixed the -package documentation to clarify its effect. Thanks to Gerhard Wickler for finding this bug.

Changes Incorporated into Version 4

Minor version levels are indicated in parentheses.

(0) Rebuilt jb and jf to be Java programs. Unfortunately, Java is not a good string processing language, so the new programs are detectably slower than the older Tcl versions.
(0) Replaced the use of sed by a java program called subst.
(0) Rebuilt the mechanism by which tokens are passed between the parser and the lexer. The change attempts to avoid the creation of a new token for every lexeme in the input and attempts to avoid re-allocating so many strings. This destroys compatibility with earlier versions of jb.
(0) Added support for the bison %union construct to allow typing of tokens and rules. This can simplify semantic actions by automating some casting of values.
(1) The static initializer in class YYtokentypes is incorrect. In the file jbf/yytokentypes.template, line 1 should be substituted for line 2.
1. for(int i=0;i<tokenmax;i++) {Tokentype[i] = new Integer(i);}
2. for(int i=0;i<=tokenmax;i++) {Tokentype[i] = new Integer(i);}
(2) Incorporated suggestion from Steven G. Parker (sparker@taz.cs.utah.edu), who notes as follows. ``By default, Java doesn't buffer File*Streams. So each read/write to that stream translates to a read/write system call. If that isn't bad enough, the implementation of threads in Java makes each read/write system call take about 4 calls - to getpid, fcntl, and some other junk. Adding the buffer makes it read in 4K chunks. For my one test case, it knocked it down from around 60 seconds to around 20 seconds. ...You have to flush them manually - I had a program that would chop off the tail end because the buffer wasn't getting flushed when the program exited.''

Changes Incorporated into Version 3

Minor version levels are indicated in parentheses.

(0) Major renaming of various classes to avoid name clashes. This destroys compatibility with earlier versions of jb.
(0) Moved the calc and idl examples to a general examples directory and added a grammar for Java.
(0) Renamed the bison package to be named "jbf" (java+bison+flex).
(0) Added the ability to store line number and character positions; this also required modifications to jbf/yylex.generic.
(0) Introduced a general nonterminal mechanism that can be used to build an intermediate representation tree (ala the Arcadia IRIS mechanism) for any parse. Both the new non-terminal class (YYnonterm) and the old token class (now called YYtoken) are both subtypes of a new class called YYnode. The names and an index of the non-terminals is inserted into generated class YYtokentypes (was Tokentypes).
(0) Dropped JavaLex in favor of the Gnu Flex system for generating lexers. Flex produced much smaller lexers, and ones that are competitive in size to the generic lexer.
(1) Fixed bug in YYlexbuffer that miscounted the number of lines after purge was finished.

SERL <serl@cs.colorado.edu>
[ Research | People | Software | Papers ]