LLK 0.5

Release notes

LLK Changes

  • CHANGE - LLK v0.5 generated code now use Java5 features and require JDK1.5.0 or above to run.
  • CHANGE - Support files templates are now splitted into the llk-share project which can also be used directly and avoid having to generate a separate set of support files for each grammar. All samples grammars now use support files from the llk-share project.
  • NEW - Added llk-share-formatter project which provide support for code formatters.
  • NEW - Added sample grammars for CSS2, HTML4, XHTML1.0 and XML1.1 that demonstrate context switching in lexers.


  • CHANGE - Lexer grammar now default to has grammar option Greedy=false instead of Greedy=true.
  • CHANGE - Lexer grammar no longer accept Import option since it is not used anyway.
  • CHANGE - Alias rule now define a token type with ExplicitNodePrefix + rule name instead of just using the rule name as node name.
  • CHANGE - Rule name can no longer be used to reference token type of rule that define an implicit or explicit node. The actual node name must be used. Example:
    	void test01(): {
    Here, ASTtest01 must be used, test01 no longer works.
  • FIXED - try{} catch{} finally{} element is now allowed in syntactic predicts.
  • FIXED - now issue error if error action is not the last alternative in the choices block or the choices block is unconditional, eg. [ {} | alt ] or ( alt | {} ).


  • NEW - Support file templates are splitted to a new project llk-share. LLK use files in llk-share as templates to generate the support files. Projects that use multiple related parser can also share support files from llk-share using the -spath option instead let LLK generate a set of support files for each grammar.
  • NEW - ILLKMain added methods
    	Map getOptions();
    	void setQuiet(); // suppress warning and error messages.
    	boolean isQuiet();
    	void setMaxErrors(int max);
    	void setMaxWarns(int max);
    	char[] getFileContent(String path, Charset charset);
    	char[] getFileContent(Charset charset);
  • CHANGE - ILLKMain.hasError() is renamed to hasErrors() to be consistent with hasWarnings().
  • CHANGE - ILLKNode added method
    	/** @return Length of node source text. */
    	int getLength();
  • CHANGE - Lexer, Parser, and TreeParser default constructor no longer call llkReset() and no longer reset the input (ILLKLexerInput,ILLKParserInput and ILLKTreeParserInput respectively). The default constructor would still call llkInit().
  • CHANGE - Also LLKParserInput constructor would no longer call lexer.llkReset().
  • CHANGE - LLK now generate LLKToken.java,LLKNode.java, visitor and visitor adapter in the output directory instead of the support directory, so that files in the support directory can be shared more easily.
  • CHANGE - accept() method is member of LLKNode only, no longer a member of ILLKNode.
  • CHANGE - Code generate no longer generate keyword context related stuffs when grammar has no keyword context.
  • CHANGE - Code generate now export the aliases of the token types (with has same id but different name), so that alias defined in the lexer can be used in parsers.
  • CHANGE - Since alias rules are exported, they are marked as protected instead of private.
  • CHANGE - TreeParser now keep the top of llkStack as llkParent field for quick access.
  • CHANGE - NodeFileGenerator no longer generate import of the support directory if the grammar do not generate visitor.
  • CHANGE - Added ILLKLexerInput.replace(int start, int end, char[] text) and LLKLexerInput.replace(int start, int end, char[] text).
  • CHANGE - TokenSet now print the token names in the token set for lexer (ie. characters) in the generated source.
  • CHANGE - Added LLKLexerBase.llkCreateToken(int type).
  • CHANGE - Added LLKTree.peek(int n).
  • CHANGE - SimpleLLKMain.getOptInt() now works with both Integer or String option value.
  • CHANGE - LLKAnalyzer.warnChoicesFollowConflict() now print lookahead sets at all k up to max k instead of just at max k. The information would be useful since conflicts should often be resolved at depth less than max k and conflicts at max k are often just artifacts.
  • NEW - LLKTreeParserGenerator now takes a -notrim options. If specified it would not trim the input grammar and print the token references that would otherwise be trimmed as comment. The output TreeParser grammar would not be valid, but it would useful for debugging and as basis for formatters.
  • NEW - sf.llk.impl.LLKMain added -import command line option. If specified, it override the Import grammar option specified in the grammar file.
  • NEW - Added support interface IReadOnlyLocator. ISourceLocator now extends IReadOnlyLocator.


  • FIXED - Fixed analyzer bug#426 that failed to report choice-choice conflict when one of the choices has incomplete depths.
  • FIXED - Fixed bug#428 that conflicts with code rule not reported.
  • FIXED - Code generator used llkInvertedBitTest() for more than 4 token types for parser and treeparser is fixed.
  • FIXED - Code generator now create directory if needed before writing files instead of reporting file not found error.
  • FIXED - Code generator now do not generate accept() methods for ILLKNode.java and LLKNode.java if grammar option Multi=false.
  • FIXED - JavaCodeGenerator.genMatchString() that probe at character beyond length of string.
  • FIXED - Added IDirectiveHandler.isDefined(String name) which is required by llk-csharp-model DefaultDirectiveHandler, ... etc.

Support Projects Changes


  • CHANGE - Added IReadOnlyLocator interface. ISourceLocator now extends IReadOnlyLocator. ILLKParserInput.getLocator() now returns IReadOnlyLocator instead of ISourceLocator.

Sample Grammar Changes


  • llk-java5 project is obsoleted by llk-java-model project.
  • llk-csharp obsoleted by llk-csharp-model project.


  • NEW - llk-java-model project now comes with a Java 5 code formatter.
  • FIXED - A number of bugs in the Java5 grammar.
  • FIXED - JavaLexer now recognize HexadeciamalFloatingPointLiteral.


  • NEW - llk-csharp-model project now comes with a CSharp 2.0 code formatter.
  • FIXED - A number of bugs in the CSharp 2.0 grammar.


Requirements to run LLK

The current distribution is developed and tested with Java 1.5.0 and Linux. Since it is written completely in Java, it MAY work in other environment. These are packages required and being tested on. There are a number of support packages required that are bundled with the binary distribution:

Requirement to run generated parsers

Same as requirement to run LLK above.

Requirement to compile LLK

The current distribution is developed and tested only in Linux. Since it is written completely in Java, it may work in other environment. These are packages required and being tested on.
  • Java 1.5.0 for Linux.
  • Eclipse v3.1 GTK for Linux and its bundled org.apache.ant_1.6.5 and org.junit_3.8.1 plugins.
You also need the support packages that are bundled in the binary distribution. Simply unpack the binary distribution into the ext/ directory. Also it is highly recommended that you have
  • black-sun plugins for Eclipse, which provide interface to the fireant builder and an LLK editor that support syntax highlighting and formatting, ... etc.

Running the LLK binary

LLK binary distribution include the ready to run binary llk.jar and required libraries. The main class (sf.llk.impl.LLKMain) should run if the jar files to llk-share, getopt, Jakarta-oro, blacksun-util packages are in the classpath. To run the binary, for example:
	java -classpath $(classpath) sf.llk.impl.LLKMain \
		-o <output-dir> -module <module> \
		-sdir <support-dir> -smodule <support-module> \
Or inside the directory that contains the unpacked binary distribution, run:
	java -jar llk.jar -o <output-dir> -module <module> \
		-sdir <support-dir> -smodule <support-module> \

Compiling LLK from the source

The package is developed and tested in Eclipse v3.1 GTK for Linux and Java 1.5.0. This release use the fireant build system from the black-sun project. The builders are located in the llk-fireant project.

The following directory structure is assumed:

where eclipse/ is the Eclipse installation. ext/ is the directory that contains the support libraries (eg. Jakarta-oro). llk/, ... etc are the project directories from the source distribution.

Build with Eclipse:

  • Make sure you are running with Java1.5 and Eclipse 3.1.
  • Setup directory structure as above.
  • Unpack the binary distribution into the ext/ directory. Unpack source distribution archive into the workspace and start Eclipse.
  • Make sure Java compliance level is set to Java 5.0 in Window/Prefereences/Java/Compiler/Compiler Compliance Level.
  • Import the llk and other projects with Import.../Existing project into Workspace
  • Refresh and rebuild all projects.
  • To install, invoke the fireant builder sf.llk.fireant.Builder Install target. That would create all the binary and source jar files in the ext/ directory. There are a number of ways to invoke the fireant builder inside Eclipse:
    • If you have the black-sun plugins for Eclipse installed, you can invoke the builder with the Fireant menu item on thesf.llk.fireant.Builder class in the PackageView popup menu or from an editor (see the black-sun documentation for details on this).
    • If you do not have the black-sun plugins installed, the easiest way is to use Run As/Java Application ... menu item on the sf.llk.fireant.Builder class in the PackageView. Eclipse would create a Launch configuration for the builder with correct classpaths. Just enter the target Install as argument and launch the configuration would invoke the fireant builder to build the target.
    • Otherwise the fireant.sh script inside the llk-fireant project directory may works for you:
          bash ./fireant.sh Install

Build with shell scripts:

If you do not have Eclipse, you may try the build scripts.
  • Make sure you are running with Java1.5
  • Setup directory structure as above.
  • Make sure you have the ant and junit plugins from Eclipse 3.1 in the workspace.
  • Unpack the binary distribution into the ext/ directory. Unpack source distribution archive into the workspace.
  • Inside the llk-fireant project directory, run:
    	bash ./build.sh
        bash ./fireant.sh Install

Regenerating the LLK parser

The LLK parser is generated by LLK itself. To regenerate the lexer, parser and treeparsers from LLK.ll ... etc, a LLK binary is required (one is included in the binary distribution).
  • Make sure you setup the correct environment as described in Compiling the source distribution section above.
  • If you have the black-sun builder plugin installed (for Eclipse), any changes to LLK.ll or other grammar files would trigger the fireant incremental builder to rebuild of parsers.
  • Otherwise, in the llk-fireant project directory, run:
        bash ./fireant.sh BlacksunFireantBuild
    to rebuild any modified grammar or
        bash ./fireant.sh RebuildParsers
    to rebuild all the parsers.

Status and plans