LLK Changes

• CHANGE - LLK v0.5 generated code now use Java5 features and require JDK1.5.0 or above to run.
• CHANGE - Support files templates are now splitted into the llk-share project which can also be used directly and avoid having to generate a separate set of support files for each grammar. All samples grammars now use support files from the llk-share project.
• NEW - Added llk-share-formatter project which provide support for code formatters.
• NEW - Added sample grammars for CSS2, HTML4, XHTML1.0 and XML1.1 that demonstrate context switching in lexers.

Grammar

• CHANGE - Lexer grammar now default to has grammar option Greedy=false instead of Greedy=true.
• CHANGE - Lexer grammar no longer accept Import option since it is not used anyway.
• CHANGE - Alias rule now define a token type with ExplicitNodePrefix + rule name instead of just using the rule name as node name.
• CHANGE - Rule name can no longer be used to reference token type of rule that define an implicit or explicit node. The actual node name must be used. Example:

  	void test01(): {
  	    ...
  	}
  	aliasTest01=<ASTtest01>;
  

  Here, ASTtest01 must be used, test01 no longer works.
• FIXED - try{} catch{} finally{} element is now allowed in syntactic predicts.
• FIXED - now issue error if error action is not the last alternative in the choices block or the choices block is unconditional, eg. [ {} | alt ] or ( alt | {} ).

Implementation

• NEW - Support file templates are splitted to a new project llk-share. LLK use files in llk-share as templates to generate the support files. Projects that use multiple related parser can also share support files from llk-share using the -spath option instead let LLK generate a set of support files for each grammar.
• NEW - ILLKMain added methods

  	Map getOptions();
  	void setQuiet(); // suppress warning and error messages.
  	boolean isQuiet();
  	void setMaxErrors(int max);
  	void setMaxWarns(int max);
  	char[] getFileContent(String path, Charset charset);
  	char[] getFileContent(Charset charset);
  

• CHANGE - ILLKMain.hasError() is renamed to hasErrors() to be consistent with hasWarnings().
• CHANGE - ILLKNode added method

  	/** @return Length of node source text. */
  	int getLength();
  

• CHANGE - Lexer, Parser, and TreeParser default constructor no longer call llkReset() and no longer reset the input (ILLKLexerInput,ILLKParserInput and ILLKTreeParserInput respectively). The default constructor would still call llkInit().
• CHANGE - Also LLKParserInput constructor would no longer call lexer.llkReset().
• CHANGE - LLK now generate LLKToken.java,LLKNode.java, visitor and visitor adapter in the output directory instead of the support directory, so that files in the support directory can be shared more easily.
• CHANGE - accept() method is member of LLKNode only, no longer a member of ILLKNode.
• CHANGE - Code generate no longer generate keyword context related stuffs when grammar has no keyword context.
• CHANGE - Code generate now export the aliases of the token types (with has same id but different name), so that alias defined in the lexer can be used in parsers.
• CHANGE - Since alias rules are exported, they are marked as protected instead of private.
• CHANGE - TreeParser now keep the top of llkStack as llkParent field for quick access.
• CHANGE - NodeFileGenerator no longer generate import of the support directory if the grammar do not generate visitor.
• CHANGE - Added ILLKLexerInput.replace(int start, int end, char[] text) and LLKLexerInput.replace(int start, int end, char[] text).
• CHANGE - TokenSet now print the token names in the token set for lexer (ie. characters) in the generated source.
• CHANGE - Added LLKLexerBase.llkCreateToken(int type).
• CHANGE - Added LLKTree.peek(int n).
• CHANGE - SimpleLLKMain.getOptInt() now works with both Integer or String option value.
• CHANGE - LLKAnalyzer.warnChoicesFollowConflict() now print lookahead sets at all k up to max k instead of just at max k. The information would be useful since conflicts should often be resolved at depth less than max k and conflicts at max k are often just artifacts.
• NEW - LLKTreeParserGenerator now takes a -notrim options. If specified it would not trim the input grammar and print the token references that would otherwise be trimmed as comment. The output TreeParser grammar would not be valid, but it would useful for debugging and as basis for formatters.
• NEW - sf.llk.impl.LLKMain added -import command line option. If specified, it override the Import grammar option specified in the grammar file.
• NEW - Added support interface IReadOnlyLocator. ISourceLocator now extends IReadOnlyLocator.

Bugs

• FIXED - Fixed analyzer bug#426 that failed to report choice-choice conflict when one of the choices has incomplete depths.
• FIXED - Fixed bug#428 that conflicts with code rule not reported.
• FIXED - Code generator used llkInvertedBitTest() for more than 4 token types for parser and treeparser is fixed.
• FIXED - Code generator now create directory if needed before writing files instead of reporting file not found error.
• FIXED - Code generator now do not generate accept() methods for ILLKNode.java and LLKNode.java if grammar option Multi=false.
• FIXED - JavaCodeGenerator.genMatchString() that probe at character beyond length of string.
• FIXED - Added IDirectiveHandler.isDefined(String name) which is required by llk-csharp-model DefaultDirectiveHandler, ... etc.

Support Projects Changes

llk-share

• CHANGE - Added IReadOnlyLocator interface. ISourceLocator now extends IReadOnlyLocator. ILLKParserInput.getLocator() now returns IReadOnlyLocator instead of ISourceLocator.

Sample Grammar Changes

General

• llk-java5 project is obsoleted by llk-java-model project.
• llk-csharp obsoleted by llk-csharp-model project.

llk-java-model

• NEW - llk-java-model project now comes with a Java 5 code formatter.
• FIXED - A number of bugs in the Java5 grammar.
• FIXED - JavaLexer now recognize HexadeciamalFloatingPointLiteral.

llk-csharp-model

• NEW - llk-csharp-model project now comes with a CSharp 2.0 code formatter.
• FIXED - A number of bugs in the CSharp 2.0 grammar.

Requirements

Requirements to run LLK

• Java 1.5.0 for Linux.

• Gnu getopt v.1.0.9.
• Jakarta-oro v2.0.8.
• blacksun-util.jar from Black-sun v0.8.0.

Requirement to run generated parsers

Requirement to compile LLK

• Java 1.5.0 for Linux.
• Eclipse v3.1 GTK for Linux and its bundled org.apache.ant_1.6.5 and org.junit_3.8.1 plugins.

• black-sun plugins for Eclipse, which provide interface to the fireant builder and an LLK editor that support syntax highlighting and formatting, ... etc.

Running the LLK binary

	java -classpath $(classpath) sf.llk.impl.LLKMain \
		-o <output-dir> -module <module> \
		-sdir <support-dir> -smodule <support-module> \
		<grammar.ll>

	java -jar llk.jar -o <output-dir> -module <module> \
		-sdir <support-dir> -smodule <support-module> \
		<grammar.ll>

Compiling LLK from the source

The following directory structure is assumed:

    eclipse/
        plugins/
            org.apache.ant_1.6.5/
            ...
    ext/
        getopt.jar
        jpackage/oro-2.0.8.jar
        blacksun-util.jar
        llk-share.jar
        llk.jar
        ...
    llk/
        src/
        bin/
        ...
    llk-fireant/
        ...
    ...

Build with Eclipse:

• Make sure you are running with Java1.5 and Eclipse 3.1.
• Setup directory structure as above.
• Unpack the binary distribution into the ext/ directory. Unpack source distribution archive into the workspace and start Eclipse.
• Make sure Java compliance level is set to Java 5.0 in Window/Prefereences/Java/Compiler/Compiler Compliance Level.
• Import the llk and other projects with Import.../Existing project into Workspace
• Refresh and rebuild all projects.
• To install, invoke the fireant builder sf.llk.fireant.Builder Install target. That would create all the binary and source jar files in the ext/ directory. There are a number of ways to invoke the fireant builder inside Eclipse:
  • If you have the black-sun plugins for Eclipse installed, you can invoke the builder with the Fireant menu item on thesf.llk.fireant.Builder class in the PackageView popup menu or from an editor (see the black-sun documentation for details on this).
  • If you do not have the black-sun plugins installed, the easiest way is to use Run As/Java Application ... menu item on the sf.llk.fireant.Builder class in the PackageView. Eclipse would create a Launch configuration for the builder with correct classpaths. Just enter the target Install as argument and launch the configuration would invoke the fireant builder to build the target.
  • Otherwise the fireant.sh script inside the llk-fireant project directory may works for you:

        bash ./fireant.sh Install
    

Build with shell scripts:

• Make sure you are running with Java1.5
• Setup directory structure as above.
• Make sure you have the ant and junit plugins from Eclipse 3.1 in the workspace.
• Unpack the binary distribution into the ext/ directory. Unpack source distribution archive into the workspace.
• Inside the llk-fireant project directory, run:

  	bash ./build.sh
      bash ./fireant.sh Install
  

Regenerating the LLK parser

• Make sure you setup the correct environment as described in Compiling the source distribution section above.
• If you have the black-sun builder plugin installed (for Eclipse), any changes to LLK.ll or other grammar files would trigger the fireant incremental builder to rebuild of parsers.
• Otherwise, in the llk-fireant project directory, run:

      bash ./fireant.sh BlacksunFireantBuild
  

  to rebuild any modified grammar or

      bash ./fireant.sh RebuildParsers
  

  to rebuild all the parsers.

Status and plans