Using ANTLR plugin to process separate lexer and parser grammars

Help/Discuss
1

This is a follow up question to Using Gradle 2.10's ANTLR plugin to import an ANTLR 4 lexer grammar into another grammar
As suggested by @st_oehme I opened a new topic.

I took @jbduncan’s project and modified it to match my use case (see attachment). The difference is that in the original project the lexer grammar was imported as a grammar, while I just want to use separate lexer and parser grammars.

It works fine when the generated CommonLexerRules.tokens is copied to src/main/antlr/org/jbduncan but otherwise fails with

error(160): org/jbduncan/Expr.g4:4:11: cannot find tokens file /home/henry/tmp/antlr/antlr-example/src/generated/java/CommonLexerRules.tokens

antlr-example.zip (55.1 KB)

1 Like
2

I know its been 4 years since this was posted but I couldnt find a good solution, so I wrote a kotlin helper function to solve it, append and call from your build.gradle.kts:

import java.io.BufferedReader
import java.io.File
import java.io.InputStreamReader

//your build.gradle.kts file...
fun Project.antlr(
        grammarFile: String,
        packageName: String = "$group.$name",
        outputDir: String = "$rootDir/src/gen/java",
        inputDir: String = "$rootDir/src/main/antlr"
) {
    val pkgPath = packageName.replace(".", "/")

    val compileOnlyDepFiles = configurations.compileOnly.get().files
    project.logger.debug("checking for antlr4 ($antlrToolJarRegex) configuartions.compileOnly.files: $compileOnlyDepFiles")
    val antlrJars = compileOnlyDepFiles.filter { it.name.matches(antlrToolJarRegex) }
    project.logger.info("found antlr jars: ${antlrJars.joinToString("\n","\n")}")

    val antlrJar = antlrJars.firstOrNull()

    if(antlrJar == null || ! antlrJar.exists()) { throw RuntimeException("failed to find antlr compiler tool $antlrJar") }

    File(outputDir).mkdirs()

    val cmd = arrayOf(
            "java", "-jar", antlrJar.absolutePath,
            "-encoding", "UTF-8",
            "-o", "$outputDir/$pkgPath",
            "-package", packageName,
            "-lib", outputDir,
            "$inputDir/$grammarFile"
    )

    println("exec ~= ${cmd.joinToString(" ")}")
    val exec = Runtime.getRuntime().exec(cmd)
    val messages = BufferedReader(InputStreamReader(exec.errorStream)).use { err ->
        generateSequence { err.readLine() }.toList()
    }
    val result = exec.waitFor()

    if (result != 0) {
        throw RuntimeException(
                """antlr exec ~= ${cmd.joinToString(" ")}
                  |failed with code=${result} because:
                  |    ${messages.joinToString("")}
                """.trimMargin()
        )
    }
}

usage:

dependencies {
    //...
    compileOnly("org.antlr:antlr4:4.8-1:complete")
}

tasks {
    register("generateLexer") {
        doLast {
            antlr("MyLexer.g4")
        }
    }
    register("generateParser") {
        doLast {
            antlr("MyParser.g4")
        }
    }
}
3

It’s sad that this is the solution.

I think the basic issue with the plugin as it stands seem to be that it compiles everything in the antlr source directory. Even if you are using imports you don’t want to compile each .g4 file, you only want to compile the “main” one. So I think a style of just being able to specify explicitly “compile this one file” seems much more sensible.

I suspect it would be possible to use the AntlrTask tasks directly. I’m far from a gradle expert though, and sadly need to use groovy rather than kotlin