Using ANTLR plugin to process separate lexer and parser grammars

This is a follow up question to Using Gradle 2.10's ANTLR plugin to import an ANTLR 4 lexer grammar into another grammar
As suggested by @st_oehme I opened a new topic.

I took @jbduncan’s project and modified it to match my use case (see attachment). The difference is that in the original project the lexer grammar was imported as a grammar, while I just want to use separate lexer and parser grammars.

It works fine when the generated CommonLexerRules.tokens is copied to src/main/antlr/org/jbduncan but otherwise fails with

error(160): org/jbduncan/Expr.g4:4:11: cannot find tokens file /home/henry/tmp/antlr/antlr-example/src/generated/java/CommonLexerRules.tokens

antlr-example.zip (55.1 KB)

1 Like

I know its been 4 years since this was posted but I couldnt find a good solution, so I wrote a kotlin helper function to solve it, append and call from your build.gradle.kts:

import java.io.BufferedReader
import java.io.File
import java.io.InputStreamReader

//your build.gradle.kts file...
fun Project.antlr(
        grammarFile: String,
        packageName: String = "$group.$name",
        outputDir: String = "$rootDir/src/gen/java",
        inputDir: String = "$rootDir/src/main/antlr"
) {
    val pkgPath = packageName.replace(".", "/")

    val compileOnlyDepFiles = configurations.compileOnly.get().files
    project.logger.debug("checking for antlr4 ($antlrToolJarRegex) configuartions.compileOnly.files: $compileOnlyDepFiles")
    val antlrJars = compileOnlyDepFiles.filter { it.name.matches(antlrToolJarRegex) }
    project.logger.info("found antlr jars: ${antlrJars.joinToString("\n","\n")}")

    val antlrJar = antlrJars.firstOrNull()

    if(antlrJar == null || ! antlrJar.exists()) { throw RuntimeException("failed to find antlr compiler tool $antlrJar") }

    File(outputDir).mkdirs()

    val cmd = arrayOf(
            "java", "-jar", antlrJar.absolutePath,
            "-encoding", "UTF-8",
            "-o", "$outputDir/$pkgPath",
            "-package", packageName,
            "-lib", outputDir,
            "$inputDir/$grammarFile"
    )

    println("exec ~= ${cmd.joinToString(" ")}")
    val exec = Runtime.getRuntime().exec(cmd)
    val messages = BufferedReader(InputStreamReader(exec.errorStream)).use { err ->
        generateSequence { err.readLine() }.toList()
    }
    val result = exec.waitFor()

    if (result != 0) {
        throw RuntimeException(
                """antlr exec ~= ${cmd.joinToString(" ")}
                  |failed with code=${result} because:
                  |    ${messages.joinToString("")}
                """.trimMargin()
        )
    }
}

usage:

dependencies {
    //...
    compileOnly("org.antlr:antlr4:4.8-1:complete")
}

tasks {
    register("generateLexer") {
        doLast {
            antlr("MyLexer.g4")
        }
    }
    register("generateParser") {
        doLast {
            antlr("MyParser.g4")
        }
    }
}