Gradle fails to compile complex antlr4 grammar file structure


I’m having trouble compiling a specific set of antlr4 files using the antlr plugin in gradle 4.8.1/4.9. To reproduce, I’ve created a minimal repository:

The build.gradle file contains some of the config snippets I have tried in comments with their associated errors. I believe that part of the problem may lie within the antlr tooling itself so I opened an issue on antlr that will probably need some clarification from our exploration here.

The problem in a nutshell is that I can get one set of errors or another depending on settings:

When using defaults or manually specifying “-lib src/main/antlr”, FooLexer.g4 is unable to import FooTokens.g4 which is in the same directory:

  1. com/example/foo/FooLexer.g4:3:7: can’t find or load grammar FooTokens

If I specify “-lib src/main/antlr/com/example/foo” then FooLexer.g4 can resolve FooTokens but then Foo.g4 can’t resolve FooLexer:

  1. com/example/foo/Foo.g4:3:21: cannot find tokens file com/example/foo/FooLexer.tokens

There is a subtly different import mechanism at play here. Foo.g4 uses “options { tokenVocab=FooLexer; }” while FooLexer.g4 uses “import FooTokens;”

The first one (tokenVocab) relies on being able to find compiled products (FooLexer.tokens) which go into the generated-src directory while the import mechanism relies on parsing the other source files directly. Since the -lib argument overrides where antlr is looking and I don’t see a way to tell antlr to look in two places, we hit this behavior.

Additionally, if I set the -lib argument as the generated-src directory then FooLexer still can’t resolve FooTokens. It seems that I am caught in a catch 22: I can either import FooTokens or I can reference FooLexer, but not both in the same gradle configuration.

The real pain is that the antlr4 cli technically produces code in this situation but only if you construct the command very carefully. There is an example of this in the build.gradle file as well.

Anyway, if there are any hints on how I might proceed here or if someone is willing to chime in on the antlr issue, I would appreciate it.