We are running a gradle build that during its testing phase accumulates thousands of open pipes, until eventually it fails with a “too many files open” IOException.
This is with gradle 5.4.1 on macOS 10.14.6 and 10.15, JDKs 11 and 12.
As far as I can tell, those pipes are a gradle thing, not created by our tests. Is this possible? Any idea of what could be the cause and/or how to make gradle close those pipes?
A bit more of puzzling detail: 1-2 year old MacBooks accumulate about 8000 such pipes during the build, which is surprising but actually harmless. The problem appears in newer MacBooks, in which the daemons accumulate over 10240 such pipes, which is the limit in the JVM and triggers the IOException.
This is how the output of “lsof” looks like:
…
java 99069 mija 4292u KQUEUE count=0, state=0x8
java 99069 mija 4293 PIPE 0xcefa84f33b5d7d39 16384 ->0xcefa84f33b5d7c79
java 99069 mija 4294 PIPE 0xcefa84f33b5d7c79 16384 ->0xcefa84f33b5d7d39
java 99069 mija 4295u KQUEUE count=0, state=0x8
java 99069 mija 4296 PIPE 0xcefa84f33b5d84b9 16384 ->0xcefa84f33b5d8579
java 99069 mija 4297 PIPE 0xcefa84f33b5d8579 16384 ->0xcefa84f33b5d84b9
…
Interestingly, the pipes seem to loop in pairs. Maybe this is why they don’t get closed…?
(I already reported this in Expire daemons when open file descriptors count reach a treshold · Issue #755 · gradle/gradle · GitHub, but not sure whether to consider this a but, or a missing feature, or whether this is gradle’s problem at all. I have also tried looking for documentation about the gradle daemon internals, but couldn’t find anything interesting.)