Why doesn’t this work in gradle (using 1.4)? I tested with and without a daemon and got the same results. The threads are submitted to the pool and I see the starting message for each thread but I don’t see the finished message. The same code works fine in a standard groovy script.
import java.util.concurrent.*
task threads << {
def pool = Executors.newFixedThreadPool(10)
def defer = { closure -> pool.submit(closure as Callable) }
for (def i=0; i<10; i++) {
def j = i+1
defer {
println "Starting task ${j}"
Thread.sleep(j*1000)
println "Finished task ${j}"
}
}
pool.shutdown()
}
Probably because ‘println’ does not work after the build terminates.
Regardless why, appending 'pool.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS) ’ should solve your problem.
Hmm… I guess the question is, why is the build terminating if there are still running threads (which doesn’t seem to be the case either for a real world example). Is that the expected behavior?
I was aware of the awaitTermination method but was hoping to avoid having that because I didn’t want to put some arbitrary large enough value in there to ensure that it doesn’t timeout prematurely. I guess I can live with that if there are no other options.
Why shouldn’t it? Personally, I expect the build to terminate after all the tasks completed. For example, in a daemon, there are running threads after the build terminates (otherwise the daemon would stop). It is practically unfeasible for Gradle to keep track which thread you believe part of the build and which not.
The issue is that when the build terminates it ends up killing any threads that were spawned by the process. That doesn’t seem like normal behavior to me.
I don’t think it does (in a daemon, I would find that quite surprising). I believe that ‘println’ doesn’t work after the build terminates. That is, it is executed but does nothing (or throws an exception). But a Gradle dev. could verify that.
To test it you might try to write to a file instead of ‘println’ or something else what doesn’t depend on Gradle.
A file is exactly what I did to test it out. Here’s another example of what I have without using a thread pool. Maybe I’m doing something wrong. When I run this with the last line commented out, no files get created. If I uncomment the last line and let the gradle process run for a long enough time, all the files get created. I get the same result if I use the gradle daemon or not. If you could try this and see what results you get I would appreciate it.
task threads << {
for (def i=0; i<10; i++) {
def j = i+1
Thread.start {
Thread.sleep(j*1000)
File file = new File("./thread_${j}")
file << "hello world ${j}!\n"
}
}
//Thread.sleep(180000)
}
Gradle forcibly exits the JVM instead of waiting for all threads to finish. In practice this has proven to be the safest strategy. What is the core problem you are trying to solve here?
I tried your code writing to a file and using a daemon it did create the files.
Ok, let me step back a little. We have a project that contains all of our regression tests and these tests happen to use junit. There’s nothing I can really do in terms of how this project is setup. The existing ant script for this project orders each test suite/class according to how long the suite takes to run. It then runs each test class in this order in a single thread and has up to n threads running at a time. So at any given time it would be running let’s say 20 individual test classes in parallel. Now I have to implement that in gradle.
The first problem I had was to create a task/function in gradle that could execute a single test class that it would take in as a parameter (see http://forums.gradle.org/gradle/topics/executing_a_single_junit_test_suite_programmatically_from_a_task). I still don’t have that piece working properly. Anyway, with that I could just iterate over the tests in order and submit them to a thread pool of whatever size. This was the second problem because I wasn’t able to get the thread pool working as I expected either.
I think waiting for the threads of the executor is the smaller problem here. I mean, how would you expect the Gradle task to fail, if the task (Callable, Runnable, whatever) submitted to executor has failed? That is, even if Gradle waited for the threads of the executor, the build would succeed even if your asynchronous task fails.
I don’t expect it to fail. I’m ok with gradle returning success even though some test threads may have failed.
Then I think you should call:
gradle.buildFinished {
pool.shutdown()
pool.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS)
}
Yeah that seems to be the easiest workaround so I’ll go with that. Thanks for the help!