That’s certainly an aspect of the Gradle cache we’d like to optimize. It would make sense to delete caches of older (and potentially unused) Gradle versions after a while. The same is true for outdated SNAPSHOT versions. At the moment there’s no way to tell Gradle to delete old dependency artifacts from the cache.
It would certainly be great if we could have the functionality in Gradle core. If you are interested in contributing, a good place to start would a discussion on the Gradle dev list.