Why Gradle doesn’t provide “provided”?

EDIT May 10, 2016: Gradle 2.12 finally brings compileOnly dependency configuration for Java plugin (until then available only with WAR plugin). It does not model exactly what provided means in Maven, but covers most of the cases like using Java EE compile time dependencies in libraries, etc.

EDIT Jan 15, 2016: Gradle itself recently recommended nebula.provider-base plugin that introduces provided scope. I added build-nebula.gradle to the repo, check it out! Too bad I can’t link the resource, I simply cannot find it anymore, but the plugin works.

Honestly, I don’t know. I’ve been watching Gradle for around 3 years already, but except for primitive demos I didn’t have courage to switch to it. And – believe it or not – provided scope was the biggest practical obstacle in my case.

What is provided, anyway?

Ouch, now I got myself too. I know when to use it, or better said – I know with what kind of dependencies I use it. Use it with any dependency (mostly an API) that are provided (hence the name I guess :-)) at the runtime platform where your artifact will be run. Typical case is javax:javaee-api:7.0. You want to compile your classes that use various Java EE API. This one is kinda “javaee-all” and you can find separate dependencies for particular JSRs. But why not to make your life easier when you don’t pack this into your final artifact (WAR/EAR) anyway?

So it seems to be like compile (Maven’s default scope for dependencies) except that it should not be wrapped in WAR’s lib directory, right? I guess so, except that provided is not transitive, so you have to name it again and again, while compile dependencies are taken from upstream projects.

BTW: This is why I like writing blog posts – I have to make it clear to myself (sometimes not for the first time, of course). Maven’s dependency scopes are nicely described here.

But Gradle has provided!

Without being strict what Gradle is and what are its plugins, when you download Gradle, you can use this kind of scope – if you use ‘war’ plugin, just like in this simple example. If you want to run it (and other examples from this post), just try the following commands in git-bash (or adjust as necessary):

$ svn export https://github.com/virgo47/litterbin/trunk/demos/gradle-provided
$ cd gradle-provided
$ gradle -b build-war.gradle build

Works like a charm – but it’s WAR! Good thing is you can now really check that the provided dependency is not in the built WAR file, only Guava sits there in WEB-INF/lib directory. But often we just need JARs. Actually, when you modularize your project, you mostly work with JARs that are put together in a couple of final artifacts (WAR/EAR). That doesn’t mean you don’t need Java EE imports in these JARs – on the contrary.

So this providedCompile is dearly missed in Gradle’s java plugin. And we have to work around it.

Just Google it!

I tried. Too many results. Various results. Different solutions, different snippets. And nothing worked for me.

The main reason for my failures must have been the fact that I tried to apply various StackOverflow answers or blog advices into an existing project. I should have tried to create something super-simple first.

Recently I created my little “litterbin” project on GitHub. It contains any tests, demos or issue reproductions I need to share (mostly with my-later-self, or when I’m on a different computer). And today, finally, I tried to proof my latest “research” in provided scope – you can check various gradle.build files using vanilla aproach or propdeps plugins (read further). You can also “svn export” (download) the project as I showed higher and play with it.

My final result without using any fancy plugin is this:

apply plugin: 'maven'
apply plugin: 'java'
apply plugin: 'idea'

repositories {
    mavenCentral()
}

configurations {
    provided
}

sourceSets {
    main {
        compileClasspath += configurations.provided
        test.compileClasspath += configurations.provided
        test.runtimeClasspath += configurations.provided
    }
}

// if you use 'idea' plugin, otherwise fails with: Could not find method idea() for arguments...
idea {
    module {
        /*
         * If you omit [ ] around, it fails with: Cannot change configuration ':provided' after it has been resolved
         * This is due Gradle 2.x using Groovy 2.3 that does not allow += for single elements addition.
         * More: https://discuss.gradle.org/t/custom-provided-configuration-not-working-with-gradle-2-0-rc2-in-multi-project-mode/2459
         */
        scopes.PROVIDED.plus += [configurations.provided]
        downloadJavadoc = true
        downloadSources = true
    }
}

dependencies {
    compile 'com.google.guava:guava:17.0'
    provided 'javax:javaee-api:7.0'
}

In the comments you can see the potential problems.

With strictly contained proof-of-concept “project” I can finally be sure what works and what doesn’t. If it works here and doesn’t work when combined with something else, the problem is somewhere else (or in the interaction of various parts of the build). Before I always tried to migrate some multi-module build from Maven, and although I tried to do it incrementally, it simply got over my head when I wanted to tackle provided dependencies.

Just use something pre-cooked!

If you want provided scope you can also use something that just gives it to you. Spring Boot plugin does, for instance, but it may also add something you don’t want. In this StackOverflow answer it was suggested to use propdeps plugin managed by Spring. This just adds the scope you may want – and nothing else. Let’s try it! I went to the page and copied the snippets – the build looked like this:

apply plugin: 'maven'
apply plugin: 'java'
apply plugin: 'idea'

repositories {
    mavenCentral()
}

buildscript {
    repositories {
        maven { url 'http://repo.spring.io/plugins-release' }
    }
    dependencies {
        classpath 'org.springframework.build.gradle:propdeps-plugin:0.0.6'
    }
}

configure(allprojects) {
    apply plugin: 'propdeps'
    apply plugin: 'propdeps-maven'
    // following line causes Cannot change configuration ':provided' with Gradle 2.x (uses += without [ ] internally)
    apply plugin: 'propdeps-idea'
    apply plugin: 'propdeps-eclipse'
}

dependencies {
    compile 'com.google.guava:guava:17.0'
    provided 'javax:javaee-api:7.0'
}

As the added comment suggest, it wasn’t complete success. Without IDEA plugin and the section, it worked. But the error with the IDEA parts was this:

Cannot change configuration ':provided' after it has been resolved.

You google and eventually find this discussion, where the key message by Peter Niederwieser (core Gradle developer) is:

Gradle 2 updated to Groovy 2.3, which no longer supports the use of ‘+=’ for adding a single element to a collection. So instead of ‘scopes.PROVIDED.plus += configurations.provided’ it’s now ‘scopes.PROVIDED.plus += [configurations.provided]’.

Funny part is, that it is actually fixed in the spring-projects/gradle-plugins version 0.0.7, they have just forgotten to update the examples in the README. 🙂 So yeah, with 0.0.7 instead of 0.0.6 in the example, it works fine.

How can this stop you?

Maybe provided scope is not that trivial. Scope is actually not the right word in Gradle world, but my mentality and vocabulary is rooted in Maven world after all the years. If provided was obvious and easy they’d probably resolve this never ending story already. Now the issue is polluted with advocates for the scope (yeah, I didn’t resist either) and it’s difficult to understand what the problem is on the side of the Gradle team, except it seems they’re just ignoring it for a couple of years.

Original reporter claimed it doesn’t make sense to stay with Maven for this – and he is right. He is also right that many developers don’t understand how Configuration works (true for me as well) and how it relates to ClassLoader (true again). I’ve read some Gradle book and read many parts of the manual, trouble is that my problems were always about migrating existing Maven builds. Not big ones, but definitely multi-module with provided dependencies. And it really is not easy from this position.

I successfully used Gradle for one-time projects, demos, etc. Every time I try to learn something new about it. I acknowledge that building domain is a hard domain. Gradle has good documentation, but it doesn’t mean it’s always easy to find the right recipe. I never worked with a team where someone was dedicated for this task and I was mostly (and sadly) the best learned member when it came to builds with tons of other stuff on my hands. (Sorry for rant. It springs from the fact that builds are considered secondary matter, or worse. And there is too much primary concerns anyway.)

When one doesn’t know how to get to “provided” scope – that was available “for free” in Maven – any obstacle seems much bigger than it really is. There is simply too much we don’t know when we tackle the Gradle the first time. Nobody tells you “don’t use propdeps-plugin:0.0.6, try 0.0.7 instead”.

Or you get Gradle like message “Cannot change configuration ‘:provided’ after it has been resolved” which is probably perfectly OK from Gradle point of view – it nicely covers underlying technology. But it also covers the root cause that Groovy 2.3 simply doesn’t support += without wrapping the right side into […] – and even that only in some cases:

// correct line, but fails without [ ]
idea { module { scopes.PROVIDED.plus += [configurations.provided] }}

Even –stacktrace –debug will not help you to find the root cause. Maybe if you’d debug the build in IDE, but I’m definitely not there yet (not with Gradle, I debug Maven builds sometimes).

I hope you can now appreciate how subtle the whole problem is and how much difficulty it may cause.

provided or providedCompile?

And that is another trick – people call it differently. “providedCompile” is probably more Gradle-like (and available with war plugin), “provided” is what we are used to from Maven. Now imagine you experiment with various solutions how to introduce this kind of scope – that is you test different plugins. And all these call it differently. Every time you have to go to your dependency list and fix it there, or wonder why it doesn’t work when you forget. It just adds to the chaos when you already navigate unknown territory.

And it also nicely underlines the fact how much it is missing for java plugin out of the box. Because “it is supported in ‘war’ plugin” is not satisfactory answer. I want to use Java EE imports in my JAR that may be later put to WAR. Or I may run it in embedded container that will be declared with different dependencies. “This mostly affects only library developers” is also not true. Sure, it affects my Java Simon (which is a library), but I used provided scope for JAR modules on every single project in my past.

Now imagine this is your first battle with Gradle (which more or less was in my case). How should I be confident about releasing to Maven Central? It reportedly works, but then, for experienced Gradle users everything is easy…

Conclusion

During my research I found also the article Provided Scope in Gradle. I don’t know how accurate it is for Gradle 2.x or whether Android guys didn’t solve it already somehow. Author added nice pictures and also started with “What is provided anyway?” question (I swear it was a natural choice for my first subheader too :-)). And again it just shows how much complicated Gradle builds are when it’s not available out of the box.

It doesn’t mean I don’t want to try to get to Gradle build. I don’t like Maven’s rigidity – although I appreciate the conventions and I’ll follow those with my Gradle builds too. But sometimes you just want to switch something from false to true – and it takes 10 XML lines. You may say, meh! But it means you see less on the screen, builds are not readable, etc. And we already agreed, I hope, that building is a (potentially) complex domain. Readability is a must.

Sure there is something about polyglot Maven, but there is still also the issue with the lack of flexibility. I’m absolutely convinced that Gradle is the way to go. I tried it for simple things and I liked it, and I have no doubt I’ll learn it well enough to master bigger builds too.

Hopefully, provided will not be problem anymore. 🙂

Happy New Year 2013!

Happy New year, of course! My last year was a bit poorer blog-wise. For some reasons I was more lazy to write about things. Heck, sometimes I think that I was less lucky with new technology in overall. I achieved some nice results with testing in our company during the previous year. This year I wanted to push Continuous Integration, testing a bit further, maybe Gradle – but results in CI area are mixed and the rest brought no real results at all.

On the brighter side, I managed to finish my quest for system time shifter on JVM that would be usable for testing purposes – all documented in my post. Blogging is not all of course and I am quite happy how topics around Clean Code got some attention around me. We pushed Java Simon project a bit further too, I learned a few interesting things around Spring, MVC and jQuery… Add this beautiful Scala class on Coursera and this year was more than fun after all.

Still I’d like to make some resolutions. I discovered QueryDSL (thanks to a colleague of mine) and this seems to be answer to readable and compile time safe Criteria – because those shipped with JPA2 are simply horrible to read. It works well with IDEA’s annotation processor, Maven and it should be no problem with Gradle either. Ah, Gradle! For around two years I’m watching this guy but for whatever reason I was not able to use it for anything more than a few tests – but that is not Gradle’s fault. I like it, I like the idea, I like the language – and I think this year is time to switch Java Simon from Maven to Gradle. And after that I’ll go on with projects in our company, although the battle there will be more difficult I guess.

Out of technology, I managed to put together a few songs with my colleagues and it was fun – the first time I played in something close to a band. We played only on our company party but it doesn’t change anything… it was a real fun. We didn’t have a drummer so I used my Native Instruments Maschine Mikro and pre-programmed our songs – and I was really happy with the results. I’ll probably dedicate a post to Maschine Mikro, because it is one really interesting controller (and software too!).


Maschine Mikro controller

Talking about music, I managed to upload two full-blown tracks to my Soundcloud and later added two simple guitar+voice tracks. While mixing/mastering is still my weakness, I’m happy that I was able to pull through this recording-wise. And just how I imagined – my songs composed with paper, pen and acoustic guitar many years ago can really work as rock recording too.

So what about this year and those resolutions? Gradle – sure. More testing methodology on our projects – maybe I’ll even manage to document it here on the blog. Pushing Continuous delivery just a bit further again. Scala or other JVM language? I don’t know. Maybe, maybe for tests. And a bit of my music – I need to practice more with keyboard, guitar and bass guitar (yeah, I bought lovely Yamaha bass too).


Bass guitar Yamaha RBX375

Last resolution is no resolution at all – we have to survive somehow “socialistic” experiments of our government here in Slovakia (although there is nothing social about them). Europe has its own deal of problems – and USA? Well they saved themselves from falling down that fiscal cliff or what – just a few hours ago. And it probably means to make the cliff a bit higher for the next time. So we might have escaped one Doom’s day lately at the end of 2012, but who knows how our civilization will fare in the future.

Then I remember those really poor and I know we have nothing really horrible to complain about. So once again – Happy New year – and whole year of 2013!

Honestly… I hate Maven

And I don’t give a damn that I don’t know it good enough. Why “good enough” in Maven is so difficult when it was so easy with Ant? I remember how we came from “make” to Ant for our projects. I remember what we tried with Ant. Sometimes we failed when we wanted too much.

And then I remember trying Maven. The Next Big Thing (was it 2005? sooner?), revolution in builds (and dependency management, and… everything, right?), and probably the next best thing after wheel. So I tried it. Maybe I was one day from our final goal, maybe just an hour. But I eventually gave up. I failed. Maybe I was just plain stupid. Or Maven too smart.

I did my best to forget about it when I was asked to provide Java Simon in some Maven repository. It was pain again. Not just to restructure our modules, but to understand that magic. And deployment. And plugins. And dependencies, tons of documentation. Maybe Maven makes complex things simpler. But Maven also makes simple things complex. And then repository of my choice changed their configuration and I decided to move on to Maven central.

Documentation again, javadoc generation (still have to figure this out), …a lot of learning for such an obvious goal. Because people want everything in Maven repository. Understandably of course. I, too, want our library to be used – Maven is our standard, our salvation.

Yes, I don’t understand Maven. I understand the concept, but I don’t understand why it has to be so complicated when one needs something very simple. Why things just don’t work. The whole infrastructure around Maven is crazy. If something isn’t right with build most of my colleagues just try to ignore the problem because they don’t want to mess with Maven. Yet we use it.

Recently I checked Gradle. It starts where Maven ended. I switched one of my older projects from Ant to Gradle. I had to do these things to do so:

  • switch structure to Maven-like POM-compliant structure.
  • call Ant’s native2ascii target (Gradle has simple facility to do that), because my project have resource bundles in ISO Latin 2.
  • and… that was it!

It was just so much more satisfying. Second step was a bit troublesome, but I was just happy when it all worked and my build file was just a few lines long. I also noticed that Gradle offers not only declarative approach, but you can say what and how you want things done when you need it. Right now I’m not doing any further research, but I know I will carry on with Gradle later when necessary.

Right now I have some Maven work to do. And I’m biased, I know it, I’m also frustrated and it all came to me – and I know that I just hate Maven. Not because it is bad – I actually don’t care. But because it’s everywhere, like a plague, it’s too complex (is parent + six sub-modules so difficult to comprehend? yes, with Maven) and you have to live with it if you want to offer anything that looks like library to other people. And worst of all you have to follow tons of additional rules when you need the stuff hosted somewhere. Maybe it’s necessary evil – but still, evil it is. There is no beauty, there is no elegance, there is just… POM. And XML, of course.