Siberia Elite Prism – my last buy from SteelSeries

And it’s actually also my first one. I wanted to replace my older headset and I chose SteelSeries Siberia Elite Prism – the white one, but that doesn’t really matter. It costed me 160 EUR and first I felt excited. Ok, so you actually buy headset and pay also for fancy led lights on them. Reportedly up to 16M colour, which obviously is b/s, because the fact that you can control something with 24 bits doesn’t mean there are really 16M colours. But you don’t see them when you play, do you?

Then you pay also for funny led light on the mic, that tells you when it’s muted. Some say it should be the other way around, but as I expect mic normally on, I think it makes more sense to indicate muted state. But both these led thingies work only when you drive the headset through their USB audio card – that, conveniently, is provided. And here we come to my problems.

Their software sucks

It’s called SteelSeries Engine 3 and it caused crashes related to audio playback on my Lenovo notebook. Actually the whole system went completely dead, except I could have send it to sleep with a power button and then it awoke in locked screen running for 10 more minutes or so if you listened to the audio through their card. On the desktop there was no problem – again luckily, because I disabled the power button and I’d just have to hard-reset it. It seems that without Engine started it worked better. Or maybe it had to start as administrator (why software that requires it doesn’t say so? I don’t know then!)… or maybe it was the driver.

After some time I managed to listen to the music without interruptions but I don’t know how and why. And because I didn’t use it that much on the notebook, I can’t elaborate on the problem. Let’s go to my desktop then.

I want to plug both headset and speakers

And you can! Their audio card offers the special connector for the headset. This – obviously – makes the headset kinda useless if the cable breaks after the warranty. I can solder new jack on an audio cable, but here you’re out of luck. In this price level removable and replaceable cable may be expected.

But their audio card also contains audio jacks for microphone and speakers. Great! So I thought. My setup is plain silly simple – I use my headphones and my speakers! Yes, sometimes even both. When I have a Skype call I leave the speakers on a bit for the kids to listen what grandma is saying and I use the headset. But to my shock, grandma can’t hear me! What-the… we tried this particular headset already! Microphone is detected, it just doesn’t pick any sound.

After some messing around I disconnected the speakers and voila – microphone works…

The really designed it this way?!

If it was common jack for speakers and mic (TRRS), I’d be inclined to understand. But how hard can it be to split the sound? Not at all – and they do it! Except they turn off the microphone that is completely unrelated to the output path.

I thought it silly, so I contacted their support. They responded within two days, they can reproduce the issue (half the solution, right? :-)) and they will ask the technical team. I expressed my patience and waited.

We solve tickets, not problems

After seven days my ticket got closed. I’m just a customer and not expert on their Zendesk setup, so I was rather surprised. “Hey, c’mon, you didn’t resolve my problem! I want to know what is happening!” I opened a follow-up expressing my dissatisfaction being treated this way. These support people, or anyone who sets up the workflow, absolutely ignores how things really work. They want to get rid of the problem, get rid of your ticket (and maybe even you…). For them, ticket is the problem, not the real cause of the problem.

I know this mentality because I got familiar with it in one of my previous employments. KPIs are set around how long you have the problem (read “ticket”) open, you get rid of the problem (read “ticket”) and the customer (even the one from the same company) has a couple of days to express their satisfaction. But how this solves the problems that occur when I do something I do once a month? My previous ticket is closed, I didn’t express my satisfaction, I open the new one, they can’t reproduce it, ticket gets closed and the cycle repeats.

Actually it’s not only about the things I alone discovered and felt I need to report them whatever the cost (although my one of my mottos is “if you don’t tell, nobody can care”). There were notorious problems everybody knew abound and the company was searching for ways, heck – processes even! – how to deal with them. “We’re good in resolving incidents, but we should get better at solving the problems.”

Like, really? Isn’t starting to do something about the problems good enough? Why we are creating meta-problem (that is a new process for it) instead of trying some involved hard work. Maybe the problem would be resolved within hours. But I know, I know… first we have to think through all the KPIs. How to measure the productivity of this problem solving. Oh, c’mon…

It really is designed that way

SteelSeries support came back to me within two days after I opened the follow-up ticket. And I learned that they are sorry for the automatic ticket closing (maybe other people don’t care I guess) and that the hardware of their USB audio is designed this way. Yes, even if it doesn’t make any sense.

So the problem is simple – if I’m receiving Skype call, I have to crawl under the desk quickly, disconnect the speaker jack, and get back up – and I can chat. Easy. What are the alternatives? Let’s say I’ll put the USB sound card on the desk, so I can easily pull out the jack within seconds. Actually – that’s how I have it now, so I can share the experiences. You may expect some inconvenience, “yeah, the guy needs to lead the cables the way he didn’t plan to”, but there’s more than that.

Putting USB audio card on the extension

You need to USB extension cable. And that may be problem. I don’t have any ultra high quality shielded USB extension cable. And I don’t know how much it would help. The cable I used brought me back to my student’s times when any mouse movement created buzz in the speakers whose cable went around mouse’s USB cable. Here I use much better cable for the speaker, but the buzz somehow gets stronger as I add more USB length between the USB audio and the computer. Funny enough, it does not translate to the headset.

Another problem is with the jack in the USB card. It doesn’t feel very robust really. You touch the cable and it cracks and creates noise. When you don’t move the card and just bend the cable, it’s alright (it’s a new cable and I will not bend it any more, at last not on purpose :-)), so it really is inside the USB card. Jacks on cards are susceptible to this and if I could, I’d not touch it. But here I have to plug/unplug it regularly. I’ll see how long the audio card will last.

But in any case the noise in the speakers is my biggest trouble now. I have to turn the volume all the way down and live with funny noises between the songs. And yes, it really is that annoying and that audible.

Or just use jacks and soundcard in the computer!

And that is actually my last backup option. I can use provided jack converter, I’ll split the sound into headset and speakers with hardware Y splitter and I’m done. That renders the USB card useless, it will be another fancy thing in the box I didn’t need. With this solution you may miss: 1) colours on the headphones (probably not), 2) muted led indication on the mic (that one is handy but not a show stopper), 3) noise cancelling for the mic that works in tandem with the card. And this last thing sounds useful although I don’t know how well it actually works.

Reportedly mic muting and volume control rings on the headset both work even without USB sound card. And both are actually not just cool, but also practical things.

Anything positive?

It’s really hard to say something positive now. I don’t know how much is the headset and its sound quality worth for me. If Windows could route the sounds through two soundcards, my speaker setup would be saved. But Windows and sound routing is incredibly inflexible – I found out when I wanted something so primitive like stereo reversing, because my cable setup forces me to switch my active monitors. Now I switch left to right using cable with jack on one side and RCA connectors on the other – and these I can just switch easily. But back to the headset.

Sound is definitely better than any headphones I had home until now – but then, none of them costed more than 40 EUR or so.

It’s not like the whole headset is completely useless, but I’m frustrated when someone creates something fancy with such an incredibly stupid design flaw. Am I the only one who want’s to drive both speakers and headphones from one sound card? The additional jack absolutely begs me to do it this way – and it works! Just… it disables the microphone.

I’d say 1/5 stars would be harsh and unfair rating (although I feel like that), but 2/5 is just spot on when I consider the software quality, useless fancy things and this design bug. If you check for other people’s problems with SteelSeries, the pattern is very similar. Sometimes it just doesn’t work for you (I mean altogether, not like my case) and support can’t help you. I know it just happens, PCs vary, etc. But the more they complicate their products (and 16M colours of your ears are waste of effort really) the more problems one can expect.

I don’t know yet what my next headset will be, but it will not be SteelSeries for sure. Though I hope this one will last reasonably long. And I hope I’ll find no more design problems.

IntelliJ IDEA – subscription renewed

It has been over 16 months now since JetBrains presented their IntelliJ IDEA Personal Licensing Changes. The discussion under the post speaks for itself – there was hardly anybody who really liked it. Before you paid 100% when you bought it the first time (not to mention sales they offered ;-)) and 50% anytime you bought an upgrade. That means, you skipped major version or two (roughly a year each) and then bought the one you liked/needed for an upgrade price of 50%.

Now you buy it the first time for 100% and every next year you pay 50% without worrying anymore. Or you don’t renew your subscription and buy next year for 75% as a returning customer. That is like 50% + 50% of it. Long story short – version skipping is now more expensive, whatever your reasons for skipping are.

New model is not all bad…

One good point is that nobody can now complain that they bought version X and the next day you can get X and upcoming major version X+1 for the same price. What can still happen is that the next major version will be released the day after your subscription ran out. Which is more or less the same problem actually, except that now you can make decision that costs you with more information (before you had to be an oracle).

Again – long story short – if all customers stayed and all subscribed, JetBrains will have more money for their (and our favourite!) IDE. I guess there are ever more features, broader scope of problems, and whatnot. Price difference for every version skipping would be 50€ here in Slovakia. Nothing terrible actually.

JetBrains defence of this step was based also on possibility to release more often, even major versions. With IDEA 13 coming on Dec 3rd, 2013 and IDEA 14 on Nov 5th, 2014, we can’t say they lied, but it’s far from radical. And the question is whether it shows in reality, not just on the splash-screen.

…but I can’t see any big change

So that’s how I feel. I more or less (rather less actually) agreed to continuing partnership with the producer of my favourite IDE. It costs a little bit more, obviously there are no sales you can speculate on, etc. Business. But then, it’s not really anything that would ruin me and it’s worth the productivity you get, especially when you’re used to it. There is still Community Edition, if that’s all you need. And man, if you need just Java, version control and build tool support, it’s just incredible.

I wasn’t sure what to imagine regarding potentially faster release cycle and we’ve got just that – nothing radical, no harm done. Some versions are better, some worse, fresh 14.1 definitely needs some refinement as sometimes it stops checking files and finding usages altogether, but it restarts quickly and I hope it will be fixed soon.

What I miss

If I could channel IDEA developer’s energy into any particular area it would be fixing already existing Youtrack issues in general. I was rather a vigorous reporter – sometimes even successful. (Did you know you can copy the error message from status bar clicking on it with right mouse button? How handy is that when you want to paste it into a bug report!) But there are open issues that are probably obsolete already, some cleanup would be great. “Forgotten” issues are no good.

I remember how long it took to bring reasonable javadoc formatting into IDEA – and it still lacks here and there, although it was postponed one or two major versions. These are the things were I’d like to see more effort. But I understand there are customers waiting for support of their favourite framework as well.

Final words

So that’s it. Not a big change in the price, especially if IDEA is your primary axe, or whatever you like as a metaphor for a sharp tool (of course we can talk about tens of percent, but really…). Perceived quality is good, although varying – like anytime before. No tens of percent change there. :-) But anytime I see my colleagues struggling with something in Netbeans (“you can’t run normal main method from test sources?!”) or Eclipse (SVN and Maven working normally already?) I know that IDEA is still worth it. Although some people should learn their IDEs in the first place, whatever they use. Sometimes what I see is like a woodcutter beating the tree with the axe instead of cutting it – when we used that metaphor before. But that’s beyond the scope of this post.

Jetty hardening

Hardening may be a bit of stretch here, so before you spent your time with the post, I’ll tell you what we’re going to do here:

  • Disable directory browsing (definitely security hole) in the web application context.
  • Disable default Jetty error messages – both in application context and out of it (if you’re not running application in the root context).
  • And a little shutdown hook that helps your application lifecycle in case of unexpected process termination (covers break/ctrl+C, not kill -9 obviously :-)).

This all applies to Jetty 9.2.x, I’m pretty sure these things change slightly between major Jetty versions, so you may need to adjust it for versions 8 or 7.

This is a compilation of some research on the internet and actual Jetty debugging, applied to our Spring based web application run under Jetty (and wrapped with Launch4J) and if you know how to do things better I’ll gladly hear about it in comments of course. It is also loose continuation of our previous parts:

So, let’s see some code!

The result

Ok, maybe not the best story-telling, but let’s see the actual result here first. I left also lines with logging, we can argue about log levels, but that’s not the point here. Because it’s a longer listing with all the features from previous post, I highlighted the lines that are related to today’s post:

import java.io.IOException;
import java.nio.file.DirectoryStream;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.security.ProtectionDomain;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.eclipse.jetty.server.Request;
import org.eclipse.jetty.server.Server;
import org.eclipse.jetty.servlet.ErrorPageErrorHandler;
import org.eclipse.jetty.webapp.WebAppContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * Port is customizable using -Dport=8181 - default is 8080.
 * Application context is customizable using -Dcontext=xxx - default is "". Initial slash is
 * always prepended internally after any leading or trailing slashes (/) are discarded from the provided value,
 * so both -Dcontext= and -Dcontext=/ define root context.
 */
public class RestMain {

    private static final Logger log = LoggerFactory.getLogger(RestMain.class);

    public static void main(String[] args) throws Exception {
        int port = Integer.parseInt(System.getProperty("port", "8080"));
        String contextPath = System.getProperty("context", "");

        log.debug("Going to start web server on port {} with context path {}", port, contextPath);
        Server server = new Server(port);

        WebAppContext context = new WebAppContext();
        context.setServer(server);
        context.setContextPath('/' + contextPath);
        context.setInitParameter("org.eclipse.jetty.servlet.Default.dirAllowed", "false");
        context.setErrorHandler(new ErrorHandler());

        ProtectionDomain protectionDomain = RestMain.class.getProtectionDomain();
        String warPath = protectionDomain.getCodeSource().getLocation().toExternalForm();
        if (warPath.toLowerCase().endsWith(".exe")) {
            warPath = prepareWarPathFromExe(warPath, "WEB-INF");
        } // else we assume dir or jar/war
        context.setWar(warPath);
        server.setHandler(context);
        log.debug("WebAppContext set for server with location {}", warPath);

        // default error handler for resources out of "context" scope
        server.addBean(new ErrorHandler());

        server.start();
        if (!context.isAvailable()) {
            //noinspection ThrowableResultOfMethodCallIgnored
            log.error("Application did NOT started properly: {}", context.getUnavailableException().toString());
            server.stop();
        } else if (context.getWebInf() == null) {
            log.error("Application was NOT FOUND");
            server.stop();
        } else {
            log.debug("Application READY");
            addJettyShutdownHook(server);
        }
        server.join();
        log.debug("Exiting application");
    }

    private static void addJettyShutdownHook(final Server server) {
        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
                try {
                    log.debug("Exiting application (shutdown hook)");
                    server.stop();
                } catch (Exception e) {
                    log.warn("Exception during server stop in shutdown hook", e);
                }
            }
        });
    }

    private static String prepareWarPathFromExe(String pathToExe, String... prefixes) throws IOException {
        Path tmpWarDir = Files.createTempDirectory("restmod");
        final String warPath = tmpWarDir.toString();
        log.debug("Extracting WAR from EXE into {}, prefixes {}", warPath, prefixes);

        WarExploder warExploder = new WarExploder(pathToExe, warPath);
        warExploder.explode(prefixes);

        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
                try {
                    deleteRecursive(Paths.get(warPath));
                    log.debug("Temporary WAR directory deleted");
                } catch (IOException e) {
                    log.warn("Problems with deleting temporary directory", e);
                }
            }
        });
        return warPath;
    }

    private static void deleteRecursive(Path dir) throws IOException {
        if (Files.isDirectory(dir)) {
            try (DirectoryStream<Path> directoryStream = Files.newDirectoryStream(dir)) {
                for (Path path : directoryStream) {
                    deleteRecursive(path);
                }
            }
        }
        Files.delete(dir);
    }

    /**
     * Dummy error handler that disables any error pages or jetty related messages and returns our
     * ERROR status JSON with plain HTTP status instead. All original error messages (from our code) are preserved
     * as they are not handled by this code.
     */
    static class ErrorHandler extends ErrorPageErrorHandler {
        @Override
        public void handle(String target, Request baseRequest, HttpServletRequest request, HttpServletResponse response) throws IOException {
            response.getWriter()
                .append("{\"status\":\"ERROR\",\"message\":\"HTTP ")
                .append(String.valueOf(response.getStatus()))
                .append("\"}");
        }
    }
}

Now we can walk through these parts.

Directory browsing

This was the easy part after we accidently found out it is allowed. Just add this line to your WebAppContext:

context.setInitParameter("org.eclipse.jetty.servlet.Default.dirAllowed", "false");

We are running REST-like API under some context path and if we comment out this line, Jetty will nicely allow us to browse the directory (WEB-INF included). This line ensures that HTTP 403 Forbidden will be returned instead. How this status is treated…

Error handling in your application

There are errors you can treat in your application – these are not problem. And then there are cases that somehow slip out, there is no reasonable way how to intercept them and server displays some ugly error for it, introducing itself to the user completely. If I run application on http://localhost:8080/xxx and I hit that URL, I get the following output:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<title>Error 404 Not Found</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /finrisk/easd. Reason:
<pre>    Not Found</pre></p><hr><i><small>Powered by Jetty://</small></i><hr/>

</body>
</html>

That is not bad, but I’d like short JSON we use for error messages. This can be customized in web.xml:

<!-- somewhere at the end of web.xml →
<error-page>
    <location>/api/0.1/error</location>
</error-page>

This will point the application to our error resource and this is returned:

{"status":"ERROR","message":"HTTP 404"}

Alternatively we can achieve the same with custom Jetty ErrorHandler as we defined it at the end of our JettyMain class:

context.setErrorHandler(new ErrorHandler());
...
static class ErrorHandler extends ErrorPageErrorHandler {
  @Override
  public void handle(String target, Request baseRequest, HttpServletRequest request, HttpServletResponse response) throws IOException {
    response.getWriter()
      .append("{\"status\":\"ERROR\",\"message\":\"HTTP ")
      .append(String.valueOf(response.getStatus()))
      .append("\"}");
  }
}

This effectively replaces your web.xml definition (if present) and if you debug your application you can confirm that with a breakpoint that would be reached without this jetty line and will not be when it’s present.

How about URLs out of webapp context?

Let’s now access some URL out of our application – like http://localhost:8080/yy – this will result in very similar default Jetty error page like we’ve seen already:

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /. Reason:
<pre>    Not Found</pre></p>
<hr /><i><small>Powered by Jetty://</small></i>
</body>
</html>

This is also easy to fix, because we can reuse our ErrorHandler – just add this line for your Jetty Server instance:

server.addBean(new ErrorHandler());

But are we really done?

Not so fast…

The trouble is that error handler (or error resource configured in web.xml) is only used when Jetty (or any servlet container) thinks there was an error. If you handled the exception some other way, then this ErrorHandler is not used – whatever HTTP status you send.

We use Jersey as our REST API provider where you can register ExceptionMapper for particular exception type (and all its subtypes). When this is triggered and you populate javax.ws.rs.core.Response with the output and set HTTP status, it will not trigger this error page. We handled the error once, no reason to do it twice.

But in case of Jersey there are exception mappers for JSON parse errors and these may leak some information. Let’s try any URL that expects JSON in POST with unpaired curly brace and you’ll get this error (going around your custom ErrorHandler or error page):

Unexpected end-of-input: expected close marker for OBJECT (from [Source: org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream@1986598; line: 1, column: 0])
 at [Source: org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream@1986598; line: 1, column: 3]

This is not acceptable, but in this case you have to fight the framework (Jersey) to unregister internal mappers and register yours to get back on track. But obviously, this is out of scope of this blog post. Just remember that we’re covering only unhandled errors and any other error/exception handlers has to be checked too.

Shutdown hook

Finally something straightforward and not Java EE at all. When your Jetty server starts and application (Java process) gets terminated somehow, it will not stop the Jetty server in a graceful manner. With it goes down our Spring application – and if you have components with @PreDestroy, these method will likely not get run.

To do better than this we utilize Java’s shutdown hooks. The code goes like this:

// after Jetty starts OK with our app running
addJettyShutdownHook(server);
...
private static void addJettyShutdownHook(final Server server) {
  Runtime.getRuntime().addShutdownHook(new Thread() {
    @Override
    public void run() {
      try {
        log.debug("Exiting application (shutdown hook)");
        server.stop();
      } catch (Exception e) {
        log.warn("Exception during server stop in shutdown hook", e);
      }
    }
  });
}

All we have to do is call server.stop(). Would finally block in the main method do the same? Probably – I’m too tired to try… but mainly – shutdown hook communicates “I’ll be run when the process goes down” much better I think.

Finally

There’s a lot more to do with Jetty – and I don’t know most of it. I’m not doing any SSL here or Jetty based authentication, but aforementioned little patches should help it a lot. In case of error handler vs error page within the webapp context – it’s your choice probably. If you don’t have error page yet, then it is the easy one. But unless you run the application on the root context (/) you definitely want to shut up Jetty’s default messages in production. Finally – shutdown hook will help the webapp going down nicely. This is twice so important if you have another container inside (like Spring). In any case, it makes our applications graceful, right? Who doesn’t want to have graceful applications? :-)

JPA joins without mapped attributes

OK, whoever knows it is possible, just stop reading right away (before my JPA rant starts). And congratulation you’ve learnt it, hopefully on time.

JPA is hard

Java Persistence API is not easy – let’s face it. Please. Because it is easy only for two kinds of programmers – those that know everything (again, congratulation) and those who think they know everything. Or something similar. While I’m all for minimizing complexity to make things easy, it is known fact that making things simple (that is non-complex) is hard. Everytime I hear “no problem, that’s easy” I know what I’ll get. Complex ball of mud that does not correspond to “easy” at all.

I’ve read parts of Hibernate reference, I’ve read Pro JPA 2/2.1 – and I’d love to read it all finally, but there is always something else to read too. And so while I was aching for solution how to get rid of some my @Many/OneToOne mappings from entities (because these are not lazy and can trigger too many selects), I thought I couldn’t because I’ll be unable to join in the direction of that association. I was wrong, but first…

Can’t you avoid triggered selects for *ToOne somehow?

Yes, you can. You can eagerly fetch that association. Let’s talk about these entities:

jpa-security-deleteI took them from my previous JPA post  (and you can ignore the Domain completely now) where I compared some of worse features of our beloved leading ORMs (and JPA implementations). I tackled *ToOne there as well, but only superficially so I don’t need to correct anything essential. Not that I have problem to correct my previous work and admit I was wrong and mistaken. Back to the topic…

A Security points to a Client (acting as a security issuer in our case). How can I populate table for a user with selected attributes from the security and its issuer as well?

  • The worst case (and still traditionally used!) – select Securities and don’t care. This will likely result in N+1 select (officially name does not reflect the cause and effect – as it is rather 1+N). This plain sucks. I really don’t know why mainstream ORM solutions still execute this as N+1 even when it’s eager (*ToOne is by default), but that just underlines… well, the state of ORM, obviously.
  • Fetch the relationship! Of course. It is bound to happen anyway, so let’s deal with it in one select. Does it mean we’re done in single select? If you have more *ToOne relations on your Security – or Client – then you’re not.
  • Fetch and select columns explicitly. This works, but you have to deal with Lists of arrays of Objects, or – in better case – Querydsl’s handy Tuple. Or you may use projection to DTO. Do this with 60 columns (maybe not for table row, but later for detail) and you’ll know the pain.

But yes, it is possible. The question is if it’s worth to use JPA if 80% of time you go around these problems. Depending on relationship optionality you need to use LEFT JOIN of course. “Use LEFT JOIN, Luke!” is my answer when another developer asks me questions like “when I want to display client’s fields, some rows just disappear from my results!” or similar with a bit of fun in it “count select returns different count than is the size of final result!” – because you don’t need to join for count unless you perform WHERE part on joined columns.

Summed up, it’s hard. Gone is the premise that you’ll enjoy working with some mapped objects. No, no, deal with Lists of Object[] or DTOs! It’s hard to say what is better – DTO in theory, but if the list of columns is changing a lot then it’s just another place you’ll have to change. If you still use vanilla JPA, consider Querydsl, seriously. Tuple may be lightweight DTO for you and on the other side you can get the stuff out using expression paths that are compile time safe – without the need to mess with accessors. (Old enough to remember this article? :-))

Dropping @ManyToOne annotation

(The same applies for @OneToOne where relevant.)

To write left join from Security to Client you can find virtually exclusively examples like this (Querydsl, but the same goes for JPQL):

QClient issuer = new QClient(“issuer”); // alias
List<Tuple> result = new JPAQuery().from(QSecurity.security)
  .leftJoin(QSecurity.security.issuer, issuer)
  .list(QSecurity.security.name, issuer.name);

That is, in left join you first state the path how to get to the issuer from the security and then you assign an alias to it (issuer). Sometimes you can live without aliases, but in general you need them for any deeper joins. Notice also how the leftJoin implies ON clause. This is logical and expected, that’s why we have the mapping there.

But thing I never realized (and decided to try it the first time today) is that you can just leftJoin to alias and add your ON explicitly – just like you would in SQL!

QClient issuer = new QClient(“issuer”); // alias
List<Tuple> result = new JPAQuery().from(QSecurity.security)
  .leftJoin(issuer).on(issuer.id.eq(QSecurity.security.issuerId))
  .list(QSecurity.security.name, issuer.name);

Obviously you have to have issuerId attribute mapped on Security – but that is probably some plain type like Integer. This will not trigger any additional select. BTW – if you really want, you can have dual mapping for the same column like this:

@Column(name = "issuer_id"<b>, insertable = false, updatable = false</b>)
private Integer issuerId;

@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "issuer_id")
private Client issuer;

Notice the insertable/updatable=false – this has to be present on one of those mappings. The question is on which one. If you save/update the entity a lot and issuer’s ID is good enough then move it to @JoinColumn annotation. But if you don’t need Client in most cases at all, remove it completely. Also notice how the code is lying to you about LAZY fetch. It is not. In theory it can be with some byte-code modified Client, but that is not portable anyway. If you absolutely need that attribute in Security class, you can still make it @Transient, fetch it and list it explicitly (here I show it for singleResult for brevity):

QClient issuer = new QClient(“issuer”); // alias
Tuple result = new JPAQuery().from(QSecurity.security)
  .leftJoin(issuer).on(issuer.id.eq(QSecurity.security.issuerId))
  .singleResult(QSecurity.security, issuer);
Security security = result.get(QSecurity.security);
security.setIssuer(result.get(issuer)); // sets transient field

No problem at all. Full control. Couple of lines longer. Probably way more performant on scale.

What if JPA allowed us to ignore relations?

If you use only id value that copies exactly FK in your entities then you’ll lose some of the ORM benefits – but as said you’ll also dump a lot of its burden that is in many cases unconsidered. Imagine you could specify mappings like this:

@Column(name = "issuer_id")
@ManyToOne(<b>targetEntity = Client.class</b>) // exists, but invalid when types don’t match
private Integer issuerId;

This way you’d say what the target entity is, generated metamodels could offer you joins, etc. I’m not going to evolve this idea at all because there are some obvious disadvantages. Your JOIN path would be security.issuerId, not issuer, so if you followed it to Client’s attributes it would read confusingly (assuming that when used as a path it would act as Client of course). Then maybe this:

@ManyToOne(fetch = <b>FetchType.JOIN_ONLY</b>) // does not exist
@JoinColumn(name = "issuer_id")
private Client issuer;

This would mean that unless you require JOIN explicitly in your select this field will NOT be populated. Actually, this would be best to combine with dual mapping as shown sooner. ID is easy to get without JOIN – and referenced entity would be insertable/updatable false. Otherwise semantics of save would be questionable. Does null mean that entity is removed and FK should be nulled? Or does it mean to ignore it? Should JPA ignore it you need that explicit issuerId field. If not ignored even trivial find/change single attribute/save (or let it happen automatically at the transaction commit) would delete your relation unexpectedly. You can add some dirty checking magic of course (“if the attribute changes then do that, otherwise don’t do anything”), but we’d just add to existing mess. So this definitely breaks “least surprise” principle. But with dual mapping that is already possible, I’d love to have this really lazy fetch type.

In case you wonder why LAZY currently does not work (without byte-code modification magic)… The answer is that issuer attribute should be null only if NULL in database. So if the FK is there, you need to instantiate some Client and if that supposes to be lazy, it has to be some kind of proxy to a real one that is lazily loaded when you actually touch (get something from) the issuer object. This is very easy with collections (*ToMany) and ORM’s can do that (although Eclipselink’s Vector is spectacularly wrong way to do it) – that’s why I’m so possessed with *ToOne problem. :-)

Is it still ORM?

Had dual mapping with possibility of ignoring the relationship (unless explicitly joined) worked, it would still be ORM as we know it. You’d have bigger control, single em.find would not trigger 47 selects (real story, seriously, and all traversing *ToOne relationships) and there would be minimal implications to JPA (although there are more competent to say so or otherwise).

Currently I plan to drop quite a lot of *ToOne mappings because:

  • There is no portable way to avoid the fetch when I know I don’t want it.
  • When I want it, I can join it myself anyway.
  • If I want to pick specific relation I can just find it by ID that is available on the entity (and then store it to @Transient field for example).

Am I still using it as ORM then? I’m dropping a lot of “mappings” of my “relations(hips)” from my “objects”. But I can still utilize a lot of JPA (and Querydsl). There still is persistence context, you can use L2 cache (if you want), and maybe you don’t have uber-cool mapping to your domain… but let’s face it. How often we really map it? How often we just mirror our databases?

Do you map @ManyToMany or do you map the association table with two IDs instead? Looking back to our picture up there – can you get all Domain IDs for known Security ID (or list of IDs)? Yes you can. In SQL by querying a single table. Eclipselink joins Security, SecurityDomain and Domain to get information that is clearly available on SecurityDomain table itself (not mapped explicitly). Is it so difficult to treat IDs properly? It probably is. How is the JPA for lowering the complexity?

Execution postponed

JPA adds tons of accidental complexity every time. Not first, but later. And virtually all teams working with it have no real JPA expert. I study JPA more than most people I know and it still keeps surprising me. Rarely in a good way. Left join using unrelated alias (at least from mapping point of view) and explicit ON is the good one. I don’t even know why I never tried it when I officially longed for it! I ranted: “Why don’t we have free joins on any numbers (PK/FK) just like SQL can do?” And nobody ever contradicted me. Most examples always start with path going from FROM entity, using alias only as their second argument (Querydsl, but JPQL examples are the same). We are slaves of our *ToOne relations and can’t get rid of undesired selects – and there is a simple way how to do it all the time (although I don’t know how it worked before JPA 2.1, but it probably did).

If you don’t care about *ToOne problem, no problem. If you do though – the treatment is:

  • Replace your *ToOne mapping with simple ID as a value attribute.
  • Change anyJoin(x.y, alias) to anyJoin(alias).on(x.yId.eq(alias.id)) – this is Querydsl example but it transforms to JPQL directly. List both entities in projection if you need it (maybe you just WHERE on it).
  • If you really want +1 select, do it with em.find yourself. That may actually use L2 cache (if it’s not in the persistence context already).

With this tiny speck of knowledge I’m backing off from my plans to get rid of JPA in our project (you can imagine what pain that would be). Now it gives me so-so enough control to contain the select explosions to required levels.

From HTC Wildfly to Samsung Galaxy S3 Mini

I bought my first smartphone (HTC Wildfire) in Dec 2010 and my second (Samsung Galaxy S3 Mini) in Jan 2015. So it’s roughly 4 years difference. The latter is a tad bigger (didn’t want too big anyway), display is much better (height from 320px to 800px), is significantly faster (HTC Wildfire was a sloth really :-)), is driven by newer Android (4.x compared to 2.x on HTC)… but I expected more. And as always – God/Devil is in the detail(s).

What does not work on new mobiles?

So what was wrong when I got together with my new Samsung pet? Many more things than I expected actually – and many of them would not come to me in the worst dreams:

  • No notification LED! Seriously. If you miss the call you find out only when you check it. It may vibrate when you pick it up, but no way to look at it and spot that something is missing. The same goes for weak battery or any other event that made HTC Wildfire blink its diode. Shame, really.
  • Funny ringtone starting quiet – reportedly this can be disabled on Galaxies with pocket ring option turned off, but this one is not available on my phone at all. Or you can get some app that resolves this, but only on rooted devices. Thank you…
  • Default Clock/Weather widget? Big and always goes to Weather. Every single person I asked would expect to go to alarm/stopwatch/timer application after touching time (EZ Weather widgets are nice replacement). After all these Android years maybe producers should do the things in similar fashion. This limited offering is a big (or rather large, with size 4×2) letdown rendering Samsung’s widget useless.
  • Lock button on the side is exposed a bit more than necessary – but if you accidently restart the phone in your pocket, you’ll get gentle vibration as a reminder. :-)
  • Samsung Kies! Are you kidding me? Where is USB mass storage? At least for flash card. Later I found out that while phone does not appear as a drive on my Windows 8.1, it can be browsed from This PC. (Sometimes I think Samsung should really stop trying to develop any software. Smart TV? People don’t want them? Not because they wouldn’t like the features, but they don’t like the actual execution! Sadly, this is probably rather management/strategic flaw than incapable developers. Waste of money in any case.)
  • Lastly the minor point – compared to HTC it takes more touches to start a call. Gesture starting a call when you are on a contact is helping a bit. But on HTC I got my history right on the first screen of phone application and it was one more touch to repeat a recent call.

After all this time I’d expect general Android environments being a bit further. Sure HTC has good clock+weather widget – is it patented or what? Is it so difficult to copy the good ideas? Or is being “different” so important even when it means being plain stupid?

Good things

Power compared to old HTC. Here it’s not only about 4 years younger device, it is about HTC Wildfire being too limited even for 2.x Android. Galaxy S3 Mini is adequate and usable. It plays videos from youtube, no problem (not to mention the resolution).

Let’s do some bullets again (in no particular order):

  • I like widget to turn on/off led lamp (torch)! :-)
  • I like options to use home key to answer the call and lock button to end it. Easy to set up and feels more natural than display touching.
  • Notification area and all the icons easily accessible (drag notification panel down, then click to the upper right corner). And battery status widget. I used some Settings application on HTC (from store) that worked after holding Home button for a while and also offered me all the options at a glance. This is here out of the box.
  • Compared to HTC you really can browse the internet here. Wildfire’s display was really coarse (320px high). I don’t use it that much because whatever I want to do on the internet I rather do on PC, but it is handy here and there.
  • The battery can still keep the phone alive for 6 days! (I was used to 7 with HTC.) Of course this is probably just me – other people who actually use their phone report 2 days top. I’m on 2G (GSM), wifi scheduled only for short times to sync with my account (Wifi Scheduler is cool app!) and I just call here and there. And no, I don’t turn it off for nights.

Well, and that’s all! :-) I didn’t get so many nice surprises, but I didn’t expect to anyway. Mobile works, sound is reasonably good, no quiet earpiece (HTC One V) or “can’t answer the call after pulling phone from pocket” (HTC Wildfire). I’m in no mood to return the Galaxy S3 Mini. Not that I love it – but hey, it’s just a phone. ;-)

Losing control

The first negative surprise with Samsung came actually soon after it booted the very first time. It asked for Wifi connection in the process and even though I provided it (and it worked) Galaxy decided to use mobile data without bothering to ask (HTC Wildfire was much better behaved in this aspect and asked kindly first and I could suppress mobile data). My provider sent me SMS informing me that I’ll be charged for one day of internet (up to some limit) 50 eurocents. This was actually pretty good because I could act – god knows when I would realize that Samsung uses mobile data otherwise. Very bad default indeed – especially for people who are not used to mobile data and turn them off once and for all.

This is my general observation lately – things get simplified (let’s face it, often plain dumbed down), streamlined – in any way (good or bad). Many products are now offered for free, but they push forward things we’re not interested in. Sometimes “we” may be minority, maybe even just “me”, but I’m pretty sure in many cases it is majority of people who don’t like the change (unless they don’t care at all of course). I don’t want to talk about some human freedoms here, how we yield our control to bad corporations or what… but I really don’t understand why we can’t turn off many services we don’t want in our mobile phones. HTC always restarted some Stock service I never wanted so it always ate memory where RAM was just 384 MiB. Samsung is no better obviously.

So better or worse?

Obviously, newer Samsung is better. It’s cheaper and gives me more than HTC 4 years ago. HTC Wildfire felt more solid in hand, also its surface was nicer than plastic Galaxy S3 Mini, but these are minor things. I’ve been excited about technology for many years actually – we take it for granted now, but these are all small miracles. I just wish software was better, maybe more evolutionary than revolutionary, because it’s mostly more buggy and unfinished than new.

Would I recommend Samsung S3 Mini? No. :-) It’s not sold anymore anyway so it seems.

I generally like Samsung, their design, menu on their TVs feels always familiar… But next time I’ll try some China mobile with better parameters (on paper) and even lower price. I’m no newest/hottest freak and just as I decided not to try HTC anymore, I’ll probably skip Samsung as well. If I’m surprised by omission of totally expected features (notification LED?!) then let it not bear name Samsung.

Jetty with self-extracting WAR from Launch4J EXE

In my previous post I was hacking Tomcat and Jetty with Launch4J – Jetty eventually won as it was willing to read JAR as WAR (not so picky about file extension as Tomcat was) and find WEB-INF/web.xml there without problems.

Yesterday I decided to go on with my quest and populate temporary WAR directory from my classpath programmatically – which would actually get Tomcat (or any other embedded container reading exploded WARs from disk) back into game. We will utilize Java 7 NIO2 classes, so if you’re stuck with Java 6 you have to adjust parts of the solution.

POM simplification

We will stick with Jetty for other reasons, but it will provide some flexibility there as well – now we can wrap JAR into EXE which we carefully avoided previously with plugin options (plugin was com.akathist.maven.plugins.launch4j:launch4j-maven-plugin, see the previous post for the listings):

  • <jar>${project.artifactId}-${project.version}.jar</jar>
  • <dontWrapJar>true</dontWrapJar>

These overrode parent POM’s defaults:

  • <jar>${project.build.directory}/${project.artifactId}-${project.version}.jar</jar>
  • <dontWrapJar>false</dontWrapJar>

So let’s minimise the module’s POM – without changing parent’s one we just mention both plugins we need – launch4j to get EXE and dependency plugin to populate directory with dependencies for us:

  <profiles>
    <profile>
      <id>exe-build</id>
      <build>
        <plugins>
          <plugin>
            <groupId>com.akathist.maven.plugins.launch4j</groupId>
            <artifactId>launch4j-maven-plugin</artifactId>
          </plugin>
          <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
          </plugin>
        </plugins>
      </build>
    </profile>
  </profiles>

Neat – my EXE module does not differ from other EXE modules (those without containers) anymore! Let’s code now, yeah… I mean not in XML.

Extracting web.xml from classpath

Using our previous bootstrap code, we just want to check whether our warPath is EXE file or not. If not, we can leave it that way, Jetty will read from JAR/WAR or even target/classes (especially handy during development). If it is EXE, however, we want to get the WAR stuff out. Explode it ourselves:

int port = Integer.parseInt(System.getProperty("port", "8080"));
String contextPath = System.getProperty("context", "app");

Server server = new Server(port);
WebAppContext context = new WebAppContext();
context.setServer(server);
context.setContextPath('/' + contextPath);

ProtectionDomain protectionDomain = RestMain.class.getProtectionDomain();
String warPath = protectionDomain.getCodeSource().getLocation().toExternalForm();
if (warPath.toLowerCase().endsWith(".exe")) {
  warPath = prepareWarPathFromExe(protectionDomain);
} // else we assume dir or jar/war
context.setWar(warPath);
server.setHandler(context);

server.start();

if (!context.isAvailable() || context.getWebInf() == null) {
  server.stop();
}
server.join();

Ok, that was easy – tiny change in the original code. Now the hard part – but first we just extract a single file – web.xml.

  private static String prepareWarPathFromExe(ProtectionDomain protectionDomain) throws IOException {
      String warPath = temporaryWar(protectionDomain.getClassLoader());
      log.debug("Extracting WAR from EXE into: " + warPath);
      final String finalWarPath = warPath;
      Runtime.getRuntime().addShutdownHook(new Thread() {
        @Override
        public void run() {
          try {
            deleteRecursive(Paths.get(finalWarPath));
            log.debug("Temporary WAR directory deleted");
          } catch (IOException e) {
            log.warn("Problems with deleting temporary directory", e);
          }
        }
      });
      return warPath;
    }

    private static String temporaryWar(ClassLoader classLoader) throws IOException {
      Path tmpWarDir = Files.createTempDirectory("restmod");
      Path webInf = tmpWarDir.resolve("WEB-INF");
      Files.createDirectory(webInf);
      Files.copy(classLoader.getResourceAsStream("WEB-INF/web.xml"), webInf.resolve("web.xml"));
      return tmpWarDir.toString();
    }

    private static void deleteRecursive(Path dir) throws IOException {
      if (Files.isDirectory(dir)) {
        try (DirectoryStream<Path> directoryStream = Files.newDirectoryStream(dir)) {
          for (Path path : directoryStream) {
            deleteRecursive(path);
          }
        }
      }
      Files.delete(dir);
    }

I think the code is easy to follow for most of the part. Shutdown hook deletes the temporary directory. We work with ClassLoader to get our resources, etc. If all you need is WEB-INF/web.xml, you can stop here. But this is far from flexible in case you have JSPs and other resources to extract. So let’s try to extract everything starting with specified strings.

Extracting more

I don’t know whether there is some reasonable way to “crawl” through classpath resources, however it is not what we need, actually. We want to check the EXE file and treat it as a JAR. While Jetty does not like context set to this EXE path (not even if you use URL starting with jar:file:/ and ending with !/), it is still perfectly valid JAR file.

First I wanted to use one of my ancient solutions inspired by Stripes framework. This checks JARs on the classpath, and opens their files as JarInputStream. This didn’t seem to work. However I found suggested Spring solution for reading resources using patterns:

PathMatchingResourcePatternResolver resolver =
  new PathMatchingResourcePatternResolver(protectionDomain.getClassLoader());
Resource[] resources = resolver.getResources("classpath*:WEB-INF/**");
// then copy the resources where necessary

And this worked and found resources in my EXE file too. So I dag a bit deeper, found the Spring’s class PathMatchingResourcePatternResolver and the method doFindPathMatchingJarResources – and I was back on track.

First we will adjust our calling statement from main method, because we’re leaving waters of classpath (taken from ProtectionDomain). And we introduce our prefixes from which we want to extract (META-INF is just an example, choose yours):

warPath = prepareWarPathFromExe(warPath, "WEB-INF", "META");

The rest of the main method stays untouched. Now the new body of the prepareWarPathFromExe method:

    private static String prepareWarPathFromExe(String pathToExe, String... prefixes) throws IOException {
      Path tmpWarDir = Files.createTempDirectory("restmod");
      final String warPath = tmpWarDir.toString();
      log.debug("Extracting WAR from EXE into {}, prefixes {}", warPath, prefixes);
      WarExploder warExploder = new WarExploder(pathToExe, warPath);
      warExploder.explode(prefixes);
      Runtime.getRuntime().addShutdownHook(new Thread() {
        @Override
        public void run() {
          try {
            deleteRecursive(Paths.get(warPath));
            log.debug("Temporary WAR directory deleted");
          } catch (IOException e) {
            log.warn("Problems with deleting temporary directory", e);
          }
        }
      });
      return warPath;
    }

Recursive delete stays the same. I put WAR exploding into separate class so I can put both paths into fields and don’t drag them through parameters. (And I’d hate to put them into static variables in my main class too.) Here it is:

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.Arrays;
import java.util.List;
import java.util.jar.JarEntry;
import java.util.jar.JarFile;
import java.util.stream.Collectors;

/**
 * Extracts files from JAR to specified directory. Extraction can be called multiple times for various set of prefixes.
 * Prefix matches if the JAR entry name starts with it, that is "META" matches META-INF and anything under.
 */
class WarExploder {
    private final File srcJarFile;
    private final Path explodedDir;

    WarExploder(String jarFilePath, String explodedDir) {
      if (jarFilePath.startsWith("file:/")) {
        jarFilePath = jarFilePath.substring("file:/".length());
      }
      srcJarFile = new File(jarFilePath);
      this.explodedDir = Paths.get(explodedDir);
    }

    void explode(String... prefixes) throws IOException {
      try (JarFile jarFile = new JarFile(srcJarFile)) {
        List<JarEntry> entriesToExtract = jarFile.stream()
          .filter(e -> Arrays.stream(prefixes).anyMatch(e.getName()::startsWith))
          .collect(Collectors.toList());

        // throws checked exception, hence the for-each
        for (JarEntry jarEntry : entriesToExtract) {
          extractIfMatching(jarFile, jarEntry);
        }
      }
    }

    private void extractIfMatching(JarFile jarFile, JarEntry jarEntry) throws IOException {
      String name = jarEntry.getName();
      Path targetPath = explodedDir.resolve(name);
      if (jarEntry.isDirectory()) {
        Files.createDirectory(targetPath);
      } else {
        Files.copy(jarFile.getInputStream(jarEntry), targetPath);
      }
    }
}

This time I used a bit of streams too, but there should be no problem to switch it to Java 7, maybe a bit more problem to use Java 6 – but if you’re that far in the past, then you rather use some Apache libs to copy files too.

Exploder doesn’t log, everything is delegated (and exception propagated) to the main class.

Conclusion

It wasn’t that hard after all. Although the solution is pure JDK after all, I have to thank Spring developers for their incredible work and inspiration (not just here and now). In cases like these you also can feel the pain of developers still stuck with pre-7 Java, as NIO 2 does miracles with Files and Paths (finally!). It also wouldn’t be possible without JDK working with JARs so easily. Actually if I had headed into getResourcesAsStream in URLClassLoader, I’d have found the proper way of working with JAR.

Originally I thought JarURLConnection was the solution, but the real key was using JarFile instead of JarInputStream (that works perfectly OK in other circumstances). You can create JarFile using JarURLConnection or just plain File. Having the file lying around on the disk, I felt that File should suffice – and it did.

There is also related StackOverflow answer (and probably more).

Next time we will talk about “hardening” our Jetty application, so that it produces our error messages instead of the default ones (not to mention that directory browsing is on by default too!) – but that will be much shorter than today’s post. :-)

Launch4j and embedded web container

Embedded web container is nice. Of course you’ll lose some benefits of a big container, but you’ll get something else. Sometimes it’s faster startup. And sometimes it’s ability to wrap your Java web application into executable.

In our system we have multiple modules and most of them are executables on Windows machine (not my preference anyway ;-)). There is some control console that can start them, kill them, etc. And our Tomcat kinda stuck out of this exe family. Before I could start Launch4j-ize our WAR I have to turn the thing around and embed the container into JAR. Something I did many times before with Jetty – although mostly only for development purposes with our exploded WAR somewhere in target directory.

Embedding web container

First step was to run some main class and get to a state when my application is accessible on some port+context – just like it would be after normal deployment. Because we were friends with Tomcat at the moment, I went for this option.

Code was easy, there are blogs you can find – I used this one with minor adjustments and with mere two tomcat artifacts (tomcat-embed-core and tomcat-embed-logging-juli) I was up and running (Tomcat 8.0.15).

I ran into couple of problems however:

  • Tomcat logging – I know it’s not Tomcat’s fault that Sun made such a horrible decision with JUL and I understand when someone uses it (portability is probably the only reason). I tried to use org.slf4j:jul-to-slf4j to little/no avail, but I didn’t give it so much time for more serious problems.
  • I was generally unhappy with the speed. It took maybe 10s of silence until my application showed its first log. No need for concrete numbers – maybe it can be adjusted, but when I compared it with expectations (and other containers – full-blown or embedded) the pause before actually starting the application was just too long.
  • The main blow however came from different direction. This is my main trouble with all these embedded containers – they kinda expect the WAR to be somewhere on disk (exploded or packed, doesn’t matter). What if I want to run it from classpath? I have web.xml as a resource already there! It seems that this is common to all the containers I met – or this kind of webapp loading is not documented. Maybe it’s not even possible because of Java EE specification – I really don’t know. So you have to have WAR. Or do you?

So my embedded application is running, why am I complaining about the last point then? I want to run the application as Windows executable and I want to avoid messing with filesystem if possible.

When JAR and WAR unite

But this seems not possible. Let’s say I run EXE wrapper for JAR file. This can be either embedded in EXE file directly, or lying somewhere around it. How am I supposed to provide WAR? Tried options:

  • Launch4j created app.exe that used the content of separate app.jar. JAR contained WEB-INF/web.xml too, but classes were placed like in normal JAR. This works fine when exploded, because web.xml is used to configure the application and the rest is on the classpath already. But not when packaged as JAR. For one – embedded Tomcat will not accept JAR file because its extension is .jar and not .war! Whatever the reason for this is, it was showstopper for this option.
  • Let’s change packaging of my “app” artifact to war then. I had to override couple of configuration options for launch4j maven plugin because now I wasn’t so unified with other non-web-app modules (based on jar packaging). This brings problems that classes in WEB-INF/classes are not on classpath before the webapp goes up. And because my main class is in that war (don’t let me create another Maven module for that class :-)) it is not found. I can meddle with my POM now to get the main class (with package structure) to root of the WAR file (working as JAR at that moment). This would probably double the size of the POM file (easy things made Maven way, you know), but I just gave up here.
  • Back to JAR packaging and let’s use something that can read WARs with “.jar” extension. :-) And so we get to Jetty.

To be fair, these are far from all the options. Just like jenkins.war (although that one uses Jetty instead of Tomcat) it is possible to do it with Tomcat after all – just leave JAR packaging and rename the resulting file (again too many lines for such a simple thing). This leads to further changes in launch4j configuration too however + we’re still stuck with slow start. It’s simply time to try something else. :-)

From Tomcat to Jetty

I was friendly with Jetty for the last four years. Not a master of this cute pet- after all it’s actually more complex than people are generally aware. It’s typically one of the tools nobody cares to study, “because I just use it during development”. Well here I am and I don’t want to use it merely for development.

Thinking about how Jenkins is run, first I try to solve this part. Answer was found in the post Executable WARs with Jetty (or this SO question that references it too). I’ll copy my version of final code:

public static void main(String[] args) throws Exception {
    int port = Integer.parseInt(System.getProperty("port", "8080"));
    String contextPath = System.getProperty("context", "app");

    Server server = new Server(port);
    WebAppContext context = new WebAppContext();
    context.setServer(server);
    context.setContextPath('/' + contextPath);

    ProtectionDomain protectionDomain = RestMain.class.getProtectionDomain();
    String warPath = protectionDomain.getCodeSource().getLocation().toExternalForm();
    context.setWar(warPath);
    server.setHandler(context);

    server.start();
    if (!context.isAvailable() || context.getWebInf() == null) {
     server.stop();
    }
    server.join();
}

I skipped all my logging and imports (those from jetty are obvious after all). Most interesting part here is the one setting the warPath. This actually works equally great for exploded wars (during development) and for JARs lying next to executable wrapper.

Obviously, not even Jetty is reading the WAR part from classpath. And it uses some temporary directory too, just less obvious than Tomcat’s tomcat.8080 placed in working directory. But Jetty can tread JAR as WAR when told to.

How about JAR wrapped in executable?

This is in fact how Launch4j creates our exe files by default (option dontWrapJar is false). Maybe it’s time to show our plugin setup for Launch4j for your convenience. Note that you need both launch4j-maven-plugin and maven-dependency-plugin. This is the default setup from our parent POM (it’s in build/plugin-management/plugins section, in our case further nested in profiles/profile, because we don’t create EXE files by default on our Linux based CI machine):

<plugin>
    <groupId>com.akathist.maven.plugins.launch4j</groupId>
    <artifactId>launch4j-maven-plugin</artifactId>
    <version>1.7.1</version>
    <executions>
        <execution>
            <id>l4j-clui</id>
            <phase>package</phase>
            <goals>
                <goal>launch4j</goal>
            </goals>
            <configuration>
                <headerType>console</headerType>
                <jar>${project.build.directory}/${project.artifactId}-${project.version}.jar</jar>
                <outfile>${project.build.directory}/${executable.name}.exe</outfile>
                <dontWrapJar>${launch4j.dontWrapJar}</dontWrapJar>
                <classPath>
                    <mainClass>${mainClass.name}</mainClass>
                    <preCp>dependency_libs/*.jar</preCp>
                </classPath>
                <icon>src/main/resources/${executable.name}.ico</icon>
                <versionInfo>
                    <fileVersion>1.0.0.0</fileVersion>
                    <txtFileVersion>${project.version}</txtFileVersion>
                    <fileDescription>${project.name}</fileDescription>
                    <copyright>...</copyright>
                    <productVersion>1.0.0.0</productVersion>
                    <txtProductVersion>1.0.0.0</txtProductVersion>
                    <productName>${project.name}</productName>
                    <companyName>...</companyName>
                    <internalName>...</internalName>
                    <originalFilename>${executable.name}.exe</originalFilename>
                </versionInfo>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-dependency-plugin</artifactId>
    <version>2.9</version>
    <executions>
        <execution>
            <id>copy-dependencies</id>
            <phase>package</phase>
            <goals>
                <goal>copy-dependencies</goal>
            </goals>
            <configuration>
                <outputDirectory>${project.build.directory}/dependency_libs</outputDirectory>
                <overWriteReleases>false</overWriteReleases>
                <overWriteSnapshots>false</overWriteSnapshots>
                <overWriteIfNewer>true</overWriteIfNewer>
            </configuration>
        </execution>
    </executions>
</plugin>

I override our global config in my module’s POM like this (I’m mentioning maven-dependency-plugin too to trigger it):

<build>
    <plugins>
        <plugin>
            <groupId>com.akathist.maven.plugins.launch4j</groupId>
            <artifactId>launch4j-maven-plugin</artifactId>
            <executions>
                <execution>
                    <id>l4j-clui</id>
                    <configuration>
                        <jar>${project.artifactId}-${project.version}.jar</jar>
                        <dontWrapJar>true</dontWrapJar>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-dependency-plugin</artifactId>
        </plugin>
    </plugins>
</build>

There are many other ways how to do this – you can go for some uberjar solution, etc. Important thing here is that <jar> element here does not reference JAR to be included into EXE, but it must be relative path to JAR when the EXE is started. Meaning of this element is changed by dontWrapJar flag.

Do I need to do this? Isn’t it possible to use EXE as WAR? Sounds silly at first, but I was surprised to discover that you can “enter” the EXE in Total Commander using Ctrl+PgDn – just like any other archive. Obviously – it’s self-extracting archive! When you include your JAR into it, it is correctly identified as a location of the main class in the bootstrap code – but Jetty will silently ignore it. In fact, this is the reason for context.getWebInf() == null check in the bootstrap code.

Anyway we’re still much better here than with Tomcat. Startup is faster, file can be named with JAR extension, what more do we want?

Other options

Before we try to enhance our current solution, let’s talk about other options. But not for long, really. Originally I wanted to try Undertow because it just seems to be cool. After reading its documentation I realized that I’d have to do more than just show it some web.xml file to start the application for me.

How about JBoss’s WildFly then? It is full-blown AS and can be embedded! However I found no documentation how to embed it that was not related to Arquillian, that is testing. Too bad. I love WildFly otherwise.

Glassfish? Probably possible but at this stage I just decided to go for simpler web containers. If you have anything useful to this topic, or other embeddable containers, please, mention it in discussion (no registration required, BTW).

Get out of my classpath!

I’ll not go further with examples, just with one idea found during extensive browsing while I was hunting the solutions. So Jetty/Tomcat/any container wants WAR in the file – or exploded WAR somewhere on the disk. They all create some kind of temporary files… so why can’t I? My bootstrap code can read web.xml (which can in turn be placed wherever on the classpath, not just WEB-INF) and put it in some temporary directory (Files#createTempDirectory) into WEB-INF subdirectory. Then we can point our WAR path there and let any worthy embedded container do the rest! Of course – if you have more than just a web.xml (JSPs, static stuff) you have to do more work, but that doesn’t make it less possible.

You may want to avoid this process for development – but again, this is easy to detect (“if the location of the main class is within EXE then do this else just use the location directly”).

Conclusion

Creating application with embedded web containers is a bit more complicated when Java application is wrapped in EXE – but that is expected. Any way you package your application, however, the biggest problem is that you somehow have to get your WAR on the disk (or something your embedded container likes like a WAR). Using app.war and running it with java -jar app.war is probably most straightforward. However if you want to share libraries with other modules and avoid uberjars, and/or use exe wrappers, things get more and more complicated. Externalizing WAR content (without classes, those are on the classpath already) in the bootstrap code may be a solution.

There is one more class of problems if you use Spring as well. Imagine all your modules bootstrap Spring application context. But your web module is doing it differently – it bootstraps webapp somehow and Spring is configured in web.xml. This is far less flexible during development too, as you can’t easily swap @Configuration class defined in the web.xml (or XML config). You can “shadow” them from test classpath but that mostly sucks. But if you invert the process and bootstrap Spring first, then it doesn’t know about web application yet and god knows what happens (or not and should) to your Jersey resources, etc.

However, this is different story for a different day… and looking at the calendar, probably for a different year too! Merry Christmas and Happy New Year!

Follow

Get every new post delivered to your Inbox.

Join 236 other followers