Live architecture with Java, Spring, JPA and OSIV

This post is about an architecture where live (attached) JPA objects are used in the presentation layer. You can expect OSIV (Open Session In View) pattern mentioned, though I’ll focus more on ways how we made it work well enough for us – safely and without LIEs (LazyInitializationException). It is just my story with my experiences, no big discovery here.

I can’t tell if it is any official name, but we call it “Live architecture” because live JPA entities are available in the presentation layer. While we use it with Spring/Wicket mostly, it is the same with any other presentation framework – and probably applies to JavaEE without Spring too (if you use OSIV).

DTO vs Live architecture

In our company there are “DTO guys” and “live architecture guys”. We all know DTOs (Data Transfer Object) and how to work with them, more or less. Their rise to fame came with the need of coarse-grained calls to remote EJBs and they became prominent “pattern” then. Even with local calls people use them to strictly divide layers. I used them on some projects, then not on others and then again I used them with GWT/Seam applications (never liked the idea of JPA entities being preprocessed for me and dragged all the way to the GWT application).

Everytime I start talking about “live architecture” that drags entity objects into the view there are architects who just say “that is no architecture at all”. And I say “whatever…” I remember projects where we “broke” a clean architecture (e.g. “everything must go through this facade!”) and the result was less and cleaner code, easier to understand, better performance even. Was it universal? Hell no, it wouldn’t scale in most cases, but in that particular case scaling was not (and after all those years still is not) necessary.

My recent story with the live architecture is based on a project where it was settled that it will be used instead of DTOs. You have to translate DTOs somehow from business objects and back. You can generate it, you can automate it, use reflection – or do it manually. Any way always adds something that is not necessary for all cases. Our views were mostly based on JPA entities and it was just shame to translate them to DTOs for the sake of transformation itself. I’m not saying DTOs are bad – well we use them for more complicated views, mostly for lists showing joined tables. You can of course build a view and design an entity over it – and we do it too…

There is no fundamentalism in this – we use entities as much as we can. I strongly believe that in normal scope projects people often overdo it with “clean architecture” and don’t care about “clean code” as much. And I strongly believe that cleaner code itself matters much more than that cloud castle of architecture (without underestimating the architecture itself!). After all our projects are quite simple multi-tier applications with a bit of clustering. No grid, no hi-perf, no America. So we use entities, because they are placed under the presentation layer (good dependency direction) and they only carry data. And when this is not enough, we use DTOs too. Simple.

Business logic objects and dumb entities

You may have different rules for your live architecture (projects using OSIV) – and that is fine. Ours start with don’t use entities to anything else – no business logic, maybe some simple computed properties, that is alright. You may call this Anemic Domain Model – but I don’t care. Logic is in separated objects that use one or more entities. It is not exactly DCI, but it is not very far from this. For many other reasons (unrelated to the live architecture) I prefer having business logic objects that performs specific scenario – the best case is 1-to-1 mapping with a Use case from the analysis document.

Let’s talk about this picture for a while:

Presentation layer can be anything – component (Wicket) or controller (Web MVC) driven. It calls the service layer (typically a Spring bean or EJB) and this further uses that “cloud” with various business logic objects. Very often I prefer create/use/throw-away pattern. In constructor the object gets its context and then it does something – preferably in one method call, but it may be a sequence too, although this is more fragile approach. Important thing is that business object can store its state during the business logic execution – it is thread safe if it is created locally for one service call (that’s why I don’t use singletons here). Sometimes state is not necessary, but in more complex cases it is. And I like fields much more than dragging list of parameters between private methods.

This business logic uses DAOs (or @EntityManager directly) to work with the DB – and of course works with entities in the process. Because entities are dumb (DCI idea, but not only theirs) they are perfect DTOs (that are also dumb). Of course there are some concerns about entities used as DTOs and you can find many questions about this issue (and not only in the Java world). Entities are POJOs – in theory – but you may drag some proxy object up there into the presentation layer. There is a lot of magic in entities, you sometimes don’t know what they are (my class or some modified class already?) – but under the most circumstances you don’t have to care that much really.

Best practices

Now let’s talk about our best practices. Presentation layer code knows entities, but doesn’t know ORM! This is probably the most important thing. Of course the dependency on the JPA is implied somehow. Of course client programmer has to know the data model and has to know how to traverse the objects he wants to display. But he absolutely can’t use EntityManager. Our first “live architecture” project didn’t have clear separation of these roles and some LIEs were fixed like “you know, here in this page before you call the service… put evict on this object there”. I wasn’t there when this project started, so I just went like “what?!?!” And I forbade this for the next project I could affect.

Next rule is rather about the communication than the technical one – presentation programmer always has to know what he gets from the service call. Otherwise he risks that LIE again. But LIEs in presentation are easy. They are easy to fix in model, in service/business code or in the presentation code (that is the most of the cases). You always have to share some model between business logic and presentation (and developers!) – and we share the data model itself. If you don’t plan to change your layers this is perfectly acceptable. I’ve actually never saw any change of technology that would satisfy using different model introduced on the facade level. So why to do it if you ain’t gonna need it? (Of course, you may need it – and you are there to say as an architect.)

Getting data is easy (talking about live architecture problems only ). You may need separate methods for every view – especially if selects are not generic enough. We have “filter beans” with single superclass and we use these beans with a few service methods (getSingleResult, getList, etc.) that are rather generic in nature. DAO-like even. It works for us, filter beans are the common ground for client and server programmer to communicate and they are part of the service layer API. We can have common FilterBean interface, because we use our custom filter framework behind. But you can use filter beans without common ancestor and have many service methods to obtain data. This is probably even cleaner.

Transactions, saves, updates

Originally we used DAO-like save on service layer too. We also didn’t have clear strategies when objects are alive and when not when the presentation layer called the service layer. If you had in one HTTP request read and write call, then the entities were alive if the write used result of the read. If you had just an update, then they were not. “Objects may come alive or not, let’s not assume that they are alive,” was our first strategy, though I didn’t feel very well about “or” used in the sentence. Never use contradictions in your assumptions. With a big help of our tests we managed to clean this mess up.

Our tests were TestNG based, they were not unit tests but mostly we tested the service layer playing the role of the presentation layer. It was funny how often the test passed and the user test (using browser) failed, but also vice-versa! Sometimes the test didn’t prepare the same environment – and we started to realize, that the service layer must assume less and be more strict. The biggest problem was that the presentation layer could change an entity A that was read in the request (hence alive) and then call service saving an entity B. The service layer had no chance to know about the A being saved in the same transaction. This lead to one very simple idea – we always clear session before calling transactional service methods. I forgot to say that we use transactions on service layer, so you can have more transactions in one HTTP request/persistence session.

Stepping back for a bit – client programmer knows that when he calls a service, his objects are alive. He can call multiple reads – and he knows that all things are still alive and he can base the next read on an attribute that is loaded lazily. In our case there is only one write/transaction called in one HTTP request – and it’s mostly the last call as well. If I wanted to make our policies even more precise I could say “always clear the session – for every service call”. This would mean less comfort for the client programmer. Or you can go for “dead” entities instead of live ones (see Other possibilities further).

Now the business programmer knows that any object that enters transactional service is detached and he can choose what to do with it. Do you need just to save the changes? Merge it (or call JPQL update, or whatever). Do you need to compare it to its original state? Read the object by its id and do what you need. Do you want to traverse its attributes? Well, better reload it first to make it attached again. We enforce this by a custom aspect that is hooked on an existing Spring @Transactional annotation.

This assumption would be very useful for read/list method too. Now the developer never knows if he has to reload or not. But read methods are not so complex and reload of the parameter entity should never harm either. Also – read/list methods are not transactional, so whatever he does, he can’t mess up with the persisted data. So this is our compromise between the client programmer using live objects and the service layer being secured enough. There is much less LIEs in our back-end code (which are harder to catch than those on the presentation layer) – actually I didn’t see one for a long time – and there is no chance to tamper with the data accidentally.

As a side note: Many of our problems were also caused by our presentation architecture – we load data, display them, then forget the content to keep page/session small and we just remember the IDs of the objects. When edit action comes, we reload the object from the service by its ID, modify it and then call the transactional write service method. To make this more convenient we have our custom ReloadableModel class for our Wicket pages, so before the model (entity obect) is to be updated, it is always reloaded from the service too (this is not a big performance hit, it often goes from the 2nd level cache anyway). This may not be very lucky solution but it was one of those we had to stick with for the time. You may or may not run into these kinds of problems. In any case, making your contracts and policies more strict and clean is always a good thing.

Other possibilities

There is not only Live vs DTO option. You can also use entities, yet always closing the session when the service call ends. This gives you the same model, less easy presentation changes, but it definitely is cleaner from the service layer point of view. You can make more strict contracts, performance is all down there and not ruined by lazy loads on the presentation layer, etc. I know this, we use this for other projects too. But I also know that people use OSIV a lot and that is why I wanted to wrap-up our experiences with it. You can come up with other policies too – for instance one read or write per request and nothing more. Do it all in one proper service call, don’t call many selects for every single combo-box model for instance. I agree with these approaches actually. But sometimes we don’t have the luxury of choice.

In any case, try to do your best to clean up the contracts as much as possible, avoid contradictory ORs in your assumptions and – I didn’t focus on this point much in this post – test your service/business layer. Contract and policy is one thing, but you have to ensure them – force them, otherwise they are not contracts, just promises. Because that is your safety net not only from the architectural standpoint, but also from the functional one. But that is a completely different story.

Stargate, DS9 and other Heroes

I once compared Prison Break with Shawshank Redemption and I wanted to talk about other typical TV shows from the last 20 years or so, where it all goes and what I miss so much about the recent shows. Just to go quickly through what I liked and what not. I liked Star Trek Deep Space 9 – this is one of those long term relationships – and very similar it is with Star Trek: SG1 and Atlantis. I liked 24 (wrote about it here…). I liked Dexter though I stopped after second season and I simply don’t want to go on to the fourth season to see her dead (ok, I saw that scene, obviously ) – but Dexter was really refreshing. I liked first season of Tudors (but I’m not much interested in the next parts) and I really liked fantastic Game Of Thrones – though they really should not let “Boromir” die. So, sci-fi, fantasy, semi-historic, action – I like it all, though sci-fi is probably my favourite.

What I liked less was Battlestar Galactica – how I like it in overall, I just can’t stand those shifts in characters. You just don’t know what to believe. When T’ealc becomes enemy of the rest of guys from SG1, you just know that he is sick or something. You know your heroes. But BSG? You just never know what to believe. And I don’t think that Stargate show doesn’t have interesting twists here and there. But not so crazy like BSG. Or, when I wrote word heroes – I remembered my probably biggest let-down. I watched two season of Heroes. Concept, visuals, idea – all great. But so much of stupidity, so many cliches with the bad guy always running away. And then time-shifting with good heroes becoming bed. All those wannabe surprises and forced shocks – after that an episode from Stargate or DS9 just caresses me so nicely!

I don’t know what is wrong with some of these new shows. Are we running out of ideas? Do we need to push the limits further no matter what? Probably yes. A have to admit that I was able to watch BSG all the way through and I was generally satisfied in the end. I also watched Razor and The Plan – and it was nice to go the whole way. I never forget Galactica going down the atmosphere on New Caprica or Pegasus down taking few cylon star bases with it. Those were magnificent scenes and for those I can forgive the big of mystery that somehow wasn’t believable for me (especially around the final five).

Once I tried Buffy the Vampire Slayer – and while it was fun it somehow didn’t grow on me (though I definitely liked Buffy ). Lately I read xkcd.com regularly (from the old ones to the newer) and there are many hints on Firefly show. Because it wasn’t the first time I’ve heard about it, I decided to check it – and with 14 episodes total + one movie (Serenity) it was quite a brief encounter. And I was more than satisfied! Not only there was that lovely doctor from Atlantis and beautiful cold Adria from SG1 (both of it shot after the Firefly actually) the whole stuff was well thought out, mix of sci-fi and western was very catchy, but without cliches, scripts are indeed great and final movie was just overwhelming. If you don’t know what Buffy and Firefly have in common – Joss Whedon is the man – and that’s why they are in this single paragraph. (Watching the Firefly was also good thing for some further xkcd reading. )

After Firefly (not that my life is divided to B.F. and A.F.) I somehow got more and more suspicious that “classic” series are the matter of past. Now it’s important to compete with BSG, Heroes and Prison Break. Well… whatever people want. New music is still good (among tons of cheap stuff) and so will be TV shows I guess. I can still see the chance there – Game Of Thrones for instance, although it’s not something that would “caress” me like Stargate. Or The Firefly. Or the fond humour in DS9.

Of course your mileage may vary – but I bet there are other people out there that must have very similar feeling. And don’t simplify it just to “you’re getting old!”

Three years with Java Simon (4)

Today I’d like to cover the rest of my Java Simon story. In the previous posts we talked hardly about the start, but the rest was actually quite quick. With Callbacks, JMX support, JDBC proxy driver and much better design we were ready to release our 2.0 version.

June 23, 2009, Java Simon 2.0, monitoring API, released

There was one major problem with this version – we needed 2 different JDKs to build it. JDBC 3 would not compile against JDK 1.6 because Java 6 required higher version of it – which we didn’t want, so we could use it on application servers without support of newer JDBC. JMX 1.2 shipped with Java 5 – on the other hand – didn’t support features we needed, mostly around MX Beans, returning more types of objects and so on. So JMX was compiled with Java 6. You can imagine the problems we had when we started using Maven as a build (though Maven still is not exclusive build tool for us).

Well… Maven. While I like the idea of it – especially dependency management is truly great – as a build tool it is incredibly in the way unless you read tons of the stuff. Originally I hosted Java Simon on java.net repository, but then Oracle somehow made it more complicated (and malfunction altogether for a while if I recall correctly) and I decided to switch to Maven Central. That was right decision of course, but the pain behind it was just crazy. Unless you have the process mastered it takes a lot of pain to deploy your first software there. However – our clients wanted Maven repo – and I did my best to provide. I learned a lot in the process, but no one will convince me that Maven can’t be MUCH simpler. And deployment on Maven Central is just horribly bureaucratic compared to FTP upload. Guys at Sonatype do their best in support though, they probably have to answer tons of stupid questions (at least for them). After all I complained more about it previously, so let’s just skip the rest with saying that 2.5.0 version was the first on Maven Central – and someone else had to deploy it for me. 3.0.0 was delayed a lot – Maven being 95% of the reason. Now I can release (at least from that computer where release plugin doesn’t throw infamous out of bounds exception without providing reason…) and it is a tremendous relief.

Talking about 3.0.0 – release announcement was here:

Java Simon alive and kicking with 3.0.0 available

As you can read in it the biggest theme was aligning of the Java dependency – now we can build it with JDK 6 only. Aside from that it was rather just a wrap-up of all the changes in 2.x line with some bug fixes reported for 2.5. Talking about bugs and issues – this was maybe the reason why I kept working on Java Simon and eventually made all the changes that slowly but surely shape the library. And this would not be possible without users – and especially active users. Reports were coming more in bursts, often from one reporter for some time. One thing I can say with my head straight up – I was always very prompt to answer and fix where appropriate (mostly they were indeed bugs).

To talk to our users we created Java Simon Google Group shortly after version 1, but this was mostly an announcement tool. Here and there someone new asked the question though – and again, I answered as soon as possible. Luckily, Java Simon is low-profile library, so the traffic was rather negligible. To sum it up – users who had problems were my motor in the end. The main problem probably was that later we had no project to use with Java Simon. There seems to be some chance now at my current job, so I expect more enhancements.

Here and there I still change some method names (some changed in 3.1, next changes will appear in 3.2) – not that I like doing that but I rather name it properly later than never (oh, how I hate broken promises of original Java’s @deprecated!), but otherwise the core seems to be pretty stable for now. But there is still some room for improvements – especially new features:

1. delivering more useful tools like JDBC proxy driver – that one I particularly like for its simplicity, just add “simon:” in the JDBC URL and have it on the classpath – right now monitoring part comes to my mind, charts, logging, dumps to some history DB, etc.;
2. providing some neat Callbacks (many things from the point 1 are actually implemented thanks to these);
3. web console where you can easily read your Simons.

Actually – there should be web console available in our next release (3.2.0) – we acquired new committer from among our users. That’s the true open source community story. You can’t even imagine how happy I was about it.

Of course – my life is not only about Java Simon. I have a family, regular job where they’d hardly pay me for Java Simon alone, I like doing music (soon more about it too) and then I just don’t care about Simon for a few weeks, sometimes even months. Though right now I’m just taking a short break before we wrap up that 3.2.0 version – and you’ll hear about it.

Three years with Java Simon (3)

I should finish this series before it should be called “Four years with Java Simon” – but we still have some time. I’ll show you what possibilities callbacks brought to the Simon, but first I’d like to deal with our Webnode site.

It was three years back (January 13th, 2009) when I posted on our Webnode site that we need some better web for presentation than our project site on Google Code. But then we found we can’t post bunch of HTML files (Javadoc) on Webnode – and with mime-types SVN props we can do that on Google Code… oh, how quickly things change.

Webnode site could be good if Java Simon gained some bigger momentum on our side – more committers and contributors, people writing blogs or tutorials or success stories (or not so success stories too if they can help ). But this did not happen and I felt as rather annoying obligation to update this site. Especially because the edit functionality is on a separate URL – editing a Wikipedia page or a blog post on WordPress is always just one click away – but not so on Webnode.

Three years later I decided to redirect javasimon.org on our Google+ page because it is so much easy to update, posting even very short posts is not inappropriate (which would be on a blog) and it’s just so much closer to my way of living on the Internet right now. I’ll go through javasimon.webnode.com and it will soon be a matter of past.

Last time I discussed some changes from version 1 to version 2. And to preserve the little from Webnode site that has any “historical” value I hereby copy one blog post covering just these differences:

Major changes in the core of the Java Simon v2
2009-01-28 14:30
While there are some important extensions to the Java Simon (JMX, Spring integration, etc.) there are a few important changes in the core part of the API that are probably even more important. If you’ve already managed to use Java Simon 1 I strongly suggest that you use version 2 even in its alpha stages. The thing is:

• If your project is finished or close to finish (month or two) stay with version 1.
• If your project continues and you’re just experimenting with Simon, definitely use version 2! There is v2-alpha1 which is basically rework of the v1 after a few changes in Stopwatch. If there is newer alpha out (check Featured Downloads on the right on our project page) take that one of course, because it contains more features from v2.
• Version 2 is planned to be out during March or April 2009, which is really soon.

Now what are the changes and why we made them?

• Important change happened in the Stopwatch. While in the v1 it contained various start/stop methods that took care of multi-threaded environment now it has only one start method and this start doesn’t return this anymore but it returns new Split object instead. You have to take care of the Split object, you have to take care of your multi-threading, you have to call stop method on the Split. This makes our code safer as the Stopwatch doesn’t contain internal maps that were prone to memore-leaks if client forgot to stop some split. Thanks to Erik van Oosten and his great Java Simon evaluation.
• Based on the same post we changed sample methods so that they return Java Bean objects now instead of the field.
• While in v1 you had to use SimonManager now you can use non-static Manager implementation directly. SimonManager still stays your favourite convenient class full of static methods, of course. This allows to have multiple separeated Simon hierarchies which may be handy in Java EE environment.
• To provide some extensibility for the API we introduced Callback interface. This allows to hook onto various events and process these events in any way you want – to log them, send JMX notifications, whatever.

There are more features to come with version 2 and I covered only those in the core part of the API. Stay tuned, download, use, test, let us know what you think.

Now let’s take a look at those Callbacks. Based on good-old Observer pattern, Callback is a listener that performs some actions on various events. First question was where the Callback should be registered – and we decided that Manager will hold its Callbacks. We didn’t want to scatter Callbacks across various Simons because typical usage would lead to a situation where many Simons call (and point to) the same Callback. We rather decided we will centralize Callback management on a Manager (that is per Manager of course) and bring some way how to filter events based on Simon name for instance.

Simple example of Simon is in our CallbackExample:

    SimonManager.callback().addCallback(new CallbackSkeleton() {
           public void onStopwatchStart(Split split) {
               System.out.println("\nStopwatch " + split.getStopwatch().getName() + " has just been started.");
           }

           public void onStopwatchStop(Split split) {
               System.out.println("Stopwatch " + split.getStopwatch().getName()
                   + " has just been stopped (" + SimonUtils.presentNanoTime(split.runningFor()) + ").");
           }
       });

       Stopwatch sw = SimonManager.getStopwatch(SimonUtils.generateName());
       sw.start().stop();

When you work with Simon (last two lines) you don’t care about Callbacks – they will be called. Their configuration can be based on some configuration and they should do whatever you want to hook on various Simon events.

BTW: If you use Java Simon 3.1 method onStopwatchStop is still called stopwatchStop. This is quite serious flaw and poor choice of method name on my part (and I’m terribly sorry for that). While this method doesn’t show it clearly, there was another method – clear (now onManagerClear). This method is called – as you may guess from the new name – when clear method on the manager is called. Let me explain composite callbacks first to show you the whole problem…

To add more callbacks to the manager is all right but if you want to filter Simons (by name, for instance) that fire an event on a Callback you actually need to do it in the event itself. Or wrap the Callback into another one – that is exactly what FilterCallback idea is all about. Another thing is that you may need to call various callbacks for the same filter – to group them – and that is what composite callback does – holds more callbacks (children) and relays the event to all of them. There is no interface CompositeCallback – instead all these methods are on Callback already, but they are not implemented in the CallbackSkeleton for instance (used in the example above). There is one implementation called CompositeFilterCallback – and you probably can guess what it does. It can hold more callbacks and call them when the common configured filter is passed. See CallbackFilteringExample for simple use case.

Now guess how people tried to remove callbacks from composite callback. It’s just a collection of callbacks after all, right? Ah, method “clear” must do exactly what I need here. But it didn’t. And if you didn’t implement this event method (which is not very common, but JmxRegisterCallback is nice example where it is very handy) it simply did nothing. There was method to remove one callback, but not all of them (this one was in SimonUtils). This is all finally fixed with version 3.2 – all names are much better and removeAllCallbacks is in the Callback interface.

JMX is nice example why we needed callbacks just as much as we wanted to offer them to our users. There are two ways how to access Simons via JMX – you can use single point MX bean, or let Simon instantiate MX beans per Simon. The latter however requires some actions when Simon is created, destroyed or the whole manager is cleared. I mentioned JmxRegisterCallback already – check how it’s done there. Now the Simon manager knows about Callback mechanism – but it doesn’t have to know about JMX – or anything else you want to drive by these events.

Split introduction and Callbacks are two very important changes that happened in version 2 – and these things are now well proven and will probably last (though some names can change as will happen in version 3.2 ). Next time I’ll try to wrap up the rest of the story.

Open letter to Java Simon users

I bet there are people who are not on our mail group or watching Java Simon page on Google+. Roughly three years after the first official release we have another really good release.

I’m really proud about our newest Java Simon release (3.1.0) and I decided to share the mail written to javasimon@googlegroups.com also here on my blog:

Dear Java Simon users

Firstly – version 3.1.0 was released on New Year’s Day – more about it
on our project site:
http://code.google.com/p/javasimon/

We announced it also on our new stream on Google+ (it should be
available for non-google users too):
https://plus.google.com/b/115141838919870730025/115141838919870730025/posts/ZpmYGp9F2yp

Secondly – about Google+ – we decided to pull down our Webnode site
because it was a bit cumbersome to maintain and add new posts there.
Instead we are moving to the aforementioned Google+ page.

Direct link: https://plus.google.com/115141838919870730025/posts
Short link: http://gplus.to/javasimon
Or just use our domain! http://javasimon.org/ or http://www.javasimon.org/

This way it should be easier for us to post more often even smaller
facts about your favourite monitoring library.

Thirdly – Happy New Year to you all, update and share your thoughts, I
feel very well about the last release. We’re working on 3.2 already,
with our new commiter (Gerald) we should be able to deliver simple
embeddable web console too, so there is a lot to be looking forward
to.

Best regards and wishes

Richard “Virgo” Richter

You are welcome – and encouraged – to add Google+ page into your circles of course. And once more – Happy New Year – as this is my first post here in 2012.