Three years with Java Simon (4)

Today I’d like to cover the rest of my Java Simon story. In the previous posts we talked hardly about the start, but the rest was actually quite quick. With Callbacks, JMX support, JDBC proxy driver and much better design we were ready to release our 2.0 version.

June 23, 2009, Java Simon 2.0, monitoring API, released

There was one major problem with this version – we needed 2 different JDKs to build it. JDBC 3 would not compile against JDK 1.6 because Java 6 required higher version of it – which we didn’t want, so we could use it on application servers without support of newer JDBC. JMX 1.2 shipped with Java 5 – on the other hand – didn’t support features we needed, mostly around MX Beans, returning more types of objects and so on. So JMX was compiled with Java 6. You can imagine the problems we had when we started using Maven as a build (though Maven still is not exclusive build tool for us).

Well… Maven. While I like the idea of it – especially dependency management is truly great – as a build tool it is incredibly in the way unless you read tons of the stuff. Originally I hosted Java Simon on java.net repository, but then Oracle somehow made it more complicated (and malfunction altogether for a while if I recall correctly) and I decided to switch to Maven Central. That was right decision of course, but the pain behind it was just crazy. Unless you have the process mastered it takes a lot of pain to deploy your first software there. However – our clients wanted Maven repo – and I did my best to provide. I learned a lot in the process, but no one will convince me that Maven can’t be MUCH simpler. And deployment on Maven Central is just horribly bureaucratic compared to FTP upload. Guys at Sonatype do their best in support though, they probably have to answer tons of stupid questions (at least for them). After all I complained more about it previously, so let’s just skip the rest with saying that 2.5.0 version was the first on Maven Central – and someone else had to deploy it for me. 3.0.0 was delayed a lot – Maven being 95% of the reason. Now I can release (at least from that computer where release plugin doesn’t throw infamous out of bounds exception without providing reason…) and it is a tremendous relief.

Talking about 3.0.0 – release announcement was here:

Java Simon alive and kicking with 3.0.0 available

As you can read in it the biggest theme was aligning of the Java dependency – now we can build it with JDK 6 only. Aside from that it was rather just a wrap-up of all the changes in 2.x line with some bug fixes reported for 2.5. Talking about bugs and issues – this was maybe the reason why I kept working on Java Simon and eventually made all the changes that slowly but surely shape the library. And this would not be possible without users – and especially active users. Reports were coming more in bursts, often from one reporter for some time. One thing I can say with my head straight up – I was always very prompt to answer and fix where appropriate (mostly they were indeed bugs).

To talk to our users we created Java Simon Google Group shortly after version 1, but this was mostly an announcement tool. Here and there someone new asked the question though – and again, I answered as soon as possible. Luckily, Java Simon is low-profile library, so the traffic was rather negligible. To sum it up – users who had problems were my motor in the end. The main problem probably was that later we had no project to use with Java Simon. There seems to be some chance now at my current job, so I expect more enhancements.

Here and there I still change some method names (some changed in 3.1, next changes will appear in 3.2) – not that I like doing that but I rather name it properly later than never (oh, how I hate broken promises of original Java’s @deprecated!), but otherwise the core seems to be pretty stable for now. But there is still some room for improvements – especially new features:

  1. delivering more useful tools like JDBC proxy driver – that one I particularly like for its simplicity, just add “simon:” in the JDBC URL and have it on the classpath – right now monitoring part comes to my mind, charts, logging, dumps to some history DB, etc.;
  2. providing some neat Callbacks (many things from the point 1 are actually implemented thanks to these);
  3. web console where you can easily read your Simons.

Actually – there should be web console available in our next release (3.2.0) – we acquired new committer from among our users. That’s the true open source community story. 🙂 You can’t even imagine how happy I was about it.

Of course – my life is not only about Java Simon. I have a family, regular job where they’d hardly pay me for Java Simon alone, I like doing music (soon more about it too) and then I just don’t care about Simon for a few weeks, sometimes even months. Though right now I’m just taking a short break before we wrap up that 3.2.0 version – and you’ll hear about it.

Java Simon 1.0, monitoring API, released

It is some time ago when we wanted to use JaMON API to add some monitoring capabilities to our applications. After a few we found it to lack some featrues – especially it was difficult to organize monitors. Hence we decided to create our own alternative that brings two features JaMON can’t provide – monitors organized in a tree hierarchy (similar to JDK14 logging API) and nanosecond precision (which bind us to JDK 1.5 or higher – which we see as no problem for most newly developed applications).

That’s how Java Simon API was born – the project is hosted on Google Code: http://code.google.com/p/javasimon/

Short description from our page:

Java Simon is a simple monitoring API that allows you to follow and better understand your application. Monitors (familiarly called Simons) are placed directly into your code and you can choose whether you want to count something or measure time/duration. Access to Simons is provided via static SimonManager.

We hope all necessary documentation is there and it will be extended in the future for sure. Aside from core functionality featuring standard stopwatch/counter monitor types we provide Simon JDBC proxy driver featuring these monitors – so you can check how various JDBC calls perform, pin down slowest selects, etc. Driver is JDBC 3 compliant with data source implementations available and it was tested with Glassfish/XA datasource with Oracle as the underlying database.

What can typical Stopwatch do?

  •  it can measure time splits, parallel measurement is supported;
  • it provides total time, number of time splits, tracks max/min time, time stamp when extremes were measured, deviation, …;
  • and like every Simon it tracks first/last usage time stamp and features like enable/disable/reset are also available.

Why we brought in a new API when there is JaMON available? Well, we tried it with JaMON but than we wanted to crucial things that JaMON doesn’t provide:

  • as said – we wanted to measure time in nanoseconds (hence JDK 1.5 or higher is required);
  • we wanted all monitors to be organized somehow – so we implemented tree hierarchy similar to java.util.logging API – this provides powerful features like enable/disable/reset of the whole subtrees of this monitor hierarchy.

Version 1.0 contains core functionality and we realize it lacks many important features in the bigger picture of application monitoring. Our plans in the future covers:

  • Declarative Simon configuration.
  • JMX access to Simons, their values and enable/disable features.
  • Sampling, collecting, agregating with persistence backend (file/DB).
  • …and probably more, but not all in the next major release. 😉

JPA, PostgreSQL and bytea vs. oid type

I started to work on a private project using JBoss Seam, Facelets, RichFaces (proven stack from Seam’s Booking demo :-)) with PostgreSQL 8.3 as a backing database – database I prefer for years for whatever reason. Hibernate is used as the JPA provider (packaged with Seam anyway) and the most recent JDBC 4 driver for PostgreSQL is used.

I added column:

picture bytea;

Somehow – after reading 8.4 Binary Data Types – I thought that bytea is the right guy for the job. Of course I had an entity with proper annotation and field:

    private byte[] picture;
    @Lob
    @Basic(fetch = FetchType.LAZY)
    public byte[] getPicture() {
        return picture;
    }

    public void setPicture(byte[] picture) {
        this.picture = picture;
    }

What was my surprise when following exception occurred:

java.sql.SQLException: ERROR: column "picture"
  is of type bytea but expression is of type oid

Now – when I know the solution – this message is pretty clear. It would be in case different types were mentioned in the message. While I understood that “he” somehow dislikes bytea, I couldn’t understand what the hell “oid” (as an “object identifier”) has to do with it?

I started my investigation with Google in one hand and PostgreSQL in the other. While 8.16 Object Identifier Types was not very helpful – as in “what does have ‘unsigned four-byte integer’ to do with my byte array?!” – some googled pages convinced me that oid indeed is that type I should use – mostly one from postgresql.com domain. Truth is that I didn’t understand why and how that oid type manages to store my byte array. But I’ve tried it, it stores it, it reads it, no exception is thrown.

Now I’ve finished second part of my investigation and the results are:

  • There is special Large Object facility in PostgreSQL and it’s based on using oid type that references actual large object stored in different table behind the scene.
  • JDBC documentation also mentions that you can use oid instead of bytea.
  • In that same chapter you can learn that bytea is supported since JDBC 7.2 version – that is not very long ago.
  • I don’t know if JPA/Hibernate can use bytea, but – honestly – I don’t care while this works fine. And it does.

So if you use PostgreSQL along with JPA and bytea type is throwing an exception, try oid. I’m not sure how well JDBC supports bytea, I don’t know why JPA wants to use oid and if other JPA providers (Toplink?) can use bytea. It’s not straightforward to use oid as a type for binary data – because you can learn this only in JDBC documentation and not in the mainstream PostgreSQL manual (I don’t count that chapter 31 because you’ll likely never use it while creating your tables). If you google this with along with “JPA” or “@Lob” your results might not lead you to proper articles quickly enough… so hopefully this post might help it a bit.

If you have some facts to add, feel free to comment.

Edit 8-Feb-2013 – warning: My colleague informed me that after removing oid row from a table, large object itself stays in the DB if you use Hibernate (can’t tell anything about other providers). Reportedly, there is hot ongoing debate between Hibernate and PostgreSQL guys. Both may be right that it’s not their job to do the stuff and they don’t have enough information to delete the object, because large object itself can be shared and referenced from more tables and in fact only you, the programmer can know if the object can go away or not. Check the PostgreSQL documentation to find out how to remove the object, also check if vacuumlo doesn’t solve your problem.