Archivo mensual: mayo 2012

Beyond DJN 2.1 (III)

In this post I will consider features which I consider just “nice to have” and therefore not a must.

Nice to have features

  • Enhancements to unexpected server exception handling
    Right now, when there is an unexpected exception at the server, DJN provides a message, as per the Ext Direct specification.

    While this message provides both the exception type (without the package) and the exception message, I would like to have them apart. This way, it would be possible to handle exceptions without having to dissect the message, making things easier.

    Besides, I would love to return the whole exception chain, if in a limited way: just the exception type and the message for every exception in the chain would be enough.

    My proposal is return serverException with the error information, that would provide the following entries:

    • exception.type: the full exception name of the topmost exception.
    • exception.message: the message of the topmost exception.
    • rootException.type: the full exception name of the root exception.
    • rootException.message: the message of the root exception.
    • exceptions: an array of elements containing the type and message for all exceptions in the exception chain.

    It is important to understand that serializing all exception information looks nice but is not a good idea. What if we find an exception with an structure whose serialization is not feasible? Then we would get the ugly scenario of an error inside of an error handler. You really don’t want that to happen, believe me!

    Better not to try: I think that getting the types and messages of the whole stack of exceptions is enough for 99% of the cases.

  • Support client-based polymorphic objects
    Ok, you’ve got a nice user interface that allows you to handle projects and complex projects. Looks like they both should be inheriting from a base project class, right? Now, how do you handle that in the server side? If you write a method that receives a base project, then, how is DJN/Gson to know the exact class to which a json object belongs?

    This is a feature that is relevant for some projects with rich object models, but I really need to ruminate about it a lot more.

  • Sencha IDE support: this one is nice too, but there has been very little demand for it, strange as it might seem.

    And, I don’t use Sencha’s IDE: they gave me a free license for their first version so that I might support it, but since then I’ve got nothing else that an offer for a discount for the new version. I thought they would keep sending it for free to contributors like me, but that’s not the case: I would be crazy to pay for a product I will not use…except to provide free support for it. Yup! Writing Open Source is hard :).

    Besides, not having committed to support it and not knowing what will happen when newer version appear, I don’t feel like asking Sencha for the new IDE.

  • Method interception, so that you can call your own code before and after the Java method itself is called to perform security checks or… You get the idea.

    Well, this is already supported, just that I have not publicized it. I think this feature deserves its own post.

    For those of you who are interested, take a look at how SsmDispatcher is implemented and used, as well as to Dispatcher and DispatcherBase.

  • CDI & JEE Integration
  • Spring Integration
  • Tracking appengine to make sure the last version is supported.

There is another feature that feels very important to me, but I have not discussed here: implementing serialization control.

For rich object models it can happen that too many unneeded objects are sent to the server because the object model is a giant net of interconnected objects. You need to limit what is serialized in that scenario. When you consider this, as well as interaction with JPA lazy loading, or you take into account the need to exclude some sensitive data, you can see that things can get really complex.

This feature deserves a whole post, and this one is already too long.

ExtJs + Ext Direct + DJN: working with dates

Passing dates between ExtJs and Java code using Ext Direct is a bit convoluted, because there is no standard for passing dates in json: in fact, there is not such a thing as a date type in json.

All in all, I have found that the easiest way to pass dates back and and forth is to pass them as a numeric value, the number of milliseconds elapsed since January 1, 1970 00:00:00.000 GMT.

The java.lang.Date type represents just this, take a look here. And a javascript Date stores the same value, look here. Therefore, recreating dates is trivial once you have the milliseconds.

Besides, by passing milliseconds we are guaranteed that we will not lose data, as both types do not have more accuracy: there are no microseconds or nanoseconds in a java.lang.Date or a javascript date.

Because we are passing a number, we need to tell both ExtJs and DJN how to handle dates. Doing this in ExtJs is trivial, just redefine Ext.JSON.encodeDate as follows:

Ext.JSON.encodeDate = function(d) {
   return d.getTime();

Next, we need to tell DJN how to handle dates. DJN uses Gson for json handling, and it will ask us to define classes to perform date serialization and deserialization. Here’s the date serializer:

   public class DateSerializer implements JsonSerializer<Date> {
      public JsonElement serialize( Date src, Type typeOfSrc,
            JsonSerializationContext context) {
         assert src != null;
         assert context != null;
         assert typeOfSrc != null;

         JsonElement result = new JsonPrimitive( src.getTime() );
         return result;

And here is the deserializer:

   public class DateDeserializer implements JsonDeserializer<Date> {
      public Date deserialize( JsonElement json, Type typeOfT,
            JsonDeserializationContext context)
                  throws JsonParseException
         assert json != null;
         assert context != null;
         assert typeOfT != null;

         if( !json.isJsonPrimitive() ) {
            throw new JsonParseException( "The value must be a valid number");
         JsonPrimitive primitivejson = (JsonPrimitive)json;
         if( !primitivejson.isNumber()) {
            throw new JsonParseException( "The value must be a valid number");
         return new Date( primitivejson.getAsLong() );

Once we have defined these classes, we need to register them with gson. To do this, we need to define a GsonBuilderConfigurator, as follows:

public class CustomGsonBuilderConfigurator extends DefaultGsonBuilderConfigurator
   public void configure(GsonBuilder builder,
         GlobalConfiguration configuration)
      super.configure(builder, configuration);
      builder.registerTypeAdapter( Date.class, new DateDeserializer());
      builder.registerTypeAdapter( Date.class, new DateSerializer());

Finally, you need to tell DJN to use this configurator, by adding the following lines to web.xml:


Back to ExtJs, we still need to tell it how to convert milliseconds to dates.

In many scenarios, you will tell ExtJs that a value is a 'date', and that its format is 'time‘. For example, for a Model object you can define an startDate field as follows:

fields: [
   { name: 'startDate',<strong> type: 'date', dateFormat: 'time'</strong>}
   // ...

At other times, you will have to handle the milliseconds yourself, creating a new date as follows:

date = new Date(theValueYouReceivedViaJson);

Built in support for dates in future DJN versions

While all of this is boilerplate and therefore easy to set up, I will probably add this support to DJN in some version beyond 2.1, so that all you will have to do is to override Ext.JSON.encodeDate and take care of the javascript side.

Of course, you will always be able to redefine how to handle dates if this approach does not work for you.

Findbugs 2.0: an update

I’m a big advocate of FindBugs, to which have devoted several posts.

My experience in “porting” a project to use FindBugs 1.x to the maximum extent possible was pretty good, and I decided to define a set of recommendations for using FindBugs in all new projects.

These days I have decided to take a look at FindBugs 2.0, and I have to say that the experience was good. It felt unobtrusive, worked well, and found some new potential issues. These were not real issues for my project, but would have been in most other projects, and it felt good to have them pinpointed.

FindBugs 2.0 works like 1.x did: in fact, the recommendations I laid out for using 1.x remain unchanged nd are fully applicable. Nothing new under the sun, just small improvements well worth it.

By the way, I decided to add FindBugs plugins, that help check additional potential issues. I’m using fb-contrib. I have to say that I did not find additional potential issues in my code, but this might be just my project. Since this did no harm, I’m keeping these extra checks.

Kudos to the FindBugs guys for their 2.0 release. I hope they keep the good work, FindBugs is a nice and helpful tool that everybody should use if they are concerned with code quality.

Beyond DirectJNgine 2.1 (II)

In my first entry about what to add to DJN 2.1+, I though I had a pretty complete list of “important” features. But I forgot this one…

Important features

  • Be able to receive hererogenous data in Java methods

    What do I mean? Sometimes it happens that you really truly have to handle data of an unknown type, much like myFunction does:

    MyRemoteThing.myFunction( 55, .... );
    MyRemoteThing.myFunction( "hello", ... );
    MyRemoteThing.myFunction( {value1:33, value2:"Unknown"}, ... );
    MyRemoteThing.myFunction( [5, "bye"] );
    MyRemoteThing.myFunction( [1, "anyway", {value1:33, 
                                 [33, null, 5], value2:"Unknown"}], ... );

    Here, myFunction has to handle a number, then a String, an object, then an array, and finally an array containing numbers, strings and even an object that contains itself another array. Yuk!

    How are we going to handle this monster in the Java side? Well, up until now you had to write a method to receive the almost raw json (a JsonArray parameter), as generated by Gson, and process it on your own.

    My proposal is to write a Java method with this signature:

    public void myFunction( Object param ) {
      // ...

    Now, what will DJN pass to the poor myFunction method? Let’s dive into each case:

    • myFunction( 55, ...)

      Here param will be a Double set to 55.0.

      Not an Integer, mind you, because we need to honour the fact that json numbers can have fractional part. There is no separated integral numeric type in json.

    • myFunction( "hello", ... )

      Here param will be a String set to "hello".

    • myFunction( {value1:33, value2:"Unknown"}, ... )
      Here param will be a Map<String,Object> with two entries, a "value1" key with a Double value set to 33.0, and a "value2” key with a String value set to "unknown".

    • myFunction( [5, "bye"] )
      Here param will be an array of objects (Object[]), with two elements: a Double set to 5.0 and a String set to "bye".

      I think that arrays will be used most of the time to pass elements of the same type: always numbers, always strings, etc.

      This contrived example just illustrates the whole potential -that includes potential for mess, of course.

    • myFunction( [1, "anyway", {value1:33, [33, null, 5], value2:"Unknown"}], ... )

      You’ve got the idea by now. Here the method will receive an array of objects, the third one will be a Map<String,Object>, etc.

      I do not think this last scenario will be very popular, but it shows how far we can go if we need to.

    But this is *dirty* Java…!


    Yes, this function is not a very good Java-world citizen, what the hell does myFunction think it is doing? Since when is working with Object a good idea? Because an Object is just an alias for whatever, right?

    And, in fact, a very important goal for DirectJNgine was to provide and almost enforce a degree of type safety at the server side, and that’s why I have not implemented this kind of functionality up until now: if you write a server function that receives an Object, it will not work because DJN considers there is not much one can meaningfully do with whatever, so it almost forces you to devise a class that organizes your data.

    Even then, I provided the JSonArray backdoor to allow the server side to receive whatever, because sometimes you have very special needs.

    …so, why allow this level of exposure?

    Have I changed my mind? Is untyped data good nowadays?

    No, not at all. It just happens that my mind has expanded. Well, maybe I have changed my mind -but just a tiny bit 🙂

    Here is my reasoning.

    Firstly, having to handle the json itself is not too difficult, but neither is it very easy. For example, for moderately complex structures you will need a bit of recursion to process the JsonArray with the underlying json. I know people will like me better if I write that code for them.

    Secondly, a guy might tell me this: “ok, I understand you are making it difficult for me to mess with data in the whatever format. Limits can be good. But sometimes I inherit the mess. In fact, I’ve got that nice UI component from Joe: it allows end users to edit multiple items easily…but it will handle me data of whatever type. Sometimes the user will edit a string, sometimes a number, etc. And *I* don’t want to have to rearrange the data to make it fit a type-safe but uber-complex class I have to invent to make Java happy just once”.

    Well, that seems reasonable.

    Besides, one might think that, if there is a need to massage very complex data to make it amenable to complex processing, it might be better to massage it at the Java side, where you have all those Java utility libraries that have no Javascript equivalent. In that scenario, it might be better to pass raw data to the Java side and clean the mess there.

    That said, let me tell you that I still feel it will be bad style to allow highly unstructured data to get deep into the business core (unless it is just to be stored as-is and handled back the same way), and that most of the time you can be well served by defining a handful of helper data-only classes to structure the information.

    Yes, that’s me: I am a quality code bigot that enjoys Test Driven Development like there is no tomorrow.

    A pair of little details to take into account

    I expect two scenarios to be very common:

    1. Handling arrays, essentially acting as a collection of items of the same type, whatever that might be.
    2. Handling objects as bags of properties, probably of different types.

    To enforce those scenarios you will find it nice to be able to write your Java function so that it takes an Object[] (scenario 1) or Map<String,Object> (scenario 2), not a raw Object. That makes the intent clear and helps with type safety.

    Caveat emptor & more to come

    Of course, I’m just thinking aloud, though I would very much like to implement this given some spare time. Maybe in DJN 2.3.

    There are still other features that, if not important, would be nice to have. So a third entry about post 2.1 DJN future is guaranteed.

Beyond DirectJNgine 2.1 (I)

With DJN 2.1 out, it is time to start thinking about features for the future versions of DJN.

I’ve been very happy with DJN reliability. Users report very, very, very little bugs. I’m really fond of the way Test Driven Development has been put into practice in DirectJNgine.

Of course, everyone has a desired feature that he misses and will consider that a bug, but a missing feature is a bug only if it was part of the specifications -if it was not there, how come we “miss” it? 😉

I consider the underlying structure absolutely stable and rock solid: therefore, no need to plan for architecture improvements, refactorings and such for the near future.

Must have features

I don’t miss any must have feature.

Yes, some guys will justly consider some features must-have: there are requests such as “I have to download 100 Mb files and need to support end user feedback, and DJN does not allow me!”, or “I love Spring, please provide support asap”, which are completely reasonble. But I don’t have an avalanche of requests with regards to any missing feature.

Important features

  • Generic type support for method parameters

    Runtime handling of Java generics is problematic because they are an afterthough. All JSON deserializers for Java use one trick or another to work around this (I envy .net guys in this respect, really).

    The gson guys (it is the underlying json handling library) explain this very well here.

    I have found that using arrays, which provide all necessary information at runtime, is the way to go. In reality this missing feature looks worse than it really is. Once they get into the real business of using DJN, most users just don’t care -and even wonder why it looked so important back then when I question about it.

    But, yet, this looks so bad. Especially for those that do not get into understanding why this, and so assume it is a stupid mistake. I would love to have this for DJN 2.2 (no promises, though!).

  • Make methods that can receive arrays capable of receiving a single item gracefully

    Now, why is this relevant? Sometimes ExtJs generates calls to my Java method with a single object, and sometimes it generates calls with many items and sends an array. Why not send an array in both cases?

    Java is a type-safe language, so you can’t have a method that receives both kinds of parameters. Depending on how you use ExtJs this can get really annoying.

    DJN to the rescue! What if you write a method that receives an array, and DJN creates a single item array when ExtJs insists on sending a single item? Would be nice!

    This might be something to have in DJN 2.2 or maybe 2.3.

  • Provide reasonable default Java Date support

    Dates are not a JSON type, so everybody sends them their way, and you have to define that way in every project.

    Of course, I have my own, but I found that I always encode them the same way, and everybody seems to like that way. I might add that code to DJN by default, and whoever does not like it can keep defining his own way (I bet everybody out there has already defined serializers/deserializers for java Date).

    Might be part of DJN 2.3.

  • Support for donwnloading unlimited size files

    I implemented file downloading in the simplest possible way, especially from the user point of view.

    But that way is not the more efficient way. When file size grows big, it can get ugly because you have no way to provide feedback. What’s worse, if you have little memory available, you can get out of memory errors.

    Would be nice for DJN 2.4, right?

More considerations to come

That’s all for today. I will continue ruminating about future directions for DJN in the following days…

Your feedback will be greatly appreciated: feel free to post your ideas as comments in this blog or in the main ExtJs thread for ExtJs here.



DirectJNgine 2.1 final is out

Today I am releasing DirectJNgine 2.1 final, which adds support for ExtJs 4.1. You can get it here.

DirectJNgine 2.x has been out in the wild for a long time, with an alpha label that really made it no justice, as it was fully production ready.

In order to make it absolutely clear that DJN 2.0 works perfectly well with ExtJs 4.x I have decided to release 2.1 final. This release adds very little to 2.0 alpha 1, just a minor tweak to circumvent a change in ExtJs from 4.0.x to 4.1 that broke an automated test, and a handful of tests. I decided to label this version as 2.1 to make it clear that this version is targeting ExtJs 4.1, a great improvement over 4.0.x.

As always, all ExtJs Direct examples are ported to use DJN 2.1: you can fully port the PHP based examples provided by Sencha to Java in less than five minutes.

Upgrade to ExtJs 4.1

For those of you that use ExtJs 4.0.x, let me tell you that DJN 2.1 will probably work ok, in fact it passes all but one test. But I encourage everybody to move to ExtJs 4.1, as it is a great improvement to 4.0, fixes many issues and is the target platform against which DJN is developed.

Hope you enjoy this new version 🙂