SQL Zone is brought to you in partnership with:

Marty has posted 1 posts at DZone. View Full User Profile

Lazy Loading with Flex, BlazeDS and Hibernate

10.21.2010
| 41772 views |
  • submit to reddit

This article takes a look at some of the problems associated with sending complex object graphs from Java to Flex, a few common workarounds and how lazy loading presents a more effective solution.

We'll take a look at one of the open source solutions available to facilitate the solutions discussed – dpHibernate - and step through setting it up.

Additionally, we'll cover off a brief overview of how it all works, and some of the common pitfalls.

The sample project discussed in this article is available to be checked out here


The problem part one:  LazyInitializationException

BlazeDS does not provide any out-of-the-box support for Lazy Loading, or support of Hibernate lazy-loaded entities.

When a server method returns an entity fetched from the database via hibernate, developers will often see a LazyInitializationException  thrown from Hibernate / BlazeDS.

This is because the hibernate session that was used to fetch from the database is closed after the call exits the DAO layer.  During serialization, BlazeDS traverses the accessors of an object, calling each and serializing it's value.  If it hits a property which Hibernate has proxied, a LazyInitializationException is thrown, as Hibernate no longer has an active session to fetch the value from the database.


dpHibernate

dpHibernate is an open source serialization adpater for BlazeDS, which provides lazy loading and persistence for Hibernate.  It addresses the LazyInitializationException issue by managing the hibernate session in a fashion similar to Springs' OpenSessionInViewFilter.

The database session gets opened and maintained by dpHibernate, ensuring that when BlazeDS goes to serialize a response, the database session is still available, should it be needed.



This prevents the  LazyInitializationException, and would allow the full object graph to be serialized, and returned to the flex client.


Too much data

While having the ability to return the full object graph to the client is handy, it's not very realistic in a real-world scenario.  This can often result in very bloated object graphs being sent across the wire, and needlessly incurring expensive serialization & deserialization.  DpHibernate addresses this issue, by providing true support for lazy loading of entities between Flex and Java, proxying objects being sent across the wire.

For example, here's a couple of classes from a question/answer forum application:

public class User {
String name;
Set<Post> posts;
}
public class Post {
String title;
String body;
User author;
Set<Post> replies;
}

This seemingly simple domain model turns out to be extremely expensive to serialize without lazy loading.  For example, when serializing a user, their collection of posts is serialized;  Each post includes it's own set of reply posts;  each reply includes it's author, and each author includes their full set of posts; etc.

As result, a simple call to return a single user object, could result in a heavy object graph similar to the one shown below:

public User getUser(int userId) {}

 

A common work-around to this problem is to use the DTO pattern, creating a trimmed down version of the User class with a subset of fields, specifically for transferring between layers.

Custom methods are then written for further drilling down into fields of an object, eg:

    public UserDTO getUser(int userId) {}
public List<PostDTO> getPostsByUser(int userId) {};
public List<PostDTO> getRepliesToPost(int postId) {};

Along with these, each Domain Object / DTO pair generally needs a translator to manage mapping between the two objects.


Lazy loading

By using dpHibernate and lazy loading, none of this extra work is required.  Instead, proxies are sent initially for collections and complex objects, and fetched on-demand when requested by the client. 

For example, the the same call to load a user against a dpHibernate service returns the following:




The response is significantly more lightweight and suitable for sending across to the client, as none of the complex objects – (eg., the Posts) are actually serialized in the initial call.  Instead, dpHibernate sends proxies for all the complex properties of the User.  As the client calls the getters for the proxied properties, their values are loaded asynchronously from the server, and populated back on the user object.

As the “posts”  collection is filled with proxies, it's length is still reported correctly.  This is useful when a collection is bound to a datagrid or list, as the grids still size correctly, and render scrollbars appropriately, even though much of the data hasn't actually been loaded on the client.  In fact, because of the way that lists &  dataGrids operate within Flex, only the items which are being rendered on-screen will be loaded from the server, rather than the full collection – a feature especially useful when working with very large datasets.


Getting started with dpHibernate

Lets look at how to set up dpHibernate in a Spring/BlazeDS project.  For this example, I'll be using the latest dpHibernate (2.0-RC3), along with Spring 3.0.  (Note that dpHibernate doesn't depend on Spring, and works just as well in environments not using Spring.)


Downloads

To begin with, grab the dpHibernate download from here.

Because we're working in a Spring environment, we'll need to add the following two jars to the classpath:

dpHibernate-core-2.0-RC3.jar
dpHibernate-springExtensions3.0-2.0-RC3.jar

 

Configuration:  web.xml

dpHibernate needs an active database session when sending & receiving messages with BlazeDS.  To enable this, we need a custom filter to keep the session alive:

    <filter>
<filter-name>dpHibernateSessionFilter</filter-name>
<filter-class>org.dphibernate.filters.HibernateSessionServletFilter</filter-class>
</filter>
<filter-mapping>
<filter-name>dpHibernateSessionFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

Note – this has the same effect as adding a Spring OpenSessionInViewFilter.  So, if you already have that declared in your web.xml, you can skip this step.


Spring configuration

Next up, we need to add a few custom Spring beans.  This is a simple configuration that allows for lazy loading, and batch loading (something covered off later).

    <!-- Defines the remoting adapter, which intercepts inbound & outbound messages, and routes them thruogh dpHibernate -->
<bean id="dpHibernateRemotingAdapter"
class="org.springframework.flex.core.ManageableComponentFactoryBean">
<constructor-arg
value="org.dphibernate.adapters.RemotingAdapter" />
<property name="properties">
<value>
{"dpHibernate" :
{
"serializerFactory" : "org.dphibernate.serialization.SpringContextSerializerFactory"
}
}
</value>
</property>
</bean>
<!-- Provides a basic service for lazy loading operations through dpHibernate.
It's also exported as a remoting destination, which makes it accessible to flex clients
-->
<bean id="dataAccessService"
class="org.dphibernate.services.SpringLazyLoadService"
autowire="constructor">
<flex:remoting-destination />
</bean>
<!-- The main serializer. Converts outbound POJO's to ASObjects with dpHibernate proxies for lazy loading. Required -->
<bean id="dpHibernateSerializer"
class="org.dphibernate.serialization.HibernateSerializer"
scope="prototype">
<property name="pageSize" value="10"/>
</bean>
<bean id="dpHibernateDeserializer"
class="org.dphibernate.serialization.HibernateDeserializer"
scope="prototype" />


<!-- Set up the dpHibernate adapter to be the default adapter for BlazeDS -->
<flex:message-broker services-config-path="/WEB-INF/flex/services-config.xml">
<flex:remoting-service default-adapter-id="dpHibernateRemotingAdapter"
default-channels="my-amf,my-secure-amf" />
</flex:message-broker>

Setting up the model

In order for dpHibernate to facilitate lazy loading, entity classes must implement the org.dphibernate.core.IHibernateProxy interface, both on the Java and Actionscript classes.  Additionally, Actionscript classes must either implement IManaged, or use the [Managed] metatag.

Here's a simple example:

BaseEntity.java

@MappedSuperclass
public abstract class BaseEntity implements IHibernateProxy {

@Id
private Integer id;
// Marked as Transient so the field is not persisted to the database by Hibernate
@Transient
private Boolean proxyInitialized = true;

public Integer getId()
{
return id;
}
public void setId(Integer id)
{
this.id = id;
}
@Override
public Boolean getProxyInitialized() {
return proxyInitialized;
}
@Override
public Object getProxyKey() {
return id;
}
@Override
public void setProxyInitialized(Boolean value) {
proxyInitialized = value;
}



In this example, I'm using a common base class – BaseEntity – which is the base class for all entities in the project, and implementing IHibernateProxy.

BaseEntity.as

[RemoteClass(alias="com.mangofactory.pepper.model.BaseEntity")]
public class BaseEntity implements IHibernateProxy
{
public var id:Number;
private var hibernateProxy:IHibernateProxy=new HibernateBean();

/* Constructor */
public function BaseEntity():void
{
super();
}

//==================================================
// IHibernateProxy impl..
//==================================================
public function get proxyKey():Object
{
return hibernateProxy.proxyKey;
}

public function set proxyKey(value:Object):void
{
hibernateProxy.proxyKey=value;
}

public function get proxyInitialized():Boolean
{
return hibernateProxy.proxyInitialized;
}

public function set proxyInitialized(value:Boolean):void
{
hibernateProxy.proxyInitialized=value;
}

       
Post.as (snippet)

    [Managed]
[RemoteClass(alias="com.mangofactory.pepper.model.Post")]
public class Post extends BaseEntity
{
 }

Declare & initialize a dpHibernate service

The final step is to add a dpHibernate service to the project.  This service extends flex's RemoteObject class, and provides dpHibernate's functionality within the client.

First we declare the remote object within a <Declarations /> tag:

    <fx:Declarations>
<dphibernate:HibernateRemoteObject id="dataService" destination="dataService" bufferProxyLoadRequests="true" fault="faultHandler(event)"/>
</fx:Declarations>

Then we initialize dpHibernate with this service as it's default:

    protected function onAppComplete(event:FlexEvent):void
{
// Setup a default service.
// This is used by beans to perform dpHibernate operations
HibernateManaged.defaultHibernateService=dataService;
}

Note that here we set “bufferProxyLoadRequests” to true.  This instructs the service to batch up requests to load proxies from the server where possible.  For example, when scrolling around in a List or DataGrid, several requests to fetch proxies would typically be issued to the server., (and therefore, the database).  When batching is turned on, dpHibernate uses a short delay between the first time a proxy is requested, and actually sending the call to the server.  Any other proxies that are needed during that pause get batched up into the same server call, minimizing the load on the server & database.


Setup complete!

That's it.  dpHibernate is now set up & ready to go on the project.  Calls are executed through a dpHibernate remote object the same way they would be through a normal remote object (either by declaring an operation, or be executing the method directly).

However, any results that are returned from a method call will have lazy loading enabled for all the objects which implement IHibernateProxy.


Demo Project

The above examples are taken from a demo project put together for this article – a question/answer site built on the Stack Overflow creative commons dump.  (Which is not to say I think Stack Overflow belongs in Flex, it's just a nice, freely available source of lots of data!)

The demo project displays a list of recent questions, including detailed user information, tags, comments, and the ability to drill down further into individual questions and users. 

The data is all lazy loaded on demand, keeping the amount of data transferred to a minimum, without sacrificing the richness of the flex data model.

Normally this would require many different server calls (often one per drill-down entity), translators and DTO objects.  However, thanks to the ability to lazy-load the data, only a single server method was required, and the same rich entity model is used from Java domain through to Flex domain.  (ie., No more DTO's!)


Loading some data

Now that we have our project setup and configured, dpHibernate stays out of the way.  Loading data from a remote service is exactly the same as it always was.  Let's take a look at some sample code:

  <fx:Script>
<![CDATA[
[Bindable]
public var questionList:ArrayCollection;

public function loadData():void {
var token:AsyncToken = dataService.getRecentPosts();
token.addResponder(new mx.rpc.Responder(resultHandler,faultHandler));
}
protected function resultHandler(event:ResultEvent):void
{
questionList = event.result as ArrayCollection;
}

protected function faultHandler(event:FaultEvent):void
{
Alert.show("Fault when loading data: " + event.fault.faultString);
}

]]>
</fx:Script>
<s:List width="100%" dataProvider="{questionList}" height="100%" contentBackgroundAlpha="0"
itemRenderer="com.mangofactory.pepper.view.QuestionRenderer"
borderAlpha="0"/>


Here we're using the standard responder pattern to issue a call to the getRecentPosts() method on the server, and handle the result, updating the questionList ArrayCollection.  The arrayCollection serves as the dataprovider for a List, which renders as follows:



As you scroll around within in the list, you should start to see data load from the server in batches, as required.  Note that for each Post that is loaded, we're actually also expanding their “tags” collection, as required.



Clicking and drilling down into either a question or a user causes changes the view to a drill-down view, and because of the databinding in use on each of the views, triggers the data to be loaded by dpHibernate.

So, through a service method which efficiently returned a list of Post objects, we've also been able to drill down into Tags, Replies, Users and Comments, without writing any additional server calls, and without requiring any DTO objects, or translator classes.


Beware the Asynchronous server calls!

One of the major differences between a lazy loading solution implemented in Flex and one in Java, is that Flex executes all it's server calls asynchronously.  The first time the getter of a proxied value is called, dpHibernate issues a call to the server to load the value.  However, until the server's result is received, the value on the client is  either null, or an empty proxy.  Flex's databinding makes all this happen fairly transparently when working in MXML, however when working in Actionscript, developers must be wary of null values.

For example – take a look at this dummy item renderer:

<s:ItemRenderer xmlns:fx="http://ns.adobe.com/mxml/2009" >
<fx:Script>
<![CDATA[
[Bindable("dataChange")]
public function get post():Post
{
return data as Post;
}

private function doSomething():void
{
// A NPE could be thrown here while data is being loaded
var name:String = post.author.displayName;
}
]]>
</fx:Script>
<!-- Flex will swallow any NPEs thrown here while data is being lazy loaded -->
<s:Label text="{post.author.displayName}" />
</s:ItemRenderer>



Accessing the post.author.displayName property is safe within a databinding statement, as Flex's dataBinding mechanism swallows null pointer exceptions.  However, accessing the same property from within actionscript can cause a null pointer to be thrown.


Further reading

More advanced configuration options are available to allow finer grained control over lazy loading behaviour – customizing how and when data is proxied.  This gives the developer the ability to further optimize the trips to the server.  Check out this article for more information.

Also, dpHibernate provides support for full CRUD operations from the client, with only a little more configuration than what has already been covered in this article.  For more information on the persistence features, check out this article.  Also, the demo project we've been working on today has been further enhanced with full persistence features.  Check out the project here. 

Finally, the dpHibernate project site and mailing list are available for support.


Conclusion

In this article you've seen how easy it is to get dpHibernate set up and running with lazy loading.  By using lazy loading to return a rich domain model from the server, we can minimize the amount of boilerplate services and code required to pass data between the client and server.  dpHibernate is designed to quietly work behind the scenes, with minimal configuration and pollution of your entity model.  


About the Author

Marty Pitt is an independent software developer who specializes in enterprise Java / Flex applications.  He is the lead developer of dpHibernate, and the upcoming code review tool, Marmalade.

AttachmentSize
demo screenshot 1.jpg45.45 KB
demo screenshot 2.jpg28.5 KB
demo screenshot 3.jpg40.62 KB
Payload with dpHibernate.jpg20.14 KB
Payload without dpHibernate.png129.46 KB
Server overview - dpHibernate.jpg63.73 KB
Server overview - no dpHibernate.jpg64.37 KB
Thumbs.db_.zip65.17 KB
uml.jpg17.45 KB
Published at DZone with permission of its author, Marty Pitt.

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Mark Unknown replied on Thu, 2010/10/21 - 9:00am

I know it is not the point of the article but the DTO pattern is not what you describe. It is pattern to deal with network issues, not layers. DTOs do not contain a subset of attributes. They contain all the attributes to "rehydrate" a full domain object. This was [mainly] a EJB 1/2 issue. What you describe (and every other person who does the same thing) is a "View" object or a partial domain object. http://java.sun.com/blueprints/corej2eepatterns/Patterns/TransferObject.html Anyway, I will look more at this project. Having to create a whole set of objects because the client is not the same technology as the server is a MAJOR pain. FYI, GWT has this very same issue.

Dimitris Menounos replied on Thu, 2010/10/21 - 10:34am

OpenSessionInViewFilter is not a good solution. Instead transaction boundaries should align with service methods and the resulting entities be pre-hydrated as neccessary. Finally any hibernate proxy leftovers (of properties not loaded) should be replaced with null prior to serialization.

 

Marty Pitt replied on Thu, 2010/10/21 - 3:14pm in response to: Dimitris Menounos

Hi

> and the resulting entities be pre-hydrated as neccessary.

Unfortunately, this is an extremely expensive alternative in terms of bandwidth.

If I understand your intentions (please correct me if not), pre-hydrating an object is essentially the same as "serializing the full object graph", which is not practical when sending an payload over a network connection, such as the internet.

The aim of this project is to keep the payload as small and as immediately relevant as possible, and lazy load the rest of the data on demand, with minimal friction.

Daz Za replied on Thu, 2010/10/21 - 6:15pm

Hi Marty, so glad to see that dpHibernate has almost released 2.0 status. The Hibernate lazy-loading issue has been a big sticking point for developers wanting to move to BlazeDS ever since it was released.

Can you tell me how your approach differs to to that takien with LiveCycle?

Charlie Mordant replied on Fri, 2010/10/22 - 2:40am

Hi,

Don't  you think that lazy loading is here for something?

Either you can put your relation in eager mode or make a finder, so it implies you have to brainstorm on your model and DAO objects.

This kind of pattern leads to brainless model programming, and I think it's a bad thing.

In the other hand, your framework is good, and deals with some common ejb programming problematics, thank you for that nice job.

 

Jack Smack replied on Sat, 2010/11/20 - 4:49pm

Marty - thanks for the great article, and the great tool. Are there setup instructions for using this with ColdFusion? For those of us that are not JavaHeads. Thanks!

Tu Pham replied on Tue, 2010/11/23 - 11:32pm

Hi,

This is good artical about entity managment, I wonder that dpHibernate work as filter and each time there is request come, it perform initize session by hold reference to SessionFactory and perform attacted object to session, get data and return to client. But when use as filter may leading to Session leak.

 

Regards,

Damian Pooley replied on Tue, 2010/11/23 - 11:50pm

 

Dimitris Menounos suggest what close to what I'll describe below; which answers your main concern Marty (bandwidth, how much serialised).

An alternative I 'copied' from a former Architect and overall cluey guy looks at the requirement: transferring data from A to B without a continious open session this way:

Firstly, what the client is doing is asking for information. Limit the response (as much as possible) to what the client wants. If the client wants more data, they can ask for it. Now, DTO/VO (current project suffers from calling a VO a DTO) might be what jumps into people's minds here but no, this isn't that. I also think that creating an entirely new marshalling and unmarshalling data layer and all the code and unit tests and maintenance required to support it is wasteful and missing the point of OO and ORMs like hibernate.

Instead of a complete object graph being sent off for serialisation (that's not a good thing either), its up to the ask to ask for what they want, in an OO manner. For example if the client wanted to display a User and all their Posts (but no Reply(ies) because the client has to initiate a button press to show that data, the code would be like something this:

User askForUser = (User) LoadingPolicyFactory.ask(User.class); // creates a User proxy
askForUser.getUsername(); // fetch the Username - although fields will probably be loaded by default
askForUser.getPosts().getText() // initialise a collection example
... more method calls to explicity get relationships and possibly fields if wanting to be very specific

By using interfaces and Proxy objects the actual domain model can be used to define the data that will be loaded and serialised back to the client. If more data is required (and exists in the graph), just some additional method calls are required.

This technique allows client's to define what they want in a very simple manner. There is no need to know two or more domain models (and those who think exposing the domain model is bad... maybe this model isn't the real model? They could be bluffing! And well... if a hacker gets into your system they're probably good enough to get to the database directly or cause whatever mischief they want regardless), and there is no link between the client and server outside of the service contacts.

Now if a clever programmer calls a method on a field that isn't initialised they'll still get a LazyLoadException. We should change this to LazyProgrammerException. Obviously if the data isn't loaded you'll die, so ask for the data and stop complaining about the tool. The serialisation mechanism knows about the LoadingPolicy so it won't call methods that weren't specified. In our case because it was all in the same VM and java on the client and server, clones were created or the loaded data set into a full-graph template (that the client needed, such as a JSF form). If some bright spark on the client end calls for an unloaded piece of data they'll get null or a NullPointerException instead (but at least not a Hibernate exception, those are wrong!! :p ). Again, ask and you shall receive.

I had also considered the above way (but a proxied domain loading up its service bean from spring - ouch) of doing it and had two issue:

  1. If the service is a true 'service', giving data that is persistence or connection aware to remote/barely trusted clients probably isn't a good thing.
  2. Coupling the client and its use of the domain to the services. It's cleaner, imho, if the client truly be operating with POJOs and only interact with the service tiers via the service contract. When a 'backdoor' is opened, expect programmers to knock down some more and goodby architecture.
That said, when 'you' own everything (the application and environment), those issues don't matter really.

Dane Bezuidenhout replied on Sat, 2010/11/27 - 1:22am

Thanks for the article. What is very unclear from the dpHibernate documentation and forum posts available is if dpHibernate 1.3/2.0 will work with JPA, i.e. without a Hibernate config.xml file or a Hibernate session factory. I am using Flex 3, BlazeDS 3.2, JPA/Spring 3 (backed by Hibernate). When I use the config above then I get:

Error creating bean with name 'dataAccessService' defined in class path resource [META-INF/spring/spring-flex-servlet.xml]: Unsatisfied dependency expressed through constructor argument with index 0 of type [org.hibernate.SessionFactory]: : No matching bean of type [org.hibernate.SessionFactory] found for dependency: expected at least 1 bean which qualifies as autowire candidate for this dependency.

I.e. there is no Hibernate session factory to autowire into SpringLazyLoadService (I don't have one configured as I'm using JPA). How can I remedy this? Is there a way to configure JPA to work with the dpHibernate beans above? -------- Update: I have created a sessionFactory bean that retrieves a session factory from my entityManagerFactory set up by Roo, and wired this to the SpringLazyLoadService bean. Theoretically this should provide a Hibernate session factory to dataAccessService above, but I get the following as soon as a client request is made to the application: java.lang.NullPointerException at org.dphibernate.filters.AbstractHibernateSessionServletFilter.doFilter(Unknown Source)

Jogesh Shamra replied on Fri, 2011/02/11 - 7:29am

This article is about the transfer of the graphs from java to flex. It is a lazy exercise to transfer the complex graphs from the java to flex. The aim of this project is to keep the payload as small and as immediately relevant as possible, and lazy load the rest of the data on demand, with minimal friction. While having the ability to return the full object graph to the client is handy, it's not very realistic in a real-world scenario.  This can often result in very bloated object graphs being sent across the wire, and needlessly incurring expensive serialization & deserialization. <a href="
http://www.cartagenahostels.com" rel="dofollow">
Hostel Cartagena</a> is also making the use of java to save their records.

Gar Labs replied on Sun, 2011/10/23 - 10:00am

Hi.

 I am pretty new to the stuff you mentioned above. To start things off, I would like to know what is lazy loading? I don't have the slightest clue of what it is. - GAR Labs

Carla Brian replied on Wed, 2012/07/18 - 8:17am

This make sense actually. Now  I know the reasons. Good thing I saw your post. I have learned new things today. - Mercy Ministries

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.