2 years ago I made a post about performance/benchmarking, and the fact that some groups like some magic black and white “X is better than Y” (and that there is only one measure of performance so it doesn’t matter what object graphs are used it will always be the same). The evidence is that they are wrong. Needless to say there will always be groups that don’t share our philosophy, or don’t have time to do a complete analysis (though publish their results knowing that they are incomplete and likely invalid, after all it’s not their software they’re maybe not presenting in a fair light). Recently we had another performance exercise. This came to the conclusion “Hibernate is better than DataNucleus, and you should really just get ObjectDB“. So we’re back in the territory of black and white. Yes, an OODBMS ought to be way faster than RDBMS, particular when the RDBMS has a persistence layer in front of it (and you have to pay for the OODBMS besides), but that is not the subject of this post. We’ll concentrate on the former component of that conclusion.
There is nothing to add to the previous blog post in terms of correctness, we stand by all of it and nothing has been demonstrated to the contrary. This blog post simply takes the recent exercise sample and demonstrates how enabling/disabling certain features has a major impact on (DataNucleus) performance. The author of that exercise demonstrated results showing that JDO and JPA with DataNucleus were on a par in terms of performance, but below Hibernate in terms of INSERTs (anything between 1.5 and 2 times) and on a par for SELECTs (some faster, some slower but more or less the same). Since JDO and JPA are shown to be equivalent, we’ll just run the exercise with JDO here, but the same is easily demonstratable using JPA (because in DataNucleus you have full control over all persistence properties and features regardless of API).
The sample data used by this case is that of 3 classes. Student has a (1-N unidirectional) List of Credit and has a (1-1 unidirectional) Thesis. We persist 100000 Students each with 1 Credit and 1 Thesis. So that’s 300000 objects to be inserted, and then 100000 Students queried.
The INSERT is as follows
try
{
pm.currentTransaction().begin();
for (int x = 0; x < 100000; x++);
{
Student student = new Student();
Thesis thesis = new Thesis();
thesis.setComplete(true);
student.setThesis(thesis);
List credits = new ArrayList();
Credit credit = new Credit();
credits.add(credit);
student.setCredits(credits);
pm.makePersistent(student);
}
pm.currentTransaction().commit();
}
finally
{
pm.close();
}
try
{
Query q = pm.newQuery(
"select from " + Student.class.getName() +
" where thesis.complete == true && credits.size()==1");
Collection result = (Collection) q.execute();
... loop through results, so we know they're loaded
}
finally
{
pm.close();
}
So we’ll run (on H2 database, on a Core i5 64-bit PC running Linux, 4Gb RAM) and vary our persistence properties to see the effect.
Original persistence properties (from original author)
Disabled L2 cache
Disabled Optimistic Locking
Use BoneCP connection-pooling
Conclusion
As you can see, with very minimal tweaking we’ve reduced the INSERT time by a factor of 3, and the SELECT time by a factor of 1.7! That would equate to being noticeably faster than Hibernate in the authors original timings (for both INSERT and SELECT). Note that we already had the detach flags set to not detach anything, so they didn’t need tuning (but should be included if you hadn’t already looked at those in your performance tests, similarly all of the other features listed in the Performance Tuning Guide referenced above).
Does the above mean that “DataNucleus is faster than Hibernate” ? Not as such, it is in some situations and not in others. We can turn on/off many things and get different results, just as Hibernate likely can (though I’d say DataNucleus is more configurable than the majority if not all of the other persistence solutions so at least you have significant flexibility to do this with DataNucleus). In the same way we could persist other object graphs and get different results due to some parts of the persistence process being more optimised than others. One thing you can definitely say is that DataNucleus has very good performance (300000 objects persisted in 42secs on a PC, and 100000 objects queried in less than 4secs) and that performance can be significantly tuned.
The other thing that we said in the original blog post and repeat here, if you are serious about performance analysis you have to dig into the details to understand why and, as a consequence, you have an idea what to tune. You also need to assess what your application really needs to perform and what is considered acceptable performance; if you’re not going to make a proper attempt at tuning a persistence solution (whether that is DataNucleus, Hibernate, or any other), best not bother at all and just use what you were going to use anyway since you don’t have the time to give a fair representation (which is why we don’t present any Hibernate results here, so nothing hypocritical in that).
One important thing to note is that it is extremely useful to have the ability to set many of these properties on a PersistenceManager (or EntityManager) basis (so you could have a PM just for bulk inserts and disable L2 caching, or set the transaction to not be “optimistic”). JDO 3.1 adds the ability to set persistence properties on the PersistenceManager, though DataNucleus only currently supports a minimal set there – SVN trunk now has the ability to turn off the L2 cache in a PM while have it enabled for the PMF as a whole.
DataNucleus 