Configuring persistence of fields/properties using TypeConverters with JDO

JPA 2.1 allows a user to specify a converter on the value of a field/property for how it is persisted in the datastore. The way we implement that in DataNucleus is to have the JPA converter as a wrapper to our own internal TypeConverter class. This means that we can make the TypeConverter mechanism available to JDO users too. Here’s how it works. The first thing to do is to define a TypeConverter. As you can see, it requires implementation of just 2 methods, one for use in converting the (raw) field value into the datastore value, and one for use in converting the datastore value back into the (raw) field value. This means that the user has significant flexibility on how their values are stored, and can define their own converters and not rely on them being part of DataNucleus. If we take an example, here we want to store a field as serialised, but not standard Java serialised, instead using the Kryo library. So we define our TypeConverter as

public class KryoSerialiseStringConverter implements TypeConverter
{
    ThreadLocal kryo = new ThreadLocal();
    public Kryo getKryo()
    {
        Object value = this.kryo.get();
        if (value == null)
        {
            value = new Kryo();
            this.kryo.set(value);
        }
        return (Kryo)value;
    }

    public String toDatastoreType(Serializable memberValue)
    {
        if (memberValue == null)
        {
            return null;
        }

        Kryo kryo = getKryo();
        String str = null;
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        Output output = null;
        try
        {
            output = new Output(baos);
            kryo.writeClassAndObject(output, memberValue);
        }
        finally
        {
            output.close();
            str = new String(Base64.encode(baos.toByteArray()));
        }
        return str;
    }

    public Serializable toMemberType(String datastoreValue)
    {
        if (datastoreValue == null)
        {
            return null;
        }

        Kryo kryo = getKryo();
        byte[] bytes = Base64.decode(datastoreValue);
        Object obj = null;
        ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
        try
        {
            Input input = null;
            try
            {
                input = new Input(bais);
                obj = kryo.readClassAndObject(input);
            }
            finally
            {
                try
                {
                    bais.close();
                }
                finally
                {
                    if (input != null)
                    {
                        input.close();
                    }
                }
            }
        }
        catch (Exception e)
        {
            throw new NucleusException("Error Kryo deserialising " + datastoreValue, e);
        }
        return (Serializable)obj;
    }
}

Now we need to register this converter under a name with DataNucleus runtime. Here we define a plugin.xml at the root of the plugin jar as

<type-converter name="kryo-serialise" member-type="java.lang.Serializable" datastore-type="java.lang.String"
converter-class="org.datanucleus.store.types.converters.KryoSerialiseStringConverter"/>

So this converter is internally known as kryo-serialise. We add a MANIFEST.MF to our plugin jar as (something like)

Manifest-Version: 1.0
Bundle-ManifestVersion: 2
Bundle-Name: DataNucleus Converter using Kryo
Bundle-SymbolicName: org.datanucleus.store.types.converters.kryo;singleton:=true
Bundle-Version: 3.2
Bundle-Localization: plugin
Bundle-Vendor: DataNucleus
Require-Bundle: org.datanucleus
Import-Package: com.esotericsoftware.kryo;version="2.21",
 com.esotericsoftware.kryo.io;version="2.21"

just defining the dependencies of this plugin for OSGi and the plugin mechanism. Now we just need to use it on a sample class and apply the conversion to a field/property.

@PersistenceCapable
public class Sample
{
    @PrimaryKey
    long id;

    @Extension(vendorName="datanucleus", key="type-converter-name", value="kryo-serialise")
    String longString;

    ...
}

Now whenever instances of Sample are persisted the field longString will be converted using our kryo-serialise converter. Easy! You can get the code for this example converter at DataNucleus GitHub repo

Advertisements
This entry was posted in DataNucleus, java, JDO, Persistence. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s