Skip to content

Jackson: Serializing and Deserializing Extended Types #238

Closed

Description

@fdinoff

Currently we can deserialize an ExtendedType into a MessagePackExtendedType with an ObjectMapper from the getEmbbededObject method in MessagePackParser

One way of deserializing the object would be to have an customer deserializer. The following is an example of that where we have a java class called Buffer that is represented as an extension type where the extType is called bufferType. The contents of the extension type is just a Message Packed integer.

module.addDeserializer(Buffer.class, new JsonDeserializer<Buffer>() {
    @Override
    public Class<Buffer> handledType() {
        return Buffer.class;
    }

    @Override
    public Buffer deserialize(JsonParser jsonParser, DeserializationContext deserializationContext) throws IOException, JsonProcessingException {
        MessagePackExtendedType extendedType = (MessagePackExtendedType) jsonParser.getEmbeddedObject();
        if (extendedType.extType() != bufferType) {
            throw new IllegalStateException("extendedType != bufferType");
        }
        byte[] b = new byte[extendedType.byteBuffer().remaining()];
        extendedType.byteBuffer().get(b);
        long id = MessagePack.newDefaultUnpacker(b).getCursor().nextRef().asInteger().asLong();
        return new Buffer(id);
    }
});

This to me feels brittle since the object deserializer has guessed the extType before reading it. If it guessed wrong it just throws an exception.

I think a more robust solution would be to have getEmbeddedObejct return the Buffer class itself. However I don't know if this is any better.


On the other side, there does not appear to be any way to serialize a class into an Extended Type. If we add a writeExtendedType to the MessagePackGenearator class we could add a custom serializer such as the following.

module.addSerializer(Buffer.class, new JsonSerializer<Buffer>() {
    @Override
    public Class<Buffer> handledType() {
        return Buffer.class;
    }

    @Override
    public void serialize(Buffer buffer,
                          JsonGenerator jsonGenerator,
                          SerializerProvider serializerProvider) throws IOException, JsonProcessingException {
        MessagePackGenerator generator = (MessagePackGenerator) jsonGenerator;

        ByteArrayOutputStream out = new ByteArrayOutputStream();
        MessagePack.newDefaultPacker(out)
                .packLong(buffer.getId())
                .close();
        MessagePackExtendedType extendedType =
                new MessagePackExtendedType(bufferType, ByteBuffer.wrap(out.toByteArray()));
        generator.writeExtendedType(extendedType);
    }
});

Where writeExtendedType was made using the following patch

diff --git a/msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java b/msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java
index bc23e13..0375c27 100644
--- a/msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java
+++ b/msgpack-jackson/src/main/java/org/msgpack/jackson/dataformat/MessagePackGenerator.java
@@ -175,6 +175,12 @@ public class MessagePackGenerator extends GeneratorBase {
         else if (v instanceof Boolean) {
             messagePacker.packBoolean((Boolean) v);
         }
+        else if (v instanceof MessagePackExtendedType) {
+            MessagePackExtendedType extendedType = (MessagePackExtendedType) v;
+            ByteBuffer buf = extendedType.byteBuffer();
+            messagePacker.packExtendedTypeHeader(extendedType.extType(), buf.remaining());
+            messagePacker.writePayload(buf);
+        }
         else {
             throw new IllegalArgumentException(v.toString());
         }
@@ -324,6 +330,10 @@ public class MessagePackGenerator extends GeneratorBase {
         addValueToStackTop(null);
     }

+    public void writeExtendedType(MessagePackExtendedType extendedType) {
+        addValueToStackTop(extendedType);
+    }
+
     @Override
     public void close() throws IOException {
         try {

I'm not sure if this is the best way to handle this though.

What should the canonical way to serialize and deserialize extended types with the jackson api be?

Activity

  1. self-assigned this
    on May 29, 2015
  2. added this to the v07 milestone on Jun 23, 2015
  3. komamitsu commented on Jun 28, 2015

    @komamitsu
    Member

    @fdinoff I added org.msgpack.jackson.dataformat.MessagePackExtensionType.Serializer in #250. That way you can serialize an extension type (the name changed from extended type in #246) now

  4. fdinoff commented on Jun 28, 2015

    @fdinoff
    ContributorAuthor

    MessagePackParser#getEmbeddedObject:401 no longer returns org.msgpack.jackson.dataformat.MessagePackExtensionType, it returns ImmutableExtensionValue. Should writeExtendedType take an ExtensionValue instead? (And the name should be changed to writeExtensionType/writeExtensionValue)

  5. komamitsu commented on Jun 29, 2015

    @komamitsu
    Member

    Good point. The reason I used MessagePackExtensionType instead of ExtensionValue is to use JsonSerialize annotation that tells Jackson to serialize it with MessagePackExtensionType.Serializer.

    I forgot the change of MessagePackParser#getEmbeddedObject and I'll modify it to return MessagePackExtensionType for consistency.

  6. fdinoff commented on Jun 29, 2015

    @fdinoff
    ContributorAuthor

    Another option is to create a module and use mixins. It might be worthwhile if you want to add annotations to the other types as well. (Although if you go this route you don't need that annotation since you created the serializer)

  7. komamitsu commented on Jun 29, 2015

    @komamitsu
    Member

    Fixed the inconsistency in #262.

    As for module, it's what I considered but I decided not to use it in this case because of the following reasons:

    • MessagePackGenerator can access ObjectCodec in the constructor that is the parent class of ObjectMapper and ObjectReader.
    • Only ObjectMapper has registerModule method while ObjectReader doesn't have the method.
    • If we register a module to inject a serializer for Extension type into ObjectMapper, we need to check ObjectCodec codec passed in the constructor is ObjectMapper or not. It can be a naive implementation.
  8. fdinoff commented on Jun 30, 2015

    @fdinoff
    ContributorAuthor

    Thanks, that makes sense. I created a pull request with a couple of suggestions #264. Thanks for adding this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions