I am writing a network wrapper around boost::asio and was wondering what is a good and simple way to serialize my messages. I have a message factory which can take care of dispatching the data to the correct builder, but I want to know if there are any established solutions for getting the binary data on the sender side and consequently passing the data for deserialization on the receiver end.

Some options I've explored are: passing a pointer to a char[] to the serialize/deserialize functions (for serialize to write to, and deserialize to read from), but it's difficult to enforce buffer size this way; building on that, I decided to have the serialize function return a boost::asio::mutable_buffer, however ownership of the memory gets blurred between multiple classes, as the network wrapper needs to clean up the memory allocated by the message builder.

I have also seen solutions involving streambuf's and stringstream's, but manipulating binary data in terms of its string representation is something I want to avoid. Is there some sort of binary stream I can use instead? What I am looking for is a solution (preferrably using boost libs) that lets the message builder dictate the amount of memory allocated during serialization and what that would look like in terms of passing the data around between the wrapper and message factory/message builders.

PS. Messages contain almost exclusively built-in types and PODs and form a shallow but wide hierarchy for the sake of going through a factory.

Note: a link to examples of using boost::serialization for something like this would be appreciated as I'm having difficulties figuring out the relation between it and buffers.

  • 1
    if you are passing pointer you should also pass the length of the remaining memory available to be used (otherwise it's a bufferoverflow waiting to happen) Commented Sep 24, 2012 at 11:58
  • I am doing that, but was looking for a way to invert things - instead of giving the serializer memory to work with, have it give me the memory it allocated, since it knows how much it needs. Also, if I just use one buffer it'll need protecting for threads and eventually become contested. If the wrapper allocates memory to give to the message builder, it's the same as the builder doing it itself. What's the best way to have the builder manage its memory is my question
    – mtsvetkov
    Commented Sep 24, 2012 at 12:02
  • @mtscetkov if you do that you'll get a lot of allocations, most C++ implementations will pass in the buffer and return it (I'm thinking of the cstreams operator<< as bad as the choice of operator is, not allocating the strings to copy into the output buffer is a good idea) Commented Sep 24, 2012 at 12:48
  • std::string is an arbitrary array of bytes and can contain NUL bytes and even more so std::stringstream is simply a memory-based stream. Using it for binary data is normal and expected.
    – Jan Hudec
    Commented Sep 25, 2012 at 12:55

1 Answer 1


Since you can't have your cake and eat it (see comment thread for the question), I've decided on the following solution:

The network wrapper will keep a boost::asio::streambuf, while serialization/deserialization will happen over ostream/istream. This way the message builders don't have to deal with memory management, can use as much/as little as they need and can choose whether they want to serialize as binary or text. As an added benefit, I can also easily serialize messages to/deserialize from a file for record/replay during debugging.

  • I think this is most reasonable approach.
    – Jan Hudec
    Commented Sep 25, 2012 at 12:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.