Say someone calls a web service on a server, the relevant data is being retrieved in batches and the message is being streamed to the client. In the middle of the message some kind of exception is encountered. My current implementation sort of just stops streaming the (in this case soap) message and starts streaming an error message, but obviously the result won't be a valid message or even valid XML.
I considered a few options, none of them seem to make much sense:
- Put all the relevant data in memory and then only stream the message itself.
- Do a double-take, so see if it works out with a fake stream and then do the real stream.
- Stream to a disk first and then use a file input stream to stream to the client
Because this is an implementation for a generic application platform, I do not know how big the messages will be, but I have heard people are using web services in our app platform with gigabytes of data.
As you can see all have serious downsides in terms of memory/disk consumption, speed/resource efficiency or even validity of the data. How would I best implement error handling in this situation? Do I just have to pick the least worst of these options? If so I tend to favor to the current situation where the message is just invalid in cases like these, because it at least doesn't affect the base case where everything is fine.