How to design resilient web service clients?

Question

I am writing an app that is supposed to talk to a web service. Let's say it's a chat app. It would need to receive a stream of new messages (so that they are pushed to the interface as quickly as possible), as well as send its own messages. Furthermore, I'd like the interface to be as responsive as possible, so immediately after the user sends a message, it should be visible in the list of his messages. However, the network might be faulty, or the chat server may be unavailable. Therefore, after the user presses the "send" button, his message is not guaranteed to be delivered to the server at all. The app should try to reach the network, and in case of failure notify the user. The user then have the option to retry or give up entirely.

---

I am trying to come up with a good design for this use case. My current idea is this: I am implementing a proxy object for the chat server — essentially, I am re-implementing a part of the server itself. I am running two separate processes, one of which maintains message histories that the user has with all of his peers. Let's say the user has n peers. Then this process runs n+2 threads, of which:

• n are responsible for polling for updates from each peer and pushing them to a queue. Let's call them Event Feeds.
• 1 thread consumes that queue and is responsible for maintaining a local copy of message histories. It also exposes an interface to send new messages and to subscribe for event updates. Let's call it State Maintainer.
• 1 thread executes requests to send new messages, which it receives from the State Maintainer. Let's call it Sender.

The interface exposed by State Maintainer is powered by ZeroMQ. It is composed of two sockets: one REP and one PUB. On each iteration of the internal loop, this thread retrieves updates from the queue from Event Feeds, pushes them to the PUB socket and checks for new requests to send messages from the REP socket. For each new request, it pushes the update to the message history it maintains (setting a flag maybe sent on it) and transfers it to another queue. This queue is consumed by the Sender. For each new item in the queue, it attempts to perform a request to the chat server. In case of success, it pushes an event confirming success to a queue which is consumed by the State Maintainer, so maybe sent will become sent. In case of failure, it pushes an event to the same queue, so that State Maintainer will change maybe sent to not sent. In either case, the result is sent through the REP socket.

The UI process is connected to the proxy process via two ZeroMQ sockets. It runs two threads. The primary thread receives updates on the SUB socket and handles the UI. It talks to the secondary thread via two queues. When the user sends a new message, it pushes that message to the queue. The secondary thread picks it up and sends it to the REQ socket, storing a unique ID for that request. When the answer arrives (which can be distinguished by the UID), it pushes it to the second queue, which allows the UI thread to update the interface.

So I'd like to ask:

• Is this approach good as a general approach to building resilient web service clients? Having a proxy, maintaining a piece of the remote state, two channels for information exchange between the proxy and the app logic?
• What are some general practices of solving this problem? For example, I heard that Telegram is very fault-tolerant. Is it worthwhile crunching through its networking code?

Answer 1

It looks like you want to spool the result of HTTP requests. Indeed, you might use some caching HTTP proxy like squid (you might use it, instead of coding your own).

You could use both HTTP client libraries (like libcurl) and HTTP server libraries (like libonion) in the same process (but probably in different threads, or thru some event loop like libev or libevent).

You still have to define some staleness criteria (and that would be application and domain specific; e.g. weather forecasts are surely rotten in a few days!), and you might want (perhaps that is not a very good idea) to pool the servers periodically.

You could also use (and provide) the Expires and Age HTTP headers (in replies).