1

I was reading this article, and the author uses HTTP long polling to design a chat application.

I was wondering how does this compare with a dedicated Message Broker services like Apache Kafka, or RabbitMQ. I'm trying to understand system design, and I am confused about HTTP Long Polling.

  • Is it something can work with a Message Broker? Or is it an alternative to a Message Broker?
  • Or is it something that a typical Message Broker is build on top of?
  • Or is Long Polling a feature supported by a server, so it can work irrespective of if REST/GRPC/AMPQ etc is being used?

I'm not able to find any articles online that talk about these things together. If anyone can put these blocks together and explain, that'd be great!

Edit: I was also watching this video on the same topic, where the author puts a websocket-handler layer on top of Kafka/Redis. Not sure if the intention of this layer was to simply contain the application logic for Kafka/Redis or to have actual websocket connection handling code.

Improve this question

2 Answers 2

Reset to default
4

HTTP long polling is just a buzzword for opening an HTTP connection to a web server, and keeping it open in order to repeatedly receive chunks of data. This was a workaround for web browsers that did not support Web Sockets or Server-sent events and did not have a mechanism to do real-time updates using streaming data.

Think of Web Sockets and server-sent events in a browser as the client side compliment to traditional message brokers like RabbitMQ and Kafka.

For browsers that did not support Web Sockets, HTTP long polling allowed the client (the web browser) to open a connection to a web server. The browser would keep the connection open by specifying a long "timeout" period. The client initiates the connection. JavaScript configured the XMLHttpRequest object to automatically disconnect after a long period of inactivity (hence the "long" in "HTTP long polling"). This allows the web server to use chunked transfer encoding to send multiple discrete messages back to the browser to simulate the kind of streaming data you would get from a message broker.

JavaScript needed to be written to process each chunk separately and automatically reconnect to the server after exceeding the inactivity timeout in order to complete the "live updates" effect.

I remember implementing this once about 10 years ago. I don't remember that we needed to configure the web server any different. HTTP long polling is not a feature of a web server. It basically means "obscenely long inactivity timeout for both server and client."

Since the client initiates any kind of connection to the web server, the client is also responsible for closing the socket connection. I imagine both client and web server need to be configured to allow open connections with no activity for an extended period of time. I think we settled on 1 minute.

Since I last implemented HTTP long polling, modern browsers support Web Sockets and server-sent events. Prefer web sockets if you need bi-directional communication or to transmit binary data. Server sent events are preferred for things like push notifications, where transmitting data in text is all you need, and the client does not need to send data back to the server.

Improve this answer
6
  • 1
    Great answer. You may also mention Server-sent events, as the first choice for anything requiring to receive notifications. Web sockets is more complex, and should be used only when Server-sent events are not enough (for instance when one needs to transmit binary data, or to have a bi-directional communication).
    – Arseni Mourzenko
    May 17, 2021 at 15:47
  • Thanks! That helps a lot. Can you shed some light on how it compares to Message Brokers, and in which situation would you prefer web sockets (or HTTP Long Polling) vs Message Brokers?
    – Ufder
    May 17, 2021 at 15:54
  • @ArseniMourzenko: thanks for the info. I will add that to my answer.
    – Greg Burghardt
    May 17, 2021 at 15:58
  • @Ufder: Use long polling when you need to support old browsers. HTTP long polling was basically a workaround from the first day it was used. Web Sockets (and server-sent events) are the successor technologies supported by newer browsers.
    – Greg Burghardt
    May 17, 2021 at 15:59
  • 1
    Got it, and thank you! Can you compare this to message brokers? Still unclear about that part!
    – Ufder
    May 20, 2021 at 1:12
0

Message brokers are suitable for server to server communication, such as in a micro-service architecture. web-sockets are used for client-server architecture, where client is a web browser. Http Polling can be used for both and I assume so can be long polling.

Improve this answer
1
  • 1
    Obviously message brokers are not used in an browser based internet application, but client/server does not stop to one implementation even if it is the most common one.
    – FluidCode
    May 16, 2022 at 12:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.