3

Currently I have implemented a simple UDP server using java sockets, as soon as the packet is received it is added to the queue and there are four consumer threads which process the queue.

This approach is fine till 10000 req/sec, but there is significant packet loss of packets as the number of packets/sec is >10000 req/sec.

After careful investigation the reasons are queues, due to latency of adding the packet to the queue and socket.receive blocks till the packet is added to the queue

Also used the datagram channel which is non blocking IO but the results are poorer than the blocking IO(java socket). Here is sample code

try (DatagramSocket socket = new DatagramSocket(port)) {
            System.out.println("Udp Server started at port :" + port);


            while (true) {
                byte[] buffer = new byte[1024];
                DatagramPacket incomingDatagramPacket = new DatagramPacket(buffer, buffer.length);
                try {
                    socket.receive(incomingDatagramPacket);
                    LinkedTransferQueue.add(incomingDatagramPacket);
                } catch (IOException e) {
                    e.printStackTrace();
                    continue;

                }


            }

        } catch (SocketException e) {
            e.printStackTrace();
        }

Is there any other proven way to design UDP server in a better way ?

enter image description here

  • You need a simpler data structure, preferably one that doesn't need to allocate memory. Have you considered a ring buffer (e.g. as used in LMAX Exchange's "Disruptor" architecture)? – Jules Aug 20 '18 at 21:53
  • @Jules I will try and update the results here – forum.test17 Aug 21 '18 at 15:00
  • Perhaps it's time to think about how to distribute it over multiple servers. Every machine has its limit. – dagnelies Aug 21 '18 at 15:42
  • I know it's a silly question but what is the CPU and memory usage on the server running the java app? – Richard Aug 21 '18 at 21:57
  • Just to give you an idea.. I have friends that run a java UDP service handling a popular dynamic DNS service. Their service handles hundreds of millions of requests a day.. and until recently was running on an aws micro instance. How big are the UDP packets? – Richard Aug 21 '18 at 22:00
5

You didn't note whether the latency is due to the queue depth or if it's the time it's taking to get onto the queue by itself. There is one iron-law of queuing: if your overall rate of inbound messages exceeds the overall rate of consuming messages, the queue depth will grow over time. The maximum size of the queue is simply a buffer that allows for a temporary imbalance. You are using an unbounded queue but there are implicit limits. As you grow the queue, you will need to allocate memory and that takes time. At some point, there will be no more memory available and that's your hard limit. You need the consumers to read faster that you write. So, if your consumers can't keep up, you need to scale to more consumers. If your consumers are IO bound, consider NIO options. If they are CPU bound you need more threads/cores/servers.

If it is that your queue itself is the bottleneck, you might want to consider the Disruptor data structure that was developed as part of the LMAX project. They claim that it can achieve much higher throughput. If nothing else, you will learn about some really smart software engineering. The main big idea here is that the design of a queue forces excessive L3 cache invalidation and the Disruptor structure avoids this.

  • Any other approach as I am not allowed to use any third party libraries which are not from apache foundation :( – forum.test17 Aug 21 '18 at 14:58
  • Have you determined where the bottleneck is? A simple way to do that is to watch the queue size. If you've got a lot of messages in the queue when things bog down, it's the reading that's the lag, not the writing. If things are bogging down and you have an empty queue, it's writing to the queue that's the problem. Keep in mind that this is a kind of a whack-a-mole thing. As you eliminate one bottleneck side, another part of the system becomes the limiting factor. – JimmyJames Aug 21 '18 at 16:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.