0

I'm currently designing a server with the following structure:

  • A TCP thread pool that receives data from the network
  • A queue that holds these requests
  • A worker thread pool of a fixed size that takes requests from the queue and executes some work where a ConcurrentHashMap is read, and results analyzed. After the work is complete, the results are posted to a write queue.
  • The write queue stores write requests for ConcurrentHashMap
  • A write thread pool of a fixed size that takes requests from the write pool and writes them to the ConcurrentHashMap

Server constraint:

  • For a particular entry in ConcurrentHashMap, a write will get posted to the network several seconds before a read gets posted to the network. The threads must never allow the read to get scheduled before the write occurs.

My ideas so far:

  • First, I'm hoping that the 3-5 seconds or so is a big enough head start for a write to get through before the read
  • Set the write threads to have high priority and the read threads to have low priority
  • A read can check the last write time, and if large enough (like several minutes or hours ago), it knows to ignore the last write because it is out of date. However, it then knows that it can't make an informed decision. I could have the thread retry like twice in the case of an outdated entry, but this feels clumsy.
  • Added the write queue so that the TCP pool can post directy to the write queue (write requests don't need to be processed by the worker pool) instead of the job queue while read requests are put in the worker thread queue, and then post some logging info to the write queue later. after the worker pool

Are there any ways to ensure that reads have a low chance of occurring before a write or are the precautions I've already taken sufficient? Should I use the read retry mechanism?

Also, is a write queue actually needed or can I just use worker threads more generally to also handle writes? Write order is not important.

1

Don't mess with time. They're fickle to work with and error prone in a distributed system, especially because you'll need to synchronise clocks and deal with latency. Instead use a version number. For every write request, increment the version number on the data; with every read request, send a version number to conditionally perform the request, if the read request's version number does not match the current version of the resource (or, if your application can tolerate reading from future versions, then loosen the matching criteria so it's a less-than match), then reject the read request (the requesting client can then wait for a few second and resend the read request) or put it back to the read queue and wait until a suitable write request arrives.

For extra reliability, your read queue can be a priority queue. The version number is the priority, if the front of the priority queue (the item with the lowest version number) is higher than the current version of the resource, then the read thread should block until either the write thread wakes it up due to a write event, or its woken up by an enqueue to the read queue that have a lower version number than any other entries in the queue (i.e. it's been inserted to the front of the priority queue).

  • This is the correct answer! Thanks. – wdavies973 Jul 18 at 3:48
2

I do not fully understand what your application does but your concerns and approaches seem off.

  • Do not mess with thread priorities, these will give you nothing.
  • If you do not want reads and writes to get in each others way, just lock on the queue object.
  • a read thread looking in a queue to find nothing is there (yet) is not a bad thing. Trying to control the order in which different threads run is pointless and defeats the purpose of threads.
  • do not do work within the lock, only lock while using the shared resource, that is write to the queue or read from the queue.
  • Put the read thread to sleep for a while before looking for new items again.
  • 1
    Well I'm concerned because the Java ConcurrentHashMap does not actually lock segments during reads as far as I know. The server needs to be threaded to be able to handle multiple requests. The hash map is storing statuses, and for a particular entry X, a write always is submitted a few seconds before a read from X, but the read must not see the past value or an intermediate value. I'm just curious of how to make this system ensure read/write order for X, particularly if the write queue is backlogged. My understanding is that locking a read won't work because it's a race between read/write. – wdavies973 Jul 11 at 21:38
  • 1
    If you do not trust ConcurrentHashMap to lock properly, wrap your reads and writes into your own lock. The first one to obtain the lock will be the only one to access your hash map, the other thread will wait for the lock to be released. – Martin Maat Jul 11 at 22:17
  • @wdavies973 Either your understanding of Java concurrency is much deeper than mine, or you are misunderstanding how ConcurrentHashMap works. Yes, it doesn't lock during reads. No, that doesn't mean that you might retrieve corrupted data. The lack of read locks means that you see old data while a write is in progress. You are attempting to linearize accesses via a queue, but in that case you don't need a ConcurrentHashMap. That's also likely to be slower because it defers reads that could complete immediately. – amon Jul 12 at 14:59
1

First, I'm hoping that the 3-5 seconds or so is a big enough head start for a write to get through before the read

It's really a bad idea to assume something like this. You should just write your code to deal with the fact that a read may show up first.

So if data is stale, you want the read operation to wait until it's updated, is that correct? You might want to use Condition objects. If the data is stale, create/reset a Condition object that the write operation will signal when the data is updated.

Putting threads to sleep can work but has some tradeoffs. For example, how long does it sleep? If your sleep is too short, the thread spins doing nothing but taking up CPU time. If your sleep is too long, your application is less responsive.

Using a lock, semaphore, or Condition object is simpler to code and more responsive (the read operation just blocks until there's something useful to do.)

Also, check out the book Java Concurrency in Practice. Concurrency is really hard to get right and this book discusses a lot of unexpected gotchas and how to do all of this stuff correctly, including how to test it!

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.