What are the most efficient and low-latency approaches for sharing data between threads in a C++ system, and why? My primary concern is minimizing latency and maximizing performance, as I have two threads that are each pinned to their own core: a thread that builds an orderbook snapshot and a thread that reads the snapshot object. I am considering using a lock-free queue with a size of just under 3 and either putting the snapshot object into the queue or passing a pointer to it. Another option mentioned to me was using a WAMP router to publish and subscribe to orderbook data and persist it in a LMDB database that is shared via memory mapping of database files. Are there any other options to consider, and what are the trade-offs and benefits of each approach in terms of latency and performance?

for those who dont have an idea regarding what an orderbook snapshot is :

class OrderbookSnapshot
    OrderbookSnapshot(std::array<Order, 10> bids, std::array<Order, 10> asks);

    std::array<Order, 10> get_bids() const;
    std::array<Order, 10> get_asks() const;

    std::array<Order, 10> bids_;
    std::array<Order, 10> asks_;

This is what i want to share between the two threads.

  • 3
    If you only have two threads you're overthinking all of this. Go for the simplest first (lock-full FIFO queue of objects, (not pointers) probably), then measure to find the hotspots (if any). Don't forget memory layout (false sharing). Don't forget to also benchmark without pinning threads.
    – Mat
    Dec 27, 2022 at 9:16
  • 1
    What are your actual latency requirements? In many scenarios, it might be sufficient to just sleep repeatedly for a couple of ms until the next item is available. Or you could use the features of a threading API to wake the consumer thread once the next item is available. Lock-free data structures are great, but don't solve the problem of notifying consumers that items are available. Busy wait loops / spinlocks that check whether new values have become available (without sleeping) have their uses, but might be inefficient if there are rarely new values.
    – amon
    Dec 27, 2022 at 16:49

2 Answers 2


My primary concern is minimizing latency and maximizing performance

This can be concern only after you measured performance and if this particular piece of application contributes to the performance essentially. For instance, if it takes 20% of response time, it makes sense to analyze if it can be optimized. But if it takes only 0.001% of response time, you don't need to change it at all.


One thing I can suggest from a lot of experience profiling code (although admittedly not an expert) and working in latency-critical areas a lot is that lock-free data structures don't always perform as well as they sound depending on how they're implemented. They can be lock-free and even wait-free but suffer heavily from false sharing which kills their performance.

It's not always true but sometimes the more intuitive approaches involving locks, like read-write locks (with care to align the write lock data in cache-aligned boundaries separate from the read locks) -- and occasionally even spinlocks (but use them very reluctantly) -- can outperform a lot of conventional lock-free structures. It's easy for enthusiasts seeking lock-free structures to lose sight of the nature of the CPU cache. It's always useful to fiddle with it and profile it and benchmark it and try things. I've worked with some of the greatest computer architecture wizards and still managed to beat them in some cases without nearly as much expertise on instruction selection and assembly code just because I measured and tried things a lot more.

You know, don't marry yourself to an idea and commit yourself wholeheartedly until you've tested and measured things and have satisfactory results. That's useful advice not just in optimizing code but in everything you design and implement.

Immediately though, if you're sharing data this way, you usually want a very low-latency way to notify the consumer thread that new data is available or has changed in some way. Definitely at least start with condition variables. I've seen a lot of naive code in my career using sleeps and yields with a polling approach and they consistently perform quite horribly.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.