Low-Latency Data Sharing Between Threads in C++: Options and Considerations

Question

What are the most efficient and low-latency approaches for sharing data between threads in a C++ system, and why? My primary concern is minimizing latency and maximizing performance, as I have two threads that are each pinned to their own core: a thread that builds an orderbook snapshot and a thread that reads the snapshot object. I am considering using a lock-free queue with a size of just under 3 and either putting the snapshot object into the queue or passing a pointer to it. Another option mentioned to me was using a WAMP router to publish and subscribe to orderbook data and persist it in a LMDB database that is shared via memory mapping of database files. Are there any other options to consider, and what are the trade-offs and benefits of each approach in terms of latency and performance?

for those who dont have an idea regarding what an orderbook snapshot is :

class OrderbookSnapshot
{
public:
    OrderbookSnapshot();
    OrderbookSnapshot(std::array<Order, 10> bids, std::array<Order, 10> asks);

    std::array<Order, 10> get_bids() const;
    std::array<Order, 10> get_asks() const;

private:
    std::array<Order, 10> bids_;
    std::array<Order, 10> asks_;
};

This is what i want to share between the two threads.

Answer 1

My primary concern is minimizing latency and maximizing performance

This can be concern only after you measured performance and if this particular piece of application contributes to the performance essentially. For instance, if it takes 20% of response time, it makes sense to analyze if it can be optimized. But if it takes only 0.001% of response time, you don't need to change it at all.

Answer 2

One thing I can suggest from a lot of experience profiling code (although admittedly not an expert) and working in latency-critical areas a lot is that lock-free data structures don't always perform as well as they sound depending on how they're implemented. They can be lock-free and even wait-free but suffer heavily from false sharing which kills their performance.

It's not always true but sometimes the more intuitive approaches involving locks, like read-write locks (with care to align the write lock data in cache-aligned boundaries separate from the read locks) -- and occasionally even spinlocks (but use them very reluctantly) -- can outperform a lot of conventional lock-free structures. It's easy for enthusiasts seeking lock-free structures to lose sight of the nature of the CPU cache. It's always useful to fiddle with it and profile it and benchmark it and try things. I've worked with some of the greatest computer architecture wizards and still managed to beat them in some cases without nearly as much expertise on instruction selection and assembly code just because I measured and tried things a lot more.

You know, don't marry yourself to an idea and commit yourself wholeheartedly until you've tested and measured things and have satisfactory results. That's useful advice not just in optimizing code but in everything you design and implement.

Immediately though, if you're sharing data this way, you usually want a very low-latency way to notify the consumer thread that new data is available or has changed in some way. Definitely at least start with condition variables. I've seen a lot of naive code in my career using sleeps and yields with a polling approach and they consistently perform quite horribly.