6

I need to write a server application that fetches mails from different mail servers/mailboxes and then needs to process/analyze these mails. Traditionally, I would do this multi-threaded, launching a thread for fetching mails (or maybe one per mailbox) and then process the mails.

We are moving more and more to servers where we have 8+ cores, so I would like to make use of these cores as much as possible (and not use 1 at 100% and leave the seven others untouched). So conceptually, as an example, it would be nice that I could write the application in such a way that two cores are "continuously" fetching emails and four cores are "continuously" processing/analyzing the emails (since processing and analyzing mails is more CPU intensive than fetching mails).

This seems like a good concept, but after studying some parallel patterns, I'm not really sure how this is best implemented. None of the patterns really fit. I'm working in VS2012, native C++, but I guess from a design point of view this does not really matter and just some pointers on how to organize this would be great!

Improve this question
2
  • 3
    I'd just like to make the point that a single process NodeJS instance on a single core often beats threaded servers not doing IO correctly. Most of the time in these applications tend to be wasted waiting for I/O. How much of your use case is the actual processing/analyzing and how time is spent waiting for emails? If you're not sure I think that profiling that specific aspect will really help you in making a good design decision.
    – Benjamin Gruenbaum
    Commented Jul 11, 2013 at 22:38
  • @Benjamin: that was indeed also my worry. Therefore I started first with writing an email generator script and different models for fetching the mails, to see whether there is actually a point in going multiprocessor. Thx for the input
    – Wim Van Houts
    Commented Jul 13, 2013 at 10:11

1 Answer 1

Reset to default
3

The actor model of concurrency seems like it might be a good fit for this.

The Model

In case you're not familiar with this model, it goes as follows:

Actors are threads that run in a loop. Each actor has a producer-consumer message queue; external code and other actors communicate with an actor by sending a message to it (queuing it in its message queue).

An actor's thread will block waiting for a message in its message queue, and when one appears the actor will deal with it, then loop back to process or wait for the next message. Repeat.

Note: "Actors" are sometimes called "Agents", but that term is misapplied. See the comment thread below for more.

Architecture

You could create actors specifically for downloading messages (say one per mailserver/mailbox) and other actors for processing the e-mails once they've been downloaded.

Connecting the two you could have a single routing actor that would receive references to the downloaded mail files from the fetching actors and either send each reference to an available processing actor or spin up another one to process it if all the other processing actors were busy. When a processing actor was finished processing, it would send a message to the routing actor saying it was done so the routing actor would know it could send another message to it to process.

I'm betting by this point there's a library for actors for C++ [UPDATE: See comment by @rwong below]. If all else fails you could try Erlang ;)

I'm not sure how the C++ threading libraries work -- whether they map threads to a single core or multiple cores -- but if this doesn't do it for you you could take the same concept and instead of using threads have them be discrete processes and use some sort of message passing framework for the communication.

Edit: I'm betting you'll have a bottleneck at the network, though, so it might not even make sense to want to occupy all the cores at once (unless the processing takes a loooong time).

Edit: Expanded answer and corrected terminology (Agent -> Actor)

Improve this answer
11
  • Do you mean by bottleneck at the network that a gigabit connection would not be fast enough to download sufficient emails to keep a core busy? Processing would take reasonable time (several regex to execute per mail, on mails having possibly large body).
    – Wim Van Houts
    Commented Jul 11, 2013 at 20:54
  • Yes, that was what I meant about the bottleneck. My thought was that if you had umpteen e-mails all trying to come in through a connection at once (with multiple download agents running) that it'd slow things down and that the processor agents wouldn't have enough to do. That's just my gut instinct though, I don't have any hard evidence :)
    – paul
    Commented Jul 11, 2013 at 20:58
  • @paul Agents are autonomous beings in their computational environment showing goal driven behavior. This answer suggests no goal driven behavior, no deliberation and no autonomy . While these might be considered purely reactive agents (much like a light switch could) I don't think it's the right term to use in this case. I think producer/consumer is more what you had in mind. Multiagent systems excel at huge scaling problems where coordination is needed and conflict resolution and common knowledge are issues - an 8 core server is not such a system.
    – Benjamin Gruenbaum
    Commented Jul 11, 2013 at 22:35
  • 1
    Microsoft Parallel Patterns Library (PPL) provides an implementation of asynchronous agents also known as actor mode. This online book by Campbell and Miller provides a step-by-step guide to write high-performance concurrent software.
    – rwong
    Commented Jul 12, 2013 at 2:59
  • 2
    @BenjaminGruenbaum "Actor model" is the correct term. However, for historical reasons there are multiple frameworks that used the term "Agents" for what is actor model, therefore when we discuss practical programming issues the reader has to understand that these terms were used interchangeably. (Sort of a "do as I say, not as I do" situation.)
    – rwong
    Commented Jul 12, 2013 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.