2

I would like to implement interprocess communication between an Ubuntu Linux 15.10 mono 4.1.2 C# recorder client and Ubuntu Linux 15.10 mono 4.1.2 C# video server using a C++ mutex class and C++ event class which harnesses pthreads, shm_open and mmap.

The recorder client and video server reside on the same Ubuntu Linux machine. In addition, a Windows 7 C# client running on a separate machine communicates with an Ubuntu Linux 15.10 C# client-server program using TCP/IP sockets.

In Advanced Programming in the UNIX Environment Second Edition by W. Richard Stevens and Stephen A. Rago, I read page 489 which states

A memory-mapped region is inherited by a child across a fork (since it's part of the parent's address space) , but for the same reason, is not inherited by the new program across an exec

I need to know if I have to fork the video server program as an "C" child process in order for interprocess communication to take place by casting the mmap return value to a pthread_mutex_t pointer . I want to share pthread mutex and pthread condition variable between the video server process and recorder client process.

Have I confused threads with processes?

Boost wrote an interesting article about this topic which I have extracted an excerpt from shown below,

http://www.boost.org/doc/libs/1_37_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.mapped_region_object_limitations

Limitations When Constructing Objects In Mapped Regions Offset pointers instead of raw pointers

When two processes create a mapped region of the same mappable object, two processes can communicate writing and reading that memory. A process could construct a C++ object in that memory so that the second process can use it. However, a mapped region shared by multiple processes, can't hold any C++ object, because not every class is ready to be a process-shared object, specially, if the mapped region is mapped in different address in each process.

When placing objects in a mapped region and mapping that region in different address in every process, raw pointers are a problem since they are only valid for the process that placed them there. To solve this, Boost.Interprocess offers a special smart pointer that can be used instead of a raw pointer. So user classes containing raw pointers (or Boost smart pointers, that internally own a raw pointer) can't be safely placed in a process shared mapped region. These pointers must be replaced with offset pointers, and these pointers must point only to objects placed in the same mapped region if you want to use these shared objects from different processes.

Of course, a pointer placed in a mapped region shared between processes should only point to an object of that mapped region. Otherwise, the pointer would point to an address that it's only valid one process and other processes may crash when accessing to that address.

Basile Starynkevitch wrote on April 21 2016, "The point is won't use pthread mutex and condition variable like you dream of. Be creative too..." , in response to my question about how to emulate a Windows event in Linux.
So, I found the pdf, Implementing Condition Variables with Semaphores , http://research.microsoft.com/pubs/64242/implementingcvs.pdf, written by Microsoft researcher , Andrew D. Birrell . Below . I show an excerpt from this article ,

class Lock {
 Semaphore sm;
public Lock() { // constructor
 sm = new Semaphore(); sm.count =1; sm.limit = 1;
 }
public void Acquire() { sm.P(); }
public void Release() { sm.V(); }
}
You can come quite close to implementing a condition variable in a similar way:
class CV {
 Semaphore s;
 Lock m;
public CV(Lock m) { // Constructor
 this.m = m;
 s = new Semaphore(); s.count = 0; s.limit = 1;
 }
public void Wait() { // Pre-condition: this thread holds “m”
 m.Release();
 s.P();
 m.Acquire();
 }
public void Signal() {
 s.V();
 }
}

Could I harness Andrew D. Birrell's research to emulate a Windows event with a class like this:

class Event {
    Lock theLock;
    CV   theCV;
    bool triggered;

    Event* MakeEvent(string Name);
    Event* OpenEvent(string Name);
    void   CloseEvent(string Name, Event* anEvent);
    void   NukeEvent(string Name, Event* anEvent);
    void   SetEvent(Event* anEvent);
    void   ResetEvent(Event* anEvent);
    int    WaitForSingleObject(Event* anEvent, int millisecond);
}

?

16
  • 1
    Have you read computing.llnl.gov/tutorials/pthreads ? Commented Apr 15, 2016 at 10:49
  • There is a very nice article titled Sharing Mutex and Condition Variable Between Processes on April 10 2014 in this URLhttp://www.gonwan.com/page/3/. which forks a child process.
    – Frank
    Commented Apr 15, 2016 at 10:49
  • @Basile Starynkevitch, Thank you for the URL. I am reading it right now. How might I apply it to solving my problem?
    – Frank
    Commented Apr 15, 2016 at 10:52
  • I don't exactly know, because I don't have access to all your source code. But that URL is a possible answer to your "Have I confused threads with processes?" question. Commented Apr 15, 2016 at 10:54
  • @Basile Starynkevitch, May I ask what source code I should add to the original question?
    – Frank
    Commented Apr 15, 2016 at 11:01

3 Answers 3

1

Don't use Pthread mutexes to synchronize between processes, at least on Linux. (I am not sure that Linux is implementing pthread_mutexattr_setpshared correctly and efficiently, at least not in GNU glibc 2.21).

Use POSIX semaphores, see sem_overview(7). Or consider the Linux specific eventfd(2) probably with poll(2) & read(2) & write(2). Both (semaphores & eventfd-s) are alternative synchronization mechanisms (to mutex between processes).

You might also study the source code of your C# implementation (both Mono and Microsoft CLR are today free software). And you might also use strace(1) or ltrace(1) to understand what your C# program is doing.

The point is that you won't use pthread mutex & condvar like you dream of. You need to be creative. If using eventfd, you might have some additional thread polling it, and then broadcast a condvar ... If using semaphores, be creative too...

7
  • Where could I find an example of using POSIX semaphores and eventfd to signal pthread condition variables? Thank you
    – Frank
    Commented Apr 21, 2016 at 6:17
  • You won't signal condvar with semaphores or eventfd-s. Both are different synchronization mechanisms. Commented Apr 21, 2016 at 7:29
  • If I cannot signal pthread condition variables with semaphores or eventfd-s, how can I signal pthread condition variables for interprocess communication purposes? Thank you.
    – Frank
    Commented Apr 21, 2016 at 9:15
  • You need to give up using condition variables the way you dream. Commented Apr 21, 2016 at 9:21
  • @ Basile Starynkevitch, Thank you for your response today "about giving up using condition variables the way I dream of." Could I ask you to briefly review my question I just inserted at the end of the original question about emulating Windows events in Linux?
    – Frank
    Commented Apr 21, 2016 at 12:59
2

pthread_mutex_t is not designed to work cross process like that. It was built with the assumption that all threads using it will have the same address space.

There are however other options to share a mutex between processes. As discussed in the comments there is futex(7) (which will work over shared memory) and sem_overview(7) (an explicit named semaphore shared across processes) both of these will work cross process.

8
  • I just read your excellent answer. I ran the Linux command , ps -ef , and it showed different process ids for the video server and the recorder.
    – Frank
    Commented Apr 15, 2016 at 13:19
  • How is a Linux explicit named semaphore shared across processes? I realize a named semaphore is different than a pthreads mutex. Thank you.
    – Frank
    Commented Apr 15, 2016 at 13:43
  • @Frank when you do "sem_open" the OS will look for other semaphores with the same name and return a handle to an existing semaphore if there is already one (creating one otherwise if the flag was set). Waiting and signaling semaphores are system calls and the kernel will deal with the exclusive access. Commented Apr 15, 2016 at 14:00
  • I have a question about Figure 11.10, Using a mutex to protect a data structure on page 372 of "Advanced Programming in the UNIX Environment Second Edition" by W. Richard Stevens and Stephen A. Rago. Figure 11.10 uses malloc(sizeof(struct foo)) to allocate the reference counted object. May one use this example to share a mutex between processes?
    – Frank
    Commented Apr 15, 2016 at 14:14
  • A pthread_mutex_t is for locking inside threads of the same process. Use sem_overview(7) for inter-process synchronization. So don't share any pthread_mutex_t between different processes. Commented Apr 17, 2016 at 6:43
1

If I were you, I'd forget about using either pthreads (which only works on threads and not processes, and you're clearly using multiple processes) or shared memory of the kind you're talking about, as transferring the handles between processes is going to be a bit of a pain point when working in C#.

A simpler technique, which is becoming increasing frequent in Linux software, is to create a file in a transient storage area (e.g. /run/shm in Linux distributions using the Filesystem Heirarchy Standard 3.0 or higher, or /var/shm in older systems - I don't use Ubuntu but I can see that the version you're using is recent, and as it's based on Debian whose current version is based on FHS 3.0 I suspect you should be using /run), and then open that in both processes, and use mmap (or Mono.Unix.Native.Syscall.mmap in your case) to map the file into both processes' memory space. Once you've done this, you can share data simply by reading & writing into the mapped area.

For mutual exclusion, a similar process can be performed by creating an empty file (typically in /run/lock or /var/lock) and then using lockf (also both in Mono.Unix.Native.Syscall). Alternatively, you can use a System.Threading.Semaphore constructed with a system-wide name.

9
  • Thank you for your excellent answer. As shown below, public static int lockf (int fd, LockfCommand cmd, long len) takes an integer file descriptor rather than a file pointer. How do I handle that? Could I convert a linux file pointer to an linux file descriptor?
    – Frank
    Commented Apr 18, 2016 at 12:32
  • Do we have to mmap the empty file for mutual exclusion? Thank you.
    – Frank
    Commented Apr 18, 2016 at 15:16
  • The simplest way is to use the open method in the same class to open a file and return its file descriptor. And you don't need to map the lock file, on the data file.
    – Jules
    Commented Apr 18, 2016 at 15:36
  • What is the purpose of creating a file in transient storage area of and then opening that in both processes, and use mmap to map the file into both processes' memory space. as well as sharing the data by reading & writing into the mapped area? We understand the purpose of your mutual exclusion. Thank you.
    – Frank
    Commented Apr 18, 2016 at 19:05
  • The transient file exists to create a globally-accessible name that any process can refer to. It also defines the size of the memory block that can be shared between the processes. By using mmap on the same file, any processes can gain access to the same shared data. Using the /run filesystem prevents the overhead of the data ever being flushed to disk, as it is (typically) a memory-only filesystem ("tmpfs").
    – Jules
    Commented Apr 18, 2016 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.