I know that operating systems facilitate inter-process and inter-thread data sharing. I want to know about the mechanisms used to facilitate such sharing. I read about "pipes". What are the other ways?
-
4This question borders on the unanswerable unless you give the specifics of your data and your desired end-result. Is this simple text data, or multidimensional structures? Does it occur in a stream, or in fits-and-starts? Regarding dataflow: is the producer process pushing or is the consumer process pulling? Etc., etc.– Peter RowellNov 7, 2012 at 15:14
-
There are MANY ways to do this, so this question can't be reasonably answered. Though you didn't know that so that's ok, change your question to ask for the term used to refer to sharing data between processes. Given the correct term for the technique you refer to, you can then start your research on the topic far more effectively. (Hint: there is a well known term.)– Jimmy HoffaNov 7, 2012 at 15:38
-
See: en.wikipedia.org/wiki/Inter-process_communication– FrustratedWithFormsDesignerNov 7, 2012 at 16:42
-
Let me save you a lot of pain. Just use 0MQ (zeromq.org).– Evan PlaiceNov 8, 2012 at 3:04
Add a comment
|
3 Answers
Threads operate in the common address space, so in-memory data structures can be shared without any restrictions (assuming, of course, that proper synchronization has been put in place). One can pass an address of an in memory data structure to another thread in order for the thread to be able to access that data structure.
Processes, on the other hand, operate in separate memory spaces, so they use one of several approaches:
- Serialized data exchange - this can be implemented on top of pipes or other in-memory exchange data protocols (e.g. connecting via TCP to a server running on
localhost). The data needs to be serialized and deserialized, making this approach less practical as the volume of data goes up. Note that this is not precisely sharing, because all the data is copied both ways. - Shared Memory - this lets you exchange large amounts of data without serialization. This is a true data sharing, because no copying is done. However, ensuring proper synchronization becomes a responsibility of your application. Memory-mapped files is another variation of shared memory, retaining all the advantages and the disadvantages of the approach.
answered Nov 7, 2012 at 15:12
-
These are both good solutions, I just wanted to throw out a caveat that neither of them scale very well. However, both in-process data sharing and shared memory are much faster than most "scalable" alternatives (message queues or databases).– TMNNov 7, 2012 at 16:32
Message Queue is another option. A process can write to the tail of a queue and another process can read from the head of that queue. Message queue's advantage is that it supports asynchronous operation.
Depending on the scope and type of an application, if what you need are command line applications you could use pipes by writing and reading from console, if you are writing an intranet application or standalone application and processes accessing data are parallel or on different machines you can use a database server.
If what you need is not complex you can use files or database systems that are integral part of the application like SQL lite, but here you would need to implement flagging so you don't corrupt data if both process try to write data.
answered Nov 7, 2012 at 15:18
