Read much more about serialization (e.g. this) and persistence and distributed data store and big data and data science. There are libraries, formats like XDR, tools like SWIG which could help. Consider keeping the data in some centralized database server (PostGresQL, MongoDb, Cassandra ...). Read about the challenge of Distributed Database Systems, ACID and the CAP theorem and Rice's theorem and π calculus. Read about middleware for distributed applications.
If you use any relational database, put efforts on defining well your database schema and database indexes and be aware of database normalization.
If the processes involved are on the same computer, read about MPI, inter-process communication, shared memory. So read much more about operating systems.
On Linux, see also socket(7), fifo(7), shm_overview(7), mq_overview(7)
Textual formats and text based protocols like JSON, HJSON, XML, s-expressions could be helpful (easier to debug) but requires CPU time to be parsed. So read about parsing. Given that CPU are a lot faster than disks (see http://norvig.com/21-days.html ...) there are situations where compressing the data (using lossless compression à la zlib) before storing it on disk is worthwhile.
The devil is in the details. Put efforts on documentation of your software architecture and data format. The data (and its collection) usually costs more than the software processing it.
Your question is really too broad. I tried to give some hints.
PS. You could get a PhD in solving your problem efficiently. Look into ACM conferences (e.g. this one) and ACM papers related to it.