Linking two or more different programming languages

Question

I would like to ask how optimizer software written in C/C++ can be linked to simulator software written in FORTRAN or any other language. I have copied the section of journal article of Singh and Chakrabarty (2011).

In the present study, MODFLOW 2000 and MT3DMS 5.0 are linked as independent modules with the optimization code. Modifications are made to the codes of the simulators in such a way that they can be run through input files, and both of them are then compiled. One of the files required to run MODFLOW 2000 is prepared by the optimization algorithm. This file contains the latest values of the decision variables. The NSGA II separately calls both MODFLOW 2000 and MT3DMS 5.0 in their compiled forms as independent modules. MODFLOW 2000 uses the file prepared by NSGA II to calculate hydraulic head and prepare an output file containing these values. Similarly, MT3DMS 5.0 calculates contaminant concentrations at selected locations and produces an output file containing the concentration values. The output files produced by the simulators are used by NSGA II as input files to produce updated values of the decision variables. The optimization problem, thus, runs until convergence. Obviously, writing of interface programs for linking codes written in different languages is altogether eliminated in this technique of linking.

I believe from here I can move forward with my research. Thanks to you all.

Answer 1

Linking code (using the linker, e.g. ld often started by gcc or gfortran compilation commands) written in two different languages is implementation specific. It is often called foreign function interface. It depends upon calling conventions specific to the implementations.

On Linux, with code compiled by gfortran and gcc (both from the GCC compiler) is quite easy (in particular because both share a common middle-end and back-end layers of the Gnu Compiler Collection). The gfortran documentation has a chapter about mixed language programming

BTW, you can make two processes (e.g. running two different -or the same- programs, perhaps coded in two different languages) communicate using Inter-process communication; this is provided by the operating system so is OS specific. On Linux and POSIX you could use pipe(7)-s, socket(7)-s, in particular unix(7) ones (and poll(2) them appropriately in some event loop), shared memory (see shm_overview(7) & sem_overview(7) for synchronization with semaphores), etc. Read Advanced Linux Programming. You might be interested by Remote Procedure Calls (e.g. JSON-RPC, ....), MPI, etc...

PS. Don't call IPC linking. When talking about software, linking means using the linker (e.g. to merge & mix code from various object files and libraries into a single executable).

Answer 2

I think all they're saying here is that there are two components (one containing MODFLOW 2000 and MT3DMS 5.0, and the other containing NGSA II, with the two components being written in different languages) which simply use files as their interprocess communication (IPC) mechanism - hence the two components are logically linked while not being linked as in the "linker" step of a traditional compiler-linker chain.

However, this is not an innovative technique in any way, so either I've misinterpreted things or the authors are spinning things a little bit too much in the quote you've emphasised.

Answer 3

It sounds like you are saying each of the simulator programs is a separate executable, and you run it by 1) writing a file for it to input, 2) spawning a process to run it and waiting for it to complete, and 3) reading its result file. As they say, in this way the various modules can be written in any language, it doesn't matter what languages.

The problem with this approach is performance. If it takes longer to execute than you want to wait, chances are you will find, if you use this technique, that a quite large fraction of time is spent in each of the three steps - writing, spawning, and reading. If so, that implies the execution time could be reduced by a big factor.

Compilers like gcc and gfortran play together nicely, as @Basile points out. If the various tools can be compiled and linked together into a single executable, and communicate directly through in-memory data passing, eliminating the need for file formatting and parsing, and eliminating the need for process startup, you may find it a big speedup. Possibly one or two orders of magnitude.