I've recently joined a team working on radar observations. The team has access to a repository of software (written in C++) used for dealing with these observations, but their work is focused towards research, not software development.
There are various variables associated to reading and analysing radar information, and they are all currently stored as local variables (somewhere) in the aforementioned C++ software. It is common for a researcher to want to query some of these variables for their research. Today, I found out that the standard practice here is to check out a new branch of the code, modify it to print out what you want, and then rebuild and run the code locally. This modified code is never checked back in, since it is too localized to be useful to anyone else.
To me, this looks like very bad practice (though I've not been developing software that long, so maybe one of you will correct me). I have always held the opinion that version control (they used a modified form of SVN) should be used primarily for making changes to the code that you intend to check back in, and should never be used for making one-off queries. That is what SQL is for.
Unfortunately, I am not at all sure how easy it will be to implement an SQL front end for this program. At the moment, all the information is stored away in local C++ variables in various modules throughout the program. I have had two ideas for how to fix this:
- Write a function (probably involving some rewriting of the existing code) that writes all of the information that might be useful into a database in some piece of database software, using standard libraries. Then users can run this program over some data, and write SQL queries on the resulting database, using the database software's built-in parser.
The good thing about this approach is that the SQL side of things (as opposed to the database side) is entirely handled by existing software. The downside is that it requires users to build the entire database even if they only want a small query. The team frequently deals with large amounts of data coming in on an hourly basis, and it might be impractical to keep having to build enourmous databases when you're only interested in small queries. The other problem is that the organization as a whole is reluctant to install new software on company computers, and it might be difficult to get appropriate database software installed.
- Use a standard SQL parser library for C++ to query that somehow queries the relevant local variables in the C++ program as if they were stored in a database. So if I passed in the SQL query
SELECT RadarID, Height, Signal Strength, Wavelength FROM RadarObservations WHERE Date = '03/07/2015'
then the program would give me out all the specified fields for radar observations on 03/07/2015, without creating the whole database, but getting the values directly from the local variables in C++ where they are stored.
Of the two, option (2) looks the more attractive, but I doubt that it's even possible. Has anyone any experience with either option, or can anyone suggest anything different? One course of action might be to have a rolling build that builds the databases each day so people can query them, but people here are interested in historical data too, and holding on to all these databases might pose storage issues.