We have a project that is meant to do signal processing. If all we had to do with the software was that one job, we would have a pretty clean architecture. Unfortunately, we've run into an issue most other projects standard enterprise projects don't ever deal with. We need to generate a qualitative scientific analysis of our algorithms and software in a way that interfaces with domain experts that aren't software engineers. This analysis is used to develop new novel algorithms to improve the signal processing chain further.
The "not being software engineers" isn't the issue here; the tools we create for them also end up helping us developers a lot. We also are not at odds with the importance of these metrics, IE, the entire software development team, agrees that these metrics need to be exposed to better our software. Not exposing these metrics is not an option.
We have to dig deep into the implementation details of our signal processing chain to get the metrics we need for analysis, with many callbacks and configuration parameters that are completely ignored when our application is not run under a development environment. It creates massive coupling issues and complicated code paths. For example, we originally would have a function signature that looks like this:
but with the added metrics, becomes something like:
SignalProcessingChainCommunicator(output_port, signal_processing_configuration..., enable_publish_a, ... enable_publish_z, log_results_a, ... log_results_z)
And resulted in lots of conditionals inside the code and required adding callbacks to internal dependency injection objects. We decided we needed to refactor when we started getting dozens of parameters that obfuscated the real code underneath. We created a "
CallbackConfigurator" object which dealt with enabling the publishing and logging, and assigning callbacks to the DI (we couldn't remove the callback members, we still needed the internal signal processing objects to send out the information we needed) turning the above into this:
SignalProcessingChainCommunicator(output_port, signal_processing_configuration..., callback_configurator)
and turning this:
if enable_publish_a: object_a.callback = publish_a ... if enable_publish_z: object_z.callback = publish_z if enable_log_a: old_callback = object_a.callback def new_callback(obj): old_callback(obj) log_a(obj) object_a.callback = new_callback ... if enable_log_z: old_callback = object_z.callback def new_callback(obj): old_callback(obj) log_z(obj) object_z.callback = new_callback
self.callback_configurator.configure_a(a) ... self.callback_configurator.configure_z(z)
and an additional update function (which passes new values from the
SignalProcessingChainCommunicator to the internal callbacks, i.e., what the current signal processing iteration is)
It cleaned up the noise in our signal processing chain, but it didn't remove the need to configure internal messaging of implementation details. It moved this to a new class.
Is this the only solution to this issue? Is there another way we could solve this problem of ever-growing analysis outputs that depend on implementation details?