2

There is a db with tables of genes. And these genes would need to be annotated based on the client's needs. So if a gene matches condition x, then add a new column y and value z to that gene. Every cancer is different, so we supplied every client with a template configuration file where they can insert their queries and supply it to our app.

The issues are that:

1) these queries have become so complex that they are now a language of their own and only the developer can be confident in making changes with some back and forth with the clinician. So the whole benefit of providing a config file that the client can edit seems to be lost now.

2) the difference between code and config is becoming blurred because the queries are too complex. For example, we use pandas.query(), but soon we found out that it doesn't handle all our use cases and we now start to patch the query inserted by the client and replacing it on the fly with some inline code which does the job...

I imagine it will get only worse from now on and wonder what other solutions could be there.

4

As developers, we often gravitate to interfaces that are very code-like, but this isn't what a typical user finds intuitive. Think more along the lines of a GUI that guides the user into creating a correct configuration. Yes, these sorts of GUIs are usually among the most difficult to create, but that's what we are going for. We want to offload the difficulty from the users and onto the developers. You can still have the GUI output your config file as a means to store the config, but that shouldn't be the primary means of creating something as complex as you describe.

3

Complexity can be divided into two kinds: essential and accidental. Essential complexity is property of the domain, accidental complexity is something that developers create and can fix.

One way to handle growing essential complexity in configuration is to develop a domain-specific language to describe the configuration, as suggested in another answer.

For non-technical users it's important to provide good tool support, in which case language workbenches help build a full-featured IDE with syntax highlighting, code completion, refactoring, type checking, value tracing/debugging, etc.

Here's a YouTube video about a DSL being developed for a biological domain using the JetBrains MPS language workbench: https://www.youtube.com/watch?v=207HPy7BHeM

Campagne Laboratory, a biology lab, is using JetBrains MPS for its projects (though apparently not for the core domain).

1

One general principle of computer science is that you can add one more indirection layer to make things simpler. Maybe you could add another DSL on top of your current configuration language to simplify it to your clients. In this case, you would have to find patterns in usage and handle them in a similar way.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.