1

I have this conondrum I have to deal with. We have this already pretty big application which main purpose can be classified into 2 main functions:

  • Core data (deals with a pretty complex data relationships and data analysis).
  • Data presentation/visualization (this is the piece that keeps growing, the reasons to come next.)

This information presented to the user is done in all sorts of unpredictable ways. There is really no way to automate or predict what the users will come up with next time they request a new data visualization. Currently, specialized code is build as part of the main solution for every visualization requested. This has been going for years, and we have come up with ways to standardize and classify what is common and repetitive, however the code keeps growing and growing increasing complexity and complicating automated testing (600 visualization so far). One important characteristic about this visualizations is that they are only useful for a while, after they are used they become disposable. We can’t really dispose of them in about a month after they are used. But in the meantime we want to keep them around for support purposes and as reference for future pieces.

The question is there a way to maintain this?

Someone recommended using docker and containerizing all individual data visualizations (Views and all directly coupled code), but that feels like an overkill to me and doesn't really solve the ever-growing code problem.

Project is a .Net core MVC.

Any ideas, or previous experience that have worked?

Thanks!

  • 1
    "We can’t really dispose of them in about a month after they are used" Why not? – Philip Kendall Oct 31 '17 at 6:47
  • 1
    WHen you say 600 VIzualizations ? How is it transcribe into your architecure ? 600 mini specific applications with their own .NET project module ? 1 Global applications that cointains everything ? Other ? – Walfrat Oct 31 '17 at 10:32
  • 2
    Kinda of a long shot, but can't you use some embeded scripting language like lua with access to some api from your application to create the presentation part. I know games does something like this for UI. – Mandrill Oct 31 '17 at 21:15
  • 1
    Could you please describe whether these visualizations are interactive or static (the latter meaning that results are exported as graphs or images)? Interactions with visualizations may be the biggest contributor to code complexity. – rwong Nov 7 '17 at 21:25
  • 1
    @rwong: Visualizations are dynamic both in the data that is rendered and the interaction the user can have with them (Sorting, Links to different reports and general static content), so they are not static reports. – Chepech Nov 14 '17 at 19:05
2

Without knowledge about your domain, it would be hard to tell. However, I happen to be working on something similar.

It sounds like your real problem is choice of infrastructure.

Due to their nature, visualizations are limited since there is a predefined set of UI elements you can display. Hence, your visualizations can be modeled as a composition of elements.

So, instead of coding visualizations- store data about how they are composed in a document database. Then, simply render it on demand. I Strongly reccommend React which is naturally outfitted for this task.

1

Currently, specialized code is build as part of the main solution for every visualization requested

This is the root of your problem. Writing one-off reports/visualizations is usually the starting place for organizations. You've obviously outgrown this stage with 600 custom reports.

I recommend you invest in a visualization tool. There are many out there. I've used JasperReports, and while the experience with it was not without problems, it was still a far cry better than coding custom visualizations.

There is really no way to automate or predict what the users will come up with next time they request a new data visualization

Many of these tools come with self-serve solutions for users to build their own custom visualizations using SQL queries or pre-packaged data. For any special needs that your customers have you can either take a hardline approach and only allow them to use what's available, or you add features to your tool that will be available for any other user that will need it in the future. You need a systematic and repeatable way to create reports without involving coding and testing. Once in place it should pay for itself.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.