Choosing the design of a scientific DSL: purely or impurely functional?

Question

My aim is to create a language specific to the scientific field (which would be used mainly in the field of machine learning and physics) which would be based on the functional paradigm, a paradigm which seems to me to be perfect for data analysis and processing.

Ideally and in theory, this would imply that there would be no side effects. However, the DSL must nevertheless offer the possibility of processing data in real time, representing and adjusting it on a scalable graph, communicating with external devices or servers, or even more common tasks such as reading a file or writing to the terminal.

Personally, I appreciate the functional paradigm very much, but it becomes relatively complex to use for the purposes mentioned, especially if the language is intended for non-computer specialists whose objective is to be able to manipulate data without worrying about the functioning of the tools they use (they nevertheless have sufficient minimal training).

I therefore have a design question for this DSL concerning its purely functional nature. The question could be resolved fairly quickly by quoting for example a language from the ML family (such as F# or OCaml), which sacrifice a little purity in favour of a more user-friendly interface that is easy to understand and use.

On the other hand, there are languages such as Haskell where this style of problem invokes a certain "monadic complexity" in the management of edge effects.

I'm not looking for the complexity of Haskell for this DSL, but the ML style prevents, or makes complicated, certain tasks that are easily possible in Haskell thanks to the edge effect framing, the purity and the lazy evaluation (and I find that cleaner too, but it's subjective).

Thus, I am curious as to the in-between that would be possible in a DSL with the uses mentioned. So my question is, which design(s) for a functional language that is "as pure as possible" are best suited to meet the style of the problems stated?

Answer 1

I have written my own language for analytics, so my thoughts here are based on my perspective.

Purity was important for my needs because of metaprogramming. In particular, I needed compile-time function evaluation to happen automatically and on arbitrary expressions. (The need for transparent CTFE was based in-turn on some type-system tricks I wanted.)

I looked into several different ways of addressing purity, including monads and uniqueness types. Eventually I started looking into algebraic effects, which became the inspiration for my solution.

If types represent the kind of value, then effects represent the kind of computation. The most commonly used effects systems are checked exceptions, though there has been lots of research on using effects for other domains, including state and IO.

I knew that forcing users to think about effects would be way too much work. So I inverted the paradigm with traits. If effects track something bad that can happen, then traits track something good that can happen. A common trait is that a mathematical operator is commutative, which allows for certain compiler optimizations.

Therefore, I track function purity with traits. The compiler knows that a compound expression is pure if all subexpressions are pure. That simple insight---plus the requirement that base parameters be constant values---allows me to know when a function can be evaluated at compile time! The gory details are in this blog post.

For your needs, I think it might be helpful to track purity on behalf of your user. Mark which builtin functions are pure and then have your language implementation determine traits the same way you determine type.