If functional programming is expensive and memory-consuming for small data, why do we need it for Big Data ?

Specifically I can already see a few applications where this is extremely useful. ex. Statistics, ie calculating a Gaussian function on the fly with different parameters or a set of parameters for data analytics. There is also interpolation for numerical analysis, etc.

What are the best practices to use functional programming (Scala) for Big Data ?

To answer on efficiency there are also techniques to help increase your efficiency in space or time, specifically recursion, tail recursion and continuation passing style, currying, etc. Some languages have their pros and cons (example lazy vs eager.) For something simple like the Fibonnacci sequence I might just use the imperative way (I still prefer functional for fibonnacci since it is easier to read) in something based off of a C language but for other purposes I might use some of the techniques and functional programming for writing quick, clean, "easy to read" (although I find this subjective) code.

If functional programming is expensive and memory-consuming for small data, why do we need it for Big Data ?

Specifically I can already see a few applications where this is extremely useful. ex. Statistics, ie calculating a Gaussian function on the fly with different parameters or a set of parameters for data analytics. There is also interpolation for numerical analysis, etc.

What are the best practices to use functional programming (Scala) for Big Data ?

To answer on efficiency there are also techniques to help increase your efficiency in space or time, specifically recursion, tail recursion, continuation passing style, higher-order functions, etc. Some languages have their pros and cons (example lazy vs eager.) For something simple like the Fibonnacci sequence I might just use the imperative way as I find at times some of my co-workers are reluctant and may not be as comfortable with functional programming and hence take up more development time... (I still prefer to use functional programming when I can[applications that I am in charge of]) since I find it quick, clean and "easy to read" (although I find this subjective) code.

Wikipedia has a "fast" version of the fibonnacci sequence posted. https://en.wikipedia.org/wiki/Functional_programming#Scala

def fibTailRec(n: Int): Int = {
  @tailrec def f(a: Int, b: Int, c: Int): Int = if (a == 0) 0 else if(a < 2) c else f(a-1, c, b + c)
  f(n, 0, 1)
}

Using streams/hof

val fibStream:Stream[Int] = 0 #:: 1 #:: (fibStream zip fibStream.tail).map{ t => t._1 + t._2 }