Return to Answer

Here's how I see it:

• Let's ignore the words "big data" for a while, as they are a pretty vague notion

• You mentioned Hadoop. Hadoop does 2 things: allows you to have a sort of "virtual" drive which is distributed on multiple machines, with redundancy, that can be accessed via Hadoop's API as if it were a single, unitary, drive. It's called HDFS as in Hadoop Distributed File System. The other thing Hadoop does is allow you to execute Map-Reduce jobs (it's a framework for Map-Reduce). If we check out MapReduce's Wikipedia page, we see that:

MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.

...

A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies)

...

'MapReduce' is a framework for processing parallelizable problems across huge datasets using a large number of computers

Also on this page, Hadoop is described as

Hadoop, Apache's free and open source implementation of MapReduce.

Now, Hadoop is written in Java, which is not a functional language. Also, if we look on Hadoop's page, we also find an example of how to create a MapReduce job in Java and deploy it in a Hadoop cluster.

Here's a Java example of a Fibonnaci MapReduce job for Hadoop.

I hope this answer your question, namely that BigData, and in particular a Fibonacci-creating MapReduce job doesn't "need" to be functional, aka you can implement it in OO languages if you want to (for example).

Of course that doesn't mean BigData "needs" to be OO-only either. You can very well use a functional language to implement a MapReduce like job. You can, for example, use Scala with Hadoop if you want to, via Scalding.

Other points I think are worth mentioning.

When doing recursion in Scala, if your code allows for it, Scala will do tail-call-optimization. However, since the JVM doesn't (yet) support tail-call-optimization, Scala achieves this by replacing, at compile time, your recursive calls with code equivalent to loops, as explained here. What this basically means is that doing recursive vs non-recursive code benchmarks using Scala is pointless, as they both end up doing the same thing at run time.

Here's how I see it:

• Let's ignore the words "big data" for a while, as they are a pretty vague notion

• You mentioned Hadoop. Hadoop does 2 things: allows you to have a sort of "virtual" drive which is distributed on multiple machines, with redundancy, that can be accessed via Hadoop's API as if it were a single, unitary, drive. It's called HDFS as in Hadoop Distributed File System. The other thing Hadoop does is allow you to execute Map-Reduce jobs (it's a framework for Map-Reduce). If we check out MapReduce's Wikipedia page, we see that:

MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.

...

A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies)

...

'MapReduce' is a framework for processing parallelizable problems across huge datasets using a large number of computers

Also on this page, Hadoop is described as

Hadoop, Apache's free and open source implementation of MapReduce.

Now, Hadoop is written in Java, which is not a functional language. Also, if we look on Hadoop's page, we also find an example of how to create a MapReduce job in Java and deploy it in a Hadoop cluster.

Here's a Java example of a Fibonnaci MapReduce job for Hadoop.

I hope this answer your question, namely that BigData, and in particular a Fibonacci-creating MapReduce job doesn't "need" to be functional, aka you can implement it in OO languages if you want to (for example).

Of course that doesn't mean BigData "needs" to be OO-only either. You can very well use a functional language to implement a MapReduce like job. You can, for example, use Scala with Hadoop if you want to, via Scalding.

Other points I think are worth mentioning.

When doing recursion in Scala, if your code allows for it, Scala will do tail-call-optimization. However, since the JVM doesn't (yet) support tail-call-optimization, Scala achieves this by replacing, at compile time, your recursive calls with code equivalent to loops, as explained here. What this basically means is that doing recursive vs non-recursive code benchmarks using Scala is pointless, as they both end up doing the same thing at run time.

Here's how I see it:

• Let's ignore the words "big data" for a while, as they are a pretty vague notion

• You mentioned Hadoop. Hadoop does 2 things: allows you to have a sort of "virtual" drive which is distributed on multiple machines, with redundancy, that can be accessed via Hadoop's API as if it were a single, unitary, drive. It's called HDFS as in Hadoop Distributed File System. The other thing Hadoop does is allow you to execute Map-Reduce jobs (it's a framework for Map-Reduce). If we check out MapReduce's Wikipedia page, we see that:

MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.

...

A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies)

...

'MapReduce' is a framework for processing parallelizable problems across huge datasets using a large number of computers

Also on this page, Hadoop is described as

Hadoop, Apache's free and open source implementation of MapReduce.

Now, Hadoop is written in Java, which is not a functional language. Also, if we look on Hadoop's page, we also find an example of how to create a MapReduce job in Java and deploy it in a Hadoop cluster.

Here's a Java example of a Fibonnaci MapReduce job for Hadoop.

I hope this answer your question, namely that BigData, and in particular a Fibonacci-creating MapReduce job doesn't "need" to be functional, aka you can implement it in OO languages if you want to (for example).

Of course that doesn't mean BigData "needs" to be OO-only either. You can very well use a functional language to implement a MapReduce like job. You can, for example, use Scala with Hadoop if you want to, via Scalding.

Other points I think are worth mentioning.

When doing recursion in Scala, if your code allows for it, Scala will do tail-call-optimization. However, since the JVM doesn't (yet) support tail-call-optimization, Scala achieves this by replacing, at compile time, your recursive calls with code equivalent to loops, as explained here. What this basically means is that doing recursive vs non-recursive code benchmarks using Scala is pointless, as they both end up doing the same thing at run time.

Here's how I see it:

• Let's ignore the words "big data" for a while, as they are a pretty vague notion

• You mentioned Hadoop. Hadoop does 2 things: allows you to have a sort of "virtual" drive which is distributed on multiple machines, with redundancy, that can be accessed via Hadoop's API as if it were a single, unitary, drive. It's called HDFS as in Hadoop Distributed File System. The other thing Hadoop does is allow you to execute Map-Reduce jobs (it's a framework for Map-Reduce). If we check out MapReduce's Wikipedia page, we see that:

MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.

...

A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies)

...

'MapReduce' is a framework for processing parallelizable problems across huge datasets using a large number of computers

Also on this page, Hadoop is described as

Hadoop, Apache's free and open source implementation of MapReduce.

Now, Hadoop is written in Java, which is not a functional language. Also, if we look on Hadoop's page, we also find an example of how to create a MapReduce job in Java and deploy it in a Hadoop cluster.

Here's a Java example of a Fibonnaci MapReduce job for Hadoop.

I hope this answer your question, namely that BigData, and in particular a Fibonacci-creating MapReduce job doesn't "need" to be functional, aka you can implement it in OO languages if you want to (for example).

Of course that doesn't mean BigData "needs" to be OO-only either. You can very well use a functional language to implement a MapReduce like job. You can, for example, use Scala with Hadoop if you want to, via Scalding.

Other points I think are worth mentioning.

When doing recursion in Scala, if your code allows for it, Scala will do tail-call-optimization. However, since the JVM doesn't (yet) support tail-call-optimization, Scala achieves this by replacing, at compile time, your recursive calls with code equivalent to loops, as explained here. What this basically means is that doing recursive vs non-recursive code benchmarks using Scala is pointless, as they both end up doing the same thing at run time.

Here's how I see it:

• Let's ignore the words "big data" for a while, as they are a pretty vague notion

• You mentioned Hadoop. Hadoop does 2 things: allows you to have a sort of "virtual" drive which is distributed on multiple machines, with redundancy, that can be accessed via Hadoop's API as if it were a single, unitary, drive. It's called HDFS as in Hadoop Distributed File System. The other thing Hadoop does is allow you to execute Map-Reduce jobs (it's a framework for Map-Reduce). If we check out MapReduce's Wikipedia page, we see that:

MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.

...

A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies)

...

'MapReduce' is a framework for processing parallelizable problems across huge datasets using a large number of computers

Also on this page, Hadoop is described as

Hadoop, Apache's free and open source implementation of MapReduce.

Now, Hadoop is written in Java, which is not a functional language. Also, if we look on Hadoop's page, we also find an example of how to create a MapReduce job in Java and deploy it in a Hadoop cluster.

Here's a Java example of a Fibonnaci MapReduce job for Hadoop.

I hope this answer your question, namely that BigData, and in particular a Fibonacci-creating MapReduce job doesn't "need" to be functional, aka you can implement it in OO languages if you want to (for example).

Of course that doesn't mean BigData "needs" to be OO-only either. You can very well use a functional language to implement a MapReduce like job. You can, for example, use Scala with Hadoop if you want to, via Scalding.

Other points I think are worth mentioning.

When doing recursion in Scala, if your code allows for it, Scala will do tail-call-optimization. However, since the JVM doesn't (yet) support tail-call-optimization, Scala achieves this by replacing, at compile time, your recursive calls with code equivalent to loops, as explained here. What this basically means is that doing recursive vs non-recursive code benchmarks using Scala is pointless, as they both end up doing the same thing at run time.

Here's how I see it:

• Let's ignore the words "big data" for a while, as they are a pretty vague notion

• You mentioned Hadoop. Hadoop does 2 things: allows you to have a sort of "virtual" drive which is distributed on multiple machines, with redundancy, that can be accessed via Hadoop's API as if it were a single, unitary, drive. It's called HDFS as in Hadoop Distributed File System. The other thing Hadoop does is allow you to execute Map-Reduce jobs (it's a framework for Map-Reduce). If we check out MapReduce's Wikipedia page, we see that:

MapReduce is a programming model for processing large data sets with a parallel, distributed algorithm on a cluster.

...

A MapReduce program is composed of a Map() procedure that performs filtering and sorting (such as sorting students by first name into queues, one queue for each name) and a Reduce() procedure that performs a summary operation (such as counting the number of students in each queue, yielding name frequencies)

...

'MapReduce' is a framework for processing parallelizable problems across huge datasets using a large number of computers

Also on this page, Hadoop is described as

Hadoop, Apache's free and open source implementation of MapReduce.

Now, Hadoop is written in Java, which is not a functional language. Also, if we look on Hadoop's page, we also find an example of how to create a MapReduce job in Java and deploy it in a Hadoop cluster.

Here's a Java example of a Fibonnaci MapReduce job for Hadoop.

I hope this answer your question, namely that BigData, and in particular a Fibonacci-creating MapReduce job doesn't "need" to be functional, aka you can implement it in OO languages if you want to (for example).

Of course that doesn't mean BigData "needs" to be OO-only either. You can very well use a functional language to implement a MapReduce like job. You can, for example, use Scala with Hadoop if you want to, via Scalding.

Other points I think are worth mentioning.

When doing recursion in Scala, if your code allows for it, Scala will do tail-call-optimization. However, since the JVM doesn't (yet) support tail-call-optimization, Scala achieves this by replacing, at compile time, your recursive calls with code equivalent to loops, as explained here. What this basically means is that doing recursive vs non-recursive code benchmarks using Scala is pointless, as they both end up doing the same thing at run time.