24

Is there one?

All the definitions I can find describe the size, complexity / variety or velocity of the data.

Wikipedia's definition is the only one I've found with an actual number

Big data sizes are a constantly moving target, as of 2012 ranging from a few dozen terabytes to many petabytes of data in a single data set.

However, this seemingly contradicts the MIKE2.0 definition, referenced in the next paragraph, which indicates that "big" data can be small and that 100,000 sensors on an aircraft creating only 3GB of data could be considered big.

IBM despite saying that:

Big data is more simply than a matter of size.

have emphasised size in their definition.

O'Reilly has stressed "volume, velocity and variety" as well. Though explained well, and in more depth, the definition seems to be a re-hash of the others - or vice-versa of course.

I think that a Computer Weekly article title sums up a number of articles fairly well "What is big data and how can it be used to gain competitive advantage".

But ZDNet wins with the following from 2012:

“Big Data” is a catch phrase that has been bubbling up from the high performance computing niche of the IT market... If one sits through the presentations from ten suppliers of technology, fifteen or so different definitions are likely to come forward. Each definition, of course, tends to support the need for that supplier’s products and services. Imagine that.

Basically "big data" is "big" in some way shape or form.

What is "big"? Is it quantifiable at the current time?

If "big" is unquantifiable is there a definition that does not rely solely on generalities?

Improve this question
10
  • 7
    "What is "big"? Is it quantifiable at the current time?". Sure. Big is more than you can handle at the current time ;)
    – Oded
    Commented Jun 7, 2012 at 19:47
  • 1
    @Oded, you'd need to define "handle" then :-).
    – Ben
    Commented Jun 7, 2012 at 19:48
  • 14
    If you have to ask, yours isn't big enough to count. ;)
    – FrustratedWithFormsDesigner
    Commented Jun 7, 2012 at 19:54
  • @Ben - That's defined differently for each individual and system...
    – Oded
    Commented Jun 7, 2012 at 20:13
  • 4
    "Big" most likely refers to "difficult to handle". Enough to not fit in memory, fill up the disk, take time to transfer over the network, etc.
    – user1249
    Commented Jun 8, 2012 at 21:39

4 Answers 4

Reset to default
43

There isn't one; it's a buzzword.

The delineator though is that your data is beyond the capabilities of traditional systems. The data is too large to store on the largest disk, the queries take tons too long without special optimization, the network or disk can't support the incoming traffic flow, a plain old dataview isn't going to handle visualization for the shape/size/breadth of data...

Basically, that your data is beyond some ill-defined tipping point where "just add more hardware" isn't going to cut it.

Improve this answer
3
  • +1 and furthermore, what counts as "big" is always changing as better hardware catches up and previously customized tools become mature, standardized and sold commercially to deal with such problems.
    – FrustratedWithFormsDesigner
    Commented Jun 7, 2012 at 21:11
  • In other words: no, no idea, no, no :-).
    – Ben
    Commented Jun 8, 2012 at 11:02
  • In addition, before big data became a big thing, many companies and research institutes already did big data things. Only now with all the social media / online big data challenges has it become more main stream.
    – Paul Hiemstra
    Commented Jun 28, 2013 at 8:33
2

As it is pointed in Oracle link (comment by Immad Careem) oracle.com/us/technologies/big-data/index.html. The Big Data is everything which is not a relational data stored in a RDBMS. Few years before the hype it was just "a lot of data". Now it did grow and was promoted by marketers to be some kind of special data.

There are several secondary reasons (other than marketing) to consider Big Data a real thing.

  1. Invention of Map-Reduce
  2. NOSQL technologies like Hadoop
  3. Some evolution in traditional RDBMS influenced by demand of unstructured data types
  4. Possibly some hardware technologies offered by EMC2 corporation
Improve this answer
2
  • 2
    "Invention of Map-Reduce"? You have to be kidding.
    – Telastyn
    Commented Jun 9, 2012 at 20:27
  • 1
    "Everything which is not relational data" is a definition that could only come from someone as RDB-centered as Oracle (and it's wrong). Under that definition, every SolR index, every MongoDB database and every Berkley DB is "big data". And that's just stupid.
    – Joachim Sauer
    Commented May 29, 2013 at 10:55
0

Using Doug Laney's answer as a starting point, we reverse-engineered a list of Big Data definitions, now over 30 and going strong. Our list of definitions for "Big Data" is located here.

We welcome corrections, entries, graphics, etc.

Improve this answer
-1

Great to see O'Reilly and others finally latch on to Gartner's 3Vs of big data that we first introduced over 11 years ago. For reference, here's the original piece I wrote in 2001: http://blogs.gartner.com/doug-laney/deja-vvvue-others-claiming-gartners-volume-velocity-variety-construct-for-big-data/.

Gartner's recently updated definition also recognizes the value aspect: "Big Data are information assets with volumes, velocities and/or variety requiring innovative forms of information processing for enhanced insight discovery, decision-making and process automation."

We have also developed a method to quantify data magnitude along the three vectors that are prescriptive in terms of technology adoption. However I cannot share it publicly.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.