A company I'm going to interview with says they deal with huge amounts of data. How does one prepare for an interview where that fact is telegraphed beforehand as something that one really ought to ponder over?

Should I memorize maximum file sizes for Ext4 file systems or something? Or just be aware of the need for the best possible algorithms...


closed as off-topic by durron597, user40980, user22815, Ixrec, gnat Jun 15 '15 at 23:32

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions seeking career or education advice are off topic here. They are only meaningful to the asker and do not generate lasting value for the broader community. Furthermore, in most cases, any answer is going to be a subjective opinion that may not take into account all the nuances of a (your) particular circumstance." – durron597, Community, Community, Ixrec
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 2
    Doesn't it depend on what they're doing with that data? – user16764 Jul 7 '11 at 22:56
  • 1
    They move it from point A to point B. They do streaming. – Ponk Jul 7 '11 at 22:59
  • 1
    @CodeMonkey You should edit your question to include that additional bit of information. – Adam Lear Jul 7 '11 at 23:02
  • 1
    What job are you being interviewed for? What will be your role? What is the company about? ... very vague question. The only concrete piece of information is "huge amounts of data" and even that ("huge") is relative nowadays. – Rook Jul 7 '11 at 23:09

I worked at a storage company and it sucked immensely. Basically data is stored on large hard drives which are attached to servers. In addition they attempted to have a clever scheme for data integrity/security/recovery etc. Hard drives are dirt-cheap, but they are very likely to die within 5 years. So, how do you keep the data safe?

Google came up with this back in 2003:

http://en.wikipedia.org/wiki/Google_File_System http://labs.google.com/papers/gfs.html

The company that I worked for never got even close. As the result hospitals lost important documents, CIA lost track of terrorists, banks lost customer data - I exaggerate not!

I highly recommend reading the Google File System paper. They did a good job at explaining stuff.

Also, be able to compare different file systems (at a high level): ext3, ext4, reiserFS, NTFS, the one that Apple uses ...

Also try to learn what a company Akamai does and why they are in business - who would pay them and why.

Knowing something about networking, caching, proxy servers, BGP tables http://en.wikipedia.org/wiki/Border_Gateway_Protocol would help.

Look up what the company does, try to figure out who their customers are, what problem they have and why they pay $$$ to your employer. Then try to imagine what low-level details are involved in streaming stuff from point A to point B, including protocols, etc.

Hopefully this will help you 'WOW' them.

EDIT: Do not be cocky at the interview, even if you happen to know some things that they do not, do not show off. You are trying to get a job, right?

I realize that my answer is not complete, albeit somewhat popular.

I would add knowing the difference between TC and UDP, and when you would need each one.


I would take this with a pinch of salt unless you get some concrete figures such as number of Terabytes and number of locations.

In may experience most sites think they have a "huge" amount of data because they are inward looking and have no idea what the rest of the world is doing. Currently I am looking into a system with a billion rows in the database, and I would classify this as a "medium" to "large" system.I have worked on systems where the transaction history alone held millions of rows.

Never trust terms like "huge", "large", "high volume" etc. always ask for concrete numbers. In most cases these will turn out to be rather modest.


We at our company deal with a lot of data (social stuffs), what I have learned is:

  • know how hardware deals with data and how does the OS (kernel), know the bottlenecks/limits.
  • have a good understanding over the data you are dealing with, this helps you to understand and solve problems faster and easier.
  • learn at least basic algorithms and read best practices for dealing with data.
  • don't be afraid it's just data :)

I know this is not the answer you have expected but I hope it would be of some help and good luck.

  • Luckily these are topics that interest me. I don't know whether I have a shot at this job (I'm a nervous intervieweee) but I'm very much into learning about bandwidth issues, cache utilization etc. Thanks. – Ponk Jul 7 '11 at 23:53

It depends on the job.

If you're doing database work, you should know your joins so you don't create a massive Cartesian Join / bog down the system. This is a big problem with some of the developers at my workplace.

Knowing how to string together subqueries and writing efficient code is always a plus. Our team prefers programmers who don't need to test every single snippet on the db2 server. Being confident in your code prior to testing keeps you in good graces on my team.


Safe bets would be network topologies and technologies, as well as application architectures. High-volume filtering and routing techniques might also be good. If their primary business involves serving or storing data from external customers, then security should be involved. Showing some database knowledge is probably a good idea too. If their data is transactional, then you should know basic transaction semantics.

Not the answer you're looking for? Browse other questions tagged or ask your own question.