39

I use both Python (for data analysis ofcourse including numpy and scipy) and R next to each other. However, I use R exclusively to perform data analysis, and Python for more generic programming tasks (e.g. workflow control of a computer model). In terms of basic operations, say operations on arrays and the sort, R and Python + numpy are very comparable. ...


36

Background: I'm a data scientist at a startup in Austin, and I come from grad school (Physics). I use Python day-to-day for data analysis, but use R a bit. I also use C#/.NET and Java (just about daily), I used C++ heavily in grad school. I think the main problem with using Python for numerics (over R) is the size of the user community. Since the language ...


24

There are many differences between these two, but in practical terms, there are three main things to consider: speed, interpretability, and accuracy. Decision Trees Should be faster once trained (although both algorithms can train slowly depending on exact algorithm and the amount/dimensionality of the data). This is because a decision tree inherently "...


15

Sun paths can be predicted, so I imagine you can get the mirror aligned pretty closely already if you know the time of day, the day of the year, and the latitude and longitude. You don't need machine learning for this. If you have mirrors that don't know which way they're pointed (i.e. you can't correlate their position with elevation and azimuth ...


15

To answer just your title, yes. Neural nets can give non-boolean answers. For example, neural nets have been used to predict stock market values, which is a numeric answer and thus more than just yes/no. Neural nets are also used in handwriting recognition, in which the output can be one of a whole range of characters - the whole alphabet, the numbers, and ...


15

(1) What all features should I extract? First, realize that you're not classifying documents. You're classifying (document, query) pairs, so you should extract features that express how well they match. The standard approach in learning to rank is to run the query against various search engine setups (e.g. tf-idf, BM-25, etc.) and then train a model on the ...


15

Joel actually answered this one a few years back. The actual meaning of "teach a machine how to program by itself" is "teach a machine how to take a spec and create a program that corresponds to that spec." And with that in mind: The problem, here, is very fundamental. In order to mechanically prove that a program corresponds to some spec, the spec ...


14

An iterative algorithm is said to converge when, as the iterations proceed, the output gets closer and closer to some specific value. More precisely, no matter how small an error range you choose, if you continue long enough the function will eventually stay within that error range around the final value. In some circumstances, an algorithm will not ...


12

So, I have primarily done data analysis in Matlab, but have done some in Python (and more used Python for general purpose) and also I've started a bit of R. I am going to go against the grain here and suggest you use Python. The reason why is because you are doing data analysis from a Machine Learning perspective, not stats (where R is dominant) or digital ...


11

First of all, it seems to me you are missing (or perhaps misunderstanding) the "explicitly" bit in "without being explicitly programmed" (from the quote in the question). It doesn't mean that no programming is required at all, it means that you are not programming a specific solution to the problem, but instead what you are making is a more general program ...


9

For this sort of application, a field of mirrors trying to point at a solar collector, you can very much calculate where you think the sun should be, where the mirrors should be, what angle they should be at, and how to position them so they point towards your collector. You know, a mathematical model. It'll be close. Probably close enough. As for ...


9

Take a look at n-grams. One n-gram is a sequence of n words. In your case you want n to be 3, since you need two query words and a resulting word. One 3-gram would be for example "I am tired", another one "I am happy". What you then need is a collection of these 3-grams that are collected over your target language, say English. Since you cannot collect it ...


8

Machine learning is a whole field in Computer Science, which is quite different from Artificial Intelligence. Stanford provides a free online class for machine learning which will get you started. You need to understand that machine learning alone will most probably not be sufficient to create good trading strategies; you need to understand well how the ...


8

For each class (breed, gender...) of categorical attributes, you can add a number of components to your feature vector equal to the number of possible values in that class. Then, if a data point has the ith value, you set the ith one of those components to 1, and the rest for that attribute to 0. In your example, for gender, you would add two new components ...


8

Is Machine Learning a part of Data Science? No. Big Data vs Data Science Not the same. Birds and Bird Watching are also not the same. Machine learning is a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on the development of computer programs that ...


7

Maching Learning and Natural Language Processing is somewhat data-driven. Without a continuous supply of high-quality data (which must be re-captured whenever new criteria are added), the software development may miss its intended target. The customer and product owner may devote a bigger fraction of their time toward test data collection. The adaptations ...


7

That course is specifically designed to be accessible to folks without 'much' math background. Of course 'much' is a relative term. In this case it means 'knowledge of calculus is helpful but not required'. The course does use some results from differential calculus, but you can answer the quizzes and complete the programs without knowing calculus yourself. ...


7

You'll need to know Linear Algebra through Eigenvectors if you want things to be "easy". Also a good statistical background with strong emphasis on Regression, Clustering, and Baye's Theorem. Knowing something about gradients doesn't hurt either. As with any CS, graph theory is helpful as well. Obviously the course can be taken with only the most basic ...


7

As an old school (over 50) scientist who has and continues to use a number of these tools I will add my two cents. I have worked with colleagues who still write every piece of code in Fortran, from trivial one-off data analysis jobs to code that dominates some of the worlds supercomputers. Recent Fortran dialects (F90, F95, F2003, F2008) are IMHO, some of ...


7

Usually, such filters are programmed to output not only a yes/no value for each sample, but rather a probability: a sample may, for example, be reported as 95% likely to be abusive. Then you set two thresholds, to divide your results in three groups: very unlikely to be abusive, very likely to be abusive, and uncertain. For example, you might consider ...


7

Work in binary: 0, 1, 10, 11, 100, 101... Know your math: 0+0=00, 0+1=01, 1+0=01, 1+1=10 Know your logic: or, and, not, xor... Find that the low bit is a XOR and the high bit is a AND. Expand the principle of one bit to 8, 16, 32, 64 bits Build it with logic gates. If you want to know more, see my answer to How is fundamental mathematics efficiently ...


6

MATLAB tends not to find its way into production code for several reasons. Among these: It's proprietary It's old and hasn't aged well as a language It was traditionally difficult to to embed in other applications Points 1 and 3 aren't so much true today, since we've got Octave and there are many ways to interface MATLAB with external programs, but these ...


6

Solutions to this problem are often ambiguous, and it's sometimes difficult to decide an optimum solution, even for a human. If you need only a single solution where all words exist in a dictionary, then a naive approach will fail as soon as you encounter a word which prefix is also a valid dictionary entry. For example, if your input string was: ...


6

Fuzzing is a testing method where machine learning can & has been applied. Fuzzing is a method of testing in the realm of automated exploratory testing. It attempts to find defects in software by running a large number of inputs and looking for errors. Unhandled exceptions are the simplest category, but a smart implementation can use ML to find suspect ...


6

You are correct about AI which includes ML which includes DL. NN can indeed be included in ML, be it inside or outside of the DL context. An example for the latter is when neuronal nets are used in simple task based learning (e.g. recognize car number plates in pictures). Data mining is somewhat broader than your definition, because it's not only ...


5

Books are good if you are already knowledgeable about something. But if you are just starting out, a real course is better. For the human brain, is easier to learn something from another person, rather than from an inanimate book. Anyhow it is not always possible to follow a course at your local college, or maybe you want to have the best possible education,...


5

It seems to me that you're comparing apples & oranges. A machine needs 1 day to produce x units, so, this implies that two machines like the first would need 1/2 day. No, it implies 2 machines can produce 2x units in 1 day. We don't have enough information to know if x is divisible, if the machines can cooperate, if there are external constraints (...


4

I don't think you can learn much in 2 months. Image processing is really broad field, and to get better in it you'll need at least several years. Some of the very basics stuff you can do : take a look into 2d filters (or better yet find a book describing 2d image filtering). get octave and try to play with some filters. Try to process images on your own ...


4

Sure, we do this all the time (for extremely limited subsets of problems). It is fairly trivial to imagine taking another step or two and tying something like Siri into the input of these code generators (or something like Wolfram Alpha) which in turn writes code and solves your problem. I would expect that something already exists somewhere to do the most ...


Only top voted, non community-wiki answers of a minimum length are eligible