39

I use both Python (for data analysis ofcourse including numpy and scipy) and R next to each other. However, I use R exclusively to perform data analysis, and Python for more generic programming tasks (e.g. workflow control of a computer model). In terms of basic operations, say operations on arrays and the sort, R and Python + numpy are very comparable. ...


36

Background: I'm a data scientist at a startup in Austin, and I come from grad school (Physics). I use Python day-to-day for data analysis, but use R a bit. I also use C#/.NET and Java (just about daily), I used C++ heavily in grad school. I think the main problem with using Python for numerics (over R) is the size of the user community. Since the language ...


15

To answer just your title, yes. Neural nets can give non-boolean answers. For example, neural nets have been used to predict stock market values, which is a numeric answer and thus more than just yes/no. Neural nets are also used in handwriting recognition, in which the output can be one of a whole range of characters - the whole alphabet, the numbers, and ...


15

(1) What all features should I extract? First, realize that you're not classifying documents. You're classifying (document, query) pairs, so you should extract features that express how well they match. The standard approach in learning to rank is to run the query against various search engine setups (e.g. tf-idf, BM-25, etc.) and then train a model on the ...


15

Sun paths can be predicted, so I imagine you can get the mirror aligned pretty closely already if you know the time of day, the day of the year, and the latitude and longitude. You don't need machine learning for this. If you have mirrors that don't know which way they're pointed (i.e. you can't correlate their position with elevation and azimuth ...


15

Joel actually answered this one a few years back. The actual meaning of "teach a machine how to program by itself" is "teach a machine how to take a spec and create a program that corresponds to that spec." And with that in mind: The problem, here, is very fundamental. In order to mechanically prove that a program corresponds to some spec, the spec ...


14

An iterative algorithm is said to converge when, as the iterations proceed, the output gets closer and closer to some specific value. More precisely, no matter how small an error range you choose, if you continue long enough the function will eventually stay within that error range around the final value. In some circumstances, an algorithm will not ...


12

So, I have primarily done data analysis in Matlab, but have done some in Python (and more used Python for general purpose) and also I've started a bit of R. I am going to go against the grain here and suggest you use Python. The reason why is because you are doing data analysis from a Machine Learning perspective, not stats (where R is dominant) or digital ...


11

First of all, it seems to me you are missing (or perhaps misunderstanding) the "explicitly" bit in "without being explicitly programmed" (from the quote in the question). It doesn't mean that no programming is required at all, it means that you are not programming a specific solution to the problem, but instead what you are making is a more general program ...


9

For this sort of application, a field of mirrors trying to point at a solar collector, you can very much calculate where you think the sun should be, where the mirrors should be, what angle they should be at, and how to position them so they point towards your collector. You know, a mathematical model. It'll be close. Probably close enough. As for ...


8

For each class (breed, gender...) of categorical attributes, you can add a number of components to your feature vector equal to the number of possible values in that class. Then, if a data point has the ith value, you set the ith one of those components to 1, and the rest for that attribute to 0. In your example, for gender, you would add two new components ...


8

Is Machine Learning a part of Data Science? No. Big Data vs Data Science Not the same. Birds and Bird Watching are also not the same. Machine learning is a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on the development of computer programs that can ...


7

As an old school (over 50) scientist who has and continues to use a number of these tools I will add my two cents. I have worked with colleagues who still write every piece of code in Fortran, from trivial one-off data analysis jobs to code that dominates some of the worlds supercomputers. Recent Fortran dialects (F90, F95, F2003, F2008) are IMHO, some of ...


7

Work in binary: 0, 1, 10, 11, 100, 101... Know your math: 0+0=00, 0+1=01, 1+0=01, 1+1=10 Know your logic: or, and, not, xor... Find that the low bit is a XOR and the high bit is a AND. Expand the principle of one bit to 8, 16, 32, 64 bits Build it with logic gates. If you want to know more, see my answer to How is fundamental mathematics efficiently ...


7

Usually, such filters are programmed to output not only a yes/no value for each sample, but rather a probability: a sample may, for example, be reported as 95% likely to be abusive. Then you set two thresholds, to divide your results in three groups: very unlikely to be abusive, very likely to be abusive, and uncertain. For example, you might consider ...


7

You'll need to know Linear Algebra through Eigenvectors if you want things to be "easy". Also a good statistical background with strong emphasis on Regression, Clustering, and Baye's Theorem. Knowing something about gradients doesn't hurt either. As with any CS, graph theory is helpful as well. Obviously the course can be taken with only the most basic ...


7

That course is specifically designed to be accessible to folks without 'much' math background. Of course 'much' is a relative term. In this case it means 'knowledge of calculus is helpful but not required'. The course does use some results from differential calculus, but you can answer the quizzes and complete the programs without knowing calculus yourself. ...


6

MATLAB tends not to find its way into production code for several reasons. Among these: It's proprietary It's old and hasn't aged well as a language It was traditionally difficult to to embed in other applications Points 1 and 3 aren't so much true today, since we've got Octave and there are many ways to interface MATLAB with external programs, but these ...


6

Fuzzing is a testing method where machine learning can & has been applied. Fuzzing is a method of testing in the realm of automated exploratory testing. It attempts to find defects in software by running a large number of inputs and looking for errors. Unhandled exceptions are the simplest category, but a smart implementation can use ML to find suspect ...


6

You are correct about AI which includes ML which includes DL. NN can indeed be included in ML, be it inside or outside of the DL context. An example for the latter is when neuronal nets are used in simple task based learning (e.g. recognize car number plates in pictures). Data mining is somewhat broader than your definition, because it's not only ...


5

It seems to me that you're comparing apples & oranges. A machine needs 1 day to produce x units, so, this implies that two machines like the first would need 1/2 day. No, it implies 2 machines can produce 2x units in 1 day. We don't have enough information to know if x is divisible, if the machines can cooperate, if there are external constraints (...


4

Sure, we do this all the time (for extremely limited subsets of problems). It is fairly trivial to imagine taking another step or two and tying something like Siri into the input of these code generators (or something like Wolfram Alpha) which in turn writes code and solves your problem. I would expect that something already exists somewhere to do the most ...


4

The numbers in the table indicate the number of four connected positions which include that space for example: the 3 in the upper left corner is for one each of horizontal, vertical, and diagonal lines of four which can be made with it. the 4 beside it is for two horizontal (one including starting in the corner, one starting on it, one vertical, and one ...


4

Keep a copy of the users' purchasing history within your recommendations service. Then, for each new batch run, it only has to request the updated users from the users service (a delta update, if you want to call it like that). You will need a big update to ramp up the recommendations service, and possibly a big synchronization update from time to time, ...


4

This is a supervised learning problem. It seems to be a classification task i.e. the output variable D takes class labels (the other group is the regression task). There are a lot of algorithms for classification and you should probably start with something simple e.g. logistic regression. If I misunderstood the example and yours is a symbolic regression ...


4

The Coursera course only covers "batch" or "offline" methods for machine learning. In batch methods you train the model once, and then use the trained model as a static resource. What you are looking for are online machine learning methods. Typically this involves finding an algorithm where the new data can be combined with the existing model to generate a ...


4

This is a good candidate for Differential Evolution. DE is a very simple (but powerful) population based, stochastic function minimizer/maximizer. A key point for integrating DE in your scheme is the fitness function: double fitness(Agent_k) fit = 0 repeat M times randomly extract an individual Agent_i (i <> k) switch (result of ...


4

I believe the body of knowledge you are looking for is "Mathematical Programming". In general you want to build a model to support decision making. To do this i would start with a "toy" model. This is where you take a very small example, say you only have one machine and three orders to process. You then need to answer some fundamental questions, like ...


4

Microsoft has been developing DeepCoder to use deep learning to predict a method body from a given input and outputs. That's the only example I know offhand. I can tell you that Meta-Genetic Programming is a field of study with a similar ambition, but I can't say I know enough about it to be knowledgeable. Genetic Programming was in the news in 2015 when ...


Only top voted, non community-wiki answers of a minimum length are eligible