147

A proper solution would probably be some learned/statistical model, but here are some fun ideas: Semi-colons at the end of a line. This alone would catch a whole bunch of languages. Parentheses directly following text with no space to separate it: myFunc() A dot or arrow between two words: foo.bar = ptr->val Presence of curly braces, brackets: while (...


54

I would be curious to see what are the average metrics of written English on one side, and code on the other side. length of paragraphs length of lines size of words chars used ratio between alphabetic, numeric and other symbol characters number of symbols per word etc. Maybe that alone could discriminate already between code and the rest. At least I ...


39

I use both Python (for data analysis ofcourse including numpy and scipy) and R next to each other. However, I use R exclusively to perform data analysis, and Python for more generic programming tasks (e.g. workflow control of a computer model). In terms of basic operations, say operations on arrays and the sort, R and Python + numpy are very comparable. ...


36

Background: I'm a data scientist at a startup in Austin, and I come from grad school (Physics). I use Python day-to-day for data analysis, but use R a bit. I also use C#/.NET and Java (just about daily), I used C++ heavily in grad school. I think the main problem with using Python for numerics (over R) is the size of the user community. Since the language ...


23

Typically, Markov chains are used to generate text, but they can also be used to predict the similarity of text (per C.E. Shannon 1950) to a trained model. I recommend multiple Markov chains. For each prevalent language, train a Markov chain on a large, representative sample of code in the language. Then, for a Stack Overflow post for which you want to ...


23

There are many differences between these two, but in practical terms, there are three main things to consider: speed, interpretability, and accuracy. Decision Trees Should be faster once trained (although both algorithms can train slowly depending on exact algorithm and the amount/dimensionality of the data). This is because a decision tree inherently "...


15

Sun paths can be predicted, so I imagine you can get the mirror aligned pretty closely already if you know the time of day, the day of the year, and the latitude and longitude. You don't need machine learning for this. If you have mirrors that don't know which way they're pointed (i.e. you can't correlate their position with elevation and azimuth ...


15

To answer just your title, yes. Neural nets can give non-boolean answers. For example, neural nets have been used to predict stock market values, which is a numeric answer and thus more than just yes/no. Neural nets are also used in handwriting recognition, in which the output can be one of a whole range of characters - the whole alphabet, the numbers, and ...


15

(1) What all features should I extract? First, realize that you're not classifying documents. You're classifying (document, query) pairs, so you should extract features that express how well they match. The standard approach in learning to rank is to run the query against various search engine setups (e.g. tf-idf, BM-25, etc.) and then train a model on the ...


15

Joel actually answered this one a few years back. The actual meaning of "teach a machine how to program by itself" is "teach a machine how to take a spec and create a program that corresponds to that spec." And with that in mind: The problem, here, is very fundamental. In order to mechanically prove that a program corresponds to some spec, the spec ...


13

May I suggest a radically different approach? On SO the only human-language allowed is English, so anything that is non-English has 99.9% of chances to be a code snippet. So my solution would be: use one of the many English language-checkers out there (just make sure they also signal - beside misspellings - syntax mistakes like double dots, or non-language ...


13

An iterative algorithm is said to converge when, as the iterations proceed, the output gets closer and closer to some specific value. More precisely, no matter how small an error range you choose, if you continue long enough the function will eventually stay within that error range around the final value. In some circumstances, an algorithm will not ...


12

So, I have primarily done data analysis in Matlab, but have done some in Python (and more used Python for general purpose) and also I've started a bit of R. I am going to go against the grain here and suggest you use Python. The reason why is because you are doing data analysis from a Machine Learning perspective, not stats (where R is dominant) or digital ...


11

I'm probably going to get a few down votes for this but I think you are approaching this from the wrong angle. This line got me: people have to go in and manually format code for people that are somehow unable to figure this out IMO that standpoint is kind of arrogant. I find this a lot in software design where programmers and designers get annoyed ...


11

Pseudo code would pose a real challenge because all programming language depend on special characters like '[]', ';', '()', etc. Simply count the occurrence of these special characters. Just like you would detect a binary file (more than 5% of a sample contains byte value 0).


11

I think there're clear rules on how cities are partitioned into quarters or regions. You should ask your local administration on where they draw the borders. Then you could, for example, retrieve the location data of the address (latitude and longitude might work) and simply check in which region's boundaries this address is in. There's no need for a ...


11

First of all, it seems to me you are missing (or perhaps misunderstanding) the "explicitly" bit in "without being explicitly programmed" (from the quote in the question). It doesn't mean that no programming is required at all, it means that you are not programming a specific solution to the problem, but instead what you are making is a more general program ...


10

If a machine can look at the state of the board for a few games of chess (or a few games of checkers) in the beginning and after each move, can it be programmed to learn the rules of the game? Certainly not for a few games of chess; you'd need to analyse incredibly large numbers of them to stop it from making invalid moves. How much, I don't know; this ...


9

Take a look at n-grams. One n-gram is a sequence of n words. In your case you want n to be 3, since you need two query words and a resulting word. One 3-gram would be for example "I am tired", another one "I am happy". What you then need is a collection of these 3-grams that are collected over your target language, say English. Since you cannot collect it ...


9

For this sort of application, a field of mirrors trying to point at a solar collector, you can very much calculate where you think the sun should be, where the mirrors should be, what angle they should be at, and how to position them so they point towards your collector. You know, a mathematical model. It'll be close. Probably close enough. As for ...


8

The heart of Watson is IBM DeepQA software. We find some answers on it's FAQ: Q: What data is stored in Watson? A: All of Watson's data will be self-contained. Watson will perform without a connection to the web or any external resource. The vast majority of Watson's data will be a wide variety of natural language text. Some structured (...


8

Machine learning is a whole field in Computer Science, which is quite different from Artificial Intelligence. Stanford provides a free online class for machine learning which will get you started. You need to understand that machine learning alone will most probably not be sufficient to create good trading strategies; you need to understand well how the ...


8

For each class (breed, gender...) of categorical attributes, you can add a number of components to your feature vector equal to the number of possible values in that class. Then, if a data point has the ith value, you set the ith one of those components to 1, and the rest for that attribute to 0. In your example, for gender, you would add two new components ...


8

Is Machine Learning a part of Data Science? No. Big Data vs Data Science Not the same. Birds and Bird Watching are also not the same. Machine learning is a type of artificial intelligence (AI) that provides computers with the ability to learn without being explicitly programmed. Machine learning focuses on the development of computer programs that ...


7

Maching Learning and Natural Language Processing is somewhat data-driven. Without a continuous supply of high-quality data (which must be re-captured whenever new criteria are added), the software development may miss its intended target. The customer and product owner may devote a bigger fraction of their time toward test data collection. The adaptations ...


7

That course is specifically designed to be accessible to folks without 'much' math background. Of course 'much' is a relative term. In this case it means 'knowledge of calculus is helpful but not required'. The course does use some results from differential calculus, but you can answer the quizzes and complete the programs without knowing calculus yourself. ...


7

You'll need to know Linear Algebra through Eigenvectors if you want things to be "easy". Also a good statistical background with strong emphasis on Regression, Clustering, and Baye's Theorem. Knowing something about gradients doesn't hurt either. As with any CS, graph theory is helpful as well. Obviously the course can be taken with only the most basic ...


7

As an old school (over 50) scientist who has and continues to use a number of these tools I will add my two cents. I have worked with colleagues who still write every piece of code in Fortran, from trivial one-off data analysis jobs to code that dominates some of the worlds supercomputers. Recent Fortran dialects (F90, F95, F2003, F2008) are IMHO, some of ...


7

Usually, such filters are programmed to output not only a yes/no value for each sample, but rather a probability: a sample may, for example, be reported as 95% likely to be abusive. Then you set two thresholds, to divide your results in three groups: very unlikely to be abusive, very likely to be abusive, and uncertain. For example, you might consider ...


7

Work in binary: 0, 1, 10, 11, 100, 101... Know your math: 0+0=00, 0+1=01, 1+0=01, 1+1=10 Know your logic: or, and, not, xor... Find that the low bit is a XOR and the high bit is a AND. Expand the principle of one bit to 8, 16, 32, 64 bits Build it with logic gates. If you want to know more, see my answer to How is fundamental mathematics efficiently ...


Only top voted, non community-wiki answers of a minimum length are eligible