1

Lately, there has been a lot of hype around Neural Networks. They took over the field of AI, finding applications in computer vision, medicine, natural language processing, etc.

There are active fields of research in understanding them and their capabilities, but it seems like solving more and more AI problems is more about finding the right NN architecture (there is also active research on metalearning - automating the design of neural nets).

Since more and more problems can be solved by finding the right architecture of a neural network, can we say that a new programming paradigm is being born?

Edit: Most answers argue that neural networks are by no means new. Indeed, they have been present for at least 50 years, at least since when the perceptron was introduced in late fifties. I would say that neural networks, as we know them, have been around since backpropagation started being applied to neural networks in eighties. However, there seems to be an exponential increase in usage of neural networks. The question was revolving around the idea that with neural networks, solving a problem is about designing an architecture that solves the problem. Considering this exponential increase in their usage and number of applications, at which point may neural networks be considered a paradigm on its own?

  • 5
    Your question basically boils down to precisely defining the term "programming paradigm", which is notoriously difficult to find a useful definition for. As such, the answer to your question is simply a matter of opinion, and therefore off-topic on this site. (FWIW, though, ANNs have been a hot topic in AI since the 90s, and describing them as a "new" anything is possibly a bit of a stretch...) – Jules Jun 11 '17 at 0:51
  • @Paul92: If you can define "programming paradigm", then I can answer the question. If you can't provide a definition, any answer will useless. – JacquesB Jun 11 '17 at 13:20
  • The programming paradigm behind neural networks is called dataflow – mouviciel Jun 12 '17 at 9:46
4

can we say that a new programming paradigm is being born?

You can. But you're either being silly or you're in advertising.

The programming paradigm (and it's techniques) that you're talking about is making new achievements but it isn't new.

AI was born before the digital computer. It's been the dream of kings. Computer science wouldn't be here without it. AI didn't start with ones and zeros. It started with math. Hell it's older even then the clocks that fill clock towers.

What's changed is the hardware and the scale of tasks we're able achieve.

Yes architecture makes a big difference. But so does matching the procedure to the task. I've seen massive Hadoop clusters put to shame by a single computer running a well written Bash one liner.

So no this isn't a paradigm shift. If anything it's the power of marketing.

But still, I like being able to tell Siri "take me home".

  • "you're either being silly or you're in advertising." There's a difference? – Alexander Jun 12 '17 at 17:06
3

Are neureal nets a paradigm ?

Neural nets are a domain for a set of specific problems and their solutions. As neural nets deeply influence the way a system is build, they could be considered as an architectural pattern. As they further define a set of related concepts and thought patterns they could be called a paradigm.

What is a programming paradigm ?

A programming paradigm is in general understood as a way to program and structure code that is supported by programming language features and used throughout the code.

You may for instance find functional programming, or object oriented programming features in several programming language, and throughout almost every piece of code written according to this paradigm. But you'll never find this for neural nets. A neural nets is not a structure that is directly in the program code: it's something that emerges from the whole system. This is why the architectural pattern would be more appropriate.

Furthermore, the neural net code is useless without the training/learning acquired by the net. So the neural net is not only about programming.

Conclusion

Whereas neural nets are used in programming and form a paradigm, they are not a programming paradigm according to the sens that is generally given to that term.

Its just like graphs: these are also a quite old concept, with related thought patterns (quote of the day "once you have an A* algorithm in your brain, every problem looks like a GPS"). So they are clearly a paradigm. Graphs are now ubiquitous in programming. There are lots of frameworks and application domains (e.g. neural nets are graphs). There are even dedicated database engines with graph-specific querying language. But nobody dares to call this a "programming paradigm".

2

There's a fuzzy line between a technique and a paradigm. There's no great definition for "programming paradigm" beyond "general characteristics of how a program is written" (e.g. we might say that a program is written in a functional paradigm if it works primarily with immutable objects and heavily with higher order functions). Importantly, a paradigm is an overall characteristic of how code is written and not any object named in the code: in a Haskell program there is generally no object called functionalness.

By contrast a neural network is going to be named in the code. You can have a neural network library in a way that you cannot have a functional programming library.

I would also say that neural networks lack a certain amount of genericity that precludes them from becoming a paradigm: they are unparalleled for creating particular functionality in a program (e.g. "identify if this picture is a cat") but not at all suited for other functionality ("find the logarithm of this number", "map this function over a list", "parse this csv into a list of lists", etc.).

  • I agree that a paradigm is a way of writing programs, and programming languages usually offer support for this. However, even without explicit support, using clever tricks such as structures with function pointers and unions you can fundamentally do oop in C, which is seen as a procedural language. Also, function pointers and immutable objects can be used for fp, even if you don't have explicit support. The reason why you don't have fp libraries i think has more to do with demand (you have more sensible options). And you have lambdas, which are a touch of fp in oop languages. – Paul92 Jun 11 '17 at 0:43
  • @Paul92 I agree that paradigms are features of programs rather than programming languages and don't see where I said otherwise. – walpen Jun 11 '17 at 0:46
  • You could even easily question the programming part of NN. The thing that actually does the work in a NN is not programmed per se. – tofro Jun 11 '17 at 9:33
  • @tofro You got the main motivation for my question - I feel like there is another way of solving problems; rather than write a program, you design a NN architecture which solves your problem. The question was when and if can this be called a (programmign) paradigm. – Paul92 Jun 11 '17 at 10:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.