0

I've read about NEAT/Evolutionary Artificial Neural Networks/Genetic Algorithms.

I understand the concept of choosing the fittest neural networks and breeding them to produce another one, but how exactly does this work?

Do you simply choose at random, weights from the parents, for the child network? Do you cross over the bias weights too?

How would this produce a network as fit as its parents, would it not just produce a new random network, because the weights have lost their correlation?

After trying this weight-swapping, I got very bad results.

What are the in-depth steps to evolving a feed-forward neural network?

Something like this

  • 1
    "Your questions should be reasonably scoped..." (help center) – gnat Sep 13 '17 at 20:34
  • @gnat ,This question is supposed to be migrated here Cross Validated for effective feedback – quintumnia Sep 16 '17 at 11:46
1

Do you simply choose at random, weights from the parents, for the child network?

no.

NEAT is an algorithm as described in the paper "Evolving Neural Networks through Augmenting Topologies" by Kenneth O. Stanley & Risto Miikkulainen.

Which you can find a copy of (as of the time of writing...) here: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf

The relevant section you're looking for is 3.1 (Genetic encoding).

This is a complex process, and I can't possibly offer you a step-by-step guide to implementing it, however, in summary:

  • Each connection is modelled as a gene.
  • Each gnome is modelled as a sequence of genes.
  • A specific algorithm converts the gnome sequence into a NN.

Evolution is achieved by additional connection genes which are added to the sequence or individual nodes are altered. This generates trivial connection changes to the network, generating minor variance in the network structure.

and breeding them...

Read the paper, page 12.

The specific layout of genes in a linear sequence allows two parents to be combined by 'overlaying' the two gnomes on top of each other, and selecting effectively the union of the two.

What are the in-depth steps to evolving a feed-forward neural network?

That's far beyond the scope of what be easily answered here.

  • Thanks for the tips. I will conquer my fear of academic papers, tonight. – Tobi Sep 14 '17 at 2:55
  • You can also always dig into the source code of an implementation, eg. github.com/colgreen/sharpneat, but the implementations vary slightly from the actual paper's method. – Doug Sep 14 '17 at 3:57
  • I'd tried looking into various implementations, but the code was either seemingly doing what I was doing, or consisted of thousands of lines. – Tobi Sep 14 '17 at 13:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.