How do you evolve a neural network?

Question

I've read about NEAT/Evolutionary Artificial Neural Networks/Genetic Algorithms.

I understand the concept of choosing the fittest neural networks and breeding them to produce another one, but how exactly does this work?

Do you simply choose at random, weights from the parents, for the child network? Do you cross over the bias weights too?

How would this produce a network as fit as its parents, would it not just produce a new random network, because the weights have lost their correlation?

After trying this weight-swapping, I got very bad results.

What are the in-depth steps to evolving a feed-forward neural network?

Something like this

Answer 1

Do you simply choose at random, weights from the parents, for the child network?

no.

NEAT is an algorithm as described in the paper "Evolving Neural Networks through Augmenting Topologies" by Kenneth O. Stanley & Risto Miikkulainen.

Which you can find a copy of (as of the time of writing...) here: http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf

The relevant section you're looking for is 3.1 (Genetic encoding).

This is a complex process, and I can't possibly offer you a step-by-step guide to implementing it, however, in summary:

• Each connection is modelled as a gene.
• Each gnome is modelled as a sequence of genes.
• A specific algorithm converts the gnome sequence into a NN.

Evolution is achieved by additional connection genes which are added to the sequence or individual nodes are altered. This generates trivial connection changes to the network, generating minor variance in the network structure.

and breeding them...

Read the paper, page 12.

The specific layout of genes in a linear sequence allows two parents to be combined by 'overlaying' the two gnomes on top of each other, and selecting effectively the union of the two.

What are the in-depth steps to evolving a feed-forward neural network?

That's far beyond the scope of what be easily answered here.