Genetic Programming
Genetic Programming is a technique for the automatic generation of programs based on the ideas of evolution and survival of the fittest. It is the topic I chose for my MSc thesis at Edinburgh University.
The abstract is:
This dissertation presents and investigates a new technique for using sub-routines in genetic programming. The technique differs from previous ideas by having separate populations for programs and functions and evolving the members of each concurrently. It was hoped that this technique will solve problems as successfully as existing methods with the additional advantage of having less structure determined in advance by the experimenter. It was also hoped that the technique would be superior to existing methods at finding generally useful sub-routines. The initial algorithm was found to be better than genetic programming techniques that don’t use sub-routines but inferior to Koza’s automatically defined functions (ADFs). The reasons behind the performance are analysed and three refinements are investigated. One of these improved the performance but remained inferior to ADFs. The system was found to discover generally useful sub-routines on successful runs.
The final dissertation is available.
I also have the source code for the Lisp Genetic Programming system I wrote to do the investigations. If you want to have it then drop me a line, however be warned that the code ended up as a bit of a mess. As such I am in the process of rewriting in a much more powerful and better designed version that will additional functionality. More news here soon(ish).
Neural Networks
About the simplest form of Neural Network is a multilayer perceptron network. However they are extraordinarily powerful as they can approximate almost any well behaved mathematical function. Here is a proof for the approximation of continuous functions.