Contact info
Word
Research
   Publications
Studies
Free Software
Hobbies
Articles
Photography
About me
   Curriculum Vitae

©Marko Grönroos, 1998

USENET News comp.ai.genetic

Säie: GA and Neural Networks

Lionel Barnett <lionelb AT cogs PISTE susx PISTE ac PISTE uk> writes: > In the case of backprop at least, convergence to an optimal solution is > notoriously slow. It is conceivable that a GA (or other search > algorithm) could find a good solution faster, given limited time > resources. Some backprop variants, such as quickprop or RProp (resilient backprop), are much faster than the original "vanilla" backprop.       What comes to GAs vs. backprop, I think even ordinary backprop is much faster than a GA, because bp can find the direction of the optimum just by directly calculating it from the derivative of the error surface, while GA has to "try" the surface indirectly by generating a population of instances randomly. Certainly GA is in this case less likely to get stuck in a local minimum, but some backprop variants have some techniques (such as momentum) to prevent that.       Quickprop makes estimates about how far the optimum is from the current point by using second-order derivatives and assuming that the error surface is somewhat quadratic. Certainly a GA could also do such analysis of the surface to calculate the optimal mutation rate, but that would again make it a specialized ad hoc algorithm.       Thus, as usual, specialized algorithms perform better than more generic ones. > Beyond backprop, and in particular for recurrent networks > there are generally no known learning algorithms, slow or otherwise. > GA's may work well; the beauty of GA's is that they don't necessarily > have to be "closely tied" to the structure of the problem at hand... I think some people have used backprop with recurrent nets too, but I'm not certain.       I would think that it is trivial to transform ("unloop") any recurrent ANN that runs for a certain number of iterations to a chain of feedforward ANNs, which you can then train with backprop.       But you don't have to do the "unlooping", but just first iterate the recurrent net forward, then calculate the error at some iteration, and then propagate the signal back, but again recurrently.       I've also thought of some nice variants of this technique, but I'll leave them for some other time. But yeh, I too find GAs (EAs) very interesting approaches to certain recurrent ANN training problems. -- -- Marko Grönroos, magi AT iki PISTE fi (http://www.iki.fi/magi/) -- Paradoxes are the source of truth and the end of wisdom

Edellinen säie: NN evolving techniques
[Muut säikeet] [Muut uutisryhmät]