by roberto » Fri Sep 20, 2013 2:44 pm
Thinking deeper, there is alternative to Back Propagation algoritm to train a Neural Network: to use Genetic Algoritm to evolve a population of weights till reach the element that sutusfy the NN. This is a interesting path to explore, cos it is demostrate that can give great speed up to train and in this case, parallela is a overkill because AG need a simple classic cpu to be runned. I played with GA in past and they can give extraordinary results (few minutes of calculation) where a classic algoritm "brute force" need centuries to produce a result. In this case, the Back Propagation algoritm is concettually a brute force system, no matter if multy-layer-few-neurons-per-layer or single-layer-with-many-neurons NN.