How to Initialize weights in a neural net so it performs well?
▻https://hackernoon.com/how-to-initialize-weights-in-a-neural-net-so-it-performs-well-3e9302d449
How to Initialize weights in a neural net so it performs well? — Super fast explanation for Xavier’s Random Weight Initialization▻http://www.mdpi.com/1099-4300/19/3/101We know that in a neural network, weights are initialized usually randomly and that kind of initialization takes fair / significant amount of repetitions to converge to the least loss and reach to the ideal weight matrix. The problem is, this kind of initialization is prone to vanishing or exploding gradient problems.One way to reduce this problem is carefully choosing the random weight initialization. Xavier’s random weight initialization aka Xavier’s algorithm factors into the equation the size of the network (number of input and output neurons) and addresses these problems.Xavier Glorot and Yoshua Bengio are the (...)
#andrew-ng #deep-learning #deep-neural-networks #machine-learning #neural-networks