Some State of the Art Optimizers in Neural Networks
▻https://hackernoon.com/some-state-of-the-art-optimizers-in-neural-networks-a3c2ba5a5643?source=
We are going to study Momentum, Nesterov Accelerated Momentum, AdaGrad, AdaDelta, RMSProp, Adam, AdaMax, AMSGrad.Optimization is the heart of Machine learningBefore begin on our topic, let’s understand..Why do we need Optimization?According to Merriam-Webster dictionary, meaning of the word optimize is “to make as perfect, effective, or functional as possible”. This is the definition to understand why we need Optimization in neural networks.So in machine learning, we perform optimization on the training data and check its performance on a new validation data.We already have a cost function which will tell us about the behavior of our model. Initially, our model contains arbitrary defined parameters like weights and biases and now we need to find the best possible state of those parameters (...)
#optimization-algorithms #data-science #gradient-descent #neural-networks #machine-learning