High learning rates often lead to weight change oscillations in the training process, which can cause non-convergence or return a non-optimal solution. Momentum makes it less likely for such undesirable cases to occur by making the next weight change a function of the previous weight  change  to  provide  a  smoothing  effect.   The value for momentum (between 0 and 1) determines the proportion of the last weight change that is added to the next weight change.