In this tutorial we'll implement vanilla gradient descent and gradient descent with momentum from scratch. We'll also visualize the algorithms and compare different optimizers using PyTorch, including:
For a more thorough breakdown of the code, check out the full tutorial Gradient Descent and Optimization in Deep Learning on the blog.
