Technical interview guideNew
Gradient Descent and Optimizers
How gradient descent minimises a loss by stepping against its gradient, and how learning rate, batch size, momentum, adaptive methods and schedules change how that descent behaves in practice. Built for ML engineers who must explain and compare optimizers in interviews.
- Read
- 14 min
- Practice MCQs
- 25
- Interview Q&A
- 25
- Edition
- v1
- Editorial status
- Reviewed
Scope: Current to PyTorch 2.x (torch.optim defaults: Adam lr 1e-3, betas (0.9, 0.999), eps 1e-8, weight_decay 0; AdamW weight_decay 0.01) and TensorFlow 2.x / Keras 3 optimizers. Convergence statements assume smooth losses unless stated; learning-rate limits quoted for quadratic losses with curvature lambda..
Curated: · Written: · Reviewed: