Optimization lies at the heart of machine learning. In this talk, I will introduce continuous-time analyses of discrete-time optimization algorithms for both convex and nonconvex optimization. By viewing gradient based optimization algorithms as the discretization of continuous-time (stochastic) differential equations, we will be able to better understanding the acceleration mechanism of Nesterov’s method and design new simple algorithms that achieve the optimal convergence rate. In particular, I will discuss novel stochastic differential equations (SDEs) for accelerated stochastic mirror descent and several new discrete-time algorithms derived from the SDEs. I will also introduce a more general SDE, the Langevin dynamics, that enables us to study the global convergence in nonconvex optimization.