A gradient is the vector of partial derivatives of a scalar-valued function with respect to each of its input variables, pointing in the direction of steepest ascent at a given point. In machine learning it is the quantity computed by backpropagation and consumed by optimisers to update model parameters. Its magnitude and direction drive first-order optimisation methods such as gradient descent and its momentum-based variants.

Provenance