Training & Fine-tuning

Gradient Descent

The algorithm models use to gradually improve during training by reducing errors.

Definition

Gradient descent is the core learning algorithm in deep learning. During training, the model makes predictions and measures how wrong they are (the loss). Gradient descent calculates which direction to adjust each parameter to reduce the loss, and makes a small step in that direction. Repeat this billions of times across the training dataset and the model gradually improves. The 'gradient' is a mathematical quantity indicating the direction and size of the adjustment needed.

Related Terms

Loss Function

The measure of how wrong a model's outputs are — training aims to minimise this.

Backpropagation

The process of passing error signals backwards through a network to update weights.

Learning Rate

How quickly a model updates its weights during training — a critical training hyperparameter.

Heard enough terminology — ready to talk outcomes?

We translate AI concepts into measurable business results. No upfront fees — you pay only when independently verified results are delivered.

← Back to glossary

Disclaimer

This definition is provided for educational and informational purposes only. It represents a general explanation of a technical concept and does not constitute professional, technical, or investment advice. Artificial intelligence is a rapidly evolving field; terminology, techniques, and capabilities change frequently. Coaley Peak Ltd makes no warranty as to the accuracy, completeness, or currency of the information provided. Nothing on this page should be relied upon as the sole basis for commercial, technical, legal, or investment decisions without independent professional advice.

Document reference: ISO_webpage_knowledge-base_glossary_v1

Last modified: 29 March 2026

Knowledge Base·Training & Fine-tuning·Gradient Descent