Training & Fine-tuning

Pre-training

The initial, large-scale training phase where a model learns from massive datasets.

Definition

Pre-training is the first and most expensive phase of building an AI model. The model is exposed to enormous quantities of data and adjusts its billions of parameters to become better at predicting text (or images, or other modalities). This phase can take months and cost millions of pounds in computing resources, which is why only a handful of organisations do it. The resulting pre-trained model forms the foundation on which more specialised capabilities are built.

Related Terms

Fine-tuning

Further training a pre-trained model on specific data to specialise it for a task.

Training Data

The text, documents, or other content an AI learns from during its development.

Foundation Model

A large AI model trained on broad data that can be adapted for many specific tasks.

Heard enough terminology — ready to talk outcomes?

We translate AI concepts into measurable business results. No upfront fees — you pay only when independently verified results are delivered.

← Back to glossary

Disclaimer

This definition is provided for educational and informational purposes only. It represents a general explanation of a technical concept and does not constitute professional, technical, or investment advice. Artificial intelligence is a rapidly evolving field; terminology, techniques, and capabilities change frequently. Coaley Peak Ltd makes no warranty as to the accuracy, completeness, or currency of the information provided. Nothing on this page should be relied upon as the sole basis for commercial, technical, legal, or investment decisions without independent professional advice.

Document reference: ISO_webpage_knowledge-base_glossary_v1

Last modified: 29 March 2026

Knowledge Base·Training & Fine-tuning·Pre-training