Model Architecture

Softmax

The mathematical function that converts a model's raw scores into probabilities.

Definition

When a language model decides what token to generate next, it first produces a raw score for every token in its vocabulary. Softmax converts these raw scores into a probability distribution — a set of percentages that add up to 100%, showing how likely each token is to be the right next choice. The model then selects a token based on these probabilities. Temperature settings interact with softmax to make outputs more or less predictable.

Related Terms

Temperature

A setting that controls how creative or random an AI's outputs are.

Logits

The raw numerical scores a model produces before they're converted to probabilities.

Token

The basic unit an LLM reads and writes — roughly equivalent to a word or part of a word.

Heard enough terminology — ready to talk outcomes?

We translate AI concepts into measurable business results. No upfront fees — you pay only when independently verified results are delivered.

← Back to glossary

Disclaimer

This definition is provided for educational and informational purposes only. It represents a general explanation of a technical concept and does not constitute professional, technical, or investment advice. Artificial intelligence is a rapidly evolving field; terminology, techniques, and capabilities change frequently. Coaley Peak Ltd makes no warranty as to the accuracy, completeness, or currency of the information provided. Nothing on this page should be relied upon as the sole basis for commercial, technical, legal, or investment decisions without independent professional advice.

Document reference: ISO_webpage_knowledge-base_glossary_v1

Last modified: 29 March 2026

Knowledge Base·Model Architecture·Softmax