Performance & Evaluation

Human Evaluation

Using human raters to judge AI output quality — the gold standard for open-ended tasks.

Definition

For tasks where quality cannot be measured automatically — the helpfulness of a response, the accuracy of a summary, the appropriateness of a tone — human evaluation is required. Evaluators review AI outputs and rate them on specified dimensions. While slower and more expensive than automated metrics, human evaluation is the most reliable way to assess AI quality in real-world terms. It should be part of any serious AI deployment process.

Related Terms

Benchmark

A standardised test used to compare and measure AI model performance.

Factual Accuracy

Whether an AI's statements are true and verifiable.

RLHF (Reinforcement Learning from Human Feedback)

Teaching an AI to improve its responses using human ratings to align it with human preferences.

Heard enough terminology — ready to talk outcomes?

We translate AI concepts into measurable business results. No upfront fees — you pay only when independently verified results are delivered.

← Back to glossary

Disclaimer

This definition is provided for educational and informational purposes only. It represents a general explanation of a technical concept and does not constitute professional, technical, or investment advice. Artificial intelligence is a rapidly evolving field; terminology, techniques, and capabilities change frequently. Coaley Peak Ltd makes no warranty as to the accuracy, completeness, or currency of the information provided. Nothing on this page should be relied upon as the sole basis for commercial, technical, legal, or investment decisions without independent professional advice.

Document reference: ISO_webpage_knowledge-base_glossary_v1

Last modified: 29 March 2026

Knowledge Base·Performance & Evaluation·Human Evaluation