Short answer
Measurement reliability asks whether results are consistent; measurement validity asks whether evidence supports what those results are taken to mean or justify. A test can produce dependable scores without sufficient evidence for its intended interpretation. Reliability contributes to validity assessment, but it is not enough on its own. 1 2
On this page
At a glance
Scroll to compare all columns →
| Question or attribute | Measurement reliability | Measurement validity |
|---|---|---|
| Central question | Are measurements consistent? | Is this interpretation or use supported? |
| Typical comparison | Items, occasions, raters, or versions | Scores and evidence about the intended outcome |
| Context | Assessment conditions matter | Purpose, setting, and learners matter |
| What it cannot establish alone | That scores measure the intended outcome | A context-free endorsement of the tool |
These are connected questions, not interchangeable labels. 1 2
What each thing is
Reliability describes relationships within or across measurements: repeated administrations, different raters, assessment items, or alternate forms can each be examined. Validity concerns the usefulness and justification of inferences from those measurements. Here, neither term describes everyday trustworthiness or whether an argument is logically valid; the focus is assessment scores and their interpretation. 1 2
Key differences
Reliability evidence addresses consistency under the comparison being studied. Validity evidence addresses the intended meaning or purpose: for example, whether scores relate to later outcomes or to assessments of the same construct. Consequently, a strong reliability result does not answer every validity question. The required evidence depends on the claim being made from the scores. 1 2
How to tell them apart
A practical rule: ask what the comparison is meant to establish. Agreement across raters points toward reliability; support for interpreting scores as the intended outcome points toward validity. The limit is that reliability evidence can also contribute to the validity argument. 2
Examples
- Hypothetically, two observers give matching clinical-simulation ratings. That supports interrater reliability, not automatically the claim that the ratings represent clinical competence.
- Hypothetically, an assessment’s scores relate strongly to a later related measure. That supplies predictive validity evidence, rather than directly testing repeat-administration consistency. 1
Where they overlap
Reliability and validity are not rival qualities. The primer treats reliability as part of validity assessment and says both matter for credible study results. Relationships among items may therefore be examined for consistency while also contributing evidence about whether the assessment’s internal structure fits the intended interpretation. 1 2
Edge cases
A reliability label can hide different questions: raters may agree even when repeat-administration consistency has not been examined. Likewise, validity evidence for one purpose should not be treated as a blanket endorsement for another setting or learner group. The primer also reports that reliability is sometimes called internal validity or internal structure, so surrounding terminology needs careful reading. 1 2
Why the distinction exists
Separating the concepts prevents reproducibility from standing in for meaning. An assessment is used to draw conclusions, not merely to generate repeatable numbers. Examining validity makes the intended inference explicit and connects it to supporting evidence; examining reliability addresses consistency within that broader assessment. 1 2
Common misconceptions
“Highly reliable” does not mean “valid for every use.” Nor does validity require comparison with a universally accepted benchmark: the primer notes that such a benchmark often does not exist, and comparisons may instead involve other reasonable assessments. Finally, calling a tool “valid” without specifying its interpretation and context leaves an important qualification unstated. 2
Examples
Hypothetical cases are included under how_to_tell_them_apart.