Skip to content
Industrial Inspection & Measurement Instruments

Resolution, Accuracy, Repeatability: Three Specifications People Constantly Confuse

Resolution, Accuracy, Repeatability: Three Specifications People Constantly Confuse

The bottom line

The three most misused words in a specification table are resolution, accuracy and repeatability. They describe completely different things: resolution is about the display, accuracy is about the true value, repeatability is about consistency. Acceptance decisions rest on the last two.

What each one means

Specification Definition What to look at Typical magnitude (coating gauges)
Resolution Smallest displayed or output step Display digits 0.1 μm / 1 μm
Accuracy Maximum permitted deviation from the true value Comparison against shims or a reference ±(1%-3%)H+1.5 μm
Repeatability Agreement of repeated measurements at one spot under one condition Scatter of consecutive readings Usually better than nominal accuracy
Measurement uncertainty Quantified confidence interval for the result Expanded uncertainty on the calibration certificate Stated on the certificate

How to read “±(1%-3%)H+1.5 μm”

This is how Dongru contractor-grade models such as DR280A and DR360A express accuracy. It has two parts added together:

  • Proportional term: 1%-3% × H, where H is the current reading. At 1000 μm the proportional term alone permits 10-30 μm;
  • Fixed term: +1.5 μm, coming from zero, signal-to-noise and other thickness-independent effects.

So the same gauge performs better on thin coatings and permits more on thick ones — that is physics, not a defect. When comparing two gauges you must compare them at the same thickness, otherwise the comparison means nothing.

Why repeatability usually beats accuracy

Accuracy has to absorb systematic error — substrate differences, calibration curve, thermal drift — whereas repeatability is agreement under a single condition. In practice repeatability is clearly better than nominal accuracy, which is exactly why acceptance work focuses on the mean rather than individual readings.

SSPC-PA 2 requires at least three readings averaged per spot. That is repeatability being traded for stability.

Specifications side by side

Model Category Accuracy Range
DR280A Coating · dual mode ±(1%-3%)H+1.5 μm 0-1250 μm
DR360A Coating · magnetic (zinc weight) ±(1%-3%)H+1.5 μm 0-2000 μm
DR15A Coating · high range ±3%H+2 μm 100 μm-15 mm
DR87A Ultrasonic · through-coating ±(0.5%H+0.05) mm 0.55-500 mm
DR89S Ultrasonic · ultra-thin single crystal ±(0.01+0.3%H) mm (15P6 included) 0.15-300 mm

Uncertainty on the certificate is the real endpoint

Nominal accuracy is a design target; actual credibility comes from the measurement uncertainty stated on the calibration certificate. The uncertainty expression guide referenced by ASTM D7091-22 (ISO/IEC Guide 98-3, the GUM) requires an expanded uncertainty and a coverage factor. A certificate without them is usually rejected at audit.

Common misconceptions

  • “0.1 μm resolution means very high accuracy” — resolution only describes display capability, not deviation from the true value.
  • “±2% is always better than ±3%” — look at the fixed term too. Between ±2%H+3 μm and ±3%H+1.5 μm, the latter can be better on thin coatings.
  • “Three identical readings at one spot means high accuracy” — that means good repeatability. A systematic bias may still be present, and only a shim or reference comparison will reveal it.