Resolution and accuracy

Accuracy

Whether a measuring instrument is suitable for a measurement task or not largely depends on the requirements the instrument must meet to perform a sufficiently precise measurement for the user. An important requirement is the resolution and accuracy of the measurement. The measurement accuracy of an instrument is often referred to as “basic accuracy.” This value is expressed as a percentage [%] and indicates the maximum deviation of the measured result from the physically correct value. A basic accuracy of ±0.05% means that the measured value deviates by a maximum of +0.05% or -0.05% from the physically correct value. The basic accuracy specified by the manufacturer usually represents the smallest possible deviation, which may increase by changing parameters such as measurement frequency or measurement speed.

Resolution

Another important factor for a qualitative measurement is the resolution. It indicates the step sizes in which measured values can be reproduced. The higher the resolution of a measuring instrument, the more accurately a value can be displayed. In other words, the higher the resolution, the more different values can be represented in this measurement range. For example, if the measurement range is 0Ω – 1000Ω with a resolution of 1Ω, 1000 different measurement values can be displayed. However, if the resistance value is to be measured accurately to 0.5 Ω, a measuring instrument with a minimum resolution of 0.5Ω is required. The more requirements are known beforehand, the better the measuring instrument can be matched to the application.

Example 1: “Calculation of Measuring Instrument”

TARGET values: You want to measure a resistance between 0.01 Ω and 0.05 Ω. Your tolerance should be +/- 5%. The following calculation then results: 10 mΩ (0.01 Ω) * 5% = 0.5 mΩ

ACTUAL values: e.g. resistance measuring instrument DU5010 with a measurement range of 200 mΩ: 0.01 mΩ – 199.99 mΩ. The following calculation then results: 200 mΩ * 0.1% = 0.2 mΩ

Since 0.2 mΩ change is smaller than 0.5 mΩ, the accuracy of the resistance measuring instrument DU5010 would be completely sufficient.

Example 2: “Calculation of Resolution and Accuracy”

Resistances of 5 Ω are to be measured. The maximum deviation of the resistances may be 0.1 Ω. From the maximum deviation to be measured, the minimum required resolution of 0.1 Ω results. The deviation of 0.1 Ω then corresponds to the following percentage deviation: (0.1 Ω / 5 Ω) * 100 = 2 %

Thus, a measuring instrument with a basic accuracy of less than 2% is required.
Thus, 2 requirements are clear:

Resolution ≤ 0.1 Ω

Basic accuracy ≤ 2 %

« Back