## Follow ISOBudgets blog by Rick Hogan on Feedspot

or

ISOBudgets blog by Rick Hogan by Richard Hogan - 1M ago

Introduction

Linearity uncertainty (a.k.a. linearity error or non-linearity) is a source of uncertainty that should be included in most uncertainty budgets. It is a common characteristic published in the manufacturer’s specifications for various types of measurement equipment. However, I do not see it included in uncertainty budgets as often as it should be.

If you have a test or measurement function that spans across a range of values, then you may need to include linearity in your uncertainty analysis. Therefore, I thought that it would be a great idea to develop a guide to show you how to estimate linearity uncertainty step by step using Microsoft Excel.

In this guide, you are going to learn everything that you need to know about linearity uncertainty, including;

1. What is Linearity Uncertainty
2. Why is Linearity Uncertainty Important
3. When Should You Include Linearity Uncertainty
4. Linearity Uncertainty Methods
5. Which Uncertainty Method You Should use
6. How to Calculate Linearity Uncertainty (Step By Step)

What is Linearity Uncertainty

Linearity is the property of a mathematical relationship or function that can be graphically represented as a straight line.

Non-linearity is the deviation from a straight line over a desired range.

Therefore, linearity uncertainty would the uncertainty associated with non-linear behavior observed across the range of an assumed linear function.

When you think about how measurement equipment functions, you probably assume that its measurement performance is linear across the measurement range.

However, this is not typically the case.

The measurement functions of most devices are not actually linear. Instead, they are approximately linear. So, we try to correct them with coefficients and linear or polynomial equations to make their performance more predictable.

Still, prediction equations and coefficients do fully correct for their non-linear behavior. Therefore, we must take linearity uncertainty into consideration.

Non-linear behavior is most commonly observed for many mechanical devices and physical materials. For example, here is a list of devices that are commonly evaluated for linearity;

• Pressure Gauges (with bourdon tubes)
• Pressure Transducers (with strain gauges)
• Force gauges,
• Scales and analytical balances,
• Torque transducers,
• Resistance Thermometers (e.g. PRT, RTD, thermistors, etc.),
• Liquid in glass thermometers (e.g. mercury, spirit-filled, etc.),
• Hygrometers,
• Dial indicators,
• and much more.

Additionally, many electrical devices can exhibit non-linear behavior too.

• Digital Multimeters,
• Multifunction Calibrators,
• Oscilloscopes,
• LCR Meters,
• Phase Meters,
• Thermocouple Simulators,
• Power Sensors,
• Signal Generators,
• and much more.

No matter what type of equipment you are using, do not forget to consider linearity in your uncertainty analysis unless it is negligible or inappropriate to do so.

Why is Linearity Uncertainty Important

Linearity uncertainty is important because it allows you to consider the effects of non-linear behavior in a measurement function. If you use an equation to estimate uncertainty across a measurement range, then you may need to consider evaluating linearity uncertainty.

I often hear people say that linearity is not important or doesn’t need to be included in an uncertainty budget. I say, test it and let the results speak for themselves.

If the result is small or negligible, great! Now, you have objective evidence to support your opinion. However, I would still include it your uncertainty budget to prove you considered it.

If the result is significant, then include the results in your uncertainty budget.

If you are unable to calculate linearity, try reading the manufacturer’s manuals and datasheets to see if they list it in the specifications.

Nonetheless, linearity uncertainty is important. At least consider whether or not it affects your measurement uncertainty.

When Should You Include Linearity Uncertainty

You should include linearity in your uncertainty analysis anytime you are estimating uncertainty for a continuous measurement range.

If you plan to use a linear equation to predict the measurement uncertainty of a given measurement range, then you should include linearity into your uncertainty analysis.

When estimating measurement uncertainty across a measurement range, you will typically estimate uncertainty at test-points close to the minimum and maximum of the measurement range. Since your uncertainty analysis doesn’t estimate uncertainty at the points in between the minimum and maximum values, you need to take non-linearity of the function into consideration.

Additionally, many measurement instruments exhibit non-linear behavior below 10% of the measurement range. When you estimate uncertainty for values below 10% of the measurement range, you are more likely to see non-linear behavior the closer you get to zero.

So, be sure to take that into consideration when selecting test-points for your uncertainty analysis.

Linearity Uncertainty Methods

There are two common methods that you can use when estimating linearity uncertainty. They are;

1. Maximum Deviation from Linearity
2. Typical Deviation from Linearity
Method 1: Maximum Deviation from Linearity

Maximum error provides the maximum deviation from the linear behavior of a fitted line prediction equation (e.g. regression, interpolation, B.F.S.L.).

Method 2: Standard Error from Linearity

Standard error provides the typical deviation from the linear behavior of a fitted line prediction equation (e.g. regression, interpolation, B.F.S.L.).

Both methods evaluate the deviation from linearity. The difference between the two methods is one method evaluates the worst case scenario and the other evaluates the most likely or most probable scenario.

Which Uncertainty Should You Use: Max or Standard

The maximum deviation method is the most commonly used method for evaluating linearity uncertainty. Additionally, it is the most recommended method. If you decide to do some research, you are more likely to find information on the maximum deviation method.

NIST recommends the maximum deviation method in their NIST/SEMATECH Engineering Statistics Handbook. See the excerpt provided below.

When evaluating linearity uncertainty, I prefer to use the standard error method. I believe that it is more applicable to an uncertainty analysis and the development of a CMC Uncertainty predication equation, especially if I have already considered bias or error in my uncertainty analysis.

If you choose to use the maximum deviation for linearity, you need to be careful not to confound your results and overstate your estimated measurement uncertainty.

When you include bias or error in your uncertainty analysis, you are more likely to overstate your uncertainty using the maximum deviation method. Especially, since the maximum deviation and bias could end up being the same result!

If you choose to use the standard error method, you are more likely to understate your measurement uncertainty if you do not include bias in your uncertainty budget.

When you include bias in your uncertainty analysis, using the standard error for linearity uncertainty is more likely to give you a better estimate of uncertainty in measurement.

So, use the method you like best. At least you should know what options you have available and why you selected to use the method you have chosen if anyone ever asks you a question.

How to Calculate Linearity Uncertainty

To calculate linearity uncertainty, I am going to show you how to perform regression analysis in Microsoft Excel and find the maximum deviation and standard error.

In Microsoft Excel, there are two processes that you can use to easily use to get results;

1. Data Analysis ToolPak, and
2. LINEST and INTERCEPT functions.
Option 1
Finding Linearity Uncertainty with Data Analysis ToolPak

In this section, you will learn how to use Data Analysis ToolPak to find your linearity uncertainty following the four steps below;

1. Install Data Analysis ToolPak,
2. Enter Your Standard and UUT Data,
3. Perform Regression Analysis, and
1. Install Data Analysis ToolPak

To calculate linearity uncertainty, you will need to perform regression analysis. To do this in Microsoft Excel, you will need to install Data Analysis Tool Pack.

Since this add-in comes built into Microsoft Excel, all you need to do is activate it. To activate Data Analysis Tool Pack, follow the steps below:

a. Click the File tab

b. Click Options (on the left side panel)

c. A new window will open. Click Add-ins.

d. At the bottom of the screen, use the drop-down menu to select Excel Add-ins, then click the Go button.

e. Check the box next to Analysis ToolPak, then click the Ok button.

Data Analysis ToolPak will be added to Microsoft Excel. You can add under the Data tab.

2. Enter Your Standard and UUT Data

a. Enter your Nominal Values into column X
Now that Data Analysis ToolPak is added to Microsoft Excel, pick a column and enter your nominal or standard values. You want to use all of test-points calibrated for the measurement range you are evaluating linearity uncertainty.

b. Enter Your Actual Values into column Y
Next, select another column and enter the calibration results for the unit under test (UUT).

3. Perform Regression Analysis

a. Open Data Analysis ToolPak
Now, we are going to put Data Analysis ToolPak to work. Click on the Data tab. Look at the right-side of the toolbar and click on the Data Analysis button.

b. Select Regression Analysis
A new window will open with a list of analyses. Scroll down and select Regression Analysis. Then, click the Ok button.

c. Select Column Y
A new window will open that requires you to enter information needed to perform regression analysis. In the Input section, find the Input Y Range cell and click the button to the right of the cell.

Select all of the cells that contain the UUT calibration results.

d. Select Column X
Find the Input X Range cell and click the button to the right of the cell.

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

Proficiency testing is an important element of ISO/IEC 17025 accreditation. So important, that it has its own ISO standard (i.e. ISO/IEC 17043).

In addition to the international standard, proficiency testing has plenty of endorsement via ILAC and accreditation body policies. There is a ton of information on proficiency testing available to laboratories seeking accreditation.

So, why do so many laboratories struggle with proficiency testing?

Most of the labs that I have spoken with either have a hard time finding a proficiency testing provider or have a unique test or calibration where proficiency testing is not available.

To get accredited, you need to know;
• all of the requirements that you need to meet beyond the ISO/IEC 17025,
• how to find a proficiency testing provider,
• how to understand your proficiency testing results,
• what to do if you fail a proficiency test.

In this guide, I am going to cover everything that you need to know about proficiency testing for ISO/IEC 17025 accreditation including the answers to your problems listed above.

So, grab something to drink and take a seat, I am about to give you a lot of information.

Background

Personally, I have never had much of a problem meeting proficiency testing requirements.

Even with a 30+ page scope of accreditation, most of the measurement functions were able to be covered using a single proficiency testing provider.

The only measurement function that I had a hard time finding a PT provider was for chemical gas concentration.

Otherwise, I have only had to organize two interlaboratory comparisons in my career.

Looking at my experience, it seems that proficiency testing is a pretty easy requirement to meet. However, many labs seem to have a hard time.

So, what is the deal?

Is it a lack of knowledge, training, available information, or time? Whatever the reason, it appears that Proficiency Testing is a problem for many accredited labs.

On average, I probably encounter 3 clients a month who need help with proficiency testing and 20 or more subscribers who have questions.

Therefore, I decided to create a Proficiency Testing Guide to help you and others who struggle to meet ISO/IEC 17025 requirements.

In this guide, you will learn;

If you would like to jump ahead to a particular section, just use the links provided above. Otherwise, take a seat. We have a lot of information to cover.

What is Proficiency Testing

According to ISO/IEC 17043:2010, proficiency testing (PT) is the evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons.

In other words, a proficiency test is a method used to demonstrate competency and validate a laboratory’s measurement process by comparing your results to the results of a reference laboratory and other participant laboratories.

To get a better idea of what a proficiency test is, look at the image below.

A coordinating body sends a test item or artifact to a reference laboratory for testing. Then, the coordinating body sends the item to each participating laboratory for subsequent testing.

Each participant laboratory will independently test the item, submit their results to the coordinating body, and forward the item to the next participating laboratory.

After each participating laboratory has completed testing, the artifact is returned to the coordinating body.

The coordinating body will evaluate the all the test results and issue a performance report to each participating laboratory.

This is typically referred to as Round Robin Testing, and is one of the most common proficiency testing schemes used by PT providers.

Why is Proficiency Testing Important

Proficiency Testing is important for several reasons. Primarily, it enables your laboratory to demonstrate competency for a particular measurement discipline which can be used to validate;
• A measurement method;
• Technical training of personnel;
• Traceability of standards, and
• Estimates of measurement uncertainty.

Imagine that your laboratory is adding a new measurement or testing capability to your scope of accreditation. You have purchased equipment, had it calibrated, written methods, trained personnel, estimated uncertainty, and performed numerous internal verification studies.

Even with all of that hard work, time, and money invested into your new process, are you confident that your results are adequate and comparable to other laboratories?

By participating in proficiency testing, your laboratory can externally validate your new measurement or testing process.

Hence, the reason your accreditation body requires you to successfully complete a proficiency test before they will add it to your scope of accreditation.

We will cover more about that later in this guide.

Proficiency Testing vs Interlaboratory Comparisons

Proficiency testing and inter-laboratory comparisons are terms used synonymously in the Test and Measurement industry. However, they are not exactly the same.

They are similar, but slightly different.

According to ISO/IEC 17043:2010, inter-laboratory comparison (ILC) is the organization, performance, and evaluation of measurements or tests on the same or similar items by two or more laboratories or inspection bodies in accordance with predetermined conditions.

According to ISO Guide 43, Proficiency Testing is a formal exercise managed by a coordinating body which includes a standard or reference laboratory. The results are issued in a formal report that clearly provides the En and Z score.

Furthermore, ISO Guide 43 describes an inter-laboratory comparison as an exercise that is performed by agreement between two or more participating laboratories where the results are issued in a formal report.

So, the difference is a proficiency test is an inter-laboratory comparison that is organized and managed by an independent third party. Additionally, a proficiency test includes the participation of a reference laboratory and uses their results to determine participant performance.

An inter-laboratory comparison does not require the use of a reference laboratory or a coordinating body. Therefore, participant laboratories are only comparing performance amongst the group of participating members.

As you can see, they are similar, but different.

Proficiency Testing Schemes

When it comes to proficiency testing, there are a few different schemes used to conduct interlaboratory comparisons.

Each scheme is unique to maintain homogeneity and stability of the artifact throughout the testing process. Otherwise, the test results could contain errors and become invalid for use.
In this section, I will show you the two most common proficiency testing schemes recommended by the ISO/IEC 17043:2010;
• Sequential Participation Schemes
• Simultaneous Participation Schemes

If you conduct interlaboratory comparisons without the use of a proficiency testing provider, these test schemes may be of value to you.

Sequential Participation Schemes

In a sequential participation scheme, artifacts are successively circulated from one participant to the next, or occasionally returned to the proficiency testing provider or reference laboratory for retesting.

Sequential participation schemes are very common in proficiency testing. Two of the most common sequential designs are;
• Ring Test
• Petal Test

The ring test (i.e. round-robin test) is a proficiency testing scheme where a reference laboratory initially measures an artifact (to establish a reference) and then successively submits the artifact to each participant laboratory.

This ring test is typically used for artifacts known to have better long term stability.

The petal test is a proficiency testing scheme where a pivot laboratory is used to measure an artifact more than once during testing. In some cases, artifacts are returned to the pivot laboratory before and after each shipment to a participant laboratory.

The petal test is typically used for artifacts with short-term stability or when the participants are national metrology institutes (NMI’s).

Simultaneous Participation Schemes

In a simultaneous participation scheme, sub-samples are randomly selected from a material source and simultaneously distributed to participant laboratories for concurrent testing.

Simultaneous participation schemes are very common in proficiency testing and typically used for reference materials or single use samples that are destroyed or discarded after testing.

Three of the most common simultaneous testing designs are;
• Split-Level Test,
• Split-Sample Test, and
• Partial-Process Test.

In a split-level proficiency testing scheme, similar (but not identical) levels of a measurand are incorporated into two separate proficiency test items.

In a split-sample testing scheme, material or product samples are split into two or more parts, where each participant only test one part of the sample.

In a partial-process scheme, participants only perform specific parts of the overall testing or measurement process.

How to Evaluate Proficiency Testing Results

If you are going to participate in proficiency testing or inter-laboratory comparisons, it is beneficial for you to know how to evaluate your testing results.

This is especially true if you need to perform an interlaboratory comparison without the aid of a proficiency testing provider.

Furthermore, it is always a good idea to double-check your PT results even if you are using a proficiency testing provider.

I have found mistakes in proficiency testing reports before.

One time, a provider issued a report to my accreditation body indicating that I had an unsatisfactory result. However, when I double-checked the calculation of En, I discovered that I was well within.

So, I contacted the PT provider and notified them of my findings. When they double-checked the calculations, they discovered that there was a mistake. As a result, the proficiency testing report was updated and A2LA retracted their discrepancy letter.

Every once in a while it is nice to get a lucky break. However, you will never know if you do not verify your proficiency testing results. Therefore, let’s check out some of the methods used by proficiency testing providers.

Proficiency testing results are commonly evaluated using two methods described in ISO/IEC 17043;

1. Normalized Error, and
2. Z-Score.

Normalized Error

Normalized error is a statistical evaluation used to compare proficiency testing results between the participant and the reference laboratory where the uncertainty in the measurement result is included.

Typically, it is the first evaluation used to determine conformance or nonconformance (i.e. Satisfactory/Unsatisfactory) in proficiency testing.

When determining whether a participant’s results are satisfactory or unsatisfactory, the following rules are used;

• When the value of |En| ≤ 1 (i.e. between -1 and +1), the results are considered satisfactory.

• When the value of |En| > 1 (i.e. greater than +1 or less than -1), the results are considered unsatisfactory.

To calculate normalized error, use the equation provided below;

If you are having a hard time understanding the equation above, use the step-by-step instructions below to calculate normalized error (i.e. En);

1. Subtract the result from the participating laboratory by the result of the reference laboratory (i.e. laboratory bias).
2. Calculate the root sum of squares for both laboratories’ reported estimates of measurement uncertainty.
3. Divide the value calculated in step 1 and by the value calculated in step 2.
Z-Score

Z-score is a statistical measurement of a score’s relationship (i.e. how many standard deviations above or below the population mean) to the mean in a set of scores.

It is a statistical evaluation used to review the results of all participants and identify outliers and exclude their data from proficiency testing results.

When determining whether a participant’s results are satisfactory, unsatisfactory, or questionable, the following rules are used;
• When the value of Z <=2, the results are considered satisfactory.
• When the value of Z >=3, the results are considered unsatisfactory.
• When the value of Z >=2 and Z <=3, the results are considered questionable.

To calculate z-score, use the equation provided below;

If you are having a hard time understanding the equation above, use the step-by-step instructions below to calculate z-score;

1. Subtract the participant laboratory’s result by the population mean (i.e. average).
2. Calculate the standard deviation of all participant results.
3. Divide the result of step 1 by the result of step 2.

Proficiency Testing Standards

The International Organization for Standardization (ISO) has a standard for just about everything, including proficiency testing.

If you are interested in becoming an accredited proficiency testing provider or want to learn more about proficiency testing standards, check out the list of active and obsolete ISO standards provided below.

Current and Active Standards Obsolete and Withdrawn Standards

ISO/IEC 17025 Requirements For Proficiency Testing

The new ISO/IEC 17025:2017 standard requires laboratories to participate in proficiency testing. In the 2005 standard, proficiency testing was recommended in section 5.9.1b, but not required.

Now, it is required (where available and appropriate). Therefore, I recommend that you develop and implement a proficiency testing program if you do not have one in place.

To familiarize yourself with the ISO/IEC 17025:2017 standard, let’s take a look at all of the sections that mention proficiency testing so you can be better prepared for accreditation.

Proficiency Testing Definition

In section 3.5, the ISO/IEC 17025 standard defines the term “proficiency testing.” According to the standard, proficiency testing is the evaluation of participant performance against pre-established criteria by means of interlaboratory comparison.

The definition comes from the proficiency testing standard, ISO/IEC 17043:2010.

See the excerpt below:

3 Terms and definitions
3.5 proficiency testing
evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons (3.3)

[SOURCE: ISO/IEC 17043:2010, 3.7, modified — Notes to entry have been deleted.]”

Ensuring the Validity of Results

In section 7.7.2, the ISO/IEC 17025 standard states that laboratories shall monitor their performance by comparing their results with other laboratories.

The two methods that are recommended are:

a. Proficiency Testing or
b. Interlaboratory Comparisons

See the excerpt below:

7.7 Ensuring the validity of results
7.7.2 The laboratory shall monitor its performance by comparison with results of other laboratories, where available and appropriate. This monitoring shall be planned and reviewed and shall include, but not be limited to, either or both of the following:

a) participation in proficiency testing;

NOTE – ISO/IEC 17043 contains additional information on proficiency tests and proficiency testing providers. Proficiency testing providers that meet the requirements of ISO/IEC 17043 are considered to be competent.

b) participation in interlaboratory comparisons other than proficiency testing.”

Externally Provided Products and Services

In section 6.6.1, the ISO/IEC 17025 standard states that laboratories must use only suitable externally provided services when it affect laboratory activities. If you read the note below the section, you will see that proficiency testing services should be included in externally provided services.

If you maintain an Approved Supplier List (like many other labs), then you may want to add your proficiency testing providers to it.

See the excerpt below:

6.6 Externally provided products and services
6.6.1 The laboratory shall ensure that only suitable externally provided products and services that affect laboratory activities are used, when such products and..

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

Each time that you begin an uncertainty analysis, you need to create an outline before you start to estimate uncertainty.

I call this process specifying your measurement function. It is the first step in my 7 Steps to Estimating Measurement Uncertainty Guide. However, it is simply creating an outline for your uncertainty analysis.

The goal of this process is to help you focus on estimating uncertainty for one measurement function and one measurement range at a time. Ultimately, it will help you save time and avoid mistakes.

When you know where to focus your attention, it is easier to start estimating uncertainty and finish the process.

Similar to other laboratory activities, you need a procedure to complete a test or calibration. However, you need to know which procedure to use. If you outline what you are testing or calibrating, the process becomes much easier.

Uncertainty Analysis: Create An Outline

Each time you perform an uncertainty analysis you should create an outline by performing the following steps;

1. Select the measurement function,
2. Select the measurement range,
3. Select the test points,
4. Select the method,
5. Select the equipment, and
6. Find the mathematical equation or formula.

In the sections below, you will learn how to perform each of these steps in more detail.

1. Select the Measurement Function

The first thing that you want to do when starting to estimate uncertainty is to identify what you are doing.

Therefore, you need to specify your measurement function.

Just ask yourself, “What am I doing?”

Think about the process that you are performing and the results that you will record. How would you describe them?

For example, maybe you are;

• Measuring DC voltage with a digital multimeter,
• Generating DC voltage with a multifunction calibrator,
• Measuring temperature with a thermometer,
• Measuring frequency with a counter,
• Calibrating pipettes with a precision balance,

There are many different activities that your laboratory could perform. You just need to briefly describe what you are doing.

Then, you will summarize the process in just a few words. Using the list above, some common names that you could use would be;

• DC Voltage Measure,
• DC Voltage Generate,
• Temperature Measure,
• Frequency Measure,
• Volume Measure

Additionally, you can describe what you are testing or calibrating. For example, you can specify your measurement function as;

• Scales,
• Pipettes,
• PRT Thermometers,
• Gage Blocks,
• Flowmeters

For inspiration, you should look at other laboratory’s scopes of accreditation to see what they are naming measurement functions.

This will ensure that you are using function names and descriptions that have been approved by your accreditation body.

Another good piece of advice would be to use names and descriptions that your customers will recognize.

Therefore, start every uncertainty analysis by creating an outline and specifying your measurement function so you know where to focus the attention.

2. Select the Measurement Range

The next thing that you want to do is select your measurement range.

This will help you determine the range of values that your uncertainty analysis will cover so your focus is kept between these two values.

For example, your measurement functions could have one of the following ranges;

• (0 to 100) V,
• (0 to 250) °C,
• (1 to 100) Hz,
• (100 to 1,000) µL,
• (0.1 to 1) in

If you are having a hard time determining the measurement range, try looking the equipment manufacturer’s specifications. You should be able to find this information in manufacturer’s manuals and datasheets.

Typically, the specification sheets will list all of the ranges of the equipment’s measurement capability.

To determine the equipment’s measurement range;

1. Find the smallest value of the range,
2. Find the largest value of the range, and
3. List of the range from smallest to largest.

Look at the image above, you should be able to identify the following ranges;

• (0 to 200) mV
• (0 to 2) V
• (0 to 20) V
• (0 to 200) V
• (0 to 1000) V

Alternatively, you can list the measurement ranges similar to how they will be presented in your scope of accreditation;

• (0 to 200) mV
• (0.2 to 2) V
• (2 to 20) V
• (20 to 200) V
• (200 to 1000) V

In this example, you are listing your measurement ranges where they are typically used and crossover to the next range.

The key takeaway is to make sure that you cover your system’s measurement capability.

However, use common sense when listing your measurement range. Your estimated uncertainty in measurement may not adequately represent equipment’s measurement capability below 5% of the range.

For example, your measurement equipment may not perform well at 0.1% of the measurement range in comparison to 100% of the range.

Typically, measurement equipment is recommended to be used from 10% or 20% of the measurement range to 100% of the range. Therefore, using equipment at 1% or less of the measurement range may not be desirable.

Make sure that you read manufacturer’s specifications and instructions.

3. Select the Test Points

Now that you have identified your measurement function and range, it is time to select the test points that you will use to estimate measurement uncertainty.

To select test points for estimating measurement uncertainty;

1. Select a test point at the low end of the range,
2. Select a test point at the high end of the range, and
3. If necessary, select a test point in the middle of the range.

As a minimum, you will need to select two test points, one low and one high.

It is quite common to select test points at 10% and 100% of the range or 20% and 100% of the range.

The best practice is to select test points where your equipment is calibrated. Take a look at your calibration reports and find two points per range (i.e. one low and one high).

If your equipment is not calibrated at two points per measurement range, then you may need to estimate uncertainty at your cross-over points. This is where one measurement range overlaps with the next measurement range.

For example, a digital multimeter has five measurement ranges (e.g. 0.1V, 1V, 10V, 100V, and 1,000V) and is only calibrated at one point per range.

In this situation, it is best to estimate uncertainty at each calibration point and use the data from the previous range to calculate CMC uncertainty equations.

To estimate uncertainty for the 10V range, you would use the uncertainty at 10V and at 1V to develop a cmc uncertainty equation. This would allow you to estimate uncertainty at two points across the measurement range by using crossover or overlapping test points.

4. Select the Method or Procedure

To estimate measurement uncertainty for a test or calibration process, you need to know the process.

Therefore, you need to select the method or procedure that will be used to perform the process.

With the method, you can review the process to find sources of uncertainty from the;

• List of recommended equipment,
• Recommended environmental conditions,
• Steps of the process,
• Equations used to obtain results, and/or
• Precautionary notes to avoid errors.

5. Select the Test Equipment

You should select your test equipment before beginning an uncertainty analysis. This will ensure that you choose the right equipment to adequately estimate uncertainty in measurement.

It is a good idea to select your best equipment or the equipment that you would typically use.

Selecting inferior or substandard equipment can significantly affect your uncertainty analysis results.

If your goal is to report less uncertainty and you are measuring DC Voltage, selecting a Keysight 3458A multimeter may be a better choice than a 34401A multimeter.

If you are measuring length and have similar goals, selecting a grade 1 gauge block may be a better choice than a grade 3 gauge block.

So, make sure to pick the right test equipment for your uncertainty analysis.

6. Find the Mathematical Equation or Formula

Another task that you should perform is finding the mathematical equation or formula that represents your measurement process.

It is a step that many people often overlook when estimating uncertainty. However, using the mathematical equation can significantly help you.

When available, I recommend that you use the mathematical equation when estimating uncertainty in measurement.

Mathematical equations are like a map. They can;

• Give you sources of uncertainty,
• Reduce the time you spend conducting research,
• Ensure that you are appropriately estimating uncertainty.

With an equation, you should be able to find what components contribute to measurement uncertainty and determine how each variable contributes to the uncertainty of the test or measurement result.

Additionally, you can use the equation to perform simulations and a calculate sensitivity coefficients.

I highly recommend that you use equations when they are available.

For example, if you are estimating uncertainty for a dead weight tester, you can use the mathematical equation below.

As you can see, this equation has a lot of variables. However, if you quantify the uncertainty for each variable, you can estimate uncertainty for the calculated pressure.

Furthermore, you do not have to worry (as much) about understating uncertainty because you have considered all of the sources of uncertainty included in the equation.

So, make sure to spend some time to find and use the mathematical equation (if available).

Information Needed for Uncertainty Analysis

After establishing what you will be estimating uncertainty for, you need to prepare yourself to begin a new uncertainty analysis.

Before you start, you need to collect information and data that will help you estimate measurement uncertainty.

I recommend that you gather the following items listed below;

This should not be a difficult task. You should have all of these items readily available to you with little or no research.

The purpose of this task is to give you a process that will help you prepare to estimate uncertainty in measurement.

Uncertainty Analysis Example

Instead of ending this guide here, I want to give you some examples of this process so you can see how to implement it when you perform uncertainty analysis.

DC Voltage Measure with a Fluke 8508A

For this example, imagine that you want to estimate uncertainty for your Fluke 8508A digital multimeter.

To outline your uncertainty analysis, you need to:

1. Identify the measurement function,
2. Identify the measurement range,
3. Identify the test points,
4. Identify the method,
5. Identify the equipment,
1. Identify the measurement function

Which measurement function of the multimeter will you be using?

Additionally, you will want to whether you are sourcing, generating, or measuring.

For this example, you will be measuring DC Voltage. Therefore, your measurement function would be title: DC Voltage Measure.

2. Identify the Measurement Range

Which measurement range do you want to evaluate?

Even though you want to estimate uncertainty each range, you should only pick one at a time.

For this example, let’s pick the 200mV range.

To specify the measurement range from beginning to end, identify the lowest and highest points of the range.

Some examples that you can use may include;

• Up to 200mV
• 1mV to 200mV
• 0.000001mV to 200mV

However, be realistic. List the measurement range that you actually intend to use.

3. Identify the Test Points

What test points will you use to estimate uncertainty?

At a minimum, pick two test points across the range. I would recommend one low test point and one high test point.

Look at your calibration reports. I would recommend using test points that have calibration results. It will be much easier to estimate uncertainty at these points.

For this example, the multimeter is only calibrated at 100mV. Therefore, I would estimate uncertainty at 0mV and 100mV.

Otherwise, I recommend that you pick test points that are close to 10% of the range and 100% of the range.

For the next range (i.e. 2V), I would use your uncertainty at 100mV for the low test point and estimate uncertainty at 1V for your high test point.

4. Identify the Method

Which method do you use to measure DC Voltage?

Select the method that best represents your typical workload or laboratory activities.

For this example, the manufacturer’s calibration procedure was chosen for measuring DC Voltage from a multifunction calibrator.

5. Identify the Equipment

What equipment will you use?

Select the equipment that will provide your best measurement capability or your most common measurement capability.

I recommend that you select equipment that will provide your best measurement capability because you cannot report uncertainty that is smaller than advertised in your scope of accreditation.

For this example, a Fluke 8508A Multimeter was selected since it provides the best measurement capability for our hypothetical laboratory.

Additionally, I have selected a multifunction calibrator to act as the unit under test since it will help provide smaller uncertainties for repeatability and reproducibility.

If I had selected to use a power supply as my UUT, the results of repeatability and reproducibility testing would have been much larger which would lead to a larger uncertainty estimate.

So, make sure to select equipment (i.e. STD and UUT) with the best measurement capability. It will help achieve smaller uncertainties.

Now, that you have completed the previous five steps, you can record the results in your uncertainty budgets.

I recommend that you list the following information;

1. Measurement function,
2. Equipment or system description,
3. Equipment identification or serial numbers,
4. Measurement range, and
5. Test-point

Take a look at the image below to see an example. This is the way that I present information in every uncertainty budget that I create.

It gives you a great outline for your uncertainty analysis.

Using this format provides all of the important information that you or your assessor need to understand what section of your scope of..

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

Reporting measurement uncertainty in test and calibration certificates is a common practice for ISO/IEC 17025 accredited laboratories. It is also a common problem for a lot of laboratories.

Many struggle to do it right.

Most accredited laboratories are required to include uncertainty in their certificates. However, many of them do not know how to report uncertainty correctly.

Some of the most common deficiencies laboratories get cited for are;

• Not reporting uncertainty in certificates,
• Not reporting uncertainty correctly,
• Not reporting uncertainty to two significant figures,
• Not rounding uncertainty correctly,
• Reporting uncertainty smaller than their scope of accreditation.

Many of these deficiencies are easily avoidable. However, so much focus is put into learning how to estimate measurement uncertainty that many forget to learn how to report uncertainty.

The good news is the rules for reporting uncertainty have become much clearer over the last 10 years; and, today, we are going to cover them all!

In this guide, you will learn the requirements, the process, and the skills to report uncertainty in measurement.

Plus, I am going to give you plenty of examples from major laboratories that you can use to report uncertainty in your own certificates.

We are going to cover;

If you have questions about reporting measurement uncertainty in your certificates, then you are going to like this guide.

Let’s get started.

Requirements For Reporting Uncertainty

Whether you know it or not, there are rules to reporting uncertainty in your test and calibration certificates.

However, these rules have not always been around.

In the beginning, the GUM only provided recommendations, not rules. These recommendations were not heavily enforced, so laboratories used a variety of methods to report uncertainty.

Within the last decade, the recommendations from the GUM have started to become published in policies and standards.

As a result, accreditation bodies are starting to enforce these requirements. This means that you should become familiar with them and make sure they are implemented in your test or calibration certificates.

Therefore, we are going to cover the recommendations, policies, and requirements published in;

JCGM 100:2008 1. Reporting Expanded Measurement Uncertainty

In section 7.2.3, the GUM states that you should include the following information when reporting the expanded measurement uncertainty;

• A full description the of the measurand Y,
• The measurement result, the measurement uncertainty, and the units of measure,
• Include the relative expanded uncertainty (e.g. percent) when appropriate,
• Give the value of the coverage factor (k),
• Give the confidence level associated with the reported uncertainty,
• Give a copy of your uncertainty budget or refer to a document that contains it (see sections 7.2.7 and 7.1.4).
2. Reporting the Measurement Results and Measurement Uncertainty

In section 7.2.4, the GUM states that you should report measurement results, for maximum clarity, similar to the example below;

3. Round Uncertainty to Two Significant Digits

In section 7.2.6, the GUM states that you should not report an excessive number of digits and recommends to round measurement uncertainty to two significant figures.

ILAC P14:01/2013 1. Reporting Uncertainty in Calibration Reports

In section 6.1, the ILAC P14 policy states that a calibration laboratory should report uncertainty;

• In the calibration report,
• With the measurement result

Additionally, section 6.1 provides exceptions to reporting uncertainty as long as you obtain an agreement with the customer and meet the listed criteria.

2. Requirements for Calibration Certificates

In section 6.2, the policy states that you should report the measured value and the measurement uncertainty together, and include the associated units of measure for both.

Optionally, you can present the results in a table or provide the measurement uncertainty as relative expanded uncertainty (e.g. percent).

Finally, you will need to add a note or statement explaining the coverage factor and coverage probability of your reported measurement uncertainty values.

3. Rounding Uncertainty for Calibration Certificates

In section 6.3, the policy states that you should round measurement uncertainty to two significant figures and use the rounding method provided in section 7 of the GUM.

4. Uncertainty Components for Calibration Uncertainty

In section 6.4, the policy states that you include the following sources of uncertainty when estimating measurement uncertainty for your calibration reports;

• CMC Uncertainty,
• UUT Resolution, and
• UUT Repeatability.
5. Do Not Report Uncertainty Smaller Than Your CMC Uncertainty

In section 6.5, the policy states that you should not report measurement uncertainty values that are smaller or less than the CMC Uncertainty listed in your scope of accreditation.

ISO/IEC 17025:2017

Overall, the requirements in the ISO/IEC 17025 standard for reporting uncertainty in measurement are very straightforward.

1. Reporting Measurement Uncertainty in Test Reports

In section 7.8.3.1c, the ISO/IEC 17025:2017 states that you should report uncertainty in your test reports when;

• It is relevant to the validity of the test results,
• A customer requests it, or
• It affects conformity to a specification limit.

Additionally, when you report uncertainty, it should be reported in the same unit of measurement as the result or in a unit relative (e.g percent) to the result.

2. Reporting Measurement Uncertainty in Calibration Reports

In section 7.8.4.1a, the ISO/IEC 17025:2017 states that you should report uncertainty in your calibration reports;

• In the same unit as the measurement result, or
• In a term relative to the measurement result (e.g. percent).

Unlike test reports, the standard does not give you the option to omit reporting uncertainty. Therefore, you should always report uncertainty in calibration reports.

3. Reporting Measurement Uncertainty When Reporting Sampling

In section 7.8.5f, the ISO/IEC 17025:2017 states that you should include information needed to evaluate measurement uncertainty in subsequent tests or calibrations.

How to Report Measurement Uncertainty

When reporting uncertainty in measurement, follow this five-step process;

1. Record the measurement result
2. Estimate the uncertainty in measurement
3. Round uncertainty to two significant figures
4. Round the measurement result to match the uncertainty
5. Report the results
6. Include an uncertainty statement
1. Record the measurement result

The first step to reporting uncertainty is to know the value of the measurement result. Therefore, you must start the process by performing a measurement and recording the result.

Using the example in the GUM, imagine that measure the resistance of a resistor and find it’s value to be 10.05762 Ohms.

Record the resistance value as the measurement result.

2. Estimate uncertainty in measurement

After recording your measurement result, you can estimate the uncertainty in measurement.

Following the recommendations of the ILAC P14, estimate your uncertainty in measurement including these three factors;

• CMC Uncertainty
• UUT Resolution
• UUT Repeatability

After combining these three factors and calculating the expanded uncertainty to 95% where k=2, you should have a value for calibration uncertainty.

3. Round uncertainty to two significant figures

Now that uncertainty has been estimated for the measurement result, it is time to round uncertainty to two significant figures.

To do this, find your first two significant figures. Then, use conventional rounding to round up or down to the nearest number.

4. Round the result to match the uncertainty

Next, round the measurement result to be consistent with the measurement uncertainty.

Since the estimated uncertainty does not give you enough accuracy to justify the value of the measurement result, round the measurement result to match the accuracy of the uncertainty.

There is no point in reporting a measurement result beyond the number of digits given in the estimated uncertainty. So, just round your measurement result to match the number of digits given in your estimated uncertainty.

5. Report the results

Report the results in your test or calibration certificate.

There several ways that you can report uncertainty in your test or calibration reports. You can report the results;

• Alongside your test or measurement results,
• In a sentence or statement, or
• In a table or uncertainty budget.

Choose the method that works best for you.

6. Include an uncertainty statement

Finally, provide a statement that explains how your customers should interpret the reported measurement uncertainty.

Your certificate must include a statement that gives;

• The coverage factor, and
• The coverage probability.

To give you an idea, here is a general statement that you could use in your certificates.

“Reported uncertainties were estimated in accordance with the [insert method here] expressed to a 95% confidence interval where k=2.”

Significant Figures

Significant figures is an important concept that confuses many people. Therefore, I am going to try to simplify it for you.

Significant Figures Definition

According to the Oxford Advanced Learner’s Dictionary, significant figures are each of the digits of a number that are used to express it to the required degree of accuracy, starting from the first nonzero digit.

Now, the definition may not make sense to you at first. However, it will become much clearer after you review the rules for determining significant figures.

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

INTRODUCTION

Performing a repeatability test is an essential part of estimating uncertainty in measurement. It is the most common experiment performed to collect Type A Uncertainty data.

Additionally, most accreditation bodies require that you include repeatability data in your uncertainty budgets. However, many people have trouble with performing a repeatability test. They do not know how to conduct the experiment, collect the data, or analyze the results.

In this guide, you are going to learn everything that you need to know about repeatability testing;

1. What is a repeatability test,
2. How to perform a repeatability test, and
3. How to calculate repeatability.

Plus, I have created some great tools to help you perform a repeatability test and analyze the results next time you need to estimate measurement uncertainty.

BACKGROUND

Every day, I work with clients who need my help to estimate uncertainty in measurement. During the process, I typically ask clients to perform repeatability and reproducibility testing.

However, many of my clients do not know how to conduct a repeatability test. So, they ask for procedures, checklists, and consultation.

After years of continually helping clients perform a repeatability test, I noticed that I had not written a formal guide to help these clients; nor automate the process.

Therefore, I decided to create a guide dedicated to repeatability testing in order to answer all of the questions that I have been asked.

In this guide you will learn;

WHAT IS TYPE A UNCERTAINTY

Type A Uncertainty is a component of uncertainty where data is collected from a series of observations and evaluated using statistical methods associated with the analysis of variance (ANOVA).

Commonly referred to as Type A Data, Type A Uncertainty is typically associated with the results of Repeatability and Reproducibility Testing. However, it can also be associated with stability testing.

According to the Vocabulary in International Metrology (VIM), Type A Evaluation of measurement uncertainty is a component of measurement uncertainty evaluated by a statistical analysis of measured quantity values that were obtained under defined measurement conditions.

WHAT IS A REPEATABILITY TEST

A repeatability test is an experiment performed to evaluate how repeatable your results are under a set of similar conditions.

When performing a repeatability test, you will want to collect data using the;

1. Same method,
2. Same operator,
3. Same equipment,
4. Same environmental conditions,
5. Same location, and
6. Same item or unit under test.

Essentially, you want to collect repeatable results over a short period of time without changing anything (if possible).

According to the Vocabulary in International Metrology (VIM), measurement repeatability is measurement precision under a set of repeatable conditions of measurement.

Furthermore, the VIM defines a repeatability condition of measurement as a condition of measurement, out of a set of conditions that includes the same measurement procedure, same operators, same measuring system, same operating conditions, same location, and same replicate measurement on the same or similar objects over a short period of time.

Therefore, you need to define your measurement conditions and collect repeatable results over a short period time so you can evaluate the precision of your process.

In the next section, you will learn step by step how to perform a repeatability test.

HOW TO PERFORM A REPEATABILITY TEST

To calculate repeatability, you need to have a procedure.

Similar to every test or measurement performed in your laboratory, you must have a method or procedure to guide you through the process and ensure consistency in your results.

In this section, you will learn to perform a repeatability test step by step. Follow the instructions below to add repeatability test data to your uncertainty budgets.

Here is a list of the steps in this process;

1. Select the measurement function to test,
2. Select the measurement range,
3. Select the test-point(s),
4. Select the method,
5. Select the equipment,
6. Select the operator,
7. Perform the test,
8. Collect the number n of repeated samples,
10. Save a record of your results (recommended),

1. Select the Measurement Function

Before you begin performing a repeatability test, it is a good idea to determine what you are going to test.

Start by selecting the measurement function that will be tested.

The measurement function will be the category that best describes your measurement or test result, such as;

1. DC Voltage Generate/Measure
2. Length
3. Pressure Generate/Measure
4. Torque Generate/Measure
5. Temperature Source/Measure

If you are having trouble, take a look at your scope of accreditation (or another laboratory’s scope of accreditation) and pick the measurement function that you would like to test.

2. Select the Measurement Range

After selecting the measurement function, pick a measurement range to test. This should consist of a chosen starting measurement value and an ending measurement value; typically low to high.

I recommend that you pick a measurement range listed in your scope of accreditation or in the equipment manufacturer’s specifications.

3. Select the Test Points

Now that you have specified a measurement range, it is time to select the test-points for your repeatability test.

If you assume that your measurement function is linear, you will need to select two test-points along the measurement range. Typically, it should be a low value and a high value.

Some common practices are to select test points that are at 10% and 90% of the measurement range or at 20% and 100% of the measurement range. For best results, I recommend that you select two calibration points along the measurement range (if available).

If you assume that your measurement function is non-linear, you may want to select three of more test points to evaluate. This will help you prevent errors due to the curvature of the measurement function.

Should you decide to perform a repeatability test at three of more test points, try to select evenly spaced test points to prevent modeling errors in your CMC Uncertainty prediction equation.

4. Select the Method

The next step to performing a repeatability test is to select the measurement method or procedure. You will want to select a method or procedure that best represents the how the measurement process is performed.

A good place to start is to use a test method or calibration procedure that best represents your measurement process. This will help you make sure that you are evaluating a measurement process that you would normally perform in your laboratory.

If you do not have a procedure, try writing one for your process. Even if the procedure minimally covers the steps of the process, it is still better than nothing. Additionally, it will help you get consistent results.

5. Select the Equipment

Following the chosen method, select the equipment recommended to perform your measurement process. Make sure that your equipment is calibrated and functioning properly before use.

For best results, select the most accurate equipment available to you. The equipment you choose will affect the results. So, select your best equipment.

6. Select the Operator

Select an operator to perform the repeatability test. Pick an operator that is qualified and experienced at performing the test or measurement.

Typically, your most experienced or qualified operators will yield the best results.

Your goal should be to achieve consistent repeatable results. Therefore, choose an operator that you will help you achieve it.

7. Perform the Repeatability Test

Now that you have established all of your conditions, it is time to perform a repeatability test for the measurement function, range, and test-points that you selected using the method, equipment, and operator that you selected.

Perform steps eight through eleven to conduct a repeatability test.

8. Collect ‘n’ Number of Samples and Record Your Results

When performing a repeatability test, collect a defined number of repeated samples. Typically, it is recommended that you collect at least 20 to 30 samples to obtain statistically significant results.

However, collecting 20 to 30 samples is not always practical for every test or measurement. Instead, collect the number of samples that is most appropriate for your situation and measurement system.

If you can only collect 5 samples because the test or measurement process is time-consuming or labor-intensive, then only collect 5 samples. If you can collect 100 samples because your test or measurement process is quick and automated, then collect 100 samples.

Only collect the number of samples that is most appropriate for your situation.

After collecting samples, you will need to analyze your data using the analysis of variance (ANOVA). Calculate the mean (i.e. average), standard deviation, and the degrees of freedom.

If you are analyzing a single set of data, you will use the calculated standard deviation and degrees of freedom in your uncertainty budget.

If you are analyzing multiple sets of data, you will need to use the method of pooled variance to calculate the pooled standard deviation and degrees of freedom for your uncertainty budget.

10. Save a Record of Your Results (Recommended)

Anytime that you collect data, it is a good idea to save a record of your results. It will come in handy if you need to go back and review your results or compare them with other repeatability test data.

I always recommend that you keep files for your repeatability test data. However, you do not have to keep records. Instead, you can just collect Type A data each time you update your uncertainty budgets. The choice is yours.

Finally, add your repeatability test results to your uncertainty budgets. Create a line item for repeatability and include the standard deviation and the degrees of freedom in your budget.

Characterize your repeatability results with a Normal distribution where ‘k’ equals one (i.e. k=1).

For step by step instructions, read the section below: How to Add Repeatability to Your Uncertainty Budget.

HOW MANY SAMPLES SHOULD YOU COLLECT

A common question people ask when performing a repeatability test is, “How many samples should I collect?”

The answer is, “As many as you practically can.”

As a general rule of thumb, it is typically recommended to collect 20 to 30 samples to be statistically sound. However, this rule is not applicable to every scenario.

If you are using automation and have the capability to collect 100 or more samples over a short period of time, then collect 100 or more samples. It is practical for your test or measurement process.

If you are performing a test or measurement that is difficult or time-consuming, it may be hard to collect 20 to 30 samples. Therefore, you should collect fewer samples. In this situation, it may be more practical to only collect three to five samples.

Make sure to select the number of samples that is appropriate for your measurement process.

If you would like to manipulate your results to achieve a desired margin of error (i.e. standard deviation), use the formula below;

How to Calculate
1. Choose your desired confidence level (z).
2. Choose your desired margin of error (MOE).
3. Multiply the result of step 1 by the value by the standard deviation of the sample set.
4. Divide the result by the margin of error selected in step 2.
5. Square the result calculated in step 4.

HOW TO COLLECT REPEATED SAMPLES

When performing a repeatability test, some people get confused on how to collect repeated samples. They believe that they must;

1. Set-up a test,
2. Collect a result,
3. Break-down the setup, and
4. Repeat the process ‘n’ number of times.

This is not true.

Conducting a repeatability test following that process would be rigorous and time-consuming.

Instead, think about how you could collect the data easier and faster if you were to follow this process;

1. Set-up a test,
2. Collect a result,
3. Repeat ‘n’ number of times, and
4. Break-down the setup.

If you follow this process, you would be able to complete a repeatability test much faster. So, to make repeatability testing less rigorous, make sure to collect repeated samples back to back over a short period of time.

If your measurement equipment repeatedly samples results and refreshes the display, collect ‘n’ number of displayed results back to back over a short period of time.

Should your measurement equipment require manual sampling, repeat the process over and over again until you collect your desired number of samples.

Do not break down your test set-up each sample and repeat. That process would actually be a form of reproducibility testing, not repeatability testing.

HOW TO CALCULATE REPEATABILITY (SINGLE TEST)

Analyzing the results of a single repeatability test is pretty simple. Just calculate the mean, standard deviation, degrees of freedom.

1. Calculate the Mean,
2. Calculate the Standard Deviation,
3. Calculate the Degrees of Freedom.

In the sections below, you will learn how to perform these calculations in Excel.

1. Calculate the Mean using Excel

How to Calculate
1. Select a cell to calculate the mean.
2. Type “=AVERAGE(“ in the cell.
3. Select the cells containing your results.
4. Type “)“ and press the “Enter” key.

2. Calculate the Standard Deviation using Excel

How to Calculate
1. Select a cell to calculate the standard deviation.
2. Type “=STDEV(“ in the cell.
3. Select the cells containing your results.
4. Type “)“ and press the “Enter” key.

3. Calculate the Degrees of Freedom using Excel

How to Calculate
1. Select a cell to calculate the degrees of..

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

Ever wondered what factors are significant contributors to uncertainty in measurement?

Are you performing an uncertainty analysis and wondering what components to include in your uncertainty budget?

Maybe an assessor cited you a deficiency and recommended that you include more sources of uncertainty in your budgets. Afterward, they provided you with a laundry list of significant contributors.

Perhaps, you wanted to reduce your measurement uncertainty by targeting your largest contributors.

If the answer to any of the previous statements was “Yes,” then this guide is for you.

Today, I am going to show you how to find significant contributors by calculating the value of a metric I call “significance.”

Plus, you will learn how to add this calculation to your uncertainty budget in Microsoft Excel, so you automate the calculation every time you estimate uncertainty.

Background

Several years ago, I was at the A2LA Annual Meeting in Maryland discussing measurement uncertainty and significant contributors with a group of technical experts (e.g. metrologists, assessors, and consultants).

One of the topics of conversation was about evaluating uncertainty components to find out how much they contribute to measurement uncertainty.

Dilip Shah, President at E=mc3 Solutions, mentioned a method for determining the amount of influence an uncertainty component contributes to the total uncertainty.

Specifically, I remember him stating that you cannot calculate a ratio of standard deviations. He said that you must convert the standard deviations to variances before calculating the ratios.

Now, I remember learning about this method in college, but never in relation to uncertainty analysis. However, when Dilip mentioned it, I knew that it would be a great tool to evaluate and validate my estimations of measurement uncertainty.

So, I added the calculation to my uncertainty budget templates that night at the hotel. I wanted to test the function to see what I could learn from using it in my uncertainty calculator.

After a few weeks, I realized that it was extremely valuable. It allowed me to quickly;

• find significant contributors,
• find negligible contributors,
• evaluate my results, and
• find errors.

Having this data in my uncertainty budgets helped me validate my results faster. At this point, I couldn’t calculate uncertainty without it.

Therefore, I added it to every uncertainty calculator that I created. Today, I continue to use it and recommend that you give a try.

In this guide, you will learn everything that you need to know about calculating significance and finding significant contributors to measurement uncertainty. I will even show you how to add it your uncertainty budgets.

Here is a list of topics that will be covered in this guide;

What is a Significant Contributor

A significant contributor is a source of uncertainty in measurement that increases the CMC Uncertainty by five percent or more.

According to A2LA, a it is “a contributor whose contribution increases the CMC by five percent (5%) or greater.”

See the excerpt from the A2LA R205 Publication below;

“Significant (A2LA): “significant” further means a contributor whose contribution increases the CMC by five percent (5%) or greater.”

Now, significant contributor is not an official term defined by the Vocabulary in Metrology; but, it is a term used repetitively in several key documents;

• The GUM states, “… significant component to uncertainty of the measurement result.”
• The ISO/IEC 17025:2005 standard states, “… major sources of uncertainty.”
• The ISO/IEC 17025:2017 standard states, “… all contributions that are of significance.”

However, none of these documents define “what is” a significant contributor or establish requirements to determine whether or not a component is a significant contributor to uncertainty.

Therefore, I prefer to refer to the definition provided in the A2LA R205 document. It is the only document to establish requirements for significance.

If laboratories’ uncertainty budgets are going to be assessed for the inclusion of significant contributors, it would be best to rely on requirements rather than opinions.

When a technical expert is allowed to make a decision based on subjective opinions rather than requirements and facts, there will be problems.

Now that you know what a significant contributor is, you may be wondering how to find them in your uncertainty analysis.

Well, it is pretty easy. You just need to calculate a parameter that I call “significance.”

What is Significance

Significance is a proportion, in percentage, of the total uncertainty that a component contributes to the CMC Uncertainty.

While this is not an actual statistical term, it is a term that I have used to describe the magnitude of influence when evaluating significant contributors to uncertainty in measurement.

However, it is statistical method derived from the analysis of variance (ANOVA). Specifically, it is based on the analysis of a proportion of variance or a proportion of total variation.

I love this method!

I use it all of the time to evaluate my uncertainty analyses. In fact, I include it in every uncertainty budget calculator that I use.

It is really helpful for evaluating uncertainty budgets to find;

• The most significant contributors,
• The least significant contributors,
• Negligible contributors, and
• Errors.

How to Calculate Significance

Calculating the significance of an uncertainty component is not difficult. The process can be completed in only four steps.

To calculate significance, convert your uncertainty components from standard deviations to variances. Next, calculate the sum of squares of all uncertainty components. Then, calculate the ratio of one uncertainty component to the total sum of squares of all the uncertainty components.

Equation

Look at the equation below to calculate significance.

Instructions

To calculate significance, just follow these step-by-step-instructions;

1. Select an uncertainty component,
2. Square the standard uncertainty component to convert it to a variance,
3. Calculate the Sum of Squares for all uncertainty components,
4. Divide the result in Step 2 by the result in Step 3.

Example

To show you how to calculate significance, take a look at the example below.

Imagine that you have 3 uncertainty components;

• CMC Uncertainty,
• UUT Resolution, and
• UUT Repeatability.

The value of each component is provided below as a standard uncertainty;

• CMC Uncertainty: 0.16mV
• UUT Resolution: 0.577mV
• UUT Repeatability: 0.55mV

Now, let’s see how much the UUT Resolution contributed to the total combined uncertainty.

In case you want to work out the equation for yourself, the significance of each uncertainty component is listed below;

• CMC Uncertainty: 3.9%
• UUT Resolution: 50.4%
• UUT Repeatability: 45.7%

How to Calculate Significance in Excel

Calculating significance can be performed using Microsoft Excel. It is a fast and easy way to evaluate your estimates of uncertainty in measurement.

If you use Excel to calculate uncertainty, you can easily add this function to your uncertainty calculator. It will automatically calculate the significance of uncertainty components each time you estimate uncertainty.

In the steps below, you will learn how to add this function to your uncertainty calculator.

1. Square The Standard Uncertainty Component

To calculate the significance of an uncertainty component, you must first square the value of the standard uncertainty component. This will convert the standard deviation to variance of the uncertainty component.

2. Calculate The Sum of Squares For All Standard Uncertainty Components

Next, you will calculate the sum of squares for all uncertainty components.

Essentially, you will convert each uncertainty component to a variance and add them all together.

In Microsoft Excel, you will use the sum of squares function, or ‘SUMSQ,’

3. Divide The Result In Step 1 By The Result In Step 2

Now, divide the functions given in step 1 by the function in step 2. Your function should look similar to the example below;

=Cell1^2/SUMSQ(Cell2:Cell3)

Where,
Cell1 = standard uncertainty component
Cell2 = First standard uncertainty component
Cell3 = Last standard uncertainty component

Every time you calculate significance, it is important to double-check your work and validate your results.

Validating your significance calculations is pretty easy. Just add all of your significance calculations together. The result should be 100%.

If your results do not add up to 100%, then you have a problem and must go back to check your equation for errors.

In the example given earlier, significance was calculated for three uncertainty components. Their values were 3.9%, 50.4%, and 45.7%.

When added together, their sum equals 100%.

Since the values equal 100%, the calculation is validated to be correct.

In fact, I add the function to every one of my uncertainty budgets. It allows me to quickly evaluate my uncertainty calculations.

In this section, I am going to show you how to add this function to your uncertainty calculator; if you use Microsoft Excel. The process is easy. Just follow the steps listed below;

1. Pick An Empty Column Next To Your Uncertainty Budget

Find an empty column next to your uncertainty budget or insert a new column in your uncertainty budget. I prefer to use the last column to the right of the uncertainty calculator.

This column will be used to calculate the significance of your uncertainty components.

2. Add The Following Excel Function Into a Cell For Each Uncertainty Component

Select the cell that is on the same row as your first uncertainty component and add the Excel function in the image below.

Next, repeat the process and add the function to each row that contains an uncertainty component.

3. Convert The Results To Percentage

Next, you want to convert the values to a percentage. It will help you compare and evaluate results.

Select the cells that contain your significance calculations and click on the percentage button in ‘Home’ tab.

You can also press Ctrl+Shift+% simultaneously to convert the values to percentage.

Afterward, your results should look similar to those in the image below.

Then, add any necessary formatting to help it blend in with your uncertainty budget or calculator.

5. Calculate The Sum Of The Values

6. Verify The Result Is 100%

Finally, verify the sum of all significance calculation is 100 percent. If it is, you have validated that your calculations are correct.

If the result does not..

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

Type A and Type B uncertainty are two elements that are commonly discussed in estimating measurement uncertainty.

Uncertainty type is covered in most measurement uncertainty guides and uncertainty training courses. Auditors review uncertainty budgets to make sure the components are categorized correctly.

However, have you ever looked at most of the information published on Type A and Type B uncertainty?

It’s very minimal. No one covers the topic of uncertainty type as well as the GUM. There is so much information left out of other guides and training.

It might be the reason why most people only evaluate type B uncertainty with a rectangular distribution when there are so many more realistic options.

Why are other options omitted?

In this guide, I am going to teach you all about Type A and Type B uncertainty as explained in the GUM. However, I am going explain in a manner that doesn’t require you to have a PhD.

So, if you want learn how to calculate uncertainty, make sure to read this guide to learn everything you need to know about Type A and Type B uncertainty.

Background

Before you learn about uncertainty type classifications, it’s a good idea to know more about why they exist and where they came from.

In 1980, the CIPM Recommendation INC-1 suggested that measurement uncertainty components should be grouped into two categories; Type A and Type B.

Below is an exert from the Vocabulary in Metrology;

“In the CIPM Recommendation INC-1 (1980) on the Statement of Uncertainties, it is suggested that the components of measurement uncertainty should be grouped into two categories, Type A and Type B, according to whether they were evaluated by statistical methods or otherwise, and that they be combined to yield a variance according to the rules of mathematical probability theory by also treating the Type B components in terms of variances. The resulting standard deviation is an expression of a measurement uncertainty. A view of the Uncertainty Approach was detailed in the Guide to the expression of uncertainty in measurement (GUM) (1993, corrected and reprinted in 1995) that focused on the mathematical treatment of measurement uncertainty through an explicit measurement model under the assumption that the measurand can be characterized by an essentially unique value. Moreover, in the GUM as well as in IEC documents, guidance is provided on the Uncertainty Approach in the case of a single reading of a calibrated instrument, a situation normally met in industrial metrology.” – VIM 2012

As you can see, the VIM gives a great explanation and recommends that you read the GUM for more details.

Here is an exert from the Guide to the Expression of Uncertainty in Measurement;

“3.3.4 The purpose of the Type A and Type B classification is to indicate the two different ways of evaluating uncertainty components and is for convenience of discussion only; the classification is not meant to indicate that there is any difference in the nature of the components resulting from the two types of evaluation. Both types of evaluation are based on probability distributions (C.2.3), and the uncertainty components resulting from either type are quantified by variances or standard deviations.” – JCGM 100

For more information on the CIPM recommendation INC-1 (1980), go to iso.org. The text is in French but can be easily translated with tools like Google Translate.

Now that you have read the VIM and the GUM, you can understand that the use of uncertainty types (i.e. A & B) are to help you quickly determine how the data was evaluated.

If you continue to read the GUM, it will teach the difference between Type A and Type B uncertainty. See the excerpt below.

“3.3.5 The estimated variance u2 characterizing an uncertainty component obtained from a Type A evaluation is calculated from series of repeated observations and is the familiar statistically estimated variance s2 (see 4.2). The estimated standard deviation (C.2.12, C.2.21, C.3.3) u, the positive square root of u2, is thus u = s and for convenience is sometimes called a Type A standard uncertainty. For an uncertainty component obtained from a Type B evaluation, the estimated variance u2 is evaluated using available knowledge (see 4.3), and the estimated standard deviation u is sometimes called a Type B standard uncertainty.” – JCGM 100

From the excerpt above, you can determine two things;
• Type A uncertainty is calculated from a series of observations,
• Type B uncertainty is evaluated using available information.

Furthermore, the GUM provides you with information about the probability distributions for each uncertainty type.

“Thus a Type A standard uncertainty is obtained from a probability density function (C.2.5) derived from an observed frequency distribution (C.2.18), while a Type B standard uncertainty is obtained from an assumed probability density function based on the degree of belief that an event will occur [often called subjective probability (C.2.1)]. Both approaches employ recognized interpretations of probability.” – JCGM 100

Type A uncertainty is characterized by the observed frequency distribution which means that you should look at the histogram to find the correct probability distribution.

Following the Central Limit Theorem, the more samples that you collect, the more the data will begin to resemble a normal distribution. Here is a link to an amazing video on the Central Limit Theorem. I recommend that you watch it.

On the other hand, Type B uncertainty is characterized using an assumed probability distribution based on available information. Without the original data or a histogram, you are left to determine how the data is characterized based on your information sources.

Most of the time, you are not given much information. Therefore, people typically assume a rectangular distribution.

However, there are plenty of other ways for you to evaluate Type B uncertainty data that no one ever references; not even in the best guides to estimating uncertainty.

Today, I am going to cover everything that you need to know about Type A and Type B uncertainty. Look at the list below to see what is covered in this guide.

1. What is Type A Uncertainty
2. Evaluation of Type A Uncertainty
3. Examples of Evaluating Type A Uncertainty
4. What is Type B Uncertainty
5. Evaluation of Type B Uncertainty
6. Examples of Evaluating Type B Uncertainty
7. Difference Between Type A and Type B Uncertainty
8. How to Choose Type A or Type B

What is Type A Uncertainty

According to the Vocabulary in Metrology (VIM), Type A Uncertainty is the “evaluation of a component of measurement uncertainty by a statistical analysis of measured quantity values obtained under defined measurement conditions.”

In the Guide to the Expression of Uncertainty in Measurement (GUM), Type A evaluation of uncertainty is defined as the method of evaluation of uncertainty by the statistical analysis of series of observations.

Essentially, Type A Uncertainty is data collected from a series of observations and evaluated using statistical methods associated with the analysis of variance (ANOVA).

So, if you collect repeated samples of similar measurement results and evaluate it by calculating the mean, standard deviation, and degrees of freedom, your uncertainty component would be classified as Type A uncertainty.

Evaluation of Type A Uncertainty

For most cases, the best way to evaluate Type A uncertainty data is by calculating the;

• Arithmetic Mean,
• Standard Deviation, and
• Degrees of Freedom

Arithmetic Mean

When performing a series of repeated measurements, you will want to know the average value of your sample set.

This is where the arithmetic mean equation can help you evaluate Type A uncertainty. You can use the value later to predict the expected value of future measurement results.

Definition
The central number of set of numbers that is calculated by adding quantities together and then dividing the total number of quantities.

Equation

How to Calculate
1. Add all the values together.
2. Count the number of values.
3. Divide step 1 by step 2.

Standard Deviation

When performing a series of repeated measurements, you will also want to know the average variance of your sample set.

Here, you will want to calculate the standard deviation. It is most common Type A evaluation used in uncertainty analysis.

So, if there were only one function to learn, this would be the one to focus your attention on.

Definition
A measure of the dispersion of a set of data from its mean (i.e. average).

Equation

How to Calculate
1. Subtract each value from the mean.
2. Square each value in step 1.
3. Add all of the values from step 2.
4. Count the number of values and Subtract it by 1.
5. Divide step 3 by step 4.
6. Calculate the Square Root of step 5.

Degrees of Freedom

After calculating the mean and standard deviation, you need to determine the degrees of freedom associated with your sample set.

It is an important value that most people neglect to calculate. Even most guides on measurement uncertainty forget to include it in their text. However, the GUM does not forget to mention it.

In fact, in section 4.2.6, the GUM recommends that you should always include the degrees of freedom when documenting Type A uncertainty evaluations.

I agree.

I always include the degrees of freedom when evaluating Type A data and in my uncertainty budgets.

You can also use it to estimate confidence intervals and coverage factors.

Definition
The number of values in the final calculation of a statistic that are free to vary.

Equation

How to Calculate
1. Count the number of values in the sample set.
2. Subtract the value in step 1 by 1.

Example of Evaluating Type A Uncertainty

To give you an example of evaluating Type A uncertainty data, I am going to show you two common scenarios people encounter when estimating measurement uncertainty.

• Single Repeatability Test, and
• Multiple Repeatability Tests

Single Repeatability Test

Imagine you are estimating uncertainty in measurement and need to obtain some Type A data. So, you perform a repeatability test and collect a series of repeated measurements.

Now that you have collected data, you need to evaluate it. Therefore, you calculate the mean, standard deviation, and the degrees of freedom.

Next, you add the standard deviation and degrees of freedom to your uncertainty budget for repeatability.

Multiple Repeatability Tests

In this scenario, let’s imagine you are estimating measurement uncertainty for a measurement system that is critical to your laboratory. Try to think of a reference standard that you own.

It is so important that you perform a repeatability test for this system every month and document the results.

Your records have the mean, standard deviation, and degrees of freedom listed for each month.

With so much Type A data, you are probably wondering, “Which results do I include in my uncertainty budget?”

The answer is all of them; or, at least, the last twelve months.

To evaluate your Type A uncertainty data, you will want to use the method of pooled variance. It is the best way to combine or pool your standard deviations.

After performing this analysis, you will want to the pooled standard deviation to your uncertainty budget for repeatability.

What is Type B Uncertainty

According to the Vocabulary in Metrology (VIM), Type B Uncertainty is the “evaluation of a component of measurement uncertainty determined by means other than a Type A evaluation of measurement uncertainty.”

In the Guide to the Expression of Uncertainty in Measurement (GUM), Type B evaluation of uncertainty is defined as the method of evaluation of uncertainty by means other than the statistical analysis of series of observations.

Essentially, Type B Uncertainty is data collected from anything other than an experiment performed by you.

Even if you can analyze the data statistically, it is not Type A data if you did not collect it from a series of observations.

Most of the Type B data that you will use to estimate uncertainty will come from;

• Calibration reports,
• Proficiency testing reports,
• Manufacturer’s manuals,
• Datasheets,
• Standard methods,
• Calibration procedures,
• Journal articles,
• Conference papers,
• White papers,
• Industry guides,
• Textbooks, and
• Other available information.

Evaluation of Type B Uncertainty

Since Type B Uncertainty can come from so many different sources, there are a lot ways that it can be evaluated.

This means that there is a lot of information to cover in this section.

Most of the time, people default to assigning a rectangular distribution to an uncertainty component and using a square root of three divisor to convert quantities to standard uncertainty.

If this describes how you evaluate uncertainty in measurement, go ahead and raise your hand.

The good news is that this will work for 90% of the uncertainty calculations that you will perform in your lifetime. However, there are many more realistic options available for you to use to evaluate Type B uncertainty.

It depends whether or not you want use them or not.

If you are interested, keep reading. I am going to cover the evaluation methods in the GUM that most measurement uncertainty guides tend to leave out.

“It should be recognized that a Type B evaluation of standard uncertainty can be as reliable as a Type A evaluation” Manufacture Specifications & Calibration Reports

In section 4.3.3 of the GUM, the guide gives recommendations for evaluating information published in manufacturer’s specifications and calibration reports.

“4.3.3 If the estimate xi is taken from a manufacturer’s specification, calibration certificate, handbook, or other source and its quoted uncertainty is stated to be a particular multiple of a standard deviation, the standard uncertainty u(xi) is simply the quoted value divided by the multiplier, and the estimated variance u2(xi) is the square of that quotient.”

Additionally, in section 4.3.4 of the GUM, the guide gives you more information for evaluating manufacture specifications.

“4.3.4 The quoted uncertainty of xi is not necessarily given as a multiple of a standard deviation as in 4.3.3. Instead, one may find it stated that the quoted uncertainty defines an interval having a 90, 95, or 99 percent level of confidence (see 6.2.2). Unless otherwise indicated, one may assume that a normal distribution (C.2.14) was used to calculate the quoted uncertainty, and recover the standard uncertainty of xi by dividing the quoted uncertainty by the appropriate factor for the normal distribution. The factors corresponding to the above three levels of confidence are 1,64; 1,96; and 2,58 (see also Table G.1 in Annex G).”

If the uncertainty is reported to a particular confidence interval (e.g. 95%), use the associated coverage factor to convert to standard uncertainty.

In the image below is an excerpt from the Fluke 5700A datasheet. You should notice that the specifications are stated for both 95% and 99% confidence intervals.

Therefore, to find standard uncertainty, simply divided published uncertainty by the coverage factor (k) that associated with the confidence interval given.

If the confidence level is not provided in the specifications (most of the time it is not provided), it is best to assume that it is given to a 95% confidence interval. Only assume a 99% confidence interval if it is stated.

PRO TIP: Next time your auditor suggests that you should evaluate manufacturer’s accuracy or uncertainty specifications with a rectangular distribution, please refer them to read sections 4.3.3 and 4.3.4 of the GUM.

50/50 Chance of Occurrence

In section 4.3.5 of the GUM, the guide tells you how to evaluate type B uncertainty when you believe that there is a 50% chance of occurrence. The guide recommends that you divide the interval by 1.48.

Therefore, you would use the following equation to convert to standard uncertainty.

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

A laboratory’s scope of accreditation is a key element of ISO/IEC 17025 accreditation and a vital asset to your customers. It lists all of the activities that your laboratory is accredited perform.

However, many laboratories forget to consider its importance when getting accredited or reaccredited. They just rush to develop a scope of their capabilities and submit it to their accreditation body.

Instead, you need to make sure to develop a scope of accreditation that is functional for both your laboratory and customers.

It is a great marketing tool that make an impact your business.

When a customer looks at your scope, they want to know one or more of the following;

1. Is your laboratory ISO/IEC 17025 accredited,
2. Can you calibrate/test “x,”
3. What is your measurement uncertainty. and(or)
4. What methods are you using to calibrate/test “x?”

If your scope of accreditation does not answer these questions, you may have a problem that could be driving away potential customers.

In this guide, I am going to show you a simple five step process that will help you develop an amazing scope of accreditation for your laboratory. Plus, I have included plenty of important links to valuable resources that will help you meet ISO/IEC 17025 requirements.

So, if you are looking to create or update your scope of accreditation, let’s get started.

Background

When I took over a calibration program for an accredited laboratory, I remember reviewing and updating the scope of accreditation for the first time.

It wasn’t difficult. Most the work had already been done for me.

However, when I needed to add new capabilities, I always had a tough time figuring out the right terminology and format to use.

So, I read the policies, requirements, and guides published by my accreditation body. Yet, I still struggled to get it right. Most of the guides were great for knowing the rules, but did not provide exact match examples that met my needs.

Therefore, I started to look at other laboratories’ scopes of accreditation for inspiration.

It was great! I was able to review several laboratory scopes that were similar to mine and replicate the format that I thought would fit my laboratory’s needs.

But meeting the needs of my laboratory was not enough. I had to learn the hard way that our scope of accreditation was not helping our current and prospective customers.

At the time, we were receiving a ton of phone calls each week with questions about our calibration capabilities. We were helping customers by answering their questions, but we were wasting our valuable time and not pushing out quotes fast enough (because we were constantly on the phone).

Now, most people would recommend that we hire more personnel to handle the volume of work. However, I disagree.

We did not have a personnel problem, we had a communication problem. We were not answering our customers’ questions with our scope of accreditation. Therefore, they had to call us to get answers to their questions.

Luckily, they were interested enough in our services to pick up the phone and call!

How many potential customers do you think we were losing because our scope did not answer their questions?

I am not certain; but, I bet it was a lot!

I am confident that the majority of potential customers looked at our scope, did not find the answer they were looking for, and moved on to look at other laboratories’ scopes of accreditation.
Bummer! Another customer lost.

Then, it dawned on me. I needed to design and format my scope to meet my customers’ needs. So, we spent time documenting customer questions and identifying the questions that we the most repetitive.

With this information, I decided to change the terminology used in our scope of accreditation to meet our customers’ needs (i.e. answering questions).

As a result, we were able to reduce the amount of phone calls related to questions and respond quicker to customer inquiries for quotes.

The content of our telephone conversations changed from “Can you calibrate X” to “Can I get a quote for calibration of X.” Furthermore, we noticed a significant bump in the number of requests for quotes (both via telephone and email).

With this in mind, what are potential customers seeing when they look at your scope of accreditation; Do they find what they are looking for or are they moving on to another laboratory’s scope of accreditation?

Whether you are creating your first scope of accreditation or considering the need to update your current scope, I hope that you find this guide helpful.

In this guide, you will learn;

What is a Scope of Accreditation

According to the ILAC G18, a scope of accreditation is the official and detailed statement of activities for which the laboratory is accredited.

Basically, it is an official list of tests and/or calibrations that your laboratory is accredited to perform.

Look at the image below. It is an excerpt of Fluke’s Everett Service Center’s scope of accreditation. If you notice, the scope of accreditation lists the laboratory’s;

• Name,
• Location,
• Technical or Quality Manager,
• Contact information,
• Certificate number,
• Expiration date, and
• List of accredited activities.

Most laboratories use their scope of accreditation as a marketing tool to showcase their capabilities to current and prospective customers. If you plan to do the same, it is important to ensure that your scope is accurate and up to date.

To provide your customers accredited calibration and test results, the activities must be listed in your scope. If not, you must report the results as non-accredited regardless of capability.

Testing Labs vs Calibration Labs

Every ISO/IEC 17025 accredited laboratory has a scope of accreditation. However, there are slight differences in the presentation of information depending on the type of laboratory.

The scope of accreditation for a testing laboratory is not formatted the same way as a calibration laboratory.

According to the ILAC G18, the scope of accreditation must include:

Testing Laboratories – The tests or types of tests performed and materials or products tested and, where appropriate, the methods used.

Calibration Laboratories – the calibrations, including the types of measurements performed, the Calibration and Measurement Capability (CMC) or equivalent.

Look at the image below. You will see a table from the ILAC G18 that lists the typical information included in the scope of accreditation based on laboratory type.

Calibration laboratory scopes typically list the measurement function, range, and CMC Uncertainty for each activity that the lab is accredited to perform. This makes it easy to find and compare laboratory measurement capability and quality.

Testing laboratory scopes typically list only the test activities and methods which they are accredited to perform. Only few testing scopes contain quantitative information about capability and uncertainty.

As a result, the majority of testing laboratory scopes of accreditation do not include statements or estimates of measurement uncertainty. This makes it difficult for customers to compare the capability and quality of testing laboratories.

In an industry of buyer beware, the scope of accreditation should allow customers to confidently find and select laboratories that will meet their requirements; and, provide customers the reassurance that they are conducting business with a laboratory that is ISO/IEC 17025 accredited.

Guides For Creating A Scope of Accreditation

If you are going to create or update your laboratory scope of accreditation, you should make sure that you are meeting ISO/IEC 17025 requirements. Luckily, there are plenty of guides available to help you.

In this section, you will find a list of guides that will help you develop a scope of accreditation.

The first guide that you should read is the ILAC G18. You can access it using the link provided below.

How to Create A Scope Of Accreditation

Now, that you have some good background information on the scope of accreditation, it is time to get to work and start making one for your laboratory.

All you need to do is follow the five step process listed below to develop your laboratory scope;

1. Contact your Accreditation Body and Get the Draft Template.
3. Research other laboratory scopes with similar capabilities.
4. Enter your data into the Scope of Accreditation template.

Using this simple five-step process will help you make an amazing laboratory scope for ISO/IEC 17025 accreditation that will benefit both your laboratory and your customers.

1. Contact Your Accreditation Body and Get the Draft Template.

To create a scope of accreditation, you will need a template. If you have selected an accreditation body and are in the application or reaccreditation process, contact your accreditation body and request a template for your Draft Scope of Accreditation.

The easiest way to do this is send an email to your accreditation officer requesting the template. Typically, they will send it to you within 24 hours. However, if you do not receive the template within 24 hours, follow up with a telephone call.

Before you begin completing your draft scope of accreditation, read your accreditation body’s policies and requirements. Each accreditation body has their own preferences for how they like scope to be designed. It will help you create your scope using the correct terminology and format.

Additionally, make sure to read your accreditation body’s guides. They will provide you with insight and guidance to prevent you from making mistakes.

To help you out, I have provided a list of accreditation requirements documents below. Find your accreditation body in the list below and click on the link to access the document(s) relevant to your laboratory.

3. Research Other Laboratory Scopes With Similar Capabilities.

A great trick to developing your scope of accreditation is to get inspiration from other accredited laboratories. Just look at their scope of accreditation to see how they are listing their activities.

The best part is the information is available online and easy to find. Just search your accreditation body’s search directory. You should be able to find plenty of examples from other laboratories that perform similar activities.

However, do not just look at one scope of accreditation. Try reviewing at least three to five scopes to see how other laboratories list their activities. If you have time, look at ten.

Pay attention to the terminology they use and how they present information. Find examples that you like and replicate it in your scope of accreditation.

While this may seem like a lot of extra work. I promise that it will save you time in the long run. When you are not sure what to do, you can spend a lot of time thinking about how to present the information. Not good!

Always remember that the scope of accreditation should be targeted toward your customers. So, make sure that it answers their questions about your laboratory’s capability.

PRO-TIP: Think of words and copy that will resonate best with your customers. After all, your scope of accreditation is a marketing document. Make sure it designed help customer quickly find what they are looking for!

4. Enter Your Data Into The Scope of Accreditation Template.

Be sure to focus on one discipline or measurement function at a time. If you do not, it will be easy to get disorganized and make mistakes. This is especially true if you are developing a large scope with a lot of activities.

With all of the work and stress that you will have during an ISO/IEC 17025 audit, the last thing you want to do is stay at work late fixing mistakes in your scope of accreditation.

The worst mistake that you can make is to forget to add a measurement function entirely. Unless you want to pay for an additional audit, you will have to wait until next time to add new activities to your scope.

So, make sure to pay attention and focus on one measurement function or parameter at a time.

a. Measurement Function / Parameter / Equipment

This column defines the measurement functions and disciplines that you are accredited to perform.

You can list functions such as;

• Length,
• Voltage,
• Pressure, and/or
• Temperature.

Or, you can use copy that specifically narrows down your capability to a niche, such as;

• Micrometers.
• Multimeters,
• Pressure Gauges, and/or
• Liquid-in-Glass Thermometers

As you can see, you have some flexibility. However, you will want to check you accreditation body’s requirements to see what type of terminology they will allow.

If you are having trouble deciding, choose the option that will best guide your customer to requesting a quote for calibration services.

b. Measurement Range

This column defines the range of your measurement function.

Try to think of the minimum and maximum values of you measurement capability. Then, add this information to your scope.

Make sure that you read your accreditation body’s policies, requirements, and guides. Some accreditation bodies will not allow you to use zero (i.e. 0) in your measurement range.

Instead, they may prefer that you use;

• the minimum resolution of the measurement range,
• the phrase “up to” the max value of the range, or
• the minimum value that you would test, measure, etc.

c. CMC Uncertainty

This column is used to define the Calibration and Measurement Capability of your measurement function at a specific measurement range.

Therefore, you will need to list your estimates of uncertainty in measurement for each measurement range included in your scope of accreditation.

• A single value,
• A range of values,
• An equation (typically a linear equation),
• A matrix of values, or
• A graph

d. Comments / Notes / Remarks

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

Estimating uncertainty can be a difficult task, especially if you are a beginner. Tools like an uncertainty budget or calculator can really make a big difference.

However, software to calculate uncertainty is typically outdated, expensive, and difficult to learn.

Many people have problems because they do not know what to do or where to begin.

Luckily, there is an easier way!

You don’t need expensive software to estimate uncertainty. All you need is a program named Microsoft Excel.

In fact, you probably have Microsoft Office on your computer. Therefore, the cost for you to begin to calculate uncertainty with Excel is zero.

In this guide, I am going to show you how to create an uncertainty budget in Excel so you can estimate uncertainty in measurement for ISO/IEC 17025 accreditation.

So, keep reading. I am going to show you how to;

1. Create an uncertainty budget in Excel,
2. Add functions to automatically calculate uncertainty, and
3. Validate that your uncertainty calculator functions correctly.

Background

When I first began to estimate uncertainty, I spent months buying and trying all kinds of software that was supposed to help me calculate uncertainty.

It was awful!

I wasted many hours searching for software, buying it, downloading it, installing it, and testing it only to find out that it didn’t calculate uncertainty like I thought it would.

Finally, I stumbled across the Hewlett Packard’s UnCal 3.2. It was the best program (at the time) that helped me estimate uncertainty. Best of all, it was free!

I used it to prepare for an ISO/IEC 17025 accreditation audit where I had to estimate uncertainty for the laboratory’s entire 25-page scope of accreditation.

At the time, I inherited a metrology program that had based all of it’s uncertainty statements on manufacturer’s specifications.

Most of the scope of accreditation was not supported with any uncertainty budgets. The few parameters that did have uncertainty budgets were awful and didn’t make much sense.

Every measurement parameter needed an uncertainty budget and I had to start from scratch!

The worst part was, I had three months to get it done and I didn’t have a clue what I was doing.

Needless to say, I got hit hard during the assessment. Most of deficiencies were related to missing uncertainty budgets.

Nearly 70% of my scope of accreditation was marked “TBD.”

It was embarrassing!

As a result, I spent the next 6 months creating uncertainty budgets and living off of scope extensions. It was not fun.

Luckily, A2LA agreed to work with me throughout the process while I created uncertainty budgets for my entire scope of accreditation.

After spending 9 months of my life calculating uncertainty, here is what I learned;

• I learned a lot about estimating uncertainty,
• I learned a lot about my calibration laboratory,
• I hated measurement uncertainty software, and
• I learned Microsoft Excel was great for calculating uncertainty.

Since then, I have used Microsoft Excel (almost exclusively) to create uncertainty budgets for estimating uncertainty.

So, if you are interested in using Excel to calculate uncertainty in measurement, let me show you how to create a powerful uncertainty budget calculator.

What Is An Uncertainty Budget

According the Vocabulary in Metrology (VIM), an uncertainty budget is a statement of a measurement uncertainty, and of their calculation and combination.

Essentially, it is a document that describes how you estimated uncertainty in measurement including the components and calculations.

For a better understanding, read the note below the definition. It states that an uncertainty budget should include the following information;

1. Measurement model
2. Estimates,
3. Uncertainties for quantities in the measurement model,
4. Covariances,
5. Probability distributions,
6. Degrees of freedom,
7. Type of Evaluation (i.e. Uncertainty Type), and
8. Coverage factor

Take a look at section 7.1.4 of The Guide to the Expression of Uncertainty in Measurement (GUM). It offers some great advice about creating uncertainty budgets. The last phrase is my favorite!

7.1.4 Although in practice the amount of information necessary to document a measurement result depends on its intended use, the basic principle of what is required remains unchanged: when reporting the result of a measurement and its uncertainty, it is preferable to err on the side of providing too much information rather than too little. For example, one should

a) describe clearly the methods used to calculate the measurement result and its uncertainty from the experimental observations and input data;

b) list all uncertainty components and document fully how they were evaluated;

c) present the data analysis in such a way that each of its important steps can be readily followed and the calculation of the reported result can be independently repeated if necessary;

d) give all corrections and constants used in the analysis and their sources.

A test of the foregoing list is to ask oneself “Have I provided enough information in a sufficiently clear manner that my result can be updated in the future if new information or data become available?”

Therefore, to create a great uncertainty budget, make sure that you include enough information to repeat the process in the future.

I recommend that you include as much information as you can in your uncertainty budgets. You can always remove information that is useless or unnecessary later.

It’s more difficult to add important information after the analysis has been completed. Proportionally, the more time that elapses since the analysis, the more difficult it will become to recall how you estimated measurement uncertainty.

Take it from me. It is rather embarrassing when an assessor asks you to explain your uncertainty analysis and you cannot remember how you achieved the results.

Leaving out important details and information about your uncertainty calculations is only setting yourself up for failure.

So, make sure to provide plenty of information in your uncertainty budgets and update them routinely (e.g. every 12 to 24 months) to recall how you estimated uncertainty.

If you need help, I have included a section in this guide that will show you what factors I include in my uncertainty budgets.

Why Is It Important

Uncertainty budgets are important because they provide detailed information for how you estimate uncertainty.

Additionally, uncertainty budgets are important if you want your laboratory to be ISO/IEC 17025 accredited.

As an accredited laboratory, you are required to estimate uncertainty for the test and measurement functions your organization will be accredited to perform.

An uncertainty budget is the tool that you will use to estimate measurement uncertainty to support your scope of accreditation. Without it, you will have a difficult time getting accredited.

Therefore, you need an uncertainty budget. It will help you convey to others exactly how you estimated uncertainty.

If you want to make the process easier, you will want to make sure that your uncertainty budget can function as an uncertainty calculator. It will save you a lot time that can be better spent elsewhere.

What To Include In An Uncertainty Budget

To calculate uncertainty effectively, you should consider including the following elements in your uncertainty budgets.

1. Definition of Measurand
2. Uncertainty Source or Component,
3. Sensitivity Coefficient,
4. Uncertainty Value,
5. Unit of Measure,
6. Uncertainty Type
7. Probability Distribution,
8. Divisor,
9. Standard Uncertainty,
10. Degrees of Freedom,
11. Significance or Influence on Total Combined Uncertainty,
12. Combined Uncertainty,
13. Total Effective Degrees of Freedom,
14. Coverage Factor,
15. Expanded Uncertainty, and

I know that this may seem like a lot of information; but, it is important for explaining how you estimated uncertainty. Plus, you can use some of the additional information to help you reduce measurement uncertainty.

In the example below, you will see how I typically include these elements into my uncertainty budgets.

DISCLAIMER: The uncertainty budget examples shown in this guide are for reference only and do not represent an actual measurement uncertainty analysis for the identified item.

Uncertainty Budget Design & Format

While there are several formats that can be used, most people tend to use a table format to demonstrate how they calculate uncertainty.

In fact, I prefer to use a table format to present the information in a clean and simple way that makes my analyses easy to read and understand.

Look at the image below to see how I format my uncertainty budgets.

If you notice, I use rows to list each uncertainty contributor and columns to provide important details about each contributor. This makes it easy to keep an uncertainty budget organized and consistent.

Additionally, I prefer to use lines to only separate important information. This minimizes distractions, eye strain, and reduces the amount of ink required for printing.

With a clean and simple format (like the example above), you can make your uncertainty budgets easy to read and understand how uncertainty is calculated. Additionally, a minimalist design will keep your file sizes small and reduce your printing costs.

However, I have seen amazing uncertainty budgets that use borders and colors in their design. The format is entirely up to you. Just make sure that you can quickly read and understand the information when needed. That’s what is really important.

How To Create An Uncertainty Budget In Excel (Step-by-Step)

Creating an uncertainty budget is actually pretty simple if you are using Microsoft Excel. However, most people seem to have trouble using Excel’s functions.

If you are not familiar with Excel, adding formulas and functions can seem like an overwhelming task.

Luckily for you, I have outlined the process below with step by step instructions. Follow them and you can create a fully functional uncertainty calculator in less than 20 minutes.

After creating your uncertainty budget, you should save the file as a template that you can use each time you need to calculate uncertainty. This will ensure that your process is repeatable and prevent you from making mistakes and miscalculations.

If you are ready to create an uncertainty budget, let’s begin.

1. Open Microsoft Excel and create a new spreadsheet

To create an uncertainty budget, get started by opening Microsoft Excel and creating a ‘New’ workbook.

When you first open the Excel program, you should see a screen that looks like this.

There will be several options to choose from, but you simply want to start with a blank workbook.

Select “Blank Workbook” to open a new spreadsheet like in the image below.

This is a great time to create a name and title for your uncertainty budget.

I recommend that you use this opportunity to define your measurement function or test method.

For example, I always use the first four rows to define;

• Measurement Function,
• Description of the equipment or method used.
• Measurement Range, and
• The Measurement Value

To see how I define the measurement function, look at the image below.

Now, before you do anything else, save your file. Use a unique file name that will help you identify your uncertainty budget later on. I recommend keeping it simple and naming your file, “uncertainty-budget-template.xlsx.”

You will not understand how important this step is until you need to find your file again.

It is really frustrating when you are not able to find your uncertainty budgets during an ISO/IEC 17025 audit.

To take it a step further, I recommend that you create a unique file architecture system for your computers and servers. This will save you plenty of headaches over the course of your lifetime.

3. Create a table for your uncertainty calculations

In this step, create a table that you will use to perform uncertainty calculations. This will become your uncertainty budget.

When developing your table, make sure to include the following parameters;

• Uncertainty Sources or Components
• Sensitivity Coefficients
• Uncertainty Values
• Units of Measure
• Probability Distributions
• Divisors
• Standard Uncertainty
• Degrees of Freedom
• Significance or Influence on Total Combined Uncertainty
• Combined Uncertainty
• Total Effective Degrees of Freedom
• Coverage Factor
• Expanded Uncertainty

In my uncertainty budgets, I like to use rows to list my sources of uncertainty and columns to define the parameters of each uncertainty source.

Refer to the image below for an example.

However, fell free to use any type of format that you wish. You want to make sure that you and your assessors are able to understand your uncertainty budgets and calculations.

It is always best to be consistent. So, choose a format that works best for you and stick to it. Remember, you are the one who has to defend it.

4. Add functions to calculate uncertainty

Now that you have created a table for your uncertainty budgets, you will want to add functions and formulas to help you calculate uncertainty.

The benefit to adding formulas and functions is to add automation to your process for estimating uncertainty. Your uncertainty budget will become a calculator that will automatically calculate uncertainty based on your input values.

The five functions that I recommend you add are;

• Calculate standard uncertainty,
• Calculate combined uncertainty,
• Calculate expanded uncertainty,
• Calculate effective degrees of freedom, and
• Calculate the significance of your uncertainty components

Calculate Standard Uncertainty

Calculating standard uncertainty is pretty easy to do, and you can add this function to your uncertainty calculator in no time.

All you need to do is multiply the value of your uncertainty component by it’s respective sensitivity coefficient and divide it by it’s respective divisor (which is based on the probability distribution assigned to the uncertainty component).

To calculate standard uncertainty, follow these simple steps;

1. Select the standard uncertainty cell,
2. Press the equals (=) key to start a new function,
3. Select the sensitivity coefficient cell,
4. Type the asterisk(*) key (i.e. Shift+8) for the multiplication function,
5. Select the Uncertainty Value cell,
6. Type the forward-slash (/) key for the division function,
7. Select the Divisor cell,
8. Hit the ‘Enter’ key.

If you followed the steps above, your formula should look similar to the image below and your uncertainty budget calculator will now calculate standard uncertainty.

Calculate Combined Uncertainty

To calculate combined uncertainty, you will need to use the Root Sum of Square method as directed by the Guide to the Expression of Uncertainty in Measurement (i.e. GUM).

Over the years I have seen people perform this calculation in Excel the hard way by creating a very long formula to square each uncertainty component and add them together.

Well, I am going to show you the easy way to calculate combined uncertainty with just two Excel functions;

• sum of squares (i.e. SUMSQ), and
• square root (i.e. SQRT).

All you need to do is follow these simple steps;

1. Select the combined uncertainty cell,
2. Press the equals (=) key to start a new function,
3. Type the square root function (i.e. SQRT),
4. Press the open parenthesis key (i.e. Shift+9),
5. Inside the SQRT parenthesis, type the sum of squares function (i.e. SUMSQ),
6. Press the open parenthesis key (i.e. Shift+9),
7. Inside the SUMSQ parenthesis, select or type in the range of values,
8. Press the close parenthesis key (i.e. Shift+0) twice, and
9. Hit the ‘Enter’ key.

If you followed the steps above, your formula should look similar to the image below and your uncertainty budget calculator will now calculate..

• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .
ISOBudgets blog by Rick Hogan by Richard Hogan - 2M ago

Introduction

Reproducibility testing is an important part of estimating uncertainty in measurement. It is a type A uncertainty component that should be included in every uncertainty budget.

However, a lot of laboratories neglect to test reproducibility and include it in their analyses.

When talking with clients and friends, I get a lot of reasons excuses for why they do not test reproducibility;

• “We don’t have time.”
• “We don’t know how.”
• “We are a one person laboratory.”
• “We only have one measurement standard.”

Well, I am here today to tell you that you can test reproducibility even if you are only a one person laboratory.

In this guide, you are going to learn all that you need to know, and more, to perform your own reproducibility tests.

If you are calculating uncertainty in measurement, you need to know how reproducible your measurement results are.

It is not only important for quality control. There are so many things that you can learn from testing reproducibility that it is worth the effort.

Background

There has been a lot of discussions and opinions about including type A uncertainty into your estimates of uncertainty in measurement.

Most laboratories include repeatability test data into their uncertainty budgets. However, many neglect to include reproducibility test data.

Why?

I am not sure, but I would like to find out.

Recently, I attended the A2LA Annual Meeting for the Measurement Advisory Committee; and, I was shocked to hear someone ask why ‘Reproducibility’ should be considered for uncertainty analysis.

After making the statement, this person continued speaking to tell everyone that it is not necessary or feasible because the laboratory only had one work station and one technician.

Honestly, I was shocked!

The person that made this statement is an intelligent person from a reputable company, and I was very confused why this person could not conceptualize the rationale for reproducibility testing.

Unfortunately, this person is not alone. Many people question the use reproducibility testing for estimating uncertainty. Over the years, I have had a number of readers, leads, and clients who have also had questions about reproducibility testing.

With this information, I can only assume;

1. They do not understand reproducibility testing,
2. They have not tried reproducibility testing, and(or)
3. They have not researched reproducibility testing

Therefore, I have decided to write a guide that will teach you everything that you will ever need to know about reproducibility testing for collecting type A uncertainty data.

Additionally, I am going to show you 5 ways that you can test measurement reproducibility for calculating uncertainty in measurement even if you are a single person laboratory with only one workbench.

So, if you are interested in collecting type A uncertainty data, keep reading because I am going to cover the following seven topics;

What is Reproducibility

According to the Vocabulary in International Metrology (VIM), reproducibility is “measurement precision under reproducibility conditions of measurement.”

While reading the definition, I am sure that you noticed the terms “measurement precision” and “reproducibility condition of measurement.”

Don’t you hate it when terms are defined by their own words? Me too. It’s frustrating.

Therefore, let’s try to simplify the definition so you don’t need a PhD to understand the concept.

First, let’s define measurement precision. The VIM defines it as “the closeness of agreement between indications.”

Essentially, this is the spread or standard deviation calculated from a set of repeated measurements, most likely from a repeatability test.

Next, let’s define reproducibility condition of measurement. The VIM defines it as a “condition of measurement, out of a set of conditions that includes different locations, operators, measuring systems, and replicate measurements on the same or similar objects.”

Therefore, a reproducibility condition of measurement is another repeatability test where a condition of measurement has been changed.

Now that we have broken down the definition of reproducibility, it can be best explained as the standard deviation of multiple repeatability test results where the conditions of measurement have been changed.

The goal is to determine how closely the results of one repeatability test agree with another to determine how reproducible your results are when performed under various conditions.

Fundamentally, you need to recreate your measurement results after changing one variable at a time and evaluate the impact it has on your results.

Hopefully, you now have a better understanding of the term reproducibility.

In the next section, you will learn why it is important to estimating uncertainty in measurement.

Why is Reproducibility Important

Reproducibility is important because it demonstrates that your laboratory has the ability to replicate measurement results under various conditions.

Additionally, reproducibility testing offers you the ability to experiment with different factors that can influence your measurement results and estimated uncertainty.

When you know which factors significantly impact your measurement results, you can take action to control your measurement process and reduce uncertainty in measurement.

For example, if your laboratory were to conduct a reproducibility test evaluating every technician and one operator’s measurement results were significantly different from the sampled group, you could investigate the cause and take appropriate action (e.g. provide them training to improve their skills).

In another example, imagine that your laboratory is evaluating various test or calibration methods using reproducibility testing. If one method yields significantly different results than the other methods, you could take action to revise the method or eliminate its use for tests and calibrations.

As you can see, reproducibility testing can be a powerful tool for quality control and optimizing your laboratory processes. This is why it is important.

However, it is only beneficial if you actually analyze the data and use it to improve your measurement process. So, think beyond estimating uncertainty, reproducibility testing can be used for;

• Monitoring the quality of work,
• Validating methods, procedures, training, etc,
• Finding problems in measurement systems, and
• Increasing confidence in measurement results.

If your goal is to provide better quality measurement results with less uncertainty, start performing reproducibility tests and use the results to improve your process.

Requirements for Reproducibility Testing

Currently, reproducibility testing is not required by ISO/IEC 17025 or any other normative document; unless you are accredited via A2LA. However, there are strong recommendations for the use of reproducibility testing.

In this section, learn how reproducibility is referenced in various policies and requirements documents.

ISO/IEC 17025 International Standard

In section 5.4.5.3 of the ISO/IEC 17025 standard, reproducibility is listed as a value that may be obtained from validated methods.

Additionally, in note 3 under section 5.4.5.3, reproducibility is listed as an example or component where the uncertainty of values can be listed.

ILAC P14 Policy

In section 5.4 of the ILAC P14 policy recommends that reproducibility should be included in your estimation of CMC uncertainty.

“A reasonable amount of contribution to uncertainty from repeatability shall be included and contributions due to reproducibility should be included in the CMC uncertainty component, when available.”

JCGM 100:2008 Guide to the Estimation of Uncertainty in Measurement

In Appendix B, section 2.16 of the Guide to the Estimation of Uncertainty in Measurement (GUM), reproducibility is defined and provides a list of conditions or variables that can be changed for reproducibility testing.

ASTM E177 & ASTM E456

In table 1 of the ASTM E177, Practice for Use of the Terms Precision and Bias, reproducibility is listed as a condition for precision. Furthermore, the table below provides a list of conditions or variables that can be changed to determine reproducibility.

As you can see, reproducibility is not strictly required for estimating uncertainty; but, it sure is recognized as a significant contributor and recommended for consideration in your uncertainty budgets.

A2LA R205

Only one accreditation body in North America, A2LA, has made reproducibility a requirement for estimates of CMC uncertainty statements.

In section 6.7.1 of the A2LA R205 Specific Requirements: Calibration Laboratory Accreditation Program, reproducibility is listed as a key component that shall be considered in every CMC uncertainty calculation.

In my opinion, reproducibility is a significant contributor to uncertainty in measurement and should be included in your uncertainty budgets.

It is a type A uncertainty component that is just as important as repeatability.

Furthermore, it is a test that you can easily perform in your laboratory. It only takes a few additional minutes of your time to collect the data and analyze it.

Types of Reproducibility Tests

When it comes to reproducibility testing, there are a lot of different conditions that you can test. However, most metrologists (that I have spoken with or surveyed) only compare operators (i.e. one technician’s results compared to another).

While I agree that comparing operators is important, there are so many additional conditions that you can test. Why only test one condition?

You will never be sure that one condition is more significant than the other unless you test it yourself.

In this section, you will learn about five types of reproducibility tests that you can perform in your laboratory to collect type A uncertainty data. Afterwards, you should have enough information to help you decide which conditions are best for testing in your laboratory.

The five conditions that we will cover in this guide are;

1. Operator vs Operator
2. Equipment vs Equipment
3. Environment vs Environment
4. Method vs Method
5. Day vs Day

Operator vs Operator

The most common reproducibility test for collecting type A uncertainty data is looking for variances in measurement results via the operator.

By comparing two or more technicians, engineers, etc, you can learn a lot about inconsistencies in your measurement process.

All you need to do is have two technicians independently perform that same measurement process.

First, have technician A perform a simple repeatability test and record their results.

With this information, calculate the average and standard deviation.

Next, have technician B perform the same repeatability test and record the results.

Again, calculate the average and standard deviation.

Now, calculate the standard deviation of technician A’s average and technician B’s average.

The result will be the operator to operator reproducibility.

If the calculated result is larger than you prefer or larger than you expected, consider providing the technicians training and repeat the experiment.

Continue this process until you achieve a result that you are happy with.

To perform an operator vs operator reproducibility test, use the following instructions;

1. Perform a repeatability test with operator A.
3. Calculate the mean, standard deviation, and degrees of freedom,
4. Perform a repeatability test with operator B,
6. Calculate the mean, standard deviation, and degrees of freedom,
7. Calculate the standard deviation of the two means recorded in steps 3 and 6.

Day vs Day

Another common way to test reproducibility of a measurement process is to test the variance in measurement results from day to day.

This method is great for collecting type A uncertainty data for laboratories with only one technician, one workbench, or both.

The only thing that you need to change is the day or time that the test is performed.

For example, you can compare;

• Morning vs Afternoon,
• Monday vs Tuesday, or
• Monday vs Friday.

You can compare any scenario you want as long as the only variable that changes if day or time of day.

To get started, have a technician perform a repeatability test on day 1.

From their results, calculate the average and the standard deviation.

Next, have the technician perform the exact same repeatability test on day 2.

Again, calculate the average and standard deviation from their results.

Now, calculate the standard deviation of the averages calculated from day1 and day 2. The result will be your day to day reproducibility.

To perform a day vs day reproducibility test, use the following instructions;

1. Perform a repeatability test on day A.
3. Calculate the mean, standard deviation, and degrees of freedom,
4. Perform a repeatability test on day B,
6. Calculate the mean, standard deviation, and degrees of freedom,
7. Calculate the standard deviation of the two means recorded in steps 3 and 6.

Method vs Method

Testing methods for reproducibility is not a common test performed for type A uncertainty. However, it is a very beneficial reproducibility test if you are seeking to reduce your measurement uncertainty.

The method that you choose can significantly affect your measurement uncertainty, Therefore, method vs method reproducibility can be used to determine the variance between two measurement methods.

From the results, you can determine which measurement process yields less uncertainty in measurement results.

To perform method vs method reproducibility test, use the following instructions;

1. Perform a repeatability test using method A.
3. Calculate the mean, standard deviation, and degrees of freedom,
4. Perform a repeatability test using method B,
6. Calculate the mean, standard deviation, and degrees of freedom,
7. Calculate the standard deviation of the two means recorded in steps 3 and 6.

Equipment vs Equipment

If you work in a medium to large-size laboratory, you probably have multiple workbenches and duplicate measurement equipment.

Therefore, you are likely to have variance in your measurement results from using different workbenches or measurement equipment. Wouldn’t it be interesting to see how selecting different equipment can affect your measurement results?

If you said “Yes,” I recommend that you try this type of reproducibility test in your laboratory. You may be surprised by the results.

Every measuring instrument or device will perform differently from another, even though they may have the same manufacture and model number.

Even if your equipment was manufactured on the same day, on the same production line, it’s performance will slightly vary.

This is why it is important to consider equipment vs equipment reproducibility testing for your type A uncertainty data. The probability or likelihood that your measurement results will be affected by the equipment you choose is pretty high!

It makes a good substitute for operator vs operator reproducibility testing.

To perform equipment vs equipment reproducibility test, use the following instructions;

1. Perform a repeatability test using equipment A.
3. Calculate the mean, standard deviation, and degrees of freedom,
4. Perform a repeatability test using equipment B,
6. Calculate the mean, standard deviation, and degrees of freedom,
7. Calculate the standard deviation of the two means recorded in steps 3 and 6.

Environment vs Environment

Performing work outside of the laboratory is becoming more common these days. If your laboratory performs calibration or testing in the field or at customer sites, your measurement results will most likely be impacted by the environment where testing and calibration is..

Articles marked as Favorite are saved for later viewing.
• Show original
• .
• Share
• .
• Favorite
• .
• Email
• .