Accuracy and Precision

For those clinical laboratory scientists, technologists, managers, and directors who have been in laboratory medicine for any length of time, this two-part presentation on quality control principles and practices will be a review. Other healthcare professionals who find themselves doing lab testing as part of their scope of practice and without specific lab training may feel that quality control is unnecessary, an excessive burden, and/or an inscrutable jumble of numbers and graphs.

Good laboratory practice, as well as state and federal regulations, requires laboratory staff of all descriptions to perform and monitor QC and to take corrective action to ensure reliable patient results. So, sit down with your favorite flavor of caffeine and here we go.

Accuracy and Precision

The purpose in performing QC activities and in evaluating the resulting data is to detect test system errors before they become clinically significant and begin to affect the reliability of patient results. To be reliable, results must be both accurate and precise.

Accuracy is the proximity of a test result to the true or accepted value. When assessing accuracy, the results of a given test system are compared to some outside, reliable reference data for the analyte in question. The standard reference method, proficiency testing data, or assayed control target values can be used for this purpose.

Standard deviation index (SDI) is used to compare your data to the reference data. An SDI of zero indicates that the two groups are identical. As the SDI moves away from zero in either direction the disagreement between the two data groups increases. SDIs between -2.0 and +2.0 are usually acceptable for clinical purposes.

Factors affecting accuracy include:

· integrity of calibrators or standards used to establish and confirm linearity;
· condition of pipetting mechanisms used to deliver patient and control samples and/or reagents;
· reagent preparation. (A reagent that is always prepared in the same, but incorrect, fashion will affect accuracy.)

Precision refers to the reproducibility of the test results. Rather than comparing the results to outside data, your lab's results are compared to each other. Two statistical manipulations are used to determine and express precision.

Standard deviation (SD) measures the scatter of values around their mean. In a normal distribution, the mean ±1SD will contain about 68% of all values for a given control and analyte. The mean ±2SD will contain about 95% of all such values, and the mean ±3SD will contain about 99.7% of all results obtained. Typically, the mean ±2SD is used in the clinical laboratory setting. You can therefore expect 5% of your QC data points to be out of range!

Coefficient of variation (CV = SD/mean x 100) expresses precision as a percentage rather than a range. CV therefore allows for comparison of precision at various concentrations of the analyte in question. A low level control can thus be compared to a high level control, for example.

Factors that may affect the precision of a test method or instrument include:

· maintenance of instruments and equipment on a regular schedule;
· consistency in pipetting, calibration, and other techniques among staff members;
· reagent preparation. (A reagent that is prepared in an inconsistent fashion will affect precision as well as accuracy.)

Assess accuracy and precision of your data by comparing the published mean, SD, and CV for a given analyte and control material to the calculated mean, SD, and CV of the actual QC data obtained in the laboratory. For example, compare the published QC mean and 2SD range for a normal glucose control with results obtained by two different labs, Accurate and Inaccurate.

Example	Published	Accurate	Inaccurate
Mean	95	94	87
2SD	90-100	90-98	83-91
CV	2.1%	2.1%	2.3%

In Table 1 Accurate's mean is slightly lower than the published target, with a tighter range and CV. All are well within acceptable limits. Inaccurate's mean is outside the published range, a strong indicator that something is very wrong with the test system.

Looking at Table 2, Precise shows the same data as Inaccurate, above, but we are now looking at precision (reproducibility), not accuracy (proximity to the target).

Example	Published	Precise	Imprecies
Mean	95	87	95
2SD	90-100	83-91	77-113
CV	2.6%	2.3%	9.5%

We also see that it is possible to be precise although inaccurate.

Statistical analysis is valuable when describing accuracy and precision, when establishing a new QC range, or evaluating an old one. Another useful tool is visual analysis, used to detect more subtle changes in test system performance, such as shifts and trends.

Shifts and Trends

A shift may be defined as six or more consecutive data points on one side of the established mean. A trend is defined as a gradual drift of six or more data points in the same direction. A trend can start on either side of or at the mean, and may or may not go beyond 2SD. Again, good laboratory practice and state and federal law, require that labs evaluate shifts and trends and take appropriate action.

Shifts and trends may be monitored by plotting QC data on graphs called Levey-Jennings (LJ) charts. Time is represented on the horizontal- or X-axis and concentration is represented on the vertical- or Y-axis. Plotted data are readily evaluated. Try your skills on the following examples:



In Chart 1, the scatter of data is fairly uniform, with about the same number of data points above and below the mean (represented by the middle horizontal line). There is one outlier at "a" that exceeds the upper limit slightly and is represented by the point just above the +2SD line. There is the threat of a slight downward shift from "b" to "c" with only four data points involved. Perhaps an astute lab tech noticed the problem and took corrective action.



Chart 2 represents a downward shift from "x" to "y." What caused the shift? What, if anything, did the lab do to identify and correct the problem? Why is it important to know?

QC Documentation

This section addresses the issue, "I've done the QC. Now what?" The answer, of course, is, write it down. The reason is twofold: 1) From an inspector's point of view, if it's not documented, it's not done, and 2) You're going to want to know later.

Qualitative tests answer the question, "Is it there or not?" Qualitative test results are binary: positive or negative; present or absent; seen or not seen. Tests for Group A Strep antigen, hCG, Infectious Mononucleosis, and Rheumatoid Factor can fall into this category. Qualitative QC data, including QC from waived methods if required by the manufacturer, should be logged in tabular form. Design your log sheet with spaces to identify the test system, lot numbers, and expiration dates. These identifiers need not be rewritten each day. Start a new sheet or draw a bold line when you change lots, making sure to note the new lot number and expiration date. Make columns for the date of testing, results of the positive and negative controls, and initials of analyst. Leave room for comments in case of a QC failure.

Semi-quantitative tests yield results that give a relative concentration or a range of
concentrations for the analyte: small, moderate or large; 1+, 2+, 3+; 100-125 mg/dL. These QC data are usually logged, but may be plotted using LJ charts.

Quantitative tests give a numerical concentration of the analyte. Most chemistry and hematology tests are of this nature, as well as some serologic procedures. QC data should be plotted on Levey-Jennings charts to readily identify shifts and trends. All LJ charts should identify the test system, lot numbers and expiration dates, and should be reviewed at least weekly for shifts and trends by testing staff and at least monthly by your technical consultant.

Your laboratory procedure manual should contain a step-by step protocol that describes what to do in the event that QC fails to fall within acceptable limits or when shifts or trends are identified that may affect patient results. These outliers, shifts, and trends should be circled or flagged in some manner on the QC sheet at the time of occurrence or noted on an Out of Control (OOC) log if QC data are stored electronically until being printed at the end of each month. Short notations like these will cover most situations:

Repeat OK;
OK post recal;
2 of 3 controls OK; or
No patients reported.

More detailed explanations about outliers and notations about shifts and trends should be documented on the OOC log. Remember to document corrective action and other troubleshooting activities also. Maintenance charts and OOC logs are logical places to make notations, since they are already in place and are usually at hand during these activities. Your notes will be used for reference when reviewing QC at the end of each month and will be helpful when troubleshooting similar problems in the future.

The Review Process

Completed QC logs and LJ charts should be checked for any missing data. Verify that QC was performed and documented at the required frequency for each test method. Make sure that each entry is dated and initialed. If "holes" are found in your QC documentation and it is possible to reconstruct what occurred from other documents to fill in a blank, it is permissible to do so. Do not falsify data.

Your technical consultant should review all QC documents at least monthly, performing a kind of quality assurance on your QC data. He/she will glean an overview of test system and control material performance from your QC data, evaluate shifts and trends, and refer to calibration records, maintenance logs, and other documents to help determine the cause of any identified problems. He/she may recommend that you perform some specific corrective action, possibly in collaboration with the test system manufacturer's technical service department.

Remember that an occasional outlier is a statistical fact of life in any laboratory. An inspector will not close your lab for being "out of control" every now and then. He/she will, however, expect you to take notice of outliers, shifts, and trends and take appropriate corrective action. And write it down.

Knapp • Frazer Consulting • 707.539.6621 • info@knapp-frazer.com
Copyright 2005. All rights reserved. Site designed by Planeteria Web Design.