Why Not Compare Meters' Accuracy?
Manufacturers of blood glucose meters and diabetes organizations seem reluctant to provide a ranking of the various meters' accuracy. Why? I might even pay more for a meter that exceeded the minimum standard. Tony Trevino, Cedar Park, Texas
Sue Kirkman, MD, the American Diabetes Association's senior vice president for medical affairs and community information, responds:
The American Diabetes Association has a long-standing interest in the accuracy of home blood glucose meters and is part of coalitions that are advocating for strengthening international and Food and Drug Administration (FDA) standards for accuracy. It looks as if the requirements will tighten up in the near future, which can only be good news for people with diabetes, although we don't know what the final recommendations will be.
The ADA would love to be able to provide comparisons of meters' accuracy for our readers, but there are a number of reasons why we cannot do so. We are a nonprofit organization without a laboratory or expertise in laboratory comparisons of products. Such testing would be very complicated (as a meter company executive explains below) and beyond our capabilities. Consumer Reports, the magazine of an organization that does have expertise in rating products, ranked 17 common blood glucose meters in its November 2011 issue. The article says that the meters were compared to "a standard laboratory analyzer," but it doesn't otherwise provide details of the methods used. CR ranked eight meters as "excellent" for accuracy, six as "very good," and three as "good," and noted that all the meters were well within the current standards for accuracy.
Meters sold in the United States are required to describe their accuracy under "Performance Characteristics" in the package insert that comes with test strips and in the meter's instruction manual. Check the website for the meter brand, and call the customer service line if necessary.
However, even with this information, a comparison among meters is difficult because the results aren't reported in the same way for each meter. Some meters report their accuracy results as a "regression line," with a correlation coefficient, slope, and Y-axis. (Yikes! Shades of forgotten algebra!) Other companies report in a table format the percentage of readings above 75 mg/dl that are within plus or minus 5 percent, plus or minus 10 percent, and so on. It would be difficult to compare one regression line to another, and even more difficult to compare a regression line to a table. One requirement we'd love to see from the FDA is that all meters report their accuracy in the same user-friendly format in their package inserts.
We asked David A. Simmons, MD, of Bayer Diabetes Care to respond to your inquiry and appreciate his response below.
David A. Simmons, MD, vice president and chief medical officer of Bayer Diabetes Care, responds:
Thank you, Mr. Trevino, for your important question. I agree that people with diabetes should be able to review comparative, head-to-head data related to the devices they must use every day to manage their blood sugars.
While this sounds like a simple proposition, it actually isn't. In order for a comparison between two devices to be fair and honest, a test would need to be done with each meter using the identical sample of blood, and compared to a reference value (laboratory standard) using the same sample. There are many factors that can affect the results of the test: the blood sample (finger stick or from a vein), how it is obtained (shallow or deep finger stick, alternate site), how the blood is processed (fresh sample, stored or frozen blood), who performs the test (patient user, a lab technician), what reference method is used, and who compares the meter result to the laboratory standard.
Finally, the performance of a meter may not be the same at every glucose value, so it is important that a wide range of sugar values are evaluated and that enough samples are tested in each range. As a consequence, comparing results from two separate studies has a strong possibility of giving an erroneous impression of the comparative performance of two devices.
Even published studies might have misleading conclusions unless they have been carefully constructed to avoid these sources of bias and the reviewers are aware of all of these pitfalls. In other words, it is a very rigorous process to provide a fair comparison. It is also important to remember that the manufacturers are regulated by the FDA with regard to what they may say about any of their products marketed in the U.S. Because of the limitations noted above, the FDA has been cautious about allowing manufacturers to make head-to-head comparisons.
My colleagues at Bayer and I are committed to providing just this kind of quality data. We believe that there are ways to overcome these many technical hurdles to providing the best information to our patients and customers about our products. We are working with advocacy organizations, such as the ADA, and regulatory authorities, such as the FDA, to move this process forward.