ASM's Top 5 TextbooksHelping educators teach tomorrow's microbiologists.
The American Society for Microbiology (ASM) appreciates the opportunity to submit comments to the Food and Drug Administration for its review of the criteria used to determine whether specific laboratory tests are waived from certain requirements of the Clinical Laboratory Improvements Amendments of 1988 (CLIA). The ASM is the largest single life science society in the world with more than 42,000 members representing a broad spectrum of subspecialties, including microbiologists who work in biomedical, clinical, public health, and industrial laboratories. The mission of ASM is to enhance the science of microbiology to better understand basic life processes and to promote the application of this knowledge for improved health and for economic and environmental well-being.
The ASM's comments regarding criteria for waived tests reflect the spirit of the law (Food and Drug Administration Modernization Act of 1997) which adheres to simplicity, accuracy, and no unreasonable risk of harm to the patient.
(1) Under FDAMA (1997), waived tests are currently defined as "laboratory examinations and procedures that have been approved by the Food and Drug Administration for home use or that, as determined by the Secretary, are simple laboratory examinations and procedures that have an insignificant risk of an erroneous result, including those that:
(A) employ methodologies that are so simple and accurate to render the likelihood of erroneous results by the user negligible, or
(B) the Secretary has determined pose no unreasonable risk of harm to the patient if performed incorrectly."
General Questions for Public Input
FDA Question 1:
What criteria should be used to demonstrate that a waived test is a simple laboratory examination and procedure with "an insignificant risk of an erroneous result?" For example:
ASM strongly recommends that the FDA Medical Device Advisory Committee be consulted to determine the various accuracy thresholds for infectious disease tests.
FDA Question 2:
What criteria should FDA use to determine if a methodology is "so simple and accurate to render the likelihood of erroneous results by the user negligible?"
Should a waived test be so accurate when performed by untrained users that inaccurate results will not occur?
FDA Question # 3:
What criteria should FDA use in determining that a test will "pose no unreasonable risk of harm to the patient if performed incorrectly?"
ASM strongly believes that a test that can cause harm should, under no circumstances, be waived. FDA's Medical Device Advisory Committee should be consulted to determine the criteria that the FDA could use in determining that a test will "pose no unreasonable risk of harm to the patient if performed incorrectly."
FDA Questions #4:
Should the waiver process be different for screening tests that require a second test for confirmation? Since there are no CLIA standards for performance of waived testing, except instructions to follow the manufacturer's package insert, what is the assurance that confirmatory testing will be performed? Should the need for confirmatory testing raise, lower, or have no impact on the threshold for a waiver decision?
The waiver process should not be different for screening tests that require a second test for confirmation. Any test that requires confirmation is a 2-step process. Currently, waived tests only require a 1-step process. Because there is no assurance that a second test would be conducted to confirm the results of the first test, ASM believes that tests requiring confirmation should not be considered for a waiver decision.
Specific Questions for Public Input
FDA Question 5:
Should accuracy be determined using comparison of the waiver test to a well- characterized reference method and/or materials, to a designated comparative method and/or materials, to a working laboratory method and/or materials, to a clinical algorithm for diagnosis, and/or to other endpoints?
Accuracy should be determined using a well-characterized reference method and a clinical algorithm for diagnosis. ASM believes that clinical algorithms allow for clinical significance. The test must be evaluated on patient specimens in a "clinical trial." For example, a false negative test result for streptococcal A antigen could result in a life-threatening case of rheumatic fever or suppurative sequelae, whereas a false negative test result for vaginitis would not.
ASM encourages the FDA to consult with the FDA Medical Devices Committee to determine whether the device has been evaluated adequately.
FDA Question # 6:
How many samples, what types of samples (real or artificial) by how many users and how many sites are appropriate to evaluate accuracy? (Current guidelines being followed by FDA are for performance to be demonstrated by laboratory users at a minimum of one site.)
ASM recommends that 100 patient samples be tested at a minimum of three sites by 2 different individuals to appropriately evaluate accuracy. The test should be performed by a lay person without any prior training where it is likely to be performed, i.e. POL, and a trained laboratory person. The 100 samples should include approximately 40 positive samples of varying degrees of reactivity. More samples may be necessary to generate statistically meaningful assurance that a waived test is performing acceptably.
FDA Question # 7:
What should be the background of these users?
Waived tests should be conducted by both those who work in CLIA certified laboratories and by a cross section of individuals who are likely to perform the testing.
FDA Question # 8:
What performance criteria (statistical or clinical) should FDA apply to the accuracy threshold for a waived test (e.g., t-test or McNemar test at key decision points, description of performance with confidence intervals at key decision points, use of set performance standards using a receiver operator curve --80%, 90%, 95%, or other-- at key decision points, and/or others)?
The ASM recommends that FDA apply clinical performance criteria and a reference method as the accuracy threshold for a waived test because the majority of tests for infectious diseases provide only a positive or negative result. Waived tests should be held to the same performance criteria as all other FDA-approved tests, and the decision regarding waiver status be made independently. The latter decision should be based on an assessment of the risk of harm to the patients, should testing be performed or results interpreted incorrectly by untrained users.
FDA Question #9:
How should FDA define precision for purposes of waiver determination, what types of samples, how many and what types of operators/sites are appropriate? The current CDC recommendation is for 20 samples at three levels representing appropriate decision points to be tested at three sites by lay users using materials in either artificial and/or real matrices depending on availability and biohazard issues.
ASM recommends that FDA should define precision with a panel of 20 patient specimens at three levels representing appropriate decision points tested three times in a row at three sites by lay users.
FDA Question # 10:
What performance thresholds should FDA use to determine whether the precision studies are appropriate for waiver status (e.g., ANOVA analysis, use of a predefined performance goals such as Tonks' formula, or percent agreement out of total repeat runs)?
The appropriate performance threshold for a waived test should be determined by a statistician.
FDA Question # 11:
What interference studies are appropriate to establish performance of waived tests (e.g., effects of hemolysis, lipemia, etc.)?
The same interference studies required for approval of non-waived tests should be required for waived tests. The manufacturer should list all known interfering substances in the package insert. The package insert should be written so that a lay individual who performs the test in any setting where a waived test is performed could be able to interpret and record the results accurately.
FDA Question # 12
What environmental studies or flex (stress) studies are appropriate to establish performance of waived tests (e.g., temperature or humidity stresses, short fills)?
The same environmental studies required for approval of non-waived tests should be required for waived tests. The manufacturer should define the environmental conditions in the package insert. The test should be performed according to these instructions. The package insert should be written so that a lay individual who performs the test in any setting where a waived test is performed could be able to interpret and record the results accurately.
FDA Question # 13:
What additional studies (if any) should be submitted for evaluation of qualitative tests for waiver?
No additional studies are required.
FDA Question # 14:
What additional studies (if any) should be submitted for evaluation of quantitative tests for waiver?
ASM is against waiving quantitative tests which require two or more steps because of the test complexity. Otherwise the test must contain failsafe mechanisms which render no result when the test system malfunctions or is out of the reportable range.
In closing, the ASM recommends that FDA separate the difference between the simplicity of tests and the safety and accuracy of tests, in relation to the type of medical follow up that will be required as a result of the test. The accuracy of results can mean the difference between the life and death of patients.
ASM appreciates the opportunity to comment as FDA decides the appropriate criteria for determining whether or not certain laboratory tests can be classified as "waived." We are pleased to provide any additional information you may require as this process moves forward.