Case Studies

Competency Assessment
Who needs it? We all do and here’s why.

If your laboratory performs flawlessly all of the time— never makes a mistake, never loses a specimen, never gives out misinformation, and never misreads a stain — you probably don't need Competency Assessment.  But most of us are not that lucky because our laboratories do make mistakes.  We know that specimens can sometimes be mishandled and ruined, we sometimes miss things on Gram stains, we occasionally misidentify organisms, we sometimes report erroneous antimicrobial test results, and the list goes on. In a year’s period of time hundreds of mistakes throughout the laboratory are documented, and these are only the mistakes that we know about!  We surely make errors that are never discovered or never reported.  The bottom line is that ALL laboratories need Competency Assessment.

In accepting the responsibility for technical competency programs and delving into the spirit of what you should be accomplishing, you may become convinced that your existing competency assessment efforts are not really working.  This is apparent because, although you know you have failures in practice, your employees test almost 100% perfect in every area of assessment. 

The design of many Competency Assessment programs tend to prove competency rather than to detect incompetence.  Our entire laboratory profession is based on the design and execution of test systems that identify diseases, quantitate risk factors and measure the effects of clinical interventions.  We need to put these same efforts into our principles of employee competency.

You must accept the fact that you will not be 100% perfect in providing clinical diagnostic services.  After all, laboratories perform thousands to millions of tests per year so mistakes are to be expected.  We hope that by reading the following cases you may change the way you think about Competency Assessment.  Competency Assessment should not be performed solely because it is mandated by law, but because it can truly help you to provide better patient care if implemented appropriately.    

Case 1: TB or not TB; that is the question?  

Case 2: Incompetent! Who ME ??    

Case 3: If THEY would just label the tubes properly...

Case 4: Spot or API?

Case 5:  Extended-what?  

Case 6:  Mixed or pure?  

Case 7:  Site specific!  

Case 8:  Which identification do you believe?

Case 1: TB or not TB; that is the question?

An occasion developed where cultures from two patients that were processed for mycobacteria on the same day both grew Mycobacterium tuberculosis.  One of the patients (patient A) was smear positive with numerous acid-fast bacilli, while the other patient (patient B) was smear negative for acid-fast bacilli.  The culture from patient A was positive after 10 days of incubation, while the culture from patient B was positive after 18 days of incubation.  Contamination of patient B’s sample from the smear positive sample from patient A was suspected.  Both M.tuberculosis isolates were sent for molecular testing to determine if they were in fact the same organism.  The supervisor questioned the senior technologist who initially processed the specimens in order to determine how this might have happened.  No obvious reason was identified.  The supervisor decided that competency assessment might shed some light on the situation and she directly observed the technologist processing specimens for mycobacterial smear and culture.  Upon this interaction the supervisor found that the technologist was not capping specimen transfer tubes after adding a patient’s sample prior to transferring specimen from the next patient.  The supervisor believed that this break in protocol led to the suspected contamination (which was subsequently confirmed by molecular testing).  Due to this deviation from the standard protocol, it was decided that direct observations were warranted for all the mycobacteria processing technologists to assure proper techniques were being adhered to by everyone.  One other minor deviation from protocol was observed during these competency assessments that was corrected with all the technical staff.  In this instance, competency assessment by direct observation assisted the laboratory in improving the quality of future results from the mycobacteriology laboratory.

back to top

Case 2: Incompetent! Who ME ??  

One evening, at the height of RSV season, several specimens were received for DFA viral respiratory panels.  A senior technologist was covering the late shift and set up the panels and read them before she left at the end of the shift.  She did not, however, enter the results in the computer system as she should have.  On the following morning the early shift discovered the pending orders, found the slides next to the microscope, read the slides for a second time and entered the results in the computer.  Shortly thereafter, the physician phoned to say that the computer results conflicted with results that were given by phone on the previous evening.  This is not the way that you want to find out about incompetence.  It took some time to determine how this happened, but it was finally discovered that the senior technologist had incorrectly interpreted positive fluorescence in the specimens.  It was further discovered that the laboratory supervisor had never tested the technologist’s competency for reading these stains after her initial training verification. 

Justin Kruger and David Dunning's 1999 paper, dealing with inflated self-assessment versus actual performance, concludes that people who lack a skill are incapable of knowing that they lack a skill unless there is some external means of measurement.[1]  Kruger and Dunning's work deals with social skills but experience with competency assessment leads us to believe that the same is true of technical skills.  Thus, in absence of a meaningful assessment program, a person can be blissful ignorant of their own incompetence as was the issue with our senior technologist in this case.  We find incompetence all of the time and yet we have really good people who seem to care very much about their performance and patients.  That they are unskilled and unaware explains how caring people can be incompetent.  As soon as caring people discover that they are incompetent, they can take corrective action.  That's how competency measurement works.  Good people will find a way to improve but they must first know about their incompetence, else they think that they are doing a fine job.

1.      Kruger J, Dunning D. Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self assessments. J Pers Soc Psychol, 1999;77: 1121-1134.

back to top

Case 3: If THEY would just label the tubes properly...

A laboratory received a complaint that a spinal fluid submitted for culture had been rejected.  Investigation showed that the collecting physician recorded the patient's room number on the tubes and left the samples for a nursing assistant to label and complete the paperwork. The nursing assistant attached the requisition but did not properly label the tubes according to standard laboratory practice. The puzzling thing was that all other tests on this sample, except for the cultures, were completed in a timely fashion.  Upon further investigated to see if the tests other than the microbiology cultures were done as ordered, the supervisors of these other sections of the laboratory had intervened to save the sample. The steps that each of them took were slightly different but in each case the supervisor contacted nursing personnel to verify the source of the sample and they documented the entire incident prior to accepting the sample, but then processed it according to standard procedures.  In other words, the microbiology laboratory followed the rules and rejected the sample but three labs developed an alternative action and they solved a problem.

Are we saying that it's OK to break rules? Yes, whenever it is appropriate.  In fact breaking rules, making exceptions and devising alternative actions are at the very heart of problem solving.  When is it appropriate to break a rule? This is a tough question and the answer relies our clinical judgement. In the case of the unlabeled spinal fluid one must consider that it is a precious sample that is not easily re-collected.  Also, few patients have this procedure on any given day and the nursing staff can easily and positively identify which patient had the lumbar puncture.  Most laboratory professionals recognize that it is OK to break the rule in this case, whereas specimen processing personnel might not even recognize the rejection of a spinal fluid as a problem.

back to top

Case 4: Spot or API?

The Microbiology Laboratory at a 300 bed hospital routinely identifies E. coli from superficial wounds based on colonial morphology on MacConkey Agar and spot indole reaction.  Colonies not meeting the requirements are identified completely using an API strip at a cost of approximately $3.00 per organism.  Part of the ongoing competency assessment of the laboratory is a review of all positive culture worksheets by the department supervisor.  As part of this review, the supervisor, Karen,  notices that one of her employees, Michelle, seldom reports an E. coli without performing an API strip.  Karen’s impression is that Michelle performs far more API’s than other technologists in the laboratory.  Karen doesn’t want to discourage Michelle from performing API’s where appropriate, but also does not want unnecessary expense on her tight microbiology budget.  How might Karen proceed?

Karen will want to assess Michelle’s understanding of the protocol for working up superficial wound cultures, as well as her competency in the interpretation of colonial morphology. This will best be done by first reviewing the situation with Michelle, and insuring that she completely understands the criteria for organism workup in the superficial wound area.  If it is clear that Michelle understands the protocol, direct observation of Michelle as she interprets colonial morphology will help to further identify the source of the problem.  Michelle may require guidance in understanding the limits of interpretation – how should she work up young colonies, what if the morphology is slightly abnormal (e.g. a more mucoid colony), etc. Once it is determined that Michelle can both describe and follow the protocol, Karen should encourage Michelle to bring any future isolates to her for consultation if there is any uncertainty about whether they fit the morphologic criteria.  Finally, Karen should continue to monitor Michelle’s worksheets (and perform spot checks if deemed necessary) to see if the problem has been resolved.  

back to top

Case 5:  Extended-what?

An 87 year-old nursing home patient was admitted through the ER with a diagnosis of pneumonia.  She was started on IV ceftriaxone + levofloxacin and transferred to the ICU.  A sputum specimen was collected in the ER which showed many neutrophils and many gram negative rods, resembling enterics on Gram stain.  The culture grew many E.coli which was placed on the Vitek for antimicrobial susceptibility testing (AST).  The results of the AST were as follows:

As per the laboratory’s cascading protocol, the technologist reports out the following results to the patient chart:

Based on the AST report from the laboratory, the doctors in the ICU discontinued levofloxacin and left the patient on ceftriaxone only.  The patient showed no improvement and developed shaking chills and fever spikes.  Blood cultures were obtained.  The blood cultures were positive within 24 hours for E.coli that had the same susceptibility pattern as the isolate from the sputum culture.  An infectious disease consult was called.

The infectious disease physician came to the laboratory to review the antibiotic pattern for both of these organisms.  She switched the patient to imipenem and the patient showed rapid improvement. 

WHAT WAS THE PROBLEM HERE?

The AST on this organism indicated the possibility of an E.coli that harbored an extended-spectrum beta-lactamase (ESBL).  Confirmation testing for these organisms are recommended by the NCCLS.  If confirmed as an ESBL-producing organism, the organism would be resistant to all cephalosporins (including ceftriaxone), all penicillins and aztreonam (this organism would NOT be resistant to the cephamycins; e.g.; cefoxitin and cefotetan).  

This organism should have had confirmation testing done for possible ESBL production, and if confirmed,reported as follows:

COMPETENCY ASSESSMENT:  Verify atypical antibiograms.  This error lead to morbidity to the patient and increased length of stay in the hospital that could have been avoided if the laboratory personnel had been competent in recognizing this atypical AST pattern as a possible ESBL-producing organism.

back to top

A 66 year-old male was taken to ICU following bladder reconstruction and symptoms indicated that an infection had developed post-surgery.  The patient was placed on cefepime + gentamicin.  The culture of the infected surgical site revealed Enterobacter cloacae from the wound and E.cloacae from the tracheal aspirate specimen as well.  However, when tested for antimicrobial susceptibility, different antibiograms were noted (tested by a commercial MIC method).  The following susceptibilities were reported for these two isolates of E.cloacae.

The physician questions the laboratory results as the patient seems to be responding to cefepime (which was reported as R from the tracheal aspirate). 

WHAT WAS THE  PROBLEM HERE?

The laboratory checks the “saved” primary plates planted on the tracheal aspirate and noted a few colonies of a second gram negative bacilli.  The second organism was identified as Stenotrophomonas maltophilia.   S.maltophilia are routinely resistant to the aminoglycosides as well as the extended penicillins and cephalosporins.   A tip here that they AST of the tracheal aspirate was mixed with another organism is that E.cloacae should not be resistant to amikacin.

COMPETENCY ASSESSMENT:  Check purity plates and recognize and verify atypical antibiograms as well as checking previous organisms isolated on the patient.  Falsely resistant results were reported out for this patient’s isolate and this could have been avoided if a mixed culture or purity plate were noted.

back to top

Case 7:  Site specific!

A 34 year-old, previously healthy woman was admitted through the ER with signs and symptoms of meningitis.  A spinal tap was performed in the ER, the patient was started on ceftriaxone + vancomycin and transferred to the ICU.  The spinal fluid showed many neutrophils and many gram positive cocci in pairs on Gram stain.  The next day the culture grew Streptococci pneumoniae.  AST on this isolate revealed the following:

The laboratory reported the CSF organism as resistant to penicillin, intermediate to levofloxacin, and susceptible to ceftriaxone, erythromycin, clindamycin, trim-sulfa and vancomycin.  The vancomycin was discontinued and the patient was left on ceftriaxone alone.  The patient showed no improvement and died 2 days later. 

WHAT WAS THE PROBLEM HERE?

The ceftriaxone MICs were interpreted by the laboratory using the NCCLS non-meningitis breakpoints leading to a falsely susceptible result for ceftriaxone. 

Streptococcus pneumoniae
NCCLS M100-S12

Also, the laboratory reported several agents that are not recommended for CSF infections (e.g.; erythromycin will not work even if S), and meropenem was not tested or reported as recommended by NCCLS. 

COMPETENCY ASSESSMENT:

Reporting inappropriate drugs for specimen source and misinterpretation of MIC results.  Reporting antibiotics that will not work in cases of meningitis can lead to adverse outcomes for the patient.  As seen here, misinterpretation of the ceftriaxone results contributed to the death of this patient.

back to top

Case 8:  Which identification do you believe? 

Bob, an experienced microbiologist,  recovered an anaerobic gram-negative bacilli from a pelvic abscess along with an anaerobic gram positive cocci, and an aerobic “enteric” looking organism.  Bob performed special potency antibiotic disk testing and other spot and rapid tests on the anaerobic gram negative bacilli to provide preliminary grouping and identification of the organism.  He recorded the pattern of the special potency antibiotic disks as resistant to Kanamycin (1000 ug), resistant to Vancomycin (5 ug), and sensitive to Colistin (10 ug).  The organism was non-pigmenting, non-fluorescing, spot indole negative, and the organism did not grow on BBE (bile negative). Bob issued a preliminary report of Prevotella sp.  The following day a definitive identification would be performed using a commercial identification kit to further characterize the organism.

The next day another microbiologist was assigned to the anaerobe section and inoculated the anaerobic gram negative bacilli into a commercial identification system.  This technologist was much less experienced and wanted to set the organism up quickly since it was Friday afternoon and he wanted to leave early.  After four hours of incubation, the rapid identification test system identified the organism as Bacteroides caccae.  What is going on?  Which identification do you believe?

The supervisor reviewing the final reports was surprised about the difference between the preliminary and final identification of the organism and began investigating the situation.  The supervisor asked the second technologist how the organism was set up, the length of incubation, how old the plate was from which the inoculum was prepared, from which media the inoculum was produced, and the inoculum size—all of which can affect the identification of the isolate.  The supervisor explained to the technologist that Bacteroides caccae is bile resistant (will grow on BBE), and is resistant to all three special potency antibiotic disks, so something unusual was going on.

The supervisor determined that the technologist used a higher than standard inoculum size when inoculating the identification kit.  A higher inoculum than is recommended by the manufacturer can result in false-positive reactions and change the identification of the organism.  The supervisor explained to the second technologist that a too high or low inoculum size, the plating medium from which the inoculum was prepared, and  the age of the organism when the inoculum is prepared can all effect the biochemical reactions and thereby change the identity of the organism.

The supervisor needs to test the competency of all employees who use commercial identification kits for anaerobes to ensure they are aware of correct procedures and any potential problems and limitations.   In this case, the commercial system was over-inoculated , causing false-positive reactions on some of the biochemical tests.  It is recommended that other data, such as Gram stain characteristics, colony morphology, results of rapid spot tests, and special potency antibiotic disk results be used in conjunction with kit results before an identification is reported.

back to top